Cross-high-resolution optical satellite marine target searching method

Abstract

The invention discloses a cross-high-resolution optical satellite marine target searching method, which comprises the following steps: acquiring optical satellite images at different moments in a task area, and continuously acquiring AIS (Automatic Identification System) data at the same time; identifying marine target data of the optical satellite image; identifying a non-cooperative maritime target according to the maritime target data and AIS data, and taking a corresponding optical satellite image as a query image; constructing a re-identification network for extracting the depth appearance feature vector of the optical satellite image; obtaining an optical satellite image at a subsequent moment and constructing an image library; inputting the query image and the image library into the re-identification network to respectively obtain a query image depth appearance feature vector and an image library depth appearance feature vector; and calculating the similarity between the depth appearance feature vector of the query image and the depth appearance feature vector of an image library, and determining the optical satellite image of the non-cooperative marine target in the image library. According to the method, marine target search in the cross-space-time optical satellite image is realized.

Description

Technical Field

[0001] This invention belongs to the field of satellite remote sensing information processing technology, and specifically relates to a method for searching maritime targets across high-resolution optical satellites. Background Technology

[0002] With the large-scale deployment of low-orbit optical remote sensing satellites and the continuous improvement of image resolution, multi-satellite collaborative operation enables continuous observation of large-scale sea areas, making cross-resolution optical satellite maritime target search possible. In existing technologies, maritime target search methods based on optical satellite imagery mainly utilize information such as the initial direction and velocity of maritime targets for motion prediction, and then use position matching for searching. However, due to the large imaging intervals between different optical satellites, and the high density of maritime targets near ports and important straits, methods based solely on motion prediction are difficult to effectively search for maritime targets of interest. Summary of the Invention

[0003] The purpose of this invention is to provide a method for searching for maritime targets across high-resolution optical satellites, solving the problem of efficient search for maritime targets across high-resolution optical satellites in complex imaging environments.

[0004] To achieve the above objectives, the present invention provides the following technical solution: A method for searching maritime targets across high-resolution optical satellites, the method comprising: The mission area is continuously imaged using a high-resolution optical satellite constellation to obtain optical satellite images at different times, while AIS data within the mission area is continuously acquired. The system identifies maritime target data from the optical satellite imagery based on a pre-defined target detection network. Non-cooperative maritime targets are identified based on the maritime target data and AIS data, and the corresponding optical satellite images are used as query images. Construct a re-identification network for extracting depth appearance feature vectors from optical satellite images; Acquire optical satellite images obtained from high-resolution satellite constellation imaging at subsequent times and construct an image library; The query image and the image database are input into the re-identification network to obtain the depth appearance feature vector of the query image and the depth appearance feature vector of the image database, respectively. Calculate the similarity between the depth appearance feature vector of the query image and the depth appearance feature vector of the image library, and determine the optical satellite image of the non-cooperative maritime target in the image library.

[0005] Preferably, the target detection network is a network model built based on the YOLOv11 target detection algorithm with a rotated box. The maritime target data includes the imaging time, latitude and longitude coordinates of the rotating frame, length of the maritime target, and slices of the maritime target in the optical satellite image.

[0006] Preferably, the AIS data includes the MMSI number of the maritime target, its latitude and longitude coordinates, and its length. The steps of “identifying non-cooperative maritime targets based on the maritime target data and AIS data” include: Global nearest neighbor matching is performed based on the latitude and longitude coordinates and length information of the maritime targets in the AIS data. Maritime targets that can be matched are defined as cooperative targets, and maritime targets that cannot be matched are defined as non-cooperative maritime targets.

[0007] Preferably, prior to the step of "constructing a re-identification network for extracting depth appearance feature vectors from optical satellite images", the method further includes: Acquire a large amount of AIS data and maritime target data identified based on optical satellite images at different imaging times; Using AIS data as the ground truth, we construct a maritime target re-identification dataset by taking the maritime target slices corresponding to the same MMSI number as cross-high resolution optical satellite observation samples of the same maritime target.

[0008] Preferably, the re-identification network uses ResNet50 as the backbone network, and embeds non-local attention modules in its last four residual blocks; and, The re-identification network is a model optimized using weighted regularized triplet loss and cross-entropy loss.

[0009] Preferably, during the network training phase, the re-identification network further includes data augmentation and normalization processing on the maritime target re-identification dataset.

[0010] Preferably, the step of "acquiring optical satellite images obtained from the high-resolution satellite constellation imaging at subsequent times and constructing an image library" includes: Based on the maximum speed and search time interval of non-cooperative maritime targets, the possible activity area of ​​non-cooperative maritime targets is obtained; The image library is constructed by acquiring optical satellite images of the active area at subsequent times.

[0011] Preferably, the step of "calculating the similarity between the depth appearance feature vector of the query image and the depth appearance feature vector of the image database and determining the optical satellite image of the non-cooperative maritime target in the image database" includes: Calculate the similarity between the depth appearance feature vector of the query image and the depth appearance feature vector of the image database, and form a similarity ranking list; The similarity sorting list is optimized based on reverse order reordering and query expansion methods; Optical satellite images of the non-cooperative maritime targets are filtered out by setting a confidence threshold.

[0012] The advantages of this invention are: The cross-high resolution optical satellite maritime target search method provided by this invention achieves efficient search for maritime targets of interest through reasonable network design and optimization strategies, and is particularly suitable for non-cooperative maritime target surveillance scenarios in the open sea. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the main steps of a method for searching maritime targets across high-resolution optical satellites according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a process for acquiring initial features of a maritime target based on initial satellite imagery in an embodiment of the present invention; Figure 3 This is a schematic diagram of a typical target slice in a re-identification dataset according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a re-identification network in an embodiment of the present invention; Figure 5 This is a schematic diagram of a typical re-identification result using the method of this invention in a test set, as described in an embodiment of this invention. Figure 6 This is a schematic diagram of a search result for a maritime target in an embodiment of the present invention. Detailed Implementation

[0014] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0015] To achieve efficient search for maritime targets across high-resolution optical satellites in complex imaging environments, this invention employs a deep learning-based maritime target search method. Based on target detection results acquired by different optical satellites in the same sea area and matching results with Automatic Identification System (AIS) data, a maritime target re-identification dataset is constructed. By designing and training a maritime target re-identification network, the network extracts depth and appearance features from query images and image databases. Combined with constraints such as motion and length, and depth and appearance feature matching, this improves the accuracy, robustness, and efficiency of maritime target search across long time spans between different optical satellites, providing strong technical support for maritime target surveillance applications.

[0016] See Figure 1 and Figure 2This describes the main process of a method for searching maritime targets across high-resolution optical satellites. The method provided in this embodiment includes: Step S1: Utilize a high-resolution optical satellite constellation to continuously image the mission area, obtaining optical satellite images at different times, while continuously acquiring AIS data within the mission area.

[0017] Specifically, a high-resolution optical satellite constellation is used to continuously image the mission area, obtaining high-resolution optical satellite images at different imaging times. Simultaneously, AIS data of the mission area is continuously acquired, including the MMSI number, latitude and longitude coordinates, and length of the maritime targets.

[0018] Step S2: Identify maritime target data from optical satellite images based on a preset target detection network.

[0019] Specifically, the target detection network is a network model built based on the YOLOv11 target detection algorithm with rotated bounding boxes. Addressing the multi-angle variation characteristics of maritime targets in optical satellite images, an improved YOLOv11 target detection algorithm with rotated bounding boxes is employed to extract maritime targets from complex imaging backgrounds. The target detection network is a network model trained with a large number of labeled maritime target samples, maintaining high detection accuracy under different lighting conditions and complex marine environments. The target detection network process includes inputting optical satellite images in blocks, extracting depth features, generating and discriminating region candidates, transforming image-to-object coordinates, and cropping target slices, ensuring that the output target bounding boxes have high confidence. Ultimately, it obtains maritime target data for each moment in the optical satellite image, including the imaging time of the maritime target in the optical satellite image, the latitude and longitude coordinates of the rotated bounding box, the length of the maritime target, and the maritime target slice data.

[0020] Step S3: Identify non-cooperative maritime targets based on maritime target data and AIS data, and use the corresponding optical satellite image as the query image.

[0021] Specifically, based on the latitude and longitude coordinates and length information of maritime targets in the maritime target data and AIS data, global nearest neighbor matching is performed within certain distance and length thresholds. Maritime targets that can be matched are defined as cooperative targets, while those that cannot be matched are defined as non-cooperative maritime targets. Optionally, targets of interest can be selected according to mission requirements. For tasks such as maritime traffic management and detection of illegal maritime activities, specific non-cooperative maritime targets are filtered out based on target location, length, and other information. Non-cooperative maritime targets are then added to the query image as key targets in subsequent maritime target searches.

[0022] Step S4: Construct a re-identification network for extracting depth appearance feature vectors from optical satellite images.

[0023] Specifically, due to the differences in imaging angle, imaging background, solar elevation angle, image resolution, and other parameters of different optical satellites, the same maritime target in different optical satellite images has significant differences in radiation, geometry, etc. This invention uses a deep learning-based re-identification network to extract the depth appearance features of maritime targets and construct a robust feature representation.

[0024] First, a maritime target re-identification dataset is constructed by acquiring a large amount of AIS data and maritime target data identified based on optical satellite images at different imaging times. Using AIS data as the ground truth, maritime target slices corresponding to the same MMSI number are used as cross-resolution optical satellite observation samples for the same maritime target to construct the maritime target re-identification dataset. Considering that most optical satellites use a linear array pushbroom system, a single optical satellite can only obtain one slice of a single maritime target. By accumulating a large number of maritime target samples from multiple sea areas, a diverse and large-scale maritime target re-identification dataset can be ultimately constructed.

[0025] See Figure 3 This is a schematic diagram of a typical target slice in the re-identification dataset. This embodiment constructs a high-resolution optical satellite maritime target re-identification dataset based on commercial satellite remote sensing data and AIS data. It contains over 1000 maritime targets, each with 4-10 optical satellite image slices. A typical target slice in the re-identification dataset is shown below. Figure 3 As shown, the same target corresponds to a target slice from 10 satellite images.

[0026] See Figure 4 The re-identification network uses ResNet50 as its backbone, embedding non-local attention modules in the last four residual blocks to enhance the global contextual dependence of feature representations. It captures fine-grained features of maritime targets through generalized mean pooling, optimizing the network's ability to represent target features. Furthermore, the re-identification network jointly employs weighted regularized triplet loss and cross-entropy loss to optimize the model's classification and feature discrimination capabilities.

[0027] During the network training phase, data augmentation was performed on target slices collected from the maritime target re-identification dataset. This included operations such as flipping, random erasing, and occlusion to improve the re-identification network model's adaptability to diverse scenes. Target slices were normalized to ensure consistent image size, providing standardized data for subsequent training. A pre-warm-up learning rate strategy was employed to enhance stability in the early stages of training, while the learning rate was dynamically adjusted to adapt to the model's convergence process.

[0028] In this embodiment, the designed and trained maritime target re-identification network is validated on a re-identification dataset. The re-identification results of 100 targets are shown in Table 1.

[0029] Table 1. Here, rank1 represents the proportion of correctly matched target slices appearing in the first position of the search results, measuring the algorithm's ability to accurately find the target in one go; a higher value indicates higher algorithm accuracy. rank5 represents the proportion of correctly matched target slices appearing in the top 5 positions of the search results; a higher value indicates better fault tolerance and robustness of the algorithm. mAP is the mean precision, which comprehensively considers precision and recall. It is obtained by calculating the average precision of each query target and then averaging it over all query results, reflecting the overall performance of the algorithm under samples of different difficulty levels, and is currently the most mainstream comprehensive evaluation indicator. It can be seen that the maritime target re-identification network provided in this embodiment has higher retrieval performance than the ResNet50 baseline network.

[0030] Step S5: Acquire optical satellite images obtained from high-resolution satellite constellation imaging at subsequent times and construct an image library.

[0031] Specifically, the search for non-cooperative maritime targets is achieved in optical satellite images at subsequent imaging times. Using the target detection network mentioned earlier, maritime targets are extracted from the optical satellite images at subsequent imaging times, constructing an image database. Based on the query images and the maritime targets in the image database, constraints such as motion and length are applied to reduce the target search space. The maximum velocity of the maritime targets is then considered. and search time interval This yields the target's possible activity area, i.e., the maximum search radius. By considering the characteristics of actual sea lanes, the search area can be further reduced. Using the maximum search radius as a distance threshold and the maximum length difference as a length threshold, only maritime targets that meet both distance and length thresholds can be matched for depth and appearance features.

[0032] Step S6: Input the query image and the image database into the re-identification network to obtain the depth appearance feature vector of the query image and the depth appearance feature vector of the image database, respectively.

[0033] Specifically, target slices of non-cooperative maritime targets are input into the re-identification network, which outputs a depth and appearance feature vector of the query image. Target slices from the image database are input into the re-identification network, which outputs depth and appearance features of the image database.

[0034] Step S7: Calculate the similarity between the depth appearance feature vector of the query image and the depth appearance feature vector of the image database, and identify the optical satellite images of non-cooperative maritime targets in the image database.

[0035] To improve cross-satellite target feature matching performance, a deep spatial reconstruction mechanism can be introduced to enhance local feature alignment capabilities. Then, cosine distance is used to calculate the similarity between the depth and appearance feature vectors of the query image and the depth and appearance feature vectors of the image database, forming a similarity ranking list. Based on reverse order reordering and query expansion methods, the similarity ranking list is optimized to reduce the impact of low-confidence matches on search results. Optical satellite images of non-cooperative maritime targets are filtered out by setting a confidence threshold. The search results further guide the planning of subsequent satellite imaging missions.

[0036] In the test set, typical re-identification results using the deep learning-based maritime target search method of this invention are as follows: Figure 5 As shown in the figure, the first column of each group of images is the query image, and columns 2-10 are the retrieved images (only column 2 is shown in the figure). The images are sorted from high to low confidence. Solid boxes represent correctly retrieved images, and dashed boxes represent incorrectly retrieved images. It can be seen that the deep learning-based maritime target search method provided by this invention has a good retrieval effect.

[0037] In practical applications, taking optical satellite images of a certain area taken at two different times as an example, a slice of interest representing a maritime target is selected from the satellite image at the previous time. Based on a re-identification network, a search for the maritime target in the satellite image at the next time is performed, resulting in the correct target slice. For example... Figure 6 As shown, the feasibility and effectiveness of the search method of the present invention are verified.

[0038] In summary, this invention employs a deep learning-based method for searching maritime targets. Based on target detection results acquired by different optical satellites in the same sea area and matching results with AIS data, it identifies non-cooperative maritime targets. By designing and training a maritime target re-identification network, and utilizing this network to extract depth and appearance features from query images and image databases, combined with constraints such as motion and length, and depth and appearance feature matching, the accuracy, robustness, and efficiency of maritime target searching over long time spans between different optical satellites are improved, providing strong technical support for maritime target surveillance applications.

[0039] The above description describes the preferred embodiments of the present invention and the technical principles applied thereto. For those skilled in the art, any obvious changes such as equivalent transformations or simple substitutions based on the technical solutions of the present invention, without departing from the spirit and scope of the present invention, shall fall within the protection scope of the present invention.

Claims

1. A method for searching maritime targets across high-resolution optical satellites, characterized in that, The method includes: The mission area is continuously imaged using a high-resolution optical satellite constellation to obtain optical satellite images at different times, while AIS data within the mission area is continuously acquired. The system identifies maritime target data from the optical satellite imagery based on a pre-defined target detection network. Non-cooperative maritime targets are identified based on the maritime target data and AIS data, and the corresponding optical satellite images are used as query images. Construct a re-identification network for extracting depth appearance feature vectors from optical satellite images; Acquire optical satellite images obtained from high-resolution satellite constellation imaging at subsequent times and construct an image library; The query image and the image database are input into the re-identification network to obtain the depth appearance feature vector of the query image and the depth appearance feature vector of the image database, respectively. Calculate the similarity between the depth appearance feature vector of the query image and the depth appearance feature vector of the image library, and determine the optical satellite image of the non-cooperative maritime target in the image library.

2. The method for searching maritime targets across high-resolution optical satellites as described in claim 1, characterized in that, The target detection network is a network model built based on the YOLOv11 target detection algorithm with a rotating frame. The maritime target data includes the imaging time, latitude and longitude coordinates of the rotating frame, length of the maritime target, and slices of the maritime target in the optical satellite image.

3. The method for searching maritime targets across high-resolution optical satellites as described in claim 2, characterized in that, The AIS data includes the MMSI number, latitude and longitude coordinates, and length of the maritime target; The steps of "identifying non-cooperative maritime targets based on the maritime target data and AIS data" include: Global nearest neighbor matching is performed based on the latitude and longitude coordinates and length information of the maritime targets in the AIS data. Maritime targets that can be matched are defined as cooperative targets, and maritime targets that cannot be matched are defined as non-cooperative maritime targets.

4. The method for searching maritime targets across high-resolution optical satellites as described in claim 3, characterized in that, Prior to the step of "constructing a re-identification network for extracting depth appearance feature vectors from optical satellite images", the method further includes: Acquire a large amount of AIS data and maritime target data identified based on optical satellite images at different imaging times; Using AIS data as the ground truth, we construct a maritime target re-identification dataset by taking the maritime target slices corresponding to the same MMSI number as cross-high resolution optical satellite observation samples of the same maritime target.

5. The method for searching maritime targets across high-resolution optical satellites as described in claim 4, characterized in that, The re-identification network uses ResNet50 as its backbone network, embedding non-local attention modules in its last four residual blocks; and... The re-identification network is a model optimized using weighted regularized triplet loss and cross-entropy loss.

6. The method for searching maritime targets across high-resolution optical satellites as described in claim 5, characterized in that, During the network training phase, the re-identification network also includes data augmentation and normalization processing on the maritime target re-identification dataset.

7. The method for searching maritime targets across high-resolution optical satellites as described in claim 4, characterized in that, The steps of "acquiring optical satellite images obtained from high-resolution satellite constellation imaging at subsequent times and constructing an image library" include: Based on the maximum speed and search time interval of non-cooperative maritime targets, the possible activity area of ​​non-cooperative maritime targets is obtained; The image library is constructed by acquiring optical satellite images of the active area at subsequent times.

8. The method for searching maritime targets across high-resolution optical satellites as described in claim 7, characterized in that, The steps of "calculating the similarity between the depth appearance feature vector of the query image and the depth appearance feature vector of the image database, and determining the optical satellite image of the non-cooperative maritime target in the image database" include: Calculate the similarity between the depth appearance feature vector of the query image and the depth appearance feature vector of the image database, and form a similarity ranking list; The similarity sorting list is optimized based on reverse order reordering and query expansion methods; Optical satellite images of the non-cooperative maritime targets are filtered out by setting a confidence threshold.

Images