A method for detecting specific targets at sea using medium- and low-resolution multispectral imagery

Abstract

This invention belongs to the field of spectral image recognition technology, specifically a method for detecting specific maritime targets in medium- and low-resolution multispectral images. The specific steps of this method are as follows: Based on the selected satellite image data source, a target sample T is collected from the historical archive database. This invention utilizes multi-directional template matching and comparative spectral feature extraction to comprehensively extract the spatial texture features and spectral features of the target, thereby achieving the search for specific maritime targets. Compared with deep learning methods, this method does not require sample labeling and has the characteristics of low development cost, low computational complexity, flexibility and controllability, and suitability for rapid deployment in the cloud. It is suitable for complementing deep learning technology.

Description

Technical Field

[0001] This invention relates to the field of spectral image recognition technology, specifically to a method for detecting specific targets at sea in medium- and low-resolution multispectral images. Background Technology

[0002] Detection, identification, and tracking of maritime targets has always been an important research area. It is crucial to quickly and accurately detect and identify targets such as ships and marine facilities on the sea surface.

[0003] Currently, Automatic Identification Systems (AIS) are widely used in ship monitoring and navigation management, capable of reporting real-time dynamic information such as a ship's position, course, and speed. However, AIS has some inherent limitations:

[0004] 1) AIS can be manually turned off by crew members, making the vessel "invisible" to surveillance when needed, especially when carrying out illegal or special missions;

[0005] 2) AIS data in the open ocean mainly relies on satellite communication, which has a low update frequency and may even be lost;

[0006] 3) AIS can only provide a limited amount of message information such as the ship's position and cannot reflect the status of the ship and its surrounding environment.

[0007] Therefore, the technology of detecting maritime targets based on remote sensing imagery plays an important role and often serves as a supplement in the system.

[0008] In recent years, target detection algorithms based on deep convolutional neural networks (CNNs) have made significant progress, demonstrating a powerful ability to automatically learn data feature representations and have begun to be gradually applied to the field of maritime target detection, with representative algorithms such as YOLO and Faster-RCNN. However, directly applying these methods to specific maritime target detection tasks still faces some challenges and limitations:

[0009] 1) Reliance on large amounts of high-quality labeled data: CNNs require supervised training with a large amount of labeled data to learn effective feature representations, which is very costly in terms of time and manpower.

[0010] 2) Insufficient utilization of spectral information: Most existing mainstream CNN models are based on RGB three-channel images, which fail to fully utilize the rich multispectral and hyperspectral information contained in remote sensing images;

[0011] 3) Limited applicability: Remote sensing images are diverse, and high-resolution images, multispectral images, and radar images have significantly different characteristics. A single model often cannot be compatible with data from different modes.

[0012] 4) Scene limitations: CNNs are often trained on high-resolution images, but for large areas of the sea, the coverage of high-resolution satellites is very limited. Medium and low resolution payloads with a resolution of more than 10 meters, such as Sentinel-2, Gaofen-6, and Landsat, are more realistic choices. Their swath width is often more than 100 kilometers. At this resolution, the target texture is blurred, and CNNs need to be modeled and trained in a specific way.

[0013] To address these issues, a method for detecting specific targets at sea using medium- and low-resolution multispectral imagery is proposed. Summary of the Invention

[0014] The purpose of this invention is to provide a method for detecting specific targets at sea using medium- and low-resolution multispectral imagery, in order to solve the problems mentioned in the background art.

[0015] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting specific targets at sea using medium- and low-resolution multispectral imagery, the specific steps of which are as follows:

[0016] Step 1: Based on the selected satellite imagery data source, collect a target sample T from the historical archive database;

[0017] Step 2: Rotate sample T to construct a multi-directional template set T Θ To avoid interpolation, let Θ∈{0,π / 2,π,3π / 2};

[0018] Step 3: Based on the pre-determined search area, obtain the list of images to be analyzed (I). C ;

[0019] Step 4: From I C Extract a scene from the image

[0020] Step 5: For the image and multi-directional template set T Θ Using a multi-directional template matching method with a threshold of 0.5, the first-stage candidate target set was extracted.

[0021] Step Six: Construct sample T and The spectral features of all candidate targets are analyzed, and the cosine similarity between the features of the sample and each candidate target is calculated. A threshold of 0.5 is set to filter out the high-suspicious targets in the second stage.

[0022] Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full Collect global target set O CIn the process, a spatial spectrum visualization image of each target is generated for easy manual verification;

[0023] Step 8: k = k + 1, if k is less than image list I C Given the length, skip to step three;

[0024] Step 9: Output the final result O C ,Finish.

[0025] Preferably, in step one, target sample acquisition refers to acquiring at least one target sample based on historical image data under the adopted data source. Taking a ship target as an example, based on the ship or other ships of the same type, matching sample data is acquired from the satellite historical archive database using AIS ship trajectory data. Assuming the number of bands of the image data source used is c, the sample is a three-dimensional matrix data of (h, w, c), where h and w are the height and width, respectively, and the center of the sample is the target. It is required that h and w be slightly larger than the width and height of the target to retain certain background information.

[0026] Preferably, in step five, the extraction of the candidate target set for the first stage is achieved by using the RGB three bands to rotate the sample in multiple directions for template matching, thus obtaining the candidate targets for the first stage. Since the data are from the same source and the spatial resolution of the test data and the sample is consistent, there is no need to consider multi-scale issues. The template matching adopts the normalized cross-correlation coefficient matching method, and the output value range of the matching is [-1, 1]. Therefore, the matching result in each direction is a single-channel matching score image with the same pixel size as the test data. The maximum value of the matching scores in multiple directions is taken at each pixel location to obtain the final matching score map SM of the image under test. final And continue to use the threshold method to SM final Convert the image into a binary map Mask to indicate candidate targets. Each patch in the Mask represents a candidate target. The maximum value within the patch is taken as the score s1, and the location of the maximum value is taken as the center of the candidate target (x). c ,y c ), set the length and width to be consistent with the sample, and the resulting candidate targets are in (x c ,y c The candidate targets are collected in a list as the first-stage search results.

[0027] Preferably, the formula for calculating the final matching score image is as follows:

[0028]

[0029] ①②③ represent multi-directional template matching, and ④ represents taking the maximum value of the multi-directional matching results; where I represents the image, T θkThe template is rotated, and w and h are the width and height of the template.

[0030] Preferably, in step six, the high-suspicious targets in the second stage are selected based on the location coordinates of the candidate targets obtained in the first stage, and their coordinates (x, y, y) are extracted. c ,y c The spectral features of the (h,w,c) image data block at location (h,w,c) are constructed and compared with the samples to achieve the purpose of screening and identification.

[0031] Preferably, the specific steps for screening highly suspicious targets in the second stage are as follows:

[0032] S1: Constructing Spectral Features: Due to factors such as changes in solar illumination angle and atmospheric conditions, the reflectance intensity of a target at the same location varies across different images. Therefore, when constructing spectral features, the seawater in the target background is used as a calibration. The OTSU adaptive threshold segmentation method is used to segment the target data into foreground and background. First, outliers in each band of the background pixels are removed using the quartile outlier detection method. Then, the mean is calculated to obtain the average spectral curve of the background pixels. Next, for all foreground pixels, the average spectral curve of the background is subtracted, and the first-order difference is calculated in the spectral dimension. The mean is then calculated to obtain the average first-order difference curve of the foreground target, which serves as the spectral feature. The specific process is as follows:

[0033] Let d be the input data, d rgb To extract its RGB three bands

[0034] fg mask ,bg mask =OTSU(d rgb )

[0035]

[0036] In the formula, removeOutliers is the quartile outlier removal function, which acts independently on each band;

[0037] diff is the first-order difference in the spectral dimension, where f is the feature of the final output.

[0038] S2: Similarity Calculation: For sample S and candidate target P to be evaluated, the corresponding spectral features f are calculated using the method in S1. s and f p Calculate their cosine similarity:

[0039] s2 = cosineSimilarity(f s ,f p )

[0040] The cosine similarity ranges from [-1, 1]. Using a threshold method, empirically set between 0.2 and 0.5, all candidate targets exceeding the threshold are retained as the final highly suspicious targets output, with (x... c ,y c Records in the format of w, h, s1, s2).

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] This invention utilizes multi-directional template matching and comparative spectral feature extraction to comprehensively extract the spatial texture and spectral features of a target, thereby enabling the search for specific targets at sea. Compared to deep learning methods, this method does not require sample labeling and features low development cost, low computational complexity, flexibility and controllability, and suitability for rapid deployment in the cloud. It is suitable for complementing deep learning technology.

[0043] Overall framework of the method: This invention achieves target search over a wide range of sea areas by combining spatial texture features and spectral features for screening. This combination utilizes the complementarity of spatial and spectral information, improving the accuracy and robustness of target detection.

[0044] Application of spectral features: The construction of spectral features in this invention takes into account the influence of the difference in image reflectance at different times. It is corrected by the background spectral curve after denoising and the first-order difference is calculated so that cosine similarity can be used for comparison, which significantly improves the accuracy and reliability of target recognition.

[0045] Application scenarios and usage environment: This invention is designed for the search of maritime targets using medium- and low-resolution multispectral wide-swath satellites. Applicable targets include, but are not limited to, objects with specific appearance characteristics such as ships and offshore operating platforms. Attached Figure Description

[0046] Figure 1 This is a flowchart of the processing of a single scene image;

[0047] Figure 2 Example image of the target sample;

[0048] Figure 3 This is a sample image of the spatial spectrum of a suspected target obtained by the algorithm search. Detailed Implementation

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0050] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0051] Example:

[0052] Please see Figure 1-3 The present invention provides a technical solution:

[0053] A method for detecting specific targets at sea using medium- and low-resolution multispectral imagery is disclosed. The specific steps of this method are as follows:

[0054] Step 1: Based on the selected satellite imagery data source, collect a target sample T from the historical archive database;

[0055] Step 2: Rotate sample T to construct a multi-directional template set T Θ To avoid interpolation, let Θ∈{0,π / 2,π,3π / 2};

[0056] Step 3: Based on the pre-determined search area, obtain the list of images to be analyzed (I). C ;

[0057] Step 4: From I C Extract a scene from the image

[0058] Step 5: For the image and multi-directional template set T Θ Using a multi-directional template matching method with a threshold of 0.5, the first-stage candidate target set was extracted.

[0059] Step Six: Construct sample T and The spectral features of all candidate targets are analyzed, and the cosine similarity between the features of the sample and each candidate target is calculated. A threshold of 0.5 is set to filter out the high-suspicious targets in the second stage.

[0060] Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Collect global target set O C In the process, a spatial spectrum visualization image of each target is generated for easy manual verification;

[0061] Step 8: k = k + 1, if k is less than image list I CGiven the length, skip to step three;

[0062] Step 9: Output the final result O C ,Finish.

[0063] In step one, target sample acquisition refers to acquiring at least one target sample based on historical image data, using the data source adopted (images taken by different satellite payloads are different data sources). Taking a ship target as an example, based on the ship or other ships of the same type, matching sample data is acquired from the satellite historical archive database using AIS ship trajectory data. Assuming the number of bands of the image data source used is c, the sample is a three-dimensional matrix data of (h, w, c), where h and w are the height and width, respectively. The center of the sample is the target. It is required that h and w be slightly larger than the width and height of the target to retain some background information.

[0064] In step five, the candidate target set for the first stage is extracted by rotating the sample in multiple directions using the RGB bands and performing template matching to obtain the candidate targets for the first stage. Since the data are from the same source and the spatial resolution of the test data and the sample is consistent, there is no need to consider multi-scale issues. The template matching adopts the normalized cross-correlation coefficient matching method (corresponding to OpenCV's CV_TM_CCOEFF_NORMED mode). The output value range of the matching is [-1, 1] (1 indicates complete consistency). Therefore, the matching result in each direction is a single-channel matching score image with the same pixel size as the test data. The maximum value of the matching scores in multiple directions is taken at each pixel location to obtain the final matching score map SM of the image under test. final And continue to use the threshold method to SM final Convert the image into a binary map Mask to indicate candidate targets. Each patch in the Mask represents a candidate target. The maximum value within the patch is taken as the score s1, and the location of the maximum value is taken as the center of the candidate target (x). c ,y c ), set the length and width to be consistent with the sample, and the resulting candidate targets are in (x c ,y c The candidate targets are collected in a list as the first-stage search results.

[0065] The formula for calculating the final matching score image is as follows:

[0066]

[0067] ①②③ represent multi-directional template matching, and ④ represents taking the maximum value of the multi-directional matching results; where I represents the image, T θk The template is rotated, and w and h are the width and height of the template.

[0068] In step six, the screening of highly suspicious targets in the second stage involves extracting their coordinates (x, y) based on the location coordinates of the candidate targets obtained in the first stage. c ,y c The spectral features of the image data block at (h,w,c) are constructed and compared with the samples to achieve the purpose of screening and identification.

[0069] The specific steps for screening highly suspicious targets in the second phase are as follows:

[0070] S1: Constructing Spectral Features: Due to factors such as changes in solar illumination angle and atmospheric conditions, the reflectance intensity of a target at the same location varies across different images. Therefore, when constructing spectral features, the seawater in the target background is used as a calibration. The OTSU adaptive threshold segmentation method is used to segment the target data into foreground and background (here we assume the background is water). First, the quartile outlier detection method is used to remove outliers from each band of the background pixels (eliminating interference from mixed pixels). Then, the mean is calculated to obtain the average spectral curve of the background pixels. Then, for all foreground pixels, the average spectral curve of the background is subtracted, and the first-order difference is calculated in the spectral dimension. The mean is then calculated to obtain the average first-order difference curve of the foreground target, which is used as the spectral feature. The specific process is as follows:

[0071] Let d be the input data, d rgb To extract its RGB three bands

[0072] fg mask ,bg mask =OTSU(d rgb )

[0073]

[0074] In the formula, removeOutliers is the quartile outlier removal function, which acts independently on each band; diff is the first-order difference in the spectral dimension, and f is the feature of the final output;

[0075] S2: Similarity Calculation: For sample S and candidate target P to be evaluated, the corresponding spectral features f are calculated using the method in S1. s and f p Calculate their cosine similarity:

[0076] s2 = cosineSimilarity(f s ,f p )

[0077] The cosine similarity ranges from [-1, 1]. Using a threshold method, empirically set between 0.2 and 0.5, all candidate targets exceeding the threshold are retained as the final highly suspicious targets output, with (x... c ,y c Records in the format of w, h, s1, s2).

[0078] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0079] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting specific targets at sea using medium- and low-resolution multispectral imagery, characterized in that, The specific steps of this method for detecting specific targets at sea using medium- and low-resolution multispectral imagery are as follows: Step 1: Based on the selected satellite imagery data source, collect a target sample from the historical archive database. ; Step 2: Sample Rotational construction of multi-directional template sets To avoid interpolation, ; Step 3: Based on the pre-determined search area, obtain a list of images to be analyzed. ; Step 4: From Extract a scene from the image ; Step 5: For the image and multi-directional template collection Using a multi-directional template matching method with a threshold of 0.5, the first-stage candidate target set was extracted. Extracting the candidate target set in the first stage is done by utilizing Three bands are used to rotate the samples in multiple directions for template matching to obtain candidate targets in the first stage. Since the data are from the same source and the spatial resolution of the test data and the samples is consistent, there is no need to consider multi-scale issues. The template matching adopts the normalized cross-correlation coefficient matching method, and the output value range of the matching is [-1, 1]. Therefore, the matching result in each direction is a single-channel matching score map with the same pixel size as the test data. The maximum value of the matching scores from multiple directions is taken at each pixel location to obtain the final matching score map of the image under test. And continue to use the threshold method to... Convert to a binary image indicating candidate targets , Each patch represents a candidate target, and the maximum value within the patch's range is taken as the score. The location of the maximum value is used as the candidate target center. Set the length and width to be consistent with the sample, and the resulting candidate targets are... These candidate targets are recorded and collected into a list as the first-stage search results; Step Six: Constructing Samples and The spectral features of all candidate targets are analyzed, and the cosine similarity between the features of the sample and each candidate target is calculated. A threshold of 0.5 is set to filter out the high-suspicious targets in the second stage. ; Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full Collect global target set In the process, a spatial spectrum visualization image of each target is generated for easy manual verification; Step 8: ,if Smaller than image list Given the length, skip to step three; Step 9: Output the final result ,Finish.

2. The method for detecting specific maritime targets in medium- and low-resolution multispectral imagery according to claim 1, characterized in that: In step one, target sample acquisition refers to acquiring at least one target sample based on historical imagery data, using the chosen data source. For ship targets, matching sample data is acquired from the satellite historical archive database using AIS ship trajectory data, based on the ship or other ships of the same type. Assuming the image data source has [number of bands]... The sample is Three-dimensional matrix data, The height and width are respectively, with the sample center as the target, and the requirements are... and It should be slightly larger than the width and height of the target, while retaining some background information.

3. The method for detecting specific targets at sea in medium- and low-resolution multispectral imagery according to claim 1, characterized in that: The formula for calculating the final matching score image is as follows: ① ② ③ ④ ①②③ represent multi-directional template matching, and ④ represents taking the maximum value of the multi-directional matching results; where For images, The rotated template, This refers to the width and height of the template.

4. The method for detecting specific maritime targets in medium- and low-resolution multispectral imagery according to claim 1, characterized in that: In step six, the screening of highly suspicious targets in the second stage involves extracting their coordinates based on the location coordinates of the candidate targets obtained in the first stage. place Image data blocks are constructed by comparing spectral features with samples to achieve the purpose of screening and identification.

5. A method for detecting specific maritime targets in medium- and low-resolution multispectral imagery according to claim 4, characterized in that: The specific steps for screening highly suspicious targets in the second phase are as follows: S1: Constructing Spectral Features: Due to the influence of factors such as the angle of solar illumination and changes in atmospheric conditions, the reflectance intensity of a target at the same location varies in images from different periods. Therefore, when constructing spectral features, the seawater in the target background is used as a calibration. The OTSU adaptive threshold segmentation method is used to segment the target data into foreground and background. First, the quartile outlier detection method is used to remove outliers from each band of the background pixel data. Then, the mean is calculated to obtain the average spectral curve of the background pixels. Then, for all foreground pixels, the average spectral curve of the background is subtracted, and the first difference is calculated in the spectral dimension. The mean is then calculated to obtain the average first difference curve of the foreground target, which is used as the spectral feature. The specific process is as follows: set up For input data, To intercept it Three bands ； ； ； ； In the formula, removeOutliers is the quartile outlier removal function, which acts independently on each band; diff is the first-order difference in the spectral dimension. Features for the final output; S2: Similarity Calculation: For samples and candidate targets to be evaluated The corresponding spectral features were calculated using the method in S1. and Calculate their cosine similarity: ； The cosine similarity ranges from [-1, 1]. A threshold method is used, empirically set between 0.2 and 0.

5. All candidate targets exceeding the threshold are retained as the final highly suspicious targets in the output. Formatted records.

Images