Multi-source ship target identification method based on two-stage collaborative fusion

Abstract

The invention discloses a multi-source ship target identification method based on two-stage collaborative fusion, and relates to the technical field of signal processing, and the method comprises the steps: firstly collecting target sea area data, including original optical image and radar image data, by means of an optical camera and a radar sensor on an aircraft, and then carrying out the preprocessing of the collected data through Gaussian filtering; performing sea-land segmentation by using a deep learning algorithm, constructing a loss function optimization network parameter to obtain a segmentation mask, and removing a land background area according to a mask result to obtain processed optical and radar images; an improved SURF feature and graph cut algorithm combined registration method is adopted, optimal transformation parameters are obtained by screening feature point pairs and solving the minimum value of an energy function, and image registration is achieved; for the registered images, fusing the images based on an adaptive fusion rule; and finally, extracting corresponding features through manual feature extraction and feature extraction based on deep learning, and fusing the features to obtain a final target recognition result.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a multi-source ship target recognition method based on two-level collaborative fusion. Background Art

[0002] Multi-source refers to the use of data from different types of sensors for information fusion. Common multi-source data include optical images, radar images, etc. Optical images provide high-resolution image information, which can clearly present the texture, color and details of objects; radar images obtain target information through electromagnetic wave reflection, especially in severe weather conditions, such as haze, night, rain and snow, etc., and have strong robustness. By fusing these two types of data, the limitations of a single data source can be compensated and the accuracy and robustness of target recognition can be improved.

[0003] Multi-source ship target recognition refers to the technology of comprehensive analysis and recognition of ship targets by integrating multiple information sources such as optical images and radar images. Ship target recognition is of great significance in the fields of maritime surveillance, maritime security, military reconnaissance, etc. Optical images and radar images each have different advantages and disadvantages. Optical images perform well under visible light, but are greatly affected by environmental conditions; while radar images can provide stable target detection capabilities in complex environments, but the resolution is relatively low. By fusing optical images and radar images, not only can the advantages of the two data sources be fully utilized, but more comprehensive and accurate ship target recognition results can be provided under different scenarios and conditions. For example, during the day or when the visibility is good, the optical image can clearly present the appearance characteristics of the ship, while at night or when the visibility is poor, the radar image can obtain the outline information of the ship through the reflected wave, thereby improving the recognition accuracy and adaptability.

[0004] Multi-source ship target recognition technology has great application potential, but it still faces many technical difficulties in practical applications. First, optical images and radar images come from different sensors and have different imaging mechanisms and data characteristics. Optical images usually have high spatial resolution and rich texture information, but they are easily affected by environmental factors such as weather, lighting, and haze, resulting in loss or blurring of target information. Radar images detect targets through electromagnetic wave reflection, have strong anti-interference ability, can penetrate clouds and night environments, but have relatively low resolution and can only provide target contour information. Therefore, how to overcome the differences and complementarity of these two data sources while ensuring recognition accuracy has become the primary challenge facing multi-source ship target recognition. Secondly, the fusion of image data is one of the difficulties in the technology. Optical images and radar images not only differ in spatial resolution, but also in their imaging principles and data formats. For example, optical images are based on visible light reflection, while radar images rely on the reflection characteristics of electromagnetic waves. This difference makes the image fusion process complicated. Common image fusion methods include pixel-level fusion, feature-level fusion, and decision-level fusion, but the fusion effects and adaptation scenarios of different methods are different. Pixel-level fusion performs well in detail recovery, but is easily affected by noise; feature-level fusion can extract more recognizable features, but some detail information may be lost during the fusion process; and decision-level fusion is integrated after target recognition, but this method has high requirements for the accuracy of the recognition model. In addition, in the process of multi-source image fusion, data synchronization and alignment are also important challenges. When different types of sensors collect images, due to differences in time, space, and perspective, the image data may be inconsistent, which in turn affects the quality of the fusion results. Therefore, how to accurately align images from different sources and effectively fuse data is the key to improving the performance of multi-source ship target recognition. Finally, feature extraction and selection issues also pose challenges to multi-source ship target recognition technology. The fused image contains a lot of redundant information and noise, and for the accurate recognition of ship targets, relying solely on traditional manual feature extraction methods may not meet the requirements of high precision and high real-time performance. Although deep learning methods have shown powerful capabilities in feature extraction, how to efficiently extract useful features and optimize them is still an urgent problem to be solved.

[0005] In summary, although multi-source ship target recognition technology has great potential in improving target recognition accuracy and expanding its scope of application, due to technical issues such as sensor differences, image fusion, data synchronization and feature selection, further research and innovation are still needed to improve its application effect and stability in complex environments. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-source ship target recognition method based on two-level collaborative fusion to solve the problems of poor accuracy and low efficiency of target recognition results caused by sensor differences, image fusion, data synchronization and feature selection in the prior art.

[0007] To achieve the above object, the present invention provides a multi-source ship target recognition method based on two-level collaborative fusion, comprising the following steps: Step 1: Use the optical camera and radar sensor on board the aircraft to synchronously collect data on the target sea area and obtain the original optical image data. and radar image data ,in Represents the pixel coordinates in the image plane, and then preprocesses it with Gaussian filtering to obtain the filtered optical image and the filtered radar image ; Step 2: Use deep learning algorithms to perform land-sea segmentation on the preprocessed optical image and radar image, and obtain the optical image and radar image results after setting the land area of ​​the original image to zero, which are , ; Step 3: Use the registration method based on the improved SURF (Speed ​​Up Robust Features) feature and the graph cut algorithm to and Perform image registration to obtain the registered optical image and radar images ; Step 4: For the registered optical image and radar images The fused image is obtained by using an adaptive fusion rule based on regional energy, gradient and structure tensor information. ; Step 5: Extract corresponding features from the fused image obtained in step 4 through manual feature extraction and feature extraction based on deep learning, and then fuse the extracted features to obtain the final target recognition result.

[0008] Preferably, the formula of Gaussian filtering is as follows: ; in is the Gaussian kernel, defined as: ; In the formula, represents the filtered image, represents the original image, Represents the standard deviation of the Gaussian kernel, controlling the width and height of the kernel, Indicates the horizontal coordinate increment of the input data point, Indicates the vertical coordinate increment of the input data point.

[0009] Preferably, in step 2, the deep learning algorithm is used to perform land-sea segmentation on the preprocessed optical image and radar image respectively, and the process of obtaining the optical image and radar image after setting the land area of ​​the original image to zero is as follows: S21. Construct loss function Optimize the parameters of the sea-land segmentation network to obtain the sea-land segmentation mask; among them, the loss function The expression is as follows: ; In the formula, is the sample size, is the number of categories (both land and sea), For sample Belongs to category The real label, Predict samples for the model Belongs to category The probability value of For Category The weight coefficient is adjusted adaptively according to the proportion of each category in the sample. For example, if the proportion of sea pixels is , the proportion of land pixels is , then the sea category weight , weight of category ; S22. Based on the result of the segmentation mask, only the pixel parts corresponding to the coastal and sea areas in the image are retained, where the sea and land are represented by specific pixel values, 1 for the sea area and 0 for the land area, and the land background area is removed to obtain the optical image and radar image results after setting the land area of ​​the original image to zero. The expression is as follows: ; ; In the formula, and They are the optical image and radar image results after setting the land area of ​​the original image to zero. represents the land and sea segmentation mask of the optical image, Represents the land and sea segmentation mask of the radar image. Indicates that the pixel value of the point with coordinate (x, y) in the sea-land segmentation mask of the optical image is 1, Indicates that the pixel value of the point with coordinates (x, y) in the optical image sea and land segmentation mask is 0. Indicates that the pixel value of the point with coordinates (x, y) in the sea and land segmentation mask of the radar image is 1. Indicates that the pixel value of the point with coordinates (x, y) in the sea and land segmentation mask of the radar image is 0.

[0010] Preferably, in step 3, a registration method based on a combination of improved SURF features and graph cut algorithm is used to and Perform image registration to obtain the registered optical image and radar images The process is as follows: S31. Use the improved SURF algorithm to extract feature points. In the feature descriptor generation stage, introduce texture features based on local binary patterns to enhance the description ability of feature points and their robustness to illumination changes. For feature points in optical images, and feature points in radar images ,Reliable feature point pairs are screened out through the feature matching algorithm based on the bidirectional nearest neighbor distance ratio (BB-NNDR); S32. Use the graph cut algorithm to solve the minimum value of the energy function and obtain the optimal transformation parameters so that the radar image can be accurately transformed into the same coordinate system as the optical image. Energy function By data item and smoothing term The composition is as follows: ; ; ; ; ; in, is the set of all pixels in the image, is a set of matching feature point pairs, is a set of adjacent pixel pairs, is the transformation matrix used to map points in the radar image to the optical image coordinate system. is the balance coefficient, is a pair of adjacent pixels and The distance between is a set of adjacent pixel pairs, is a parameter that controls the degree of smoothing. is a similarity measurement function based on conditional probability, is the similarity measure, It represents the smoothness measure between adjacent pixel pairs; S33, find the corresponding relationship between the images through S31 and S32, achieve accurate registration, and obtain the registered optical image and radar images .

[0011] Preferably, in step 4, for the registered optical image and radar images The fused image is obtained by using an adaptive fusion rule based on regional energy, gradient and structure tensor information. The process is as follows: S41. Calculation of images by The local window at the center The regional energy , gradient amplitude and the structure tensor eigenvalues , , , the expression is as follows: ; ; ; By changing the structure tensor matrix Perform eigenvalue decomposition to obtain and ; S42. Calculate the weight coefficient of the optical image and the weight coefficient of the radar image , the expression is as follows: ; ; In the formula, For an image , which is based on The local window at the center The regional energy within and For local window The horizontal and vertical coordinate values ​​of the pixel points within, Representing images In coordinates The gradient amplitude at and is an image exist and Directional partial derivatives, subscripts opt and rad denote optical and radar, respectively; Represents a structure tensor The characteristic value of S43, obtain the fused image , the expression is as follows: .

[0012] Preferably, manual feature extraction utilizes an edge detection algorithm based on the Canny operator to extract the outline of the ship, and accurately determines the boundary of the ship target by performing morphological processing (such as dilation and erosion operations) and a contour tracking algorithm on the edge image, and then calculates multiple geometric shape features of the ship, including aspect ratio, perimeter, and area change rate.

[0013] Preferably, the feature extraction of deep learning adopts an architecture combining a multi-branch convolutional neural network (M-CNN) and an attention mechanism; the multi-branch convolutional neural network processes different scales and feature levels of the fused image respectively, and each branch contains several convolution layers, pooling layers and residual connection modules to extract rich image features; the attention mechanism enables the model to focus on the ship target area and suppress the interference of background noise by learning the importance weights of different areas in the image; in the training stage, a large number of labeled ship target fusion image samples (including the above-mentioned manual features as auxiliary input) are used for training; the features extracted by the manual features and the image features extracted by each branch network are spliced ​​and fused before the fully connected layer, and finally used as the comprehensive input feature vector of the model.

[0014] Therefore, the present invention adopts the above-mentioned multi-source ship target recognition method based on two-level collaborative fusion, which has the following beneficial effects: (1) With the help of scene segmentation technology, the effective part containing only the coastal and sea areas is accurately retained, and irrelevant background information such as land is discarded, which greatly reduces the amount of data processing and enables efficient use of computing resources, thereby significantly improving the overall processing efficiency and laying a good foundation for subsequent target recognition work; (2) For ship targets, we innovatively extract features such as geometric shape and aspect ratio that are easy to obtain manually and simple to calculate, and use these features as powerful auxiliary inputs for the deep learning model. As a result, the deep learning model no longer needs to spend a lot of time and resources to repeatedly extract these simple features, but can focus more on mining deep and more discriminative feature information, effectively reducing the model training and recognition time and greatly improving the recognition efficiency. (3) A two-level collaborative fusion strategy of pixel level and feature level is adopted. The original information of optical image and radar image is fully integrated in pixel level fusion, and the feature combination is further optimized in feature level fusion. The two complement each other, making the feature description of ship target more comprehensive and accurate, thereby improving the accuracy of target recognition in all aspects and providing a more reliable technical means for ship target recognition in complex marine environments.

[0015] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention is an overall flow chart of a multi-source ship target recognition method based on two-level collaborative fusion. DETAILED DESCRIPTION

[0017] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1 , a multi-source ship target recognition method based on two-level collaborative fusion, comprising the following steps: Step 1: Use the optical camera and radar sensor on board the aircraft to synchronously collect data on the target sea area and obtain the original optical image data. and radar image data ,in Represents the pixel coordinates in the image plane, and then preprocesses it with Gaussian filtering to obtain the filtered optical image and the filtered radar image ; The formula for Gaussian filtering is as follows: ; in is the Gaussian kernel, defined as: ; In the formula, represents the filtered image, represents the original image, Represents the standard deviation of the Gaussian kernel, controlling the width and height of the kernel, Indicates the horizontal coordinate increment of the input data point, Indicates the vertical coordinate increment of the input data point.

[0019] Step 2: Use deep learning algorithms to perform land-sea segmentation on the preprocessed optical image and radar image, and obtain the optical image and radar image results after setting the land area of ​​the original image to zero, which are , ; The specific process is as follows: S21. Construct loss function Optimize the sea-land segmentation network parameters to obtain the sea-land segmentation mask; the loss function uses a combination of the dice coefficient loss function and the cross entropy loss function to deeply learn and optimize the network parameters to solve the problem of imbalance between sea and land sample categories; among them, the loss function The expression is as follows: ; In the formula, is the sample size, is the number of categories (both land and sea), For sample Belongs to category The real label, Predict samples for the model Belongs to category The probability value of For Category The weight coefficient is adjusted adaptively according to the proportion of each category in the sample. For example, if the proportion of sea pixels is , the proportion of land pixels is , then the sea category weight , weight of category ; S22. Based on the result of the segmentation mask, only the pixel parts corresponding to the coastal and sea areas in the image are retained, where the sea and land are represented by specific pixel values, 1 for the sea area and 0 for the land area, and the land background area is removed to obtain the optical image and radar image results after setting the land area of ​​the original image to zero. The expression is as follows: ; ; In the formula, and They are the optical image and radar image results after setting the land area of ​​the original image to zero. represents the land and sea segmentation mask of the optical image, Represents the land and sea segmentation mask of the radar image. Indicates that the pixel value of the point with coordinate (x, y) in the sea-land segmentation mask of the optical image is 1, Indicates that the pixel value of the point with coordinates (x, y) in the optical image sea and land segmentation mask is 0. Indicates that the pixel value of the point with coordinates (x, y) in the sea and land segmentation mask of the radar image is 1. Indicates that the pixel value of the point with coordinates (x, y) in the sea and land segmentation mask of the radar image is 0.

[0020] This step greatly reduces the amount of data and computational complexity of subsequent processing, while eliminating land background interference, laying the foundation for subsequent precise target identification.

[0021] Step 3: Considering the differences in imaging principles between optical images and radar images and factors such as platform motion, the two images need to be registered with high precision before fusion. The registration method based on the improved SURF (Speeded Up Robust Features) and the graph cut algorithm is used to register the two images. and Perform image registration to obtain the registered optical image and radar images ; The specific process is as follows: S31. Use the improved SURF algorithm to extract feature points. In the feature descriptor generation stage, introduce texture features based on local binary patterns to enhance the description ability of feature points and their robustness to illumination changes. For feature points in optical images, and feature points in radar images ,Reliable feature point pairs are screened out through the feature matching algorithm based on the bidirectional nearest neighbor distance ratio (BB-NNDR); S32. Use the graph cut algorithm to solve the minimum value of the energy function and obtain the optimal transformation parameters so that the radar image can be accurately transformed into the same coordinate system as the optical image. Energy function By data item and smoothing term The composition is as follows: ; ; ; ; ; in, is the set of all pixels in the image, is a set of matching feature point pairs, is a set of adjacent pixel pairs, is the transformation matrix used to map points in the radar image to the optical image coordinate system. is the balance coefficient, is a pair of adjacent pixels and The distance between is a set of adjacent pixel pairs, is a parameter that controls the degree of smoothing. is a similarity measurement function based on conditional probability, is the similarity measure, It represents the smoothness measure between adjacent pixel pairs; S33, find the corresponding relationship between the images through S31 and S32, achieve accurate registration, and obtain the registered optical image and radar images .

[0022] Step 4: For the registered optical image and radar images The fused image is obtained by using an adaptive fusion rule based on regional energy, gradient and structure tensor information. ; The specific process is as follows: S41. Calculation of images by The local window at the center The regional energy , gradient amplitude and the structure tensor eigenvalues , , , the expression is as follows: ; ; ; By changing the structure tensor matrix Perform eigenvalue decomposition to obtain and ; S42. Calculate the weight coefficient of the optical image and the weight coefficient of the radar image , the expression is as follows: ; ; In the formula, For an image , which is based on The local window at the center The regional energy within and For local window The horizontal and vertical coordinate values ​​of the pixel points within, Representing images In coordinates The gradient amplitude at and is an image exist and Directional partial derivatives, subscripts opt and rad denote optical and radar, respectively; Represents a structure tensor The characteristic value of S43, obtain the fused image , the expression is as follows: .

[0023] This adaptive fusion rule can dynamically assign weights according to the local features of the image, so that better fusion effects can be obtained in both the target area (such as the ship part) and the background area (such as the sea surface), improving the contrast and clarity between the target and the background, and enhancing the recognizability of the image.

[0024] Step 5: Extract the corresponding features of the fused image obtained in step 4 through manual feature extraction and feature extraction based on deep learning, and then fuse the extracted features to obtain the final target recognition result; for the image after pixel-level fusion, first perform manual feature extraction to obtain some intuitive and easy-to-calculate ship target features, which can provide important prior information for the subsequent deep learning model and reduce the learning burden and calculation amount of the model. Among them, manual feature extraction uses the edge detection algorithm based on the Canny operator to extract the outline of the ship, and accurately determines the boundary of the ship target by performing morphological processing (such as expansion and corrosion operations) and contour tracking algorithm on the edge image, and then calculates multiple geometric shape features of the ship, including aspect ratio, perimeter, and area change rate. The feature extraction of deep learning adopts the architecture that combines the multi-branch convolutional neural network (M-CNN) with the attention mechanism. The multi-branch convolutional neural network processes different scales and feature levels of the fused image respectively. Each branch contains several convolution layers, pooling layers and residual connection modules to extract rich image features. The attention mechanism learns the importance weights of different areas in the image to make the model focus on the ship target area and suppress the interference of background noise. In the training stage, a large number of labeled ship target fusion image samples (including the above-mentioned manual features as auxiliary input) are used for training. The features extracted by the manual features and the image features extracted by each branch network are spliced ​​and fused before the fully connected layer, and finally used as the comprehensive input feature vector of the model.

[0025] Therefore, the present invention adopts the above-mentioned multi-source ship target recognition method based on two-level collaborative fusion, and achieves the final target recognition result through the combination of multiple steps and multiple technologies; first, the target sea area data, including the original optical image and radar image data, are collected by means of the optical camera and radar sensor on the aircraft, and then the collected data is preprocessed by Gaussian filtering. Next, the deep learning algorithm is used to segment the land and sea, and the loss function is constructed to optimize the network parameters to obtain the segmentation mask, and then the land background area is removed according to the mask result to obtain the processed optical and radar images. After that, the registration method combining the improved SURF feature and the graph cut algorithm is used to extract the feature points and enhance their description ability, and the optimal transformation parameters are obtained by screening the feature point pairs and solving the minimum value of the energy function, thereby realizing image registration. For the registered image, the image is fused based on the adaptive fusion rule of regional energy, gradient and structure tensor information. On the basis of the fused image, on the one hand, manual feature extraction is performed by edge detection algorithms based on the Canny operator, and on the other hand, a deep learning method combining a multi-branch convolutional neural network with an attention mechanism is used to extract features, and finally the manual features and deep learning features are fused to obtain the final target recognition result.

[0026] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A multi-source ship target recognition method based on two-level collaborative fusion, characterized in that: The following steps are involved: Step 1: Use the optical camera and radar sensor on board the aircraft to synchronously collect data on the target sea area and obtain the original optical image data. and radar image data ,in Represents the pixel coordinates in the image plane, and then preprocesses it with Gaussian filtering to obtain the filtered optical image and the filtered radar image ; Step 2: Use deep learning algorithms to perform land-sea segmentation on the preprocessed optical image and radar image, and obtain the optical image and radar image results after setting the land area of ​​the original image to zero, which are , ; Step 3: Use the registration method based on the improved SURF feature and the graph cut algorithm to and Perform image registration to obtain the registered optical image and radar images ; Step 4: For the registered optical image and radar images The fused image is obtained by using an adaptive fusion rule based on regional energy, gradient and structure tensor information. ; Step 5: Extract corresponding features from the fused image obtained in step 4 through manual feature extraction and feature extraction based on deep learning, and then fuse the extracted features to obtain the final target recognition result.

2. According to claim 1, a multi-source ship target recognition method based on two-level collaborative fusion is characterized in that: The formula for Gaussian filtering is as follows: ； in is the Gaussian kernel, defined as: ； In the formula, represents the filtered image, represents the original image, Represents the standard deviation of the Gaussian kernel, controlling the width and height of the kernel, Indicates the horizontal coordinate increment of the input data point, Indicates the vertical coordinate increment of the input data point.

3. According to claim 2, a multi-source ship target recognition method based on two-level collaborative fusion is characterized in that: In step 2, the deep learning algorithm is used to segment the land and sea of ​​the preprocessed optical image and radar image respectively. The process of obtaining the optical image and radar image after setting the land area of ​​the original image to zero is as follows: S21. Construct loss function Optimize the parameters of the land-sea segmentation network to obtain the land-sea segmentation mask; the loss function The expression is as follows: ； In the formula, is the sample size, is the number of categories, For sample Belongs to category The real label, Predict samples for the model Belongs to category The probability value of For Category The weight coefficient is adaptively adjusted according to the proportion of each category in the sample; S22. Based on the result of the segmentation mask, only the pixel parts corresponding to the coastal and sea areas in the image are retained, where the sea and land are represented by specific pixel values, 1 for the sea area and 0 for the land area, and the land background area is removed to obtain the optical image and radar image results after setting the land area of ​​the original image to zero. The expression is as follows: ； ； In the formula, and They are the results of the optical image and radar image after setting the land area of ​​the original image to zero, represents the land and sea segmentation mask of the optical image, Represents the land and sea segmentation mask of the radar image; Indicates that the pixel value of the point with coordinate (x, y) in the optical image sea and land segmentation mask is 1, Indicates that the pixel value of the point with coordinate (x, y) in the optical image sea and land segmentation mask is 0. Indicates that the pixel value of the point with coordinates (x, y) in the sea and land segmentation mask of the radar image is 1. Indicates that the pixel value of the point with coordinates (x, y) in the sea and land segmentation mask of the radar image is 0.

4. The multi-source ship target recognition method based on two-level collaborative fusion according to claim 3 is characterized by: In step 3, a registration method based on improved SURF features and graph cut algorithm is used to and Perform image registration to obtain the registered optical image and radar images The process is as follows: S31, using the improved SURF algorithm to extract feature points, in the feature descriptor generation stage, the texture feature based on the local binary pattern is introduced; for the feature points in the optical image and feature points in radar images ,Reliable feature point pairs are screened out through a feature matching algorithm based on the bidirectional nearest neighbor distance ratio; S32, use the graph cut algorithm to solve the minimum value of the energy function and obtain the optimal transformation parameters. The energy function By data item and smoothing term The composition is as follows: ； ； ； ； ； in, is the set of all pixels in the image, is a set of matching feature point pairs, is a set of adjacent pixel pairs, is the transformation matrix used to map points in the radar image to the optical image coordinate system. is the balance coefficient, is a pair of adjacent pixels and The distance between is a set of adjacent pixel pairs, is a parameter that controls the degree of smoothing. is a similarity measurement function based on conditional probability, is the similarity measure, It represents the smoothness measure between adjacent pixel pairs; S33, find the corresponding relationship between the images through S31 and S32, achieve accurate registration, and obtain the registered optical image and radar images .

5. The multi-source ship target recognition method based on two-level collaborative fusion according to claim 4 is characterized in that: In step 4, the optical image after registration and radar images The fused image is obtained by using an adaptive fusion rule based on regional energy, gradient and structure tensor information. The process is as follows: S41. Calculation of images by The local window at the center The regional energy , gradient amplitude and the structure tensor eigenvalues , , , the expression is as follows: ； ； ； By changing the structure tensor matrix Perform eigenvalue decomposition to obtain and ; S42. Calculate the weight coefficient of the optical image and the weight coefficient of the radar image , the expression is as follows: ； ； In the formula, For an image , which is based on The local window at the center The regional energy within and For local window The horizontal and vertical coordinate values ​​of the pixel points within, Representing images In coordinates The gradient amplitude at and is an image exist and Directional partial derivatives, subscripts opt and rad denote optical and radar, respectively; Represents a structure tensor The characteristic value of S43, obtain the fused image , the expression is as follows: 。 6. The multi-source ship target recognition method based on two-level collaborative fusion according to claim 5 is characterized in that: Manual feature extraction uses an edge detection algorithm based on the Canny operator to extract the outline of the ship. By performing morphological processing and contour tracking algorithm on the edge image, the boundary of the ship target is accurately determined, and then multiple geometric shape features of the ship are calculated, including aspect ratio, perimeter, and area change rate.

7. The multi-source ship target recognition method based on two-level collaborative fusion according to claim 6 is characterized by: The feature extraction of deep learning adopts the architecture that combines multi-branch convolutional neural network with attention mechanism. The multi-branch convolutional neural network processes different scales and feature levels of the fusion image respectively. Each branch contains several convolutional layers, pooling layers and residual connection modules to extract rich image features. The attention mechanism learns the importance weights of different areas in the image, allowing the model to focus on the ship target area and suppress the interference of background noise.

Images