Multi-source remote sensing image fusion method, medium and system for ship detection

Abstract

The invention provides a ship detection-oriented multi-source remote sensing image fusion method, a ship detection-oriented multi-source remote sensing image fusion medium and a ship detection-oriented multi-source remote sensing image fusion system, and belongs to the technical field of remote sensing images. Constructing an electromagnetic scattering constraint model based on physical optical approximation and a geometric diffraction theory to calculate a predicted value of the backscattering cross section of the target, introducing a fusion cost function, generating a fusion weight matrix by adopting an adaptive feature weighting model, and performing multi-scale fusion on the image through hypercomplex wavelet transform; and iteratively optimizing the fusion result by using an information diffusion heat conduction equation, and performing physical consistency correction to output an enhanced fusion image, thereby solving the technical problem of poor physical consistency of the fusion result caused by lack of physical model constraint when the synthetic aperture radar image and the multispectral image are fused.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image technology, and specifically relates to a multi-source remote sensing image fusion method, medium and system for ship detection. Background Technology

[0002] Ship target detection relies on the spatial resolution and spectral information of remote sensing imagery. Traditional methods utilize the fusion of synthetic aperture radar (SAR) imagery and multispectral imagery to leverage the advantages of both types of data. SAR imagery offers all-weather, all-day imaging capabilities and high spatial resolution, while multispectral imagery provides rich spectral information for target identification. Existing fusion methods employ techniques such as pixel-level weighting, wavelet transform, and deep learning to achieve image fusion, playing a crucial role in applications such as ship detection, marine monitoring, and port management. However, existing fusion methods only perform image synthesis from a mathematical perspective, neglecting the physical characteristics of electromagnetic scattering from ship targets. This leads to discrepancies between the intensity values ​​of target regions in the fused image and the actual radar backscattering cross section. The fusion result violates the physical laws of electromagnetic scattering, resulting in decreased fusion quality in complex sea conditions and scenarios with multiple types of ship targets, thus affecting the accuracy of subsequent detection algorithms. In other words, existing technologies suffer from a lack of physical model constraints when fusing SAR and multispectral imagery, leading to poor physical consistency in the fusion results. Summary of the Invention

[0003] In view of this, the present invention provides a multi-source remote sensing image fusion method, medium and system for ship detection, which can solve the technical problem in the prior art that the lack of physical model constraints when fusing synthetic aperture radar images and multispectral images leads to poor physical consistency of the fusion results.

[0004] The present invention is implemented as follows: The first aspect of the present invention provides a multi-source remote sensing image fusion method for ship detection, comprising the following steps: acquiring a synthetic aperture radar (SAR) image to be fused and a multispectral image to be fused; performing speckle suppression processing on the SAR image to be fused to obtain a speckled SAR image; extracting the full polarization data of the SAR image to be fused and constructing a covariance matrix; performing parallel eigenvalue decomposition on the covariance matrix to obtain polarization eigenvectors; constructing an electromagnetic scattering constraint model for the ship target; calculating the predicted backscattering cross section of the target based on the physical optics approximation principle and geometric diffraction theory; introducing the residual between the predicted backscattering cross section of the target and the intensity value of the SAR image to be fused as a constraint term into the fusion cost function to obtain a physically constrained fusion cost function; and inputting the speckled SAR image and the multispectral image to be fused into an adaptive feature weighting model. Output the fusion weight matrix; organize the real and imaginary parts of the synthetic aperture radar image to be fused and the multispectral image to be fused into a hypercomplex form, and perform hypercomplex wavelet transform decomposition into low-frequency sub-bands and high-frequency sub-bands. For the low-frequency sub-band, the local variance weighted fusion rule combined with the fusion weight matrix is ​​used to obtain the fused low-frequency sub-band, and the absolute value selection rule is used to obtain the fused high-frequency sub-band. Define the local entropy value of the image as the information temperature, construct the finite difference iterative format of the information diffusion heat conduction equation, and use the fused low-frequency sub-band and fused high-frequency sub-band as the initial temperature field. Iterative calculation realizes the information flow from the high-entropy region to the low-entropy region. When the system free energy converges to the minimum value, the iteration terminates to obtain the equilibrium temperature field; perform inverse transform reconstruction on the hypercomplex wavelet coefficients corresponding to the equilibrium temperature field, and perform physical consistency correction combined with the physical constraint fusion cost function to output the enhanced fused image for ship target detection.

[0005] Specifically, the speckle suppression process involves using a nonlocal mean filter to calculate pixel block similarity weights and combining a wavelet domain Bayesian shrinkage estimator to perform adaptive thresholding of high-frequency coefficients.

[0006] Specifically, the nonlocal mean filter involves setting a search window around the pixel to be processed, calculating the Euclidean distance between the pixel image block to be processed and other image blocks, converting the Euclidean distance into similarity weights, and using the similarity weights to perform a weighted average of all pixels within the search window to obtain the filtered output value.

[0007] Specifically, the wavelet domain Bayesian shrinkage estimator performs multi-scale wavelet decomposition on the image corresponding to the filtered output value, establishes a generalized Gaussian distribution prior model for the high-frequency coefficients at each scale, estimates the Bayesian shrinkage threshold based on the noise variance and signal variance, and shrinks the high-frequency coefficients toward zero.

[0008] The fully polarized data includes data from four channels: HH, HV, VH, and VV. The covariance matrix is ​​constructed as a complex Hermitian covariance matrix with three rows and three columns or four rows and four columns.

[0009] Specifically, the parallelized eigenvalue decomposition involves dividing the synthetic aperture radar image to be fused into multiple image blocks and assigning them to different threads of the graphics processing unit. Each thread independently completes the eigenvalue decomposition of the covariance matrix of all pixels within an image block, extracting eigenvalues ​​and eigenvectors as polarization eigenvectors.

[0010] Specifically, the application of the physical optics approximation principle involves discretizing the surface of the ship target into multiple surface elements, calculating the induced current density under incident wave illumination for each surface element, and calculating the far-field scattering field intensity by integrating the induced current density of the surface elements on the target surface.

[0011] Specifically, the application of the geometric diffraction theory involves identifying the edges, tips, and corners of ship targets as diffraction sources, calculating the diffraction field generated by the incident wave at the diffraction source, and superimposing the diffraction field with the direct reflection field corresponding to the approximate backscattering cross-section value of physical optics.

[0012] Specifically, the construction of the physical constraint fusion cost function involves calculating the difference between the intensity value of the synthetic aperture radar image to be fused and the predicted value of the target backscattering cross section as the physical constraint residual term, calculating the spectral distortion between the fused image and the multispectral image to be fused as the spectral fidelity term, and calculating the spatial gradient of the fused image as the detail enhancement term.

[0013] Specifically, the structure of the adaptive feature weighting model is as follows: the input layer receives speckle-removing synthetic aperture radar images and multispectral images to be fused; the feature extraction layer uses multi-scale convolutional kernels to extract spatial features of different receptive fields in parallel; the attention layer calculates feature importance weights based on a query key-value pair mechanism; and the fusion weight generation layer maps the enhanced feature representation to the fusion weight matrix corresponding to each pixel.

[0014] Specifically, the attention layer calculates feature importance weights based on a query key-value pair mechanism. Specifically, it maps the spatial features output by the feature extraction layer into query vectors, key vectors, and value vectors. It calculates the attention score by performing a dot product operation between the query vector and the key vector, normalizes the attention score to obtain the attention weight, and uses the attention weight to perform a weighted summation of the value vector to obtain the enhanced feature representation.

[0015] Specifically, the organization of the hypercomplex form involves taking the real part of the synthetic aperture radar image to be fused as the first hypercomplex component, the imaginary part of the synthetic aperture radar image to be fused as the second hypercomplex component, and the different bands of the multispectral image to be fused as the third and fourth hypercomplex components, respectively.

[0016] Specifically, the local variance weighted fusion rule calculates the variance of pixel values ​​within a neighborhood window for each pixel position in the low-frequency sub-band as the local variance, normalizes the local variance to obtain the local variance fusion weight, and multiplies the local variance fusion weight with the weight value at the corresponding position in the fusion weight matrix to obtain the comprehensive fusion weight.

[0017] Specifically, the construction of the information diffusion heat conduction equation involves calculating the Shannon entropy based on the probability distribution of pixel values ​​within a local window of the image, using it as the information temperature of the local window. A two-dimensional heat conduction partial differential equation is then constructed to describe the evolution of the information temperature over time and space. In the two-dimensional heat conduction partial differential equation, the heat diffusion coefficient is determined by the local gradient of the image and the structure tensor.

[0018] Specifically, the finite difference iterative scheme discretizes the fused low-frequency subband and the fused high-frequency subband into a two-dimensional grid, discretizes the two-dimensional heat conduction partial differential equation in space and time, calculates the spatial second derivative using the central difference scheme, calculates the temporal first derivative using the forward difference scheme, and obtains the recursive formula for the information temperature at the next time step.

[0019] Specifically, the physical consistency correction involves calculating the deviation between the enhanced fused image and the predicted value of the target backscattering cross section based on the physical constraint fusion cost function, adjusting the intensity value of pixels with deviations exceeding a threshold, so as to minimize the residual between the adjusted enhanced fused image and the predicted value of the target backscattering cross section.

[0020] A second aspect of the present invention provides a computer-readable storage medium storing program instructions that, when executed in a computer, perform the aforementioned multi-source remote sensing image fusion method for ship detection.

[0021] A third aspect of the present invention provides a multi-source remote sensing image fusion system for ship detection, comprising the aforementioned computer-readable storage medium, wherein the system is a computer, the computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor for executing program instructions stored in the computer-readable storage medium.

[0022] This invention constructs an electromagnetic scattering constraint model for ship targets and calculates predicted backscattering cross-sections based on physical optics approximation principles and geometric diffraction theory. The residual between the predicted values ​​and the intensity values ​​of synthetic aperture radar (SAR) images is introduced as a constraint term into the fusion cost function. During the fusion process, physical consistency correction is performed on the enhanced fused image, ensuring the fusion result conforms to the physical laws of electromagnetic scattering. This invention addresses the deficiency in existing fusion methods that rely solely on mathematical calculations, leading to inconsistencies between the intensity values ​​of the target region in the fused image and the actual backscattering cross-section. By minimizing the physical constraint fusion cost function, electromagnetic scattering characteristics are embedded into the fusion process, ensuring consistency between the radar intensity characteristics of ship targets in the fused image and the physical model predictions, thus improving the reliability of the fusion result under different sea states and target types. In summary, this invention solves the technical problem mentioned in the background art, where the lack of physical model constraints in the fusion of SAR and multispectral images leads to poor physical consistency in the fusion result. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention.

[0024] Figure 2 The image shows a comparison of synthetic aperture radar (SAR) images before and after speckle suppression, including two sub-images: (A) is the SAR image before speckle suppression, and (B) is the SAR image after speckle suppression.

[0025] Figure 3 This is a distribution diagram of subband coefficients fused by supercomplex wavelet transform.

[0026] Figure 4 This is a graph showing the evolution of the system's free energy during the thermodynamic equilibrium iterative process.

[0027] Figure 5 To ultimately enhance the overlay image of the fused image and the target detection results. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0029] like Figure 1 The diagram shown is a flowchart of a multi-source remote sensing image fusion method for ship detection provided by the first aspect of the present invention. This method includes the following steps:

[0030] S01. Acquire the synthetic aperture radar image to be fused and the multispectral image to be fused. Perform speckle suppression processing on the synthetic aperture radar image to be fused. Calculate the pixel block similarity weight using a nonlocal mean filter. Combine the wavelet domain Bayesian shrinkage estimator to perform adaptive threshold processing on the high-frequency coefficients to obtain the speckle-free synthetic aperture radar image.

[0031] S02. Extract the full polarization data of the synthetic aperture radar image to be fused, construct the covariance matrix of the full polarization data and perform parallel eigenvalue decomposition to obtain the polarization eigenvector.

[0032] S03. Construct an electromagnetic scattering constraint model for ship targets. Calculate the predicted backscattering cross section of the target based on the physical optics approximation principle and geometric diffraction theory. Introduce the residual between the predicted backscattering cross section of the target and the intensity value of the synthetic aperture radar image to be fused as a constraint term into the fusion cost function to obtain the physical constraint fusion cost function.

[0033] S04. Input the speckled synthetic aperture radar image and the multispectral image to be fused into the adaptive feature weighting model. The adaptive feature weighting model extracts multi-scale spatial features and outputs the fusion weight matrix corresponding to each pixel position.

[0034] S05. The real and imaginary parts of the synthetic aperture radar image to be fused and the multispectral image to be fused are organized into a hypercomplex form. The hypercomplex form is decomposed into a low-frequency sub-band and a high-frequency sub-band by hypercomplex wavelet transform. The low-frequency sub-band is obtained by combining the local variance weighted fusion rule with the fusion weight matrix. The high-frequency sub-band is obtained by using the absolute value selection rule.

[0035] S06. Define the local entropy of the image as the information temperature, construct a finite difference iterative format for the information diffusion heat conduction equation, take the fused low-frequency sub-band and fused high-frequency sub-band as the initial temperature field, realize the information flow from the high-entropy region to the low-entropy region through iterative calculation, terminate the iteration when the system free energy converges to the minimum value, and obtain the equilibrium temperature field.

[0036] S07. Perform inverse transform reconstruction on the hypercomplex wavelet coefficients corresponding to the equilibrium temperature field, combine with the physical constraint fusion cost function to perform physical consistency correction, and output the enhanced fusion image for ship target detection.

[0037] The implementation steps of the nonlocal mean filter are as follows: a search window is set around the pixel to be processed; an image patch of the same size is extracted for each pixel within the search window; the Euclidean distance between the image patch of the pixel to be processed and other image patches is calculated; the Euclidean distance is converted into a similarity weight; and the similarity weight is used to perform a weighted average of all pixels within the search window to obtain the filtered output value.

[0038] The implementation steps of the wavelet domain Bayesian shrinkage estimator are as follows: multi-scale wavelet decomposition is performed on the image corresponding to the filtered output value to obtain high-frequency coefficients at different scales; a generalized Gaussian distribution prior model is established for the high-frequency coefficients at each scale; the Bayesian shrinkage threshold is estimated based on the noise variance and signal variance; the magnitude of shrinking the high-frequency coefficients to zero is determined by the Bayesian shrinkage threshold; and wavelet inverse transform is performed on the shrunken high-frequency coefficients and the retained low-frequency coefficients to reconstruct the speckled synthetic aperture radar image.

[0039] The fully polarized data includes data from four channels: HH, HV, VH, and VV.

[0040] The steps for constructing the covariance matrix are as follows: construct a 3×3 or 4×4 complex Hermitian covariance matrix from the HH, HV, VH, and VV channels of the fully polarized data.

[0041] The implementation steps of parallel eigenvalue decomposition are as follows: the synthetic aperture radar image to be fused is divided into multiple image blocks and assigned to different threads of the graphics processing unit. Each thread independently completes the eigenvalue decomposition of the covariance matrix of all pixels in an image block, and extracts the eigenvalues ​​and eigenvectors as polarization eigenvectors. The polarization eigenvectors include scattering entropy, scattering angle, and anisotropy parameters.

[0042] The application of the physical optics approximation principle is as follows: the surface of the ship target is discretized into multiple surface elements, the induced current density under incident wave illumination is calculated for each surface element, and the far-field scattering intensity is calculated by integrating the induced current density of the surface elements on the target surface, thus obtaining the physical optics approximation backscattering cross section value.

[0043] The application of geometric diffraction theory is as follows: identify the edges, tips, and corners of ship targets as diffraction sources, calculate the diffraction field generated by the incident wave at the diffraction source, and superimpose the diffraction field with the direct reflection field corresponding to the physical optics approximate backscattering cross section value to obtain the predicted value of the target backscattering cross section.

[0044] The steps for constructing the physical constraint fusion cost function are as follows: calculate the difference between the intensity value of the synthetic aperture radar image to be fused and the predicted value of the target backscattering cross section as the physical constraint residual term; calculate the spectral distortion of the fused image and the multispectral image to be fused as the spectral fidelity term; calculate the spatial gradient of the fused image as the detail enhancement term; normalize the physical constraint residual term, the spectral fidelity term, and the detail enhancement term, and then sum them by weight to form the physical constraint fusion cost function; optimize the fusion parameters by minimizing the physical constraint fusion cost function.

[0045] The structure of the adaptive feature weighting model is as follows: the input layer receives speckle-removing synthetic aperture radar images and multispectral images to be fused; the feature extraction layer uses multi-scale convolutional kernels to extract spatial features of different receptive fields in parallel; the attention layer calculates feature importance weights based on a query key-value pair mechanism; the attention layer maps the spatial features output by the feature extraction layer into query vectors, key vectors, and value vectors; the attention score is calculated by the dot product operation of the query vector and the key vector; the attention score is normalized to obtain the attention weights; the value vectors are weighted and summed using the attention weights to obtain the enhanced feature representation; the fusion weight generation layer maps the enhanced feature representation to the fusion weight matrix corresponding to each pixel; and the output layer outputs the fusion weight matrix for subsequent fusion operations.

[0046] The steps for establishing the training dataset of the adaptive feature weighting model are as follows: collect multiple sets of registered training synthetic aperture radar images and training multispectral images as input samples; manually label the ship target area and background area on the input samples; use a traditional fusion algorithm to fuse the input samples and calculate the fusion quality evaluation index; use the fusion results with fusion quality evaluation index higher than the threshold as reference fused images; extract the optimal fusion weight distribution corresponding to the reference fused images as training labels; pair the input samples with the training labels to form a training dataset; and perform data augmentation operations on the training dataset, including rotation, flipping, and scaling, to generate expanded training samples.

[0047] The training steps of the adaptive feature weighted model are as follows: initialize the model parameters to random values, divide the training dataset into a training subset and a validation subset, read input samples in batches from the training subset and input them into the adaptive feature weighted model to obtain the prediction fusion weight matrix, calculate the mean squared error loss function between the prediction fusion weight matrix and the training labels, use the backpropagation algorithm to calculate the gradient of the mean squared error loss function with respect to the model parameters, use the adaptive moment estimation optimizer to update the model parameters according to the gradient, repeat the above training process until the mean squared error loss function on the validation subset converges or reaches the maximum number of iterations, and save the model parameters corresponding to the minimum mean squared error loss function on the validation subset as the final training result.

[0048] The implementation steps of the feature importance evaluation framework based on the attention mechanism in the adaptive feature weighting model are as follows: the spatial features output by the feature extraction layer are mapped to a query feature matrix, a key feature matrix, and a value feature matrix through three independent linear transformation layers, respectively. The matrix product of the query feature matrix and the transpose of the key feature matrix is ​​calculated to obtain the attention score matrix. The attention score matrix is ​​normalized row by row to obtain the normalized attention weight matrix. Each element of the normalized attention weight matrix represents the importance of the corresponding spatial feature. The normalized attention weight matrix is ​​multiplied with the value feature matrix to achieve a weighted combination of spatial features. The weighted combination process automatically suppresses redundant features and highlights the key features that contribute significantly to the prediction of the fused weight matrix.

[0049] The technical effects of the attention-based feature importance assessment framework are as follows: Traditional fusion methods assign equal importance to all extracted spatial features, leading to redundant and noisy features interfering with the estimation of the fusion weight matrix and reducing fusion quality. The attention mechanism dynamically evaluates the contribution of each spatial feature to the prediction of the current pixel's fusion weight matrix by querying key-value pairs, concentrating attention resources on highly discriminative feature dimensions and suppressing the weights of low signal-to-noise ratio features and redundant features. The attention mechanism adaptively adjusts the distribution of spatial feature importance in different scenarios, strengthening the weights of edge gradient features and texture features in ship target areas and strengthening the weights of spectral consistency features in sea clutter background areas, achieving scene-adaptive feature selection and avoiding the failure of fixed feature weight strategies in diverse scenarios. The normalized attention weight matrix provides interpretability of spatial feature importance, facilitating the analysis of the decision basis of the adaptive feature weighting model and the optimization of feature extraction strategies. By jointly optimizing the attention mechanism and the fusion weight matrix prediction through end-to-end training, the spatial feature importance assessment directly serves the fusion task objective, improving the ability of enhanced fused images to preserve ship target features and suppress background noise, and enhancing the target background contrast and edge sharpness of subsequent detection algorithms.

[0050] The organization steps of the hypercomplex form are as follows: the real part of the synthetic aperture radar image to be fused is taken as the first component of the hypercomplex number, the imaginary part of the synthetic aperture radar image to be fused is taken as the second component of the hypercomplex number, and the different bands of the multispectral image to be fused are taken as the third and fourth components of the hypercomplex number, respectively, to form a quaternion-based hypercomplex form.

[0051] The implementation steps of the hypercomplex wavelet transform are as follows: define the hypercomplex wavelet basis function, perform multi-scale decomposition on the hypercomplex form, and decompose the hypercomplex form into a low-frequency sub-band and multiple high-frequency sub-bands at each decomposition scale. The low-frequency sub-band retains the main energy and low-frequency information of the image, while the high-frequency sub-band contains image details and edge information.

[0052] The implementation steps of the local variance weighted fusion rule are as follows: calculate the variance of the pixel values ​​in the neighborhood window for each pixel position in the low-frequency sub-band as the local variance, normalize the local variance to obtain the local variance fusion weight, multiply the local variance fusion weight with the weight value at the corresponding position of the fusion weight matrix to obtain the comprehensive fusion weight, and take the low-frequency sub-band coefficients corresponding to the synthetic aperture radar image to be fused and the multispectral image to be fused according to the comprehensive fusion weight to obtain the fused low-frequency sub-band.

[0053] The implementation steps of the absolute value selection rule are as follows: for each pixel position of the high-frequency sub-band, compare the absolute values ​​of the high-frequency coefficients of the synthetic aperture radar image to be fused and the multispectral image to be fused, and select the high-frequency coefficient with the largest absolute value as the output value of the corresponding position of the fused high-frequency sub-band.

[0054] The information temperature is defined as follows: calculate the probability distribution of pixel values ​​within a local window of the image, and calculate the Shannon entropy based on the probability distribution as the information temperature of the local window. The larger the information temperature value, the richer the information content of the local area.

[0055] The steps for constructing the information diffusion heat conduction equation are as follows: the image fusion process is analogized to the heat conduction process of heat transfer from a high-temperature region to a low-temperature region. The high information temperature region corresponds to the heat source, and the low information temperature region corresponds to the cold source. A two-dimensional heat conduction partial differential equation is constructed to describe the evolution of information temperature with time and space. In the two-dimensional heat conduction partial differential equation, the heat diffusion coefficient is determined by the local gradient of the image and the structure tensor. The heat diffusion coefficient is reduced in the edge region to maintain edge sharpness, and the heat diffusion coefficient is increased in the flat region to promote information smoothing.

[0056] The reason for using the thermal conduction process as an analogy in the information diffusion heat conduction equation is that a thermodynamic system spontaneously tends towards equilibrium through heat exchange, eventually reaching a stable state with uniform temperature distribution and maximum entropy. This process conforms to the principle of minimum energy and the principle of maximum entropy. The fusion of synthetic aperture radar images and multispectral images to be fused needs to achieve spatial consistency while maintaining the information of each source image, which is consistent with the goal of thermodynamic equilibrium. The amount of image information is likened to heat, and the process of information diffusion from high-information-content areas to low-information-content areas is similar to heat conduction. Image smoothing and noise suppression are achieved through the diffusion term of the two-dimensional thermal conduction partial differential equation, and edge preservation and detail enhancement are achieved by controlling the spatial distribution of the thermal diffusion coefficient. When the thermodynamic system reaches equilibrium, the free energy is minimum, which corresponds to the optimal state of enhanced fused images with balanced information distribution and coordinated spatial structure.

[0057] The steps for implementing the finite difference iterative scheme of the information diffusion heat conduction equation are as follows: Discretize the fused low-frequency subband and fused high-frequency subband into a two-dimensional grid; discretize the two-dimensional heat conduction partial differential equation in space and time; calculate the spatial second derivative using the central difference scheme; calculate the temporal first derivative using the forward difference scheme; obtain the recursive formula for the information temperature at the next time step; set the initial temperature field as the initial fused image composed of the fused low-frequency subband and fused high-frequency subband; iteratively calculate the spatiotemporal evolution of the information temperature according to the recursive formula; calculate the system free energy after each iteration; the system free energy consists of an internal energy term and an entropy term; the internal energy term represents the magnitude of the image spatial gradient; and the entropy term represents the uniformity of the image information distribution; terminate the iteration when the change in system free energy is less than the convergence threshold after multiple consecutive iterations, and obtain the equilibrium temperature field.

[0058] The technical effect of the information diffusion heat conduction equation is as follows: Traditional fusion methods use fixed fusion rules for the synthetic aperture radar (SAR) images and the multispectral images to be fused, without considering information interaction and dynamic balance during the fusion process. This leads to excessive enhancement in high-contrast regions, resulting in halo artifacts, and insufficient information in low-contrast regions, causing loss of detail. The information diffusion heat conduction equation achieves adaptive information exchange between images through an information diffusion mechanism. Edge features of the high-information-temperature SAR images diffuse to the flat regions of the low-information-temperature multispectral images, enhancing the spatial resolution of the multispectral images. Spectral information from the multispectral images diffuses to the single-band SAR images, enriching and enhancing the spectral dimensions of the fused images. The diffusion process is controlled by the edge-preserving thermal diffusion coefficient. It automatically reduces the diffusion rate at the target edge to prevent edge blurring and increases the diffusion rate in a uniform background to promote information sharing. The iterative process gradually brings the enhanced fused image toward the optimal equilibrium state, avoiding the parameter selection difficulties and local optima problems caused by one-step fusion. The system free energy minimization criterion comprehensively considers spatial smoothness and information preservation, achieving the optimal trade-off between noise suppression and detail preservation under physical constraints. The information diffusion thermal conduction equation does not require manual setting of fusion parameters and naturally reaches the fusion equilibrium point through the physical process, improving the robustness and generalization ability of the algorithm. It enhances the fused image to maintain clear edges and rich textures in the ship target area and achieves noise smoothing and radiation consistency in the sea clutter background area, thereby enhancing the separation between the target and the background.

[0059] The calculation steps for the system free energy are as follows: calculate the spatial gradient magnitude of the enhanced fused image and sum it over the entire image range as the internal energy term; calculate the local entropy of each pixel in the enhanced fused image and sum it over the entire image range as the entropy term; subtract the entropy term from the internal energy term and multiply it by the temperature parameter to obtain the system free energy value; the temperature parameter controls the relative importance of the internal energy term and the entropy term.

[0060] The steps of physical consistency correction are as follows: calculate the deviation between the enhanced fused image and the predicted value of the target backscattering cross section according to the physical constraint fusion cost function, adjust the intensity value of the pixel position where the deviation exceeds the threshold, so as to minimize the residual between the adjusted enhanced fused image and the predicted value of the target backscattering cross section.

[0061] It is important to note that scattering entropy characterizes the degree of randomness in target scattering, scattering angle characterizes the type of scattering mechanism, and anisotropy parameter characterizes the directional characteristics of scattering. These three parameters together describe the polarization scattering characteristics of the target.

[0062] It should be noted that the fusion quality evaluation indicators include information entropy, average gradient, spatial frequency, and spectral fidelity. These fusion quality evaluation indicators comprehensively evaluate the information content, sharpness, detail retention, and spectral distortion of the fused image.

[0063] It is important to note that the adaptive moment estimator optimizer combines the first-order moment estimation and the second-order moment estimation of the gradient to adaptively adjust the learning rate of each model parameter, thereby accelerating the convergence of the adaptive feature-weighted model and improving training stability.

[0064] It is important to note that the edge-preserving thermal diffusivity coefficient is set to a smaller value in the edge region to maintain edge sharpness, and a larger value in the flat region to promote information smoothing.

[0065] The specific implementation methods of the above steps are described in detail below.

[0066] The specific implementation of step S01 is as follows: First, the pixel data of the synthetic aperture radar image to be fused is read. Then, a search window is set around the pixel to be processed, with a reference size of 21×21 pixels. Image patches are extracted for each pixel within the search window, with a reference size of 7×7 pixels. The Euclidean distance between the image patch of the pixel to be processed and other image patches within the search window is calculated. A Gaussian kernel function is used to convert the Euclidean distance into similarity weights, with a reference value of 0.5 for the filtering parameter in the weight calculation. All pixels within the search window are weighted and averaged according to the similarity weights to obtain the filtered output value. The nonlocal mean filter utilizes the self-similarity principle of the image, searching for image patches with similar structures to the pixel to be processed and performing weighted averaging. This suppresses speckle noise while preserving edge and texture details. Next, the image corresponding to the filtered output value is subjected to three-level wavelet decomposition. After decomposition, the image is obtained... High-frequency and low-frequency coefficients at different scales are obtained. A generalized Gaussian distribution prior model is established for the high-frequency coefficients at each scale. The Bayesian shrinkage threshold is calculated based on the noise variance and signal variance of the image. The threshold calculation adopts the minimum mean square error criterion. The high-frequency coefficients are shrunk to zero according to the Bayesian shrinkage threshold. The shrinkage magnitude is related to the ratio of the absolute value of the high-frequency coefficient to the threshold. The shrunk high-frequency coefficients and the retained low-frequency coefficients are reconstructed by wavelet inverse transform to obtain the despeckled synthetic aperture radar image. The wavelet domain Bayesian shrinkage estimator, based on the multi-resolution characteristics of wavelet transform, separates the signal and noise at different scales. The optimal shrinkage threshold is determined by Bayesian statistical inference. While removing speckle noise, the target edge information is preserved to the greatest extent. The purpose of this step is to suppress multiplicative speckle noise in synthetic aperture radar image, improve the contrast between target and background, and provide high-quality input image for subsequent fusion processing.

[0067] The specific implementation of step S02 involves extracting the full polarization data of the synthetic aperture radar image to be fused. The full polarization data includes complex data from four channels: HH, HV, VH, and VV. The data from these four channels is organized into a 3×3 or 4×4 complex Hermitian covariance matrix according to the polarization scattering matrix. Each element of the covariance matrix represents the correlation between different polarization channels. The synthetic aperture radar image to be fused is divided into multiple image blocks of 256×256 pixels. These image blocks are assigned to different threads in the graphics processing unit for parallel processing. Each thread independently completes the eigenvalue decomposition of the covariance matrix of all pixels within its assigned image block. Eigenvalue decomposition employs the Jacobi iterative algorithm or the fast eigenvalue decomposition algorithm to extract eigenvalues ​​and eigenvectors. After extracting these eigenvalues ​​and eigenvectors, the scattering entropy, scattering angle, and anisotropy parameters are calculated to form a polarization eigenvector. The scattering entropy reflects the randomness of target scattering, the scattering angle characterizes the dominant scattering mechanism type, and the anisotropy parameters describe the direction dependence of scattering. This step utilizes multi-channel information from a fully polarimetric synthetic aperture radar to obtain the target's physical scattering characteristics through covariance matrix eigenvalue decomposition, providing discriminative features for distinguishing ship targets from sea clutter backgrounds. Parallel processing significantly reduces computation time. The purpose of this step is to extract the essential scattering characteristics of the target from fully polarimetric data, enhancing target identification capabilities.

[0068] The specific implementation of step S03 involves discretizing the ship's surface into multiple triangular or rectangular facets based on the ship's geometric model and attitude parameters. The facet size is approximately 0.5m × 0.5m. For each facet, the induced current density under radar incident wave illumination is calculated. The induced current density is related to the facet's normal vector and the polarization direction of the incident wave. The induced current densities of all facets are integrated over the target surface to calculate the far-field scattered field intensity. A numerical integration method is used to obtain the physical optical approximation backscattering cross-section value. The physical optical approximation is based on the high-frequency approximation assumption and is suitable for targets much larger than the radar wavelength. Next, the ship's edge, tip, and corner are identified as diffraction sources. The diffraction field generated at the diffraction source is calculated using the diffraction coefficients from geometric diffraction theory. The diffraction field is then coherently superimposed with the direct reflection field corresponding to the physical optical approximation backscattering cross-section value. The process involves obtaining the predicted backscattering cross section (RSS) value of the target, then calculating the difference between the intensity value of the synthetic aperture radar image to be fused and the predicted RSS value as a physical constraint residual term. The spectral distortion between the multispectral image to be fused and the fused image is calculated as a spectral fidelity term, and the sum of squared spatial gradients of the fused image is calculated as a detail enhancement term. The physical constraint residual term, spectral fidelity term, and detail enhancement term are normalized by dividing by their respective maximum values. The three normalized terms are then weighted and summed according to weight coefficients to form a physical constraint fusion cost function. The reference values ​​for the weight coefficients are 0.4, 0.3, and 0.3, respectively. This step is based on an electromagnetic scattering physics model to constrain the fusion process, ensuring that the fusion result conforms to the physical laws of radar imaging and improving the detection accuracy of small and low-contrast targets. The purpose of this step is to introduce prior physical knowledge to guide the fusion process and avoid physically unreasonable artifacts in the fusion result.

[0069] The specific implementation of step S04 involves inputting the speckle-reduced synthetic aperture radar image and the multispectral image to be fused into the input layer of the adaptive feature weighting model. The input layer performs size and pixel value normalization on the input image, and then passes the normalized image to the feature extraction layer. The feature extraction layer uses three convolutional kernels of different sizes to extract spatial features in parallel. The reference values ​​for the convolutional kernel sizes are 3×3, 5×5, and 7×7, corresponding to different receptive field ranges. Each convolutional kernel is followed by an activation function for nonlinear transformation. The activation function uses a modified linear unit function. The spatial features output by the feature extraction layer contain edge, texture, and structural information at different scales. These spatial features are then input into the attention layer for feature importance evaluation. The attention layer contains three linear transformation layers that map the spatial features into a query feature matrix, a key feature matrix, and a value feature matrix, respectively. The matrix product of the query feature matrix and the transpose of the key feature matrix is ​​calculated to obtain the attention score matrix. Each row of the attention score matrix is ​​normalized using a normalized exponential function to obtain a normalized attention weight matrix. The normalized attention weight matrix reflects the contribution of each spatial feature to the prediction of the fusion weight matrix. The normalized attention weight matrix is ​​multiplied by the value feature matrix to achieve a weighted combination of features. The enhanced feature representation after weighted combination is input into the fusion weight generation layer. The fusion weight generation layer uses a fully connected layer to map the enhanced feature representation to the fusion weight matrix corresponding to each pixel. Each element of the fusion weight matrix takes a value from 0 to 1, representing the relative weight of the synthetic aperture radar image and the multispectral image to be fused during fusion. This step automatically learns the optimal fusion weight allocation strategy for different scenarios based on the attention mechanism, avoiding the degradation of fusion quality caused by the fixed weights used in traditional methods. The purpose of this step is to adaptively determine the fusion weights according to the local scene characteristics, thereby improving the detail preservation and noise suppression capabilities of the fused image.

[0070] The specific implementation of step S05 involves taking the real part of the synthetic aperture radar image to be fused as the first component of the hypercomplex number, the imaginary part as the second component, and the first band of the multispectral image to be fused as the third component and the second band as the fourth component, forming a hypercomplex form in quaternion form. Then, a hypercomplex wavelet basis function is defined, using either a bioorthogonal wavelet or a symplectic wavelet. A three-level wavelet decomposition is performed on the hypercomplex form, with each level generating one low-frequency subband and three high-frequency subbands. The low-frequency subband contains the main energy and low-frequency information of the image, while the high-frequency subbands correspond to detail information in the horizontal, vertical, and diagonal directions, respectively. For each pixel in the low-frequency subband, the variance of the pixel values ​​within a 7×7 neighborhood window is calculated as the local variance. The local variance is divided by the maximum value of all pixel local variances for normalization to obtain the local variance fusion weight. The local variance fusion weight is multiplied by the corresponding weight value in the fusion weight matrix to obtain the comprehensive fusion weight. The process involves weighting the low-frequency sub-band coefficients of the synthetic aperture radar (SAR) image to be fused and the multispectral image to be fused according to a comprehensive fusion weight to obtain the fused low-frequency sub-band. The local variance weighted fusion rule is based on the activity metric principle, where regions with large local variances represent more detailed information and are given higher fusion weights. For each pixel in the high-frequency sub-band, the absolute values ​​of the corresponding high-frequency coefficients of the SAR image to be fused and the multispectral image to be fused are compared, and the high-frequency coefficient with the larger absolute value is selected as the output value of the corresponding position in the fused high-frequency sub-band. The absolute value selection rule retains the most significant edge and detail features. The supercomplex wavelet transform is used to uniformly represent the complex and real images, achieving collaborative fusion of multimodal information in the transform domain. While maintaining phase coherence, amplitude information is fused. The purpose of this step is to achieve multi-scale fusion of multi-source images in the wavelet transform domain, processing low-frequency energy information and high-frequency detail information separately, thereby improving the information preservation capability of the fused image.

[0071] The specific implementation of step S06 involves calculating the probability distribution of pixel values ​​within a 9×9 neighborhood window for each pixel in the initial fused image composed of fused low-frequency and fused high-frequency sub-bands. Based on this probability distribution, Shannon entropy is calculated as the information temperature. The calculation of Shannon entropy divides the pixel values ​​into 256 gray levels, and the frequency of occurrence of each gray level is statistically analyzed. A higher information temperature indicates richer local information. Then, a two-dimensional heat conduction partial differential equation is constructed to describe the evolution of information temperature over time and space. The heat conduction partial differential equation includes a time derivative term and a spatial second derivative term. The coefficient of the spatial second derivative term is the thermal diffusivity coefficient, which is calculated from the local gradient of the image. The thermal diffusivity coefficient for edge regions with large gradients has a reference range of 0.1 to 0.3, while the thermal diffusivity coefficient for flat regions with small gradients has a reference range of 0.6 to 0.9. The two-dimensional heat conduction partial differential equation is then spatially and temporally discretized. Spatial discretization uses a central difference scheme to calculate the second derivative, and temporal discretization uses a forward difference scheme to calculate the first derivative. The recursive calculation formula for the information temperature at the next moment is obtained after diffusion. The initial temperature field is set as the initial fused image. Iterative calculation is performed according to the recursive calculation formula. The information temperature value of all pixels is updated in each iteration. After the iteration, the system free energy is calculated. The system free energy is equal to the internal energy term minus the entropy term multiplied by the temperature parameter. The internal energy term is the global summation of the spatial gradient magnitude of the fused image, and the entropy term is the global summation of the local entropy of the fused image. The reference value of the temperature parameter is 0.5. The iteration is terminated when the change of the system free energy in 5 consecutive iterations is less than the convergence threshold. The reference value of the convergence threshold is 0.01 times the initial value of the system free energy. The information temperature field at the time of termination of iteration is the equilibrium temperature field. The above steps are based on the principle of thermodynamic equilibrium. Adaptive information exchange and redistribution between images are realized through information diffusion. Sharpness is maintained in the edge region and smoothness is promoted in the flat region, achieving the optimal balance between spatial consistency and information preservation. The role of the above steps is to optimize the fusion result through a physically driven iterative process and avoid the subjectivity and limitations of manual parameter adjustment.

[0072] The specific implementation of step S07 involves performing an inverse supercomplex wavelet transform on the fused low-frequency subband and fused high-frequency subband corresponding to the equilibrium temperature field. The inverse transform process is the inverse operation of the forward transform. An enhanced fused image in the spatial domain is obtained through layer-by-layer upsampling and filtering reconstruction. Then, the deviation between the intensity value of the enhanced fused image and the predicted value of the target backscattering cross section is calculated according to the physical constraint fusion cost function. The intensity value of pixels with an absolute deviation exceeding a threshold is adjusted. The threshold reference value is 15% of the predicted value of the target backscattering cross section. The adjustment method is to move the pixel intensity value in the direction of the predicted value of the target backscattering cross section, with a moving step size of 0.5 times the deviation value. The adjusted enhanced fused image satisfies the electromagnetic scattering physical consistency constraint. Finally, the enhanced fused image is output for subsequent ship target detection algorithms. This step completes the reconstruction from the transform domain to the spatial domain and uses a physical model for final correction to ensure that the fusion result conforms to the physical laws of radar imaging while maintaining image quality. The purpose of this step is to generate a high-quality fused image, providing enhanced input data for ship target detection.

[0073] It should be noted that the key technical ideas of this invention include adaptive feature weighting based on an attention mechanism, physical consistency fusion under electromagnetic scattering constraints, and a thermodynamic equilibrium iterative fusion algorithm. Adaptive feature weighting based on an attention mechanism dynamically evaluates the contribution of different spatial features to the predicted fusion weights by querying key-value pairs. It concentrates computational resources on highly discriminative feature dimensions, suppresses interference from redundant and noisy features, and achieves scene-adaptive feature selection. Compared to the traditional strategy of assigning fixed weights to all features, this mechanism can automatically adjust the feature importance distribution according to local scene characteristics, strengthening edge and texture features in the ship target area and strengthening spectral consistency features in the background area. This avoids the failure of fixed-weight strategies in diverse scenes and improves the ability of the fused image to preserve target features and suppress background noise. The physical consistency fusion method constrained by electromagnetic scattering utilizes physical optics approximation and geometric diffraction theory to construct a forward electromagnetic scattering model of ship targets. The model-predicted backscattering cross section is introduced as a constraint into the fusion cost function. By minimizing the residual between the measured intensity and the model prediction, the fusion result is ensured to conform to the physical laws of radar imaging. Compared to traditional purely data-driven fusion methods, this physical constraint avoids artifacts that violate electromagnetic scattering laws during the fusion process, improving the detection accuracy of small and low-contrast targets and enhancing the physical interpretability and reliability of the fusion result. The thermodynamic equilibrium iterative fusion algorithm analogizes image fusion to the process of a thermodynamic system reaching equilibrium. Through an information diffusion mechanism, it achieves adaptive information flow from high-information-temperature regions to low-information-temperature regions. Edge-preserving diffusion coefficients automatically reduce the diffusion rate at target edges to prevent edge blurring, while increasing the diffusion rate in uniform backgrounds to promote information sharing. Compared to traditional one-step fusion methods, this iterative process gradually brings the fused image towards the optimal equilibrium state with the minimum system free energy. It comprehensively considers spatial smoothness and information preservation, achieving an optimal trade-off between noise suppression and detail preservation under physical constraints, without requiring manual setting of complex fusion parameters. The synergistic effect of the three technical approaches lies in the fact that adaptive feature weighting provides the initial fusion image and weight allocation basis for thermodynamic iteration, thermodynamic equilibrium iteration further optimizes the information distribution based on adaptive weights, and electromagnetic scattering constraints perform physical consistency correction on the final fusion result. The three form a complete processing chain from feature extraction and information fusion to physical correction. Compared with the single fusion strategy of existing technologies, the synergistic mechanism combines data-driven feature learning, physical-driven iterative optimization, and model-driven constraint correction, which improves the fusion quality while ensuring physical reliability and significantly enhances the contrast and separability of ship targets and sea clutter background.

[0074] It should be noted that this invention also solves the following technical problem: the existing fusion methods assign fixed weights to different spatial features, leading to unstable fusion quality in diverse scenarios. Traditional fusion methods use preset fusion rules to apply the same feature weight allocation strategy to all scenarios, without considering the different requirements for feature importance between the ship target area and the sea clutter background area. In the ship edge area, gradient features need to be strengthened to maintain the clarity of the target outline, and in the uniform sea surface area, spectral consistency features need to be strengthened to suppress noise interference. The fixed weight strategy cannot adapt to the changing feature requirements of different areas. This invention constructs an adaptive feature weighting model and introduces an attention layer based on a query key-value pair mechanism. It dynamically calculates the contribution of each spatial feature to the current pixel fusion weight prediction, concentrates attention resources on highly discriminative feature dimensions and suppresses redundant features. It automatically increases the weight of edge gradient features and texture features in the ship target area and automatically increases the weight of spectral consistency features in the sea clutter background area, achieving scene-adaptive feature selection and solving the problem of decreased fusion quality in complex marine environments caused by the fixed feature weight strategy.

[0075] A second aspect of the present invention provides a computer-readable storage medium storing program instructions that, when executed in a computer, perform the aforementioned multi-source remote sensing image fusion method for ship detection.

[0076] A third aspect of the present invention provides a multi-source remote sensing image fusion system for ship detection, comprising the aforementioned computer-readable storage medium. The system is any one of a computer, a server, or a microcontroller. The computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor that executes the program instructions stored in the computer-readable storage medium.

[0077] Specifically, the principle of this invention is as follows: The reason this invention can solve the aforementioned technical problems lies in the introduction of a physical constraint mechanism based on electromagnetic scattering theory. The radar echo characteristics of ship targets are determined by their geometry and materials. The physical optics approximation principle calculates the direct reflection field through the integral of the induced current density of surface elements, while geometric diffraction theory compensates for the diffraction contributions from edges and corners. The combination of these two approaches can accurately predict the target's backscattering cross-section. This predicted value is introduced as prior physical knowledge into the fusion cost function, subjecting the fusion process to the physical laws of electromagnetic scattering and avoiding non-physical results generated by purely mathematical fusion. The physical constraint fusion cost function comprehensively considers physical consistency, spectral fidelity, and spatial detail enhancement. Multi-objective optimization is achieved by minimizing this function, ensuring that the fused image satisfies the physical characteristics of electromagnetic scattering while maintaining spectral and spatial information. The physical consistency correction step adjusts the intensity of the fusion result based on the residuals, ensuring that the output image conforms to the predictions of the physical model and improving the accuracy of the fused image in representing the characteristics of ship targets.

[0078] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0079] The specific implementation of step S01 is as follows: First, acquire the synthetic aperture radar image to be fused and the multispectral image to be fused, and then perform speckle suppression processing on the synthetic aperture radar image to be fused. In the implementation of the nonlocal mean filter, at the pixel location to be processed... Set search window around For each pixel position within the search window Extraction size is Given image patches, calculate the Euclidean distance between the pixel image patch to be processed and other image patches. Euclidean distance The specific calculation is expressed as follows:

[0080] ;

[0081] In the formula, The Euclidean distance between image patches; The side length of the image block is in pixels, with an empirical value of 5 to 9. Location of the pixel to be processed In-neighbor offset The grayscale value of the location; pixel position In-neighbor offset The grayscale value of the location; The offset index is used within the neighborhood. The Euclidean distance is converted into similarity weights. Specifically, it is expressed as follows:

[0082] ;

[0083] In the formula, pixel position and pixel position Similarity weights between them; The filter strength parameter has an empirical value of 10 to 30. The normalization coefficient is expressed as: The filtered output value is obtained by weighting all pixels within the search window using similarity weights. Specifically, it is expressed as follows:

[0084] ;

[0085] In the formula, For position The nonlocal mean filter output value; For position The grayscale value. In the implementation of the wavelet domain Bayesian contraction estimator, multi-scale wavelet decomposition is performed on the image corresponding to the filtered output value to obtain high-frequency coefficients at different scales. Noise variance The robust median estimation method is used, as shown below:

[0086] ;

[0087] In the formula, For noise variance; It is a median function; For the first High-frequency coefficients of the scale; 0.6745 is the scale parameter of the standard normal distribution. Signal variance. The specific calculation is expressed as follows:

[0088] ;

[0089] In the formula, The signal variance; This represents the total number of high-frequency coefficients at this scale. Based on the noise variance... and signal variance Estimating the Bayesian shrinkage threshold Specifically, it is expressed as follows:

[0090] ;

[0091] In the formula, The Bayesian shrinkage threshold; This represents the signal standard deviation. The value of the high-frequency coefficient after shrinkage. Specifically, it is expressed as follows:

[0092] ;

[0093] In the formula, The high-frequency coefficients after contraction; The sign function is used. The shrunken high-frequency coefficients and the retained low-frequency coefficients are reconstructed using inverse wavelet transform to reconstruct the speckled synthetic aperture radar image.

[0094] The specific implementation of step S02 is to extract the full polarization data of the synthetic aperture radar image to be fused. The full polarization data includes data from four channels: HH, HV, VH, and VV. Covariance matrix. Construct a 3×3 complex Hermitian matrix, specifically represented as follows:

[0095] ;

[0096] In the formula, It is the covariance matrix; , , These are the complex scattering coefficients for the HH, HV, and VV channels, respectively; superscript This represents the conjugation operation. In parallelized eigenvalue decomposition, the synthetic aperture radar image to be fused is divided into multiple image blocks and assigned to different threads of the graphics processing unit. Each thread independently performs eigenvalue decomposition of the covariance matrix of all pixels within an image block, extracting eigenvalues. And the eigenvectors are used as polarization eigenvectors. Scattering entropy Scattering angle Anisotropy parameters The specific calculation is expressed as follows:

[0097] ;

[0098] ;

[0099] ;

[0100] In the formula, It is the scattering entropy; For feature value index; The normalized eigenvalue probability is expressed as: ; The scattering angle is expressed in radians. For the first The scattering angle components corresponding to each eigenvector; For anisotropy parameters; These are the eigenvalues ​​of the covariance matrix; For summation index.

[0101] The specific implementation of step S03 is to construct an electromagnetic scattering constraint model for the ship target. Applying the principle of physical optics approximation, the surface of the ship target is discretized into... The element, for the first face element, Calculate the induced current density for each face element. Physical optics approximate backscattering cross section value Specifically, it is expressed as follows:

[0102] ;

[0103] In the formula, This is the approximate backscattering cross section value in physical optics, in units of... ; Wave number, unit: ; For the first The induced current density of each surface element, in units of ; For the first The area of ​​each face unit, in units of ; For the first The unit normal vector of each face element; For the first The position vector of each face element, in units of ; The unit is the imaginary unit. In applications of geometric diffraction theory, the edges, tips, and corners of ship targets are identified as diffraction sources. The scattering contribution of the incident wave at the diffraction source is calculated, and the target backscattering cross-section is predicted. Specifically, it is expressed as follows:

[0104] ;

[0105] In the formula, The predicted value of the target backscattering cross section, in units of ; For the first The diffraction field contribution of each diffracting source, in units of ; The total number of diffracting sources. Physical constraint fusion cost function. In the construction process, the intensity values ​​of the synthetic aperture radar images to be fused are calculated. Compared with the predicted value of the target backscattering cross section The difference is used as the physical constraint residual term. Physical constraint residual term Specifically, it is expressed as follows:

[0106] ;

[0107] In the formula, For physical constraint residuals; For the synthetic aperture radar image to be fused at the location The intensity value; The reference intensity value is the average intensity value of the synthetic aperture radar images to be fused. For position Predicted backscattering cross section of the target at location, in units of ; For reference backscattering cross-section values, the unit is... The empirical value is the average of the predicted values ​​of the target backscattering cross section. Spectral fidelity item. Specifically, it is expressed as follows:

[0108] ;

[0109] In the formula, For spectral fidelity; For image fusion Each band in position Pixel values; For the first multispectral image to be fused Each band in position Pixel values; Total number of bands; Band index. Detail enhancements. Specifically, it is expressed as follows:

[0110] ;

[0111] In the formula, For detail enhancements; To merge images Partial derivatives in the direction, in units of ; To merge images Partial derivatives in the direction, in units of ; Spatial resolution of pixels, in units of This is used for dimensional normalization. The physical constraint fusion cost function is specifically expressed as follows:

[0112] ;

[0113] In the formula, The cost function for physical constraint fusion; These are weighting coefficients, with empirical values ​​of 0.3, 0.5, and 0.2 respectively; These are the maximum values ​​of each item, used for normalization.

[0114] The specific implementation of step S04 is the same as described above, and will not be repeated in detail here.

[0115] The specific implementation of step S05 is to fuse the real part of the synthetic aperture radar image to be synthesized. and the virtual part Different bands of the multispectral image to be fused The organization is in hypercomplex form Specifically, it is expressed as follows:

[0116] ;

[0117] In the formula, It is in hypercomplex form; The real part of the synthetic aperture radar image to be fused; The imaginary part of the synthetic aperture radar image to be fused; These are two bands of the multispectral image to be fused; The imaginary part of the hypercomplex number is represented by the unit. The hypercomplex form is decomposed into low-frequency subbands using a hypercomplex wavelet transform. and high-frequency subband For the low-frequency subband, a local variance weighted fusion rule is applied, where the local variance... The specific calculation is expressed as follows:

[0118] ;

[0119] In the formula, For position Local variance at; The side length of the neighborhood window, in pixels; For A neighborhood window centered on the center; The average pixel value within the neighborhood window is represented as... ; This refers to the coordinate index within the neighborhood window. (Comprehensive fusion weighting) Specifically, it is expressed as follows:

[0120] ;

[0121] In the formula, For comprehensive integration and weighting; The weight values ​​at corresponding positions in the fusion weight matrix output by the adaptive feature weighting model; Use the summation coordinate index within the neighborhood window. Fuse low-frequency subbands. Specifically, it is expressed as follows:

[0122] ;

[0123] In the formula, To integrate low-frequency subbands; The low-frequency subband coefficients corresponding to the synthetic aperture radar images to be fused; The low-frequency subband coefficients correspond to the multispectral images to be fused. An absolute value selection rule is applied to the high-frequency subbands to fuse them. Specifically, it is expressed as follows:

[0124] ;

[0125] In the formula, To integrate high-frequency subbands; The high-frequency subband coefficients corresponding to the synthetic aperture radar images to be fused; The coefficients of the high-frequency subbands corresponding to the multispectral images to be fused.

[0126] The specific implementation of step S06 is to define the local entropy value of the image as the information temperature. Calculate local window of image Probability distribution of inner pixel values Probability distribution The specific calculation is expressed as follows:

[0127] ;

[0128] In the formula, grayscale In position The probability within a local window centered on the target; The grayscale value within a local window is equal to The number of pixels; This represents the total number of pixels within the local window. This is a grayscale index, with values ​​ranging from 0 to 255. The temperature information is specifically represented as follows:

[0129] ;

[0130] In the formula, For position The information temperature at that location. A two-dimensional partial differential equation for heat conduction is constructed based on the information diffusion heat conduction equation, specifically expressed as follows:

[0131] ;

[0132] In the formula, For information temperature field; For time variables, the unit is ; is the thermal diffusivity, which takes a smaller value in the edge region and a larger value in the flat region; This refers to the gradient operator. The specific functions are as follows:

[0133] ;

[0134] ;

[0135] In the formula, The gradient of the information temperature field; For divergence operators; For the temperature field in Partial derivatives in the direction, in units of ; For the temperature field in Partial derivatives in the direction, in units of Thermal diffusivity Specifically, it is expressed as follows:

[0136] ;

[0137] In the formula, The thermal diffusivity; The baseline diffusion coefficient has an empirical value of 0.1 to 0.5. For position The normalized image gradient magnitude at point is expressed as ; For images in Partial derivatives in the direction, in units of ; For images in Partial derivatives in the direction, in units of ; This represents the maximum value of the image gradient, in units of... , used for normalization; The normalized gradient threshold parameter has an empirical value of 0.1–0.3. In the finite difference iterative scheme, the central difference scheme is used to calculate the spatial second derivative, and the forward difference scheme is used to calculate the temporal first derivative. The information temperature is determined at the next time step. The recursive formula is expressed as follows:

[0138] ;

[0139] In the formula, For the first Position at the next iteration Information temperature; For the first Position at the next iteration Information temperature; For time step, the unit is The empirical value is 0.01 to 0.1. Spatial step size, unit: System free energy The specific calculation is expressed as follows:

[0140] ;

[0141] ;

[0142] ;

[0143] In the formula, The system's free energy; Internal energy term; For entropy; For temperature parameters, the empirical value is 0.1 to 0.5; This is the fused image from the current iteration; Spatial resolution of pixels, in units of Used for dimensional normalization. When the system's free energy changes during multiple consecutive iterations... Less than the convergence threshold The iteration terminates at a certain time, and the convergence condition is specifically expressed as follows:

[0144] ;

[0145] In the formula, This represents the change in the system's free energy. For the first The system free energy of the next iteration; For the first The system free energy of the next iteration; The convergence threshold is, empirically, [value]. ~ .

[0146] The specific implementation of step S07 involves reconstructing the hypercomplex wavelet coefficients corresponding to the equilibrium temperature field using inverse transform, and then performing physical consistency correction by combining the physical constraint fusion cost function. In the physical consistency correction, the enhanced fused image is calculated. Compared with the predicted value of the target backscattering cross section Normalized bias Specifically, it is expressed as follows:

[0147] ;

[0148] In the formula, For position Normalization bias at the location; To enhance the location of fused images The intensity value; To enhance the maximum intensity value of the fused image; For position Predicted backscattering cross section of the target at location, in units of ; The maximum value of the predicted backscattering cross section of the target, in units of For deviations exceeding the threshold The intensity value is adjusted at the pixel position, resulting in an enhanced fused image. Specifically, it is expressed as follows:

[0149] ;

[0150] In the formula, The corrected enhanced fused image; This is a correction factor, with an empirical value of 0.5 to 0.9; The normalization bias threshold is set to an empirical value of 0.05–0.15. The output enhanced fused image is used for ship target detection.

[0151] To better understand and implement this invention, the following is a specific application scenario example 2: To solve the problem of missed detection of small ship targets due to poor fusion quality of multi-source remote sensing images in marine monitoring, technicians used the technical solution of this invention to process synthetic aperture radar (SAR) images and multispectral image data of a certain sea area. The acquired SAR image to be fused has a resolution of 3m and an image size of 4096×4096 pixels, using C-band full polarization mode with an equivalent number of views of 4.2. The acquired multispectral image to be fused has a resolution of 10m and includes two channels: red light band and near-infrared band. The image size is aligned with the SAR image to be fused after resampling and registration. The sea state of the image coverage area is sea state 3, and there are 12 known ship targets with a target size range of 15m to 85m.

[0152] Technicians first performed speckle suppression processing on the synthetic aperture radar (SAR) image to be fused. The search window size for the nonlocal mean filter was set to 21×21 pixels, the image patch size to 7×7 pixels, and the filter parameter to 0.5. Each pixel in the image was filtered, followed by a three-level wavelet decomposition using biorthogonal wavelet basis functions. A Bayesian shrinkage estimator was applied to the high-frequency coefficients of each level, estimating a noise variance of 0.082 and a signal variance of 1.35. The calculated Bayesian shrinkage thresholds were 0.28 for the first level, 0.19 for the second level, and 0.13 for the third level. After shrinking the high-frequency coefficients according to the thresholds, the despeckled SAR image was reconstructed. Figure 2 As shown, the speckle noise in the image after speckle removal is significantly reduced, and the sharpness of the target edge is significantly improved.

[0153] Next, the technicians extracted the full polarization data from the synthetic aperture radar image to be fused, constructed a 4×4 complex Hermitian covariance matrix, and divided the image into 256×256 pixel image blocks. These blocks were then distributed to 128 parallel threads in the graphics processing unit for eigenvalue decomposition, with each thread processing 512 image blocks. The extracted polarization feature vectors included scattering entropy, scattering angle, and anisotropy parameters. The average scattering entropy of the ship target area was 0.65, the average scattering angle was 42 degrees, and the average anisotropy parameter was 0.72. The average scattering entropy of the sea clutter background area was 0.38, the average scattering angle was 18 degrees, and the average anisotropy parameter was 0.41. The polarization feature vectors showed that the ship target and the sea clutter background had significant differences in scattering characteristics.

[0154] Technicians constructed an electromagnetic scattering constraint model based on the typical geometric model of the ship target. The ship surface was discretized into 0.5m×0.5m surface elements, and the radar incident angle was set to 35 degrees. The induced current density of each surface element was calculated based on the physical optics approximation and integrated to obtain the backscattering cross section. The contribution of the diffraction field at the edge and corner was calculated by combining the geometric diffraction theory to obtain the predicted value of the target backscattering cross section. The residual between the intensity value of the synthetic aperture radar image to be fused and the predicted value was calculated as the physical constraint residual term. The spectral distortion and spatial gradient were calculated as the spectral fidelity term and the detail enhancement term, respectively. The normalized weight coefficients of the three terms were set to 0.4, 0.3 and 0.3, respectively, to form the physical constraint fusion cost function.

[0155] Technicians input the speckle-removed synthetic aperture radar (SAR) image and the multispectral image to be fused into an adaptive feature weighting model. The model employs three feature extraction layers with convolutional kernel sizes of 3×3, 5×5, and 7×7, respectively. The extracted multi-scale spatial features contain 128-dimensional feature vectors. The attention layer maps the spatial features into a query feature matrix, a key feature matrix, and a value feature matrix. After calculating the attention score and normalizing it, a normalized attention weight matrix is ​​obtained. The normalized attention weight matrix exhibits a high weight distribution in the ship target edge region and the sea surface wave texture region, and a low weight distribution in the uniform background region. The fusion weight generation layer outputs the fusion weight matrix corresponding to each pixel. The average weight of the SAR image to be fused in the ship target region is 0.73, the average weight of the multispectral image to be fused is 0.27, and the weight distribution in the background region is relatively balanced.

[0156] Technicians organized the real and imaginary parts of the synthetic aperture radar image to be fused and the two bands of the multispectral image to be fused into a quaternion hypercomplex form. A three-level hypercomplex wavelet transform was performed on the hypercomplex form to decompose it into low-frequency and high-frequency subbands. For the low-frequency subband, a local variance weighted fusion rule was applied, with a neighborhood window size of 7×7 pixels. The local variance was calculated, normalized, and multiplied by the corresponding weights in the fusion weight matrix to obtain the comprehensive fusion weight. For the high-frequency subband, an absolute value selection rule was used, selecting high-frequency coefficients with larger absolute values ​​to form the fused high-frequency subband. Figure 3 As shown, the fusion of the low-frequency subband preserves the spatial resolution of the synthetic aperture radar image to be fused and the spectral information of the multispectral image to be fused, while the fusion of the high-frequency subband preserves the edge and texture details of the ship target.

[0157] Technicians defined the local entropy of the image as information temperature, set the neighborhood window size to 9×9 pixels, and calculated the information temperature of each pixel in the initial fused image. The average information temperature of the ship target area was 5.82 bits, and the average information temperature of the sea clutter background area was 4.15 bits. A two-dimensional heat conduction partial differential equation was constructed to describe the evolution of information temperature. The thermal diffusivity coefficient of the edge region was set to 0.2, and the thermal diffusivity coefficient of the flat region was set to 0.8. Spatial-temporal discretization was performed using central difference and forward difference, with a time step set to 0.05. Iterative calculations were performed, updating the information temperature of all pixels and calculating the system free energy in each iteration. The temperature parameter was set to 0.5. After 45 iterations, the system free energy converged. The change in system free energy in five consecutive iterations was less than 0.01 times the initial value. Figure 4 As shown, during the iteration process, the high information temperature region diffuses to the low information temperature region, the information temperature at the edge of the ship target remains stable, and the information temperature in the background region gradually becomes uniform, thus obtaining a balanced temperature field.

[0158] Technicians reconstructed the spatial domain enhanced fused image by performing inverse transform on the hypercomplex wavelet coefficients corresponding to the equilibrium temperature field. Based on the physical constraint fusion cost function, they calculated the deviation between the intensity value of the enhanced fused image and the predicted value of the target backscattering cross section. A deviation threshold of 15% of the predicted value was set. Pixels exceeding the threshold had their intensity values ​​adjusted in a step size of 0.5 times the deviation value. After physical consistency correction, the final enhanced fused image was output, as shown below. Figure 5 As shown, all 12 known ship targets in the enhanced fused image are clearly distinguishable with sharp edges, significantly improved contrast with the sea clutter background, and effective suppression of background noise. Technicians input the enhanced fused image into a ship target detection algorithm, using a constant false alarm rate (CFAR) detector for target detection, with the CFAR set to a specific value. The positions of 12 ship targets were detected and matched the positions of known targets. The target detection accuracy reached 100%, with no missed detections or false alarms. The processing time statistics are shown in Table 1.

[0159] Table 1. Time Consumption Statistics for Each Processing Step

[0160]

[0161] The technological advancements of this invention compared to traditional fusion methods lie in the synergistic optimization of multiple aspects. Traditional methods employ fixed fusion rules to weightedly fuse multi-source images, neglecting the differences in local scene characteristics. This leads to the use of the same fusion strategy for both the ship target area and the background area, resulting in loss of target details or enhanced background noise. This invention, through an adaptive feature weighting model based on an attention mechanism, dynamically evaluates the importance of different spatial features and automatically adjusts the fusion weight allocation according to the local scene. It strengthens edge and texture features in the target area and enhances spectral consistency in the background area, achieving a scene-adaptive fusion strategy that improves target preservation and noise suppression capabilities. Traditional methods do not incorporate prior physical knowledge during the fusion process, making the fusion results prone to artifacts that violate the physical laws of radar imaging, reducing the reliability of subsequent detection algorithms. This invention, through an electromagnetic scattering constraint model, incorporates the physical scattering characteristics of the ship target as a constraint condition into the fusion cost function, ensuring that the fusion result conforms to electromagnetic scattering theory, avoiding physically unreasonable artifacts, and improving the detection accuracy of small targets and low-contrast targets. Traditional methods employ a one-step fusion strategy, requiring manual adjustment of fusion parameters, which makes it difficult to achieve optimal fusion results in different scenarios. This invention uses a thermodynamic equilibrium iterative fusion algorithm to analogize image fusion to the process of a thermodynamic system reaching equilibrium. It utilizes an information diffusion mechanism to achieve adaptive information exchange between images, maintaining edge sharpness by reducing the diffusion rate at target edges and increasing the diffusion rate in the background to promote information smoothing. Through iterative optimization, the fused image gradually approaches the optimal equilibrium state with the minimum system free energy. It comprehensively considers spatial smoothness and information preservation, eliminating the need for manual parameter adjustment and improving the robustness and generalization ability of the algorithm.

[0162] It should be noted that the variables involved in this invention are explained in detail in Tables 2, 3, and 4.

[0163] Table 2. Variable Explanation Table (Part 1)

[0164]

[0165] Table 3. Variable Explanation Table (Part Two)

[0166]

[0167] Table 4. Variable Explanation Table (Part 3)

[0168]

[0169] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-source remote sensing image fusion method for ship detection, characterized in that, The method comprises the following steps: The method comprises the following steps:

2. The method of claim 1, wherein, The coherent speckle suppression processing step is specifically using a non-local mean filter to calculate the similarity weight of a pixel block, and combining a wavelet domain Bayesian shrinkage estimator to perform adaptive threshold processing on high-frequency coefficients.

3. The method of claim 2, wherein, The non-local mean filter is specifically setting a search window around the pixel to be processed, calculating the Euclidean distance between the image block of the pixel to be processed and other image blocks, converting the Euclidean distance into a similarity weight, and using the similarity weight to perform weighted averaging on all pixels in the search window to obtain a filtering output value.

4. The method of claim 3, wherein, The wavelet domain Bayesian shrinkage estimator is specifically performing multi-scale wavelet decomposition on the image corresponding to the filtering output value, establishing a generalized Gaussian distribution prior model for the high-frequency coefficients of each scale, estimating a Bayesian shrinkage threshold according to a noise variance and a signal variance, and shrinking the high-frequency coefficients to zero.

5. The method of claim 4, wherein, The full polarization data includes data of four channels of HH, HV, VH and VV, and the covariance matrix is constructed as a three-row-three-column or four-row-four-column complex Hermitian covariance matrix.

6. The method of claim 5, wherein, The parallelized eigenvalue decomposition is specifically dividing the synthetic aperture radar image to be fused into multiple image blocks and distributing the image blocks to different threads of a graphics processing unit, and each thread independently completes eigenvalue decomposition of the covariance matrix of all pixels in an image block to extract eigenvalues and eigenvectors as polarization feature vectors.

7. The method of claim 6, wherein, The application of the physical optics approximation principle, specifically, the ship target surface is discretized into multiple facets, the induced current density under the incident wave irradiation is calculated for each facet, and the far-field scattering field intensity is calculated by integrating the facet induced current density on the target surface.

8. The method of claim 7, wherein, The application of the geometric diffraction theory, specifically, the edges, tips and corners of the ship target are identified as diffraction sources, the diffraction field generated by the incident wave at the diffraction sources is calculated, and the diffraction field is superimposed with the direct reflection field corresponding to the physical optics approximation backscattering cross-section value.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores program instructions, and the program instructions are used to execute the multi-source remote sensing image fusion method for ship detection when running in the computer.

10. A multi-source remote sensing image fusion system for ship detection, characterized in that, The system comprises the computer readable storage medium, and the system is a computer. The computer readable storage medium is arranged in the system, and the system is provided with a microprocessor for executing the program instructions stored in the computer readable storage medium.

Images