Image fusion method and device, equipment and storage medium

Abstract

The invention discloses an image fusion method and device, equipment and a storage medium. When the method provided by the embodiment of the invention is executed, firstly, a synthetic aperture radar (SAR) image and an optical image of a target view finding area are acquired, and feature point extraction is performed on the SAR image and the optical image to obtain an SAR feature point set and an optical feature point set; the method comprises the following steps: constructing shallow and deep feature descriptors of all SAR feature points in an SAR image; in the optical image, shallow and deep feature descriptors of all optical feature points are constructed. Based on the feature descriptors, SAR feature points and optical feature points are matched. And carrying out fusion processing on the SAR image and the optical image through the matched feature points, and finally generating a fused image. According to the method based on feature point matching, the accuracy of feature point matching is improved, the reservation capability of detail information is enhanced, the registration process is simplified, and the overall fusion effect is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image fusion method, device, equipment and storage medium. Background Art

[0003] At present, remote sensing image processing technology on the financial side mainly relies on a single optical image. This optical image is obtained through a passive optical sensor, which leads to the loss of a lot of detailed information, causing misjudgment of the relevant identification system, which may cause unreasonable allocation of loan funds by business personnel. For this reason, active remote sensing technology is introduced to improve recognition accuracy. Remote sensing technology is a highly comprehensive discipline involving multiple fields such as physics, earth science, space science and computer science. It has become an important means of observing the earth by obtaining data from remote sensing equipment installed on satellites, aircraft and other aviation or spacecraft. It uses the difference in electromagnetic waves emitted and reflected by ground objects to analyze the structural information of objects, thereby obtaining target information that cannot be obtained by traditional technology. Due to its wide range of applications, remote sensing technology has played an important role in national economy and military aspects such as remote sensing estimation of crop yields, earthquake rescue, resource exploration, meteorological observation, surveying and mapping, and military reconnaissance.

[0004] Radar imaging plays an important role in remote sensing technology, especially Synthetic Aperture Radar (SAR), which is an active sensor with strong penetration ability and can obtain the shape characteristics of difficult-to-detect ground objects. Related technologies fuse SAR images with visible light images to obtain richer image information. In this process, registration and fusion are the key steps in SAR and visible light processing, and their accuracy directly affects the subsequent application effect of remote sensing image processing.

[0005] However, due to the different imaging mechanisms of SAR images and visible light images, there are significant differences in the geometric structure and texture features of these images. Therefore, when performing the registration and fusion of SAR images and visible light images, mismatching and high complexity problems may occur. In particular, when marking high-frequency feature areas, weak target information in the SAR image may be lost, thus affecting the final fusion result. Summary of the invention

[0006] Based on the above problems, the present application provides an image fusion method, apparatus, device and storage medium, which can improve the accuracy and robustness of SAR images and visible light images in the registration and fusion process to reduce mismatching and information loss, thereby obtaining higher quality fusion results.

[0007] The embodiments of the present application disclose the following technical solutions:

[0008] An image fusion method, the method comprising:

[0009] Acquire a synthetic aperture radar SAR image and an optical image of a target viewing area, and extract feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set;

[0010] On the SAR image, shallow feature descriptors and deep feature descriptors of all SAR feature points in the SAR feature point set are constructed; on the optical image, shallow feature descriptors and deep feature descriptors of all optical feature points in the optical feature point set are constructed;

[0011] Based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image, feature point matching is performed on feature points in the SAR feature point set and feature points in the optical feature point set;

[0012] The SAR image and the optical image are fused according to the matched feature points to obtain a fused image.

[0013] In a possible implementation manner, constructing shallow feature descriptors of all SAR feature points in the SAR feature point set on the SAR image includes:

[0014] Converting the SAR image into a nonlinear diffusion scale image;

[0015] Converting the nonlinear diffusion scale image into a shear wave scale image and a phase consistency scale image;

[0016] Scale-fusing the shear wave scale image and the phase consistency scale image to obtain a multi-scale fused image;

[0017] On the multi-scale fused image, shallow feature descriptors of all SAR feature points in the SAR feature point set are constructed.

[0018] In a possible implementation, constructing shallow feature descriptors of all SAR feature points in the SAR feature point set on the multi-scale fusion image includes:

[0019] For each SAR feature point in the SAR feature point set, a first image sub-region is determined on each image layer of the multi-scale fusion image with the SAR feature point as the center, to obtain a plurality of first image sub-regions of the SAR feature point on the multi-scale fusion image; one SAR feature point corresponds to a plurality of first image sub-regions, and the number of first image sub-regions corresponding to the SAR feature point is equal to the number of image layers of the multi-scale fusion image;

[0020] The multiple first image sub-regions corresponding to each SAR feature point are merged to obtain a shallow feature descriptor of each SAR feature point.

[0021] In a possible implementation manner, constructing deep feature descriptors of all SAR feature points in the SAR feature point set on the SAR image includes:

[0022] Converting the SAR image into a multi-layer first sparse feature image;

[0023] On the multi-layer first sparse feature image, deep feature descriptors of all SAR feature points in the SAR feature point set are constructed.

[0024] In a possible implementation, constructing deep feature descriptors of all SAR feature points in the SAR feature point set on the multi-layer first sparse feature image includes:

[0025] For each SAR feature point in the SAR feature point set, a second image subregion is determined on each layer of the first sparse feature image with the SAR feature point as the center, to obtain a plurality of second image subregions of the SAR feature point on the multi-layer first sparse feature image; one SAR feature point corresponds to a plurality of second image subregions, and the number of the second image subregions corresponding to the SAR feature point is equal to the number of image layers of the second image subregion;

[0026] For each of the second image sub-regions, converting the first sparse feature image where the second image sub-region is located to obtain a second sparse feature image;

[0027] The plurality of second sparse feature images corresponding to each SAR feature point are merged to obtain a deep feature descriptor of each SAR feature point.

[0028] In a possible implementation manner, extracting feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set includes:

[0029] By using a coherent enhanced diffusion (CED-Harris) detector that introduces coherent enhanced diffusion and spatial constraints, feature points are extracted from the SAR image and the optical image to obtain the SAR feature point set and the optical feature point set.

[0030] In a possible implementation, the matching of feature points in the SAR feature point set with feature points in the optical feature point set based on the shallow feature descriptor and the deep feature descriptor of the SAR image and the shallow feature descriptor and the deep feature descriptor of the optical image includes:

[0031] Based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image, and using a sparse representation (SR) matching criterion that introduces locality constrained sparse coding, feature points in the SAR feature point set and feature points in the optical feature point set are matched.

[0032] An image fusion device, comprising:

[0033] An acquisition unit, used for acquiring a synthetic aperture radar SAR image and an optical image of a target viewing area;

[0034] A feature point extraction unit, used for extracting feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set;

[0035] A descriptor construction unit is used to construct shallow feature descriptors and deep feature descriptors of all SAR feature points in the SAR feature point set on the SAR image; and to construct shallow feature descriptors and deep feature descriptors of all optical feature points in the optical feature point set on the optical image;

[0036] A feature point matching unit, configured to perform feature point matching on feature points in the SAR feature point set and feature points in the optical feature point set based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image;

[0037] The image fusion unit is used to fuse the SAR image and the optical image according to the matched feature points to obtain a fused image.

[0038] An image fusion device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the image fusion method described above is implemented.

[0039] A computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the image fusion method as described above.

[0040] Compared with the prior art, this application has the following beneficial effects:

[0041] The present application provides an image fusion method, apparatus, device and storage medium. Specifically, when executing the image fusion method provided in the embodiment of the present application, first, a synthetic aperture radar (SAR) image and an optical image are obtained from the target viewing area. Then, feature points are extracted for these two types of images to obtain a SAR feature point set and an optical feature point set, respectively. These feature points represent key points in the image and are used for subsequent feature matching and image fusion. Next, on the SAR image, a shallow feature descriptor and a deep feature descriptor are constructed for each extracted SAR feature point. Similarly, on the optical image, a shallow feature descriptor and a deep feature descriptor are also constructed for each extracted optical feature point. Shallow feature descriptors usually capture the basic local structure and texture information of the image, while deep feature descriptors can better reflect higher-level semantic information and complex structures in the image. After obtaining all necessary feature descriptors, feature point matching is performed based on the shallow and deep feature descriptors of the SAR image and the optical image. By comparing and matching the feature points in the two sets, corresponding feature point pairs can be found, thereby achieving preliminary alignment between images. This multi-level feature descriptor matching method improves the accuracy and robustness of matching and reduces the possibility of mismatching. Finally, the SAR image and the optical image are fused according to the matched feature point pairs to generate the final fused image. In this way, not only can the advantages of the two images be retained, but also the differences in their geometric structure and texture features can be overcome, thereby obtaining richer and more accurate image information.

[0042] This application extracts feature points on SAR images and visible light images respectively, and constructs shallow and deep feature descriptors. By matching these descriptors, the accuracy of feature point matching can be improved. This method can better capture the key feature points in the image, thereby reducing the situation of mismatching. At the same time, through feature point matching, the high-frequency features and weak target information of the target area can be better retained. In the traditional fusion process, some detail information may be lost due to differences in geometric structure and texture features. The matching method based on feature points can align and fuse these detail information more accurately. In addition, using shallow and deep feature descriptors for feature point matching can simplify the complex registration process to a certain extent. Deep feature descriptors can capture deeper semantic information, which helps to improve the stability and robustness of matching, thereby reducing the complexity of the fusion process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A method flow chart of an image fusion method provided in an embodiment of the present application;

[0045] Figure 2 A method flow chart of a method for constructing a shallow feature descriptor provided in an embodiment of the present application;

[0046] Figure 3 A method flow chart of a method for constructing a deep feature descriptor provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of an image fusion device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0049] In recent years, some banks have begun to use satellite remote sensing technology to solve the problem of "difficulty in granting loans to farmers". Through this technology, banks can obtain all the production and operation records of farmers in history and the real-time situation after issuing loans, including identifying crop growth, possible natural disasters and predicting crop yields, so as to more accurately evaluate farmers' personal credit and production and planting behavior, and realize the refined management of loan funds. However, the current remote sensing image processing technology on the financial side mainly relies on a single optical image. This passive optical sensor causes a lot of detailed information to be lost, resulting in misjudgment and affecting the rationality of loan fund allocation. For this reason, active remote sensing technology is introduced to improve recognition accuracy. Among them, SAR, as an active sensor, has strong penetration ability and can obtain the shape characteristics of difficult-to-detect ground targets. Fusion processing of SAR images with visible light images can obtain richer image information. However, due to the significant differences in geometric structure and texture features between SAR images and visible light images, mismatching and high complexity may occur in the registration and fusion process, especially when marking high-frequency feature areas, which may lead to the loss of weak target information and affect the final fusion result.

[0050] In order to solve this problem, an image fusion method, device, equipment and storage medium are provided in the embodiment of the present application. First, a SAR image and an optical image of the target viewing area need to be obtained. Next, feature points are extracted from the two images to obtain a SAR feature point set and an optical feature point set respectively. On the SAR image, a shallow feature descriptor and a deep feature descriptor are constructed for each feature point in the SAR feature point set. Similarly, on the optical image, the same two types of feature descriptors are also constructed for each feature point in the optical feature point set. Shallow feature descriptors usually capture local details and texture information, while deep feature descriptors can capture deeper semantic information, which helps to improve the stability and robustness of matching. Based on these feature descriptors, feature points in the SAR feature point set are matched with feature points in the optical feature point set. By comparing the shallow and deep feature descriptors between the two, the best correspondence can be found, thereby improving the accuracy of matching. Finally, the SAR image and the optical image are fused according to the matched feature points to obtain the final fused image. In this way, not only the high-frequency features and weak target information of the target area can be retained, but also the information loss problem caused by the difference in geometric structure and texture features in the traditional fusion process can be overcome. The present application can significantly improve the accuracy of feature point matching by extracting feature points on SAR images and visible light images respectively, and constructing shallow and deep feature descriptors. By matching these descriptors, the accuracy of feature point matching can be significantly improved. This method can better capture the key feature points in the image and reduce the situation of mismatching. At the same time, through feature point matching, the high-frequency features and weak target information of the target area can be more effectively retained, avoiding the loss of detail information caused by the difference in geometric structure and texture features in the traditional fusion process. In addition, the use of shallow and deep feature descriptors for feature point matching can simplify the complex registration process to a certain extent. The deep feature descriptor can capture deeper semantic information, which helps to improve the stability and robustness of matching, thereby reducing the complexity of the fusion process.

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] See also Figure 1 , which is a method flow chart of an image fusion method provided in an embodiment of the present application, such as Figure 1 As shown, the image fusion method may include steps S101-S104:

[0053] S101: Acquire a synthetic aperture radar SAR image and an optical image of a target viewing area, and extract feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set.

[0054] When studying the SAR image and optical image of the target viewing area, it is first necessary to understand that the "target viewing area" refers to the specific area or scene of interest. This area may contain the target objects of interest, buildings, terrain, etc. By acquiring SAR images and optical images, image data collected by two different sensors can be obtained, providing diverse information.

[0055] In order to obtain these images, a common method is to use remote sensing photography such as satellites or drones. These platforms are equipped with SAR and optical sensors that can capture image data of the target viewing area. Next, feature point extraction is performed on these two images, which means finding key points with significant properties from the image. These feature points can represent textures, corners or other important information in the image, which helps with subsequent image analysis and processing.

[0056] Through feature point extraction, we obtain a set of SAR feature points and a set of optical feature points, each of which has a unique descriptor. These descriptors can help quantify the properties of feature points, thereby achieving matching and correspondence between different images. Through such detailed analysis and processing, we can better understand the target viewing area, identify the target objects or ground features therein, and carry out further application research.

[0057] For example, if you want to study a certain area in a city, you can obtain SAR images and optical images of the area through aerial photography or satellite images. Then extract feature points from these two images to find key regional features. For example, in an urban area, we may focus on features such as road intersections and building outlines. These feature points will help us better understand the structure and characteristics of the entire urban area.

[0058] In a possible implementation manner, extracting feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set includes:

[0059] A coherent enhancing diffusion (CED-Harris) detector that introduces coherent enhancing diffusion and spatial constraints is used to extract feature points from the SAR image and the optical image to obtain the SAR feature point set and the optical feature point set.

[0060] Based on CED-Harris (also known as CEDH), coherent enhanced diffusion and spatial constraints are introduced to obtain similar and highly repeatable feature points while suppressing image speckle noise.

[0061] In one possible implementation, the CED-Harris matrix with coherent enhanced diffusion and spatial constraints is calculated as follows:

[0062] In order to reduce the impact of noise on traditional Harris-Laplace feature point detection, the traditional solution is to construct multi-scale Harris in the nonlinear scale space constructed by Additive Operator Splitting (AOS). However, AOS, a method based on scalar diffusion, will cause feature information to be lost to a certain extent. Therefore, we introduced the coherent enhancement diffusion function to solve the above problem. Compared with the nonlinear diffusion filter that uses a scalar-valued diffusion rate to control the diffusion process, the method used by this function is to control the diffusion process through the diffusion tensor. It extends the previous nonlinear diffusion filter AOS scheme for scalar-valued diffusion to the case of full anisotropy with a diffusion tensor. This enables true anisotropic behavior, not only adapting the diffusion process to the position, but also allowing different smoothness in different directions, so that noise can be suppressed while maintaining feature information.

[0063] First, the traditional multi-scale Harris matrix is ​​expressed as:

[0064]

[0065] Among them, Φ is the standard deviation, which is Gaussian kernel

[0066] C CEDH (x, y, δ) = C MSH (x, y, δ)·D·C MSH (x, y, δ) T ; G x , G y is the gradient in the x and y directions. The calculation method of the improved multi-scale CED-Harris matrix after introducing enhanced diffusion is:

[0067]

[0068] where α allows for a small diffusion rate (α = 0.05). k is given by (λ1-λ2) 2 A threshold value (k = 125). The eigenvalues ​​of the λ1, λ2 tensors. In addition, m = 2, Cm = 2.33666. The corresponding multi-scale Harris function is expressed as:

[0069] RCEDH (x, y, δ) = det(C CEDH (x, y, δ)-d*tr 2 (C MSH (x, y, δ) T ).

[0070] The purpose of this step is to construct the CED-Harris scale space (Coherent Enhanced Diffusion-Harris Detector (CED-HD)). Where det(a) is the determinant of the a matrix; tr2(a) is the trace of the matrix; d is a threshold variable.

[0071] After that, the local extrema are calculated, and the edge points and low-contrast points are suppressed using the threshold to obtain the initial feature points. Then, the spatial consistency strategy is used to obtain reliable feature points.

[0072] S102: constructing shallow feature descriptors and deep feature descriptors of all SAR feature points in the SAR feature point set on the SAR image; and constructing shallow feature descriptors and deep feature descriptors of all optical feature points in the optical feature point set on the optical image.

[0073] Constructing shallow and deep feature descriptors in SAR images and optical images refers to extracting feature description information of key feature points from these two images. When processing SAR images, shallow feature descriptors and deep feature descriptors of these feature points can be established by analyzing the set of SAR feature points in the image. These descriptors help to characterize the texture, edge, shape and other features of different regions in the image based on the local structure and semantic information around the feature points. Similarly, the same operation is performed on optical images to extract shallow and deep feature descriptors of each feature point from the set of optical feature points.

[0074] The purpose of this step is to convert the feature points in SAR images and optical images into more representative and comparable numerical descriptions, so that the subsequent feature point matching and image fusion processes are more accurate and efficient. Shallow feature descriptors usually cover more direct pixel information, while deep feature descriptors represent more abstract and semantic features. The generation of such descriptors can help identify and distinguish the similarities or differences between different feature points, providing more data support and basis for subsequent image analysis and processing. Therefore, after constructing these feature descriptors, it will be possible to better understand and utilize the information in SAR and optical images, thereby achieving more accurate and comprehensive image processing and analysis goals.

[0075] See also Figure 2 , Figure 2A method flow chart of a method for constructing a shallow feature descriptor provided in an embodiment of the present application. Accordingly, on the SAR image, shallow feature descriptors of all SAR feature points in the SAR feature point set are constructed, which can be specifically implemented through steps A1-A4:

[0076] A1: Convert the SAR image into a nonlinear diffusion scale image.

[0077] Converting SAR images to nonlinear diffusion scale images refers to transforming the original SAR images through a specific processing method to obtain more layers and rich image details. This process can improve image quality, enhance image features, and help to more accurately analyze and understand image content.

[0078] In a possible implementation, converting the SAR image into a nonlinear diffusion scale image can be achieved by the following steps:

[0079] Preprocessing: First, the SAR image is preprocessed, such as denoising, contrast enhancement, etc., to prepare the raw image data to be processed;

[0080] Nonlinear diffusion filtering: By applying a nonlinear diffusion filter, the contrast and brightness of different areas in the image can be changed, thereby achieving nonlinear enhancement of the image. This filter usually adjusts the pixel value according to the gradient value of the image to highlight details such as edges and textures;

[0081] Scale space transformation: After nonlinear diffusion filtering, scale space transformation techniques can be applied to convert the resulting image into representations of different scales. This helps capture features of different scales in the image;

[0082] Generate nonlinear diffusion scale image: Finally, the images processed by nonlinear diffusion filtering and scale space transformation are merged to generate nonlinear diffusion scale image. This image retains the structural information of the original image while enhancing subtle textures and features, helping to better understand the image content.

[0083] By converting SAR images into nonlinear diffusion scale images, a richer and more expressive image representation can be obtained, which helps to improve image quality and information content, and provides a better basis for subsequent image analysis and processing tasks. This processing method can make images easier to understand and use, thereby supporting further research and application in various application fields.

[0084] A2: Convert the nonlinear diffusion scale image into a shear wave scale image and a phase consistency scale image.

[0085] The purpose of converting nonlinear diffusion scale images into shear wave scale images and phase consistency scale images is to better characterize the structural characteristics and phase information of the image, thereby providing more useful information for subsequent image analysis and processing. This conversion process involves extracting different types of features from the original image and synthesizing them to obtain a more comprehensive image representation.

[0086] Specifically, converting the nonlinear diffusion scale image into the shear wave scale image and the phase congruency scale image may include the following steps:

[0087] Shearlet scaling images: Shearlet scaling images are generated by applying some form of shearlet transform to nonlinear diffusion scaling images. This transform is usually used to highlight local features such as edges, textures, and contours of an image, making these features more obvious and easier to identify. Shearlet scaling images can help capture local structural information in images and play an important role in feature detection and matching.

[0088] Phase-consistent scaling: Phase-consistent scaling involves extracting the phase information of an image, which is an important aspect of image processing. By maintaining the phase consistency of the image and constructing an image representation with coherent phase information in the scale space, it can help to more accurately describe the geometric structure and spatial layout of the image. This information is crucial for tasks such as target detection, tracking, and matching.

[0089] By converting nonlinear diffusion scale images into shear wave scale images and phase consistency scale images, the characteristics and structures of the image can be understood from different angles and scales. This diversified feature extraction method can provide more comprehensive and richer information for subsequent image analysis, recognition and processing, and help enhance the image representation ability and application potential. This processing process enables us to understand the image content more deeply and obtain more valuable information from it to support various practical application fields.

[0090] A3: Scale-fusing the shear wave scale image and the phase consistency scale image to obtain a multi-scale fused image.

[0091] Based on the fusion weight factor of shear wave scale image and phase consistency scale image, these two different feature images can be scale-fused to obtain a multi-scale fused image. By introducing the weight factor to control the proportion of each feature in the final fusion result, the characteristics of shear wave scale image and phase consistency scale image are comprehensively utilized to further improve the fusion effect, making the multi-scale fused image more accurate and comprehensive.

[0092] In a possible implementation, step A3 scale-fusing the shear wave scale image and the phase congruency scale image to obtain a multi-scale fused image can be implemented by the following steps:

[0093] Determine weight factors: According to specific problems and application scenarios, set appropriate weight factors between shear wave scale images and phase consistency scale images. These weight factors can reflect the contribution of the two features to the image information and their correlation;

[0094] Weight factor fusion: linearly combine the shear wave scale image and the phase consistency scale image according to the weight factor or other fusion methods to calculate the weighted fusion result. This process can effectively adjust the trade-off relationship between different features, making the fused image more in line with the actual situation;

[0095] Multi-scale fusion image generation: Based on the fusion results, the final multi-scale fusion image is obtained. This image integrates the information of the shear wave scale image and the phase consistency scale image, combines their respective advantages, and shows richer and more comprehensive image features.

[0096] By introducing weight factors for scale fusion, the contribution ratio of various feature information in the fusion result can be flexibly controlled, making the multi-scale fusion image more targeted and accurate. This process combines feature fusion technology and weight adjustment strategy to optimize image feature representation and provide more refined and effective data support for various image processing and analysis tasks.

[0097] A4: constructing shallow feature descriptors of all SAR feature points in the SAR feature point set on the multi-scale fusion image.

[0098] Constructing shallow feature descriptors of all SAR feature points in the set of constructed SAR feature points on the multi-scale fused image means generating corresponding shallow feature descriptors for each SAR feature point in the determined set of SAR feature points for the multi-scale fused image. This process aims to capture and quantify local features of different locations and regions in the SAR image for subsequent tasks such as image matching, target recognition or image retrieval.

[0099] In a possible implementation, step A4 constructs shallow feature descriptors of all SAR feature points in the SAR feature point set on the multi-scale fusion image, which can be implemented by the following steps:

[0100] First, for each SAR feature point in the SAR feature point set, a first image sub-region is determined on each image layer of the multi-scale fusion image with the SAR feature point as the center, and multiple first image sub-regions of the SAR feature point on the multi-scale fusion image are obtained, wherein one SAR feature point corresponds to multiple first image sub-regions, and the number of first image sub-regions corresponding to the SAR feature point is equal to the number of image layers of the multi-scale fusion image. Then, the multiple first image sub-regions corresponding to each SAR feature point are merged to obtain a shallow feature descriptor of each SAR feature point.

[0101] Specifically, this process includes the following steps:

[0102] Determine the first image sub-region: According to the position of each SAR feature point on the multi-scale fusion image, the point is used as the center to determine the corresponding first image sub-region. These sub-regions can be local image blocks around the SAR feature point to capture the image content around the point.

[0103] Generate multiple first image sub-regions: For each SAR feature point, according to the different levels of the multi-scale fusion image, obtain the first image sub-region of the point on each layer. Therefore, a SAR feature point will correspond to multiple first image sub-regions, each sub-region corresponds to image information at different scales;

[0104] Merge sub-regions: Integrate or merge multiple first image sub-regions corresponding to each SAR feature point to synthesize image information at all levels and generate a shallow feature descriptor for the SAR feature point. This descriptor represents the local features and contextual relationship of the entire SAR feature point on the multi-scale fusion image.

[0105] Through the above process, more comprehensive and diverse local feature descriptors can be extracted for each SAR feature point, and combined with information at different scales, these descriptors are more representative and distinguishable. This method helps to improve the description and understanding of specific feature points in SAR images, and provides a more reliable data basis for subsequent target tracking, detection or recognition tasks.

[0106] It should be noted that the process of "constructing shallow feature descriptors of all optical feature points in the optical feature point set on the optical image" is similar to the process of "constructing shallow feature descriptors of all SAR feature points in the SAR feature point set on the SAR image", and please refer to steps A1-A4 for details.

[0107] See also Figure 3 , Figure 3A method flow chart of a method for constructing a deep feature descriptor provided in an embodiment of the present application. Accordingly, on the SAR image, the deep feature descriptors of all SAR feature points in the SAR feature point set are constructed, which can be specifically implemented through steps B1-B2:

[0108] B1: Convert the SAR image into a multi-layer first sparse feature image.

[0109] In the convolutional sparse coding model, by converting SAR images into multi-layer first sparse feature images, the aim is to use the feature representation of multi-layer convolutional dictionaries to improve the image representation and analysis capabilities. This conversion process involves multi-layer convolution processing to obtain richer and more accurate sparse feature representations, thereby promoting a deeper understanding of the image content.

[0110] In a possible implementation, step B1 converts the SAR image into a multi-layer first sparse feature image, which can be implemented by the following steps:

[0111] Set up multi-layer convolutional dictionary: Set up multi-layer convolutional dictionary in the convolutional sparse coding model, which contains sparse encoders at different levels. The convolutional dictionary of each layer consists of multiple convolution kernels, and the number and size of these convolution kernels will be adjusted according to experimental requirements and performance optimization. By building a multi-layer convolutional dictionary, you can increase the perception range of image features and structures and improve the ability to abstractly express image content;

[0112] Generate the first sparse feature image: For the SAR image, first use the convolution sparse dictionary of the first layer to perform convolution operation to obtain the first layer of sparse feature image. This step helps to extract the most basic and local feature representation from the original image, providing a basis for subsequent feature extraction and image reconstruction;

[0113] Multi-layer stacking: Subsequently, by stacking the information of different convolutional dictionaries layer by layer, a multi-layer first sparse feature image is generated. This can combine the feature information from each layer to form a richer and more comprehensive image representation, while retaining useful information at different scales and levels.

[0114] By converting SAR images into multi-layer first sparse feature images, and using the multi-layer convolution dictionary in the convolutional sparse coding model to extract and integrate a variety of feature information, the image representation is made more hierarchical and rich. This processing method can better understand the structure and content of the image, and provide a more comprehensive and effective foundation for subsequent image recognition, analysis and processing tasks. Ultimately, through the generation of multi-layer first sparse feature images, we can obtain a deeper and more comprehensive description of the original image, providing more powerful support for image processing problems in various application fields.

[0115] B2: constructing deep feature descriptors of all SAR feature points in the SAR feature point set on the multi-layer first sparse feature image.

[0116] Constructing deep feature descriptors of all SAR feature points in the SAR feature point set on the multi-layer first sparse feature image is a step aimed at more comprehensively and accurately characterizing the characteristics and information of these feature points by performing deep feature extraction and description on the key feature points in the image. This process involves associating and describing each SAR feature point on the multi-layer feature image to provide a more powerful and accurate feature representation for subsequent pattern recognition, target detection and image matching tasks.

[0117] In a possible implementation, step B2 constructs deep feature descriptors of all SAR feature points in the SAR feature point set on the multi-layer first sparse feature image, which can be implemented by the following steps:

[0118] C1: For each SAR feature point in the SAR feature point set, a second image sub-region is determined on each layer of the first sparse feature image with the SAR feature point as the center, to obtain multiple second image sub-regions of the SAR feature point on the multiple layers of the first sparse feature image.

[0119] One SAR feature point corresponds to a plurality of second image sub-regions, and the number of the second image sub-regions corresponding to the SAR feature point is equal to the number of image layers of the second image sub-regions.

[0120] For each SAR feature point in the SAR feature point set, a second image sub-region is determined with the SAR feature point as the center on each layer of the first sparse feature image, and multiple second image sub-regions of the SAR feature point on multiple layers of the first sparse feature image are obtained, which means that for each SAR feature point, a series of second image sub-regions with the point as the core are extracted with the SAR feature point as the center in the first sparse feature images of different levels. The purpose of this is to deeply explore and describe the local structure and texture features around each SAR feature point in the multi-level and multi-scale feature information.

[0121] Specifically, this process can be achieved through the following steps:

[0122] SAR feature point positioning: For a given set of SAR feature points, first accurately locate the position of each SAR feature point in the multi-layer first sparse feature image;

[0123] Second image sub-region extraction: With each SAR feature point as the center, the corresponding second image sub-region is determined on the first sparse feature image of each layer. These sub-regions represent local image blocks with the SAR feature point as the core, which are used to capture the contextual information around the feature point;

[0124] Multi-layer feature description: By extracting multiple second image sub-regions, multiple second image sub-regions of each SAR feature point on the multi-layer first sparse feature image can be obtained. In this way, feature descriptions at different levels can be obtained, thereby more comprehensively characterizing the context information of each SAR feature point;

[0125] By extracting multiple second image sub-regions for each SAR feature point in the SAR feature point set and obtaining the feature information of these sub-regions in the multi-layer first sparse feature image, it is possible to more comprehensively reveal and describe the local features around important feature points in the image, and provide a richer and more accurate feature representation for further feature analysis and image processing tasks. This approach helps to enhance the understanding of the intrinsic structure and content of the image, and promotes the precise implementation and performance improvement of various tasks in the field of image processing.

[0126] C2: for each of the second image sub-regions, converting the first sparse feature image where the second image sub-region is located to obtain a second sparse feature image

[0127] The extracted second image sub-region is further processed using the second layer convolutional sparse dictionary in the convolutional sparse coding model. Specifically, the second image sub-region is convolved using 64 convolution kernels (each with a size of 32) in the second layer convolutional sparse dictionary. Through the convolution operation, a more abstract and rich feature representation can be obtained, thereby generating a second sparse feature image. The second sparse feature image not only contains the information of the original SAR feature points on the first layer sparse feature map, but also further enhances the feature expression capability through the second layer convolution operation.

[0128] For each second image sub-region, the first sparse feature image is converted to obtain a corresponding second sparse feature image.

[0129] C3: merging the multiple second sparse feature images corresponding to each SAR feature point to obtain a deep feature descriptor of each SAR feature point.

[0130] The generated multiple second sparse feature images are merged to obtain a comprehensive, higher-level feature description. This merging can be done by simply stacking multiple channel feature maps, or by more complex fusion methods such as weighted averaging or further processing using deep neural networks.

[0131] By merging multiple second sparse feature images, a richer and more abstract feature representation can be obtained, namely the deep feature descriptor of each SAR feature point. This descriptor not only contains the local information of the original feature point, but also integrates the deep features extracted by multi-layer convolution operations, thereby providing higher expression ability and stronger robustness.

[0132] It should be noted that the process of "constructing deep feature descriptors of all optical feature points in the optical feature point set on the optical image" is similar to the process of "constructing deep feature descriptors of all SAR feature points in the SAR feature point set on the SAR image", and refer to steps B1-B2 for details.

[0133] S103: Based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image, feature point matching is performed on feature points in the SAR feature point set and feature points in the optical feature point set.

[0134] Based on the different levels of feature descriptors extracted from SAR images and optical images, the feature points in the SAR image are matched with the feature points in the optical image. This process aims to effectively combine the image information obtained by the two sensors and establish a corresponding relationship through this matching process. This method can improve the expressiveness and robustness of image features, help achieve more precise and accurate image analysis, target detection or other visual tasks, and provide strong support for the processing of cross-modal image data.

[0135] In a possible implementation, the matching of feature points in the SAR feature point set with feature points in the optical feature point set based on the shallow feature descriptor and the deep feature descriptor of the SAR image and the shallow feature descriptor and the deep feature descriptor of the optical image includes:

[0136] Based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image, and using a sparse representation (SR) matching criterion that introduces locality constrained sparse coding, feature points in the SAR feature point set and feature points in the optical feature point set are matched.

[0137] By combining the shallow and deep feature descriptors of the SAR image and the optical image, and using the SR matching criterion with locality-constrained sparse coding, the feature points in the SAR image are matched with the feature points in the optical image. Specifically, this process can be achieved through the following steps:

[0138] (1) SR matching criteria:

[0139] The sparse coding method with local constraints is introduced to construct the matching criterion. The objective function of the matching criterion is: Among them, r n is the reference feature point, F(r n ) is its feature descriptor, c n | is the reference feature point descriptor r n The sparse coefficient, D(s n ) is the feature point s to be registered n A sparse dictionary (basis vector) formed by a local neighborhood centered on . dn is a local adaptation that provides different degrees of freedom for each basis vector: dist(Fr n , Ds n )=[dist(Fr1,Ds1),...,dist(Fr n , Ds n )] T Denotes the reference feature point descriptor F(r n ) and dictionary D(s n ) is the Euclidean distance of each element in . According to the obtained sparse coefficient, the reference feature point r n and the feature point to be registered s n The reconstruction error is: e n =||D(s n )c n -F(r n )||2.

[0140] Finally, we find the n ) The smallest reconstruction error e1, and the second smallest reconstruction error e2. Threshold matching is performed through the ratio of e1 / e2. In this process, the threshold is set to 0.5, which can achieve the best matching performance.

[0141] (2) Feature point matching: Using the sparse representation matching criteria constructed above, the feature point set extracted from the SAR image is matched with the feature point set extracted from the optical image. Specifically, for each feature point in the SAR image, the best matching feature point in the optical image is found, thereby achieving feature point pairing.

[0142] S104: Fusing the SAR image and the optical image according to the matched feature points to obtain a fused image.

[0143] By fusing the SAR image and the optical image using the matched feature points, a fused image is generated that integrates the data information of two different sensors. This process can combine the features of the SAR image and the optical image, provide richer and more comprehensive information, and show higher details and quality in the fused image, thus providing a more accurate and comprehensive data foundation for image processing, analysis and other application fields.

[0144] Based on the contents of S101-S104, it can be known that the SAR image and the optical image of the target viewing area are first obtained, and the feature points of the two images are extracted to obtain the SAR feature point set and the optical feature point set. For the SAR image, the shallow feature descriptors and deep feature descriptors of all SAR feature points are constructed; and on the optical image, the corresponding shallow and deep feature descriptors of all optical feature points are also constructed. Next, the feature points in the SAR feature point set are associated with the feature points in the optical feature point set to find the corresponding matching relationship between them. Finally, the SAR image and the optical image are fused according to the matched feature points to generate a new fused image. The present application can accurately capture key feature points and reduce mismatching by extracting and matching feature points, combining the shallow and deep feature descriptors of SAR and visible light images. This method helps to retain high-frequency features and weak target information and improve the quality of fusion results. The feature point matching method is more accurate in aligning and fusing image details and simplifies the complex registration process. The use of deep feature descriptors can stabilize matching, reduce complexity, increase the robustness of matching, and improve the accuracy of the fusion process.

[0145] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an image fusion device provided in an embodiment of the present application. Figure 4 As shown, the image fusion device comprises:

[0146] An acquisition unit 401 is used to acquire a synthetic aperture radar SAR image and an optical image of a target viewing area;

[0147] A feature point extraction unit 402 is used to extract feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set;

[0148] The descriptor construction unit 403 is used to construct shallow feature descriptors and deep feature descriptors of all SAR feature points in the SAR feature point set on the SAR image; and to construct shallow feature descriptors and deep feature descriptors of all optical feature points in the optical feature point set on the optical image;

[0149] A feature point matching unit 404 is used to match feature points in the SAR feature point set with feature points in the optical feature point set based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image;

[0150] The image fusion unit 405 is used to fuse the SAR image and the optical image according to the matched feature points to obtain a fused image.

[0151] In a possible implementation, the descriptor construction unit 403 specifically includes:

[0152] A first image conversion unit, used for converting the SAR image into a nonlinear diffusion scale image;

[0153] A second image conversion unit, used for converting the nonlinear diffusion scale image into a shear wave scale image and a phase consistency scale image;

[0154] An image scale fusion unit, used for scale fusion of the shear wave scale image and the phase consistency scale image to obtain a multi-scale fused image;

[0155] The shallow feature descriptor construction unit is used to construct shallow feature descriptors of all SAR feature points in the SAR feature point set on the multi-scale fusion image.

[0156] In a possible implementation, the shallow feature descriptor construction unit specifically includes:

[0157] A first image sub-region determining unit is used to determine, for each SAR feature point in the SAR feature point set, a first image sub-region on each image layer of the multi-scale fusion image with the SAR feature point as the center, to obtain a plurality of first image sub-regions of the SAR feature point on the multi-scale fusion image; one SAR feature point corresponds to a plurality of first image sub-regions, and the number of first image sub-regions corresponding to the SAR feature point is equal to the number of image layers of the multi-scale fusion image;

[0158] The sub-region merging unit is used to merge the multiple first image sub-regions corresponding to each SAR feature point to obtain a shallow feature descriptor of each SAR feature point.

[0159] In a possible implementation, the descriptor construction unit 403 further includes:

[0160] A third image conversion unit, used for converting the SAR image into a multi-layer first sparse feature image;

[0161] The deep feature descriptor construction unit is used to construct deep feature descriptors of all SAR feature points in the SAR feature point set on the multi-layer first sparse feature image.

[0162] In a possible implementation, the deep feature descriptor construction unit specifically includes:

[0163] A second image sub-region determining unit is configured to determine, for each SAR feature point in the SAR feature point set, a second image sub-region on each layer of the first sparse feature image with the SAR feature point as the center, to obtain a plurality of second image sub-regions of the SAR feature point on the multi-layer first sparse feature image; one SAR feature point corresponds to a plurality of second image sub-regions, and the number of the second image sub-regions corresponding to the SAR feature point is equal to the number of image layers of the second image sub-region;

[0164] a fourth image conversion unit, configured to convert, for each of the second image sub-regions, the first sparse feature image where the second image sub-region is located to obtain a second sparse feature image;

[0165] The sparse feature image merging unit is used to merge the multiple second sparse feature images corresponding to each SAR feature point to obtain a deep feature descriptor of each SAR feature point.

[0166] In a possible implementation, the feature point extraction unit 402 is specifically used to:

[0167] By using a coherent enhanced diffusion (CED-Harris) detector that introduces coherent enhanced diffusion and spatial constraints, feature points are extracted from the SAR image and the optical image to obtain the SAR feature point set and the optical feature point set.

[0168] In a possible implementation, the feature point matching unit 404 is specifically used to:

[0169] Based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image, and using a sparse representation (SR) matching criterion that introduces locality constrained sparse coding, feature points in the SAR feature point set and feature points in the optical feature point set are matched.

[0170] In addition, an embodiment of the present application further provides an image fusion device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image fusion method described above is implemented.

[0171] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the image fusion method as described above.

[0172] The embodiment of the present application provides an image fusion device, which first uses an acquisition unit 401 to acquire a synthetic aperture radar SAR image and an optical image of a target viewing area, and uses a feature point extraction unit 402 to extract feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set. The descriptor construction unit 403 constructs shallow feature descriptors and deep feature descriptors of all SAR feature points in the SAR feature point set on the SAR image; and constructs shallow feature descriptors and deep feature descriptors of all optical feature points in the optical feature point set on the optical image. Then, the feature point matching unit 404 performs feature point matching on the feature points in the SAR feature point set and the feature points in the optical feature point set based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image, so that the image fusion unit 405 can fuse the SAR image and the optical image according to the matched feature points to obtain a fused image. The present application extracts feature points from SAR images and visible light images respectively, constructs shallow and deep feature descriptors, and matches these descriptors to significantly improve the accuracy of feature point matching. This method can better capture the key feature points in the image, thereby reducing the situation of mismatching. At the same time, through feature point matching, the high-frequency features and weak target information of the target area can be more effectively retained. In the traditional fusion process, due to the differences in geometric structure and texture features between SAR images and visible light images, detail information may be lost. The matching method based on feature points can align and fuse these detail information more accurately. Using shallow and deep feature descriptors for feature point matching can simplify the complex registration process to a certain extent. Deep feature descriptors can capture deeper semantic information, which helps to improve the stability and robustness of matching, thereby reducing the complexity of the fusion process.

[0173] The above is a detailed introduction to an image fusion method, device, equipment and storage medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0174] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0175] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. An image fusion method, characterized in that: The method comprises: Acquire a synthetic aperture radar SAR image and an optical image of a target viewing area, and extract feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set; On the SAR image, shallow feature descriptors and deep feature descriptors of all SAR feature points in the SAR feature point set are constructed; on the optical image, shallow feature descriptors and deep feature descriptors of all optical feature points in the optical feature point set are constructed; Based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image, feature point matching is performed on feature points in the SAR feature point set and feature points in the optical feature point set; The SAR image and the optical image are fused according to the matched feature points to obtain a fused image.

2. The method according to claim 1, characterized in that On the SAR image, constructing shallow feature descriptors of all SAR feature points in the SAR feature point set, including: Converting the SAR image into a nonlinear diffusion scale image; Converting the nonlinear diffusion scale image into a shear wave scale image and a phase consistency scale image; Scale-fusing the shear wave scale image and the phase consistency scale image to obtain a multi-scale fused image; On the multi-scale fused image, shallow feature descriptors of all SAR feature points in the SAR feature point set are constructed.

3. The method according to claim 2, characterized in that The step of constructing shallow feature descriptors of all SAR feature points in the SAR feature point set on the multi-scale fusion image includes: For each SAR feature point in the SAR feature point set, a first image sub-region is determined on each image layer of the multi-scale fusion image with the SAR feature point as the center, to obtain a plurality of first image sub-regions of the SAR feature point on the multi-scale fusion image; one SAR feature point corresponds to a plurality of first image sub-regions, and the number of first image sub-regions corresponding to the SAR feature point is equal to the number of image layers of the multi-scale fusion image; The multiple first image sub-regions corresponding to each SAR feature point are merged to obtain a shallow feature descriptor of each SAR feature point.

4. The method according to claim 1, characterized in that: On the SAR image, constructing deep feature descriptors of all SAR feature points in the SAR feature point set, including: Converting the SAR image into a multi-layer first sparse feature image; On the multi-layer first sparse feature image, deep feature descriptors of all SAR feature points in the SAR feature point set are constructed.

5. The method according to claim 4, characterized in that The step of constructing deep feature descriptors of all SAR feature points in the SAR feature point set on the multi-layer first sparse feature image comprises: For each SAR feature point in the SAR feature point set, a second image subregion is determined on each layer of the first sparse feature image with the SAR feature point as the center, to obtain a plurality of second image subregions of the SAR feature point on the multi-layer first sparse feature image; one SAR feature point corresponds to a plurality of second image subregions, and the number of the second image subregions corresponding to the SAR feature point is equal to the number of image layers of the second image subregion; For each of the second image sub-regions, converting the first sparse feature image where the second image sub-region is located to obtain a second sparse feature image; The plurality of second sparse feature images corresponding to each SAR feature point are merged to obtain a deep feature descriptor of each SAR feature point.

6. The method according to claim 1, characterized in that The extracting feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set includes: By using a coherent enhanced diffusion (CED-Harris) detector that introduces coherent enhanced diffusion and spatial constraints, feature points are extracted from the SAR image and the optical image to obtain the SAR feature point set and the optical feature point set.

7. The method according to claim 1, characterized in that The matching of feature points in the SAR feature point set with feature points in the optical feature point set based on the shallow feature descriptor and the deep feature descriptor of the SAR image and the shallow feature descriptor and the deep feature descriptor of the optical image comprises: Based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image, and using a sparse representation (SR) matching criterion that introduces locality constrained sparse coding, feature points in the SAR feature point set and feature points in the optical feature point set are matched.

8. An image fusion device, characterized in that: The device comprises: An acquisition unit, used for acquiring a synthetic aperture radar SAR image and an optical image of a target viewing area; A feature point extraction unit, used for extracting feature points from the SAR image and the optical image to obtain a SAR feature point set and an optical feature point set; A descriptor construction unit is used to construct shallow feature descriptors and deep feature descriptors of all SAR feature points in the SAR feature point set on the SAR image; and to construct shallow feature descriptors and deep feature descriptors of all optical feature points in the optical feature point set on the optical image; A feature point matching unit, configured to perform feature point matching on feature points in the SAR feature point set and feature points in the optical feature point set based on the shallow feature descriptors and deep feature descriptors of the SAR image and the shallow feature descriptors and deep feature descriptors of the optical image; The image fusion unit is used to fuse the SAR image and the optical image according to the matched feature points to obtain a fused image.

9. An image fusion device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image fusion method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the image fusion method according to any one of claims 1 to 7.

Images