Remote sensing image fusion method and device, electronic equipment and storage medium

Abstract

The invention provides a remote sensing image fusion method and device, electronic equipment and a storage medium, and relates to the technical field of remote sensing image fusion. The method comprises the steps of obtaining a low-frequency approximate component and a plurality of high-frequency detail components according to image data, and extracting directional texture feature information corresponding to the image data based on connectivity analysis; determining a spatial offset parameter of the image data according to the directional texture feature information, performing pixel-level position calibration on the image data, performing weighted fusion on the low-frequency approximate component and each high-frequency detail component after calibration is completed to obtain the fused low-frequency approximate component and each high-frequency detail component, and performing weighted fusion on the fused low-frequency approximate component and each high-frequency detail component; and reconstructing and generating a fused target image. The method aims at solving the problems that in an existing remote sensing image fusion method, texture dislocation is caused by space deviation between images, and the fusion result lacks details and is distorted, and fusion accuracy and stability can be improved.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image fusion technology, and more specifically, to a remote sensing image fusion method, apparatus, electronic device, and storage medium. Background Technology

[0002] Multispectral remote sensing technology, with its ability to simultaneously capture information about targets across multiple spectral bands, plays an irreplaceable role in fields such as resource exploration, environmental monitoring, agricultural yield estimation, and military reconnaissance. Satellites such as Gaofen-2 and Gaofen-1 are equipped with detectors covering the visible and near-infrared spectral bands to enhance the information acquisition capabilities of remote sensing satellites. Remote sensing images in different spectral bands each have their advantages: panchromatic images typically have high spatial resolution, clearly revealing the detailed structure of ground features; multispectral images possess rich spectral information, accurately distinguishing the material and type of ground features.

[0003] However, single-type remote sensing images are insufficient to meet the application requirements of complex scenarios. For example, panchromatic images lack spectral discrimination capabilities, while multispectral images suffer from blurred spatial details. Therefore, multispectral remote sensing image fusion technology has emerged. Its core objective is to organically combine the high spatial resolution information of panchromatic images with the high spectral resolution information of multispectral images through specific algorithm models, generating a fused image that possesses both high spatial detail and rich spectral features, thereby improving the utilization rate of remote sensing data and the accuracy of subsequent interpretation and analysis.

[0004] However, existing image fusion methods often struggle to effectively address texture misalignment caused by spatial offset between images when processing high-resolution remote sensing images. This leads to artifacts such as ghosting and blurring in the fused image. Furthermore, traditional fusion algorithms typically employ fixed thresholds or simple weighting during feature extraction, failing to adaptively preserve texture features in different directions. This can easily result in the loss of important details or amplification of noise. Particularly when processing remote sensing images with complex texture structures, existing methods struggle to accurately identify and match directional texture features between different images, resulting in fusion results that fall short of ideal performance in terms of spatial detail and spectral fidelity.

[0005] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0006] The purpose of this invention is to provide a remote sensing image fusion method, apparatus, electronic device, and storage medium, which aims to solve the problems of texture misalignment caused by spatial offset between images and lack of detail and distortion in the fusion results in existing remote sensing image fusion methods, thereby improving the accuracy and stability of fusion.

[0007] In a first aspect, the present invention provides a remote sensing image fusion method, comprising the following steps: S1. By performing wavelet decomposition on the first and second image data to be fused, low-frequency approximate components and multiple high-frequency detail components containing texture information in different directions are obtained; S2. Perform feature enhancement processing on the high-frequency detail components, and extract directional texture feature information corresponding to the first image data and the second image data based on connectivity analysis; S3. Based on the directional texture feature information, determine the spatial offset parameters of the first image data and the second image data in the horizontal and vertical directions; S4. Based on the spatial offset parameter, perform pixel-level position calibration on the first image data and the second image data, and after completing the calibration, perform weighted fusion on the low-frequency approximation component and each of the high-frequency detail components to obtain the fused low-frequency approximation component and each of the high-frequency detail components. S5. By performing wavelet inverse transform on the fused low-frequency approximation components and each high-frequency detail component, the fused target image is reconstructed.

[0008] The remote sensing image fusion method provided by this invention can effectively solve the texture misalignment problem caused by spatial offset between images in existing remote sensing image fusion methods, and adaptively retain texture features in different directions, thereby generating a fused image with both high spatial detail and rich spectral features, improving the utilization rate of remote sensing data and the accuracy of subsequent interpretation and analysis.

[0009] In a second aspect, the present invention provides a remote sensing image fusion device, comprising: The decomposition module is used to perform wavelet decomposition on the first and second image data to be fused to obtain low-frequency approximate components and multiple high-frequency detail components containing texture information in different directions. The extraction module is used to perform feature enhancement processing on the high-frequency detail components and extract directional texture feature information corresponding to the first image data and the second image data based on connectivity analysis; The determining module is used to determine the spatial offset parameters of the first image data and the second image data in the horizontal and vertical directions based on the directional texture feature information. The calibration fusion module is used to perform pixel-level position calibration on the first image data and the second image data based on the spatial offset parameter, and after the calibration is completed, to perform weighted fusion on the low-frequency approximation component and each of the high-frequency detail components to obtain the fused low-frequency approximation component and each of the high-frequency detail components. The reconstruction module is used to reconstruct the fused target image by performing wavelet inverse transform on the fused low-frequency approximation components and each high-frequency detail component.

[0010] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the steps of the method provided in the first aspect above.

[0011] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0012] As can be seen from the above, the remote sensing image fusion method provided by this invention effectively solves the texture misalignment problem caused by spatial offset between images in the prior art, and avoids artifacts such as ghosting and blurring in the fused image. Simultaneously, through feature enhancement and connectivity analysis, it can adaptively preserve texture features in different directions, overcoming the problem that traditional fusion algorithms cannot effectively handle complex texture structures, and avoiding the loss of important details or the amplification of noise. Therefore, the method of this application can generate fused images with both high spatial detail and rich spectral features, significantly improving the utilization rate of remote sensing data and the accuracy of subsequent interpretation and analysis.

[0013] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0014] Figure 1 This is a flowchart of a remote sensing image fusion method provided in an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of a remote sensing image fusion device provided in an embodiment of the present invention.

[0016] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0017] Label Explanation: 100. Decomposition module; 200. Extraction module; 300. Determination module; 400. Calibration and fusion module; 500. Reconstruction module; 13. Electronic equipment; 1301. Processor; 1302. Memory; 1303. Communication bus. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0019] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0020] For this, please refer to Figure 1 , Figure 1 This is a flowchart of a remote sensing image fusion method. The remote sensing image fusion method includes the following steps: S1. By performing wavelet decomposition on the first and second image data to be fused, low-frequency approximate components and multiple high-frequency detail components containing texture information in different directions are obtained; S2. Perform feature enhancement processing on high-frequency detail components, and extract directional texture feature information corresponding to the first image data and the second image data based on connectivity analysis; S3. Based on the directional texture feature information, determine the spatial offset parameters of the first image data and the second image data in the horizontal and vertical directions; S4. Based on the spatial offset parameter, perform pixel-level position calibration on the first image data and the second image data, and after completing the calibration, perform weighted fusion on the low-frequency approximation component and each high-frequency detail component to obtain the fused low-frequency approximation component and each high-frequency detail component. S5. By performing wavelet inverse transform on the fused low-frequency approximation components and each high-frequency detail component, the fused target image is reconstructed.

[0021] For ease of understanding, the following explains some key terms in this embodiment: Wavelet decomposition: a signal processing technique that decomposes an image into components of different frequencies and directions, including low-frequency approximation components and high-frequency detail components. The low-frequency approximation components represent the overall information of the image, while the high-frequency detail components contain detailed information such as texture and edges.

[0022] Feature enhancement processing refers to the process of highlighting specific features (such as texture and edges) in an image through a series of image processing algorithms, while suppressing noise, in order to improve the accuracy of subsequent feature extraction.

[0023] Connectivity analysis: an image processing technique used to identify interconnected pixel regions in an image, which typically represent specific structures or objects within the image. In this application, it is used to extract directional texture feature information from an image.

[0024] Oriented texture feature information refers to information describing the direction, intensity, length, and other attributes of textures in an image, such as normalized position information, texture angle, texture length, and feature point pixels. This information is crucial for image registration and fusion.

[0025] Spatial offset parameter: refers to the pixel-level displacement between two images in the horizontal and vertical directions, used to correct spatial misalignment between images.

[0026] Pixel-level position calibration: refers to precisely adjusting the position of pixels in an image based on spatial offset parameters to make two images perfectly aligned in space.

[0027] Weighted fusion: An image fusion method that combines different image components into a new image by assigning different weights to them. The choice of weights is usually based on the quality, importance, or specific application requirements of the components.

[0028] Inverse wavelet transform: the inverse process of wavelet decomposition, used to reconstruct the low-frequency approximation components and high-frequency detail components after decomposition back to the original image space, generating the fused target image.

[0029] This application proposes a remote sensing image fusion method, which aims to solve the problems of high computational complexity and poor real-time performance of existing remote sensing image fusion algorithms when processing high-resolution, large-scale images, while improving the accuracy and stability of fusion.

[0030] Specifically, the method includes the following steps: In step S1, wavelet decomposition is performed on the first and second image data to be fused to obtain low-frequency approximation components and multiple high-frequency detail components containing texture information in different directions. Wavelet decomposition can decompose an image into sub-bands of different frequencies, where the low-frequency components represent the overall structure and brightness information of the image, while the high-frequency components contain detailed information such as edges and textures. For example, algorithms such as Discrete Wavelet Transform (DWT) or Wavelet Packet Decomposition (WPT) can be used for decomposition. In practical applications, different wavelet basis functions, such as Haar wavelets and Daubechies wavelets, can be selected according to image characteristics and fusion requirements. Wavelet decomposition can effectively separate the approximate information and detailed textures of the image, laying the foundation for subsequent feature extraction and fusion processing, and avoiding the high computational burden caused by directly processing the original data.

[0031] In step S2, feature enhancement processing is performed on the high-frequency detail components, and directional texture feature information corresponding to the first and second image data is extracted based on connectivity analysis. Feature enhancement processing can employ various techniques, such as histogram equalization, contrast stretching, or nonlinear filtering, to enhance texture edge features while suppressing random noise. Connectivity analysis can identify interconnected pixel regions in the image, thereby extracting texture features with specific orientations and structures. For example, 8-connectivity or 4-connectivity algorithms can be used to identify connected regions and calculate their geometric properties, such as area, perimeter, and orientation, to form directional texture feature information. By performing feature enhancement processing on the high-frequency detail components, effective texture information in the image can be highlighted, and noise interference can be suppressed, thereby improving the accuracy of subsequent feature extraction. Extracting directional texture feature information based on connectivity analysis ensures that the extracted structured features are more reliable, providing an accurate basis for subsequent spatial offset parameter calculations and reducing feature misjudgment.

[0032] In step S3, spatial offset parameters in the horizontal and vertical directions of the first and second image data are determined based on directional texture feature information. Determining these spatial offset parameters is crucial for image registration. For example, a feature-based matching method can be used, comparing directional texture feature information extracted from the two images, such as texture direction, normalized position information, and feature point information, to calculate their relative displacement. Specifically, the Euclidean distance or correlation coefficient between feature points can be calculated to find the best matching point pair, thereby estimating the horizontal and vertical offset values. Accurately estimating the spatial offset between images based on texture features avoids the errors caused by relying on global calculations in traditional methods, improving alignment accuracy.

[0033] In step S4, pixel-level position calibration is performed on the first and second image data based on spatial offset parameters. After calibration, the low-frequency approximation component and each high-frequency detail component are weighted and fused separately to obtain the fused low-frequency approximation component and each high-frequency detail component. Pixel-level position calibration can be achieved through geometric transformations such as image translation, rotation, or affine transformation to ensure accurate spatial alignment of the two images. Weighted fusion can employ different weighting strategies based on the characteristics of different components and the fusion objective. For example, for the low-frequency approximation component, average weighting or energy-based weighting can be used to preserve the overall spectral information of the image; for the high-frequency detail component, weighting based on saliency or gradient information can be used to highlight the image's detailed texture. By ensuring image alignment before fusion, misalignment distortion can be reduced, and the advantages of different images can be combined through weighting methods to improve the stability of the fusion and the preservation of details.

[0034] In step S5, the fused target image is reconstructed by performing inverse wavelet transform on the fused low-frequency approximation components and each high-frequency detail component. Inverse wavelet transform is the inverse process of wavelet decomposition, recombinating the fused low-frequency approximation components and high-frequency detail components to generate the final fused image. For example, algorithms such as inverse discrete wavelet transform (IDWT) or inverse wavelet packet transform (IWPT) can be used for reconstruction. Through inverse wavelet transform, the fusion result can be efficiently recovered from the decomposed components, preserving the multi-scale characteristics of wavelet transform and ensuring that the final image retains both spatial detail and spectral information.

[0035] The following example will provide a more detailed explanation of the above technical solution: Suppose we need to fuse a high spatial resolution panchromatic image (first image data) and a high spectral resolution multispectral image (second image data) to generate a remote sensing image that has both high spatial detail and rich spectral information.

[0036] First, after the FPGA (Field-Programmable Gate Array) program is loaded, pixel data from panchromatic and multispectral images are received from the external source and cached sequentially in the FPGA's internal storage area. Combining wavelet lifting algorithm, the FPGA performs wavelet transform synchronously during the reception and caching of panchromatic and multispectral images, achieving in-situ data substitution. In this way, the panchromatic and multispectral images are decomposed into low-frequency approximation components (labeled LL) and high-frequency detail components such as horizontal detail components (labeled HL), vertical detail components (labeled LH), and diagonal detail components (labeled HH). This decomposition effectively separates the image's overall shape and detailed texture, providing a foundation for subsequent processing.

[0037] Next, after obtaining the wavelet transform coefficients of the panchromatic and multispectral images, the FPGA divides these coefficients into approximate and detail components for storage. During storage, threshold filtering is applied to the details to remove high-frequency noise and enhance texture features. For example, threshold filtering is performed on the horizontal detail component HL_PAN of the panchromatic image and the horizontal detail component HL_MS of the multispectral image to obtain HL_PAN_filtered and HL_MS_filtered, removing noise and enhancing the features of horizontal stripes. Similarly, the vertical detail components LH_PAN and LH_MS are filtered to obtain LH_PAN_filtered and LH_MS_filtered. Subsequently, the FPGA uses connected component operations to extract texture features from the horizontal detail components HL and vertical detail components LH of the panchromatic and multispectral images. During extraction, the extraction direction of the connected component is selected based on the texture direction characteristics, resulting in the horizontal and vertical texture data sets of the panchromatic and multispectral images. For example, horizontal connected components are extracted in HL_PAN and HL_MS to obtain the horizontal texture pixel information, and the features of each horizontal texture are calculated, including normalized position information, texture angle, texture length, and feature point pixels. Similar processing is performed on the vertical detail components. This directional texture feature information provides a precise basis for subsequent image registration.

[0038] Then, based on the extracted directional texture feature information, the spatial offset parameters of the panchromatic image and multispectral image in the horizontal and vertical directions are determined. Specifically, the algorithm compares the features of the texture data to find the corresponding textures of HL_PAN and HL_MS, and calculates the horizontal offset value X_offset of the panchromatic image and multispectral image using the normalized position information of the textures and the position information of the feature points. Similarly, based on the texture features, the corresponding textures of LH_PAN and LH_MS are found, and the vertical offset value Y_offset of the panchromatic image and multispectral image is calculated using the normalized position information of the textures and the position information of the feature points. These offset values ​​X_offset and Y_offset are the spatial offset parameters.

[0039] After obtaining the positional offset information X_offset and Y_offset, the algorithm performs pixel-level alignment of the panchromatic image and the multispectral image based on X_offset and Y_offset. Specifically, the approximate components LL_PAN and LL_MS of the panchromatic image and the multispectral image are combined with X_offset and Y_offset for pixel-level alignment. Simultaneously, the detail components of the filtered panchromatic image and the multispectral image, including HL_PAN_filtered, HL_MS_filtered, LH_PAN_filtered, LH_MS_filtered, HH_PAN_filtered, and HH_MS_filtered, are combined with X_offset and Y_offset for pixel-level alignment. After calibration, the aligned low-frequency approximate components and each high-frequency detail component are fused using a weighted method. For example, LL_PAN and LL_MS are merged using a weighted method to obtain the fused approximate component LL_fusion; then, the corresponding detail components of the panchromatic image and the multispectral image are merged using a weighted method to obtain the fused detail components HL_fusion, LH_fusion, and HH_fusion.

[0040] Finally, inverse wavelet transform is performed on the fused wavelet components LL_fusion, HL_fusion, LH_fusion, and HH_fusion in the FPGA. Through reconstruction filters and interpolation operations, the fused low-frequency approximation components and each high-frequency detail component are restored layer by layer to generate a fused target image that combines spatial details and spectral information.

[0041] The aforementioned technical solution achieves efficient and accurate remote sensing image fusion through a series of steps including wavelet decomposition, feature enhancement, offset parameter determination, pixel-level position calibration, and weighted fusion. Compared to traditional fusion algorithms, this application, when processing high-resolution, large-volume remote sensing images, leverages the hardware acceleration capabilities of FPGAs, combined with wavelet lifting algorithms to achieve in-situ data substitution and synchronous wavelet transform, significantly reducing computational complexity and improving real-time performance. For example, in the feature extraction stage, threshold filtering and connected component operations on high-frequency detail components effectively enhance texture edge features and suppress noise, thereby extracting more reliable directional texture feature information. This is more robust than directly extracting features from the original image in traditional methods. In the image registration stage, determining spatial offset parameters based on accurate directional texture feature information enables pixel-level precise alignment, avoiding potential global calculation errors in traditional methods and improving alignment accuracy. In the fusion stage, pixel-level position calibration is performed before weighted fusion, ensuring the stability and detail preservation of the fusion result and effectively avoiding distortion caused by image misalignment. Finally, the fusion result is efficiently recovered through wavelet inverse transform, ensuring that the generated target image possesses both high spatial detail and rich spectral information. The overall technical concept of this application, by optimizing each step of image decomposition, feature extraction, registration, and fusion, effectively solves the problems of high computational complexity, poor real-time performance, and insufficient fusion accuracy in existing technologies, providing a high-performance, high-precision solution for the field of remote sensing image processing.

[0042] In some embodiments, the specific steps in step S1 include: S11. Wavelet lifting algorithm is used to perform wavelet decomposition on the first image data and the second image data respectively to obtain low-frequency approximate components and multiple high-frequency detail components; the multiple high-frequency detail components include horizontal detail components, vertical detail components and diagonal detail components.

[0043] Among these methods, wavelet lifting is an efficient way to implement wavelet transform. This algorithm decomposes the signal into approximate and detail components through a series of "prediction" and "update" operations. Its key feature is its ability to perform in-situ substitution calculations, significantly reducing memory consumption and computational complexity. Specifically, wavelet lifting can decompose the signal based on odd and even samples, iteratively processing with prediction and update operators to calculate wavelet coefficients without increasing storage space. Furthermore, it can employ an integer-to-integer wavelet transform, ensuring data precision during processing and avoiding the accumulation of errors from floating-point operations. This offers significant advantages for hardware implementations, such as in embedded systems like FPGAs.

[0044] Wavelet decomposition is performed on the first and second image data separately to process the two image sources to be fused independently, ensuring that their inherent features are fully preserved during the decomposition process. This can be achieved in two ways: one is sequential processing, where wavelet decomposition is performed on the first image data first, followed by decomposition on the second image data; the other is parallel processing, where multi-core processors or dedicated hardware modules are used to decompose the two image data simultaneously to improve processing efficiency.

[0045] Obtaining low-frequency approximation components and multiple high-frequency detail components is an inherent outcome of wavelet decomposition. Low-frequency approximation components typically represent the overall structure, contours, and main spectral information of an image, while high-frequency detail components carry finer information such as texture, edges, and noise. These components form the basis for subsequent image fusion, where low-frequency components maintain overall image consistency, and high-frequency components enhance spatial details.

[0046] Multiple high-frequency detail components include horizontal, vertical, and diagonal detail components, which correspond to the texture features of the image in different directions. The horizontal detail component (HL) mainly reflects the horizontal edge and texture information of the image; the vertical detail component (LH) mainly reflects the vertical edge and texture information of the image; and the diagonal detail component (HH) mainly reflects the diagonal edge and texture information of the image, while also including random noise in the image to some extent. The acquisition of these directional detail components provides rich directional texture information for subsequent feature enhancement and precise alignment.

[0047] The proposed solution employs a wavelet lifting algorithm to decompose the first and second image data using wavelet decomposition. This efficiently decomposes the original image into low-frequency approximate components and multiple high-frequency detail components containing texture information in horizontal, vertical, and diagonal directions. This decomposition method not only inherits the advantages of wavelet transform in multi-scale analysis, effectively separating the structural and detail information of the image, but also significantly reduces computational complexity and memory requirements through the introduction of the wavelet lifting algorithm. In the overall process of remote sensing image fusion, this efficient decomposition step lays a solid foundation for subsequent feature enhancement, spatial offset parameter determination, pixel-level position calibration, and weighted fusion. It ensures that when processing high-resolution, large-volume remote sensing images, the frequency domain features of the image can be obtained quickly and accurately, thus effectively supporting the real-time performance and accuracy of the entire fusion process. In this way, the proposed solution overcomes the problems of low computational efficiency and high resource consumption of traditional remote sensing image fusion algorithms when processing high-resolution remote sensing images, providing efficient and refined image components for subsequent precise fusion.

[0048] In some embodiments, the specific steps in step S2 include: S21. By performing threshold filtering on the horizontal detail component, vertical detail component and diagonal detail component, the texture edge features of the horizontal detail component and vertical detail component are enhanced, and the random noise of the diagonal detail component is suppressed while retaining the diagonal texture features, the filtered horizontal detail component, vertical detail component and diagonal detail component are obtained. S22. By extracting connected components from the filtered horizontal and vertical detail components, directional texture feature information representing the distribution of image structure is obtained; the directional texture feature information includes normalized position information, texture angle, texture length, and feature point pixels, etc.

[0049] This method effectively solves the problem of noise interference in high-frequency detail components by synergistically applying threshold filtering and connected component extraction techniques, thereby improving the accuracy and reliability of directional texture feature information. Specifically, when processing the horizontal, vertical, and diagonal detail components obtained from wavelet decomposition, threshold filtering is performed first. Threshold filtering is a commonly used image processing technique that filters or adjusts image pixel values ​​by setting one or more thresholds to remove noise and enhance specific features. Specifically, hard threshold filtering can be used, where pixel values ​​below a certain threshold are set to zero, while values ​​above the threshold remain unchanged or are scaled; or soft threshold filtering can be used, where pixel values ​​below the threshold are set to zero, while values ​​above the threshold are shrunk. In addition, adaptive threshold filtering can be used based on local image characteristics, dynamically adjusting the threshold to better adapt to the noise levels and texture intensity of different regions. This step optimizes for the characteristic differences of detail components in different directions: for the horizontal and vertical detail components, which mainly carry edge information, threshold filtering aims to enhance their texture edge features, making them more prominent in subsequent processing. The horizontal and vertical detail components mainly carry texture information such as edges and lines in the horizontal and vertical directions of the image. Thresholding filtering effectively removes random noise below a specific intensity threshold from these components, while preserving and enhancing pixels with intensity above the threshold that represent true texture edges. This makes the horizontal and vertical texture edges of the image clearer and more prominent, facilitating subsequent feature extraction. For diagonal detail components, thresholding focuses on suppressing potential random noise while carefully preserving their inherent diagonal texture features. Diagonal detail components typically contain texture information along the diagonal direction of the image but are also susceptible to random noise. Thresholding selectively suppresses lower-intensity random noise while ensuring that higher-intensity pixels representing true diagonal texture are preserved. This approach aims to remove noise while preserving the image's inherent diagonal texture structure to the greatest extent possible, avoiding information loss. This differentiated filtering strategy ensures that useful texture information is protected to the greatest extent possible while removing noise.

[0050] After thresholding, connected component extraction is performed on the filtered horizontal and vertical detail components. Connected component extraction is an image analysis technique used to identify interconnected regions in an image with the same or similar pixel values. Its basic principle is to traverse the image pixels and group adjacent pixels that meet the conditions into the same connected region according to a preset connectivity criterion (e.g., 8-connectivity or 4-connectivity). Implementation methods can include a two-pass scanning method: the first pass marks connected regions and records equivalence relationships, and the second pass resolves equivalence relationships and re-marks them; or a recursive or stack / queue filling algorithm can be used, starting from an unmarked pixel and recursively or iteratively marking all its connected neighbors as the same region. Through connected component extraction, discrete texture pixels can be organized into structures with clear boundaries and shapes, thereby obtaining directional texture feature information characterizing the distribution of image structure. This feature information includes normalized position information, texture angle, texture length, and feature point pixels. Normalized position information refers to converting the absolute coordinates of texture features in the image into relative coordinates relative to the image size or a reference point, and limiting its range to a specific interval (e.g., [0, 1)). This normalization process helps eliminate the influence of different image sizes or resolutions, making the texture feature locations of different images or regions comparable, facilitating subsequent feature matching and spatial offset calculation. Texture angle refers to the main direction or orientation of a texture feature in an image. For example, for a straight line texture, its texture angle is the tilt angle of the line. Texture angle is an important parameter describing texture directionality and is crucial for identifying and matching texture structures with specific directions. Texture length refers to the spatial extent or size of a texture feature in an image. For example, for a line segment texture, its texture length is the pixel length of the line segment. Texture length reflects the scale information of the texture, helping to distinguish texture structures of different sizes and providing additional discriminative basis for feature matching. Feature point pixels refer to pixels that are salient, unique, and easily identifiable in texture features. These feature points can be endpoints, corners, intersections, or points with local maximum / minimum gradients. As local representatives of the texture, feature point pixels can be used to accurately locate and match corresponding textures in different images and are key information for calculating spatial offset parameters. By first performing threshold filtering to remove noise and then extracting connected components, we can avoid noise points being misidentified as texture structures, thus making the extracted directional texture features purer, more accurate, and more reliable. This processing flow ensures the accuracy of subsequent steps in comparing and calculating spatial offset parameters based on these directional texture features, laying a solid foundation for achieving pixel-level positional calibration and high-quality image fusion.

[0051] The following is a concrete example to illustrate this. When performing feature enhancement processing on high-frequency detail components, threshold filtering can first be applied to the horizontal detail components HL_PAN and HL_MS of the first and second image data (e.g., panchromatic and multispectral images). This filtering aims to remove noise and enhance the horizontal texture stripe features, thus obtaining filtered HL_PAN_filtered and HL_MS_filtered. Subsequently, horizontal connected component extraction is performed on HL_PAN_filtered and HL_MS_filtered to identify and extract the horizontal texture pixel information contained therein. For each extracted horizontal texture, its features are further calculated, which may include normalized position information, texture angle, texture length, and feature point pixels. Similarly, threshold filtering can also be applied to the vertical detail components LH_PAN and LH_MS in the first and second image data to remove noise and enhance the vertical texture stripe features, thus obtaining filtered LH_PAN_filtered and LH_MS_filtered. Next, vertical connected component extraction is performed in LH_PAN_filtered and LH_MS_filtered to identify and extract the vertical texture pixel information contained therein. Similarly, for each extracted vertical texture, its features are calculated, including normalized position information, texture angle, texture length, and feature point pixels. In this way, directional texture feature information representing the distribution of image structure can be accurately extracted from detail components in different directions.

[0052] Through the above technical solution, this application effectively solves the problem in traditional remote sensing image fusion methods where random noise in high-frequency detail components interferes with texture feature extraction, leading to inaccurate feature information and affecting the determination of spatial offset parameters and the quality of the final fused image. Specifically, by applying differentiated threshold filtering to the horizontal, vertical, and diagonal detail components, this application can specifically enhance the texture edge features in the horizontal and vertical directions, while effectively suppressing random noise in the diagonal detail component and preserving its key diagonal texture. Based on this, connected component extraction is then performed, enabling the accurate identification and extraction of directional texture features representing the image structure distribution from purer and more significant texture information. These features include normalized position information, texture angle, texture length, and feature point pixels. This filtering-then-extraction strategy significantly improves the accuracy and reliability of the extracted texture features, providing a solid data foundation for the accurate calculation of the spatial offset parameters of the first and second image data in the horizontal and vertical directions in subsequent steps. This ensures the accuracy of pixel-level position calibration and ultimately generates a fused target image with both high spatial detail and rich spectral information.

[0053] In some embodiments, the specific steps in step S3 include: S31. Based on the directional texture feature information, compare the corresponding textures of the horizontal detail components and vertical detail components in the first image data and the second image data, and calculate the horizontal offset value in the horizontal direction and the vertical offset value in the vertical direction based on the corresponding position information and feature point pixels, and use the horizontal offset value and the vertical offset value as spatial offset parameters.

[0054] The directional texture feature information, obtained after enhancement processing and connectivity analysis, contains key attributes of texture structures with specific orientations in the image. These attributes, such as normalized positional information, texture angles, texture lengths, and feature pixel data, provide a quantitative basis for subsequent accurate comparison and calculation of spatial offset between images. For example, normalized positional information indicates the relative position of the texture in the image, texture angles and lengths describe the geometric shape of the texture, and feature pixel data provides information on the local saliency of the texture. Matching corresponding textures refers to identifying and matching texture structures with the same or similar geometric shapes and spatial locations in two images to be fused (first image data and second image data). This is typically achieved through feature matching algorithms. For example, descriptor-based matching methods, such as SIFT (Scale Invariant Feature Transform) or SURF (Accelerated Robust Feature Transform) algorithms, can be used to describe and match the extracted texture features; alternatively, correlation-based matching methods can be used to find the optimal matching region by calculating the cross-correlation coefficients of local texture patches in the two images. The corresponding positional information and feature pixel data are crucial data for accurate offset calculation. Location information typically refers to the center coordinates or bounding box coordinates of a texture region, while feature point pixels are highly discriminative pixels within the texture, such as corner points or edge points. Based on this information, a geometric correspondence between corresponding textures in two images can be established. For example, feature point pairs can be constructed using this information, and the transformation parameters between images can be solved using least squares or other optimization algorithms. Horizontal and vertical offset values ​​are quantified parameters describing the relative displacement of two images along the X and Y axes. Calculating these offset values ​​usually involves geometric transformation models; for example, affine or rigid body transformation models can be used. The transformation matrix is ​​estimated by matching feature point pairs, and the translation component is extracted as the offset value. Another approach is to directly calculate the statistical mean or median of the coordinate differences between corresponding textures or feature points to obtain a robust offset estimate. Spatial offset parameters are the core input for subsequent image calibration. They encapsulate the calculated horizontal and vertical offset values ​​into a unified parameter set for easy access and application by the image processing system. These parameters directly guide the pixel-level position calibration process, ensuring that the first and second image data can be accurately aligned, laying the foundation for subsequent fusion operations.

[0055] The proposed solution first acquires directional texture feature information after wavelet decomposition and feature enhancement, focusing particularly on texture patterns in the horizontal and vertical detail components. These preprocessed and enriched features lay the foundation for subsequent accurate comparison. Specifically, by comparing the textures of these specific directional detail components in the first and second image data, the precise corresponding texture structures between the two images can be identified. Based on this, a high-precision geometric correspondence can be established using the normalized position information and feature point pixels of these corresponding textures. Based on these precise correspondences, the algorithm can calculate the horizontal offset value in the horizontal direction and the vertical offset value in the vertical direction. These calculated offset values ​​are directly used as spatial offset parameters for subsequent pixel-level position calibration. This solution is closely integrated with the aforementioned steps, forming a collaborative whole. The preceding steps obtain multi-scale image information, including horizontal, vertical, and diagonal detail components, through wavelet decomposition (e.g., using wavelet lifting). Threshold filtering enhances texture edge features and suppresses random noise. Connected component extraction then yields directional texture features containing normalized position information, texture angles, texture lengths, and feature point pixels. It is based on these high-quality, high-precision directional texture features that this scheme enables accurate texture comparison and offset calculation. This logical chain from fine feature extraction to precise offset calculation effectively solves the problem of inaccurate offset estimation in traditional methods. This approach ensures the accuracy of subsequent pixel-level position calibration, significantly improving the quality of the fused image and avoiding fusion artifacts and information loss caused by image misalignment.

[0056] As a specific implementation, the corresponding textures of the horizontal detail component (HL_PAN) of the panchromatic image and the horizontal detail component (HL_MS) of the multispectral image can be found based on texture features. Using the normalized position information and feature point position information of these textures, the horizontal offset value X_offset of the panchromatic and multispectral images is calculated. Similarly, the corresponding textures of the vertical detail component (LH_PAN) of the panchromatic image and the vertical detail component (LH_MS) of the multispectral image are found based on texture features. Using the normalized position information and feature point position information of these textures, the vertical offset value Y_offset of the panchromatic and multispectral images is calculated. Finally, the calculated X_offset and Y_offset are used as spatial offset parameters for subsequent image calibration.

[0057] Through the above technical solution, this application can obtain high-precision horizontal and vertical offset values ​​by comparing the corresponding textures of the horizontal and vertical detail components in the first and second image data, and calculating them based on their precise position information and feature point pixels. This precise offset parameter estimation effectively solves the problem of inaccurate offset value estimation in traditional methods, significantly improving the accuracy of pixel-level position calibration. Ultimately, this allows the fused remote sensing image to better preserve spatial details, reduce artifacts caused by image misalignment, and thus improve the overall quality of the fused image and the accuracy of subsequent interpretation and analysis.

[0058] In some embodiments, the specific steps in step S4 include: S41. Using horizontal and vertical offset values, perform translation transformation on the low-frequency approximation component and high-frequency detail component corresponding to the first image data to align the pixel spatial positions of the first image data and the second image data. S42. After calibration, the low-frequency approximation components of the first image data and the second image data are fused by weighting to obtain the fused low-frequency approximation components. The horizontal detail components, vertical detail components and diagonal detail components of the first image data and the second image data are fused by weighting to obtain the fused horizontal detail components, fused vertical detail components and fused diagonal detail components.

[0059] The horizontal and vertical offset values ​​are calculated based on directional texture feature information and are used to quantify the relative displacement of the first and second image data in the horizontal and vertical directions. These parameters can be integer pixel values ​​or sub-pixel precision values. The purpose of the translation transformation is to spatially shift the first image data to align it with the second image data at the pixel level. The low-frequency approximation component and the high-frequency detail component are obtained from wavelet decomposition. The low-frequency approximation component represents the overall contour and brightness information of the image, while the high-frequency detail components (including horizontal, vertical, and diagonal detail components) contain detailed information such as the image's edges and textures. Performing a translation transformation on these components ensures spatial alignment of the image at different frequency levels. Weighted fusion refers to combining two or more image components according to a certain weight ratio. For the low-frequency approximation component, a simple average weighting can be used, such as each component having a 50% weight, or the weights can be dynamically adjusted based on factors such as image quality and signal-to-noise ratio. For high-frequency detail components, different weights can be assigned based on the richness and sharpness of their texture information. For example, image components with richer detail information can be given higher weights, or an adaptive weighting strategy based on local variance and gradient information can be used. The fused low-frequency approximation component is the result of weighted fusion of the low-frequency approximation components of the first and second image data, which integrates the overall brightness information of the two images. The fused horizontal detail component, fused vertical detail component, and fused diagonal detail component are the results of weighted fusion of the corresponding high-frequency detail components of the first and second image data, which integrate the texture and edge information of the two images in different directions.

[0060] This application's solution achieves pixel-level precise alignment by specifically applying spatial offset parameters to the translation transformation of image components, effectively solving the problem of inaccurate alignment in traditional methods. Based on this, a layered and directional weighted fusion strategy is adopted, enabling the fusion process to more precisely combine the advantages of different image sources, thereby improving fusion efficiency while ensuring the quality of the fused image. Specifically, firstly, using the horizontal and vertical offset values ​​calculated in previous steps, a precise translation transformation is performed on the low-frequency approximation components and high-frequency detail components corresponding to the first image data (e.g., a panchromatic image). This operation ensures that the first image data is spatially aligned with the second image data (e.g., a multispectral image) at the pixel level. By translating the low-frequency approximation components and high-frequency detail components separately, the overall structure and local details of the image are properly handled during the alignment process, avoiding fusion artifacts caused by image misalignment. After completing precise spatial calibration, this application further fuses the low-frequency approximation components of the aligned first and second image data using a weighted method, thereby obtaining the fused low-frequency approximation components. Simultaneously, the horizontal, vertical, and diagonal detail components of the first and second image data are fused separately using a weighted method to obtain fused horizontal, vertical, and diagonal detail components. This layered and directional weighted fusion strategy fully utilizes the advantages of the first image data (such as a high spatial resolution panchromatic image) in detail information and the advantages of the second image data (such as a high spectral resolution multispectral image) in spectral information. By setting appropriate weights for different components, the spatial detail and spectral fidelity of the fused image can be effectively balanced, avoiding information loss or distortion that may result from a single fusion method. For example, higher weights can be given to the detail components of the high spatial resolution image to enhance the texture clarity of the fused image; at the same time, higher weights can be given to the low-frequency components of the high spectral resolution image to maintain the spectral characteristics of the fused image.

[0061] As a specific implementation, after obtaining the horizontal offset value X_offset and the vertical offset value Y_offset, these offset values ​​can be used to perform a translation transformation on the low-frequency approximation component LL_PAN and the high-frequency detail components (HL_PAN, LH_PAN, HH_PAN) of the panchromatic image. For example, a bilinear interpolation algorithm can be used to shift LL_PAN, HL_PAN, LH_PAN, and HH_PAN by X_offset pixels in the horizontal direction and by Y_offset pixels in the vertical direction, thereby achieving precise spatial alignment between these components and their corresponding components (LL_MS, HL_MS, LH_MS, HH_MS) of the multispectral image. After completing the above pixel-level position calibration, the aligned low-frequency approximation components LL_PAN and LL_MS can be weighted and fused. For example, a simple average weighting can be used, i.e., LL_fusion = 0.5 * LL_PAN_aligned + 0.5 * LL_MS; where LL_PAN is represented as LL_PAN_aligned after the translation transformation. For high-frequency detail components, the filtered horizontal detail components HL_PAN_filtered and HL_MS_filtered can be weighted and fused separately to obtain HL_fusion; the vertical detail components LH_PAN_filtered and LH_MS_filtered can be weighted and fused separately to obtain LH_fusion; and the diagonal detail components HH_PAN_filtered and HH_MS_filtered can be weighted and fused separately to obtain HH_fusion. For example, based on experience or image characteristics, higher weights can be assigned to detail components of high spatial resolution images, such as HL_fusion = 0.7 * HL_PAN_filtered_aligned + 0.3 * HL_MS_filtered; where HL_PAN_filtered is represented as HL_PAN_filtered_aligned after translation transformation. In this way, it can be ensured that the fused low-frequency approximation components and each high-frequency detail component fully combine the advantages of the two images.

[0062] Through the above technical solution, this application effectively solves the problems of inaccurate pixel alignment and the impact on the quality and efficiency of the fused image caused by the lack of specific translation transformation and weighted fusion steps in traditional methods. Specifically, by using horizontal and vertical offset values ​​to perform translation transformations on the low-frequency approximation components and high-frequency detail components corresponding to the first image data, the precise alignment of the first and second image data in pixel space is ensured, thereby avoiding fusion artifacts introduced by image misalignment and significantly improving the spatial consistency of the fused image. On this basis, the low-frequency approximation components and each high-frequency detail component are fused separately using a weighted method, so that the fusion process can be optimized according to the characteristics of different components and the advantages of the image source. For example, it can better preserve the detail information of the high spatial resolution image while maintaining the spectral characteristics of the high spectral resolution image. This refined layered fusion strategy not only improves the quality of the fused image, making it have both high spatial detail and rich spectral information, but also improves the efficiency and stability of the entire fusion process through explicit calibration and fusion steps.

[0063] In some embodiments, the specific steps in step S5 include: S51. By using reconstruction filters and interpolation operations, the fused low-frequency approximation components and each high-frequency detail component are restored layer by layer to generate a target image that combines spatial details and spectral information.

[0064] Reconstruction filters are key components in signal processing used to reconstruct continuous signals from discrete samples or high-resolution signals from low-resolution components. Their role is to accurately restore image structure and prevent the loss of high-frequency details during the inverse transform process, thus ensuring the clarity and integrity of the fused image. Reconstruction filters can employ linear phase filters, such as Daubechies or Coiflets filters, which perform well in wavelet reconstruction; or adaptive filters, whose coefficients can be adjusted according to local image characteristics to optimize detail preservation. Interpolation is a mathematical process for estimating unknown values ​​between known data points, often used in image processing for image resizing or filling in missing pixels. Its function is to handle the spatial relationships between pixels, ensuring spatial continuity during image reconstruction and avoiding jagged or blocky artifacts. Interpolation operations can be implemented using nearest-neighbor interpolation, bilinear interpolation, or bicubic interpolation, with bicubic interpolation typically providing smoother image results. Layer-by-layer restoration refers to a systematic reconstruction strategy in which different frequency components (such as low-frequency approximation components and high-frequency detail components) are gradually combined in a certain order (usually from coarse to fine scale). This method can effectively manage the complexity of the reconstruction process, ensure the accurate integration of information at different scales, prevent information confusion, and thus guarantee the complete preservation of spatial details and spectral information during the restoration process. Generating a target image that combines both spatial details and spectral information is the ultimate goal of the entire remote sensing image fusion method. It aims to output a comprehensive image that combines high spatial resolution (usually from panchromatic images) and rich spectral information (usually from multispectral images) to meet the application requirements in complex scenes.

[0065] This application's solution optimizes the generation process of the fused image by introducing a reconstruction filter and interpolation operations, and employing a layer-by-layer recovery strategy to perform inverse wavelet transforms on the fused low-frequency approximation components and each high-frequency detail component. Specifically, after weighted fusion of the low-frequency approximation components and each high-frequency detail component of the first and second image data, these fused components need to be reconstructed back into a complete image. The reconstruction filter plays a crucial role in this process, accurately converting these frequency domain components back to pixel information in the spatial domain, ensuring faithful restoration of the image's structure and texture details, and avoiding detail blurring or distortion problems that may occur in traditional methods. Simultaneously, the interpolation operation handles the spatial relationships between pixels during reconstruction, especially when merging components of different resolutions. It ensures smooth pixel transitions and spatial continuity, thereby avoiding blocky effects or unnatural edges in the image. Furthermore, the layer-by-layer restoration strategy makes the image reconstruction process more systematic, gradually superimposing high-frequency details from basic low-frequency information. This layered and progressive approach not only reduces the computational complexity of a single operation and improves reconstruction efficiency, but also helps to better integrate information from different frequencies, ensuring that spatial details and spectral information are completely and accurately preserved throughout the restoration process. Through this refined reconstruction mechanism, the scheme in this application can efficiently generate target images with both high spatial resolution and rich spectral information, thus effectively solving the problems of computational complexity, low efficiency, and insufficient image detail restoration in traditional reconstruction operations.

[0066] The following is a concrete example. After obtaining the fused low-frequency approximation component LL_fusion, horizontal detail component HL_fusion, vertical detail component LH_fusion, and diagonal detail component HH_fusion, the fused remote sensing image can be reconstructed using inverse wavelet transform. As a specific implementation, this process should utilize a synthesis filter (i.e., a reconstruction filter) corresponding to the analysis filter used during wavelet decomposition. For example, if Daubechies wavelet basis functions were used during decomposition, then the corresponding Daubechies synthesis filter will be used during the inverse transform. The layer-by-layer recovery process can begin with the lowest-frequency LL_fusion component, which is first upsampled, for example, through bilinear or bicubic interpolation, to match its resolution with the next layer of detail components. Subsequently, the upsampled LL_fusion is combined with the HL_fusion, LH_fusion, and HH_fusion components using the synthesis filter to reconstruct the image at that scale. This process is repeated layer by layer until all fused low-frequency approximation components and each high-frequency detail component are integrated, ultimately generating a full-resolution fused target image. In each step of upsampling and combining, the interpolation operation ensures smooth transitions and spatial continuity between pixels, while the reconstruction filter guarantees accurate restoration of image structure and details.

[0067] Through the above technical solution, this application can significantly improve the efficiency and quality of wavelet inverse transform in the remote sensing image fusion process by using refined reconstruction filters and interpolation operations combined with a layer-by-layer recovery strategy. This not only effectively solves the problem of insufficient image detail recovery due to the computational complexity and low efficiency of traditional reconstruction operations, but also ensures that the final fused image has both high spatial detail and rich spectral information, thereby improving the utilization rate of remote sensing data and the accuracy of subsequent interpretation and analysis.

[0068] Please refer to Figure 2 , Figure 2 This is a remote sensing image fusion device according to some embodiments of the present invention. The remote sensing image fusion device is integrated into a back-end control device in the form of a computer program, including: The decomposition module 100 is used to perform wavelet decomposition on the first image data and the second image data to be fused to obtain low-frequency approximate components and multiple high-frequency detail components containing texture information in different directions. The extraction module 200 is used to perform feature enhancement processing on high-frequency detail components and extract directional texture feature information corresponding to the first image data and the second image data based on connectivity analysis; The determining module 300 is used to determine the spatial offset parameters of the first image data and the second image data in the horizontal and vertical directions based on the directional texture feature information; The calibration fusion module 400 is used to perform pixel-level position calibration on the first image data and the second image data based on the spatial offset parameter, and after the calibration is completed, to perform weighted fusion on the low-frequency approximation component and each high-frequency detail component to obtain the fused low-frequency approximation component and each high-frequency detail component. The reconstruction module 500 is used to reconstruct the fused target image by performing wavelet inverse transform on the fused low-frequency approximation components and each high-frequency detail component.

[0069] In some embodiments, the decomposition module 100 is executed when performing wavelet decomposition on the first and second image data to be fused to obtain low-frequency approximate components and multiple high-frequency detail components containing texture information in different directions: S11. Wavelet lifting algorithm is used to perform wavelet decomposition on the first image data and the second image data respectively to obtain low-frequency approximate components and multiple high-frequency detail components; the multiple high-frequency detail components include horizontal detail components, vertical detail components and diagonal detail components.

[0070] In some embodiments, the extraction module 200 is executed when performing feature enhancement processing on high-frequency detail components and extracting directional texture feature information corresponding to the first image data and the second image data based on connectivity analysis: S21. By performing threshold filtering on the horizontal detail component, vertical detail component and diagonal detail component, the texture edge features of the horizontal detail component and vertical detail component are enhanced, and the random noise of the diagonal detail component is suppressed while retaining the diagonal texture features, the filtered horizontal detail component, vertical detail component and diagonal detail component are obtained. S22. By extracting connected components from the filtered horizontal and vertical detail components, directional texture feature information representing the distribution of image structure is obtained; the directional texture feature information includes normalized position information, texture angle, texture length, and feature point pixels, etc.

[0071] In some embodiments, the determining module 300 performs the following when determining the spatial offset parameters of the first image data and the second image data in the horizontal and vertical directions based on the directional texture feature information: S31. Based on the directional texture feature information, compare the corresponding textures of the horizontal detail components and vertical detail components in the first image data and the second image data, and calculate the horizontal offset value in the horizontal direction and the vertical offset value in the vertical direction based on the corresponding position information and feature point pixels, and use the horizontal offset value and the vertical offset value as spatial offset parameters.

[0072] In some embodiments, the calibration fusion module 400 performs pixel-level position calibration on the first image data and the second image data based on the spatial offset parameter, and performs weighted fusion of the low-frequency approximation component and each high-frequency detail component after calibration to obtain the fused low-frequency approximation component and each high-frequency detail component. S41. Using horizontal and vertical offset values, perform translation transformation on the low-frequency approximation component and high-frequency detail component corresponding to the first image data to align the pixel spatial positions of the first image data and the second image data. S42. After calibration, the low-frequency approximation components of the first image data and the second image data are fused by weighting to obtain the fused low-frequency approximation components. The horizontal detail components, vertical detail components and diagonal detail components of the first image data and the second image data are fused by weighting to obtain the fused horizontal detail components, fused vertical detail components and fused diagonal detail components.

[0073] In some embodiments, the reconstruction module 500 is executed when reconstructing the fused target image by performing inverse wavelet transform on the fused low-frequency approximation components and each high-frequency detail component: S51. By using reconstruction filters and interpolation operations, the fused low-frequency approximation components and each high-frequency detail component are restored layer by layer to generate a target image that combines spatial details and spectral information.

[0074] Please refer to Figure 3 , Figure 3This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The present invention provides an electronic device 13, including: a processor 1301 and a memory 1302. The processor 1301 and the memory 1302 are interconnected and communicate with each other via a communication bus 1303 and / or other forms of connection mechanism (not shown). The memory 1302 stores computer-readable instructions executable by the processor 1301. When the electronic device is running, the processor 1301 executes the computer-readable instructions to perform the method in any optional implementation of the above embodiments, thereby achieving the following function: obtaining low-frequency approximate components and components containing different... Multiple high-frequency detail components of directional texture information are identified. Feature enhancement processing is performed on the high-frequency detail components, and directional texture feature information corresponding to the first and second image data is extracted based on connectivity analysis. Spatial offset parameters in the horizontal and vertical directions of the first and second image data are determined based on the directional texture feature information. Pixel-level position calibration is performed on the first and second image data based on the spatial offset parameters. After calibration, the low-frequency approximation component and each high-frequency detail component are weighted and fused to obtain the fused low-frequency approximation component and each high-frequency detail component. The fused target image is reconstructed by performing wavelet inverse transform on the fused low-frequency approximation component and each high-frequency detail component.

[0075] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the method in any optional implementation of the above embodiments to achieve the following functions: performing wavelet decomposition on first and second image data to be fused to obtain low-frequency approximation components and multiple high-frequency detail components containing texture information in different directions; performing feature enhancement processing on the high-frequency detail components and extracting directional texture feature information corresponding to the first and second image data based on connectivity analysis; determining spatial offset parameters of the first and second image data in the horizontal and vertical directions based on the directional texture feature information; performing pixel-level position calibration on the first and second image data based on the spatial offset parameters, and after calibration, performing weighted fusion on the low-frequency approximation components and each high-frequency detail component to obtain fused low-frequency approximation components and each high-frequency detail component; and reconstructing and generating the fused target image by performing inverse wavelet transform on the fused low-frequency approximation components and each high-frequency detail component.

[0076] The computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0077] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0078] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] Furthermore, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0080] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0081] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A remote sensing image fusion method, characterized in that, Includes the following steps: S1. By performing wavelet decomposition on the first and second image data to be fused, low-frequency approximate components and multiple high-frequency detail components containing texture information in different directions are obtained; S2. Perform feature enhancement processing on the high-frequency detail components, and extract directional texture feature information corresponding to the first image data and the second image data based on connectivity analysis; S3. Based on the directional texture feature information, determine the spatial offset parameters of the first image data and the second image data in the horizontal and vertical directions; S4. Based on the spatial offset parameter, perform pixel-level position calibration on the first image data and the second image data, and after completing the calibration, perform weighted fusion on the low-frequency approximation component and each of the high-frequency detail components to obtain the fused low-frequency approximation component and each of the high-frequency detail components. S5. By performing wavelet inverse transform on the fused low-frequency approximation components and each high-frequency detail component, the fused target image is reconstructed.

2. The remote sensing image fusion method according to claim 1, characterized in that, The specific steps in step S1 include: S11. Wavelet lifting algorithm is used to perform wavelet decomposition on the first image data and the second image data respectively to obtain the low-frequency approximation component and multiple high-frequency detail components; the multiple high-frequency detail components include horizontal detail components, vertical detail components and diagonal detail components.

3. The remote sensing image fusion method according to claim 2, characterized in that, The specific steps in step S2 include: S21. By performing threshold filtering on the horizontal detail component, the vertical detail component, and the diagonal detail component, the texture edge features of the horizontal detail component and the vertical detail component are enhanced, and the random noise of the diagonal detail component is suppressed while retaining the diagonal texture features, the filtered horizontal detail component, vertical detail component, and diagonal detail component are obtained. S22. By extracting connected components from the filtered horizontal and vertical detail components, directional texture feature information representing the distribution of image structure is obtained.

4. The remote sensing image fusion method according to claim 1 or 3, characterized in that, The directional texture feature information includes normalized position information, texture angle, texture length, and feature point pixels.

5. The remote sensing image fusion method according to claim 2, characterized in that, The specific steps in step S3 include: S31. Based on the directional texture feature information, compare the corresponding textures of the horizontal detail component and the vertical detail component in the first image data and the second image data, and calculate the horizontal offset value in the horizontal direction and the vertical offset value in the vertical direction based on the corresponding position information and feature point pixels, and use the horizontal offset value and the vertical offset value as the spatial offset parameter.

6. The remote sensing image fusion method according to claim 5, characterized in that, The specific steps in step S4 include: S41. Using the horizontal offset value and the vertical offset value, perform a translation transformation on the low-frequency approximation component and the high-frequency detail component corresponding to the first image data, so as to align the pixel spatial positions of the first image data and the second image data. S42. After calibration, the low-frequency approximation components of the first image data and the second image data are fused by weighting to obtain the fused low-frequency approximation components. The horizontal detail components, vertical detail components and diagonal detail components of the first image data and the second image data are fused by weighting to obtain the fused horizontal detail components, fused vertical detail components and fused diagonal detail components.

7. The remote sensing image fusion method according to claim 1, characterized in that, The specific steps in step S5 include: S51. By using reconstruction filters and interpolation operations, the fused low-frequency approximation components and each high-frequency detail component are restored layer by layer to generate the target image that combines spatial details and spectral information.

8. A remote sensing image fusion device, characterized in that, include: The decomposition module is used to perform wavelet decomposition on the first and second image data to be fused to obtain low-frequency approximate components and multiple high-frequency detail components containing texture information in different directions. The extraction module is used to perform feature enhancement processing on the high-frequency detail components and extract directional texture feature information corresponding to the first image data and the second image data based on connectivity analysis; The determining module is used to determine the spatial offset parameters of the first image data and the second image data in the horizontal and vertical directions based on the directional texture feature information. The calibration fusion module is used to perform pixel-level position calibration on the first image data and the second image data based on the spatial offset parameter, and after the calibration is completed, to perform weighted fusion on the low-frequency approximation component and each of the high-frequency detail components to obtain the fused low-frequency approximation component and each of the high-frequency detail components. The reconstruction module is used to reconstruct the fused target image by performing wavelet inverse transform on the fused low-frequency approximation components and each high-frequency detail component.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions, which, when executed by the processor, perform the steps of the remote sensing image fusion method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the steps in the remote sensing image fusion method as described in any one of claims 1-7.

Images