Heterogeneous remote sensing image transformation method and system

Abstract

The invention discloses a heterogeneous remote sensing image transformation method and system, and the method comprises the following steps: S1, image preprocessing, S2, deep multi-modal feature extraction, S3, graph theory optimization feature matching, S4, hybrid adaptive transformation model construction, S5, image fusion enhancement, and S6, perception driving quality enhancement. The heterogeneous remote sensing image transformation system comprises a multivariate data access and management module, an ultra-precision preprocessing engine module, a depth feature extraction and precise matching unit module, a self-adaptive high-precision image transformation model construction and optimization module, a refined image fusion and quality enhancement unit module and a result display and interaction module. According to the method, the problems of noise residues, radiation errors and the like in a traditional method are solved by combining technologies of ultra-precise image preprocessing, depth feature extraction and precise matching, self-adaptive high-precision image transformation model construction and refined image fusion and quality enhancement.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a heterogeneous remote sensing image transformation method and system. Background Art

[0002] Since its birth in the early 20th century, remote sensing technology has undergone a major transformation from aerial remote sensing to space remote sensing, with the types of sensors constantly enriched and the resolution continuously improved. Early aerial remote sensing mainly relied on film cameras to obtain images, and then gradually developed into photoelectric scanning imaging, which greatly improved the efficiency and flexibility of image acquisition. After entering the space age, satellite remote sensing has become the mainstream, such as the Landsat series of satellites in the United States and the SPOT satellites in France, which can periodically observe large areas of the earth's surface, providing a rich source of data for global resource monitoring, environmental change research, etc.

[0003] Traditional image transformation methods, which use simple linear transformation, fixed parameter geometric correction and conventional image fusion algorithms, often have obvious limitations in accuracy when facing complex differences in resolution, spectral characteristics, geometric shape and texture characteristics of heterogeneous remote sensing images. This method lacks the ability to accurately model and adaptively adjust complex image differences, resulting in blurring, feature misalignment, spectral information distortion and other problems in the fused image, which seriously affects the accuracy and reliability of subsequent work such as object identification, resource monitoring and environmental assessment based on these image data. Accordingly, the present application proposes a heterogeneous remote sensing image transformation method and system. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a heterogeneous remote sensing image transformation method and system.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A heterogeneous remote sensing image transformation method and system, comprising the following steps:

[0007] S1. Image preprocessing, combining non-local mean filtering NLM, variational partial differential equations PDE, radiation transfer model, machine learning, and elastic deformation and polynomial transformation methods based on DEM and GCPs, to perform noise suppression, radiation calibration and atmospheric correction, as well as geometric correction and terrain correction on heterogeneous remote sensing images to obtain image data with high signal-to-noise ratio, consistent radiation and accurate geometry;

[0008] S2. Deep multimodal feature extraction: building a deep convolutional neural network (CNN) that integrates multimodal information such as optics and radar. Through convolution kernels of different scales and specially designed feature extraction layers, combined with attention mechanism and cross-modal connection, deep features with high discriminability and stability are extracted from preprocessed images.

[0009] S3, graph theory optimizes feature matching, constructs feature points into a graph model, uses deep learning feature descriptors to initialize the edge weights of the graph, and uses the graph cut algorithm combined with iterative optimization of spatial neighborhood and local geometric consistency to achieve accurate feature matching, reduce mismatched points, and provide reliable feature correspondence for image transformation;

[0010] S4. Hybrid adaptive transformation model construction, integrating the geometric transformation model based on physical imaging principles and the image generation model based on deep learning. The model is trained by designing a loss function that comprehensively considers geometry, radiation consistency and visual perception quality. At the same time, a dynamic adaptive adjustment mechanism is established to optimize the model parameters and structure in real time according to the error feedback during the image transformation process to achieve high-precision image transformation.

[0011] S5, image fusion enhancement, based on semantics, the image is segmented, and for different types of land objects, such as vegetation, water bodies, buildings, etc., appropriate fusion rules and weight distribution strategies are used to perform fine fusion of the transformed images to improve the information integrity and accuracy of the fused image;

[0012] S6, perception-driven quality enhancement, introduces image quality evaluation indicators based on human visual perception models, such as SSIM, VIF, etc., and uses deep learning image enhancement networks to perform targeted enhancement processing on fused images according to the evaluation results, thereby improving the visual effect and readability of the image and meeting the analysis and interpretation needs of professionals.

[0013] Preferably, in step S1, NLM filtering removes noise and retains detail texture based on local self-similarity of the image, and PDE method further smoothes the residual noise, and the two are combined to adaptively adjust according to the image area and noise characteristics to provide high signal-to-noise ratio image data for subsequent processing;

[0014] The physical method of the radiation transfer model is combined with the data-driven machine learning method. The radiation transfer model is used to combine the sensor radiation calibration parameters for preliminary correction, and then a deep neural network model trained based on ground measurement and reference image data is used for further optimization to eliminate radiation errors and improve radiation consistency accuracy.

[0015] Using high-precision digital elevation model DEM and dense ground control points GCPs, combined with elastic deformation model and polynomial transformation method, preliminary geometric correction is performed through polynomial transformation, and then terrain correction is performed by integrating DEM data based on elastic deformation model, so as to maintain the shape and texture characteristics of the objects while ensuring that the geometric position accurately matches the terrain.

[0016] Preferably, in step S2, a deep convolutional neural network (CNN) architecture that integrates multiple modal information is constructed, rich features are extracted for optical images using convolution kernels of different scales, special convolution layers are designed for radar images to extract scattering and polarization features, and key areas are focused on through an attention mechanism, a cross-modal connection layer is introduced to achieve early fusion interaction, so that feature expression and robustness are improved, and the optimal extraction strategy is learned through large-scale data training.

[0017] Preferably, in step S3, the feature points are regarded as graph vertices, and the similarity is used as the edge weight to construct a feature point graph model. The deep learning feature descriptor is used to calculate the initial similarity to construct the graph, and then the graph cut algorithm is used to segment the matching and non-matching point sets. The spatial neighborhood and local geometric consistency are considered, and an iterative optimization mechanism is introduced to update the graph attributes.

[0018] Preferably, in step S4, a hybrid model is constructed by combining a geometric transformation model based on physical imaging principles and an image generation model based on deep learning, wherein the physical model ensures the physical rationality of the transformation, and the deep learning model supplements and captures complex nonlinear transformation features. By designing a suitable loss function and considering factors such as geometry, radiation consistency, and visual perception quality to train the model, adaptive high-precision transformation is achieved;

[0019] During image transformation, local feature changes and transformation errors are monitored in real time, and model parameters and structures are dynamically adjusted based on feedback information. For example, if the error in a certain area is large, control points are added or parameters of related layers of the deep learning model are adjusted. Parameters are updated through online learning to adapt to image transformation requirements of different types and scenes.

[0020] A heterogeneous remote sensing image transformation system, comprising:

[0021] Multi-data access and management module: It has strong compatibility and can receive heterogeneous remote sensing image data from various satellite platforms, different sensor types and different data formats. At the same time, it can quickly check the integrity of the data, convert the format and classify the data for storage, establish an efficient data indexing and management mechanism, and facilitate subsequent data call and processing;

[0022] Ultra-precision pre-processing engine module: It integrates the multi-scale noise adaptive removal sub-module, the precise radiation calibration and atmospheric correction integrated sub-module, and the high-fidelity geometric correction and terrain correction collaborative processing sub-module, performs comprehensive and high-precision pre-processing operations on the input image data, and outputs high-quality and consistent pre-processed image data;

[0023] Deep feature extraction and precise matching unit module: It consists of a multimodal deep convolution feature extraction submodule and a graph-theory-based feature matching optimization submodule. It uses advanced deep learning and graph theory algorithms to extract and match image features, and passes the precise feature matching results to the adaptive high-precision image transformation model construction unit.

[0024] Adaptive high-precision image transformation model construction and optimization module: Combines the hybrid transformation model submodule based on physical model and deep learning and the model's dynamic adaptive adjustment mechanism submodule to build and optimize the image transformation model. According to the input feature matching results and image data, the model parameters and structure are adaptively adjusted to achieve high-precision image transformation operations.

[0025] Refined image fusion and quality enhancement unit module: It uses a strategy submodule based on region segmentation and multi-scale fusion and a perception quality-driven image enhancement processing submodule to perform refined fusion and quality enhancement processing on the transformed image, and outputs the final heterogeneous remote sensing image transformation result with high quality, rich information and good visual effects;

[0026] Result display and interaction module: displays the processed image data in an intuitive and visual way, and provides rich interactive functions, such as image zooming, panning, contrast display, etc., to facilitate users to evaluate and analyze the processing results. At the same time, it supports the output of processing results into a variety of standard image file formats and data formats to meet the subsequent application needs of different users.

[0027] The present invention has the following beneficial effects:

[0028] 1. Through the combination of ultra-precise image preprocessing, deep feature extraction and precise matching, adaptive high-precision image transformation model construction, and refined image fusion and quality enhancement, it is possible to achieve extremely high accuracy in processing heterogeneous remote sensing image differences, effectively overcoming the problems of residual noise, radiation error, geometric distortion, and inaccurate feature matching in traditional methods, so that the fused image can highly accurately reflect the real characteristics and spatial distribution of the ground objects, providing a solid data foundation for subsequent remote sensing applications.

[0029] 2. This method and system can flexibly respond to various types of heterogeneous remote sensing image data. Regardless of whether it is images of different sensor types, different resolutions, different spectral band ranges or different imaging times, they can be accurately transformed and fused through adaptive algorithms and models. This powerful adaptability and generalization ability enables it to be widely used in many fields, such as natural resource surveys, ecological environment monitoring, disaster warning and assessment, smart city construction, etc., providing customized, high-precision remote sensing image processing solutions for users in different industries.

[0030] 3. The processed images not only have a qualitative leap in geometric accuracy and radiometric accuracy, but also have been significantly improved in information richness and visual quality. Through refined regional segmentation and multi-scale fusion strategies, the key information in different images is fully retained and integrated, so that the images can present the details and features of the objects more clearly and accurately; at the same time, the image enhancement processing based on the human visual perception model makes the images have better visual effects and readability, which is convenient for professionals to interpret and analyze images, thereby greatly improving the availability and application value of remote sensing image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A flowchart of a heterogeneous remote sensing image transformation method proposed by the present invention;

[0032] Figure 2 The figure is a schematic diagram of the module structure of a heterogeneous remote sensing image system proposed in the present invention. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0034] A heterogeneous remote sensing image transformation method comprises the following steps:

[0035] S1. Image preprocessing, combining non-local mean filtering NLM, variational partial differential equations PDE, radiation transfer model, machine learning, and elastic deformation and polynomial transformation methods based on DEM and GCPs, noise suppression, radiation calibration and atmospheric correction, as well as geometric correction and terrain correction are performed on heterogeneous remote sensing images to obtain image data with high signal-to-noise ratio, consistent radiation and accurate geometry. NLM filtering removes noise and retains detailed texture based on local self-similarity of the image, and the PDE method further smoothes the residual noise. The combination of the two is adaptively adjusted according to the image area and noise characteristics to provide high signal-to-noise ratio image data for subsequent processing;

[0036] The physical method of the radiation transfer model is combined with the data-driven machine learning method. The radiation transfer model is used to combine the sensor radiation calibration parameters for preliminary correction, and then a deep neural network model trained based on ground measurement and reference image data is used for further optimization to eliminate radiation errors and improve radiation consistency accuracy.

[0037] Using high-precision digital elevation model DEM and dense ground control points GCPs, combined with elastic deformation model and polynomial transformation method, preliminary geometric correction is performed through polynomial transformation, and then terrain correction is performed by integrating DEM data based on elastic deformation model. While ensuring that the geometric position accurately matches the terrain, the shape and texture characteristics of the object are maintained to achieve high-fidelity correction effect;

[0038] S2. Deep multimodal feature extraction: construct a deep convolutional neural network (CNN) that integrates multimodal information such as optics and radar. Through convolution kernels of different scales and specially designed feature extraction layers, combined with attention mechanism and cross-modal connection, deep features with high discriminability and stability are extracted from preprocessed images.

[0039] Construct a deep convolutional neural network (CNN) architecture that integrates multiple modal information. Use convolution kernels of different scales to extract rich features from optical images. Small-scale convolution kernels can capture subtle texture information of the image, while large-scale convolution kernels can extract the overall shape and contour features of the object. These features are of great significance for the recognition and classification of objects.

[0040] Special convolutional layers are designed for radar images to extract scattering and polarization features. The scattering features of radar images reflect the physical structure and dielectric properties of the ground objects, while the polarization features can provide more information about the direction and shape of the ground objects. Through these specially designed convolutional layers, key feature information in radar images can be effectively extracted.

[0041] It also focuses on key areas through the attention mechanism, introduces a cross-modal connection layer to achieve early fusion interaction, enhances feature expression and robustness, and learns the optimal extraction strategy through large-scale data training;

[0042] S3, graph theory optimizes feature matching, constructs feature points into a graph model, uses deep learning feature descriptors to initialize the edge weights of the graph, and uses the graph cut algorithm combined with iterative optimization of spatial neighborhood and local geometric consistency to achieve accurate feature matching, reduce mismatched points, and provide reliable feature correspondence for image transformation;

[0043] The feature points are regarded as graph vertices, and the similarity is used as the edge weight to build a feature point graph model. The deep learning feature descriptor is used to calculate the initial similarity to build the graph, and then the graph cut algorithm is used to split the matching and non-matching point sets. The spatial neighborhood and local geometric consistency are considered, and an iterative optimization mechanism is introduced to update the graph attributes, reduce mismatching, and achieve accurate feature matching.

[0044] S4. Hybrid adaptive transformation model construction, integrating the geometric transformation model based on physical imaging principles and the image generation model based on deep learning. The model is trained by designing a loss function that comprehensively considers geometry, radiation consistency and visual perception quality. At the same time, a dynamic adaptive adjustment mechanism is established to optimize the model parameters and structure in real time according to the error feedback during the image transformation process to achieve high-precision image transformation.

[0045] A hybrid model is constructed by combining a geometric transformation model based on physical imaging principles and an image generation model based on deep learning.

[0046] The collinear equation model can accurately calculate the geometric transformation parameters of the image by establishing the collinear relationship between the image point, object point and projection center. It plays an important role in correcting basic geometric distortions such as rotation, translation and scaling of the image. At the same time, the physical model ensures the physical rationality of the transformation. The deep learning model supplements the capture of complex nonlinear transformation features. By designing a suitable loss function and considering factors such as geometry, radiation consistency and visual perception quality, the model can be trained to achieve adaptive high-precision transformation.

[0047] Real-time monitoring of local feature changes and transformation errors during image transformation, dynamic adjustment of model parameters and structure based on feedback information. For example, if the error in a certain area is large, control points are added or parameters of related layers of the deep learning model are adjusted. Parameters are updated through online learning to adapt to image transformation requirements of different types and scenes, thus improving the model's adaptive capabilities.

[0048] S5. Image fusion enhancement: segment images based on semantics. For different types of objects, such as vegetation, water, and buildings, appropriate fusion rules and weight distribution strategies are used. The fully convolutional neural network (FCN) is used to classify images at the pixel level, and each pixel in the image is marked as a corresponding object category. By training on a large-scale remote sensing image dataset, the network can learn information such as spectral features, texture features, and geometric shape features of different objects, thereby accurately performing regional segmentation.

[0049] The transformed images are finely fused to improve the information integrity and accuracy of the fused images;

[0050] S6, perception-driven quality enhancement, introduces image quality evaluation indicators based on human visual perception models, such as SSIM and VIF. The SSIM indicator measures the quality of an image by comparing the similarities in brightness, contrast, and structural information of the image. The VIF indicator evaluates the fidelity of the visual information of the image from the perspective of information theory, that is, the degree of closeness between the useful visual information contained in the image and the original image.

[0051] The image enhancement network based on deep learning is used to perform targeted enhancement processing on the fused image according to the evaluation results to improve the visual effect and readability of the image. If the SSIM value of the image is low, it means that the structural information of the image has been lost. At this time, the image enhancement network can automatically adjust the network parameters by learning a large amount of high-quality image data to enhance the structural information of the image; when the VIF value is low, it indicates that the fidelity of the visual information of the image is insufficient. The image enhancement network can focus on improving the fidelity of the visual information of the image to meet the analysis and interpretation needs of professionals.

[0052] A heterogeneous remote sensing image transformation system, comprising:

[0053] The multi-data access and management module has strong compatibility and can receive heterogeneous remote sensing image data from various satellite platforms, different sensor types and different data formats. At the same time, it can quickly check the integrity of the data, convert the format and classify the data for storage, establish an efficient data indexing and management mechanism, and facilitate subsequent data calls and processing;

[0054] Ultra-precision preprocessing engine, integrating multi-scale noise adaptive removal submodule, precise radiation calibration and atmospheric correction integrated submodule, and high-fidelity geometric correction and terrain correction collaborative processing submodule, performs comprehensive and high-precision preprocessing operations on input image data, and outputs high-quality and consistent preprocessed image data;

[0055] Deep feature extraction and precise matching unit: It consists of a multimodal deep convolution feature extraction submodule and a graph-theory-based feature matching optimization submodule. It uses advanced deep learning and graph-theory algorithms to extract and match image features, and passes precise feature matching results to the adaptive high-precision image transformation model construction unit.

[0056] Adaptive high-precision image transformation model construction and optimization module: Combines the hybrid transformation model submodule based on physical model and deep learning and the model's dynamic adaptive adjustment mechanism submodule to build and optimize the image transformation model, and adaptively adjusts the model parameters and structure according to the input feature matching results and image data to achieve high-precision image transformation operations; Refined image fusion and quality enhancement unit: Adopts the strategy submodule based on regional segmentation and multi-scale fusion and the image enhancement processing submodule driven by perception quality to perform refined fusion and quality enhancement processing on the transformed image, and outputs the final heterogeneous remote sensing image transformation result with high quality, rich information and good visual effects;

[0057] Result display and interaction module: displays the processed image data in an intuitive and visual way, and provides rich interactive functions, such as image zooming, panning, contrast display, etc., to facilitate users to evaluate and analyze the processing results. At the same time, it supports the output of processing results into a variety of standard image file formats and data formats to meet the subsequent application needs of different users.

[0058] A heterogeneous remote sensing image transformation system, comprising:

[0059] Multi-data access and management module: It has strong compatibility and can receive heterogeneous remote sensing image data from various satellite platforms, different sensor types and different data formats. At the same time, it can quickly check the integrity of the data, convert the format and classify the data for storage, establish an efficient data indexing and management mechanism, and facilitate subsequent data call and processing;

[0060] Ultra-precision pre-processing engine module: It integrates the multi-scale noise adaptive removal sub-module, the precise radiation calibration and atmospheric correction integrated sub-module, and the high-fidelity geometric correction and terrain correction collaborative processing sub-module, performs comprehensive and high-precision pre-processing operations on the input image data, and outputs high-quality and consistent pre-processed image data;

[0061] Deep feature extraction and precise matching unit module: It consists of a multimodal deep convolution feature extraction submodule and a graph-theory-based feature matching optimization submodule. It uses advanced deep learning and graph theory algorithms to extract and match image features, and passes the precise feature matching results to the adaptive high-precision image transformation model construction unit.

[0062] Adaptive high-precision image transformation model construction and optimization module: Combines the hybrid transformation model submodule based on physical model and deep learning and the model's dynamic adaptive adjustment mechanism submodule to build and optimize the image transformation model. According to the input feature matching results and image data, the model parameters and structure are adaptively adjusted to achieve high-precision image transformation operations.

[0063] Refined image fusion and quality enhancement unit module: It uses a strategy submodule based on region segmentation and multi-scale fusion and a perception quality-driven image enhancement processing submodule to perform refined fusion and quality enhancement processing on the transformed image, and outputs the final heterogeneous remote sensing image transformation result with high quality, rich information and good visual effects;

[0064] Result display and interaction module: displays the processed image data in an intuitive and visual way, and provides rich interactive functions, such as image zooming, panning, contrast display, etc., to facilitate users to evaluate and analyze the processing results. At the same time, it supports the output of processing results into a variety of standard image file formats and data formats to meet the subsequent application needs of different users.

[0065] In the field of forestry resource monitoring, the above-mentioned methods and systems are used to detect heterogeneous remote sensing image applications, and are also tested against implementation cases using other different methods and systems.

[0066] Embodiment 1:

[0067] Step 1: We collected optical remote sensing images (which have high-resolution spectral information and can reflect the types and growth status of vegetation, but are easily affected by the atmosphere and have certain geometric deformation) and radar remote sensing images (which have unique detection capabilities for forest topography, tree structure, and understory vegetation, but are noisy and have significant differences in features from optical images) of a forestry area. These image data come from different satellite platforms and have different data formats, including GeoTIFF and ENVI formats.

[0068] The collected heterogeneous remote sensing image data is imported into the system. The module first performs an integrity check on the data. If it is found that some bands of optical images are missing, the system automatically marks and records these error messages, and tries to re-acquire them from the data source or perform preliminary repairs through the data repair algorithm. Then the format conversion is performed to uniformly convert all images into a standard format within the system for subsequent processing. Then the data is classified and stored according to key information such as sensor type, shooting time, and geographic location, and an efficient data index is established, so that when specific areas or specific types of data are needed later, they can be quickly retrieved and called, greatly improving the efficiency and convenience of data management.

[0069] For optical images, the non-local mean filtering (NLM) algorithm is used. For a pixel point x, its filtered pixel value NL[u](x) is calculated by the following formula:

[0070]

[0071] Where u(y) is the original pixel value of pixel y, N(x) is the neighborhood of pixel x, w(x,y) is the weight function used to measure the similarity between pixels x and y, and C(x) is the normalization constant to ensure that the sum of the weights is 1.

[0072] Then, the residual noise is further processed by variational partial differential equation (PDE) method. Consider the classic anisotropic diffusion equation:

[0073]

[0074] Among them, u(x,y,t) is the pixel value of the image at the (x,y) position and time, and c(x,y,t) is the diffusion coefficient, which is adjusted according to the gradient information of the image. In areas such as forest edges and rivers, the noise is smoothed by dynamically adjusting the parameters to avoid edge blurring caused by over-smoothing, which significantly improves the signal-to-noise ratio of the image and provides a clear image foundation for subsequent processing.

[0075] Based on the radiation transfer model and combined with the radiation calibration parameters of the optical sensor, the optical image is initially corrected for radiation. The general form of the radiation transfer equation is:

[0076] L λ =L 0λτλ +L pλ

[0077] Where L λ is the radiation brightness received by the sensor, L 0λ is the true radiance of the object, τ λ is the atmospheric transmittance, L pλ It is atmospheric path radiation. By accurately measuring parameters such as water vapor, aerosols, and the ozone layer in the atmosphere, calculating atmospheric transmittance and path radiation, converting the pixel values ​​of the image into the apparent reflectivity of the ground objects, and preliminarily correcting the radiation errors caused by atmospheric scattering and absorption,

[0078] Using ground-measured spectral data (several representative sample plots were selected in the forest, and the spectral reflectance of different tree species at different growth stages was measured using a spectrometer) and high-resolution reference optical images of the same period, the trained deep neural network model further optimized the radiation correction results and read the high-precision digital elevation model (DEM) data of the forestry area and the ground control points (GCPs) obtained through field measurements.

[0079] For optical images, we first use polynomial transformation to perform preliminary geometric correction, and use a quadratic polynomial model. For image coordinates (x, y) and their corresponding geographic coordinates (X, Y), we have the following relationship:

[0080] X=a 0 +a1 x+a 2 y+a 3 xy+a 4 x 2 +a 5 y 2

[0081] Y=b 0 +b 1 x+b 2 y+b 3 xy+b 4 x 2 +b 5 y 2

[0082] By selecting obvious feature points in the forest, such as mountain tops, road intersections, and boundaries of large open areas, as control points, the coefficients of the above polynomial are solved using the least squares method to correct basic geometric distortions such as translation, rotation, and scaling of the image, so that it is preliminarily aligned with the actual geographic coordinate system.

[0083] Next, terrain correction is performed by integrating DEM data based on the elastic deformation model. The elastic deformation model can be expressed as an energy minimization problem. The energy function E is usually composed of the internal energy term E int and the external energy term E ext composition:

[0084] E=E int +E ext

[0085] The internal energy term is used to maintain the smoothness of the image, and a regularization term based on the image gradient can be used:

[0086] E int =∫ Ω (a▽u 2 +β▽ 2 u 2 )dΩ

[0087] where u is the deformed image, a and β are weight parameters, ▽ and ▽ 2 are the gradient and Laplace operators, respectively, and Ω is the image area. The external energy term is defined based on the matching relationship between the DEM data and the image feature points, so that the objects in the image can match the actual terrain height, while keeping the morphology and texture features of the trees undistorted, achieving high-fidelity geometric correction and terrain correction effects, and providing an accurate geometric basis for subsequent feature extraction and image fusion, so that the objects in the image can truly reflect their position and shape relationship in the actual geographic space.

[0088] Step 2: Construct a deep convolutional neural network (CNN) that fuses optical and radar images. For optical images, the front end of the network uses 3x3 and 7x7 convolution kernels to extract the detailed texture of the image and the overall distribution characteristics of forest vegetation, respectively. For example, the 3x3 convolution kernel can capture the subtle texture of leaves and the direction of branches, while the 7x7 convolution kernel is used to extract the crown shape of large areas of forests and the overall contour characteristics of forest stands. These features are of great significance for tree species identification and forest structure analysis;

[0089] For radar images, specially designed convolution layers are used to extract scattering and polarization features. By analyzing the scattering mechanism of radar echoes, structural information such as tree trunks and branches is extracted. Polarization features are used to determine the growth direction of trees and the density of forest stands. For tall and straight trees, radar images can reflect the position and height of their trunks through strong scattering signals, and polarization features can help determine their growth direction.

[0090] At the same time, through the attention mechanism, the network automatically focuses on the key areas in forestry resources. The introduced cross-modal connection layer fuses and interacts the features of optical and radar images at an early stage, enabling the network to learn the complementary information between the two modal images.

[0091] Step 3: Construct a graph model from the feature points extracted from optical and radar images. Use the feature descriptors obtained by deep learning to calculate the initial similarity between feature points and construct an initial graph. Then, use the graph cut algorithm to segment the graph. During the segmentation process, the spatial neighborhood relationship and local geometric consistency of the feature points are fully considered. In the forest road network, the feature points on the road are accurately matched and grouped according to the continuity and directional consistency of the road; for the boundaries of forest blocks in the forest, the regularity of their geometric shapes is considered to ensure that the matching of feature points conforms to the boundary characteristics of the actual objects;

[0092] At the same time, an iterative optimization mechanism is introduced to continuously update the edge weights and vertex attributes of the graph. In each iteration, the similarity between feature points is recalculated, the edge weights of the graph are adjusted, and the attributes of the vertices (such as position and feature descriptors) are updated according to the current matching results and geometric constraints. After multiple iterations, the number of mismatched points is effectively reduced, accurate feature matching is achieved, and reliable feature correspondence is provided for the subsequent construction of the image transformation model, which improves the accuracy and stability of image transformation.

[0093] Step 4: Use the collinear equation model to accurately calculate the geometric transformation parameters of the image based on the geometric principles of optical imaging, and correct the basic geometric distortions such as rotation and translation caused by different shooting angles in the optical image, ensuring the physical rationality and geometric accuracy of the image transformation. At the same time, the GAN generator captures the complex nonlinear transformation features in the image by learning a large amount of forestry area image pair data. The GAN generator can learn these complex transformation patterns and generate transformed images similar to the target image. The discriminator is responsible for determining whether the input image is a real target image or a fake image generated by the generator. Through adversarial training between the generator and the discriminator, the generation ability of the generator is continuously improved;

[0094] During the image transformation process, the local feature changes and transformation errors of the image are monitored in real time, and the parameters and structure of the model are dynamically adjusted based on the feedback information. The model parameters are continuously updated through online learning to enable it to adapt to the transformation needs of heterogeneous remote sensing images of different types and scenarios, thereby achieving continuous improvement in the self-optimization and adaptive capabilities of the model, thereby improving the precision and accuracy of subsequent image transformations and ensuring that the system can handle complex and changeable forestry remote sensing image data.

[0095] Step 5: Adopting the semantic-based region segmentation method, the fully convolutional neural network (FCN) is used to segment the transformed image into different land feature categories, such as vegetation areas of different tree species, water bodies (rivers, lakes, etc.), bare land and buildings (forest stations, watchtowers, etc.). By training on a large-scale forestry remote sensing image dataset, the network can accurately identify and segment different land feature areas, providing a basis for subsequent precise fusion, and adopting different fusion rules and weight allocation strategies for different areas;

[0096] For vegetation areas, a multi-scale fusion method based on wavelet transform is used. In the high-frequency sub-band of wavelet decomposition, wavelet coefficients with higher energy are selected for fusion according to the texture characteristics of leaves and the shape of tree crowns, highlighting the edges and detail information of vegetation, which is helpful for monitoring the pest and disease situation and growth trend of trees; in the low-frequency sub-band, the low-frequency information of optical and radar images is weighted average fused in combination with factors such as image clarity and contrast, making the fused vegetation area smoother and more natural, while retaining the terrain and tree structure information in the radar image, which is conducive to evaluating the forest volume and ecological health status;

[0097] For water areas, a fusion method based on spectral angle matching is used. By calculating the angle difference between the source image and the target image in the spectral space, the pixel value of the source image with the smallest spectral angle with the target image is selected for fusion, ensuring that the fused water area can accurately retain its spectral characteristics, which is conducive to monitoring the area changes and water quality of the water body, and is of great significance for water resource management in forest ecosystems;

[0098] For building areas, a fusion method based on feature point matching is used to combine the color and texture features of buildings in optical images with the structural features in radar images, making the outline of buildings clearer and easier to identify and locate, which is helpful for the management of forestry resources and the planning of protection facilities. Through this refined regional segmentation and multi-scale fusion strategy, the information integrity and accuracy of the fused image are improved, so that the fused image can more comprehensively and accurately reflect the actual situation of the forestry area, providing richer and more accurate information for forestry resource monitoring.

[0099] Step six: introduce the image quality evaluation indicators SSIM and VIF based on the human visual perception model to evaluate the quality of the fused image.

[0100] In dense forest areas, the SSIM value of the image may be low due to the occlusion of trees and the scattering of light, indicating that the structural information of the image has been lost. By learning a large amount of high-quality forestry scene image data, the network automatically adjusts parameters to enhance the structural information of the image, and adopts super-resolution reconstruction technology based on convolutional neural networks to increase the number of pixels in the image and improve the correlation between pixels, thereby improving the clarity and detail expression of the image;

[0101] When the VIF value is low in complex terrain areas such as mountainous forests, it indicates that the fidelity of the image's visual information is insufficient. The image enhancement network adjusts image parameters such as color balance, contrast, and brightness to make the image's colors more vivid and natural, while enhancing the image's sense of layering and three-dimensionality, making the ups and downs of the mountains and the direction of the valley more prominent, thereby meeting the forestry professionals' needs for analyzing forest terrain and resource distribution and improving the image's visual effects and readability.

[0102] After the above series of steps, the final heterogeneous remote sensing image transformation results are presented to users in an intuitive and visual way through the result display and interaction module. Users can choose a variety of display modes, such as true color display, false color synthesis display, etc., to better observe the distribution and growth of forest vegetation.

[0103] Embodiment 2:

[0104] By collecting multispectral remote sensing images (which can obtain spectral information of multiple bands but have low spatial resolution) and high-resolution panchromatic remote sensing images (which have high spatial resolution but single spectral information) of a forestry area, and importing them into the system, they are pre-processed by data screening and denoising, and classified and archived according to shooting time, etc., and absolute radiation correction and atmospheric correction are performed on multispectral images in turn, and feature-based image registration and affine transformation geometric correction are used for panchromatic images.

[0105] Then, a feature extraction model based on decision trees was constructed to extract features such as vegetation types from multispectral images and edge contour features from panchromatic images, and the PCA fusion method was used to achieve spatial resolution enhancement and spectral information fusion of multispectral images. Then, images from different periods were collected to produce a time series image set, and a template matching algorithm was used in combination with context information to perform sliding window search and change area verification and correction, completing change detection based on template matching. After that, the lidar data and the corrected multispectral and panchromatic remote sensing images were combined, and CHM and DTM were extracted to calculate tree parameters through point cloud filtering and other processing.

[0106] The texture information is mapped onto the three-dimensional model to achieve three-dimensional forest structure reconstruction. Finally, VR and AR technologies are used to visualize and interactively display the forestry resource monitoring results, allowing users to immersively browse the three-dimensional forest scene or obtain real-time auxiliary information through AR, thereby improving the efficiency and scientificity of forestry resource management.

[0107] Embodiment three:

[0108] Firstly, thermal infrared remote sensing images (which can reflect temperature but have low resolution and are greatly affected by ambient temperature) and visible light remote sensing images (which show color texture but are invalid under low light) of a forestry area are obtained and transmitted to the processing system. Then, temperature calibration and histogram equalization are performed on the thermal infrared images, and color and gamma correction is performed on the visible light images. Then, the images are divided into blocks according to administrative divisions and a distributed file system is established. The thermal infrared and visible light images are segmented respectively using methods based on region growing and threshold.

[0109] Then, an MLP neural network model was constructed, and the temperature features of the thermal infrared image and the color and texture features of the visible light image were used as input for feature fusion and classification. A large amount of labeled sample data was used for training through the back propagation algorithm, and the dropout technology was introduced to prevent overfitting. After that, a spatiotemporal database of forestry resources was established, and the dynamic change rules were mined through spatiotemporal autocorrelation analysis. For forest fire monitoring, the real-time temperature data of thermal infrared and the spatiotemporal distribution information of historical fires were combined, and a fire risk prediction model was constructed using the random forest algorithm. Then, the forestry resource monitoring results were combined with GIS to construct an intelligent decision support system, and decision models such as forest resource planning were developed. The optimization algorithm was used to provide scientific and reasonable proposals.

[0110] Finally, a mobile application for forestry resource monitoring is developed, which has functions such as image acquisition, data query, map navigation, and early warning push, so as to realize real-time acquisition, analysis and display of forestry monitoring data and improve the timeliness and flexibility of forestry resource monitoring and management.

[0111] It should be noted that in the comparative examples, when processing heterogeneous remote sensing images in forestry resource monitoring, the conventional image change method is adopted in comparative example 1, and a simple linear transformation, a geometric correction of fixed parameters and a conventional image fusion algorithm are adopted. In comparative example 2, the radiation transfer model and the high-resolution DEM are mainly used to perform radiation and terrain correction of optical images, and the radar image is corrected based on the external DEM and phase unwrapping, and is compared with examples 1, 2 and 3, as shown in Table 1:

[0112] Table 1: Comparison of heterogeneous remote sensing image processing

[0113]

[0114] As shown in Table 1, Example 1 shows advantages in many key aspects. In terms of change detection accuracy, compared with other examples and comparative examples, it can more accurately monitor the dynamic changes of forest resources, provide a reliable basis for the formulation of forest management strategies, such as more accurately identifying forest growth, felling, vegetation changes caused by pests and diseases, and changes in land use types, etc., to help forestry departments to grasp the real-time dynamics of forest resources in a timely manner and achieve sustainable development and effective protection of forest resources.

[0115] Its decision-making support capabilities are also outstanding. The constructed model has higher reliability and accuracy than other comparison ratios when analyzing forestry resource data and providing decision-making recommendations. For example, in forest resource planning, fire prevention and fighting decisions, and pest and disease control strategy formulation, it can provide decision makers with more scientific and reasonable solutions based on more accurate data analysis, reduce resource waste and ecological damage risks caused by decision-making errors, and improve the efficiency and benefits of forestry resource management.

[0116] In terms of the accuracy of pest and disease identification, it is significantly higher than that of other embodiments and comparison examples. It can timely and accurately detect the occurrence area, type and severity of forest pests and diseases in the early stage, buy precious time for taking targeted prevention and control measures, effectively curb the spread of pests and diseases, reduce the damage of pests and diseases to forest vegetation, protect the health and stability of forest ecosystems, maintain biodiversity, and at the same time reduce the economic losses caused by pests and diseases, and ensure the sustainable development of the forestry industry.

[0117] For forest cover area monitoring, assuming that its monitoring error is a reasonably small value, it can provide more accurate data compared to other embodiments. This is of great significance for accurately assessing the total amount of forest resources, formulating reasonable forest protection and development goals, and monitoring the long-term change trend of forest resources. It helps the forestry department to make scientific resource planning and management decisions and ensure the sustainable use of forest resources.

[0118] The estimation accuracy of vegetation biomass is higher than that of other embodiments and comparison examples, and can more accurately evaluate forest productivity and ecosystem functions, such as carbon storage estimation and forest ecosystem service value assessment. This has important reference value for understanding the role of forests in the global carbon cycle, formulating reasonable forest management measures, and participating in the carbon trading market. It also helps to conduct a deeper study of the structure and function of forest ecosystems and provide a scientific basis for ecological protection and restoration.

[0119] It should be noted that in the comparative embodiment, the number of image preprocessing methods, the advancement of feature fusion methods and the data storage management method are compared with Embodiment 1, Embodiment 2 and Comparative Examples 1 and 2, as shown in Table 2:

[0120] Table 2: Comparison of image preprocessing, feature fusion methods, and data storage management

[0121]

[0122] As shown in Table 2, Example 1 shows significant advantages in many aspects compared with other comparative examples and embodiments. In terms of image preprocessing methods, Example 1 adopts a more diverse means. In comparison, the number of preprocessing methods used in other embodiments and comparative examples is small and the types are not rich enough. This enables Example 1 to optimize the original remote sensing images more comprehensively and in-depth, and can more accurately solve various problems such as noise interference, radiation deviation, geometric distortion, and color imbalance in the images, thereby providing higher quality, clearer details, and more accurate information image data for forestry resource monitoring, laying a solid and reliable data foundation for subsequent analysis work.

[0123] In terms of feature fusion methods, Example 1 uses more advanced and efficient technical means, and its degree of advancement is higher than other examples and comparison examples. Through this advanced feature fusion method, Example 1 can more intelligently and accurately integrate various features obtained from different images or different feature extraction algorithms, thereby digging out more useful information hidden deep in the data, and significantly enhancing the ability to describe and distinguish forestry resource characteristics. In the process of identifying and distinguishing different tree species, it can comprehensively consider information in multiple dimensions such as the spectral characteristics, texture characteristics, shape characteristics, and spatial distribution characteristics of trees, thereby greatly improving the precision and accuracy of forestry resource inventory and monitoring.

[0124] Secondly, in terms of data storage management, Example 1 constructs a more efficient system, which has obvious advantages in data retrieval speed, storage redundancy control, and security and integrity assurance compared to other examples and comparison examples. In the actual application scenarios of forestry resource monitoring, a large amount of remote sensing image data is usually involved and needs to be stored for a long time and frequently called. This advantage of Example 1 is reflected in the ability to quickly respond to data query requests, ensuring that data can be acquired and analyzed in a timely manner, and at the same time effectively reducing the risk of data loss or damage caused by poor data management. This is of vital importance for long-term and continuous dynamic monitoring of forestry resources, comparative analysis of historical data, and rapid response in the face of forest emergencies (such as fires, sudden outbreaks of pests and diseases, etc.), which effectively improves the efficiency and reliability of the entire forestry resource monitoring and management work.

[0125] In summary, Example 1, by virtue of its advantages in image preprocessing methods, feature fusion methods, data storage and management methods, can provide better quality data, more accurate analysis results and more efficient workflows for forestry resource monitoring and management. It has demonstrated obvious technical advantages and broad application value in the field of forestry resource monitoring and management, and has advantages that cannot be ignored compared to other examples and comparison examples.

[0126] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A heterogeneous remote sensing image transformation method, characterized in that: The following steps are involved: S1. Image preprocessing, combining non-local mean filtering NLM, variational partial differential equations PDE, radiation transfer model, machine learning, and elastic deformation and polynomial transformation methods based on DEM and GCPs, to perform noise suppression, radiation calibration and atmospheric correction, as well as geometric correction and terrain correction on heterogeneous remote sensing images to obtain image data with high signal-to-noise ratio, consistent radiation and accurate geometry; S2. Deep multimodal feature extraction: building a deep convolutional neural network (CNN) that integrates multimodal information such as optics and radar. Through convolution kernels of different scales and specially designed feature extraction layers, combined with attention mechanism and cross-modal connection, deep features with high discriminability and stability are extracted from preprocessed images. S3, graph theory optimizes feature matching, constructs feature points into a graph model, uses deep learning feature descriptors to initialize the edge weights of the graph, and uses the graph cut algorithm combined with iterative optimization of spatial neighborhood and local geometric consistency to achieve accurate feature matching, reduce mismatched points, and provide reliable feature correspondence for image transformation; S4. Hybrid adaptive transformation model construction, integrating the geometric transformation model based on physical imaging principles and the image generation model based on deep learning. The model is trained by designing a loss function that comprehensively considers geometry, radiation consistency and visual perception quality. At the same time, a dynamic adaptive adjustment mechanism is established to optimize the model parameters and structure in real time according to the error feedback during the image transformation process to achieve high-precision image transformation. S5, image fusion enhancement, based on semantics, the image is segmented, and for different types of land objects, such as vegetation, water bodies, buildings, etc., appropriate fusion rules and weight distribution strategies are used to perform fine fusion of the transformed images to improve the information integrity and accuracy of the fused image; S6, perception-driven quality enhancement, introduces image quality evaluation indicators based on human visual perception models, such as SSIM, VIF, etc., and uses deep learning image enhancement networks to perform targeted enhancement processing on fused images according to the evaluation results, thereby improving the visual effect and readability of the image and meeting the analysis and interpretation needs of professionals.

2. A heterogeneous remote sensing image transformation method according to claim 1, characterized in that: In step S1, the NLM filter removes noise and retains detail texture based on local self-similarity of the image, and the PDE method further smoothes the residual noise. The two are combined to adaptively adjust according to the image area and noise characteristics to provide high signal-to-noise ratio image data for subsequent processing; The physical method of the radiation transfer model is combined with the data-driven machine learning method. The radiation transfer model is used to combine the sensor radiation calibration parameters for preliminary correction, and then a deep neural network model trained based on ground measurement and reference image data is used for further optimization to eliminate radiation errors and improve radiation consistency accuracy. Using high-precision digital elevation model DEM and dense ground control points GCPs, combined with elastic deformation model and polynomial transformation method, preliminary geometric correction is performed through polynomial transformation, and then terrain correction is performed by integrating DEM data based on elastic deformation model, so as to maintain the shape and texture characteristics of the objects while ensuring that the geometric position accurately matches the terrain.

3. The heterogeneous remote sensing image transformation method according to claim 1, characterized in that: In the step S2, a deep convolutional neural network (CNN) architecture that integrates multiple modal information is constructed, rich features are extracted for optical images using convolution kernels of different scales, special convolution layers are designed for radar images to extract scattering and polarization features, and key areas are focused on through an attention mechanism. A cross-modal connection layer is introduced to achieve early fusion interaction, so that feature expression and robustness are improved, and the optimal extraction strategy is learned through large-scale data training.

4. The heterogeneous remote sensing image transformation method according to claim 1, characterized in that: In step S3, the feature points are regarded as graph vertices, and similarity is used as edge weight to construct a feature point graph model. The deep learning feature descriptor is used to calculate the initial similarity to construct the graph, and then the graph cut algorithm is used to segment the matching and non-matching point sets. The spatial neighborhood and local geometric consistency are considered, and an iterative optimization mechanism is introduced to update the graph attributes.

5. The heterogeneous remote sensing image transformation method according to claim 1, characterized in that: In step S4, a hybrid model is constructed by combining a geometric transformation model based on physical imaging principles, such as an image generation model based on deep learning, wherein the physical model ensures the physical rationality of the transformation, and the deep learning model supplements and captures complex nonlinear transformation features. By designing a suitable loss function, the model is trained by considering factors such as geometry, radiation consistency, and visual perception quality, so as to achieve adaptive high-precision transformation; During image transformation, local feature changes and transformation errors are monitored in real time, and model parameters and structures are dynamically adjusted based on feedback information. For example, if the error in a certain area is large, control points are added or parameters of related layers of the deep learning model are adjusted. Parameters are updated through online learning to adapt to image transformation requirements of different types and scenes.

6. A heterogeneous remote sensing image transformation system, characterized in that: include: Multi-data access and management module: It has strong compatibility and can receive heterogeneous remote sensing image data from various satellite platforms, different sensor types and different data formats. At the same time, it can quickly check the integrity of the data, convert the format and classify the data for storage, establish an efficient data indexing and management mechanism, and facilitate subsequent data call and processing; Ultra-precision pre-processing engine module: It integrates the multi-scale noise adaptive removal sub-module, the precise radiation calibration and atmospheric correction integrated sub-module, and the high-fidelity geometric correction and terrain correction collaborative processing sub-module, performs comprehensive and high-precision pre-processing operations on the input image data, and outputs high-quality and consistent pre-processed image data; Deep feature extraction and precise matching unit module: It consists of a multimodal deep convolution feature extraction submodule and a graph-theory-based feature matching optimization submodule. It uses advanced deep learning and graph theory algorithms to extract and match image features, and passes the precise feature matching results to the adaptive high-precision image transformation model construction unit. Adaptive high-precision image transformation model construction and optimization module: Combines the hybrid transformation model submodule based on physical model and deep learning and the model's dynamic adaptive adjustment mechanism submodule to build and optimize the image transformation model. According to the input feature matching results and image data, the model parameters and structure are adaptively adjusted to achieve high-precision image transformation operations. Refined image fusion and quality enhancement unit module: It uses a strategy submodule based on region segmentation and multi-scale fusion and a perception quality-driven image enhancement processing submodule to perform refined fusion and quality enhancement processing on the transformed image, and outputs the final heterogeneous remote sensing image transformation result with high quality, rich information and good visual effects; Result display and interaction module: displays the processed image data in an intuitive and visual way, and provides rich interactive functions, such as image zooming, panning, contrast display, etc., to facilitate users to evaluate and analyze the processing results. At the same time, it supports the output of processing results into a variety of standard image file formats and data formats to meet the subsequent application needs of different users.

Images