Unmanned aerial vehicle-satellite remote sensing data fusion method based on dynamic weight distribution

Abstract

The invention relates to the technical field of artificial intelligence and remote sensing information processing, in particular to an unmanned aerial vehicle-satellite remote sensing data fusion method based on dynamic weight distribution, which comprises the following steps: step 1, dynamic feature extraction and environment perception; 2, dynamic weight distribution and fusion execution: forming a feature vector by the feature value and an environment interference coefficient, inputting the feature vector into a pre-training weight distribution model, and outputting real-time weights of the unmanned aerial vehicle and the satellite; generating an interpolation scaling factor according to the resolution ratio, and performing up-sampling on the low-resolution data by adopting edge sensing interpolation; pixel-level weighted fusion is carried out on the unmanned aerial vehicle data and the up-sampling satellite data according to the real-time weight; and step 3, performing closed-loop feedback optimization. According to the method, the accuracy, timeliness and quality of remote sensing data fusion are effectively improved in a dynamic weight distribution and environment perception mode, and the method has wide application value.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and remote sensing information processing technology, and in particular to a method for fusing UAV-satellite remote sensing data based on dynamic weight allocation. Background Art

[0002] With the development of remote sensing technology, collaborative observation by drones and satellites has become an important data acquisition method in fields such as natural disaster monitoring and precision agriculture. Drone remote sensing possesses high spatial resolution and flexible maneuverability, enabling it to capture detailed features of the Earth's surface; satellite remote sensing, on the other hand, offers wide coverage and multispectral advantages, providing extensive spectral information. By fusing these two types of data, remote sensing products with both high resolution and wide coverage can be generated, providing decision support for scenarios such as disaster emergency response and crop growth assessment.

[0003] However, existing fusion methods have significant drawbacks, especially in dynamic environments and time-sensitive scenarios:

[0004] 1. Static weight allocation mechanisms are ill-equipped to handle dynamic environmental disturbances.

[0005] Current mainstream methods employ a fixed-weight strategy, for example, in vegetation monitoring, satellite data is consistently weighted at 0.7 or higher. The flaw in this strategy lies in its inability to dynamically adjust the data source's contribution when faced with cloud cover (satellite data failure) or sudden changes in illumination (overexposure of drone data). In flood monitoring, cloud cover distorts satellite optical data, but due to its high fixed weight, it is still forcibly incorporated into the results, leading to misjudgments of the inundation area—revealing a fundamental contradiction between environmental dynamism and fixed weights.

[0006] 2. The open-loop converged architecture poses a risk of continuous quality deterioration:

[0007] Existing solutions lack a quality feedback mechanism for the fusion results. This means that in long-term tasks such as forest fire monitoring, smoke diffusion leads to continuous changes in environmental interference. Due to the lack of real-time evaluation and weight adjustment of indicators such as local contrast and information entropy of the fusion results, initial errors accumulate with iterations, ultimately causing shifts in heat source location. This inherent flaw in open-loop systems is particularly fatal in time-sensitive scenarios.

[0008] Therefore, there is an urgent need for a UAV-satellite remote sensing data fusion method based on dynamic weight allocation to solve the above problems. Summary of the Invention

[0009] To achieve the above objectives, this invention provides a method for fusing UAV-satellite remote sensing data based on dynamic weight allocation, comprising:

[0010] Step 1: Dynamic Feature Extraction and Environmental Awareness

[0011] Extracting spatial feature sharpness values ​​from UAV images: generated by calculating the percentage of pixel gradient direction consistency.

[0012] Fidelity of spectral features extracted from satellite data: generated by comparing real-time reflectance with preset standard ground object reflectance curves;

[0013] Calculate the time-phase feature fit: generated by the time difference between drone and satellite data collection and the timeliness requirements of the target scene;

[0014] Real-time data collection of atmospheric visibility and cloud cover; input of preset nonlinear functions to generate environmental interference coefficients.

[0015] Step 2: Dynamic weight allocation and fusion execution:

[0016] The above eigenvalues ​​and environmental interference coefficients are combined to form a feature vector, which is then input into the pre-trained weight allocation model to output the real-time weights of the UAV and the satellite.

[0017] An interpolation scaling factor is generated based on the resolution ratio, and edge-aware interpolation is used to upsample the low-resolution data.

[0018] Perform pixel-level weighted fusion of UAV data and upsampled satellite data according to real-time weights;

[0019] Step 3: Closed-loop feedback optimization:

[0020] Calculate the local contrast and information entropy of the fusion result;

[0021] When the mean local contrast is lower than the preset ratio of historical data and the information entropy decreases, a weight reallocation instruction is triggered.

[0022] The weight allocation model parameters are adjusted based on the ideal fusion results with manual annotation, and then immediately applied to subsequent fusions.

[0023] Preferably, the extraction of spatial feature sharpness values ​​includes:

[0024] The drone image is traversed through a sliding window of a preset size, and the gradient magnitude and direction of each pixel within the window are calculated.

[0025] The proportion of pixel pairs whose gradient direction difference is less than a preset angle threshold is used as the gradient direction consistency ratio.

[0026] Compare the gradient direction consistency ratio with the preset sharpness threshold:

[0027] When the percentage is higher than the threshold, the sharpness value increases linearly with the increase of the percentage;

[0028] When the percentage is below the threshold, the sharpness value decreases exponentially as the percentage decreases;

[0029] The sharpness values ​​of the entire image are normalized and output to the feature vector.

[0030] Preferably, the calculation of the temporal feature fit further includes:

[0031] When the target scenario is agricultural monitoring, collect crop phenological data and historical outbreak times of pests and diseases;

[0032] Construct a time-sensitive mapping function:

[0033] During critical growth periods or periods of high incidence of pests and diseases, the absolute value of the time difference is compressed to a preset proportion of the original value.

[0034] During crop dormancy, the absolute value of the time difference is magnified to a preset multiple of the original value;

[0035] The adjusted time difference is input into the S-shaped function and mapped to the fitness value in the [0,1] interval. The steepness of the mapping curve is determined by the crop sensitivity level.

[0036] Preferably, the training and adjustment of the weight allocation model includes:

[0037] Model training phase:

[0038] Historical environmental interference coefficients and feature vectors are collected, and the optimal weight combination is manually labeled.

[0039] The mean squared error between the model output weights and the optimal weights is used as the loss function;

[0040] The parameters of the fully connected layer are updated using the backpropagation algorithm, and the learning rate decreases stepwise as the number of training rounds increases.

[0041] Feedback and Adjustment Phase:

[0042] When a weight reallocation instruction is triggered, the parameters related to the environmental disturbance coefficients of the model input layer are frozen.

[0043] Only the hidden layer parameters corresponding to the fidelity of spectral features and the fit of temporal features are disclosed;

[0044] The model parameter update amount is calculated using the decrease in information entropy as the proportional coefficient and the local contrast deviation as the direction factor.

[0045] Preferably, edge-aware interpolation includes:

[0046] Detecting edge pixels in the image to be interpolated:

[0047] Calculate the grayscale variance of the eight neighborhoods of a pixel, and mark it as an edge point when the variance is greater than the preset edge detection threshold.

[0048] Dynamically constructing interpolation kernel functions:

[0049] For edge points, expand the kernel function coverage along the edge tangent direction to the preset maximum size;

[0050] Compress the kernel function to a preset minimum size along the edge normal direction;

[0051] Non-edge points are assigned isotropic kernel functions, and their size is linearly determined by the interpolation scaling factor.

[0052] When performing interpolation, texture features of homogeneous regions are matched first to avoid sampling across edge lines.

[0053] Preferably, obtaining the ideal fusion result through manual annotation includes:

[0054] Visible light band benchmark:

[0055] Acquire airborne hyperspectral data of the same region and downscale it to the target resolution;

[0056] Pixels with spectral angles smaller than a preset similarity threshold are selected as ground truth.

[0057] Infrared band reference:

[0058] Acquire SAR images from the same period and extract water / heat source boundaries;

[0059] Regions where the temperature difference between the inside and outside of the boundary is greater than a preset significance threshold are taken as true values;

[0060] True values ​​are used to calculate the deviation between the model's output weights and the ideal weights.

[0061] Preferably, the weighting rules for the fire monitoring scenario include:

[0062] When the environmental interference coefficient is greater than the preset high interference threshold and the satellite data contains valid infrared bands:

[0063] Initiate infrared band priority detection: Calculate the information entropy ratio of the infrared and visible light bands in the smoke-covered area;

[0064] If the entropy ratio is greater than the preset infrared dominance threshold, then reset the weight allocation model output:

[0065] Set the satellite infrared data weight to the product of a preset base value and an environmental interference coefficient;

[0066] Set the weight of the visible light data from the drone to a preset base value and an inverse function of cloud coverage;

[0067] After the weights are reset, the closed-loop feedback optimization is skipped and the fusion result is output directly.

[0068] Preferably, the calculation of local contrast includes:

[0069] The fused image is divided into several square sub-blocks of preset size;

[0070] For each sub-block:

[0071] Calculate the difference between the maximum and minimum pixel grayscale values;

[0072] Remove sub-blocks whose difference is less than a preset noise threshold;

[0073] Calculate the weighted average of the differences between the remaining sub-blocks:

[0074] The weight is determined by the information entropy of the sub-block; the higher the entropy, the greater the weight.

[0075] Compare the weighted average with the sliding window mean of historical data.

[0076] Preferably, the optimization of the water body boundary of the flood inundation area includes:

[0077] Extract the coordinate set of water body edges from SAR images and locate the corresponding pixels in the fused image;

[0078] Calculate the gradient magnitude and direction consistency of the localized pixels:

[0079] Construct a detection window centered on the coordinate point;

[0080] The percentage of pixels within the statistical window whose gradient direction and the SAR edge tangent direction are at an angle smaller than a preset tolerance angle;

[0081] When the proportion is lower than the preset confidence threshold:

[0082] Increase the weighting factor of the local contrast in the feedback command for this region;

[0083] In the weighted allocation model, add decision weights based on the sharpness value of spatial features.

[0084] Preferably, the calculation of the model parameter update includes:

[0085] Define the information entropy decrease factor as the ratio of the current information entropy to the information entropy of the previous satisfactory fusion result;

[0086] Define the local contrast deviation factor as the proportion of the difference between the current local contrast and the historical mean to the historical range.

[0087] The formula for calculating the model parameter update is:

[0088] Model parameter update amount = learning rate × entropy decrease factor × contrast bias factor × gradient sign; the gradient sign is determined by the direction of the difference between the model output weights and the ideal weights;

[0089] The learning rate decreases dynamically based on the number of model iterations.

[0090] The initial learning rate is set to the preset maximum value;

[0091] After each preset number of adjustments, the learning rate decreases in a geometric progression.

[0092] The beneficial effects of this invention are:

[0093] 1. This invention utilizes dynamic feature extraction and environmental perception technologies to extract environmental interference coefficients in real time and input them into a pre-trained weight allocation model. The model dynamically adjusts the weights according to environmental changes, ensuring that the contributions of satellite and UAV data change in a timely manner when cloud cover or sudden changes in lighting occur, thus avoiding error accumulation caused by fixed weights.

[0094] 2. This invention employs a quality feedback mechanism during the fusion process to monitor the local contrast and information entropy of the fusion result in real time. When the average local contrast falls below a preset ratio and the information entropy decreases, a weight reallocation instruction is triggered to readjust the weights of the data source. This mechanism ensures that environmental changes and interference are handled promptly in long-term tasks, preventing the accumulation of errors.

[0095] 3. This invention achieves targeted weight allocation by introducing specific weight enforcement rules in specific monitoring scenarios. For example, in fire monitoring, when the environmental interference coefficient is high and satellite infrared data is valid, satellite infrared data is used first; while when cloud cover has a significant impact, visible light data from drones is relied upon first. This mechanism avoids fusion errors caused by different data source characteristics.

[0096] 4. This invention effectively improves the fusion quality of remote sensing data at different resolutions through high-precision feature extraction and edge-aware interpolation methods. By dynamically adjusting weights and combining them with edge-aware interpolation technology, data from different sources can be accurately fused, avoiding image distortion or information loss and ensuring the high quality and stability of the fused product.

[0097] 5. The closed-loop feedback optimization mechanism of this invention ensures that each fusion result can be adjusted according to the current environmental conditions and data quality, thereby avoiding the continuous accumulation of errors in long-term tasks. Attached Figure Description

[0098] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0099] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0100] Figure 2 This is a flowchart illustrating the steps involved in extracting spatial feature sharpness values ​​using the method of the present invention.

[0101] Figure 3 This is a flowchart illustrating the steps involved in training and adjusting the weight allocation model of the present invention. Detailed Implementation

[0102] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0103] Please see Figures 1-3 This invention provides a UAV-satellite remote sensing data fusion method based on dynamic weight allocation. In step 1, when extracting the spatial feature sharpness value of the UAV image, the gradient direction consistency ratio of each pixel in the image is first calculated. Gradient direction consistency reflects the sharpness and edge clarity of different regions in the image, and is usually achieved by comparing the gray value changes of adjacent pixels. Regions with high sharpness values ​​usually contain more ground feature information, meaning that the data quality of this region is higher, and should be given higher weight in the weight allocation.

[0104] The spectral fidelity of satellite imagery is extracted by comparing real-time reflectance with preset standard ground feature reflectance curves. Each ground feature (such as water bodies, buildings, and vegetation) has its specific spectral reflectance characteristics, and reflectance deviations may vary due to factors such as atmosphere, clouds, and sensor errors. By comparing with standard ground feature reflectance curves, the spectral fidelity of satellite data, i.e., the accuracy of reflectance, can be assessed, thereby determining the quality of the satellite data.

[0105] Temporal feature fit is generated by calculating the time difference between the UAV and satellite imagery, as well as the timeliness requirements of the target scene. Timeliness requirements reflect the mission's need for timely data; for example, in disaster monitoring, real-time data is more important. Therefore, the smaller the time difference, the higher the fit, and UAV data with strong real-time requirements may receive higher weight in this case.

[0106] Atmospheric visibility and cloud cover directly impact the quality of remote sensing imagery. By collecting these environmental parameters in real time and inputting them into a preset nonlinear function, an environmental interference coefficient is generated. This coefficient quantifies the degree of interference of environmental factors on image quality, thereby influencing the weight of the data source.

[0107] In step 2, the extracted feature values ​​and environmental interference coefficients are combined into a feature vector, which is then input into a pre-trained weight allocation model. The model calculates the weights of the UAV and satellite data in real time based on the weights of the feature values ​​and the interference coefficients. At this point, the weights of the data sources are dynamically adjusted to allocate appropriate contributions based on data quality and environmental changes.

[0108] Based on the resolution ratio between the UAV and satellite imagery, an interpolation scaling factor is calculated, and an edge-aware interpolation method is used to upsample the low-resolution satellite data. Edge-aware interpolation can preserve the edge details of the image and minimize information loss caused by resolution differences.

[0109] In pixel-level weighted fusion, weights calculated in real time are applied to each pixel, fusing UAV data with upsampled satellite data. After weighted fusion, the value of each pixel is calculated based on the contributions of the two data sources according to their weights, thereby fusing information from different sources and improving the accuracy and detail of the fusion result.

[0110] In step 3, local contrast and information entropy of the fusion result are two important indicators for measuring the fusion effect. Local contrast reflects the richness of image details, while information entropy reflects the amount of information in the image. When the mean local contrast is lower than a preset ratio and information entropy decreases, it means that there may be a problem with the fusion effect, and the system will trigger a weight reallocation instruction.

[0111] When the system detects a decline in the quality of the fusion result, it triggers a weight reallocation, readjusting the contribution of UAV and satellite data. Furthermore, by comparing the results with manually labeled, ideal fusion results, the parameters of the weight allocation model are adjusted. Through this closed-loop optimization mechanism, the model can continuously improve and adapt to the data fusion needs of different scenarios.

[0112] As can be understood, spatial feature sharpness refers to the clarity and detail of pixels in an image, reflecting the edge sharpness of the image. Spectral feature fidelity refers to the degree of match between the reflectance of satellite imagery and the reflectance curve of a preset standard ground feature, measuring the spectral accuracy of satellite data. Temporal feature adaptability: Calculates the timeliness adaptability of data based on the data acquisition time difference and mission timeliness requirements. Environmental interference coefficient: Quantifies the degree of influence of environmental factors such as atmosphere and clouds on the quality of remote sensing data.

[0113] In one possible implementation, to extract spatial features from the image, a sliding window of a preset size is first set and gradually slid across the UAV image. During each slide, the gradient magnitude and direction of each pixel within the window are calculated. These gradient values ​​reflect the edge information and sharpness of the image in that region. Pixels with larger gradient magnitudes indicate sharper edges in the image, while the gradient direction indicates the directional information of the edges.

[0114] For each pixel within a sliding window, the difference between its gradient direction and that of other pixels is calculated. If the difference in gradient directions between two pixels is less than a preset angle threshold, the two pixels are considered to be in the same edge region, and their gradient directions are consistent. The number of all pixel pairs with consistent gradient directions is counted and compared with the total number of pixel pairs to obtain the gradient direction consistency ratio. This ratio reflects the continuity and sharpness of edge regions in the image.

[0115] The calculated gradient direction consistency ratio is compared with the preset sharpness threshold:

[0116] When the proportion is higher than the threshold, it means that the edge areas in the image are relatively clear, and the sharpness value increases linearly with the increase of the consistency proportion. At this time, the image region has high detail and is suitable for being assigned higher weight for fusion.

[0117] When the proportion is below the threshold, it indicates that the edge areas in the image are relatively blurry, and the sharpness value decreases exponentially as the consistency proportion decreases. At this time, there is less detail in the image area, and a lower weight should be assigned.

[0118] The sharpness values ​​of the entire image are normalized to ensure that the sharpness values ​​of different images can be compared and fused within the same range. After normalization, the sharpness values ​​are output as feature vectors for use in subsequent dynamic weight allocation and data fusion processes.

[0119] By accurately extracting sharpness values ​​and dynamically adjusting weights, the performance of UAV imagery in remote sensing data fusion is significantly improved, resulting in more accurate fusion results that effectively utilize image details and optimize fusion effects.

[0120] In one possible implementation, in agricultural monitoring applications, crop phenological data (such as sowing date, heading date, and harvest date) and historical outbreak times of pests and diseases are first collected. This data reflects the crop's growth status and the temporal patterns of pest and disease occurrence, helping to analyze the crop's growth process and the extent of damage.

[0121] Based on different stages of crop growth and peak periods of pests and diseases, a timeliness requirement mapping function is designed. The core function of this function is to adjust the fit of the crop growth stage according to the time difference, specifically in the following two cases:

[0122] During critical growth periods or periods of high incidence of pests and diseases in crops: Crops have high requirements for the timeliness of remote sensing data during these periods. Therefore, the time difference between data collection and actual observation needs to be compressed to a predetermined proportion of the original value. For example, if the current remote sensing data differs significantly from the actual observation time, the absolute value of the time difference can be compressed to highlight the importance of recent data and enhance the timeliness of monitoring results.

[0123] Crop dormancy period: Crops have lower requirements for timeliness during dormancy. Therefore, the time difference of remote sensing data needs to be magnified to a preset multiple of the original value. At this time, remote sensing data with a longer time difference can still be used for monitoring, and the magnified time difference value makes dormant data have a lower weight in the fusion.

[0124] The adjusted time difference (including compression and amplification) is input into a sigmoid function, which maps the time difference to a fitness value in the interval [0,1]. The fitness value output by the sigmoid function reflects the validity and timeliness of the data at that point in time. A fitness value close to 0 indicates low timeliness of the remote sensing data, making it unsuitable for fusion; while a fitness value close to 1 indicates high timeliness of the data, making it suitable for fusion.

[0125] The steepness of the S-curve is determined by the crop's sensitivity level. Crops with higher sensitivity levels have stricter timeliness requirements, resulting in steeper mapping curves and faster changes in data fit. Crops with lower sensitivity levels have flatter mapping curves and relatively less stringent timeliness requirements. This method allows for adjusting the remote sensing data fit calculation based on crop type and growth cycle, ensuring appropriate temporal difference weights for the crop.

[0126] By combining crop phenological data and historical outbreak data of pests and diseases, the timeliness of remote sensing data can be optimized, the accuracy and flexibility of agricultural monitoring can be enhanced, and the overall effect of data fusion can be improved, thereby providing more valuable support for agricultural production.

[0127] In one possible implementation, historical environmental data is first collected during the initial stage of model training, especially environmental interference coefficients (such as meteorological factors, terrain effects, etc.) and feature vectors related to the remote sensing data. This data can describe the degree of interference to the remote sensing data, providing a basis for subsequent model training.

[0128] Based on the analysis of historical data, the optimal weight combination for each feature vector is determined through manual annotation. The optimal weights are determined by comparing them with actual observation data to ensure that, under specific environmental conditions, the remote sensing data fusion results can approximate the actual situation as closely as possible.

[0129] During model training, the mean squared error (MSE) between the model's output weights and the optimal weights is used as the loss function. MSE effectively measures the difference between predicted and actual values, optimizing the model's output to make the predicted weights as close as possible to the actual optimal weights, thereby improving the model's accuracy.

[0130] Backpropagation is used to update the model's parameters. By calculating the error at each layer, backpropagation gradually adjusts the model's parameters to reduce the loss function. During training, the learning rate decreases stepwise with each training epoch, allowing the model to converge quickly in the early stages and fine-tune more precisely in later stages, avoiding oscillations and overfitting.

[0131] When a weight reallocation instruction is triggered, the model enters a feedback adjustment phase. At this point, input layer parameters related to environmental interference coefficients are frozen to ensure they no longer directly affect the model input during subsequent adjustments. This step prevents environmental factors from excessively influencing the model's output, allowing the adjustment process to focus more on other features.

[0132] During the feedback adjustment phase, only hidden layer parameters related to spectral feature fidelity and temporal feature fit are allowed to be adjusted. This is because these features directly affect the fusion effect of remote sensing data, and adjusting these parameters helps optimize the accuracy of data fusion. In this way, the model can more accurately adapt to different remote sensing data inputs and environmental changes.

[0133] During the adjustment process, the rate of decrease in information entropy is used as a proportional coefficient, and the local contrast deviation is used as a directional factor to calculate the amount of model parameter updates. The rate of decrease in information entropy reflects the reduction in the amount of information in the system, while the local contrast deviation indicates the trend of data change within a specific region. By combining these two factors, the model can be adjusted in a targeted manner to ensure improved accuracy during the data fusion process.

[0134] Through precise training and feedback adjustment mechanisms, the accuracy and timeliness of data fusion are significantly improved, exhibiting strong adaptability and effectively addressing changing environments and complex monitoring needs.

[0135] In one possible implementation, for each pixel in the image to be interpolated, the gray-level variance of its eight neighboring pixels is first calculated. The gray-level variance reflects the gray-level changes in the area surrounding the pixel; if the gray-level changes in this area are large, it indicates that the area may be located at the edge of the image.

[0136] A pixel is considered an edge pixel when the variance of its eight-neighbor grayscale values ​​exceeds a preset edge detection threshold. This is based on the characteristics of edge regions, where grayscale changes are more drastic and typically correspond to structural boundaries of the image. This method can efficiently and accurately detect edge pixels in an image.

[0137] For pixels marked as edge pixels, a dynamically adjusted kernel function is used during interpolation. The kernel function's coverage is expanded to a preset maximum size along the tangent direction of the edge. This strategy ensures interpolation accuracy in edge regions by utilizing as much surrounding pixel information as possible without crossing the edge.

[0138] At the same time, the kernel function is compressed to a preset minimum size along the normal direction of the edge. This process prevents the interpolation function from crossing the edge line, avoids mixing pixel information inside and outside the edge, and thus reduces artifacts or distortions that may occur during interpolation.

[0139] For non-edge points, an isotropic kernel function is used, the size of which is linearly determined by the interpolation scaling factor. This ensures that the kernel size remains uniform in regions without clear edges, thus guaranteeing smooth image transitions and natural interpolation.

[0140] When performing interpolation, priority is given to matching texture features in homogeneous regions, avoiding sampling across edge lines. This means that during interpolation, regions with similar grayscale values ​​to the current pixel are preferentially selected for interpolation to maintain the natural texture and transition effects of the image. At the same time, interpolation across edges is avoided in the image's edge regions to prevent visual artifacts and unnatural transitions introduced by sampling across edge lines.

[0141] By accurately detecting edges and dynamically adjusting the interpolation kernel function, the interpolation accuracy of the image is not only effectively improved, but also artifacts and blurring during the interpolation process are prevented, thereby enhancing image quality and natural transition effects.

[0142] In one possible implementation, firstly, airborne hyperspectral data of the same region as the remotely sensed image is acquired. This hyperspectral data provides rich spectral information, thus supporting the ground truth calibration of the target region. Because airborne hyperspectral imagery has high resolution and wide band coverage, it can provide a relatively accurate reference for pixel-level fusion results.

[0143] The acquired airborne hyperspectral data is downscaled to the same resolution as the target remote sensing image. This downscaling operation ensures spatial consistency between the airborne hyperspectral data and the target image, providing a foundation for subsequent pixel matching. This step avoids scale differences between high-resolution and low-resolution data, ensuring the accuracy of the fusion result.

[0144] During the matching process between hyperspectral data and the target image, the spectral angle of each pixel is calculated. If the spectral angle of a pixel is less than a preset similarity threshold, the pixel is considered to have high similarity and can be used as the true value for the ideal fusion result. This step utilizes the concept of spectral angle, calculates the spectral differences between different pixels in the image, and selects pixels with high similarity for calibration, thereby ensuring that the spectral features of the selected pixels highly match the real scene.

[0145] Acquiring synthetic aperture radar (SAR) imagery concurrent with visible light band imagery provides structural information about ground surfaces, offering significant advantages, particularly in the detection of water bodies and heat sources. In this context, SAR imagery provides auxiliary data support for acquiring infrared band references.

[0146] Extracting the boundaries of water bodies or heat sources from SAR images. SAR images are highly sensitive to water bodies and heat source regions, and can effectively extract the boundaries of these characteristic regions as reference areas for ideal fusion results.

[0147] After extracting the boundaries of water bodies or heat sources, the temperature difference between the inside and outside of the boundaries is further analyzed. When the temperature difference exceeds a preset significance threshold, the area is considered the true value of the ideal fusion result. This step determines the true boundaries of ground features through the significance of the temperature difference, ensuring the accuracy of the calibration results.

[0148] Ground truth data is used to calculate the deviation between the model's output weights and the ideal weights. By comparing the model-generated results with manually calibrated ground truth values, the difference between the two can be calculated. This deviation provides a reference for subsequent model optimization, further guiding weight adjustments to achieve higher accuracy.

[0149] This method ensures that the fusion results are more accurate and reliable during the dynamic weighting of UAV-satellite remote sensing data fusion, and can effectively adapt to changes in different ground features and environmental conditions.

[0150] In one possible implementation, the environmental interference coefficient of the current fire monitoring area is first obtained. If this coefficient is greater than a preset high interference threshold, it indicates that the environment of the monitoring area is subject to strong interference, such as atmospheric conditions, clouds, or other meteorological factors. In this case, further weight adjustments to the data are needed to address the impact of interference and ensure the accuracy of the results.

[0151] Determine if the satellite data contains valid infrared bands. Infrared bands are crucial for fire monitoring because smoke and heat from fires are typically more visible in the infrared spectrum. If the satellite data contains infrared bands, proceed to the next step; otherwise, it may be necessary to rely on other sensor data or adjust the weighting strategy.

[0152] After confirming that the satellite data contains valid infrared bands, the information entropy ratio of the infrared to the visible light bands in the smoke-covered area is calculated. Information entropy measures the uncertainty or complexity of data. In fire monitoring, the infrared bands in the smoke-covered area typically provide more accurate information about the fire source and smoke. If the information entropy ratio of the infrared bands is greater than a preset infrared dominance threshold, it indicates that the infrared bands contribute more to the monitoring results, and infrared data should be given priority.

[0153] Based on the entropy ratio, if the infrared band has a significant advantage, the output of the weight allocation model is reset. The specific steps are as follows:

[0154] The weight of satellite infrared data is set as the product of a preset base value and an environmental interference coefficient. A higher environmental interference coefficient indicates stronger external interference, therefore the weight of the infrared band needs to be appropriately increased to strengthen its dominant role in the monitoring process.

[0155] The weight of the UAV visible light data is set as a function inversely proportional to a preset base value and cloud coverage. When there is a lot of cloud cover, the effectiveness of the ground visible light band decreases, so the weight of the UAV visible light data should be appropriately reduced to minimize its impact on the fusion result.

[0156] After weight resetting, the high environmental interference coefficient may make further closed-loop feedback optimization unsuitable. Therefore, in this case, the adjusted fusion result is directly output to avoid further processing delays and ensure timely and effective fire monitoring.

[0157] This method can dynamically adjust the weight allocation of data fusion according to different fire monitoring environments and interference conditions, ultimately achieving more accurate and efficient fire monitoring, and has great application potential.

[0158] In one possible implementation, the fused remote sensing image is first divided into several square sub-blocks according to a preset size rule. This segmentation step localizes the entire image, allowing for precise analysis of details in each region when calculating local contrast, unaffected by global information. The preset size can be flexibly adjusted according to different application scenarios; typically, smaller regions are chosen to ensure fine-grained analysis.

[0159] For each sub-block, the difference between the maximum and minimum grayscale values ​​of its pixels is calculated; this difference is the local contrast of that sub-block. Local contrast reflects the degree of texture variation within that area; the larger the grayscale difference, the richer the detail information in that area. This calculation allows for the emphasis of areas with significant detail variations in the image.

[0160] For each calculated sub-block, if the grayscale difference is less than a preset noise threshold, the sub-block is considered to be heavily affected by noise and unable to provide effective contrast information; therefore, these sub-blocks are discarded. This step helps remove noise interference with the final result and avoids meaningless contrast information affecting the accuracy of data fusion.

[0161] For the remaining valid sub-blocks, a weighted average of their grayscale differences is calculated. The weights of the weighted average are determined by the information entropy of each sub-block. Information entropy is an indicator of the amount of information and complexity in an image; the higher the entropy value, the richer the information contained in the region, and therefore it is given greater weight in the weighted average. This processing method allows regions with high contrast and rich information to have a larger proportion in the final calculation, thereby enhancing the influence of important regions in the image.

[0162] Finally, the weighted average is compared with the sliding window mean of historical data. The sliding window mean is based on the average of several past frames of data and is used to smooth out data fluctuations and noise, providing a stable reference value. By comparing it with the historical mean, it can be determined whether there are significant changes in the local contrast of the current image. If the change is large, it indicates that the monitored target (such as a fire or other emergency) in the current area may have changed significantly.

[0163] Through the steps described above, this method can perform refined processing of local areas in images under different monitoring scenarios, thereby improving the effectiveness and fusion effect of remote sensing data and ensuring more accurate and real-time monitoring tasks. This local contrast calculation method has extremely high application value, especially in fields such as fire and environmental monitoring.

[0164] In one possible implementation, continuous water body edge lines are extracted from SAR imagery using edge detection algorithms (such as Canny or Sobel), and the set of edge coordinate points is recorded. Because SAR has strong penetrating power and can effectively identify water body boundaries, its extraction results exhibit high spatial accuracy and temporal stability.

[0165] For each SAR edge point, its corresponding pixel is precisely located in the fused image based on the pixel geographic location mapping relationship. This localization ensures that subsequent analysis is performed in the same spatial location, guaranteeing the accuracy of data alignment.

[0166] In the fused image, a fixed-size detection window (e.g., 5×5 pixels) is constructed centered on each edge coordinate point. The gradient direction of all pixels within this window is calculated; in this embodiment, the Scharr operator is used to extract the gradient.

[0167] The gradient direction of each pixel is compared with the tangent direction of the SAR edge line at that point. If the angle between the two is less than a preset tolerance angle (e.g., 15 degrees), the directions are considered to be consistent. The proportion of pixels with consistent directions within the entire window is then calculated.

[0168] If the proportion of directional consistency is lower than the preset confidence threshold (e.g., 70%), it indicates that the boundary of the region is blurred or discontinuous in the fused image, and feedback adjustment is required.

[0169] Increase the weighting factor of the local contrast in this region in the fusion feedback command to enhance the intensity of the contrast enhancement operation in this region;

[0170] Meanwhile, during the weight allocation process, a decision weight for spatial feature sharpness value is added to this region, which enhances the role of image gradient and edge sharpness in the subsequent fusion process.

[0171] By introducing gradient direction consistency analysis, setting a dynamic weight adjustment mechanism, and strengthening the role of spatial features in boundary identification, the accuracy of water body boundary identification in flood monitoring has been effectively improved, the intelligence and adaptability of remote sensing data fusion have been enhanced, and solid remote sensing support has been provided for disaster prevention, mitigation and emergency response.

[0172] In one possible implementation, information entropy is used to measure the complexity or uncertainty of an image. In each fusion iteration, the ratio of the current information entropy to the entropy value of the satisfactory result after the previous fusion is calculated. If the entropy of the current fusion result is lower than the previous one, it indicates that the structure and details of the image have become clearer. Therefore, the information entropy decrease factor can reflect the degree of optimization of the current fusion, as shown in the formula:

[0173]

[0174] When the value of this factor is greater than 1, it indicates that the fusion result has improved compared to the previous one; if it is less than 1, it indicates that the fusion effect has not met expectations.

[0175] Local contrast reflects the local details and brightness differences in an image. During the fusion process, the contrast of each local region should be maintained within a stable range. The local contrast deviation factor is calculated by comparing the difference between the current local contrast and the historical mean with the historical range to determine the degree of contrast deviation in that region. The formula is:

[0176]

[0177] If this factor is close to 0, it indicates that the current contrast is consistent with the historical average, meaning that local details are not excessively magnified or reduced.

[0178] The model parameter update amount is dynamically adjusted by comprehensively considering the information entropy decrease factor, the local contrast deviation factor, and the gradient sign. The gradient sign is determined by the direction of the difference between the model output weights and the ideal weights. The formula for calculating the update amount is:

[0179] Model parameter update amount = learning rate × entropy decrease factor × contrast bias factor × gradient sign;

[0180] This update adjusts the model's weights in each iteration, ensuring that the fusion process moves towards the optimal result.

[0181] The learning rate controls the magnitude of each model parameter update. An excessively large learning rate may cause oscillations, while an excessively small learning rate may lead to slow convergence. To ensure that the model can converge faster and adapt to different image features, the learning rate is decreased after each iteration according to a preset number of iterations. The initial learning rate is set to the maximum value, and as the number of iterations increases, the learning rate decreases in a geometric progression, ensuring the stability and efficiency of the model during the convergence process.

[0182] By combining information entropy, contrast bias, gradient sign, and dynamic learning rate, the model parameters are precisely adjusted, achieving high-precision and stable remote sensing data fusion, which greatly improves the effect of remote sensing image analysis.

[0183] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0184] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for fusing UAV-satellite remote sensing data based on dynamic weight allocation, characterized in that, include: Step 1: Dynamic Feature Extraction and Environmental Awareness Extracting spatial feature sharpness values ​​from UAV images: generated by calculating the percentage of pixel gradient direction consistency. Fidelity of spectral features extracted from satellite data: generated by comparing real-time reflectance with preset standard ground object reflectance curves; Calculate the time-phase feature fit: generated by the time difference between drone and satellite data collection and the timeliness requirements of the target scene; Real-time data collection of atmospheric visibility and cloud cover; input of preset nonlinear functions to generate environmental interference coefficients. Step 2: Dynamic weight allocation and fusion execution: The above eigenvalues ​​and environmental interference coefficients are combined to form a feature vector, which is then input into the pre-trained weight allocation model to output the real-time weights of the UAV and the satellite. An interpolation scaling factor is generated based on the resolution ratio, and edge-aware interpolation is used to upsample the low-resolution data. Perform pixel-level weighted fusion of UAV data and upsampled satellite data according to real-time weights; Step 3: Closed-loop feedback optimization: Calculate the local contrast and information entropy of the fusion result; When the mean local contrast is lower than the preset ratio of historical data and the information entropy decreases, a weight reallocation instruction is triggered. The weight allocation model parameters are adjusted based on the ideal fusion results with manual annotation, and then immediately applied to subsequent fusions.

2. The UAV-satellite remote sensing data fusion method based on dynamic weight allocation according to claim 1, characterized in that, The extraction of spatial feature sharpness values ​​includes: The drone image is traversed through a sliding window of a preset size, and the gradient magnitude and direction of each pixel within the window are calculated. The proportion of pixel pairs whose gradient direction difference is less than a preset angle threshold is used as the gradient direction consistency ratio. Compare the gradient direction consistency ratio with the preset sharpness threshold: When the percentage is higher than the threshold, the sharpness value increases linearly with the increase of the percentage; When the percentage is below the threshold, the sharpness value decreases exponentially as the percentage decreases; The sharpness values ​​of the entire image are normalized and output to the feature vector.

3. The UAV-satellite remote sensing data fusion method based on dynamic weight allocation according to claim 1, characterized in that, The calculation of temporal feature fit further includes: When the target scenario is agricultural monitoring, collect crop phenological data and historical outbreak times of pests and diseases; Construct a time-sensitive mapping function: During critical growth periods or periods of high incidence of pests and diseases, the absolute value of the time difference is compressed to a preset proportion of the original value. During crop dormancy, the absolute value of the time difference is magnified to a preset multiple of the original value; The adjusted time difference is input into the S-shaped function and mapped to the fitness value in the [0,1] interval. The steepness of the mapping curve is determined by the crop sensitivity level.

4. The UAV-satellite remote sensing data fusion method based on dynamic weight allocation according to claim 1, characterized in that, The training and tuning of the weight allocation model includes: Model training phase: Historical environmental interference coefficients and feature vectors are collected, and the optimal weight combination is manually labeled. The mean squared error between the model output weights and the optimal weights is used as the loss function; The parameters of the fully connected layer are updated using the backpropagation algorithm, and the learning rate decreases stepwise as the number of training rounds increases. Feedback and Adjustment Phase: When a weight reallocation instruction is triggered, the parameters related to the environmental disturbance coefficients of the model input layer are frozen. Only the hidden layer parameters corresponding to the fidelity of spectral features and the fit of temporal features are disclosed; The model parameter update amount is calculated using the decrease in information entropy as the proportional coefficient and the local contrast deviation as the direction factor.

5. The UAV-satellite remote sensing data fusion method based on dynamic weight allocation according to claim 1, characterized in that, Edge-aware interpolation includes: Detecting edge pixels in the image to be interpolated: Calculate the grayscale variance of the eight neighborhoods of a pixel, and mark it as an edge point when the variance is greater than the preset edge detection threshold. Dynamically constructing interpolation kernel functions: For edge points, expand the kernel function coverage along the edge tangent direction to the preset maximum size; Compress the kernel function to a preset minimum size along the edge normal direction; Non-edge points are assigned isotropic kernel functions, and their size is linearly determined by the interpolation scaling factor. When performing interpolation, texture features of homogeneous regions are matched first to avoid sampling across edge lines.

6. The UAV-satellite remote sensing data fusion method based on dynamic weight allocation according to claim 1, characterized in that, Obtaining ideal fusion results through manual annotation includes: Visible light band reference: Acquire airborne hyperspectral data of the same region and downscale it to the target resolution; Pixels with spectral angles smaller than a preset similarity threshold are selected as ground truth. Infrared band reference: Acquire SAR images from the same period and extract water / heat source boundaries; Regions where the temperature difference between the inside and outside of the boundary is greater than a preset significance threshold are taken as true values; True values ​​are used to calculate the deviation between the model's output weights and the ideal weights.

7. The UAV-satellite remote sensing data fusion method based on dynamic weight allocation according to claim 1, characterized in that, The weighting rules for fire monitoring scenarios include: When the environmental interference coefficient is greater than the preset high interference threshold and the satellite data contains valid infrared bands: Initiate infrared band priority detection: Calculate the information entropy ratio of the infrared and visible light bands in the smoke-covered area; If the entropy ratio is greater than the preset infrared dominance threshold, then reset the weight allocation model output: Set the satellite infrared data weight to the product of a preset base value and an environmental interference coefficient; Set the weight of the visible light data from the drone to a preset base value and an inverse function of cloud coverage; After the weights are reset, the closed-loop feedback optimization is skipped and the fusion result is output directly.

8. The UAV-satellite remote sensing data fusion method based on dynamic weight allocation according to claim 1, characterized in that, The calculation of local contrast includes: The fused image is divided into several square sub-blocks of preset size; For each sub-block: Calculate the difference between the maximum and minimum pixel grayscale values; Remove sub-blocks whose difference is less than a preset noise threshold; Calculate the weighted average of the differences between the remaining sub-blocks: The weight is determined by the information entropy of the sub-block; the higher the entropy, the greater the weight. Compare the weighted average with the sliding window mean of historical data.

9. A method for fusion of UAV-satellite remote sensing data based on dynamic weight allocation according to claim 6, characterized in that, Optimization of water body boundaries in flood-prone areas includes: Extract the coordinate set of water body edges from SAR images and locate the corresponding pixels in the fused image; Calculate the gradient magnitude and direction consistency of the localized pixels: Construct a detection window centered on the coordinate point; The percentage of pixels within the statistical window whose gradient direction and the SAR edge tangent direction are at an angle smaller than a preset tolerance angle; When the proportion is lower than the preset confidence threshold: Increase the weighting factor of the local contrast in the feedback command for this region; In the weighted allocation model, add decision weights based on the sharpness value of spatial features.

10. The UAV-satellite remote sensing data fusion method based on dynamic weight allocation according to claim 4, characterized in that, The calculation of model parameter update includes: Define the information entropy decrease factor as the ratio of the current information entropy to the information entropy of the previous satisfactory fusion result; Define the local contrast deviation factor as the proportion of the difference between the current local contrast and the historical mean to the historical range. The formula for calculating the model parameter update is: Model parameter update amount = learning rate × entropy decrease factor × contrast bias factor × gradient sign; the gradient sign is determined by the direction of the difference between the model output weights and the ideal weights; The learning rate decreases dynamically based on the number of model iterations. The initial learning rate is set to the preset maximum value; After each preset number of adjustments, the learning rate decreases in a geometric progression.

Images