Remote sensing image fishpond extraction method of adaptive edge enhanced neural network

Abstract

The invention discloses a remote sensing image fishpond extraction method based on an adaptive edge enhanced neural network. The method comprises the following steps: selecting a sentinel No.2 satellite multispectral band to synthesize a false color image; constructing a fishpond region labeling data set; establishing an adaptive edge enhanced neural network architecture, wherein the architecture fuses an edge detection module capable of learning a threshold and semantic feature extraction; an edge perception double attention module and a pyramid pooling module are integrated, and the multi-scale feature representation capability is enhanced; optimizing a training process by adopting a multi-level depth supervision and prediction fusion strategy; and finally, extracting a fishpond area through the training model and generating a standardized result. According to the method, the fishpond boundary segmentation precision is remarkably improved, the adaptability to fishponds with different scales is enhanced, the detection rate of small-scale fishponds is particularly improved, the model convergence process is accelerated through the depth supervision strategy, misclassification of similar water bodies such as rivers and ditches is effectively reduced, and the accuracy of fishpond boundary segmentation is improved. The technical problems that in the prior art, edges are fuzzy, multi-scale adaptability is poor, complex background interference is sensitive, and the omission ratio of a small-scale fishpond is high are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing and computer vision, and specifically relates to a method for extracting fish ponds from remote sensing images based on an adaptive edge enhancement network. Background Art

[0002] Remote sensing image interpretation is an important means of obtaining geographical information, and its core lies in the collaborative utilization of the spectral and spatial characteristics of ground objects. Traditional fish pond extraction methods mainly rely on manual field surveys, which have problems such as long cycles and high costs. Although methods based on remote sensing technology can achieve large-scale monitoring, they are limited by insufficient feature expression and spatial modeling capabilities. In the prior art, the pixel-based spectral segmentation method, the Normalized Difference Water Index (NDWI), can quickly distinguish water areas, but it is difficult to capture the regular geometric features of fish ponds and is easily interfered by adjacent ground objects, resulting in boundary adhesion. Traditional machine learning models (such as random forests) can improve accuracy by artificially designing morphological features, but their generalization ability is limited in complex scenarios.

[0003] In recent years, deep learning methods such as the Fully Convolutional Network (FCN) and U-Net have made remarkable progress in remote sensing image segmentation, but they still have technical limitations: First, conventional convolution operations are insufficient in modeling long-range spatial dependencies and are difficult to effectively associate the regular features of the row and column distributions of fish ponds, resulting in edge breaks or over-smoothing. Second, the problem of insufficient multi-scale feature fusion makes the model less adaptable to the mixed distribution scenarios of dense small ponds and scattered large ponds. Especially in the complex aquaculture areas along the coast of China, the misjudgment rate of interfering ground objects such as rivers and ditches is relatively high.

[0004] In view of the limitations of the above traditional methods and deep learning methods in fishpond extraction, the present invention proposes a fusion architecture based on adaptive edge detection with parameterizable thresholds and hierarchical spatial attention. The present invention achieves a technological breakthrough through a learnable edge-enhanced U-Net architecture: by introducing a Canny detection module with learnable high and low threshold parameters and edge-enhanced convolutions, the edge intensity of the image is adaptively optimized, and dynamic feature fusion is performed with the semantic features of the U-Net backbone through a gated fusion mechanism, significantly improving the accuracy of fishpond boundary recognition; combining the channel and spatial dual attention mechanisms of the Edge-aware Dual Attention Module (EDAM) and Pyramid Scene Parsing (PSP), a multi-scale attention feature field is constructed to synchronously enhance the recognition ability of dense small fishponds and isolated large fishponds, greatly improving the recall rate of small-scale fishponds (<50㎡); based on a deep supervision strategy to optimize network training, the overall accuracy is significantly improved compared to U-Net on Sentinel-2 multispectral data (blue, green, red, near-infrared bands), successfully solving the core technical problems such as blurred fishpond boundaries, scale mixing, and complex background interference in high-resolution images. Summary of the Invention

[0005] Aiming at the technical problems existing in the existing methods for extracting fishponds from remote sensing images, such as blurred edges, poor multi-scale adaptability, sensitivity to complex background interference, and high omission rates of small-scale fishponds, the present invention proposes a method for extracting fishponds from remote sensing images based on an adaptive edge enhancement neural network. By fusing a learnable edge detection module and a hierarchical spatial attention mechanism, combined with the construction of a multi-scale feature field and a deep supervision strategy, the accuracy of fishpond boundary segmentation and the robustness in complex scenarios are significantly improved.

[0006] 1. To achieve the above object of the present invention, the following technical solutions are adopted: construct a method for extracting fishponds from remote sensing images based on an adaptive edge enhancement neural network, specifically including the following steps: S1: Multi-spectral band selection and image synthesis: Select the blue ( , 490 nm), green ( , 560 nm), red ( , 665 nm), and near-infrared ( , 842 nm) bands with a spatial resolution of 10 meters in Sentinel-2 satellite images, combine the green, red, and near-infrared bands into a false-color image (band order: ), enhance the spectral difference between water bodies and vegetation through this combination, and use histogram equalization technology to improve the image contrast and generate a three-channel basic image of the input data S2: Fish pond area annotation and dataset construction: On the synthetic false color image, polygon annotation of the fish pond area is performed based on manual visual interpretation to generate binary mask labels (1 is fish pond category , 0 is non-fish pond area ); Crop the fish pond area data into sample blocks of 256×256 pixels , construct a training dataset containing typical fish pond characteristics D , and normalize the pixel values to the [0,1] interval; at the same time, apply data enhancement methods such as random rotation and flipping Solve the sample imbalance problem and ensure that the model learns the morphological characteristics of various fish ponds in a balanced manner.

[0007] S3: Adaptive edge-enhanced neural network architecture model construction: Building an adaptive edge-enhanced network based on the U-Net architecture , introduce an edge detection module with learnable thresholds at the encoder stage , to achieve adaptive extraction of edge features of the input image; the edge detection module uses learnable parameters and Optimize the high and low thresholds of the Canny operator and combine it with the edge enhancement convolution block Improve the expressiveness of edge features and extract edge features Through the edge feature fusion module and semantic features Perform multi-level fusion to enhance the network's ability to perceive target boundaries.

[0008] S4: Neural Network Attention Mechanism Integration: Introducing Edge-aware Dual Attention Modules in Network Encoding and Decoding Stages , realize adaptive enhancement of key features; integrate pyramid pooling module at the network bottleneck layer , through multi-scale feature extraction Enhance the model's ability to represent objects of different sizes; the edge feature fusion module adopts a gating mechanism , through learnable weights Dynamically adjust the impact of edge information on semantic features to improve the recognition accuracy of water boundaries and small targets such as fish ponds.

[0009] S5: Deep supervision optimization training strategy The network adopts a multi-level deep supervision mechanism at different stages of the upsampling decoder. Setting auxiliary supervision signal , corresponding to feature representations of different scales; introduce learnable prediction fusion weight parameters , for the main output With deep supervision prediction at all levels Perform adaptive fusion to form the final segmentation result; alleviate the problem of gradient disappearance through the deep supervision strategy, accelerate the model convergence process, enhance the network's learning ability for different-scale features, and improve the overall segmentation accuracy; S6: Based on S3 - S5, construct a model based on the self-adaptive edge enhancement neural network architecture; S7: Input the preprocessed remote sensing image dataset into the model for training: Use BCE-Dice the combined loss function to combine the main branch and the auxiliary supervision branch, where is BCE the loss weight, and Adam is the Dice loss weight; use the optimizer to perform end-to-end training with the initial learning rate , and the learning rate adopts the cosine annealing strategy . In each training epoch , use samples with a batch size of B = 16 for iterative optimization, and save the weight parameters with the highest validation set accuracy after training is completed.

[0010] S8: Obtaining the remote sensing extraction result of fish ponds: Input the preprocessed image to be extracted into the trained model to output the pixel-level fish pond probability map . After being processed by the adaptive threshold , generate a binary mask , and the final result is in the standardized raster format.

[0011] In one embodiment, S3 also includes the specific implementation of the edge detection module, and its expression can be represented as:

[0012] Among them, represents the Canny edge detection operation applying the learnable threshold parameter, and are the low threshold and high threshold parameters respectively, and are optimized through backpropagation; the constraint mechanism of the learnable threshold is expressed as:

[0013]

[0014] Among them , , , is the threshold interval parameter, and the gradient calculation for edge detection uses the Sobel operator, and the expression is:

[0015] wherein and are the Sobel kernels in the horizontal and vertical directions respectively, represents the convolution operation; is the edge enhancement convolution block, which contains cascaded convolution layers, and the expression is:

[0016] wherein represents the residual connection; the edge feature fusion module adopts a gated attention mechanism, and the expression is:

[0017]

[0018] wherein is the attention gating unit, is the learnable fusion weight, represents the feature concatenation operation, represents the element-wise multiplication.

[0019] In one embodiment, the channel and spatial dual attention mechanism in step S4 The expression can be represented as:

[0020] The calculation expression of the channel attention module is:

[0021]

[0022] wherein, the MLP structure is , is the reduction ratio; is the activation function, represents the element-wise multiplication of the channel dimension.

[0023] The calculation expression of the edge perception module is:

[0024]

[0025]

[0026] wherein represents the edge feature, represents the feature concatenation operation, represents the edge weight matrix.

[0027] The calculation expression of the spatial attention module is:

[0028]

[0029] where represents concatenation in the channel dimension, is a two-dimensional convolution of

[0030] Pyramid pooling module Through multi-scale feature extraction, the expression is:

[0031]

[0032] where represents pooling operations with different ratios, and the corresponding output feature size ratio is .

[0033] In one embodiment, the expression of the deep supervision mechanism in step S5 can be represented as:

[0034] where represents the feature map of the th layer decoder, is the activation function; the feature representations of different levels can be expressed as:

[0035] where represents the upsampling operation, represents the skip connection feature corresponding to the encoder stage; the final prediction fusion adopts an adaptive weight combination, and the expression is:

[0036] where the weight parameter satisfies the constraint condition , and is optimized through end-to-end training; the normalization expression of the prediction fusion is:

[0037] where represents the main output, represents the prediction result of the auxiliary supervision branch.

[0038] In one embodiment, the training optimization expression of step S7 can be represented as:

[0039] Among them, the combined loss function includes the main branch loss and the auxiliary supervision branch loss, and the weight coefficients are , and ; BCE The loss and Dice The loss is defined as:

[0040]

[0041] where is the total number of pixels, is the ground truth label, is the predicted probability, is the smoothing factor, which is used to improve numerical stability; The loss function of the auxiliary supervision branch is defined as:

[0042] Among them, = = 0.5 is the weight coefficient, is the prediction of the th layer of the auxiliary branch.

[0043] The Adam optimizer is used for parameter update, where = 0.9, , are the hyperparameters of the optimizer, and the learning rate adopts the cosine annealing strategy:

[0044] where is the initial learning rate, is the minimum learning rate, is the current iteration number, is the total number of iterations.

[0045] In one embodiment, the expression of the fishpond extraction result in step S8 can be expressed as:

[0046] where represents the trained model, are the parameters with the optimal performance on the validation set; the expression for generating the binary mask is:

[0047] where is the indicator function, is the optimal threshold; the expression for determining the adaptive threshold is:

[0048]

[0049] Among them represents the balance factor of precision and recall; The beneficial effects of the present invention are as follows: 1. By introducing an edge detection module with a learnable threshold and an edge feature fusion mechanism, the segmentation accuracy of the fishpond boundary is significantly improved, effectively solving the technical problem of blurred water body boundaries in complex remote sensing scenarios, making the extracted fishpond boundary more accurate and continuous.

[0050] 2. Integrating the edge-aware dual attention module and the pyramid pooling module to construct a multi-scale feature field, which enhances the adaptability of the model to fishponds of different sizes, especially improves the detection recall rate of small-scale fishponds, and overcomes the limitations of traditional methods in multi-scale target recognition.

[0051] 3. Adopting a multi-level deep supervision and prediction fusion strategy, which accelerates the convergence process of the model, improves the training stability, and at the same time enhances the robustness of the network to complex background environments by adaptively fusing prediction results of different scales, effectively reducing the misclassification of similar water bodies such as rivers and ditches, and improving the overall accuracy of fishpond extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is the system architecture diagram of the fishpond extraction method from remote sensing images of the adaptive edge enhancement neural network.

[0053] Figure 2 is the structure diagram of the adaptive edge detector.

[0054] Figure 3 is the structure diagram of the edge-aware dual attention module.

[0055] Figure 4 is the experimental result comparison diagram of the embodiments of the present invention.

[0056] Figure 5 is the overview diagram of the Sentinel-2 image of the study area. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] The present invention will be described in detail below with reference to the drawings and specific embodiments.

[0058] Embodiment 1 In this embodiment, an adaptive edge enhancement neural network is used to extract fishponds from Sentinel-2 remote sensing images to achieve high-precision fishpond boundary recognition.

[0059] This embodiment selects the Sentinel-2 image data of Jiujiang Town, Danzao Town, and Xiqiao Town in Nanhai District, Foshan City (112°58′15″–113°07′21″E, 22°55′29″–23°05′08″N) (e.g. Figure 5 As shown in Figure 1, the climate in this area is suitable for aquaculture, with an average annual temperature of about 22°C, abundant rainfall, and distinct four seasons, which are conducive to the growth of fish. The water source in the town is mainly provided by surface water and groundwater. These abundant water resources make the area an ideal place for fish farming. Considering the possible impact of the spatial resolution of different bands on water body analysis, four bands with a spatial resolution of 10 meters - blue, green, red and near-red light bands - are used to improve the analysis accuracy and reduce the data deviation that may be caused by resampling.

[0060] The experimental design adopts a cross-validation method, and a total of 3 rounds of experiments are performed. This embodiment is illustrated by using the data set described in Table 1: Table 1

[0061] S1: Multispectral band selection and image synthesis: This embodiment obtains multispectral images of the study area of Nanhai District, Foshan City from the Sentinel-2 satellite, and selects the blue ( , 490 nm), green ( , 560 nm), red ( , 665 nm), near infrared ( , 842 nm). The near infrared-red-green band combination ( ) Synthetic false color image I This combination visually presents water bodies such as fish ponds as dark blue or black, and vegetation as red, enhancing the contrast between water and land boundaries. Taking the central area of Jiujiang Town as an example, the difference between the fish pond and the surrounding objects is not obvious in the original RGB true color image, but after false color synthesis, the spectral characteristics of the fish pond are more prominent. Subsequently, this embodiment applies histogram equalization processing to the false color image , further improving the image contrast, making the fish pond boundary clearer, and generating the final three-channel basic input image .

[0062] S2: Sample annotation and dataset construction: Based on the false-color composite images, this study annotated a total of 3,856 aquaculture ponds (with areas ranging from 30 to 5,000 square meters) in Danzao Town, Jiujiang Town, and Xiqiao Town of Nanhai District, Foshan City. The annotation results were converted into binary mask labels (the pond area is 1, and the background is 0). The entire-region images were regularly cropped into 256×256 pixels, and 5,420 effective sample blocks were obtained after excluding samples with incomplete boundaries. All samples were normalized (pixel values were mapped to the interval [0,1]) S3: In this embodiment, an adaptive edge enhancement network based on the U-Net architecture was constructed , as Figure 1 shown. In the encoder stage, this embodiment introduced an edge detection module with learnable thresholds , and extracted edge features by applying an improved Canny operator to the input image , where and are the low and high threshold parameters that can be learned by the model, as Figure 2 shown. In the initial state, is set to 0.1, is set to 0.3, and during the training process, these parameters will be continuously optimized through the backpropagation algorithm. For example, in the images of the dense pond area in Xiqiao Town, there are a large number of breaks and noises in the edges extracted by the traditional fixed-threshold Canny operator, while the learnable threshold module of this method optimizes the parameters after training to significantly improve the continuity and accuracy of the edges. In addition, this method also integrates an edge enhancement convolutional block , which uses cascaded 3×3 convolutions and batch normalization layers to enhance the expressive ability of edge features . Finally, through the edge feature fusion module , the edge features are dynamically fused with the semantic features of each layer of the encoder to achieve precise enhancement of the pond boundaries.

[0063] S4: In the encoding and decoding stages of the network, this embodiment integrates an edge-aware dual attention module , as Figure 3As shown. This module contains two sub-modules: channel attention and spatial attention, which focus on the two dimensions of "what to focus on" and "where to focus on" respectively. In addition, the edge perception mechanism enhances the model's ability to accurately depict the contour of the fish pond through edge feature extraction and adaptive weight allocation. Taking a complex area in Jiujiang Town as an example, the channel attention mechanism adaptively highlights the feature channels most helpful for fish pond recognition, such as edge gray gradient and water body texture features; the spatial attention mechanism emphasizes the spatial distribution features of the fish pond boundary and internal area. The collaboration of channel attention, spatial attention, and edge perception enables the model to accurately distinguish the fish pond from surrounding ground objects such as farmland and roads. At the network bottleneck layer, this embodiment integrates a pyramid pooling module , and obtains multi-scale context information through pooling operations of different ratios (1×1, 2×2, 3×3, 6×6).

[0064] S5: To improve the network training efficiency and segmentation accuracy, this embodiment adopts a multi-level deep supervision mechanism, such as Figure 3 shown. At different stages of the decoder l ∈{1,2,3}, auxiliary supervision signals are set to generate prediction outputs , where represents the feature map fused through upsampling and skip connections. This embodiment introduces learnable prediction fusion weight parameters { } , and adaptively fuses the main output with the predictions of each level of deep supervision. The expression is . The weights are ensured to be reasonably distributed through normalization processing. This mechanism significantly accelerates the model convergence and improves the recognition ability of small-scale fish ponds.

[0065] S6: According to the design of S3 - S5, this embodiment constructs a complete adaptive edge enhancement neural network model, such as Figure 1 shown. This model integrates an edge detection and enhancement module, a dual attention mechanism, a pyramid pooling module, and a deep supervision mechanism to form an end-to-end fish pond extraction framework.

[0066] S7: This embodiment uses BCE-Dice a joint loss function for model training, where BCE is the loss weight , the Dice loss weight , and the weight of the auxiliary supervision branch . The training process uses Adam an optimizer, with an initial learning rate , and a weight decay coefficient of . The learning rate strategy adopts the cosine annealing method and gradually decreases to during 150 epochs of training.。Each iteration uses a batch size B = 16 samples for optimization, and monitors the mean Intersection over Union (IoU) metric of the validation set during training.

[0067] S8: Input the preprocessed image to be extracted into the trained model to output the pixel-level fishpond probability map 。To obtain the final binary mask result, this embodiment uses an adaptive threshold method to determine the optimal segmentation threshold 。Specifically, on the validation set, with the F1 score as the evaluation metric, different thresholds (step size 0.01) are tried, and the threshold corresponding to the highest F1 score is selected as the optimal threshold 。In the test area of Jiujiang Town, the optimal threshold is determined to be to generate a binary mask 。Compared with the traditional U-Net, this method has achieved significant accuracy improvement in all three test areas, especially in the accuracy of fishpond boundaries and the recognition of small-scale fishponds.

[0068] Experimental results: This embodiment adopts a three-round cross-validation experimental design to fully verify the adaptability and generalization ability of the algorithm of this method (Adaptive Edge-detection U-Net, AEU-Net) in different geographical environments. In each round of experiments, data from two regions are selected for model training, and independent tests are carried out in the third region to ensure the complete separation of the test data and the training data in terms of geographical distribution.

[0069] Table 2-5 respectively shows the experimental results of the three-round cross-validation. In each round of experiments, AEU-Net outperforms SegNet, U-Net, Attention U-Net, U-Net++, and U-Net3+ in all cases. In the first round of experiments (Table 2), using the data from Danzao Town and Xiqiao Town for training and Jiujiang Town as the test area, the F1-score of AEU-Net reaches 0.903 and the IoU reaches 0.823, significantly higher than other comparison methods. Similarly, in the second round (Table 3) and the third round (Table 4) of experiments, AEU-Net also performs excellently in the tests of different geographical regions. Especially in the test of Xiqiao Town, the F1-score reaches 0.918 and the IoU reaches 0.843, proving the excellent generalization ability of the algorithm. Table 5 further summarizes the average performance of the three rounds of experiments. AEU-Net achieves the best results in all evaluation metrics, with the average F1-score reaching 0.905, the average IoU reaching 0.825, the overall accuracy (OA) reaching 0.968, and the MCC reaching 0.885, as Figure 4As shown in the comparison chart of segmentation results, this method demonstrates remarkable anti-interference ability, effectively avoiding the mis-segmentation of water interference objects (rivers, ditches), and verifying the stability and effectiveness of this algorithm in different regional environments.

[0070] Table 2

[0071] Table 3

[0072] Table 4

[0073] Table 5

Claims

1. A method for extracting fish ponds from remote sensing images using an adaptive edge enhancement neural network, characterized in that, It includes the following steps: S1: Select the blue, green, red, and near-infrared bands with a spatial resolution of 10 meters in Sentinel-2 satellite images. Combine the green, red, and near-infrared bands into a false-color image. Improve the image contrast through histogram equalization technology to generate a three-channel basic image of the input data; S2: On the synthesized false-color image, perform polygon annotation on the fishpond area based on manual visual interpretation to generate a binary mask label; Crop the fishpond area data into sample blocks of 256×256 pixels, construct a training dataset containing typical fishpond features, and normalize the pixel values to the interval [0,1]; S3: Build an adaptive edge enhancement network based on the U-Net architecture. Introduce an edge detection module with a learnable threshold in the encoder stage to achieve adaptive extraction of the edge features of the input image; The edge detection module uses the high and low thresholds of the learnable parameters and the optimized Canny operator, combines the edge enhancement convolutional block to improve the expression ability of the edge features, and fuses the extracted edge features with the semantic features through the edge feature fusion module at multiple levels; S4: Introduce an edge-aware dual attention module in the network encoding and decoding stages to achieve adaptive enhancement of key features; Integrate a pyramid pooling module in the network bottleneck layer to enhance the model's representation ability for targets of different sizes through multi-scale feature extraction; S5: The network adopts a multi-level deep supervision mechanism, sets auxiliary supervision signals at different stages of the upsampling decoder, corresponding to feature representations of different scales; Introduce learnable prediction fusion weight parameters to adaptively fuse the main output and the predictions of each level of deep supervision to form the final segmentation result; S6: Input the preprocessed remote sensing image dataset into the model for training, and adopt the BCE-Dice combined loss function to combine the main branch and the auxiliary supervision branch, where α = 0.7 is the BCE loss weight, β=0.3 is the Dice loss weight; use the Adam optimizer to perform end-to-end training with the initial learning rate, and the learning rate adopts the cosine annealing strategy; S7: Input the preprocessed image to be extracted into the trained model, output a pixel-level fishpond probability map, generate a binary mask after adaptive threshold processing, and the final result is in the standardized raster format.

2. The method for extracting fish ponds from remote sensing images of an adaptive edge enhancement neural network according to claim 1, characterized in that, The edge detection module in S3 includes a combination of Canny edge detection operations with learnable thresholds and edge enhancement convolutional blocks, where the high and low threshold parameters τ L and τ H are optimized through backpropagation, and τ H satisfy the constraint mechanism of being greater than τ L . The gradient calculation for edge detection uses the Sobel operator.

3. A method for extracting fish ponds from remote sensing images using an adaptive edge enhancement neural network according to claim 1, characterized in that, The edge enhancement convolutional block includes a cascaded convolutional layer and a residual connection structure; The edge feature fusion module adopts a gated attention mechanism to dynamically adjust the influence degree of edge information on semantic features through learnable weights.

4. The method for extracting fish ponds from remote sensing images of an adaptive edge enhancement neural network according to claim 1, wherein, The dual attention mechanism in S4 includes the cascade of a channel attention module and a spatial attention module. The channel attention module is based on the feature aggregation of average pooling and max pooling, the spatial attention module is based on the feature aggregation of the channel dimension, and the edge-aware module enhances the network's ability to identify the fishpond boundary by extracting edge features and generating an adaptive weight matrix.

5. The method for extracting fish ponds from remote sensing images of an adaptive edge enhancement neural network according to claim 1, characterized in that, The pyramid pooling module extracts multi-scale features through pooling operations with different ratios, and the corresponding output feature size ratios are {1, 1 / 2, 1 / 4, 1 / 8}.

6. The method for extracting fish ponds from remote sensing images of an adaptive edge enhancement neural network according to claim 1, characterized in that, The deep supervision mechanism in S5 adds auxiliary prediction branches at each decoder layer to generate supervision signals through convolutional operations and activation functions; The feature representations at different levels include upsampling operations and encoder skip connections; The final prediction fusion adopts an adaptive weight combination, and the weight parameters are optimized through end-to-end training and satisfy the normalization constraint.

Images