A giant panda staple bamboo species mapping method based on unmanned aerial vehicle image and deep learning

Abstract

The present application belongs to the technical field of unmanned aerial vehicle remote sensing, photogrammetry and deep learning semantic segmentation, and particularly relates to a big panda staple bamboo species mapping method based on unmanned aerial vehicle images and deep learning, aiming to solve the problems of high cost, long cycle and discontinuous spatial coverage in traditional big panda staple bamboo survey relying on manual investigation. It includes obtaining unmanned aerial vehicle optical images, extracting and fusing ExG, and constructing R, G, B and ExG four-channel data sets; inputting the four-channel data into a four-channel Swin-UPerNet model adapted and modified, and combining a composite loss function to complete image pixel-level semantic segmentation; building a multi-view projection model to realize spatial matching and geometric reconstruction of two-dimensional recognition results and three-dimensional DSM data; and using a majority-voting fusion strategy to generate a spatially continuous and high-precision staple bamboo species distribution map. The method enhances the recognition of bamboo species in complex backgrounds, accurately identifies the distribution of staple bamboo, and significantly improves the recognition accuracy and spatial mapping continuity of big panda staple bamboo monitoring.

Description

Technical Field

[0001] This invention belongs to the fields of UAV remote sensing, photogrammetry and deep learning semantic segmentation technology, and in particular relates to a mapping method for bamboo species that are staple foods of giant pandas based on UAV imagery and deep learning. Background Technology

[0002] As a key flagship species in biodiversity conservation, the giant panda's survival, reproduction, and population distribution patterns are highly ecologically correlated with the spatial distribution, quality, and dynamic succession of its staple bamboo within its habitat. Therefore, timely, accurate, and detailed information on the spatial distribution of staple bamboo species is not only a core indicator for evaluating giant panda habitat quality but also crucial data support for developing scientific conservation strategies, monitoring habitat degradation risks, and carrying out ecological restoration work. In traditional technical systems, resource surveys of giant panda staple bamboo have long relied on manual quadrat surveys or transect surveys. This involves surveyors conducting on-site sampling in complex mountainous environments to record bamboo species, coverage, and growth status. From a technical perspective, this survey model based on finite discrete sampling points played a vital role in a specific historical period, ensuring the authenticity of data at the sampling point scale through on-site verification. However, with the continuous development of related technologies and the increasingly stringent requirements for data timeliness, spatial continuity, and precision in ecological monitoring tasks, the aforementioned traditional technical solutions have gradually revealed insurmountable limitations in addressing new challenges.

[0003] Specifically, the manual survey method is limited by extremely high labor costs and cumbersome survey cycles, resulting in a very low data update frequency, making it difficult to meet the needs for rapid and dynamic monitoring of habitat changes. A more profound technical contradiction lies in the fact that, due to the significant spatial dispersion and discontinuity of manually deployed quadrats, this point-to-area extrapolation method easily overlooks patchy and fragmented bamboo forest boundary information when dealing with rugged terrain and distinct vertical vegetation zonation in giant panda habitats. This leads to errors in the spatial resolution and mapping accuracy of the final mapping results. Furthermore, due to differences in the professional backgrounds and subjective experience of different surveyors, there is often a lack of unified objective standards for identifying and classifying complex canopy structures under the forest canopy, which to some extent weakens the stability of the survey results and the data consistency between different regions.

[0004] In recent years, with the rapid development of UAV low-altitude remote sensing technology, its advantages such as high resolution, maneuverability, and minimal cloud interference have provided a new observation dimension for forest resource surveys. Meanwhile, deep learning algorithms, represented by semantic segmentation, have shown great potential in image feature extraction and pixel-level classification. However, in the practical application of UAV remote sensing imagery and deep learning algorithms to identify the staple bamboo species of giant pandas, deep-seated technical bottlenecks still exist. One reason is that the staple bamboo and surrounding evergreen and deciduous vegetation share highly similar spectral characteristics in the visible light band. Especially under the complex natural forest canopy, the colors, textures, and morphological features of different vegetation species are intertwined, resulting in severe "same species, different spectra" and "different species, same spectra" phenomena. Traditional deep learning models based on three-channel (RGB) information often struggle to construct sufficiently clear decision boundaries in the feature space, leading to large-scale mixing and omissions in complex lighting environments and mixed vegetation areas, making it difficult to achieve the required precision for refined management.

[0005] Another problem is that recognition results based on optical images cannot form effective spatial maps due to the lack of geographic information. Because drones are affected by factors such as terrain undulations, camera distortion, and flight attitude during flight, the original images acquired have complex geometric deformations. Even if high recognition accuracy is achieved at the two-dimensional semantic level, without multi-view geometric consistency constraints, the resulting spatial distribution map often exhibits severe topological conflicts, ghosting, or gaps at patch boundaries due to projection differences, displacement deviations, and geometric displacement caused by canopy occlusion. This secondary problem, arising from the disconnect between the recognition model and the geospatial reconstruction process, directly leads to a serious lack of spatial continuity and geometric accuracy in the mapping of staple bamboo species distribution, making it impossible to form a truly comprehensive "single map" monitoring result.

[0006] Therefore, achieving high-precision feature representation of staple bamboo species while overcoming interference from vegetation spectral similarity, and collaboratively solving the accurate mapping and fusion between multi-view identification results and complex geospatial benchmarks, has become a highly challenging key problem for technicians in the fields of forestry remote sensing and ecological monitoring. This invention aims to provide an automated mapping method that can integrate multi-dimensional prior features, possesses strong feature extraction capabilities, and achieves consistent reconstruction of multi-view geographic information. Summary of the Invention

[0007] To address the technical problems in existing technologies, such as low efficiency, poor spatial continuity, long update cycles, insufficient identification accuracy due to reliance on manual surveys for surveying staple bamboo species of giant pandas, and the disconnect between two-dimensional identification results and three-dimensional spatial distribution, this invention provides a mapping method for staple bamboo species of giant pandas based on UAV imagery and deep learning.

[0008] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows: A method for mapping the staple bamboo species of giant pandas based on UAV imagery and deep learning includes the following steps: S1: Using a multi-rotor drone equipped with a wide-angle visible light camera, optical images of giant panda habitats are acquired under preset environmental conditions, and the optical images are processed in blocks to construct a multi-channel input dataset. S2: Construct a four-channel semantic segmentation deep learning model based on Swin-UPerNet, using SwinTransformer Tiny as the encoder and UPerNet decoder. Channel dilation is performed by embedding the first layer patch of the pre-trained model into the convolutional layer to adapt to the four-channel input containing ExG, and training is performed using a composite loss function. S3: Staple bamboo species identification output: Input the original image of the UAV to be identified into the trained Swin-UPerNet model, calculate the predicted probability of each pixel belonging to each category, and extract the pixel-level mask of the staple bamboo species. S4: Obtain the internal and external orientation elements of the camera through photogrammetric adjustment calculation, construct a multi-view photogrammetric projection model, project the polygon vertices in the pixel-level mask onto the digital surface model (DSM) for three-dimensional ray optimization, and perform multi-view voting fusion to generate a spatial distribution map of staple bamboo species.

[0009] Preferably, the specific process of segmenting the optical image and constructing a multi-channel input dataset in step S1 is as follows: S11: Segmentation and resizing of optical images: Dividing the resizing image into segments... Nr OK, Nc A column of grid blocks; adjacent grid blocks maintain a specified overlap ratio in both the horizontal and vertical directions, with a target size of [missing information]. W 0 × H 0 The original data size serves as the final input. S12: Construct four-channel data R+G+B+ExG: For each image block, construct four-channel input data including R, G, BB and ExG, and normalize each channel: ExG is constructed as the fourth channel based on the original RGB image, and uniform scaling is performed across the entire library to obtain the original ExG index map of the block; and the numerical distribution of ExG is calibrated once across the entire image library to obtain a uniform lower bound L and upper bound H; S13: Manually annotate the preprocessed UAV images; delineate polygonal ranges on the images using vegetation patches as units, and classify each patch into a pre-agreed category based on canopy color, texture, and morphological characteristics; all annotation results are ultimately mapped to a fixed four-category classification system, including staple bamboo (two bamboo species), background (irrelevant elements such as snow, roads, and water bodies), deciduous vegetation, and evergreen vegetation. Assign a unique integer code to each category, and rasterize the polygon annotations into a pixel-level label matrix, which is then randomly divided into training, validation, and test sets according to a specified ratio.

[0010] Preferably, the specific process of step S2 is as follows: S21: Perform channel splicing and tensor arrangement; S22: Construct and train the Swin-UPerNet four-channel semantic segmentation network: The semantic segmentation network consisting of Swin TransformerTiny as the encoder and UPerNet as the decoder is used as the backbone structure, and it is extended to a four-channel input. S23: Set a composite loss function: During training, use the weighted sum of multi-class Focal-Tversky loss and multi-class Dice loss as the total loss function; S24: Set training strategy, early stopping mechanism and model evaluation metrics: Learning strategy: Use AdamW optimizer for end-to-end training and combine with cosine annealing with restart to automatically adjust the learning rate; Early stopping strategy: Use the mean IoU of the two types of staple bamboo in the validation set as the core metric for model selection and early stopping; Model evaluation metrics: Quantitatively evaluate model performance based on the pixel-level confusion matrix of the validation and test sets.

[0011] Preferably, the specific process of step S2 is as follows: S211: Pixel-level four-channel vector construction: For any pixel in an image block ( x, y (), which has been normalized R norm ( x, y ), G norm ( x, y ), Bnorm ( x, y ) and prior channels Concatenate them along the channel dimension to form a four-dimensional input feature vector; S212: Spatial Channel Stitching: For the entire image block, at the height... H ,width W Within the specified range, the four channels are spliced ​​together according to their channel dimensions to obtain the following shape: X HWC ∈R H×W×4 The four-channel image tensor; S213: Tensor arrangement to fit network input: Move the channel dimension to the front to obtain: X = Transpose ( X HWC )∈R H×W×4 After adding the batch dimension, the actual shape of the input tensor received by the network is: X batch ∈R N×4×H×W .

[0012] Preferably, the specific process of step S22 is as follows: S221: The pre-trained four-channel semantic segmentation deep learning model based on Swin-UPerNet uses a patch embedded in the original first-layer convolutional layer as a three-channel input. To receive four-channel input, the convolutional kernel is dilated from 3 to 4 channels: first, the element-wise mean of the weights of the first three input channels is calculated, and then this is used as the weight of the fourth channel to concatenate and obtain a new convolutional kernel; this weight replaces the original weights of the first-layer convolution, thus achieving the desired effect. X Four-channel feature extraction, while preserving the expressive power of the pre-trained model to the maximum extent; S222: During the encoding-decoding process, the network processes each pixel ( x, y Output a class log vector of length C=4, and obtain the predicted probability of each class through softmax. The final pixel prediction label is obtained. .

[0013] Preferably, the composite loss function in step S23 includes: Multiple types of Dice loss: For category c The Dice coefficient is: ; in, p i,c For the first i Each pixel belongs to the category c The predicted probability; t i,c : No.i The real label of each pixel; It is a tiny constant; The formula for multi-class Dice loss is as follows: ; Multiple Focal-Tversky Losses: Category c Define soft statistics: category c Soft count of true instances: ; category c Soft counting of false positives: ; category c Soft count of false negatives: ; The Tversky coefficient is: ; Constructing the Focal-Tversky loss: ; in, γ As a focusing factor; Total loss function: The total loss of a single branch is obtained by linearly combining the multi-class Dice loss and the multi-class Focal-Tversky loss with fixed weights. .

[0014] Preferably, the specific process of step S3 is as follows: S31: Converts the pre-processed raw optical image of the UAV to be identified into a uniformly sized four-channel input block. X batch ∈ℝ N×4×H×W ; S32: The four-channel input block is fed into the trained Swin-UPerNet semantic segmentation model to obtain the prediction result of the category to which each pixel belongs. The model output is a pixel-level category probability map. After the category maximum probability operation, the pixel-level classification map of the whole image in the categories of staple bamboo, deciduous vegetation, evergreen vegetation and background is obtained. S33: Extract the pixels belonging to the staple bamboo species from the prediction results to form a pixel-level mask image of staple bamboo, which will be used for subsequent geographic information reconstruction.

[0015] Preferably, the specific process of step S4 is as follows: S41: Data Input and Coordinate Unification: Perform aerial triangulation and multi-view stereo reconstruction on optical images to generate a Digital Elevation Model (DSM) and Orthophoto Map (DOM) for the corresponding survey area, and export the aerial triangulation results file; using the coordinate reference system of the DOM as the target coordinate system, convert the latitude and longitude camera positions recorded in the CSV into DOM coordinates using projection transformation relationships; read the interior and exterior orientation elements corresponding to each image; use the camera plane position in the aerial triangulation results to perform spatial nearest neighbor matching with the camera center (X,Y), and reconstruct the correspondence between the hash file name and the original UAV file name within the error threshold to achieve one-to-one alignment between the photogrammetric results and the labeled images; S42: Construct a multi-view photogrammetric projection model; S43: 3D projection of labeled polygon vertices and DOM plane reconstruction; S44: DOM segmentation, rasterization, and multi-view voting fusion: Preferably, the specific process of step S42 is as follows: S421: Constructing the camera intrinsic parameter model: For pixel coordinates ( u, v According to the intrinsic parameter matrix Calculate the normalized distortion coordinates: ; ; in,, f x ,f y Focal length in pixels. c x , c y The pixel coordinates of the principal point on the image plane s The slope coefficient is ( u, v ) represents pixel coordinates, ( x d ，y d ) represents the initial normalized, distortion-free pixel coordinates; S422: Inverse solution of radial and tangential distortion: Let the distortion-free coordinates of the current iteration be ( x, y radial distance r 2 = x 2 + y 2 The distortion model is then: ; ; ; according to , ; S423: Use an iterative method to reverse the process ( x, y ), until convergence, to obtain distortion-free normalized image points ( x, y ): ; ; in,, r 2 Let be the square of the radial distance from the image point to the optical axis. k 1, k 2, k3 is the radial distortion coefficient, p1, p2 are the tangential distortion coefficients, ( x, y ) represents the distortion-free normalized coordinates of the current iteration, ( x d ，y d () represents the corresponding distortion-normalized coordinates, with superscript indicating the coordinates. k Indicates the first k The variable values ​​for each iteration; S424: Ray representation in world coordinates: Treating distortion-free image points as direction vectors in camera coordinates: ; through rotation matrix R cw Obtain the world coordinate system orientation from pixel coordinates: ; With the camera center C = [X,Y,Z] T Starting from the pixel ( u, v The corresponding three-dimensional ray is: ; ; Where, d world For pixels ( u, v The corresponding ray direction vector, t max This represents the maximum ray length.

[0016] Preferably, the specific process of step S44 is as follows: S441: DOM tiled rasterization uses tiled calculation: based on the row and column range of the DOM, a tiled grid is constructed according to a fixed tile size, and each tile is recorded as a tile; for each tile, a tile-image record index is established using its range and the bounding rectangle of each image footprint polygon; S442: Polygon rasterization within a single tile: For each tile, the pixel coordinates within that tile ( x, y The function reads the valid mask of the DOM and obtains the valid region indicator:M dom ( x, y )∈{0,1}; for each image record participating in this block i : Rasterize the image footprint polygons to obtain the overlay mask: M i ( x, y The image is projected onto the DOM and rasterized into polygons of each category to obtain the label raster. L i (x,y)∈{0,1,2,…,C}; where 0 indicates that the pixel is unlabeled from this viewpoint; positive integers represent different categories; for each pixel (x,y), define the unlabeled count and category. c The vote count; S443: Multi-view majority voting fusion: for each pixel ( x, y First, find the category with the most votes, and introduce the rule of priority for unmarked categories. Only when the number of votes for a certain category is strictly greater than the number of unmarked votes will it be assigned a category; otherwise, it will be recorded as no category. S444: Result raster output: Write out the merged L * (x,y) is written to the output GeoTIFF file in the form of a single-band integer raster; a mapping table of category name → category code is saved in the data tags.

[0017] The beneficial effects of this invention include: 1. Significantly improves the accuracy of staple bamboo species identification in complex backgrounds: The RGB three-channel input plus the green saliency index ExG is constructed as a four-channel input. ExG enhances the spectral separability between vegetation and non-vegetation, and between staple bamboo and other vegetation, effectively alleviating the confusion and omission caused by different species having the same spectrum or the same species having different spectra in mountainous forest areas. At the same time, the first convolution of Swin-UPerNet is dilated from 3 to 4 channels to fully preserve the feature extraction capability of the pre-trained model. Combined with a multi-class Focal-Tversky+Dice composite loss function, it specifically solves the problems of sample imbalance, blurred boundaries, and missed detection of small patches, which significantly improves the IoU, overall accuracy and F1 value of staple bamboo identification, and maintains stable segmentation results in mixed forests, fragmented patches and complex lighting scenes.

[0018] 2. Effectively and thoroughly eliminate projection deviation from 2D recognition to 3D mapping: The camera's interior and exterior orientation elements are obtained through photogrammetric aerial triangulation. A multi-view ray projection model with distortion correction is constructed. The vertices of the 2D pixel-level mask are precisely intersected with the DSM digital surface model along the ray, realizing a strict 3D mapping of the recognition results from the image plane to the real geographic space. This eliminates the geometric offset and ghosting caused by terrain undulation, camera posture, and lens distortion from the root, ensuring that the boundaries and spatial positions of bamboo forest patches are consistent with the actual ground height.

[0019] 3. Achieve robust fusion and spatially continuous mapping of multi-view recognition results: Adopt a multi-view voting fusion strategy that prioritizes unlabeled areas, and make majority decisions on overlapping areas of multiple views. Prioritize the retention of consistent labels, filter noise and false detections, so that the final output bamboo species distribution map is spatially continuous, without topological conflicts, holes, or overlaps, forming a single map geographic information result that can be directly used for monitoring and management, solving the problem of spatial discontinuity caused by traditional single-view stitching. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the method for mapping giant panda staple bamboo species based on UAV imagery and deep learning, as described in this invention. Figure 2 This is a schematic diagram of the process for identifying and mapping the distribution of staple bamboo according to the present invention; Figure 3 This is a schematic diagram of the overall process of the present invention; Figure 4 This is a rendering of the geographic information reconstruction effect of the present invention. Detailed Implementation

[0021] The following is in conjunction with the appendix Figures 1 to 4 The present invention will be further described in detail below: Example 1 See appendix Figures 1 - 4 As shown, a method for mapping the staple bamboo species of giant pandas based on UAV imagery and deep learning includes the following steps: Step S1: Image data acquisition and preprocessing: Using a multi-rotor drone equipped with a wide-angle visible light camera, optical images of the giant panda habitat are acquired under preset environmental conditions, and the optical images are processed in blocks to construct a multi-channel input dataset. Step S2: Construct a four-channel semantic segmentation deep learning model based on Swin-UPerNet: Use SwinTransformer Tiny as the encoder and UPerNet decoder. Dilate the channels by embedding the first layer patch of the pre-trained model into the convolutional layer to adapt to the four-channel input containing ExG, and use a composite loss function for model training. Step S3: Staple bamboo species identification output: Input the original image of the UAV to be identified into the trained Swin-UPerNet model, calculate the predicted probability of each pixel belonging to each category c, and extract the pixel-level mask of the staple bamboo species (1). Step S4: Multi-view geographic information reconstruction and distribution mapping based on photogrammetry: The internal and external orientation elements of the camera are obtained through photogrammetric adjustment calculation, a multi-view photogrammetric projection model is constructed, the polygon vertices in the pixel-level mask are projected onto the digital surface model (DSM) for three-dimensional ray optimization, and multi-view voting fusion is performed to generate a spatial distribution map of staple bamboo species.

[0022] In this embodiment, the specific process of segmenting the optical image and constructing a multi-channel input dataset in step S1 is as follows: S11: Segmentation and resizing of optical images: Dividing the resizing image into segments... Nr OK, Nc The column grid blocks are set to Nr =5, Nc =4; Adjacent grid blocks retain a specified overlap ratio in both the horizontal and vertical directions. The specified ratio can be set to 20%. Let the target size be... W 0 × H 0 In this embodiment, the following settings are provided. W 0 = 512, H 0 = 1024, the size of the raw data as the final input.

[0023] S12: Constructing Four-Channel Data R+G+B+ExG: For each image patch, construct four-channel input data containing three RGB channels and one green saliency index (ExG) channel, and normalize each channel: The pixel values ​​after resampling a certain image patch are denoted as: ; in( x, y ) represents the pixel coordinates. R , G , B The values ​​represent the digital quantization (DN) values ​​of the red, green, and blue channels, respectively, with a range of [0, 255]. During the data preparation stage, this invention keeps the DN values ​​of the RGB channels unchanged and directly uses them as the first three input channels for subsequent training and inference of the deep learning model.

[0024] To enhance the separability of the staple bamboo species relative to the surrounding evergreen and deciduous vegetation, this invention constructs the Excess Green Index (ExG) as a fourth channel based on the original RGB image, and performs uniform scaling normalization across the entire database. ExG raw index calculation: In the implementation process, image blocks are read in BGR order, denoted as B( x, y ),G( x, y ),R( x, y After converting it to floating-point, calculate the original ExG value at the cell using the following formula: ; The above calculations are performed independently on each image block to obtain the raw ExG index map for that block; Quantile calibration across the entire image database: To ensure the comparability of ExG indices between images from different flights and under different lighting conditions, the numerical distribution of ExG is calibrated once across the entire image database to obtain a unified lower bound L and upper bound H. For each original BGR image, it is first converted to HSV space, and the saturation S and luminance V components are taken. The original ExG value is calculated only on suspected vegetation pixels that meet the conditions of S≥40 and 40≤V≤240. If there are too few pixels that meet the conditions, the entire image is returned for statistical analysis.

[0025] In each image, to reduce computational load, up to 120,000 pixels of raw ExG values ​​are randomly selected from the suspected vegetation pixels as samples, and the samples from all images are aggregated to form a "full library ExG sample set".

[0026] Calculate the 10th percentile and 90th percentile on this sample set, and use them as the unified lower bound L and upper bound H of ExG, respectively, i.e.: Lower bound: 10th percentile of the entire library's ExG; Upper bound: 90th percentile of the entire ExG dataset.

[0027] After calibration, (L,H) is written into the configuration file and only needs to be calculated once. Subsequently, the same set of (L,H) is used for each image to linearly normalize the ExG to [0,1], thereby achieving a fixed quantile scale and ensuring that the ExG of different images and different patches has a uniform numerical scale. S13: Staple Bamboo Species Labeling and Dataset Division: With the participation of staple bamboo classification experts, preprocessed UAV images were manually labeled; polygonal areas were delineated on the images using obvious vegetation patches as units, and each patch was assigned to a pre-agreed category based on canopy color, texture, and morphological characteristics; all labeling results were ultimately mapped to a fixed four-category classification system, including staple bamboo species, background category, deciduous vegetation category, and evergreen vegetation category. Staple bamboo species: Distribution areas of staple bamboo species for giant pandas in the study area, including staple bamboo under deciduous vegetation; Background category: includes bare ground, roads, water bodies, buildings, shadows, and other non-vegetation or non-identifiable areas; Deciduous vegetation: mainly deciduous broad-leaved forests, deciduous shrubs, etc. Evergreen vegetation includes evergreen coniferous forests, evergreen broad-leaved forests, and evergreen shrubs.

[0028] Each category is assigned a unique integer code, such as 0 for background, 1 for staple bamboo, 2 for deciduous vegetation, and 3 for evergreen vegetation. The polygon labels are then rasterized into a pixel-level label matrix.

[0029] After obtaining uniformly coded labels, the entire image is randomly divided into training, validation, and test sets in a ratio of approximately 8:1:1. This ensures that the same original image and its segmented blocks appear in only one subset, thereby avoiding spatial information leakage and guaranteeing the objectivity of model training and evaluation.

[0030] Example 2 Based on Example 1, the specific process of step S2 is as follows: S21: Perform channel splicing and tensor arrangement: S211: Pixel-level four-channel vector construction: For any pixel in an image block ( x, y (), which has been normalized R norm ( x, y ), G norm ( x, y ), B norm ( x, y ) and prior channels Concatenate them along the channel dimension to form a four-dimensional input feature vector: ; S212: Spatial Channel Stitching: For the entire image block, at the height... H ,width W Within the specified range, the four channels are spliced ​​together according to their channel dimensions to obtain the following shape: X HWC ∈R H×W×4 The four-channel image tensor (the last dimension is the channel dimension). S213: Tensor Arrangement to Adapt to Network Input: To satisfy the input conventions of convolutional networks in deep learning frameworks, this invention moves the channel dimension to the front, resulting in: X = Transpose ( X HWC )∈R H×W×4 After adding the batch dimension, the actual shape of the input tensor received by the network is: X batch ∈R N×4×H×W ,inN Batch size; The above steps achieve a unified organization of the RGB three channels and the ExG channel, enabling the model to simultaneously utilize raw optical information and vegetation index priors for staple bamboo species identification. This processing is consistent across the training, validation, and testing datasets.

[0031] S22: Construct and train the Swin-UPerNet four-channel semantic segmentation network: The semantic segmentation network (Swin-UPerNet) consisting of Swin TransformerTiny as the encoder and UPerNet as the decoder is used as the backbone structure, and it is extended to a four-channel input. S221: The pre-trained four-channel semantic segmentation deep learning model based on Swin-UPerNet has its original first-layer patch embedded into a convolutional layer with three-channel input. Its weight tensor is denoted as: To receive four-channel input, the convolution kernel is dilated from 3 to 4 channels: first, the element-wise mean of the weights of the first three input channels is calculated: Then, these weights are used as the fourth channel weights and concatenated to obtain a new convolutional kernel: Replace the original first-layer convolution weights with this weight to achieve [the desired effect]. X Four-channel feature extraction, while preserving the expressive power of the pre-trained model to the maximum extent; S222: During the encoding-decoding process, the network processes each pixel ( x, y Output a class logarithmic vector of length C=4, and obtain the predicted probabilities of each class using softmax: ; in, C =0 indicates the background class. C =1 is mainly bamboo 1, C =2 represents deciduous vegetation. C =3 represents evergreen vegetation. C =4 are the main food bamboo species; 2 The final pixel predicted label is: ; The above category encoding is consistent with the label system in the training code.

[0032] S23: Set a composite loss function: During training, the weighted sum of multi-class Focal-Tversky loss and multi-class Dice loss is used as the total loss function to balance the recognition effect of staple bamboo under class imbalance and the overall segmentation accuracy. The composite loss function in step S23 includes: Let the total number of valid pixels be N, and the total number of categories be C=4. For the first... iFor each pixel, the predicted probability is: p i,c , t i,c For the first i The real label of the first pixel, when the i The true category of each pixel is c hour, t i,c =1, otherwise 0; Multiple types of Dice loss: For category c The Dice coefficient is: ; in, p i,c For the first i Each pixel belongs to the category c The predicted probability; t i,c : No. i The real label of each pixel; To prevent the use of tiny constants with a denominator of 0, the value is set to 10. −6 ; The formula for multi-class Dice loss is as follows: ; Multiple Focal-Tversky Losses: Category c Define soft statistics: category c Soft count of true instances: ; category c Soft counting of false positives: ; category c Soft count of false negatives: ; The Tversky coefficient is: ; in, Pick α =0.7, β =0.3; Based on this, the Focal-Tversky loss is constructed: ; in, γ The focus factor is set to 1.33 in this embodiment; Total loss function: The total loss of a single branch is obtained by linearly combining the multi-class Dice loss and the multi-class Focal-Tversky loss with fixed weights. ; in, λ FT =0.7, λ Dice =0.3. This embodiment uses a single-output head structure, and the total loss during training is... L branch In the corresponding implementation FT The configuration is weight = 0.7 and DICE weight = 0.3.

[0033] S24: Set training strategy, early stopping mechanism, and model evaluation metrics: Learning strategy: End-to-end training is performed using the AdamW optimizer, with an initial learning rate of 1×10⁻⁶. −4 It also incorporates a cosine annealing warp restart strategy to automatically adjust the learning rate, enabling rapid convergence in the early stages and meticulous approximation of the optimal solution in the later stages, thus preventing excessive oscillations.

[0034] Lightweight data augmentation and training subsets: To improve the spatial invariance of the model, this invention employs lightweight geometric augmentation during the training phase: for each training sample, a horizontal mirror flip is performed with a 50% probability, while simultaneously flipping the label matrix and the ExG prior channel; no augmentation is performed on the validation and test sets.

[0035] Early cessation strategy: To highlight the recognition performance of staple bamboo, this invention uses the mean IoU of two types of staple bamboo (Category 1 and Category 4) in the validation set as the core indicator for model selection and early cessation, specifically: Record No. e After the training round, the IoU of staple food bamboo 1 and staple food bamboo 2 in the validation set were respectively and Define the metrics to focus on: ; In the current round When the current model parameters exceed the historical best value, save the current model parameters as the "optimal model"; if no improvement is seen for T consecutive rounds (T=20 in this example), trigger early stop to terminate the training process and avoid overfitting and ineffective training; Model evaluation metrics: Quantitatively evaluate model performance based on the pixel-level confusion matrix of the validation and test sets; Let the total number of categories be C=4, and the confusion matrix be... M ∈ R C×C ,in M ij Indicates the true category is i Predicted as j The number of pixels; for each categoryc : Intersection over Union (IoU): ; ; ; ; Further calculate the average IoU (mIoU): ; Overall Accuracy (OA): ; Accuracy: ; Recall: ; F1 value: .

[0036] Example 3 Based on Example 1 or Example 2, the specific process of step S3 is as follows: S31: Input of the image to be identified: The original optical image of the UAV to be identified, acquired during the leaf-fall season, is converted into a uniformly sized four-channel input block according to the data preprocessing flow in step S1 (including image cropping, ExG prior calculation, and four-channel construction). X batch ∈ℝ N×4×H×W ; S32: Semantic Segmentation Inference: The four-channel input block is fed into the trained Swin-UPerNet semantic segmentation model to obtain the prediction result of the category to which each pixel belongs. The model output is a pixel-level category probability map. After the maximum probability (argmax) operation is performed on the category, the pixel-level classification map of the entire image in terms of staple bamboo species, deciduous vegetation, evergreen vegetation, and background is obtained. S33: Generation of staple bamboo species identification results: Extract the pixels belonging to the staple bamboo species from the prediction results to form a pixel-level mask image of staple bamboo, which is used for subsequent geographic information reconstruction.

[0037] Through the above steps, the automatic identification of staple bamboo species based on the original optical images of UAVs and a four-channel semantic segmentation deep learning model was successfully achieved, which significantly improved the accuracy of bamboo species identification, especially in the identification of bamboo forest boundaries, mixed areas and sparse bamboo patches, and enhanced the model's identification ability in complex environments.

[0038] Example 4 Based on Example 1, Example 2, or Example 3, step S4 is used to accurately project the annotation results of staple bamboo species in multi-view UAV imagery onto the measured DSM / DOM to generate a spatially continuous raster distribution map of staple bamboo species. This solves the problem of poor spatial continuity and difficulty in forming high-precision spatial distribution results in traditional surveys. The specific process is as follows: S41: Data input and coordinate unification: S411: Perform aerial triangulation and multi-view stereo reconstruction on optical images to generate digital elevation models (DSM) and orthophotos (DOM) for the corresponding survey area, and export the aerial triangulation results file; Extract the camera intrinsic and extrinsic parameters for each drone image from the BlocksExchangeUndistortAT_WithoutTiePoints.xml file exported from DJI Terra, including image width and height (W, H) and intrinsic parameter matrix K (focal length). fx , fy Principal point coordinates (c x , c γ The image contains tilt coefficients (s), radial distortion coefficients (k1, k2, k3), tangential distortion coefficients (p1, p2), and exterior orientation elements (camera center C = (X, Y, Z), rotation matrix R); the latitude, longitude, and elevation information corresponding to each image is read. Read DSM and DOM data, and obtain the spatial reference system (CRS) and raster resolution of the DOM to provide a foundation for subsequent 3D ray-DSM intersection operations and unified rasterization; Obtain the staple bamboo identification results output by the four-channel semantic segmentation deep learning model based on Swin-UPerNet in step S3. Export the staple bamboo identification results to Label Studio, and then export the JSON file and its corresponding image directory. Group and manage the polygon annotation results of each image according to the image name. S412: Coordinate Reference System 1: Using the DOM's coordinate reference system as the target coordinate system, the latitude and longitude camera positions recorded in the CSV are converted into DOM coordinates using projection transformation relationships; the projection relationship can be expressed as: ; When the DSM and DOM coordinate systems are inconsistent, construct: , Unify the planar coordinates on the DSM to the DOM coordinate system; in, , These are the longitude and latitude of the camera location, respectively. X dom ,Ydom ) represents the planar coordinates of the DOM image. X dsm , Y ds m () represents the plane coordinates of the DSM elevation map; S413: Camera Recording Reconstruction and Alignment: Read the interior orientation elements (resolution W, H, principal point) for each image from BlocksExchange XML. c x ，c y ),focal length f x ，f y Distortion parameters k 1, k 2, k 3, p 1, p 2) And the exterior orientation elements (camera center C=[X,Y,Z]) T (and rotation matrix); By using the camera planar position in the aerial triangulation results and performing spatial nearest neighbor matching with the camera center (X,Y) in BlocksExchange, the correspondence between the hash file name and the original UAV file name (the file name used by Label Studio) is reconstructed within the error threshold, thereby achieving one-to-one alignment between the photogrammetric results and the labeled images. S42: Construct a multi-view photogrammetric projection model: S421: Constructing the camera intrinsic parameter model: For pixel coordinates ( u, v According to the intrinsic parameter matrix Calculate the normalized distortion coordinates: ; ; in,, f x ,f y Focal length in pixels. c x , c y The pixel coordinates of the principal point (optical center) on the image plane. s The slope coefficient is ( u, v ) represents pixel coordinates, ( x d ，y d ) represents the initial normalized, distortion-free pixel coordinates; S422: Inverse solution of radial and tangential distortion: Let the distortion-free coordinates of the current iteration be ( x, y radial distance r 2 = x 2 + y 2 The distortion model is then: ; ; ; according to , ; S423: Use an iterative method to reverse the process ( x, y ), until convergence, to obtain distortion-free normalized image points ( x, y ): ; ; in,, r 2 Let be the square of the radial distance from the image point to the optical axis. k 1, k 2, k3 is the radial distortion coefficient, p1, p2 are the tangential distortion coefficients, ( x, y ) represents the distortion-free normalized coordinates of the current iteration, ( x d ，y d () represents the corresponding distortion-normalized coordinates, with superscript indicating the coordinates. k Indicates the first k The variable values ​​for each iteration; S424: Ray representation in world coordinates: Treating distortion-free image points as direction vectors in camera coordinates: ; through rotation matrix R cw (camera→world) Obtain the world coordinate system orientation from pixel coordinates: ; With the camera center C = [X,Y,Z] T Starting from (the position of the camera center in the world coordinate system), the number of pixels ( u, v The corresponding three-dimensional ray is: ; ; Where, d world For pixels ( u, v The corresponding ray direction vector, tmax This is the maximum ray length, and it ensures that the ray points towards the ground.

[0039] S43: 3D projection of labeled polygon vertices and DOM plane reconstruction; S431: Annotation Coordinate Restoration and Simplification: For each image, read its corresponding polygon annotation results from Label Studio JSON (stored as a percentage). px [%), py [%]) coordinates), converted to pixel coordinates: ; Based on the image EXIF ​​orientation markers, for ( u, v Perform a reverse rotation to restore the original image coordinates; Each polygon vertex sequence is simplified by perimeter constraints to ensure that the number of vertices does not exceed a preset limit (set to 150), thereby reducing the number of subsequent DSM ray intersections.

[0040] S432: Intersection of vertex along ray with DSM: For each vertex cell in the polygon ( u, v Using the above photogrammetric ray model P(t), sampling is performed on the DSM at step sizes: From the ray plane coordinates ( X ( t ), Y ( t Read DSM elevation Z DSM ( X ( t ), Y ( t )); When Z(t) ≤ Z DSM ( X ( t ), Y ( t When the ray first intersects the Earth's surface, that point is considered the ground point. X g ,Y g ,Z g ); If the DSM coordinate system is inconsistent with the DOM, then through ( X dom ,Y dom =TDSM→DOM( X g ,Y gUnify vertex coordinates to the DOM plane coordinate system; S433: Polygonal Ground Feature Projection Reconstruction: Connect the set of (Xdom, Ydom) obtained from the above projection into a planar geometry to form the ground projection polygon of the label under this viewpoint; at the same time, for each image, select a subset of pixels of a regular grid, project it onto the ground in the same way, calculate its convex hull to obtain the image footprint polygon of the image on the DOM, which is used to limit the effective coverage of this viewpoint. Finally, for each UAV image participating in the annotation, a record is obtained: including the footprint polygon of the image and several ground polygons of staple bamboo / other categories after projection. S44: DOM segmentation, rasterization, and multi-view voting fusion: S441: DOM tiled rasterization reduces CPU computation pressure by employing tiled computation. Specifically, it constructs a tiled grid based on the row and column range of the DOM and a fixed tile size (e.g., 1024×1024 pixels). Each tile is denoted as a tile. For each tile, a "tile-image record" index is established using its range and the bounding rectangle of each image footprint polygon. Only images that actually cover the tile participate in subsequent voting, thereby reducing unnecessary computation.

[0041] S442: Polygon rasterization within a single tile: For each tile, the pixel coordinates within that tile ( x, y Perform the following operations on it: Read the valid mask of the DOM to obtain the function indicating the valid region: M dom ( x, y )∈{0,1}; For each image record participating in this block i : The overlay mask is obtained by rasterizing the image footprint polygons. M i ( x, y )∈{0,1}; The image is projected onto the DOM and rasterized into polygons of each category to obtain the label raster: L i (x,y)∈{0,1,2,…,C}; Where 0 indicates "not labeled from this perspective"; positive integers represent different categories (staple bamboo, deciduous vegetation, evergreen vegetation, background, etc.). For each cell (x, y), define: Unlabeled count: This cell is covered by the image footprint but is not labeled by any type of polygon within the image. ; categoryc Vote count: ; in, M i (x,y) represents the first... i Image in pixels ( x, y Whether the area is covered (1 for coverage, 0 otherwise). L i ( x, y ) represents the category label of the i-th image at this pixel (0 indicates unlabeled / background), 1 ( * The parentheses () represent an indicator function that returns 1 if the condition within the parentheses is true, and 0 otherwise. C represents the total number of categories. U ( x, y ) represents the "unlabeled number" of a pixel across all viewpoints. V c ( x, y () represents the total number of votes for a pixel in its category; S443: Multi-view majority voting fusion: for each pixel ( x, y First, find the category with the most votes: ; ; Introduce the rule of "unlabeled priority": only assign a category if the number of votes for a certain category is significantly greater than the number of unlabeled votes; otherwise, record it as unlabeled (background). ; Among them, c * (x,y) represents the candidate class for pixel (x,y), Vmax(x,y) is the maximum number of votes among all classes, and L * (x,y) represents the final merged label, with a value of 0 indicating background or no category; pixels in invalid DOM areas (such as no data areas, outside water bodies, etc.) are directly assigned the Nodata value (−9999 in the code). S444: Result raster output: Write out the merged L * (x,y) is written to the output GeoTIFF file in the form of a single-band integer raster; A mapping table of "category name → category code" is stored in the data tags to facilitate the generation and interpretation of legends for the distribution map of staple bamboo species.

Claims

1. A method for mapping the staple bamboo species of giant pandas based on UAV imagery and deep learning, characterized in that, Includes the following steps: S1: Using a multi-rotor drone equipped with a wide-angle visible light camera, optical images of giant panda habitats are acquired under preset environmental conditions, and the optical images are processed in blocks to construct a multi-channel input dataset. S2: Construct a four-channel semantic segmentation deep learning model based on Swin-UPerNet, using Swin TransformerTiny as the encoder and UPerNet decoder. Channel dilation is performed by embedding the first layer patch of the pre-trained model into the convolutional layer to adapt to the four-channel input containing ExG, and the model is trained using a composite loss function. S3: Staple bamboo species identification output: Input the original image of the UAV to be identified into the trained Swin-UPerNet model, calculate the predicted probability of each pixel belonging to each category, and extract the pixel-level mask of the staple bamboo species. S4: Obtain the internal and external orientation elements of the camera through photogrammetric adjustment calculation, construct a multi-view photogrammetric projection model, project the polygon vertices in the pixel-level mask onto the digital surface model (DSM) for three-dimensional ray optimization, and perform multi-view voting fusion to generate a spatial distribution map of staple bamboo species.

2. The method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 1, characterized in that, The specific process of segmenting the optical image and constructing a multi-channel input dataset in step S1 is as follows: S11: Segmentation and resizing of optical images: Dividing the resizing image into segments... Nr OK, Nc A column of grid blocks; adjacent grid blocks maintain a specified overlap ratio in both the horizontal and vertical directions, with a target size of [missing information]. W 0 × H 0 The original data size serves as the final input. S12: Construct four-channel data R+G+B+ExG: For each image block, construct four-channel input data containing R, G, B and ExG, and normalize each channel: ExG is constructed as the fourth channel based on the original RGB image, and uniform scaling is performed across the entire library to obtain the original ExG index map of the block; and the numerical distribution of ExG is calibrated once across the entire image library to obtain a uniform lower bound L and upper bound H; S13: Manually annotate the preprocessed UAV images; delineate polygonal ranges on the images using vegetation patches as units, and classify each patch into a pre-agreed category based on canopy color, texture, and morphological characteristics; all annotation results are ultimately mapped to a fixed four-category classification system, including staple bamboo species, background category, deciduous vegetation category, and evergreen vegetation category, assigning a unique integer code to each category, and rasterizing the polygon annotations into a pixel-level label matrix, which is then randomly divided into training set, validation set, and test set according to a specified ratio.

3. The method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 2, characterized in that, The specific process of step S2 is as follows: S21: Perform channel splicing and tensor arrangement; S22: Construct and train a four-channel semantic segmentation network using the Swin-UPerNet as the encoder and the UPerNet as the decoder as the backbone structure, and extend it to a four-channel input. S23: Set a composite loss function: During training, use the weighted sum of multi-class Focal-Tversky loss and multi-class Dice loss as the total loss function; S24: Set training strategy, early stopping mechanism and model evaluation metrics: Learning strategy: Use AdamW optimizer for end-to-end training and combine with cosine annealing with restart to automatically adjust the learning rate; Early stopping strategy: Use the mean IoU of the two types of staple bamboo in the validation set as the core metric for model selection and early stopping; Model evaluation metrics: Quantitatively evaluate model performance based on the pixel-level confusion matrix of the validation and test sets.

4. The method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 3, characterized in that, The specific process of step S2 is as follows: S211: Pixel-level four-channel vector construction: For any pixel in an image block ( x,y (), which has been normalized R norm ( x,y ), G norm ( x,y ), B norm ( x,y ) and prior channels Concatenate them along the channel dimension to form a four-dimensional input feature vector; S212: Spatial Channel Stitching: For the entire image block, at the height... H ,width W Within the specified range, the four channels are spliced ​​together according to their channel dimensions to obtain the following shape: X HWC ∈R H×W×4 The four-channel image tensor; S213: Tensor arrangement to fit network input: Move the channel dimension to the front to obtain: X = Transpose ( X HWC )∈R H ×W×4 After adding the batch dimension, the actual shape of the input tensor received by the network is: X batch ∈R N×4×H×W .

5. The method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 3, characterized in that, The specific process of step S22 is as follows: S221: The pre-trained four-channel semantic segmentation deep learning model based on Swin-UPerNet uses a patch embedded in the original first layer of the convolutional layer as a three-channel input. To receive four-channel input, the convolutional kernel of the convolutional layer is dilated from 3 to 4 channels: first, the element-wise mean of the weights of the first three input channels is calculated, and then this is used as the weight of the fourth channel and concatenated to obtain a new convolutional kernel; this weight replaces the original weights of the first layer convolution, thus achieving the desired four-channel input. X Four-channel feature extraction, while preserving the expressive power of the pre-trained model to the maximum extent; S222: During the encoding-decoding process, the network processes each pixel ( x,y Output a class log vector of length C=4, and obtain the predicted probability of each class through softmax. The final pixel prediction label is obtained. .

6. The method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 3, characterized in that, The composite loss function in step S23 includes: Multiple types of Dice loss: For category C The Dice coefficient is: ； in, p i,c For the first i Each pixel belongs to the category c The predicted probability; t i,c : No. i The real label of each pixel; It is a tiny constant; The formula for multi-class Dice loss is as follows: ； Multiple Focal-Tversky Losses: Category c Define soft statistics: category c Soft count of true instances: ; category c Soft counting of false positives: ; category c Soft count of false negatives: ; The Tversky coefficient is: ; Constructing the Focal-Tversky loss: ； in, γ As a focusing factor; Total loss function: The total loss of a single branch is obtained by linearly combining the multi-class Dice loss and the multi-class Focal-Tversky loss with fixed weights. .

7. The method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 1, characterized in that, The specific process of step S3 is as follows: S31: Converts the pre-processed raw optical image of the UAV to be identified into a uniformly sized four-channel input block. X batch ∈ℝ N×4×H×W ; S32: The four-channel input block is fed into the trained Swin-UPerNet semantic segmentation model to obtain the prediction result of the category to which each pixel belongs. The model output is a pixel-level category probability map. After the category maximum probability operation, the pixel-level classification map of the whole image in the categories of staple bamboo, deciduous vegetation, evergreen vegetation and background is obtained. S33: Extract the pixels belonging to the staple bamboo species from the prediction results to form a pixel-level mask image of staple bamboo, which will be used for subsequent geographic information reconstruction.

8. The method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 1, characterized in that, The specific process of step S4 is as follows: S41: Data Input and Coordinate Unification: Perform aerial triangulation and multi-view stereo reconstruction on optical images to generate the corresponding digital elevation model (DSM) and orthophoto (DOM) for the survey area, and export the aerial triangulation results file; use the coordinate reference system of the DOM as the target coordinate system, and use the projection transformation relationship to convert the latitude and longitude camera positions recorded in the CSV into DOM coordinates; read the interior orientation elements and exterior orientation elements corresponding to each image; By utilizing the camera's planar position in the aerial triangulation results, spatial nearest neighbor matching is performed with the camera center (X,Y). Within the error threshold, the correspondence between the hash file name and the original UAV file name is reconstructed, achieving one-to-one alignment between the photogrammetric results and the labeled images. S42: Construct a multi-view photogrammetric projection model; S43: 3D projection of labeled polygon vertices and DOM plane reconstruction; S44: DOM segmentation, rasterization, and multi-view voting fusion.

9. A method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 8, characterized in that, The specific process of step S42 is as follows: S421: Constructing the camera intrinsic parameter model: For pixel coordinates ( u,v According to the intrinsic parameter matrix Calculate the normalized distortion coordinates: ； ； in,, f x ,f y Focal length in pixels. c x , c y The pixel coordinates of the principal point on the image plane s The slope coefficient is ( u,v ) represents pixel coordinates, ( x d ，y d ) represents the initial normalized, distortion-free pixel coordinates; S422: Inverse solution of radial and tangential distortion: Let the distortion-free coordinates of the current iteration be ( x,y radial distance r 2 = x 2 + y 2 The distortion model is then: ； ； ； according to , ; S423: Use an iterative method to reverse the process ( x,y ), until convergence, to obtain distortion-free normalized image points ( x,y ): ； ； in,, r 2 Let be the square of the radial distance from the image point to the optical axis. k 1, k 2, k3 is the radial distortion coefficient, p1, p2 are the tangential distortion coefficients, ( x,y ) represents the distortion-free normalized coordinates of the current iteration, ( x d ，y d () represents the corresponding distortion-normalized coordinates, with superscript indicating the coordinates. k Indicates the first k The variable values ​​for each iteration; S424: Ray representation in world coordinates: Treating distortion-free image points as direction vectors in camera coordinates: ; through rotation matrix R cw Obtain the world coordinate system orientation from pixel coordinates: ; With the camera center C = [X,Y,Z] T Starting from the pixel ( u,v The corresponding three-dimensional ray is: ; ; Where, d world For pixels ( u,v The corresponding ray direction vector, t max This represents the maximum ray length.

10. A method for mapping giant panda staple bamboo species based on UAV imagery and deep learning according to claim 8, characterized in that, The specific process of step S44 is as follows: S441: DOM tiled rasterization uses tiled calculation: based on the row and column range of the DOM, a tiled grid is constructed according to a fixed tile size, and each tile is recorded as a tile; for each tile, a tile-image record index is established using its range and the bounding rectangle of each image footprint polygon; S442: Polygon rasterization within a single tile: For each tile, the pixel coordinates within that tile ( x,y The function reads the effective mask of the DOM and obtains the effective region indicator function: M dom ( x,y )∈{0,1}; for each image record participating in this block i : Rasterize the image footprint polygons to obtain the overlay mask: M i ( x,y The image is projected onto the polygons of each category in the DOM and rasterized to obtain the label raster. L i (x,y)∈{0,1,2,…,C}; where 0 indicates that the pixel is unlabeled from this viewpoint; positive integers represent different categories; for each pixel (x,y), define the unlabeled count and category. c The vote count; S443: Multi-view majority voting fusion: for each pixel ( x,y First, find the category with the most votes, and introduce the rule of priority for unmarked categories. Only when the number of votes for a certain category is strictly greater than the number of unmarked votes will it be assigned a category; otherwise, it will be recorded as no category. S444: Result raster output: Write out the merged L * (x,y) is written to the output GeoTIFF file in the form of a single-band integer raster; a mapping table of category name → category code is saved in the data tags.

Images