A real-time image semantic segmentation method for low-altitude unmanned aerial vehicles

Abstract

The application discloses a real-time image semantic segmentation method for low-altitude unmanned aerial vehicles, acquires low-altitude unmanned aerial vehicle visual data sets, pre-processes the data sets, builds an ultra-network encoder-decoder structure, adds a weight generation module to the last layer of the encoder, generates the weight of each block in the decoder immediately before the end of the encoding of the encoder, reduces the parameter quantity and the calculation amount of the network during prediction, utilizes a local connection layer mechanism to build a dynamic patch convolution in the decoder, fully considers semantic information of position coding when facing large segmentation objects crossing multiple patches, changes the weight of each convolution kernel in the decoder with the spatial position of an input feature map, utilizes dynamic weight to segment different objects, provides maximum adaptability of the network, utilizes a transfer learning method to pre-train the network in view of the problem that the low-altitude urban scene data set has fewer trainable images, and can effectively complete the real-time image semantic segmentation task of low-altitude unmanned aerial vehicle vision.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned aerial vehicle environment perception, and relates to a real-time image semantic segmentation method for low-altitude unmanned aerial vehicles. BACKGROUND

[0002] Unmanned aerial vehicle vision is an autonomous environment perception method based on computer vision, which mainly captures surrounding environment information through a camera and processes and analyzes the information to assist various decisions of the unmanned aerial vehicle. The unmanned aerial vehicle operates in the air and usually views the ground environment at an angle of 45° to 90°, performs segmentation on a ground object scene, and focuses on the ground environment information under the aerial view angle to perceive the position, size, state and the like of a to-be-segmented object in the current environment.

[0003] With the development of computer vision, deep learning and other technologies and the rapid improvement of hardware computing speed, more and more CNN architecture models are applied to image semantic segmentation tasks due to the strong scene adaptability and high automation of convolutional neural network. Shengke Wang combined superpixel segmentation and CNN classification to segment the accurate region of Ulva prolifera in very high resolution unmanned aerial vehicle images (Wang S, Liu L, Qu L, et al. Accurate Ulva prolifera regions extraction of UAV images with superpixel and CNNs for ocean environment monitoring [J]. Neurocomputing, 2019, 348(JUL.5): 158-168.); Keiller Nogueira used convolutional neural network to distinguish vegetation species with a small number of labeled samples (Nogueira K, Santos J, Cancian L, et al. Semantic segmentation of vegetation images acquired by unmanned aerial vehicles using an ensemble of ConvNets [C] / / IGARSS 2017-2017 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2017.); Jiankun Li used multi-scale skip connection network and Markov random field model to perform semantic segmentation on high-resolution aerial images (Jiankun, Li, Wenrui, et al. Semantic Segmentation for High-Resolution Aerial Imagery Using Multi-Skip Network and Markov Random Fields [C] / / 0.); Huasheng Huang realized the segmentation of weeds in grassland under unmanned aerial vehicle vision through improved FCN network (Huang H, Deng J, Lan Y, et al. A fully convolutional network for weed mapping of unmanned aerial vehicle (UAV) imagery [J]. PLoS ONE, 2018, 13(4): e0196302.).

[0004] Current research on UAV vision generally focuses on UAV remote sensing images, overemphasizing large objects such as buildings and roads while neglecting smaller targets from a near-ground perspective. This results in an inability to perceive detailed ground environmental information and a lack of understanding of the specific state of the ground scene. Furthermore, methods applying convolutional neural networks to image semantic segmentation often employ complex and redundant models to pursue higher accuracy. However, UAVs, limited by their low computing power and extremely high flight speed, have very high requirements for real-time performance in their algorithms. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the present invention aims to provide a real-time image semantic segmentation method for low-altitude unmanned aerial vehicles (UAVs). By combining hypernetworks and local connection layers, a smaller decoder is used to achieve higher accuracy, meeting the requirements of real-time segmentation. This method can better and faster adapt to UAV visual scenarios and achieve a good balance between accuracy and speed.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A real-time image semantic segmentation method for low-altitude unmanned aerial vehicles (UAVs) specifically includes the following steps:

[0008] Step 1: Dataset Selection: Select a visual image dataset for low-altitude drones;

[0009] Step 2: Perform data preprocessing on the dataset selected in Step 1;

[0010] 1) Data augmentation: The dataset obtained in step one is augmented by horizontal flipping and resizing to enhance the model's generalization ability;

[0011] 2) Scale transformation: The images in the dataset obtained in step 1) of step 2 are scaled to convert them to the network's default size;

[0012] 3) Divide the dataset: Divide the images obtained in step 2) of step two into a training set and a validation set according to the proportions;

[0013] Step 3: Based on the data processed in Step 2, construct an end-to-end hypernetwork encoder-decoder structure;

[0014] Step 4: Based on the network structure in Step 3, design a fixed encoding form based on the local connection layer structure, design positional encoding, and represent that the weights of the decoder are different in different spatial locations.

[0015] Step 5: Based on the network structure in Step 3, construct a weight generation module to enable the network to customize different segmentation processes according to the transformation of the input image;

[0016] Step 6: Combining Step 4 and Step 5, construct a dynamic piecewise convolution with dynamic weights by using additional spatial location encoding and downsampling features;

[0017] Step 7: Develop a transfer learning strategy and train the network to obtain the final real-time image semantic segmentation model for low-altitude UAV vision.

[0018] The specific method for step three is as follows:

[0019] 1) Use downsampling convolutional layers with a number of layers in the range of [3,5] as encoders and upsampling layers with a corresponding number as decoders to complete end-to-end pixel-level semantic segmentation; Based on the U-Net structure, use skip connections between the corresponding layers of the encoder and decoder to connect the hierarchical networks of different levels of encoders with corresponding levels of decoders.

[0020] 2) The encoder network uses a lightweight convolutional neural network, with the input image... Feature maps downsampled to different resolutions i∈[1,5], where C i H represents the number of channels in the i-th feature map, and H and W represent the number of pixels corresponding to the image height and width.

[0021] 3) To reduce the decoder size, an additional convolution is added, making each F... i The number of channels is reduced by r i ;

[0022] 4) The decoder consists of multiple decoding blocks, each containing an additional weight mapping network component w; the input to the decoder is the original image and the feature map F. i , and spatial location map P i The connections and weights are dynamically determined by the patches at each spatial location in the image;

[0023] The specific method for step four is as follows:

[0024] A local connection layer structure is introduced, and a fixed encoding method is adopted. At different positions (i,j) in the image The calculation method is as shown in equation (1):

[0025]

[0026] Given an input image and feature maps F1 to F2 n Decoder block m0 corresponds to the input image, m1 to m... n Corresponding feature maps F1 to F n After each unpacking, bilinear upsampling is performed and concatenated with the feature map of the same resolution from the previous level. This concatenation is then combined with the corresponding positions at the same resolution to encode P0 to P...n and the weight θ of de-blocking in the decoder mi outputting the segmentation prediction result where C is the number of classes of semantic segmentation;

[0027] The specific method of step five is:

[0028] 1) Introducing a super network structure to replace the last layer of the encoder network with a weight generation module h, the input and output of the parameter of the weight generation module remain the same, both are mapping the last feature map from the encoder network to a signal φ, and then sending the signal to the weight mapping network generating the weight θ of de-blocking in the decoder network mi ;

[0029] 2) The nested U-Net network structure is adopted in the module h, the convolution operation is adopted for down-sampling, and the number of output channels is half of the input, the feature map at the bottom layer is averaged and pooled in the down-sampling stage, the highest level context is extracted, and then it is up-sampled to the previous resolution using the nearest neighbor interpolation;

[0030] 3) In each level of the up-sampling path of h, the feature map thereof is connected with the up-sampled feature map corresponding thereto, and an all-connected layer is used at the end, so as to achieve the purpose of the same size of input and output;

[0031] 4) In the hierarchical network of the decoder, w is divided into multiple parts and embedded into each block of the decoder network, instead of being directly divided after weight generation and being transmitted to all blocks to wait for convolution operation;

[0032] 5) The weight θ of the encoder network, the weight generation module and the weight mapping network b , θ h , θ w is fixed in the inference process, is learned in the training process, and dynamically predicts the weight θ of de-blocking in the decoder in the inference process mi ;

[0033] 6) The weight generation module shares parameters within a channel group transmitted to the decoder network, and the number of groups g wi controls w according to formula (2) and formula (3) i is m i The trainable parameters and calculation amount invested in generating the weight;

[0034]

[0035]

[0036] The specific method of step six is:

[0037] 1) decoder in the de-blocking m i The MobileNetV2-based reverse residual block consists of a small convolution kernel pw1, a large depth convolution kernel dw, and another small convolution kernel pw2, without activation function;

[0038] 2) When the signal φ is input into each m i Convolution operation is performed with w i , the channel is divided into g wi groups and the weight is shared in each channel group, so that the input of dynamic slicing convolution is a feature map And a set of weight networks Where C in And C out are the number of input and output channels, g wi is the number of channel groups, H and W are the height and width of the input, K h And K w are the height and width of the convolution kernel, N h And N w is the number of patches along the height and width axes, and the output of each patch is as formula (4):

[0039] O i,j =X i,j *θ i,j (4)

[0040] Where i∈[0,N h ) and j∈[0,N w ) are patch indices, X i,j is the patch at image position (i,j), θ i,j is the corresponding weight in the weight network;

[0041] 3) The patch is applied to the entire input feature map X, and then the adjacent pixels in the patch X i,j at each position are combined to form a local connection layer with dynamic weights by combining additional spatial position encoding to emphasize the position information of the input image and feature map;

[0042] The specific method of the seventh step is:

[0043] In view of the problem that there are few trainable images in the low-altitude city scene data set, the method of transfer learning is used to load the city scene data set for pre-training;

[0044] 1) Select the same class as the low-altitude unmanned aerial vehicle vision data set from the image to be segmented class to generate image labels, and then perform end-to-end training on the entire network to obtain a network model;

[0045] 2) In the training stage, θ h and θ wThe network is initialized by taking random values in a normal distribution, trained using an adaptive optimizer, adopts a polynomial learning rate strategy, and after i iterations, the initial learning rate is lr0, and the decay Where t is the total number of iterations, and p is a scalar constant.

[0046] 3) The trained network model in step seven 1) is used for full training using low-altitude UAV visual data, and the parameters in the network are continuously optimized through the back propagation of gradients during the retraining process, to obtain a final real-time image semantic segmentation model for low-altitude UAV vision.

[0047] In the step three 3) of the method, r i The value is [1 / 2, 1 / 8], r i The value is preferably 1 / 4.

[0048] In the step six, the number of channel groups g wi The range is [2, 32], g wi The value is best (4, 8, 16, 16, 32).

[0049] In the step six, N h And N w The size is the same and the value is [4, 32], N h And N w The value is best 16 and 16.

[0050] In the step seven, the adaptive optimizer is preferably the Adam optimizer, the training parameter β1 of the Adam optimizer is best 0.5, β2 is best 0.999; the initial learning rate lr0 is best 10 -3 , and the decay is that the total number of iterations t is best 1440000 times, and the scalar constant p is best 0.9.

[0051] The beneficial effects of the present application are:

[0052] 1、The present application is aimed at the real-time image semantic segmentation task of low-altitude UAV vision under complex scene of inclination angle, a new encoder-decoder method is proposed by combining hypernetwork and local connection layer, wherein the encoder network is based on a lightweight encoding network in the field of image segmentation, and the encoding signal is mapped to the generated dynamic network weight of the decoder by the context weight generation module inside the U-Net network. The decoder is composed of dynamic blocks with changing weights with spatial position, and a local connection layer with dynamic weights is designed. And by using the transfer learning technology, the model can better and faster adapt to the UAV vision scene. The hot issue of difficult to balance "high precision" and "low delay" in the current low-altitude UAV vision related research is solved.

[0053] 2、For the real-time semantic segmentation scene of low-altitude vision of unmanned aerial vehicle, by using the super network and the local connection layer, the weight of the decoder depends not only on the input image, but also on the weight between different regions of the image, so that the network can customize different segmentation processes according to the input image transformation, and a good balance between accuracy and speed is achieved. Effectively combine low-level information from the network backbone with high-level information from the bottom, use a smaller decoder to achieve higher accuracy, thereby improving real-time performance. Ensure the real-time autonomous environmental perception of low-altitude unmanned aerial vehicle during task flight, make up for the problem of insufficient research on low-altitude unmanned aerial vehicle vision image of existing image segmentation technology, provide good data input for subsequent complex tasks of unmanned aerial vehicle, and have certain practical value. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 is a flowchart of the method of the present application.

[0055] Figure 2 is a super network encoder-decoder structure diagram of the present application.

[0056] Figure 3 is a network structure diagram of the context header weight generation module of the present application.

[0057] Figure 4 (a) is an internal network structure diagram of the de-blocking in the decoder of the present application.

[0058] Figure 4 (b) is a dynamic patch convolution structure diagram of the present application.

[0059] Figure 5 is a comparison diagram of segmentation results of different algorithms, wherein, Figure 5 (a) is an input image, Figure 5 (b) is an image label, Figure 5 (c) is an MSD segmentation result, Figure 5 (d) is an ABCNet segmentation result, Figure 5 (e) is a segmentation result of the present application. DETAILED DESCRIPTION

[0060] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0061] Referring to Figure 1 , a real-time image semantic segmentation method for low-altitude unmanned aerial vehicle, specifically comprising the following steps:

[0062] Step 1, select low-altitude unmanned aerial vehicle vision image dataset, preferably UAVId;

[0063] The data set is selected from the UAVId data set. The UAVId data set is used for semantic segmentation of low-altitude unmanned aerial vehicle urban scenes. The data set contains 420 images and 8 accurately labeled categories.

[0064] Step two, data preprocessing of the data set selected in step one

[0065] 1) Data augmentation: random size scaling with a ratio of [0.5, 2] and random horizontal flipping with a probability of 0.5;

[0066] 2) Scale transformation: scale the images in the data set obtained in step two 2) to 512x1024 pixels;

[0067] 3) Divide the data set: divide the images obtained in step two 2) into training and validation sets at a ratio of 8:2;

[0068] Step three, constructing the super network encoder-decoder

[0069] 1) The structure of the super network encoder-decoder is shown in Figure 2 . Four down-sampling convolutional layers are designed as the encoder and four up-sampling layers as the decoder to complete the end-to-end pixel-level semantic segmentation. Based on the idea of U-Net, a skip connection is used between the corresponding layers of the encoder and the decoder to connect the hierarchical networks of different levels of the encoder and the corresponding levels of the decoder;

[0070] 2) The encoder network uses EfficientNet, and the input image is down-sampled to feature maps of different resolutions i∈[1,5], where C i represents the number of channels of the i-th feature map, and H and W represent the number of pixels corresponding to the image height and width, which are 512 and 1024, respectively;

[0071] 3) To reduce the size of the decoder, an additional 1x1 convolution is added to reduce the number of channels of each F i by 1 / 4;

[0072] 4) The decoder is composed of multiple unblocks, and each unblock contains an additional weight mapping network component w. The input of the decoder d is the original image and the feature map F i , which is connected with the spatial position map P i , and the weight is dynamically determined by each spatial position patch in the image;

[0073] Step four, position encoding, which shows that the weights of the decoder are different at different spatial positions; a fixed encoding form is adopted by introducing the idea of local connection layer The weight of the decoder at different positions (i,j) in the image is The calculation method is as formula (1):

[0074]

[0075] Given the input image and feature maps F1 to F n , the decoder block m0 corresponds to the input image, m1 to m n correspond to the feature maps F1 to F n , and after each unblocking, bilinear up-sampling is performed, and the feature map of the same resolution is connected with the same resolution, combined with the corresponding position encoding P0 to P n and the decoder weight θ mi , the output segmentation prediction result is obtained where C is the number of semantic segmentation classes 8;

[0076] Step five, design the weight generation module

[0077] 1) Introduce the super network idea to replace the last layer of the encoder network with the weight generation module h, and the specific structure of the module h is shown in Figure 3 The input and output of the weight generation module parameters are consistent, both of which are The last feature map is mapped from the encoder network to the signal φ. Then the signal is sent to the weight mapping network to generate the weight θ mi of the unblocking in the decoder network

[0078] 2) The nested U-Net network structure is used inside the module h, 2x2 convolution with a step of 2 is used for down-sampling, and the number of output channels is half of the input. In the down-sampling stage, the feature map at the bottom is averaged pooled to extract the highest level of context, and then it is up-sampled to the previous resolution using nearest neighbor interpolation;

[0079] 3) In each level of the up-sampling path of h, its feature map is connected with the corresponding up-sampled feature map, and a fully connected layer is used at the end to achieve the same size of input and output;

[0080] 4) In the hierarchical network of the decoder, w is divided into multiple parts and embedded into each block of the encoder network, instead of being directly divided after weight generation and transmitted to all blocks for convolution operation;

[0081] 5) The weights θ b , θ h , θ w of the encoder network, the weight generation module and the weight mapping network are fixed in the inference process, learned in the training process, and dynamically predicted in the inference process. The weight θ mi of the unblocking in the decoder

[0082] 6) The weight generation module shares parameters within the channel group passed to the decoder network, the number of groups g wi According to formula (2), formula (3) controls w i For m i The trainable parameters and computational complexity invested in generating weights;

[0083]

[0084]

[0085] Step six, build a dynamic slicing convolution method

[0086] 1) Dynamic slicing convolution design reference Figure 4 , the de-blocking m i in the decoder Based on the MobileNetV2 residual block, composed of a 1x1 convolution pw1, a 3x3 deep convolution dw, and another 1x1 convolution pw2, without activation function;

[0087] 2) When the signal φ is passed into each m i w i 1x1 convolution is performed, the channels are divided into g wi groups and the weights are shared within each channel group, g wi ∈(g w1 ,g w5 ) takes the value (4,8,16,16,32), so that the input of the dynamic slicing convolution is a feature map and a group of weight networks Where C in and C out are the number of input and output channels, g wi is the number of channel groups, H and W are the height and width of the input, K h and K w are the height and width of the convolution kernel, N h and N w are the number of slices along the height and width axes 16 and 16, and the output of each patch is as formula (4):

[0088] O i,j =X i,j *θ i,j (4)

[0089] Where i∈[0,N h ) and j∈[0,N w ) are patch indices, X i,j is the patch at image position (i,j), θ i,j is the corresponding weight in the weight network;

[0090] 3) Apply patches to the entire input feature map X, and then apply patches to X at each location. i,j Adjacent pixels in the input image are combined, and the positional information of the input image and feature map is emphasized by combining additional spatial location encoding to form a local connection layer with dynamic weights.

[0091] Step 7: Develop a transfer learning strategy

[0092] To address the issue of limited trainable images in low-altitude urban scene datasets, we utilize transfer learning to pre-train on the Cityscapes dataset, which also features urban scenes.

[0093] 1) Using the Cityscapes dataset tool, select 8 classes from 19 categories to be segmented that are the same as those in the UAVId dataset and generate image labels. Then, train the entire network end-to-end to obtain the network model.

[0094] 2) Training phase targeting θ h and θ w The network is initialized with random values ​​taken from a normal distribution and trained using the Adam optimizer with training parameters β1 = 0.5 and β2 = 0.999. A multinomial learning rate strategy is used, and after i iterations, the initial learning rate lr0 = 10 is adjusted. -3 attenuation Where t is the total number of iterations, which is 1,440,000, and the scalar constant p is set to 0.9;

[0095] 3) The network model trained on the Cityscapes dataset is fully trained using UAVId data. During the retraining process, the parameters in the network are continuously optimized through backpropagation of gradients to obtain the final real-time image semantic segmentation model for low-altitude UAV vision.

[0096] To verify the beneficial effects of the present invention, the inventors conducted experiments on the UAVId dataset using the network model of the present invention. The experimental results are as follows.

[0097] 1. Evaluation Indicators

[0098] The model segmentation speed is measured by frames per second (FPS), and the model segmentation accuracy is measured by the mean intersection over union (mIoU).

[0099] mIoU is a standard metric for image semantic segmentation tasks. It is used to calculate the ratio (IoU) of the intersection and union of the ground truth set and the predicted value set for each class, and then calculate the average value to reflect the degree of overlap between the algorithm's predicted values ​​and the ground truth values. The formula for calculating mIoU is as shown in equation (1):

[0100]

[0101] k represents the number of categories, TP represents true positive; FP represents false positive; and FN represents false negative.

[0102] FPS calculates the time taken from data uploading to the GPU to prediction downloading, which is used to evaluate the algorithm speed, and the calculation method is shown in formula (2).

[0103]

[0104] wherein N represents the number of images, T j represents the time for the algorithm to process the jth image.

[0105] 2. Experimental conditions

[0106] The experimental environment setting is shown in Table 1.

[0107] Table 1 Experimental environment setting

[0108]

[0109] 3. Experimental method

[0110] The experiment compares and analyzes the segmentation accuracy and speed of the present application, MSD, BiMSANet and ABCNet on the UAVId dataset, and the resolution of the input image is 512*1024. Table 2 is the comparison of algorithm performance under the same experimental environment.

[0111] Table 2 Performance comparison of different algorithms on the UAVId dataset

[0112]

[0113] The experiment shows that the present application reduces 4.5% in inference accuracy compared with BiMSANet, but greatly improves 33.2 in inference speed, reaching the level of real-time segmentation, and is more suitable for real-time semantic segmentation task of low-altitude unmanned aerial vehicle. Compared with MSD and ABCNet, the method proposed in the present application improves the accuracy by 9.3% and 2.5% under the premise of ensuring real-time segmentation. Comprehensive analysis shows that the present application achieves a good balance between segmentation accuracy and segmentation speed, can realize efficient and accurate segmentation, and verifies the effectiveness of the super network structure of the present application.

[0114] In order to more intuitively show the superiority of the algorithm of the present application, part of the segmentation results of the present application and MSD and ABCNet algorithm are selected for visual processing, and the segmentation effect is compared and analyzed, and the results are shown in Figure 5 .

[0115] As can be seen from the comparison chart, when segmenting large continuous objects, the MSD and ABCNet methods will have erroneous segmentation and rough segmentation boundaries, while the present application is smooth and continuous in segmenting large objects. By introducing a local connection layer, the present application fully considers the contextual semantic information when facing large objects spanning multiple patches, and uses dynamic weights to segment different objects, so that the overall segmentation effect has less intra-class inconsistency and inter-class inconsistency. In summary, the present application can effectively complete the task of environment perception through semantic segmentation, and exhibits the effectiveness and superiority of the present application.

Claims

1. A real-time image semantic segmentation method for low-altitude unmanned aerial vehicles, characterized in that: Specifically comprising the following steps: Step one, dataset selection: select low-altitude unmanned aerial vehicle vision dataset; Step two, data preprocessing of the dataset selected in step one; 1) Data augmentation: expand the dataset obtained in step one by including horizontal flip and size scaling to enhance the generalization ability of the model; 2) Scale transformation: scale the images in the dataset obtained in step two 1) to convert the images to the default size of the network; 3) Divide the dataset: divide the images obtained in step two 2) into training set and validation set in proportion; Step three, according to the data processed in step two, build an end-to-end hypernetwork encoder-decoder structure; Step four, according to the network structure of step three, design a fixed encoding form based on local connection layer structure, design position encoding, and show that the weights of the decoder are different at different spatial positions; Step five, according to the super network encoder-decoder structure of step three, replace the last layer of the encoder network with a weight generation module , the weight generation module The input and output of the parameter remain the same, and the last feature map is mapped from the encoder network to a signal Then send the signal To the weight mapping network , generate the weight of the de-blocking in the decoder network Make the network be able to transform different segmentation processes according to the input image Step six, combined with step four and step five, a dynamic patch convolution with dynamic weights is constructed through additional spatial position coding and down-sampled features; including in the signal Deblocking in each decoder Time and weight mapping network Convolution operation, dividing the channel into groups and sharing weights within each channel group, so that the input of the dynamic patch convolution is a feature map and a group of weight networks; Step seven, develop a transfer learning strategy and train the network to obtain the final real-time image semantic segmentation model for low-altitude unmanned aerial vehicle vision.

2. The method of claim 1, wherein the method is a low-altitude unmanned aerial vehicle oriented real-time image semantic segmentation method. The specific method of step three is: 1) Use the down-sampling convolution layer with the number range of [3, 5] as the encoder and the corresponding number of up-sampling layers as the decoder to complete the end-to-end pixel-level semantic segmentation; based on the structure of U-Net, use the skip connection between the corresponding layers of the encoder and the decoder to connect the hierarchical network of different levels of encoder and decoder; 2) The encoder network uses a lightweight convolutional neural network, input image Downsampled to feature maps of different resolutions where denotes the number of channels of the th feature map, and represent the number of pixels corresponding to the image height and width, respectively. 3) To reduce the size of the decoder, add an extra convolution that reduces the number of channels of each of the previous layers by two. 4) The decoder consists of multiple unblocks, inside which there are additional weight mapping network components ; The input to the decoder is the original image and the feature map , and the spatial position map , and the connection, the weight is dynamically determined by the patch at each spatial position in the image.

3. The method of claim 1, wherein: The specific method of step four is: The local connection layer structure is introduced, and a fixed coding form is adopted at different positions in the image of The calculation method is as formula (1): Given input image and feature map to decoder block corresponding input image, to corresponding feature map to After each de-blocking, bilinear up-sampling is performed and connected with the feature map of the same resolution in the previous stage, combined with the corresponding position encoding of the same resolution to and the weight of de-blocking in the decoder output segmentation prediction result wherein is the number of classes of semantic segmentation.

4. The method of claim 1, wherein: The specific method of step five is: 1) Introducing a super network structure to replace the last layer of the encoder network with a weight generation module , the input and output of the weight generation module parameters remain the same, both are , map the last feature map from b to the signal , then send the signal to the weight mapping network , generate the weight of the de-blocking in the decoder network ; 2) Module The nested U-Net network structure is adopted inside, convolution operation is adopted for down-sampling, the number of output channels is half of the input, the feature map of the bottom layer is average-pooled in the down-sampling stage, the highest level context is extracted, and then it is up-sampled to the previous resolution by using the nearest neighbor interpolation. 3) in Each level in the up-sampling path concatenates its feature maps with the up-sampled feature maps corresponding to it, ending with a fully connected layer to achieve the same input and output size. 4) In the hierarchical network of decoders, the Instead of being divided into multiple parts and embedded into each block of the decoder network after weight generation, and passed into all blocks waiting for convolution operation, 5) the weights of the encoder network, the weight generation module, and the weight mapping network , , fixed during inference, learned during training, and dynamically predicted during inference ; 6) the weight generation module shares parameters within groups of channels passed to the decoder network, number of groups According to equation (2), equation (3) controls To Trainable parameters and computational effort invested in generating weights; 。 5. The method of claim 1, wherein: The specific method of step six is: 1) Deblocking in decoder MobileNetV2-based inverted residual block consisting of one small convolutional kernel , one large depthwise convolutional kernel , and another small convolutional kernel without activation function; 2) in the signal Incoming each Time and Convolution operation, the channel is divided into Group and share weights within each channel group, so that the input of dynamic slicing convolution is a feature map And a group of weight networks , Where And The number of input and output channels, The number of channel groups, And The height and width of the input, And The height and width of the convolution kernel, And The number of patches along the height and width axes, the output of each patch is as formula (4): wherein and is a patch index, is an image position of a patch, is a corresponding weight in the weight network; 3) patch is applied to the entire input feature map Then the patches at each position are combined with the neighboring pixels in By incorporating additional spatial position encodings, the position information of the input image and feature map is emphasized, forming a locally connected layer with dynamic weights.

6. The method of claim 1, wherein: The specific method of step seven is: To solve the problem of fewer trainable images in the low-altitude urban scene dataset, transfer learning method is used to load the urban scene dataset for pre-training; 1) Select the same class as the low-altitude unmanned aerial vehicle vision dataset from the image to be segmented category to generate image labels, and then train the entire network to obtain the network model; 2) Training phase aims at and Random values are taken in normal distribution to initialize the network, and the adaptive optimizer is used for training, and the training parameters are , ; the polynomial learning rate strategy is adopted, and after iterations, the initial learning rate is , and the decay is , where is the total number of iterations, is a scalar constant; 3) Use low-altitude unmanned aerial vehicle vision data to completely train the network model trained in step seven 1), and continuously optimize the parameters in the network during the retraining process through the back propagation of the gradient to obtain the final real-time image semantic segmentation model for low-altitude unmanned aerial vehicle vision.

7. The method of claim 2, wherein: in step three 3) above, takes the value 1 / 4.

8. The method of claim 5, wherein: In the step six, the number of channel groups The value is (4, 8, 16, 16, 32).

9. The method of claim 5, wherein: In the step six, the second step, and have the same size and take the values 16 and 16.

10. The method of claim 6, wherein, In the second step of the seventh step, the adaptive optimizer employs an Adam optimizer, the Adam optimizer trains parameters with a value of 0.5, with a value of 0.999; an initial learning rate of 10 -3 , a total number of iterations of 1440000, and a scalar constant of 0.9.

Images