High-altitude landing point identification method based on image semantic segmentation

Abstract

This invention relates to the field of image semantic segmentation technology, specifically to a high-speed descent point recognition method based on image semantic segmentation, including a model adaptive training stage and a real-time application inference stage. The model adaptive training stage includes the following steps: Step S1, acquiring a synthesized high-speed descent scene image and a real aerial descent scene image of the current frame, extracting initial features of the synthesized domain and initial features of the real domain, and obtaining pixel-level domain-aligned features through domain alignment processing. This invention filters pixels based on the connectivity, area, and boundary closure of the landing material pixels, eliminating scattered pixels, narrow strips, internal broken areas, and areas with insufficient area. It transforms the abstract pixel category results into the coordinates of closed flat polygon regions with clear vertex order, complete boundaries, and directly locatable ranges, thereby improving the stability and accuracy of high-speed descent point recognition in complex real-world environments.

Description

Technical Field

[0001] This invention relates to the field of image semantic segmentation technology, and in particular to a method for identifying high-speed landing points based on image semantic segmentation. Background Technology

[0002] Image semantic segmentation technology involves computer vision, digital image processing and pattern recognition. By determining the semantic category of each pixel in an image, pixels with the same semantic attributes are divided into corresponding category regions, thus forming pixel-level segmentation results with clear spatial boundaries and category labels.

[0003] Current image semantic segmentation typically relies on the pixel feature distribution formed in training images to directly determine the semantic category of each pixel position in the input image. When training scenarios and real-world application scenarios differ in terms of lighting intensity, ground material, shooting height, lens angle, and image sharpness, the pixel feature distribution is prone to shift, causing the same ground area to be classified differently in different images. For example, a flat grassy area in a synthetic image may be identified as a shrub area in a real aerial image due to shadow coverage or texture changes. Furthermore, current processing focuses on the color, texture, and contour relationships within a single frame, insufficiently utilizing the spatial displacement and semantic continuity between adjacent frames during continuous aerial photography. When the drone's attitude changes, personnel descend rapidly, or the camera shakes, the same ground area may repeatedly switch categories in consecutive frames, leading to abrupt segmentation boundaries. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a high-speed landing point recognition method based on image semantic segmentation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a high-speed landing point recognition method based on image semantic segmentation, comprising a model adaptive training stage and a real-time application inference stage; the model adaptive training stage includes the following steps: Step S1: Obtain the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image; extract the initial features of the synthesized domain and the initial features of the real domain; obtain pixel-level domain-aligned features through domain alignment processing; predict the initial features of the real domain to generate initial pseudo-labels. Step S2: Input the adjacent previous frame real aerial descent scene image and the current frame real aerial descent scene image into the optical flow estimation network to obtain the optical flow field; use the optical flow field to distort the pixel-level domain alignment features corresponding to the adjacent previous frame real aerial descent scene image to generate temporal distortion features; calculate the feature divergence residual between the temporal distortion features and the pixel-level domain alignment features corresponding to the current frame real aerial descent scene image; perform mask modulation on the pixel-level domain alignment features corresponding to the current frame real aerial descent scene image based on the feature divergence residual to output spatiotemporal consistency features; Step S3: Based on the spatiotemporal consistency features, obtain a scene cognition uncertainty probability map; extract the pixel coordinates within the scene cognition uncertainty probability map that meet the preset uncertainty limit conditions to generate a reliability mask matrix; use the reliability mask matrix to filter the spatiotemporal consistency features and the initial pseudo-labels to obtain low-uncertainty pure features and purified pseudo-labels.

[0006] Preferably, the model adaptive training phase further includes: Step S4: Perform derivative operations on the low-uncertainty pure features and the initial features of the synthetic domain respectively to extract the high-frequency edge feature maps of the real domain and the synthetic domain. Construct a high-frequency geometric consistency distillation loss based on the high-frequency edge feature maps of the real domain and the synthetic domain. Combine the high-frequency geometric consistency distillation loss, the real domain self-supervised cross-entropy loss constructed based on the purified pseudo-labels, and the synthetic domain supervised loss constructed based on the truth labels to calculate the global joint loss function and update the network parameters to obtain the high-altitude descent semantic segmentation model.

[0007] Preferably, the real-time application inference phase includes the following steps: Step S5: Input the collected single-frame real aerial descent scene image into the high-altitude descent semantic segmentation model and output a pixel-level semantic segmentation result image; extract the pixel set corresponding to the safe landing material from the pixel-level semantic segmentation result image, perform connected component morphological analysis, output the coordinates of a closed flat polygon region with an area greater than the preset landing point limit, and obtain the high-altitude descent point identification result.

[0008] Preferably, the step of obtaining the initial pseudo-label is as follows: Acquire synthetic high-altitude descent scene images with pixel-level ground truth labels, verify the pixel coordinates, semantic category numbers, and pixel-level ground truth label correspondence of the synthetic high-altitude descent scene images frame by frame, acquire unlabeled continuous frames of real aerial descent scene images arranged according to continuous acquisition time, select the current frame of real aerial descent scene images from the unlabeled continuous frames of real aerial descent scene images, unify the pixel size, channel arrangement order, and pixel value range of the synthetic high-altitude descent scene images and the current frame of real aerial descent scene images, and maintain the pixel-level ground truth labels and the pixel positions of the synthetic high-altitude descent scene images point by point to form a scene image input combination; Based on the scene image input combination, the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image are synchronously input into the feature extraction network according to the same batch position. The convolution output of the same level in the feature extraction network is called. The feature response values ​​of the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image are recorded according to the pixel row coordinates, pixel column coordinates, and feature channel numbers, respectively. The feature response values ​​of the synthesized high-altitude descent scene image are organized into the initial features of the synthesis domain, and the feature response values ​​of the current frame's real aerial descent scene image are organized into the initial features of the real domain, thus obtaining the initial features of the synthesis domain and the initial features of the real domain. The initial features of the synthetic domain and the initial features of the real domain are input into the gradient inversion layer. The real domain discrimination probability obtained by inputting the initial features of the real domain into the domain discriminator and the synthetic domain discrimination probability obtained by inputting the initial features of the synthetic domain into the domain discriminator are compared batch by batch. The difference values ​​of the two probabilities distributed in the global feature latent space are accumulated. The feature extraction network parameters are adjusted in reverse according to the difference values. The feature extraction, domain discrimination, difference value calculation and parameter adjustment are repeated until the change amplitude of the difference values ​​between two adjacent times no longer exceeds the preset convergence limit. The pixel-level domain alignment features after the domain environment interference is weakened are retained. At the same time, the initial features of the real domain are input into the pre-trained classifier. The prediction probability corresponding to each semantic category is read pixel by pixel. The maximum prediction probability and the semantic category number corresponding to the maximum prediction probability are selected for each pixel position. Pixel positions with the maximum prediction probability greater than or equal to the preset confidence limit are assigned the corresponding semantic category number. Pixel positions with the maximum prediction probability less than the preset confidence limit are assigned an invalid label. All semantic category numbers and invalid labels are rearranged according to the pixel coordinates of the real aerial descent scene image in the current frame to generate initial pseudo labels.

[0009] Preferably, the step of obtaining the characteristic divergence residual is as follows: Read the previous and current frames of real aerial descent scene images in the order of continuous acquisition time. Check the number of pixel rows, number of pixel columns, and channel arrangement order of the previous and current frames of real aerial descent scene images. Keep the same pixel coordinates corresponding to the same spatial position. Input the previous and current frames of real aerial descent scene images into the optical flow estimation network synchronously. Determine the corresponding pixel position of the current frame pixel in the previous frame pixel by pixel. Record the coordinate offset of the corresponding pixel position along the pixel row direction and the coordinate offset along the pixel column direction. Arrange the coordinate offsets of all pixel positions according to the pixel coordinates of the current frame to obtain the optical flow field. Read the pixel-level domain alignment features corresponding to the previous frame of real aerial rappelling scene image. According to the pixel row direction coordinate offset and pixel column direction coordinate offset of each pixel position in the optical flow field, reverse locate the source coordinates in the pixel-level domain alignment features corresponding to the previous frame of real aerial rappelling scene image. When the source coordinates fall on an integer pixel position, read the corresponding feature value. When the source coordinates fall between adjacent integer pixel positions, allocate the value ratio according to the distance from the source coordinates to the surrounding pixel positions. Rearrange the reading results according to the pixel coordinates of the current frame to generate temporal distortion features. Read the temporal distortion features and the pixel-level domain alignment features of the current frame of real aerial rappelling scene image pixel by pixel. Calculate the response difference of the same pixel position and the same feature channel. Summarize the response difference of all feature channels corresponding to each pixel position to obtain the feature divergence residual.

[0010] Preferably, the steps for obtaining the spatiotemporal consistency feature are as follows: Based on the feature divergence residual, the residual value of each pixel position is read item by item according to the pixel coordinates of the current frame. Each residual value is input into a negative exponential mapping function. When the residual value increases, the weight value of the corresponding pixel position is reduced, and when the residual value decreases, the weight value of the corresponding pixel position is increased. The weight values ​​of all pixel positions are restricted to the range of zero to one. All weight values ​​are arranged according to the pixel coordinates of the current frame, and the pixel-level domain alignment features of the real aerial descent scene image of the current frame are read pixel by pixel. At the same pixel position, the weight value is multiplied into all feature channels respectively. For pixel positions where the residual value exceeds the preset residual judgment limit, the feature response amplitude is reduced. For pixel positions where the residual value does not exceed the preset residual judgment limit, the corresponding feature response is retained. The weighted feature responses are arranged according to the original number of pixel rows, number of pixel columns, and number of feature channels to obtain the spatiotemporal consistency features.

[0011] Preferably, the steps for obtaining the scene cognition uncertainty probability map are as follows: The spatiotemporal consistency features are input into a semantic classifier based on evidence deep learning according to pixel row coordinates, pixel column coordinates, and feature channel numbers. The feature activation outputs generated by the classification output layer for each semantic category at each pixel position are read. Negative feature activation outputs are truncated to zero values, and non-negative feature activation outputs are converted into evidence quantity parameters for the corresponding semantic category. The evidence quantity parameters of all semantic categories are summarized according to each pixel position. The category prior is superimposed on each evidence quantity parameter to form the Dirichlet distribution parameters corresponding to each pixel position. The proportion of the number of semantic categories relative to the total number of Dirichlet distribution parameters is calculated. The proportion values ​​are arranged according to the original pixel coordinates to obtain the scene cognition uncertainty probability map.

[0012] Preferably, the steps for obtaining the low-uncertainty purity feature and the corresponding purified pseudo-label are as follows: Based on the scene cognitive uncertainty probability map, the scene cognitive uncertainty probability value corresponding to each pixel coordinate is read row by row. The scene cognitive uncertainty probability value is compared with the preset uncertainty limit item by item. The pixel coordinates whose scene cognitive uncertainty probability value is less than the preset uncertainty limit are retained and written into the pixel coordinate set. The pixel coordinates whose scene cognitive uncertainty probability value is greater than or equal to the preset uncertainty limit are excluded from the pixel coordinate set. A matrix of the same size is established according to the number of pixel rows and the number of pixel columns of the scene cognitive uncertainty probability map. The pixel positions in the pixel coordinate set are assigned valid labels and the remaining pixel positions are assigned invalid labels to generate a reliability mask matrix. Verify the pixel row coordinates and pixel column coordinates of the reliability mask matrix, the spatiotemporal consistency feature, and the initial pseudo-label. Extend the valid identifiers in the reliability mask matrix to all feature channels of the pixel positions corresponding to the spatiotemporal consistency feature. Retain the feature response values ​​of the pixel positions corresponding to the valid identifiers, set the feature response values ​​of the pixel positions corresponding to the invalid identifiers to zero, retain the semantic category numbers in the initial pseudo-labels according to the same pixel coordinates, replace the semantic category numbers of the pixel positions corresponding to the invalid identifiers with invalid labels, and maintain the pixel-by-pixel correspondence between the retained feature response values ​​and the retained semantic category numbers to obtain low-uncertainty pure features and their corresponding purified pseudo-labels.

[0013] Preferably, the steps for obtaining the high-altitude descent semantic segmentation model are as follows: Read the low-uncertainty pure features and the initial features of the synthetic domain, verify the pixel row coordinates, pixel column coordinates and feature channel numbers, establish the spatial correspondence between the center pixel position and the upper, lower, left and right pixel positions in each feature channel, calculate the second-order response change of the center pixel position along the pixel row direction, calculate the second-order response change of the center pixel position along the pixel column direction, superimpose the second-order response changes in the two directions pixel by pixel, retain the pixel positions where the absolute value of the response change exceeds the preset edge response limit, suppress the pixel positions where the absolute value of the response change does not exceed the preset edge response limit, arrange the retained response changes according to the original pixel coordinates and feature channel numbers, and obtain the high-frequency edge feature map of the real domain and the high-frequency edge feature map of the synthetic domain. Based on the high-frequency edge feature maps of the real domain and the synthetic domain, the latent space mapping responses of all pixel positions are statistically analyzed according to the same feature channel. The mean of the real domain mapping and the mean of the synthetic domain mapping are calculated for each feature channel. The difference between the mean mappings of each feature channel is accumulated to obtain the maximum mean difference metric. A high-frequency geometric consistency distillation loss is constructed based on the maximum mean difference metric. The semantic category number retained in the purified pseudo-label is read, and the real domain category prediction probability of the corresponding pixel position is called. The negative logarithm of the prediction probability corresponding to the semantic category number is accumulated to obtain the real domain self-supervised cross-entropy loss. The ground truth label of the synthetic high-altitude descent scene image is read, and the synthetic domain category prediction probability of the corresponding pixel position is called. The negative logarithm of the prediction probability corresponding to the ground truth label is accumulated to obtain the synthetic domain supervised loss. The numerical proportions of the high-frequency geometric consistency distillation loss, the real domain self-supervised cross-entropy loss, and the synthetic domain supervised loss are adjusted according to the preset loss weights. The adjusted loss values ​​are accumulated to obtain the global joint loss function. Based on the global joint loss function, the partial derivative change of the global joint loss function with respect to each network parameter is calculated layer by layer according to the parameter propagation direction of the semantic classifier, feature extraction network, and domain discriminator. The partial derivative change of each network parameter is propagated in reverse according to the corresponding parameter position. Each network parameter is corrected according to the preset parameter update step size. The synthesized high-altitude descent scene image and continuous frames of real aerial descent scene images are re-inputted. Feature extraction, semantic prediction, loss calculation, and parameter correction are repeatedly performed. The value of the global joint loss function corresponding to each training round is recorded. When the change amplitude of the global joint loss function value in multiple consecutive rounds is less than the preset convergence limit, parameter correction is stopped. The feature extraction network parameters and semantic classifier parameters at the time of stopping parameter correction are retained to obtain the high-altitude descent semantic segmentation model.

[0014] Preferably, the steps for obtaining the high-speed landing point identification result are as follows: A single-frame real-time aerial rappelling scene image captured by a drone or first-person perspective device is acquired. The number of pixel rows, number of pixel columns, channel arrangement order, and pixel values ​​of each channel in the single-frame real-frame real-frame aerial rappelling scene image are read. The pixel coordinate correspondence is adjusted according to the input size of the high-altitude rappelling semantic segmentation model. The pixel values ​​of each channel are rearranged according to the input channel order of the high-altitude rappelling semantic segmentation model. The adjusted single-frame real-frame aerial rappelling scene image is input into the high-altitude rappelling semantic segmentation model. The output value corresponding to each semantic category is read pixel by pixel. The semantic category number with the largest output value at each pixel position is selected. All semantic category numbers are arranged according to the original pixel row coordinates and the original pixel column coordinates to obtain a pixel-level semantic segmentation result image. Based on the pixel-level semantic segmentation result image, the semantic category number is read pixel by pixel. Pixel positions with semantic category numbers belonging to the safe landing material category are assigned valid identifiers, and pixel positions corresponding to other semantic category numbers are assigned invalid identifiers. All valid and invalid identifiers are arranged according to pixel row coordinates and pixel column coordinates to form a binary image of safe landing material. Starting from the first valid identifier in the binary image of safe landing material, the adjacent pixel positions are retrieved. Valid identifiers that are continuously connected in the horizontal, vertical, or diagonal directions are grouped into the same connected region. The adjacent pixel position retrieval is repeated for valid identifiers that have not yet been grouped into a connected region. All pixel coordinates corresponding to each connected region are recorded. Invalid identifiers that are completely surrounded by the same connected region are filled in, and isolated valid identifiers that cannot form a continuous boundary are deleted to obtain the connected regions of safe landing material. The number of effective pixels corresponding to each connected region of the safe landing material is counted, and the number of effective pixels is converted into the region area. The region area is compared with the preset landing point limit item by item. The connected regions of the safe landing material with a region area greater than the preset landing point limit are retained, and the connected regions of the safe landing material with a region area less than or equal to the preset landing point limit are deleted. The outermost boundary pixel coordinates of each retained region are traced in a clockwise direction. The connection status of the first and last pixels of the outermost boundary pixel coordinates is checked. The retained regions with broken boundaries are deleted. The polygon vertices are determined according to the pixel positions where the boundary direction changes. Adjacent polygon vertices are connected to form a closed boundary. The continuity of effective pixels and the distribution of internal holes inside the closed boundary are checked. Closed flat polygon regions with continuous effective pixels and no unfilled holes are retained. The coordinates of the closed flat polygon regions are output according to the order of polygon vertices to obtain the high-speed landing point identification result.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: This invention extracts features simultaneously from synthetic high-altitude descent scene images and real aerial descent scene images, and continuously adjusts pixel representations based on the differences in feature distributions between the two types of scenes. This shifts the semantic judgment criteria formed during training from color, texture, and background features to ground category, contour structure, and spatial morphology, thereby reducing the impact of environmental differences between synthetic and real scenes on recognition results. Furthermore, by utilizing the pixel coordinate offset relationship between adjacent frames, pixel-level domain alignment features of the corresponding ground region in the previous frame are mapped to the current frame position. The feature response of the current frame is adjusted based on the difference between the mapped features and the pixel-level domain alignment features of the current frame, suppressing abnormal responses caused by flight attitude changes, image jitter, motion blur, and short-term occlusion, thus maintaining a relatively stable semantic category for the same ground region in consecutive frames. Finally, features and pseudo-labels that can participate in training are selected based on the cognitive uncertainty of each pixel position, preventing pixels with insufficient category criteria, blurred boundaries, or high levels of interference from entering subsequent parameter updates, reducing the impact of these factors. The continuous influence of erroneous pseudo-labels on the distribution of real-world scene features prevents the accumulation and spread of local misjudgments during multiple training rounds. Spatial variation extraction is performed on the filtered real-world and synthetic scene features, which can concentrate training constraints on ground boundaries, regional transitions, obstacle outlines, and the outer edges of flat areas. Through joint constraints of geometric differences, real-world semantic errors, and synthetic scene supervision errors, semantic category determination, real-world environment adaptation, and regional boundaries are kept in synergistic convergence during the same parameter update process. After the above continuous processing, the real-time input single-frame real aerial descent scene image can form a pixel-level semantic segmentation result with continuous boundaries and stable categories. Then, by filtering the connectivity of safe landing material pixels, regional area, and boundary closure status, scattered pixels, narrow strips, internal broken areas, and areas with insufficient area can be excluded. The abstract pixel category result is transformed into the coordinates of a closed flat polygon region with a clear vertex order, complete boundaries, and directly locatable range, thereby improving the stability and accuracy of high-speed descent point recognition in complex real-world environments. Attached Figure Description

[0016] Figure 1 A graph showing the numerical change of the adversarial loss function; Figure 2 This is a probability diagram of uncertainty in scene cognition. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] Please see Figure 1-2This invention provides a technical solution for high-speed landing point recognition based on image semantic segmentation, comprising a model adaptive training phase and a real-time application inference phase; the model adaptive training phase includes the following steps: Step S1: Obtain the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image; extract the initial features of the synthesized domain and the initial features of the real domain; obtain pixel-level domain-aligned features through domain alignment processing; predict the initial features of the real domain to generate initial pseudo-labels. Step S2: Input the adjacent previous frame real aerial descent scene image and the current frame real aerial descent scene image into the optical flow estimation network to obtain the optical flow field; use the optical flow field to distort the pixel-level domain alignment features corresponding to the adjacent previous frame real aerial descent scene image to generate temporal distortion features; calculate the feature divergence residual between the temporal distortion features and the pixel-level domain alignment features corresponding to the current frame real aerial descent scene image; perform mask modulation on the pixel-level domain alignment features corresponding to the current frame real aerial descent scene image based on the feature divergence residual to output spatiotemporal consistency features; Step S3: Based on the spatiotemporal consistency features, obtain a scene cognition uncertainty probability map; extract the pixel coordinates within the scene cognition uncertainty probability map that satisfy the preset uncertainty limit condition to generate a reliability mask matrix; use the reliability mask matrix to filter the spatiotemporal consistency features and the initial pseudo-labels to obtain low-uncertainty pure features and purified pseudo-labels; Step S4: Perform derivative operations on the low-uncertainty pure features and the initial features of the synthetic domain respectively to extract the high-frequency edge feature maps of the real domain and the synthetic domain. Construct a high-frequency geometric consistency distillation loss based on the high-frequency edge feature maps of the real domain and the synthetic domain. Combine the high-frequency geometric consistency distillation loss, the real domain self-supervised cross-entropy loss constructed based on the purified pseudo-labels, and the synthetic domain supervised loss constructed based on the truth labels to calculate the global joint loss function and update the network parameters to obtain the high-altitude descent semantic segmentation model. The real-time application inference phase includes the following steps: Step S5: Input the collected single-frame real aerial descent scene image into the high-altitude descent semantic segmentation model and output a pixel-level semantic segmentation result image; extract the pixel set corresponding to the safe landing material from the pixel-level semantic segmentation result image, perform connected component morphological analysis, output the coordinates of a closed flat polygon region with an area greater than the preset landing point limit, and obtain the high-altitude descent point identification result.

[0019] The steps for obtaining the initial pseudo-tags are as follows: Acquire synthetic high-altitude descent scene images with pixel-level ground truth labels, verify the pixel coordinates, semantic category numbers, and pixel-level ground truth label correspondence of the synthetic high-altitude descent scene images frame by frame, acquire unlabeled continuous frames of real aerial descent scene images arranged according to continuous acquisition time, select the current frame of real aerial descent scene images from the unlabeled continuous frames of real aerial descent scene images, unify the pixel size, channel arrangement order, and pixel value range of the synthetic high-altitude descent scene images and the current frame of real aerial descent scene images, and maintain the pixel-level ground truth labels and the pixel positions of the synthetic high-altitude descent scene images point by point to form a scene image input combination; Based on the scene image input combination, the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image are synchronously input into the feature extraction network according to the same batch position. The convolution output of the same level in the feature extraction network is called. The feature response values ​​of the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image are recorded according to the pixel row coordinates, pixel column coordinates, and feature channel numbers, respectively. The feature response values ​​of the synthesized high-altitude descent scene image are organized into the initial features of the synthesis domain, and the feature response values ​​of the current frame's real aerial descent scene image are organized into the initial features of the real domain, thus obtaining the initial features of the synthesis domain and the initial features of the real domain. The initial features of the synthetic domain and the initial features of the real domain are input into the gradient inversion layer. The real domain discrimination probability obtained by inputting the initial features of the real domain into the domain discriminator and the synthetic domain discrimination probability obtained by inputting the initial features of the synthetic domain into the domain discriminator are compared batch by batch. The difference values ​​of the two probabilities distributed in the global feature latent space are accumulated. The feature extraction network parameters are adjusted in reverse according to the difference values. The feature extraction, domain discrimination, difference value calculation and parameter adjustment are repeated until the change amplitude of the difference values ​​between two adjacent times no longer exceeds the preset convergence limit. The pixel-level domain alignment features after the domain environment interference is weakened are retained. At the same time, the initial features of the real domain are input into the pre-trained classifier. The prediction probability corresponding to each semantic category is read pixel by pixel. The maximum prediction probability and the semantic category number corresponding to the maximum prediction probability are selected for each pixel position. Pixel positions with the maximum prediction probability greater than or equal to the preset confidence limit are assigned the corresponding semantic category number. Pixel positions with the maximum prediction probability less than the preset confidence limit are assigned an invalid label. All semantic category numbers and invalid labels are rearranged according to the pixel coordinates of the real aerial descent scene image in the current frame to generate initial pseudo labels.

[0020] Specifically, the process involves acquiring composite high-altitude descent scene images with pixel-level ground truth labels. This is achieved by generating 100,000 images with a resolution of 1920×1080 using Unreal Engine and an aerial photography plugin. Each image is accompanied by a pixel-level semantic label. The correspondence between the pixel coordinates, semantic category number, and pixel-level ground truth label of each composite high-altitude descent scene image is verified. Verification includes confirming that the label image size matches the image size and that the category numbers in the label image fall within eight predefined categories (e.g., 0: Sky, 1: Buildings, 2: Vegetation, 3: Roads, 4: Water, 5: Rocks, 6: Sand, 7: Flat Grass / Safe Landing Zone). The acquisition is based on continuous sampling... This dataset contains unlabeled, continuous frames of real-world aerial descent scenes arranged in time. Specifically, it extracts consecutive, uncompressed image frames from multiple video clips recorded during a drone's high-speed descent. The current frame of the real-world aerial descent scene is selected. To adapt to network input, the pixel dimensions of both the synthesized high-altitude descent scene image and the current frame of the real-world aerial descent scene image are uniformly adjusted to 1024×512 using a bicubic interpolation algorithm. The channel order is also uniformly converted from BGR (OpenCV default) to RGB. Subsequently, the pixel value range is linearly normalized, mapping integer pixel values ​​in the [0, 255] interval to floating-point values ​​in the [0.0, 1.0] interval. The normalization formula is as follows: ,in These are the original pixel values. The normalized pixel values ​​are processed to maintain the pixel-level ground truth labels and the corresponding pixel positions of the synthesized high-altitude descent scene images, forming a scene image input combination that can be directly input into the network.

[0021] Based on the aforementioned scene image input combination, a high-altitude descent scene image will be synthesized. and the current frame of real aerial descent scene image A shared-weight feature extraction network is synchronously fed into the same batch of images. This network uses a ResNet-101 architecture as its backbone, but removes its last global average pooling layer and fully connected layer to preserve spatial feature maps. The convolutional output of the fourth residual block (conv4_x) in the feature extraction network is used. The feature map output by this layer is 1 / 16 of the input image in spatial dimension, according to pixel row coordinates. Pixel column coordinates Feature channel number The feature response values ​​of the synthesized high-altitude descent scene image and the feature response value of the current frame's real aerial descent scene image are recorded respectively. These response values ​​are then organized into two four-dimensional tensors, namely the initial features of the synthesis domain. and initial features of the real domain ,in For batch size, This represents the number of feature channels (e.g., 1024). and The height and width of the feature map are used to obtain the initial features of the synthetic domain and the initial features of the real domain.

[0022] The initial features of the synthesis domain and the initial features of the real domain Input a gradient inversion layer (GRL), which performs an identity transformation during forward propagation and multiplies the gradient by a negative constant during backward propagation. The discriminant performs batch-by-batch comparisons of the ground domain discrimination probabilities obtained by feeding the initial ground domain features into the discriminant and the synthetic domain discrimination probabilities obtained by feeding the initial synthetic domain features into the discriminant. The discriminant consists of three convolutional layers, each followed by a LeakyReLU activation function, and finally outputs a single-channel probability map with an adversarial loss function. The binary cross-entropy loss is used for calculation, and the formula is as follows: in, It is a feature extraction network. It is a domain discriminator. and They are the first The first synthetic image and the first A real image, and These represent the number of synthetic and real images in the batch, respectively. The domain discriminator is trained by minimizing this loss. The feature extractor is trained by maximizing this loss (via GRL). The difference between two probability distributions in the global feature latent space is accumulated. Based on the difference, the parameters of the feature extraction network are adjusted in reverse. This process of feature extraction, domain discrimination, difference calculation, and parameter adjustment is repeated until the result is achieved in five consecutive iterations. If the average value of the variation is lower than the preset convergence limit (e.g., 1e-5), the pixel-level domain alignment features after the domain environment interference is weakened are retained. At the same time, the initial features of the real domain are fed into the head of a classifier pre-trained on ImageNet, and the prediction probability corresponding to each semantic category is read pixel by pixel. The maximum prediction probability at each pixel position is selected. and the corresponding semantic category number Set a confidence limit. For example, this limit is determined by the 95th percentile of the predicted probability distribution of statistical synthetic domain data in the early stages of training. Then assign the corresponding semantic category number to the pixel position. Otherwise, assign an invalid identifier (e.g., -1), rearrange all semantic category numbers and invalid identifiers according to the pixel coordinates of the current frame's real aerial descent scene image, and generate initial pseudo-labels.

[0023] The steps for obtaining the characteristic divergence residual are as follows: Read the previous and current frames of real aerial descent scene images in the order of continuous acquisition time. Check the number of pixel rows, number of pixel columns, and channel arrangement order of the previous and current frames of real aerial descent scene images. Keep the same pixel coordinates corresponding to the same spatial position. Input the previous and current frames of real aerial descent scene images into the optical flow estimation network synchronously. Determine the corresponding pixel position of the current frame pixel in the previous frame pixel by pixel. Record the coordinate offset of the corresponding pixel position along the pixel row direction and the coordinate offset along the pixel column direction. Arrange the coordinate offsets of all pixel positions according to the pixel coordinates of the current frame to obtain the optical flow field. Read the pixel-level domain alignment features corresponding to the previous frame of real aerial rappelling scene image. According to the pixel row direction coordinate offset and pixel column direction coordinate offset of each pixel position in the optical flow field, reverse locate the source coordinates in the pixel-level domain alignment features corresponding to the previous frame of real aerial rappelling scene image. When the source coordinates fall on an integer pixel position, read the corresponding feature value. When the source coordinates fall between adjacent integer pixel positions, allocate the value ratio according to the distance from the source coordinates to the surrounding pixel positions. Rearrange the reading results according to the pixel coordinates of the current frame to generate temporal distortion features. Read the temporal distortion features and the pixel-level domain alignment features of the current frame of real aerial rappelling scene image pixel by pixel. Calculate the response difference of the same pixel position and the same feature channel. Summarize the response difference of all feature channels corresponding to each pixel position to obtain the feature divergence residual.

[0024] Specifically, the images of the previous consecutive aerial descent scene are read in chronological order according to the continuous acquisition time. and the current frame of real aerial descent scene image The number of pixel rows, columns, and channel arrangement of the two images are verified to be 1024×512 and RGB, respectively. The same pixel coordinates correspond to the same spatial location. These two images are then simultaneously input into a pre-trained optical flow estimation network, such as PWC-Net. This network constructs a feature pyramid, calculates correlations at different scales, and progressively optimizes optical flow to determine the current frame's pixels pixel by pixel. The corresponding pixel position in the adjacent previous frame And record the coordinate offset of the corresponding pixel position along the pixel row direction. and coordinate offset along the pixel column direction The coordinate offsets of all pixel positions are arranged according to the pixel coordinates of the current frame and combined into a two-dimensional vector field to obtain the optical flow field. ,in and This represents the height and width of the image.

[0025] Read the pixel-level domain alignment features corresponding to the previous frame of the real aerial descent scene image. According to the optical flow field Each pixel position coordinate offset Reverse localization of features from the previous frame Source coordinates in Since the source coordinates are usually non-integer, when the source coordinates fall between adjacent integer pixel positions, the bilinear interpolation method is used to allocate the value ratio based on the distance from the source coordinates to the four surrounding integer pixel positions. The calculation formula is as follows: in, These are the interpolated eigenvalues. The previous frame at integer coordinates The feature values ​​are used to rearrange the read results of all positions according to the pixel coordinates of the current frame, generating temporal distortion features. Subsequently, the temporal distortion features and the pixel-level domain alignment features of the current frame's real aerial descent scene image are read pixel by pixel. Calculate the response difference for the same pixel location and the same feature channel, and summarize the results for each pixel location. All correspondences The L1 norm of the response differences of each feature channel is used to obtain the feature divergence residual map. .

[0026] The steps for obtaining the spatiotemporal consistency feature are as follows: Based on the feature divergence residual, the residual value of each pixel position is read item by item according to the pixel coordinates of the current frame. Each residual value is input into a negative exponential mapping function. When the residual value increases, the weight value of the corresponding pixel position is reduced, and when the residual value decreases, the weight value of the corresponding pixel position is increased. The weight values ​​of all pixel positions are restricted to the range of zero to one. All weight values ​​are arranged according to the pixel coordinates of the current frame, and the pixel-level domain alignment features of the real aerial descent scene image of the current frame are read pixel by pixel. At the same pixel position, the weight value is multiplied into all feature channels respectively. For pixel positions where the residual value exceeds the preset residual judgment limit, the feature response amplitude is reduced. For pixel positions where the residual value does not exceed the preset residual judgment limit, the corresponding feature response is retained. The weighted feature responses are arranged according to the original number of pixel rows, number of pixel columns, and number of feature channels to obtain the spatiotemporal consistency features.

[0027] Specifically, based on the characteristic divergence residual map Read the residual value of each pixel position item by item according to the pixel coordinates of the current frame. Each residual value is then input into a negative exponential mapping function to generate weights, the formula of which is: ,in It is a positive scaling factor used to control the sensitivity of the residuals to the weights. Set as residual plot The reciprocal of the average of all pixel residual values, i.e. As the residual value increases, the absolute value of the exponent term increases, causing the weight value to decrease exponentially. When the residual value decreases, the weight value approaches 1. The weight values ​​of all pixel positions are restricted to the range of zero to one, and all weight values ​​are arranged according to the pixel coordinates of the current frame to form a weight matrix. It reads pixel-level domain alignment features of the current frame's real aerial descent scene image, pixel by pixel. At the same pixel location, the weight values ​​are multiplied element-wise into all feature channels. This effectively suppresses the feature response amplitude of pixel locations where the residual value exceeds a preset residual threshold (e.g., locations where the residual value is greater than 1.5 times the average of all values ​​in the residual image plus one standard deviation). For pixel locations where the residual value does not exceed this threshold, most of the feature response is retained, and the pixel rows are arranged as originally. Number of pixel columns Number of feature channels The spatiotemporal consistency features are obtained by arranging and weighting the feature responses. ,in This indicates element-wise multiplication.

[0028] The steps for obtaining the scene cognition uncertainty probability map are as follows: The spatiotemporal consistency features are input into a semantic classifier based on evidence deep learning according to pixel row coordinates, pixel column coordinates, and feature channel numbers. The feature activation outputs generated by the classification output layer for each semantic category at each pixel position are read. Negative feature activation outputs are truncated to zero values, and non-negative feature activation outputs are converted into evidence quantity parameters for the corresponding semantic category. The evidence quantity parameters of all semantic categories are summarized according to each pixel position. The category prior is superimposed on each evidence quantity parameter to form the Dirichlet distribution parameters corresponding to each pixel position. The proportion of the number of semantic categories relative to the total number of Dirichlet distribution parameters is calculated. The proportion values ​​are arranged according to the original pixel coordinates to obtain the scene cognition uncertainty probability map.

[0029] Specifically, the spatiotemporal consistency feature Inputting pixel row coordinates, pixel column coordinates, and feature channel numbers into an evidence-based deep learning semantic classifier, the classifier consists of two 3×3 convolutional layers and one 1×1 convolutional layer connected in series. The number of output channels of the 1×1 convolutional layer represents the number of semantic categories. Furthermore, the last layer uses the ReLU activation function, and the classification output layer is read at each pixel position. For each semantic category The generated non-negative feature activation output, the non-negative feature activation output Directly used as the evidence quantity parameter for the corresponding semantic category ,Right now The evidence parameters for all semantic categories are summarized according to each pixel position, and a category prior of 1 is superimposed on each evidence parameter to form the Dirichlet distribution parameters corresponding to each pixel position. Then, the number of semantic categories is calculated. The total Dirichlet distribution parameters relative to the pixel location (i.e., Dirichlet intensity) The proportion of ) is used as the cognitive uncertainty of that pixel, and its calculation formula is: in, That is, pixels The cognitive uncertainty probability value is calculated, with a range of (0,1]. A larger value indicates higher uncertainty. The calculation results of all pixels are arranged according to their original pixel coordinates to obtain the scene cognitive uncertainty probability map. .

[0030] The steps for obtaining the low-uncertainty purity feature and the corresponding purified pseudo-label are as follows: Based on the scene cognitive uncertainty probability map, the scene cognitive uncertainty probability value corresponding to each pixel coordinate is read row by row. The scene cognitive uncertainty probability value is compared with the preset uncertainty limit item by item. The pixel coordinates whose scene cognitive uncertainty probability value is less than the preset uncertainty limit are retained and written into the pixel coordinate set. The pixel coordinates whose scene cognitive uncertainty probability value is greater than or equal to the preset uncertainty limit are excluded from the pixel coordinate set. A matrix of the same size is established according to the number of pixel rows and the number of pixel columns of the scene cognitive uncertainty probability map. The pixel positions in the pixel coordinate set are assigned valid labels and the remaining pixel positions are assigned invalid labels to generate a reliability mask matrix. Verify the pixel row coordinates and pixel column coordinates of the reliability mask matrix, the spatiotemporal consistency feature, and the initial pseudo-label. Extend the valid identifiers in the reliability mask matrix to all feature channels of the pixel positions corresponding to the spatiotemporal consistency feature. Retain the feature response values ​​of the pixel positions corresponding to the valid identifiers, set the feature response values ​​of the pixel positions corresponding to the invalid identifiers to zero, retain the semantic category numbers in the initial pseudo-labels according to the same pixel coordinates, replace the semantic category numbers of the pixel positions corresponding to the invalid identifiers with invalid labels, and maintain the pixel-by-pixel correspondence between the retained feature response values ​​and the retained semantic category numbers to obtain low-uncertainty pure features and their corresponding purified pseudo-labels.

[0031] Specifically, based on the aforementioned scene-based cognitive uncertainty probability graph Read the coordinates of each pixel line by line. Corresponding scene cognitive uncertainty probability value Compare this probability value with a preset uncertainty limit. The method for setting this limit is to perform a step-by-step comparison: in the early stages of model training, statistically analyze the prediction uncertainty of a batch (e.g., 1000) of synthetic images and calculate the mean of these uncertainty values. and standard deviation Set the limit to To retain most of the relatively certain prediction areas, and retain all those that meet the criteria. Conditional pixel coordinates And write these coordinates into a pixel coordinate set, which will satisfy The pixel coordinates of the condition are excluded from the set. A binary matrix of the same size is constructed according to the number of pixel rows and columns of the scene cognitive uncertainty probability map. Pixel positions in the pixel coordinate set are assigned a valid identifier 1, and the remaining pixel positions are assigned an invalid identifier 0, thus generating a reliability mask matrix. .

[0032] Verify the reliability mask matrix The spatiotemporal consistency feature and the previously generated initial pseudo-tags The pixel row coordinates and pixel column coordinates are determined to ensure spatial alignment. The valid identifier 1 in the reliability mask matrix is ​​then extended to all pixel positions corresponding to the spatiotemporal consistency feature. Each feature channel retains the feature response value corresponding to the valid identifier pixel position through element-wise multiplication, and sets the feature response value of the pixel position corresponding to the invalid identifier 0 to zero. Similarly, according to the same pixel coordinates, the initial pseudo-labels are... The semantic category numbers of valid pixel locations indicated by the reliability mask matrix are retained, and the semantic category numbers of pixel locations corresponding to invalid identifiers 0 are replaced with a special invalid label value (e.g., 255). This process maintains the one-to-one correspondence between the retained feature response values ​​and the retained semantic category numbers at each pixel location, thereby obtaining low-uncertainty pure features after uncertainty filtering. And the corresponding purification pseudo-label .

[0033] The steps for obtaining the high-altitude descent semantic segmentation model are as follows: Read the low-uncertainty pure features and the initial features of the synthetic domain, verify the pixel row coordinates, pixel column coordinates and feature channel numbers, establish the spatial correspondence between the center pixel position and the upper, lower, left and right pixel positions in each feature channel, calculate the second-order response change of the center pixel position along the pixel row direction, calculate the second-order response change of the center pixel position along the pixel column direction, superimpose the second-order response changes in the two directions pixel by pixel, retain the pixel positions where the absolute value of the response change exceeds the preset edge response limit, suppress the pixel positions where the absolute value of the response change does not exceed the preset edge response limit, arrange the retained response changes according to the original pixel coordinates and feature channel numbers, and obtain the high-frequency edge feature map of the real domain and the high-frequency edge feature map of the synthetic domain. Based on the high-frequency edge feature maps of the real domain and the synthetic domain, the latent space mapping responses of all pixel positions are statistically analyzed according to the same feature channel. The mean of the real domain mapping and the mean of the synthetic domain mapping are calculated for each feature channel. The difference between the mean mappings of each feature channel is accumulated to obtain the maximum mean difference metric. A high-frequency geometric consistency distillation loss is constructed based on the maximum mean difference metric. The semantic category number retained in the purified pseudo-label is read, and the real domain category prediction probability of the corresponding pixel position is called. The negative logarithm of the prediction probability corresponding to the semantic category number is accumulated to obtain the real domain self-supervised cross-entropy loss. The ground truth label of the synthetic high-altitude descent scene image is read, and the synthetic domain category prediction probability of the corresponding pixel position is called. The negative logarithm of the prediction probability corresponding to the ground truth label is accumulated to obtain the synthetic domain supervised loss. The numerical proportions of the high-frequency geometric consistency distillation loss, the real domain self-supervised cross-entropy loss, and the synthetic domain supervised loss are adjusted according to the preset loss weights. The adjusted loss values ​​are accumulated to obtain the global joint loss function. Based on the global joint loss function, the partial derivative change of the global joint loss function with respect to each network parameter is calculated layer by layer according to the parameter propagation direction of the semantic classifier, feature extraction network, and domain discriminator. The partial derivative change of each network parameter is propagated in reverse according to the corresponding parameter position. Each network parameter is corrected according to the preset parameter update step size. The synthesized high-altitude descent scene image and continuous frames of real aerial descent scene images are re-inputted. Feature extraction, semantic prediction, loss calculation, and parameter correction are repeatedly performed. The value of the global joint loss function corresponding to each training round is recorded. When the change amplitude of the global joint loss function value in multiple consecutive rounds is less than the preset convergence limit, parameter correction is stopped. The feature extraction network parameters and semantic classifier parameters at the time of stopping parameter correction are retained to obtain the high-altitude descent semantic segmentation model.

[0034] Specifically, read the low-uncertainty pure features. and the initial features of the synthesis domain The pixel row coordinates, pixel column coordinates, and feature channel numbers of both are verified. Within each feature channel, the Laplacian operator is used to extract high-frequency edge information. Specifically, the second-order response change of the center pixel position along the pixel row direction and the second-order response change along the pixel column direction are calculated. The discretization calculation formula is as follows: in, Representative feature map at location The response value, For the Laplacian response at a given point, the second-order response changes in both directions are superimposed pixel-by-pixel. An edge response limit is set, which is twice the global average absolute value of the Laplacian response for each feature map. Pixels whose absolute response changes exceed this preset edge response limit are retained, while pixels whose absolute response changes do not exceed the limit are set to zero. The retained response changes are arranged according to their original pixel coordinates and feature channel numbers to obtain the real-domain high-frequency edge feature maps. and high-frequency edge feature map of the synthetic domain .

[0035] Based on the real-domain high-frequency edge feature map and the high-frequency edge feature map of the synthesized domain To achieve knowledge distillation, maximum mean difference (MMD) is used to align the high-frequency feature distributions of the two domains. The latent space mapping response of all pixel positions is statistically analyzed according to the same feature channel, and the latent space mapping response of each feature channel is calculated. The corresponding real domain mapping mean Mapping mean to the synthetic domain The sum of squared differences between the means of each feature channel mapping is accumulated to obtain the maximum mean difference measure, and this is used to construct the high-frequency geometric consistency distillation loss. Its expression is: ,in and It is the total number of pixels in the real domain and the composite domain. and This corresponds to the response value of the feature map, and simultaneously, the purified pseudo-label is read. The semantic category numbering of invalid labels in China and Africa, and the calling model for pure features with low uncertainty. Predicted class probabilities, calculate real-domain self-supervised cross-entropy loss. And read the ground truth labels of the synthesized high-altitude descent scene image. Calculate the supervised loss in the synthetic domain According to the preset loss weight , and (For example, based on experience, set them to 0.1, 1.0, 1.0 respectively) Adjust the numerical proportions of each loss, and sum the adjusted loss values ​​to obtain the global joint loss function. .

[0036] Based on the global joint loss function The Adam optimizer is employed, and following the parameter propagation direction of the semantic classifier, feature extraction network, and domain discriminator, the backpropagation algorithm is used to calculate the partial derivative change (i.e., gradient) of the global joint loss function with respect to each network trainable parameter (weights and biases) layer by layer. The gradient of each network parameter is propagated backward according to its corresponding position in the network, and each network parameter is corrected based on a preset parameter update step size (i.e., learning rate, for example, initially set to 1e-4) and the momentum estimate of the Adam optimizer. The specific update rule is as follows: ,in This is the current parameter. It's the learning rate. and These are the bias correction estimates for the first and second moments, respectively. This is a small constant (e.g., 1e-8) set to prevent division by zero. A new batch of synthetic high-altitude descent scene images and consecutive frames of real aerial descent scene images are re-inputted. The complete training process of feature extraction, semantic prediction, loss calculation, and parameter correction is repeated. The value of the global joint loss function corresponding to each training round is recorded. When the decrease of the global joint loss function value in 10 consecutive training rounds is less than the preset convergence limit 1e-6, the model is determined to have converged and parameter correction is stopped. The feature extraction network parameters and semantic classifier parameters at this time are retained to obtain the final high-altitude descent semantic segmentation model.

[0037] The steps for obtaining the high-speed landing point identification result are as follows: A single-frame real-time aerial rappelling scene image captured by a drone or first-person perspective device is acquired. The number of pixel rows, number of pixel columns, channel arrangement order, and pixel values ​​of each channel in the single-frame real-frame real-frame aerial rappelling scene image are read. The pixel coordinate correspondence is adjusted according to the input size of the high-altitude rappelling semantic segmentation model. The pixel values ​​of each channel are rearranged according to the input channel order of the high-altitude rappelling semantic segmentation model. The adjusted single-frame real-frame aerial rappelling scene image is input into the high-altitude rappelling semantic segmentation model. The output value corresponding to each semantic category is read pixel by pixel. The semantic category number with the largest output value at each pixel position is selected. All semantic category numbers are arranged according to the original pixel row coordinates and the original pixel column coordinates to obtain a pixel-level semantic segmentation result image. Based on the pixel-level semantic segmentation result image, the semantic category number is read pixel by pixel. Pixel positions with semantic category numbers belonging to the safe landing material category are assigned valid identifiers, and pixel positions corresponding to other semantic category numbers are assigned invalid identifiers. All valid and invalid identifiers are arranged according to pixel row coordinates and pixel column coordinates to form a binary image of safe landing material. Starting from the first valid identifier in the binary image of safe landing material, the adjacent pixel positions are retrieved. Valid identifiers that are continuously connected in the horizontal, vertical, or diagonal directions are grouped into the same connected region. The adjacent pixel position retrieval is repeated for valid identifiers that have not yet been grouped into a connected region. All pixel coordinates corresponding to each connected region are recorded. Invalid identifiers that are completely surrounded by the same connected region are filled in, and isolated valid identifiers that cannot form a continuous boundary are deleted to obtain the connected regions of safe landing material. The number of effective pixels corresponding to each connected region of the safe landing material is counted, and the number of effective pixels is converted into the region area. The region area is compared with the preset landing point limit item by item. The connected regions of the safe landing material with a region area greater than the preset landing point limit are retained, and the connected regions of the safe landing material with a region area less than or equal to the preset landing point limit are deleted. The outermost boundary pixel coordinates of each retained region are traced in a clockwise direction. The connection status of the first and last pixels of the outermost boundary pixel coordinates is checked. The retained regions with broken boundaries are deleted. The polygon vertices are determined according to the pixel positions where the boundary direction changes. Adjacent polygon vertices are connected to form a closed boundary. The continuity of effective pixels and the distribution of internal holes inside the closed boundary are checked. Closed flat polygon regions with continuous effective pixels and no unfilled holes are retained. The coordinates of the closed flat polygon regions are output according to the order of polygon vertices to obtain the high-speed landing point identification result.

[0038] Specifically, a single-frame real-time aerial descent scene image captured by a drone or first-person perspective device is acquired. The number of pixel rows, columns, channel arrangement, and pixel values ​​for each channel are read. Following the input requirements of the high-altitude descent semantic segmentation model, the image size is first uniformly adjusted to 1024×512 using bicubic interpolation. Then, the channels are arranged in RGB order, and pixel values ​​are normalized to [0,1]. The adjusted single-frame image tensor is input into the trained high-altitude descent semantic segmentation model. The model performs one forward propagation calculation, reading the prediction score (logits) corresponding to each semantic category in the model's output layer pixel by pixel. The semantic category number with the largest output score at each pixel position is selected as the predicted category for that pixel. The mathematical expression is: ,in It is a pixel The prediction category The pixels output by the model Category The predicted category numbers of all pixels are reverse-mapped and arranged according to the pixel row and column coordinates before adjustment of the original image, based on the score or probability, to obtain a pixel-level semantic segmentation result image with the same size as the original input image.

[0039] Based on the pixel-level semantic segmentation result image, the semantic category number is read pixel by pixel. Pixels with semantic category numbers belonging to the predefined safe landing material category (e.g., category number 7, "flat grass / safe landing zone") are assigned a valid identifier of 1, while all other pixel positions corresponding to semantic category numbers are assigned an invalid identifier of 0. All valid and invalid identifiers are arranged according to pixel row and column coordinates to form a safe landing material binary image. Starting from the top left corner of this binary image, an 8-connected component analysis algorithm is performed on pixels with a value of 1. That is, starting from a seed pixel, a breadth-first search or depth-first search is used to analyze the horizontal, vertical, or diagonal components. Valid marker pixels that are consecutively connected in a direction are grouped into the same connected region. This search process is repeated for all valid marker pixels in the image that have not yet been assigned to any region until all valid pixels are assigned to a connected region. The coordinates of all pixels contained in each connected region are recorded. Then, morphological post-processing is performed. Specifically, a 5×5 structuring element is used to perform a closing operation (dilation followed by erosion) on each connected region to fill in small invalid markers (holes) that are completely surrounded by the same connected region. Then, an opening operation (erosion followed by dilation) is performed to remove isolated valid marker noise that cannot form a continuous boundary, resulting in the optimized connected region of the safe landing material.

[0040] The number of effective pixels corresponding to each safe landing material connected region is counted, and this number is multiplied by the actual area represented by a single pixel (estimated based on the drone's altitude and camera field of view), converting it into a region area in square meters. The area of ​​each region is then compared item by item with a preset minimum landing point limit (e.g., 10 square meters, set based on the drone's rotor diameter and safety redundancy distance). Safe landing material connected regions with an area greater than this limit are retained, while other regions that do not meet the condition are deleted. For each retained region, the Moore-Neighbor tracking algorithm is used to trace the pixel coordinate sequence of its outermost boundary in a clockwise direction. The first and last pixel coordinates of this sequence are checked to see if they coincide or are adjacent, confirming the boundary closure. Regions with boundary extinction are then deleted. In the non-closed region of the breakpoint, the Douglas-Puk algorithm is used to fit polygons based on the positions of pixels whose orientation changes in the boundary pixel sequence, in order to simplify the boundary and determine the polygon vertices. The tolerance parameter of the algorithm (e.g., set to 2.0 pixels) determines the degree of simplification. Connecting adjacent polygon vertices forms a closed boundary, and then checking the continuity of effective pixels inside the closed boundary, such as calculating the ratio of the convex hull area of ​​the region to its actual pixel area (solidity), and checking for internal holes that have not been filled after morphological processing. Closed flat polygon regions with a solidity greater than 0.9 and no internal holes are retained, and the geographic or image coordinates of these regions are output in clockwise order of the polygon vertices to obtain the final high-speed landing point identification result.

[0041] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for identifying high-speed landing points based on image semantic segmentation, characterized in that, It includes a model adaptive training phase and a real-time application inference phase; the model adaptive training phase includes the following steps: Step S1: Obtain the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image; extract the initial features of the synthesized domain and the initial features of the real domain; obtain pixel-level domain-aligned features through domain alignment processing; predict the initial features of the real domain to generate initial pseudo-labels. Step S2: Input the adjacent previous frame real aerial descent scene image and the current frame real aerial descent scene image into the optical flow estimation network to obtain the optical flow field; use the optical flow field to distort the pixel-level domain alignment features corresponding to the adjacent previous frame real aerial descent scene image to generate temporal distortion features; calculate the feature divergence residual between the temporal distortion features and the pixel-level domain alignment features corresponding to the current frame real aerial descent scene image; perform mask modulation on the pixel-level domain alignment features corresponding to the current frame real aerial descent scene image based on the feature divergence residual to output spatiotemporal consistency features; Step S3: Based on the spatiotemporal consistency features, obtain a scene cognition uncertainty probability map; extract the pixel coordinates within the scene cognition uncertainty probability map that meet the preset uncertainty limit conditions to generate a reliability mask matrix; use the reliability mask matrix to filter the spatiotemporal consistency features and the initial pseudo-labels to obtain low-uncertainty pure features and purified pseudo-labels.

2. The high-speed landing point recognition method based on image semantic segmentation according to claim 1, characterized in that, The adaptive training phase of the model also includes: Step S4: Perform derivative operations on the low-uncertainty pure features and the initial features of the synthetic domain respectively to extract the high-frequency edge feature maps of the real domain and the synthetic domain. Construct a high-frequency geometric consistency distillation loss based on the high-frequency edge feature maps of the real domain and the synthetic domain. Combine the high-frequency geometric consistency distillation loss, the real domain self-supervised cross-entropy loss constructed based on the purified pseudo-labels, and the synthetic domain supervised loss constructed based on the truth labels to calculate the global joint loss function and update the network parameters to obtain the high-altitude descent semantic segmentation model.

3. The high-speed landing point recognition method based on image semantic segmentation according to claim 1, characterized in that, The real-time application inference phase includes the following steps: Step S5: Input the collected single-frame real aerial descent scene image into the high-altitude descent semantic segmentation model and output a pixel-level semantic segmentation result image; extract the pixel set corresponding to the safe landing material from the pixel-level semantic segmentation result image, perform connected component morphological analysis, output the coordinates of a closed flat polygon region with an area greater than the preset landing point limit, and obtain the high-altitude descent point identification result.

4. The high-speed landing point recognition method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the initial pseudo-tags are as follows: Acquire synthetic high-altitude descent scene images with pixel-level ground truth labels, verify the pixel coordinates, semantic category numbers, and pixel-level ground truth label correspondence of the synthetic high-altitude descent scene images frame by frame, acquire unlabeled continuous frames of real aerial descent scene images arranged according to continuous acquisition time, select the current frame of real aerial descent scene images from the unlabeled continuous frames of real aerial descent scene images, unify the pixel size, channel arrangement order, and pixel value range of the synthetic high-altitude descent scene images and the current frame of real aerial descent scene images, and maintain the pixel-level ground truth labels and the pixel positions of the synthetic high-altitude descent scene images point by point to form a scene image input combination; Based on the scene image input combination, the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image are synchronously input into the feature extraction network according to the same batch position. The convolution output of the same level in the feature extraction network is called. The feature response values ​​of the synthesized high-altitude descent scene image and the current frame's real aerial descent scene image are recorded according to the pixel row coordinates, pixel column coordinates, and feature channel numbers, respectively. The feature response values ​​of the synthesized high-altitude descent scene image are organized into the initial features of the synthesis domain, and the feature response values ​​of the current frame's real aerial descent scene image are organized into the initial features of the real domain, thus obtaining the initial features of the synthesis domain and the initial features of the real domain. The initial features of the synthetic domain and the initial features of the real domain are input into the gradient inversion layer. The real domain discrimination probability obtained by inputting the initial features of the real domain into the domain discriminator and the synthetic domain discrimination probability obtained by inputting the initial features of the synthetic domain into the domain discriminator are compared batch by batch. The difference values ​​of the two probabilities distributed in the global feature latent space are accumulated. The feature extraction network parameters are adjusted in reverse according to the difference values. The feature extraction, domain discrimination, difference value calculation and parameter adjustment are repeated until the change amplitude of the difference values ​​between two adjacent times no longer exceeds the preset convergence limit. The pixel-level domain alignment features after the domain environment interference is weakened are retained. At the same time, the initial features of the real domain are input into the pre-trained classifier. The prediction probability corresponding to each semantic category is read pixel by pixel. The maximum prediction probability and the semantic category number corresponding to the maximum prediction probability are selected for each pixel position. Pixel positions with the maximum prediction probability greater than or equal to the preset confidence limit are assigned the corresponding semantic category number. Pixel positions with the maximum prediction probability less than the preset confidence limit are assigned an invalid label. All semantic category numbers and invalid labels are rearranged according to the pixel coordinates of the real aerial descent scene image in the current frame to generate initial pseudo labels.

5. The high-speed landing point recognition method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the characteristic divergence residual are as follows: Read the previous and current frames of real aerial descent scene images in the order of continuous acquisition time. Check the number of pixel rows, number of pixel columns, and channel arrangement order of the previous and current frames of real aerial descent scene images. Keep the same pixel coordinates corresponding to the same spatial position. Input the previous and current frames of real aerial descent scene images into the optical flow estimation network synchronously. Determine the corresponding pixel position of the current frame pixel in the previous frame pixel by pixel. Record the coordinate offset of the corresponding pixel position along the pixel row direction and the coordinate offset along the pixel column direction. Arrange the coordinate offsets of all pixel positions according to the pixel coordinates of the current frame to obtain the optical flow field. Read the pixel-level domain alignment features corresponding to the previous frame of real aerial rappelling scene image. According to the pixel row direction coordinate offset and pixel column direction coordinate offset of each pixel position in the optical flow field, reverse locate the source coordinates in the pixel-level domain alignment features corresponding to the previous frame of real aerial rappelling scene image. When the source coordinates fall on an integer pixel position, read the corresponding feature value. When the source coordinates fall between adjacent integer pixel positions, allocate the value ratio according to the distance from the source coordinates to the surrounding pixel positions. Rearrange the reading results according to the pixel coordinates of the current frame to generate temporal distortion features. Read the temporal distortion features and the pixel-level domain alignment features of the current frame of real aerial rappelling scene image pixel by pixel. Calculate the response difference of the same pixel position and the same feature channel. Summarize the response difference of all feature channels corresponding to each pixel position to obtain the feature divergence residual.

6. The high-speed landing point recognition method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the spatiotemporal consistency feature are as follows: Based on the feature divergence residual, the residual value of each pixel position is read item by item according to the pixel coordinates of the current frame. Each residual value is input into a negative exponential mapping function. When the residual value increases, the weight value of the corresponding pixel position is reduced, and when the residual value decreases, the weight value of the corresponding pixel position is increased. The weight values ​​of all pixel positions are restricted to the range of zero to one. All weight values ​​are arranged according to the pixel coordinates of the current frame, and the pixel-level domain alignment features of the real aerial descent scene image of the current frame are read pixel by pixel. At the same pixel position, the weight value is multiplied into all feature channels respectively. For pixel positions where the residual value exceeds the preset residual judgment limit, the feature response amplitude is reduced. For pixel positions where the residual value does not exceed the preset residual judgment limit, the corresponding feature response is retained. The weighted feature responses are arranged according to the original number of pixel rows, number of pixel columns, and number of feature channels to obtain the spatiotemporal consistency features.

7. The high-speed landing point recognition method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the scene cognition uncertainty probability map are as follows: The spatiotemporal consistency features are input into a semantic classifier based on evidence deep learning according to pixel row coordinates, pixel column coordinates, and feature channel numbers. The feature activation outputs generated by the classification output layer for each semantic category at each pixel position are read. Negative feature activation outputs are truncated to zero values, and non-negative feature activation outputs are converted into evidence quantity parameters for the corresponding semantic category. The evidence quantity parameters of all semantic categories are summarized according to each pixel position. The category prior is superimposed on each evidence quantity parameter to form the Dirichlet distribution parameters corresponding to each pixel position. The proportion of the number of semantic categories relative to the total number of Dirichlet distribution parameters is calculated. The proportion values ​​are arranged according to the original pixel coordinates to obtain the scene cognition uncertainty probability map.

8. The high-speed landing point recognition method based on image semantic segmentation according to claim 1, characterized in that, The steps for obtaining the low-uncertainty purity feature and the corresponding purified pseudo-label are as follows: Based on the scene cognitive uncertainty probability map, the scene cognitive uncertainty probability value corresponding to each pixel coordinate is read row by row. The scene cognitive uncertainty probability value is compared with the preset uncertainty limit item by item. The pixel coordinates whose scene cognitive uncertainty probability value is less than the preset uncertainty limit are retained and written into the pixel coordinate set. The pixel coordinates whose scene cognitive uncertainty probability value is greater than or equal to the preset uncertainty limit are excluded from the pixel coordinate set. A matrix of the same size is established according to the number of pixel rows and the number of pixel columns of the scene cognitive uncertainty probability map. The pixel positions in the pixel coordinate set are assigned valid labels and the remaining pixel positions are assigned invalid labels to generate a reliability mask matrix. Verify the pixel row coordinates and pixel column coordinates of the reliability mask matrix, the spatiotemporal consistency feature, and the initial pseudo-label. Extend the valid identifiers in the reliability mask matrix to all feature channels of the pixel positions corresponding to the spatiotemporal consistency feature. Retain the feature response values ​​of the pixel positions corresponding to the valid identifiers, set the feature response values ​​of the pixel positions corresponding to the invalid identifiers to zero, retain the semantic category numbers in the initial pseudo-labels according to the same pixel coordinates, replace the semantic category numbers of the pixel positions corresponding to the invalid identifiers with invalid labels, and maintain the pixel-by-pixel correspondence between the retained feature response values ​​and the retained semantic category numbers to obtain low-uncertainty pure features and their corresponding purified pseudo-labels.

9. The high-speed landing point recognition method based on image semantic segmentation according to claim 2, characterized in that, The steps for obtaining the high-altitude descent semantic segmentation model are as follows: Read the low-uncertainty pure features and the initial features of the synthetic domain, verify the pixel row coordinates, pixel column coordinates and feature channel numbers, establish the spatial correspondence between the center pixel position and the upper, lower, left and right pixel positions in each feature channel, calculate the second-order response change of the center pixel position along the pixel row direction, calculate the second-order response change of the center pixel position along the pixel column direction, superimpose the second-order response changes in the two directions pixel by pixel, retain the pixel positions where the absolute value of the response change exceeds the preset edge response limit, suppress the pixel positions where the absolute value of the response change does not exceed the preset edge response limit, arrange the retained response changes according to the original pixel coordinates and feature channel numbers, and obtain the high-frequency edge feature map of the real domain and the high-frequency edge feature map of the synthetic domain. Based on the high-frequency edge feature maps of the real domain and the synthetic domain, the latent space mapping responses of all pixel positions are statistically analyzed according to the same feature channel. The mean of the real domain mapping and the mean of the synthetic domain mapping are calculated for each feature channel. The difference between the mean mappings of each feature channel is accumulated to obtain the maximum mean difference metric. A high-frequency geometric consistency distillation loss is constructed based on the maximum mean difference metric. The semantic category number retained in the purified pseudo-label is read, and the real domain category prediction probability of the corresponding pixel position is called. The negative logarithm of the prediction probability corresponding to the semantic category number is accumulated to obtain the real domain self-supervised cross-entropy loss. The ground truth label of the synthetic high-altitude descent scene image is read, and the synthetic domain category prediction probability of the corresponding pixel position is called. The negative logarithm of the prediction probability corresponding to the ground truth label is accumulated to obtain the synthetic domain supervised loss. The numerical proportions of the high-frequency geometric consistency distillation loss, the real domain self-supervised cross-entropy loss, and the synthetic domain supervised loss are adjusted according to the preset loss weights. The adjusted loss values ​​are accumulated to obtain the global joint loss function. Based on the global joint loss function, the partial derivative change of the global joint loss function with respect to each network parameter is calculated layer by layer according to the parameter propagation direction of the semantic classifier, feature extraction network, and domain discriminator. The partial derivative change of each network parameter is propagated in reverse according to the corresponding parameter position. Each network parameter is corrected according to the preset parameter update step size. The synthesized high-altitude descent scene image and continuous frames of real aerial descent scene images are re-inputted. Feature extraction, semantic prediction, loss calculation, and parameter correction are repeatedly performed. The value of the global joint loss function corresponding to each training round is recorded. When the change amplitude of the global joint loss function value in multiple consecutive rounds is less than the preset convergence limit, parameter correction is stopped. The feature extraction network parameters and semantic classifier parameters at the time of stopping parameter correction are retained to obtain the high-altitude descent semantic segmentation model.

10. The high-speed landing point recognition method based on image semantic segmentation according to claim 3, characterized in that, The steps for obtaining the high-speed landing point identification result are as follows: A single-frame real-time aerial rappelling scene image captured by a drone or first-person perspective device is acquired. The number of pixel rows, number of pixel columns, channel arrangement order, and pixel values ​​of each channel in the single-frame real-frame real-frame aerial rappelling scene image are read. The pixel coordinate correspondence is adjusted according to the input size of the high-altitude rappelling semantic segmentation model. The pixel values ​​of each channel are rearranged according to the input channel order of the high-altitude rappelling semantic segmentation model. The adjusted single-frame real-frame aerial rappelling scene image is input into the high-altitude rappelling semantic segmentation model. The output value corresponding to each semantic category is read pixel by pixel. The semantic category number with the largest output value at each pixel position is selected. All semantic category numbers are arranged according to the original pixel row coordinates and the original pixel column coordinates to obtain a pixel-level semantic segmentation result image. Based on the pixel-level semantic segmentation result image, the semantic category number is read pixel by pixel. Pixel positions with semantic category numbers belonging to the safe landing material category are assigned valid identifiers, and pixel positions corresponding to other semantic category numbers are assigned invalid identifiers. All valid and invalid identifiers are arranged according to pixel row coordinates and pixel column coordinates to form a binary image of safe landing material. Starting from the first valid identifier in the binary image of safe landing material, the adjacent pixel positions are retrieved. Valid identifiers that are continuously connected in the horizontal, vertical, or diagonal directions are grouped into the same connected region. The adjacent pixel position retrieval is repeated for valid identifiers that have not yet been grouped into a connected region. All pixel coordinates corresponding to each connected region are recorded. Invalid identifiers that are completely surrounded by the same connected region are filled in, and isolated valid identifiers that cannot form a continuous boundary are deleted to obtain the connected regions of safe landing material. The number of effective pixels corresponding to each connected region of the safe landing material is counted, and the number of effective pixels is converted into the region area. The region area is compared with the preset landing point limit item by item. The connected regions of the safe landing material with a region area greater than the preset landing point limit are retained, and the connected regions of the safe landing material with a region area less than or equal to the preset landing point limit are deleted. The outermost boundary pixel coordinates of each retained region are traced in a clockwise direction. The connection status of the first and last pixels of the outermost boundary pixel coordinates is checked. The retained regions with broken boundaries are deleted. The polygon vertices are determined according to the pixel positions where the boundary direction changes. Adjacent polygon vertices are connected to form a closed boundary. The continuity of effective pixels and the distribution of internal holes inside the closed boundary are checked. Closed flat polygon regions with continuous effective pixels and no unfilled holes are retained. The coordinates of the closed flat polygon regions are output according to the order of polygon vertices to obtain the high-speed landing point identification result.

Images