Deep learning feature extraction method for evtol field environment

By employing a self-supervised learning method based on synthetic data pre-training and image saliency detection, real field images are automatically labeled. Combined with sparse descriptor similarity loss calculation for difficult negative sample mining, the robustness and efficiency issues of feature extraction in the field environment of eVTOL are solved, achieving efficient and real-time mapping and localization.

CN120472249BActive Publication Date: 2025-10-17SHENZHEN BOUNDARY INTELLIGENT CONTROL TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510977284.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-07-16
Publication Date
2025-10-17
Estimated Expiration
2045-07-16

AI Technical Summary

Technical Problem

Traditional feature extraction methods are easily affected by changes in light, perspective, and season in the eVTOL field environment. Furthermore, existing deep learning methods consume too much computational resources and time, failing to meet the requirements of real-time performance and efficiency. In addition, there is a lack of effective solutions to the noise problem in field scenes.

Method used

A self-supervised learning method based on pre-training of synthetic data and image saliency detection is adopted to automatically annotate real field images. Combined with sparse descriptor similarity loss calculation for difficult negative sample mining, the robustness and training efficiency of the feature extraction model are improved.

Benefits of technology

It improves the robustness and training efficiency of feature extraction, adapts to the complex field environment of eVTOL, meets the needs of real-time mapping and localization, and reduces computing resources and time costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120472249B_ABST
    Figure CN120472249B_ABST
Patent Text Reader

Abstract

The application provides a deep learning feature extraction method for eVTOL field environment, comprising: pre-training a feature detection model based on synthetic data; automatically labeling real field down-view training images based on an image saliency detection method; training a feature extraction model using real field down-view training images with labeled feature points; inputting eVTOL real down-view images into the feature extraction model to obtain image feature points and feature point descriptors. The application improves the training efficiency and performance of the deep learning model in the eVTOL field environment feature extraction task as a whole, and better meets the actual needs of eVTOL field mapping and positioning, based on feature detection and model pre-training of synthetic data, automatic labeling based on the image saliency detection method, and feature extraction model training using difficult negative samples.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the field of eVTOL (electric vertical take-off and landing aircraft) outdoor environment mapping and positioning, and particularly relates to a deep learning feature extraction method for eVTOL outdoor environment. BACKGROUND

[0002] In the eVTOL (electric vertical take-off and landing aircraft) outdoor mapping and positioning task, the traditional feature extraction method has obvious defects. It is easily affected by light, angle of view and seasonal changes. When eVTOL performs long-term positioning tasks, the outdoor scene is variable, and the traditional method is difficult to meet the demand, and cannot provide stable and reliable feature extraction results.

[0003] The existing deep learning feature extraction method is mainly suitable for structured scenes such as cities and indoor scenes. For the data noise problem caused by dense forests and repeated fields in the outdoor scene, there is no effective solution, resulting in poor feature extraction accuracy and robustness in the outdoor environment.

[0004] In addition, the existing deep learning feature extraction method adopts a dense calculation method, which is low in training efficiency. When facing large-scale data training, a large amount of computing resources and time are consumed, which cannot meet the real-time and high-efficiency requirements in actual applications. SUMMARY

[0005] The purpose of the present application is to provide a deep learning feature extraction method for eVTOL outdoor environment, to solve the problem of feature extraction in the eVTOL outdoor environment mapping and positioning task in the prior art, and to improve the robustness of feature extraction, reduce the labeling cost, and improve the training efficiency.

[0006] The following gives a brief summary of one or more aspects to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all contemplated aspects, and neither is it intended to identify key or critical elements of all aspects nor to delineate the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to the more detailed description given later.

[0007] According to one aspect of the present application, a deep learning feature extraction method for eVTOL outdoor environment is provided, comprising:

[0008] Step A100, pre-training a feature detection model based on synthetic data;

[0009] Step A200, automatically labeling real outdoor down-view training images based on an image saliency detection method;

[0010] Step A300, training the feature extraction model using real field down-view training images with labeled feature points;

[0011] Step B400, inputting the eVTOL real down-view image into the feature extraction model to obtain image feature points and feature point descriptors.

[0012] In one embodiment, the step A100 comprises:

[0013] Step A110, generating a synthetic image using simple geometric shapes or random textures and labeling feature point positions;

[0014] Step A120, inputting the synthetic image and feature point positions into the feature detection model for pre-training.

[0015] In one embodiment, the image saliency detection method in step A200 specifically comprises:

[0016] Step A211, converting the image from the RGB color space to the LAB color space according to the preprocessing step of saliency detection, and calculating color saliency;

[0017] Step A212, converting the image to a grayscale image, performing Gaussian blur, and then using the Canny algorithm to extract edges to obtain an edge map;

[0018] Step A213, normalizing the color saliency and the edge map to the same range, and performing weighted fusion according to the set weight to obtain a comprehensive saliency map;

[0019] Step A214, binarizing the comprehensive saliency map according to the set threshold to obtain a preliminary saliency mask;

[0020] Step A215, calculating the connected regions of the mask through connectivity analysis, traversing the connected regions, marking the regions with an area greater than the pre-set area threshold as salient regions, and forming a final saliency mask.

[0021] In one embodiment, the automatic labeling of real field down-view images in step A200 comprises:

[0022] Step A221, performing multiple homographic transformations on a single real field down-view image;

[0023] Step A222, using the pre-trained feature detection model to perform feature detection on multiple transformed images in combination with the saliency mask to obtain a feature map for each transformed image;

[0024] Step A223, transforming the feature map back to the original image position according to the homography matrix corresponding to the image transformation;

[0025] Step A224, superimpose the feature maps transformed back to the original image position, and fuse the feature information of multiple images.

[0026] Step A225, extract the coordinates of the maximum K values from the superimposed feature map, and determine the coordinates as the feature point labels.

[0027] In an embodiment, the number of homographic transformations in step A221 and the value of K in step A225 are set according to actual needs.

[0028] In an embodiment, the step A300 specifically comprises:

[0029] Step A310, construct an image feature encoder based on a VGG network model;

[0030] Step A320, predict the feature point positions and feature descriptors in the labeled feature points of the real field down-view training images by a feature point decoder and a descriptor decoder, respectively;

[0031] Step A330, calculate the cross-entropy loss of the label coordinates and the decoded coordinates to optimize the feature point decoder, and calculate the descriptor matching loss to optimize the descriptor decoder.

[0032] In an embodiment, in step A330, a loss function based on triplets is used to calculate the descriptor matching loss.

[0033] In an embodiment, the calculation of the descriptor matching loss in step A330 further comprises calculating a positive sample loss and a negative sample loss based on difficult negative sample mining.

[0034] In an embodiment, the calculation of the negative sample loss based on difficult negative sample mining comprises:

[0035] Step A331, in the training data set, for each target data point, calculate the feature distance between it and all other data points in the data set;

[0036] Step A332, select several data points with the farthest distance from all the calculated distances as negative samples;

[0037] Step A333, generate a corresponding negative sample set for each data point in the data set;

[0038] Step A334, calculate the negative sample loss.

[0039] In an embodiment, the calculation of the feature distance in step A331 uses Euclidean distance or cosine distance.

[0040] The beneficial effects of the embodiment of the present application are:

[0041] The image feature point labeling method based on saliency detection self-supervised learning adopted by the present application avoids the high cost and low efficiency problems of manual labeling, and at the same time, the saliency detection is used to exclude noise areas, thereby improving the robustness of feature extraction and adapting to the complex and variable environment of eVTOL in the wild.

[0042] Through the sparse descriptor similarity loss calculation method based on difficult negative sample mining, the distinguishing ability of the model to similar features is enhanced, the discriminant ability of the feature space is improved, and the feature extraction result is more accurate and reliable.

[0043] Based on the feature detection and model pre-training of synthetic data, combined with the automatic labeling based on image saliency detection method and the feature extraction model training using difficult negative sample mining, the training efficiency and performance of the deep learning model in the eVTOL field environment feature extraction task are improved as a whole, which can better meet the actual needs of eVTOL field mapping and positioning. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor by reading the following detailed description of the embodiments of the present application in combination with the following drawings.

[0045] The above features and advantages of the present application can be better understood after reading the detailed description of the embodiments of the present application in combination with the following drawings. In the drawings, the components are not necessarily drawn to scale, and the components having similar related properties or features can have the same or similar reference numerals.

[0046] Figure 1 is a flowchart of a deep learning feature extraction method for eVTOL field environment disclosed by the embodiment of the present application;

[0047] Figure 2 is a general method step diagram flowchart of the embodiment of the present application;

[0048] Figure 3 is an automatic labeling method flowchart of the embodiment of the present application;

[0049] Figure 4 is a feature extraction model training flowchart of the embodiment of the present application. DETAILED DESCRIPTION

[0050] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.

[0051] It should be noted that the terms "first", "second", "third", "fourth" and the like in the specification and claims of the present application are used to distinguish different objects, and are not used to describe a specific order. The terms "include" and "have" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0052] Please refer to Figure 1 and Figure 2 , Figure 1 is a flowchart of a deep learning feature extraction method for eVTOL field environment disclosed by the embodiments of the present application; Figure 2 is a general method step diagram flowchart of the embodiments of the present application.

[0053] The embodiments of the present application provide a deep learning feature extraction method for eVTOL field environment, including the following steps:

[0054] Step A100, pre-training a feature detection model based on synthetic data;

[0055] Step A200, automatically labeling real field down-view training images based on an image saliency detection method;

[0056] Step A300, training a feature extraction model using real field down-view training images with labeled feature points;

[0057] Step B400, inputting eVTOL real shooting down-view images into the feature extraction model to obtain image feature points and feature point descriptors.

[0058] Among them, steps A100-A300 are offline model training parts, which are operated in a non-real-time and temporarily separated environment from the actual application scenario. A large amount of synthetic image and real field down-view image data are collected before training, and then model training is carried out on a special computing device (such as a server equipped with a high-performance GPU). The entire training process does not depend on the data collected by eVTOL in real time, and a parameter-determined model is obtained after training.

[0059] Step B 400 is online real-time feature extraction, which is performed in real time in synchronization with the actual application scenario. During the flight of the eVTOL, the model that has completed offline training is used to process the real shooting down-view images collected by the aircraft in real time, so as to extract feature points and descriptors in the images in real time, so as to meet the needs of real-time positioning and mapping.

[0060] Step A 100 specifically includes: using a synthetic image generation tool to generate synthetic images containing simple geometric shapes (such as triangles, ellipses, rectangles, etc.) or random textures. The synthetic images are labeled with feature point positions, and the labeling information is used to pre-train a feature detection model containing a shared encoder, a feature point detection decoder, and a descriptor generation decoder. During the pre-training process, the network parameters are adjusted according to the input of the synthetic images and the labeled feature point positions, so that the network can initially learn the ability of feature point detection and descriptor generation. This method does not require manual labeling and is suitable for large-scale data training.

[0061] In step A 200, a saliency detection-based self-supervised learning image feature point labeling method is used. In the traditional vision field, saliency detection is usually used to extract the most attractive areas of human visual attention from images. These areas usually contain key information in the scene, such as object edges, texture changes, or areas with strong color contrast. This method proposes to apply saliency detection to the mapping and positioning tasks of eVTOL outdoor scenes to automatically identify key feature points in the environment and exclude noisy areas (such as leaves, repeated mud and grass in fields) that are not concerned about mapping and positioning, thereby improving the robustness of feature extraction.

[0062] In this embodiment, the saliency detection method specifically includes:

[0063] Step A 211, according to the preprocessing step of saliency detection, the image is converted from RGB color space to LAB color space, and color saliency is calculated;

[0064] Step A 212, convert the image to a grayscale image, perform Gaussian blur, and then use the Canny algorithm to extract edges to obtain an edge map;

[0065] Step A 213, normalize the color saliency and the edge map to the same range, and perform weighted fusion according to the set weight to obtain a comprehensive saliency map;

[0066] Step A 214, binarize the comprehensive saliency map according to the set threshold to obtain a preliminary saliency mask;

[0067] Step A215, calculate connected regions of the mask by connectivity analysis, traverse the connected regions, mark the regions with area larger than a pre-set area threshold as salient regions, and form the final saliency mask.

[0068] Part of the exemplary pseudo code of the saliency detection method is as follows:

[0069] Function:

[0070] Input:

[0071] I - input image (RGB)

[0072] Output:

[0073] M - saliency Mask (same size as I, salient region = 1, non-salient region = 0)

[0074] 1. Preprocessing:

[0075] 1.1 Read the input image I and convert to LAB color space

[0076] 1.2 Calculate color saliency S_color:

[0077] - Calculate the mean of A, B channels μ_A, μ_B

[0078] - Calculate the color contrast of each pixel point (x, y):

[0079] S_color (x, y) = (A(x, y) - μ_A)^2 + (B(x, y) - μ_B)^2

[0080] - Normalize S_color to [0, 255]

[0081] 2. Structure saliency detection:

[0082] 2.1 Convert I to grayscale image G

[0083] 2.2 Perform Gaussian blur to get G_blur

[0084] 2.3 Use Canny algorithm to extract edges to get edge map E

[0085] 3. Calculate the comprehensive saliency map S_final:

[0086] 3.1 Normalize S_color and E to the same range

[0087] 3.2 Calculate the weighted fusion S_final = a * S_color + b * E, a, b are two weight parameters, which can be adjusted according to actual situation, to balance the contribution of color and structure information to saliency.

[0088] 3.3 According to the set threshold value, binarize S_final, and convert it into a black and white image to obtain a preliminary saliency mask Mask_raw

[0089] 4. Connectivity analysis:

[0090] 4.1 Calculate the connected regions of Mask_raw, which refers to the set of connected pixels in the image.

[0091] 4.2 Initialize the final mask Mask = create a zero matrix with the same size as the input image

[0092] 4.3 Traverse all connected regions i (starting from 1, skipping the background region):

[0093] If the statistical information [i, area] > area threshold:

[0094] Mark the current connected region as a salient region Mask [component label == i] = 1

[0095] 5. Morphological optimization:

[0096] 5.1 Perform dilation operation Mask = morphological dilation (Mask, kernel size = 5x5, iteration number = 1)

[0097] 6. Return the saliency mask Mask

[0098] End Function

[0099] After obtaining the saliency mask, step A200 further includes automatically labeling the real outdoor down-view images, as shown in Figure 3 , which includes:

[0100] Step A221, multiple homography transformations are performed on a single real outdoor down-view image, the homography transformation can be realized by setting different transformation parameters (such as rotation angle, translation amount, scaling ratio, etc.), and the multiple homography transformations can simulate image changes under different viewing angles, and the number of homography transformations can be determined according to actual needs;

[0101] Step A222, using a pre-trained feature detection model, combined with a saliency mask to eliminate noise regions, perform feature detection on multiple transformed images to obtain a feature map for each transformed image. Feature map is the result obtained by convolutional neural network (CNN) in deep learning when processing images through convolutional layer, pooling layer and other operations. It is essentially a multidimensional array used to represent feature information at different levels of the image;

[0102] Step A223, according to the homography matrix corresponding to the image transformation, transform the feature map back to the original image position to ensure the correspondence between the feature map and the original image;

[0103] Step A224, superimpose the feature map transformed back to the original image position to fuse the feature information of multiple images. After superimposing the feature maps of multiple transformed images, a new feature map is obtained. This superimposed feature map integrates the feature information of multiple images, and the value at each position is the cumulative value or other way of merging the values at the corresponding position in multiple feature maps.

[0104] Step A225, extract the coordinates of the top K values from the superimposed feature map, and these coordinates are determined as the feature point labels. Specifically, to find the most representative feature points from the superimposed feature map, all elements in the feature map are sorted, and the top K elements with the largest values are selected. Here, K is a pre-set parameter that determines the number of feature points extracted. It should be noted that the extracted are not the maximum values themselves, but their coordinate positions in the feature map. These coordinates correspond to the positions of the most salient feature points in the original image, and these coordinates are used as feature point labels for subsequent tasks.

[0105] For example, assume that the superimposed feature map is a 5x5 two-dimensional array, as follows:

[0106] [[ 2, 5, 3, 1, 4],

[0107] [ 7, 9, 6, 2, 3],

[0108] [ 4, 5, 8, 3, 1],

[0109] [ 1, 2, 3, 4, 5],

[0110] [ 6, 7, 2, 3, 4]]

[0111] If K = 3, the largest 3 values are 9, 8, 7, and their coordinates in the feature map are (1, 1), (2, 2), (1, 0) respectively (here the coordinates are represented as row first, column second), which will be extracted as the feature point labels.

[0112] For the determination of K value, multiple factors need to be considered. If used for eVTOL outdoor environment mapping, a large number of feature points are needed to accurately restore the environment, and the K value should be larger; if only simple target recognition is done, a small number of key feature points are enough, and the K value should be small. For example, in the identification of specific landmark buildings in the wild to assist positioning, the key contour features are focused on, and K is set to 20-50 to meet the requirements; while constructing a large-scale high-precision map, K may be several hundred or even thousands.

[0113] If the image scene in the data set is complex and the features are rich and diverse, such as wild images containing various landforms and a large amount of vegetation, in order to fully capture the features, the K value should be large; if the image features are single and highly similar, such as large-area flat field images, the K value can be small. For example, if the data set is mainly composed of complex mountain images, K may be taken as 100-200; if it is mainly composed of simple grassland scenes, K can be set to 10-30.

[0114] During training, an initial K value can be set, such as 30, and the model is trained with it and the performance indicators on the validation set are evaluated, such as the accuracy of target recognition and the accuracy of positioning. According to the evaluation results, adjust the K value, if the performance improves, continue to fine-tune; if it decreases, adjust in the opposite direction. For example, if the initial K = 30, the mapping error is large, increase it to 50 and test, adjust several times to find the optimal K value.

[0115] Figure 4 The feature extraction model structure and training process are shown in FIG. 3, which includes the following steps: Figure 4

[0116] Step A310, an image feature encoder is constructed based on the VGG network model. The VGG network performs well in image feature extraction and can effectively extract rich feature information from the input image;

[0117] Step A320, the feature point position and feature descriptor in the real wild down-view training image of the labeled feature point are predicted through the feature point decoder and the descriptor decoder respectively;

[0118] Step A330, the cross-entropy loss of the label coordinates and the decoded coordinates is calculated to optimize the feature point decoder; the descriptor matching loss is calculated to optimize the descriptor decoder. These losses work together to guide the parameter update of the model, so that the model can extract features more accurately. ​

[0119] In the present embodiment, a triplet-based loss function (Triplet Loss) is used to calculate the descriptor matching loss. By minimizing the loss function, the discriminative ability of the feature space is improved. A triplet consists of three samples: an anchor sample (Anchor, A ), a positive sample (Positive, P ), and a negative sample (Negative, N ). The positive sample and the anchor sample belong to the same class, and the negative sample and the anchor sample belong to different classes.

[0120] The triplet-based loss function is generally defined as:

[0121] L triplet =max( d ( A , P )− d ( A , N )+ α ,0)

[0122] where d(A, P) is the distance (e.g., Euclidean distance) between the anchor sample A and the positive sample P, d(A, N) is the distance between the anchor sample A and the negative sample N, and a is a positive margin value (margin) to ensure that there is enough separation between samples of different classes.

[0123] If d(A, P)−d(A, N)+a<0, it means that the distance between the negative sample and the anchor sample is far enough, and the distance between the positive sample and the anchor sample is close enough, so the loss is 0, and the model does not need to be adjusted.

[0124] If d(A, P)−d(A, N)+a≥0, it means that the distance between the negative sample and the anchor sample is not far enough, and the distance between the positive sample and the anchor sample is not close enough, so the loss is greater than 0, and the model needs to adjust the parameters to make d(A, P) decrease and d(A, N) increase.

[0125] The calculation of the descriptor matching loss in step A330 also includes calculating the positive sample loss and the negative sample loss based on difficult negative sample mining.

[0126] To improve the performance of the model and enhance its discriminative ability, the present embodiment uses a negative sample loss calculation method based on difficult negative sample mining, which includes:

[0127] Step A331, in the training data set, for each target data point, calculate its feature distance with all other data points in the data set. There are many ways to calculate the feature distance, such as Euclidean distance, cosine distance, etc. Through these methods, the similarity between two data points is measured. The closer the distance, the more similar the features of the two data points; the farther the distance, the greater the feature difference.

[0128] Step A332, from all the calculated distances, select several data points with the farthest distance as negative samples; these negative samples have large feature differences with the target data point, and in traditional training, the model may easily distinguish them. However, the key of difficult negative sample mining is to mine those samples that are similar to positive samples (samples belonging to the same category as the target data point) but do not match. Therefore, selecting the farthest samples as negative samples is only a preliminary screening, and further adjustment will be made when constructing the negative sample set.

[0129] Step A333, generate a corresponding negative sample set for each data point in the data set; the negative sample set formula is as follows:

[0130] The formula indicates that for each data point d i (wherein d i ∈ D , D represent the entire data set), N i is its negative sample set, which contains d i the farthest K descriptors from the description of K is a pre-set parameter that determines the number of negative samples for each data point). By constructing the negative sample set in this way, the model can pay more attention to the samples that are difficult to distinguish during the training process, and enhance the model's ability to distinguish similar features.

[0131] Step A334, calculate the negative sample loss and include it in the loss function of the entire model, together with other losses (such as cross-entropy loss of label coordinates and decoded coordinates, descriptor matching loss, positive sample loss, etc.) to update the parameters of the model through backpropagation, thereby improving the performance of the model.

[0132] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between each embodiment can be referred to each other.

[0133] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0134] The above description is only a preferred example of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A deep learning feature extraction method for eVTOL field environments, characterized by: include: Step A100: pre-training a feature detection model based on synthetic data; The step A100 includes: step A110, generating a synthetic image using simple geometric shapes or random textures, and marking the locations of feature points; step A120, inputting the synthetic image and the locations of feature points into a feature detection model for pre-training; Step A200: Automatically annotate a real field downward-looking training image based on an image saliency detection method. The image saliency detection method in step A200 specifically includes: step A211: converting the image from RGB color space to LAB color space according to the preprocessing step of saliency detection, and calculating color saliency; step A212: converting the image into a grayscale image, performing Gaussian blurring, and extracting edges using the Canny algorithm to obtain an edge map; step A213: normalizing the color saliency and edge maps to the same range, and performing weighted fusion according to set weights to obtain a comprehensive saliency map; step A214: binarizing the comprehensive saliency map according to a set threshold to obtain a preliminary saliency mask; step A215: calculating connected areas of the mask through connectivity analysis, traversing the connected areas, and marking areas with an area greater than a preset area threshold as salient areas to form a final saliency mask; Step A300: Training a feature extraction model using a real, downward-looking training image in the wild with marked feature points. Step A300 specifically includes: Step A310: Constructing an image feature encoder based on a VGG network model; Step A320: Predicting the feature point positions and feature descriptors in the real, downward-looking training image in the wild with marked feature points using a feature point decoder and a descriptor decoder; Step A330: Calculating the cross entropy loss between the label coordinates and the decoded coordinates to optimize the feature point decoder; Calculating the descriptor matching loss to optimize the descriptor decoder; In step A330, calculating the descriptor matching loss using a triple-based loss function; Calculating the descriptor matching loss in step A330 also includes calculating the positive sample loss and calculating the negative sample loss based on difficult negative sample mining; Step B400: Input the real-view image taken by the eVTOL into the feature extraction model to obtain image feature points and feature point descriptors.

2. The deep learning feature extraction method for eVTOL field environment according to claim 1 is characterized in that: The automatic labeling of the real field viewing image in step A200 includes: Step A221: performing multiple homography transformations on a single real field downward-viewing image; Step A222: Using a pre-trained feature detection model in combination with a saliency mask, perform feature detection on multiple transformed images to obtain a feature map of each transformed image; Step A223: transform the feature map back to the original image position according to the homography matrix corresponding to the image transformation; Step A224: superimpose the feature maps transformed back to the original image position to fuse the feature information of multiple images; Step A225: extract the coordinates of the largest K values ​​from the superimposed feature map, and these coordinates are determined as feature point labels.

3. The deep learning feature extraction method for eVTOL field environment according to claim 2, characterized in that: The number of homography transformations in step A221 and the K value in step A225 are set according to actual needs.

4. The deep learning feature extraction method for eVTOL field environment according to claim 3 is characterized in that: The calculation of negative sample loss based on difficult negative sample mining includes: Step A331: For each target data point in the training data set, calculate its feature distance with all other data points in the data set; Step A332: From all the calculated distances, select several data points with the longest distances as negative samples; Step A333: Generate a corresponding negative sample set for each data point in the data set; Step A334: Calculate the negative sample loss.

5. The deep learning feature extraction method for eVTOL field environment according to claim 4, characterized in that: The feature distance in step A331 is calculated using Euclidean distance or cosine distance.

Citation Information

Patent Citations

  • Panoramic image saliency prediction method based on self-supervised learning

    CN115631121A

  • KR20230101974A