Unmanned aerial vehicle target geographic positioning method and system based on implicit feature enhancement
By constructing a satellite image library and using implicit feature enhancement technology, the geographic positioning model of UAV targets was optimized, solving the problems of UAV positioning accuracy and stability under adverse weather conditions, and achieving efficient and accurate geographic positioning in complex environments.
Patent Information
- Application Number
- CN202510847433.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-24
- Publication Date
- 2025-10-17
Smart Images
Figure CN120808188A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of geographic positioning, in particular to a UAV target geographic positioning method and system based on implicit feature enhancement. BACKGROUND
[0002] UAV target geographic positioning generally involves a matching task of corresponding images between UAV and satellite platforms. Given an image from the perspective of a UAV as a query, the geographic positioning system aims to identify the most relevant satellite perspective image from the candidate image library. These satellite perspective images contain GPS information, so the UAV can naturally determine its geographic location.
[0003] However, satellite navigation systems rely on the ability to capture and track satellite signals. In some application scenarios, factors such as signal blockage or radio frequency interference may cause system signal interruption, thereby affecting the normal operation of the system. The existing UAV positioning platform positioning method in the absence of satellite navigation environment lacks the ability to implicitly learn the potential structural features of the scene, resulting in insufficient robust feature representation, and does not consider the association of implicit environmental variables behind the domain transfer between training data and multiple test environments, ultimately leading to a significant decrease in UAV target geographic positioning accuracy in multiple environments due to the insufficient discriminability of explicit feature representation. For example, when the UAV encounters fog, rain, snow and other bad weather, the image contrast decreases and the feature points are difficult to extract, which easily leads to a sharp increase in visual matching error, seriously affecting the accuracy and stability of the system geographic positioning. SUMMARY
[0004] In view of this, the UAV target geographic positioning method and system based on implicit feature enhancement are proposed to solve the problems existing in the current technology.
[0005] The UAV target geographic positioning method based on implicit feature enhancement is proposed, comprising:
[0006] Obtaining a plurality of satellite images with coordinates to construct a satellite image library;
[0007] Constructing a UAV target geographic positioning model based on implicit feature enhancement, training the UAV target geographic positioning model through a data-enhanced training data set, and optimizing the UAV target geographic positioning model through a pre-set loss function to obtain a UAV target geographic positioning final model;
[0008] Obtaining a UAV perspective image, inputting the UAV perspective image and all the satellite images with coordinates in the satellite image library into the UAV target geographic positioning final model, and outputting a one-dimensional feature vector corresponding to each of the UAV perspective image and all the satellite images with coordinates;
[0009] Calculate cosine distance between one-dimensional feature vector of the UAV perspective image and one-dimensional feature vector of each corresponding satellite image with coordinates respectively, obtain the maximum cosine distance, and take the satellite image with coordinates corresponding to the maximum cosine distance as the geographical coordinate signal of the UAV perspective image, and output the geographical coordinate signal.
[0010] In some embodiments of the present application, the UAV target geographical positioning model comprises a shared weight double-branch network, a square division module, a classifier module and an implicit enhancement module connected in sequence.
[0011] In some embodiments of the present application, the implicit enhancement module is used to perform implicit enhancement on the output of the classifier module, and specifically comprises:
[0012] The feature vector F output by the Drop-out layer in the classifier module is expanded to F N×C×A =F·1 1×C×1 , to obtain a tensor with a shape of [N,C,A];
[0013] Wherein, 1 is a full one tensor; N is the batch size, that is, the number of samples input into the model for processing at a time; C is the total number of categories; A is the number of feature channels, that is, the information channel of different dimensions in the feature vector;
[0014] Generate a one-hot encoding matrix O∈R N×C , R N×C is a matrix with N rows and C columns composed of real elements, which satisfies O n,c =1 if the nth sample belongs to category c, otherwise O n,c =0; c∈C, c is the cth category, n∈N, n is the nth sample;
[0015] Expand the one-hot encoding O N×C×A =O·1 1×1×A , with a shape of [N,C,A];
[0016] Calculate the category mask feature F mask =F N×C×A ·O N×C×A ;
[0017] Calculate the feature mean vector μ c of the cth category in the current batch of samples, and μ c ∈R A , to obtain a mean matrix μ∈R c composed of c feature mean vectors μ C×A ;
[0018] The specific calculation formula of μ c is as follows:
[0019]
[0020] Among them, ε is the minimum value to avoid the denominator being zero;
[0021] Calculate the squared deviation V between each sample feature and the corresponding category mean temp :
[0022] V temp =(F mask -μ·O N×C×A ) 2 ;
[0023] V temp Sum along the sample dimension N and normalize to get the variance vector for each category and The specific calculation formula is as follows:
[0024]
[0025] Define the weight coefficient w of the c-th category sample c , w c Indicates the contribution ratio of the current batch of new samples to the historical statistics. The specific calculation formula is as follows:
[0026]
[0027] Among them, new_count c is the number of samples in the current batch that belong to the cth category; is the total number of samples belonging to category c in the historical batch samples;
[0028] Fusion of historical statistics and current batch statistics, update the covariance vector CoVariance of the cth class c , the specific calculation formula is as follows:
[0029]
[0030] Among them, CoVariance c ∈R A ; is the historical covariance vector of the cth class; is the historical mean vector of the c-th type feature; is the updated mean vector of the c-th category feature;
[0031] Update the mean vector of each c-class feature and the sample count of each c-class. The specific calculation formula is as follows:
[0032]
[0033] Among them, Amountc is the total number of samples belonging to the c-th class in the current batch and the historical batch;
[0034] obtaining a weight matrix W e R C×A of the last fully connected layer of the classifier module, which is extended to W N×C×A = W · 1 N×1×1 ;
[0035] extracting a weight vector W k e R N×1×A of the class to which the sample belongs according to a sample class label y e [1, C], k being the true class index of the sample, calculating a difference AW between the global weight formed by the weight matrix of the last fully connected layer of the classifier module and the sample class weight;
[0036] AW = W N×C×A - W k ;
[0037] combining the covariance vector to calculate the variance term of the feature space:
[0038]
[0039] wherein, is the variance disturbance of N samples on C classes; CoVariance e R C×A represents a covariance matrix composed of covariance vectors of c classes;
[0040] enhancing the output of the classifier module through variance disturbance to obtain the enhanced feature
[0041] wherein, is the variance disturbance of N samples on C classes; 0.5 is a specified feature enhancement parameter.
[0042] In some embodiments of the present application, the shared weight double-branch network is a double-branch feature extraction structure, including a satellite perspective branch and a UAV perspective branch; the double-branch feature extraction structure adopts a pre-trained ConvNeXt-Base as a backbone network for feature extraction, and inputs the feature map output by the last layer of the ConvNeXt-Base backbone network into the square division module.
[0043] In some embodiments of the present application, the square division module performs square segmentation on the last output of the shared weight double-branch network in space to obtain a plurality of non-overlapping square features, and converts each square feature into a mean-pooling feature vector through an average pooling layer.
[0044] In some embodiments of the present application, the classifier module comprises linear layers, batch normalization layers, Drop-out layers and classification layers connected in sequence, and all the mean-pooling feature vectors are taken as inputs of the classifier module, and each corresponding column vector is output.
[0045] In some embodiments of the present application, the output of the classifier module is implicitly enhanced by the implicit feature enhancement module and then taken as an input of the loss function.
[0046] In some embodiments of the present application, the loss function comprises a cross-entropy loss function and a center loss function.
[0047] In some embodiments of the present application, when the UAV target geolocation model is trained by the data enhanced training data set, the following steps are included:
[0048] The UAV images of the training data set are preprocessed by an image-based style conversion library imgaug to obtain nine data sets containing different environment images, which are respectively: haze, shower, blizzard, rain and haze, snow and haze, rain and snow, darkness, overexposure, and gale.
[0049] Compared with the prior art, the present application has the following advantages:
[0050] Based on the prior art, the present application realizes efficient and accurate UAV geolocation under the influence of various adverse environments by constructing a satellite image library, using implicit feature enhancement and data enhancement technology, using cosine distance for similarity measurement, and training and optimizing the final model. The implicit feature enhancement optimizes the feature representation, realizes the dimension expansion and semantic enrichment of the original feature vector, and significantly improves the stability and robustness of the UAV target geolocation system in complex weather and light change scenes.
[0051] On the other hand, the present application also provides a UAV target geolocation system based on implicit feature enhancement, which comprises a UAV, an onboard camera fixed on the UAV, and a computer system, the computer system is electrically connected with the onboard camera, the computer system applies the UAV target geolocation method based on implicit feature enhancement as described above, and the computer system comprises a processing unit and a storage unit:
[0052] The storage unit is used for acquiring and storing the UAV perspective images taken by the onboard camera and a plurality of satellite images with coordinates.
[0053] The processing unit is configured to receive the UAV perspective image and the plurality of satellite images with coordinates acquired by the storage unit, and perform similarity matching on the UAV perspective image and the plurality of satellite images with coordinates by using a pre-stored UAV target geographical positioning final model, output one-dimensional feature vectors corresponding to the UAV perspective image and all the satellite images with coordinates respectively, and acquire geographical coordinate signals of the satellite image with coordinates corresponding to the maximum cosine distance between the one-dimensional feature vector of the UAV perspective image and one-dimensional feature vectors of all the satellite images with coordinates as latitude and longitude coordinates corresponding to the UAV perspective image and output the latitude and longitude coordinates.
[0054] It can be understood that the UAV target geographical positioning system based on implicit feature enhancement in the embodiment has the same beneficial effects as the UAV target geographical positioning method based on implicit feature enhancement described above, and will not be described here. BRIEF DESCRIPTION OF DRAWINGS
[0055] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments and are not meant to limit the present application. Furthermore, the same reference numerals are used throughout the several drawings to designate the same or similar parts. In the drawings:
[0056] Figure 1 A flowchart of the UAV target geographical positioning method based on implicit feature enhancement provided by the embodiment of the present application. DETAILED DESCRIPTION
[0057] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the present disclosure are shown. It is to be understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
[0058] Reference Figure 1 As shown, the embodiment provides a UAV target geographical positioning method based on implicit feature enhancement, which comprises:
[0059] Acquire a plurality of satellite images with coordinates to construct a satellite image library;
[0060] The UAV target geographic positioning model based on implicit feature enhancement is constructed, the UAV target geographic positioning model is trained through the data enhanced training data set, and the UAV target geographic positioning model is optimized through a preset loss function, so that the UAV target geographic positioning final model is obtained.
[0061] The UAV perspective image is acquired, the UAV perspective image and all the satellite images with coordinates in the satellite image library are input into the UAV target geographic positioning final model, and one-dimensional feature vectors corresponding to the UAV perspective image and all the satellite images with coordinates are output.
[0062] The cosine distances between the one-dimensional feature vector of the UAV perspective image and the one-dimensional feature vectors corresponding to all the satellite images with coordinates are respectively calculated, and the geographic coordinate signal of the satellite image with the maximum cosine distance is acquired as the longitude and latitude coordinates corresponding to the UAV perspective image and is output.
[0063] It can be understood that, in the embodiment, the satellite image library is constructed to provide basic data for subsequent geographic positioning. The training data set is pre-processed for data enhancement, so as to increase the diversity of the training data and improve the generalization ability of the model. The UAV target geographic positioning final model based on implicit feature enhancement is obtained after a large amount of training and optimization, which not only improves the generalization ability of the model, but also enhances the applicability of the model in complex environments, and accurately converts the UAV perspective image into corresponding geographic coordinates.
[0064] Specifically, after the UAV target geographic positioning final model is obtained through model training, the UAV perspective image is subjected to geographic positioning inference through the UAV target geographic positioning final model, and then one-dimensional feature vectors corresponding to the UAV perspective image and all the satellite images with coordinates are output. The output of the Drop-out layer in the classifier module is spliced into a one-dimensional feature vector, that is, eight 512-dimensional feature vectors are spliced into a 4096-dimensional feature vector, so as to obtain one-dimensional feature vectors corresponding to satellite platform and UAV platform images. The cosine distance is calculated to match the feature similarity, the longitude and latitude coordinates of the satellite image corresponding to the maximum cosine distance are taken as the geographic positioning result of the UAV image, so that the aerial visual geographic positioning in multiple environments is realized.
[0065] Preferably, when the UAV target geographic positioning model is trained through the data enhanced training data set, the following steps are included.
[0066] The UAV images in the training data set are pre-processed through an image based style conversion library imgaug, so as to obtain nine data sets containing different environment images, which are: haze, shower, blizzard, rain and haze, snow and haze, rain and snow, darkness, overexposure, and gale.
[0067] It can be understood that, in the embodiment, the UAV image is combined with the background image under different weather conditions by the data enhancement technology to generate new training samples, which not only increases the diversity of the training data and enriches the content of the training data set, but also helps the model to better adapt to various actual environments, thereby improving the positioning accuracy of the model under complex conditions.
[0068] Preferably, the UAV target geographic positioning model comprises a shared weight double-branch network, a square division module, a classifier module and an implicit enhancement module connected in sequence.
[0069] Preferably, the implicit enhancement module is configured to perform implicit enhancement on the output of the classifier module, and specifically comprises:
[0070] The feature vector F output by the Drop-out layer in the classifier module is expanded to F N×C×A =F·1 1×C×1 , to obtain a tensor with a shape of [N,C,A];
[0071] wherein, 1 is a full one tensor; N is a batch size, i.e. the number of samples input to the model for processing at a time; C is a total number of categories; and A is a feature channel number, i.e. an information channel of different dimensions in the feature vector;
[0072] A one-hot encoding matrix O∈R N×C is generated, where R N×C is a matrix with N rows and C columns composed of real elements, and satisfies that if the nth sample belongs to the category c, then O n,c =1, otherwise O n,c =0; c∈C, c is the cth category, and n∈N, n is the nth sample;
[0073] The one-hot encoding O N×C×A is expanded to O 1×1×A ·1, with a shape of [N,C,A];
[0074] A category mask feature F mask is calculated, where F N×C×A =F N×C×A ·O
[0075] A feature mean vector μ c belonging to the cth category in the current batch of samples is calculated, and μ c ∈R A , to obtain a mean matrix μ∈R C×A composed of c feature mean vectors μ c .
[0076] The current batch refers to the set of samples that are input into the model at once during model training or inference. The current batch is defined by the order in which the data is input into the model. The N samples input into the model for the tth time constitute the current batch. The samples input up to and including the t-1th time constitute the historical batch. The statistical information calculated from these samples, such as the mean and variance of each category, is considered historical sample statistics.
[0077] μ c The specific calculation formula is as follows:
[0078]
[0079] Among them, ε is the minimum value to avoid the denominator being zero;
[0080] Calculate the squared deviation V between each sample feature and the corresponding category mean temp :
[0081] V temp =(F mask -μ·O N×C×A ) 2 ;
[0082] V temp Sum along the sample dimension N and normalize to get the variance vector for each category and The specific calculation formula is as follows:
[0083]
[0084] Define the weight coefficient w of the c-th category sample c , w c Indicates the contribution ratio of the current batch of new samples to the historical statistics. The specific calculation formula is as follows:
[0085]
[0086] Among them, new_count c is the number of samples in the current batch that belong to the cth category; is the total number of samples belonging to category c in the historical batch samples; the total number of historical samples is a counting indicator used to quantify the scale of historical data; the historical sample statistics are calculated based on the total number of historical samples and sample characteristics.
[0087] Fusion of historical statistics and current batch statistics, update the covariance vector CoVariance of the cth class c , the specific calculation formula is as follows:
[0088]
[0089] where CoVariance c ∈R A ; is the historical covariance vector of the c-th class; is the historical mean vector of the c-th class feature; is the updated mean vector of the c-th class feature;
[0090] The mean vector of each c-th class feature is updated with each c-th class sample count, as follows:
[0091]
[0092] where Amount c is the total number of samples belonging to the c-th class in the current batch and historical batch samples;
[0093] The weight matrix W ∈ R C×A of the last fully connected layer of the classifier module is extended to W N×C×A = W·1 N×1×1 ;
[0094] According to the sample class label y ∈ [1, C], the weight vector W k ∈R N×1×A of the class to which the sample belongs is extracted, k is the real class index of the sample, and the difference ΔW between the global weight formed by the extended dimension of the weight matrix of the last fully connected layer of the classifier module and the sample class weight is calculated.
[0095] ΔW = W N×C×A - W k ;
[0096] The variance term of the feature space is calculated in combination with the covariance vector:
[0097]
[0098] where, is the variance disturbance of N samples on C classes; CoVariance ∈ R C×A represents a covariance matrix composed of c class covariance vectors;
[0099] The output of the classifier module is enhanced by the variance disturbance to obtain the enhanced feature
[0100] where, is the variance disturbance of N samples on C classes; 0.5 is a specified feature enhancement parameter.
[0101] It can be understood that, in the embodiment, the output of the classifier module is implicitly enhanced by the implicit enhancement module, so as to avoid insufficient robust feature representation due to lack of implicit learning ability of potential structural features of the scene in a complex scene, and improve the unmanned aerial vehicle target geographic positioning accuracy in multiple environments.
[0102] Preferably, the shared weight double-branch network is a double-branch feature extraction structure, including a satellite view branch and a drone view branch; the double-branch feature extraction structure adopts a pre-trained ConvNeXt-Base as a backbone network for feature extraction, and inputs a feature map output by the last layer of the ConvNeXt-Base backbone network into the square division module.
[0103] Specifically, when the double-branch feature extraction structure adopts a pre-trained ConvNeXt-Base as a backbone network for feature extraction, the following steps are included:
[0104] Let the backbone network be F backbone ( );
[0105] Let the feature extraction process be f j = F backbone (x j ); wherein f j is a feature extraction map corresponding to an input image; x j is an input image corresponding to different image platforms; j = satellite, drone, satellite is a satellite platform, and drone is a drone platform.
[0106] Specifically, the shared weight double-branch network in the embodiment receives a drone view image and a satellite image with coordinates for feature encoding, and respectively outputs a corresponding encoded feature vector of the drone view image and the satellite image with coordinates, realizes feature space alignment between images of different platforms (satellite images and drone images), so that image features of different platforms can be effectively integrated and compared, thereby improving the efficiency and accuracy of image analysis and processing.
[0107] Preferably, the square division module performs square division on the last output of the shared weight double-branch network in space to obtain a plurality of non-overlapping square features, and converts each square feature into a mean pooling feature vector through an average pooling layer.
[0108] Specifically, the square division module is used to perform square division on the feature map f j obtained by the shared weight double-branch network in space, and the input feature map f j is divided into 8 non-overlapping square features Then apply the average pooling layer to convert each square feature into a 2048-dimensional average pooling feature vector
[0109] It can be understood that the square partitioning module in this embodiment effectively extracts and utilizes local features in the image through spatial segmentation and mean pooling, thereby improving the performance and generalization ability of the model. Compared with global pooling, square partitioning and mean pooling can reduce the amount of computation. By focusing on features at different locations, the network's generalization ability is improved, and it can better handle image recognition tasks in different scenarios.
[0110] Specifically, the square division module includes the following processing steps:
[0111] Receive the feature map f output by the shared weight dual-branch network j , and obtain the feature map f j The height H and width W;
[0112] Determine f j The center coordinates of the feature map (c h ,c w ), where c h =[H / 2], c w =[W / 2].
[0113] Preset the number of blocks and calculate the height per_h and width per_w of each block. The specific calculation formula is as follows:
[0114]
[0115] Furthermore, if per_h < 1 and per_w < 1 after size verification and difference adjustment, the feature map size is adjusted to [new_H, new_W] through bilinear interpolation. The specific calculation formula is as follows:
[0116] new_H=H+2×(block-c h );
[0117] new_W=W+2×(block-c w );
[0118] Re-obtain the resized feature map f j The height, width and center coordinates of the block are calculated, and the height and width of each block are recalculated and rounded to integer pixel values.
[0119] Based on the preset number of blocks, the square area is divided layer by layer from the center outward, including the following steps:
[0120] The divided blocks are traversed in a loop, coordinates of the i-th layer region are calculated, i = 1, 2, …, block, the top-left corner coordinates of the i-th layer region are (c h -i×per_h,c w -i×per_w), the bottom-right corner coordinates of the i-th layer region are (c h +i×per_h,c w +i×per_w), the i-th layer region feature map x curr is extracted; when the i-th layer region feature map x curr contains the previous layer region feature map, i.e., i > 1, the region feature map x pre of the previous layer (i-1) is obtained, and the region feature map x pad of the i-th layer is generated by a padding operation to have the same size as the i-th layer and eliminate the overlapping part of the region of the previous layer (i-1) and the region of the i-th layer; when i = block, if in the non-overlapping mode and i > 1, the padding amount is dynamically calculated according to the edge size of the feature map to ensure that the size after padding is consistent with the current layer, and x before -x pad is obtained by completing the non-overlapping division of the edge region, x before is the feature map before the non-overlapping division, and x is the feature map after the non-overlapping division; when in the overlapping mode and i > 1, the overlapping part of the region of the previous layer (i-1) and the region of the i-th layer is eliminated, and the specific calculation formula is as follows:
[0121] x curr_消除后 =x curr_消除前 -x pad ;
[0122] wherein x curr_消除前 is the feature map without eliminating the region of the previous layer (i-1); and x curr_消除后 is the feature map after eliminating the region of the previous layer (i-1).
[0123] After the square division is completed, mean pooling is performed on each square region feature map:
[0124] When there is a region feature map x pre of the previous layer (i-1), the effective pixel number of the i-th layer region feature map is pix_num, and the calculation formula of pix_num is as follows:
[0125] pix_num = h × w - h pre × w pre ;
[0126] wherein h and w are the height and width of the i-th layer region feature map, h pre and w preheight and width of the region feature map of the previous layer (i-1);
[0127] When the region feature map x pre of the previous layer (i-1) does not exist, i.e., there is no overlap, such as the first layer, the number of valid pixels of the region feature map of the i-th layer is pix_num, pix_num = h x w;
[0128] The region feature map x curr of the i-th layer is flattened into a two-dimensional tensor, summed along the spatial dimension, and then divided by the number of valid pixels pix_num to obtain the mean-pooled feature vector of the region feature map of the i-th layer as input to the classifier module.
[0129] Preferably, the classifier module includes linear layers, batch normalization layers, Drop-out layers, and classification layers connected in sequence, and all the mean-pooled feature vectors are input to the classifier module as input, and the classifier module outputs a respective column vector.
[0130] Specifically, the classifier module specifically includes:
[0131] All the mean-pooled feature vectors of all levels are input to the classifier as input, and the classifier module outputs a corresponding column vector The dimension of the column vector is equal to the total number C of sample class labels y ∈ [1, C].
[0132] The specific process can be represented as:
[0133] Preferably, the output of the classifier module is implicitly enhanced by an implicit enhancement module and then input to a loss function as input, specifically including:
[0134] The feature vector F output by the Drop-out layer in the classifier module is expanded to F N×C×A = F·1 1×C×1 to obtain a tensor with a shape of [N, C, A];
[0135] where 1 is an all-one tensor; N is the batch size, i.e., the number of samples input to the model for processing at a time; C is the total number of categories; and A is the number of feature channels, i.e., the information channels of different dimensions in the feature vector.
[0136] A one-hot encoding matrix O ∈ R N×C is generated, where R N×C is a matrix with N rows and C columns composed of real elements, and satisfies O n,c = 1 if the n-th sample belongs to category c, and O n,c = 0 otherwise; c ∈ C, c is the c-th category, and n ∈ N, n is the n-th sample.
[0137] Extended One-Hot Encoding O N×C×A =O·1 1×1×A , the shape is [N,C,A];
[0138] Calculate the category mask feature F mask =F N×C×A ·O N×C×A ;
[0139] Calculate the feature mean vector μ belonging to the cth class in the current batch of samples c , and μ c ∈R A , we get the c feature mean vector μ c The mean matrix μ∈R C×A .
[0140] The current batch refers to the set of samples that are input into the model at once during model training or inference. The current batch is defined by the order in which the data is input into the model. The N samples input into the model for the tth time constitute the current batch. The samples input up to and including the t-1th time constitute the historical batch. The statistical information calculated from these samples, such as the mean and variance of each category, is considered historical sample statistics.
[0141] μ c The specific calculation formula is as follows:
[0142]
[0143] Among them, ε is the minimum value to avoid the denominator being zero;
[0144] Calculate the squared deviation V between each sample feature and the corresponding category mean temp :
[0145] V temp =(F mask -μ·O N×C×A ) 2 ;
[0146] V temp Sum along the sample dimension N and normalize to get the variance vector for each category and The specific calculation formula is as follows:
[0147]
[0148] Define the weight coefficient w of the c-th category sample c , w c Indicates the contribution ratio of the current batch of new samples to the historical statistics. The specific calculation formula is as follows:
[0149]
[0150] wherein, new_count c is the number of the c-th class in the current batch of samples; old_count c is the total number of the c-th class in the history samples; the total number of history samples is a counting indicator for quantifying the size of the history data; the history sample statistics is calculated based on the total number of history samples and the sample features.
[0151] The history statistics and the current batch statistics are fused to update the covariance vector CoVariance c of the c-th class, and the specific calculation formula is as follows:
[0152]
[0153] wherein, CoVariance c ∈R A ; is the history covariance vector of the c-th class; is the history mean vector of the c-th class features; is the updated mean vector of the c-th class features;
[0154] The mean vector of each c-th class feature and the sample count of each c-th class are updated, and the specific calculation formula is as follows:
[0155]
[0156] wherein, Amount c is the total number of the c-th class in the current batch of samples and the history batch of samples; is the total number of the c-th class in the history batch of samples;
[0157] The weight matrix W ∈R C×A of the last full connection layer of the classifier module is obtained, which is extended to W N×C×A =W·1 N×1×1 ;
[0158] According to the sample class label y ∈[1,C], the weight vector W k ∈R N×1×A of the class to which the sample belongs is extracted, k is the real class index of the sample, and the difference ΔW between the global weight formed by the extended dimension of the weight matrix of the last full connection layer of the classifier module and the sample class weight is calculated.
[0159] ΔW=W N×C×A -W k ;
[0160] Compute variance terms of feature space with covariance vector:
[0161]
[0162] where, is the variance perturbation of N samples on C classes; CoVariance ∈ R C×A denotes the covariance matrix composed of c classes of covariance vectors;
[0163] Enhance the output of the classifier module by variance perturbation Get enhanced features
[0164] where, is the variance perturbation of N samples on C classes; 0.5 is the specified feature enhancement parameter.
[0165] Preferably, the loss function includes a cross-entropy loss function and a center loss function.
[0166] Specifically, two different loss functions are used to optimize the model, namely cross-entropy loss and center loss. During model training, the vector output by the classifier module is obtained after implicit feature enhancement which is input to the cross-entropy loss function, while the vector output by the classifier module is the original feature, which is input to the center loss function, and the loss is calculated respectively.
[0167] The sum of the loss values of the two loss functions L total is calculated total L CE + L Center , until and the minimum is obtained, to obtain the final model for geographic positioning of unmanned aerial vehicle targets, where L CE is the cross-entropy loss function value, and L Center is the center loss function value.
[0168] Specifically, the cross-entropy loss is mainly used to quantify the difference between the model prediction probability distribution and the actual distribution. It should be noted that for multiple different feature vectors obtained by the implicit enhancement module, the loss values are calculated respectively and then unified optimization processing is performed. The center loss function learns a center for each class, so as to ensure that the samples of the same class are more concentrated.
[0169] Based on the above loss function, optimization is performed to promote the one-dimensional feature vectors of paired aerial perspective images (i.e., drone perspective images and satellite images of corresponding locations) to gradually approach each other in the feature space, while the feature vectors of unpaired image pairs (drone perspective images and satellite images of non-corresponding locations) to gradually move away from each other. This promotes the convergence of the loss value of the drone target geolocation model based on implicit feature enhancement, and achieves the discriminability enhancement and metric space alignment of image features from different platforms (drone platform, satellite platform).
[0170] On the other hand, the present application also provides a UAV target geolocation system based on implicit feature enhancement, comprising a UAV, an onboard camera fixed to the UAV, and a computer system, wherein the computer system is electrically connected to the onboard camera, and the computer system applies the UAV target geolocation method based on implicit feature enhancement as described above, and the computer system includes a processing unit and a storage unit:
[0171] The storage unit is used to obtain and store the drone's perspective images taken by the onboard camera and a number of satellite images with coordinates;
[0172] The processing unit is used to receive the drone perspective image and several satellite images with coordinates obtained by the storage unit, and perform similarity matching on the drone perspective image and the several satellite images with coordinates through a pre-stored drone target geolocation final model, and output the one-dimensional feature vectors corresponding to the drone perspective image and all the satellite images with coordinates. The processing unit calculates the maximum cosine distance between the one-dimensional feature vector of the drone perspective image and the one-dimensional feature vectors corresponding to all the satellite images with coordinates, obtains the geographic coordinate signal of the satellite image with coordinates corresponding to the maximum cosine distance as the latitude and longitude coordinates corresponding to the drone perspective image, and outputs it.
[0173] It can be understood that the UAV target geolocation system based on implicit feature enhancement in this embodiment has the same beneficial effects as the above-mentioned UAV target geolocation method based on implicit feature enhancement, and will not be repeated here.
[0174] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware embodiments. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0175] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks
[0176] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks
[0177] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks
[0178] Finally, it should be noted that the above-mentioned embodiments are merely intended for describing the technical solutions of the present application, but not for limiting it. Although the present application is described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalent replaced without departing from the spirit and scope of the present application, and any modification or equivalent replacement should be covered in the protection scope of the claims of the present application.
Claims
1. A UAV target geolocation method based on implicit feature enhancement, characterized by: include: Obtain several satellite images with coordinates to build a satellite image library; Constructing a UAV target geolocation model based on implicit feature enhancement, training the UAV target geolocation model using the data-enhanced training dataset, and optimizing the UAV target geolocation model using a preset loss function to obtain a final UAV target geolocation model; Acquire a drone-viewed image, input the drone-viewed image and all the satellite images with coordinates in the satellite image library into the drone target geolocation final model, and output one-dimensional feature vectors corresponding to the drone-viewed image and all the satellite images with coordinates; Calculate the cosine distances between the one-dimensional feature vector of the drone perspective image and the one-dimensional feature vectors corresponding to all the coordinate satellite images respectively, obtain the geographic coordinate signal of the coordinate satellite image corresponding to the maximum value of the cosine distance as the latitude and longitude coordinates corresponding to the drone perspective image, and output it.
2. The method for UAV target geographic positioning based on implicit feature enhancement according to claim 1, characterized in that: The UAV target geographic positioning model includes a shared weight dual-branch network, a square partitioning module, a classifier module and an implicit enhancement module connected in sequence.
3. The method for UAV target geographic positioning based on implicit feature enhancement according to claim 2 is characterized in that: The implicit enhancement module is used to implicitly enhance the output of the classifier module, specifically including: Expand the feature vector F output by the Drop-out layer in the classifier module to F N×C×A =F·1 1×C×1 , get a tensor of shape [N, C, A]; Among them, 1 is a tensor of all 1s; N is the batch size, that is, the number of samples input into the model for processing at one time; C is the total number of categories; A is the number of feature channels, that is, the information channels of different dimensions in the feature vector; Generate one-hot encoding matrix O∈R N×C , R N×C is a matrix with N rows and C columns consisting of real elements, satisfying that if the nth sample belongs to category c, then O n,c =1, otherwise O n,c =0; c∈C, c is the cth category, n∈N, n is the nth sample; Extended One-Hot Encoding O N×C×A =O·1 1×1×A , the shape is [N,C,A]; Calculate the category mask feature F mask =F N×C×A ·O N×C×A ; Calculate the feature mean vector μ belonging to the cth class in the current batch of samples c , and μ c ∈R A , we get the c feature mean vector μ c The mean matrix μ∈R C×A ; μ c The specific calculation formula is as follows: Among them, ε is the minimum value to avoid the denominator being zero, and the symbol ":" is a general slice operator that represents all elements of the dimension; Calculate the squared deviation V between each sample feature and the corresponding category mean temp : V temp =(F mask -m·O N×C×A ) 2 ; V temp Sum along the sample dimension N and normalize to get the variance vector for each category and The specific calculation formula is as follows: Define the weight coefficient w of the c-th category sample c , w c Indicates the contribution ratio of the current batch sample to the historical statistics. The specific calculation formula is as follows: Among them, new_count c is the number of samples in the current batch that belong to the cth category; is the total number of samples belonging to category c in the historical batch samples; Fusion of historical statistics and current batch statistics, update the covariance vector CoVariance of the cth class c , the specific calculation formula is as follows: Among them, CoVariance c ∈R A ; is the historical covariance vector of the cth class; is the historical mean vector of the c-th type feature; is the updated mean vector of the c-th category feature; Update the mean vector of each c-class feature and the sample count of each c-class. The specific calculation formula is as follows: Among them, Amount c is the total number of samples belonging to category c in the current batch samples and historical batch samples Get the weight matrix W∈R of the last fully connected layer of the classifier module C×A , expanded to W N×C×A =W·1 N×1×1 ; Extract the weight vector W of the category to which the sample belongs according to the sample category label y∈[1,C] k ∈R N×1×A , k is the true category index of the sample, and the difference ΔW between the global weight formed by the expanded dimension of the weight matrix of the last fully connected layer of the classifier module and the sample category weight is calculated; ΔW=W N×C×A -IN k ; Combined with the covariance vector, the variance term of the feature space is calculated: in, is the variance perturbation of N samples on C categories; CoVariance∈R C×A represents the covariance matrix composed of the covariance vectors of c categories; Enhance the output of the classifier module by variance perturbation The enhanced features in, is the variance perturbation of N samples on category C; 0.5 is the specified feature enhancement parameter.
4. The method for UAV target geographic positioning based on implicit feature enhancement according to claim 2, characterized in that: The shared weight dual-branch network is a dual-branch feature extraction structure, including a satellite perspective branch and a drone perspective branch; the dual-branch feature extraction structure uses the pre-trained ConvNeXt-Base as the backbone network for feature extraction, and inputs the feature map output by the last layer of the ConvNeXt-Base backbone network into the square division module.
5. The method for UAV target geographic positioning based on implicit feature enhancement according to claim 2, characterized in that: The square division module performs spatial square division on the final output of the shared weight dual-branch network to obtain multiple non-overlapping square features, and converts each square feature into a mean pooling feature vector through an average pooling layer.
6. The method for UAV target geographic positioning based on implicit feature enhancement according to claim 5, characterized in that: The classifier module includes a linear layer, a batch normalization layer, a drop-out layer and a classification layer connected in sequence, and all the mean pooled feature vectors are used as inputs of the classifier module, and the corresponding column vectors are output.
7. The method for UAV target geographic positioning based on implicit feature enhancement according to claim 2, characterized in that: The output of the classifier module is implicitly enhanced by the implicit enhancement module and then used as the input of the loss function.
8. The method for UAV target geographic positioning based on implicit feature enhancement according to claim 1, characterized in that: The loss function includes a cross entropy loss function and a center loss function.
9. The method for UAV target geographic positioning based on implicit feature enhancement according to claim 1, characterized in that: When training the UAV target geolocation model using the data-enhanced training dataset, the following steps are included: The drone images of the training dataset are preprocessed using the image-based style transfer library imgaug to obtain nine datasets containing images of different environments, namely: haze, shower, blizzard, rain and fog, snow and fog, rain and snow, darkness, overexposure, and storm.
10. An unmanned aerial vehicle (UAV) target geographic positioning system based on implicit feature enhancement, comprising a UAV, an onboard camera fixed to the UAV, and a computer system, wherein the computer system is electrically connected to the onboard camera, and is characterized in that: The computer system applies the UAV target geolocation method based on implicit feature enhancement as described in any one of claims 1 to 9 above, and the computer system includes a processing unit and a storage unit: The storage unit is used to obtain and store the drone's perspective image and a plurality of satellite images with coordinates taken by the onboard camera; The processing unit is used to receive the drone-perspective image and the plurality of satellite images with coordinates acquired by the storage unit, and perform similarity matching on the drone-perspective image and the plurality of satellite images with coordinates using a pre-stored drone target geolocation final model, and output one-dimensional feature vectors corresponding to each of the drone-perspective image and all the satellite images with coordinates. The processing unit calculates the maximum cosine distance between the one-dimensional feature vector of the drone-perspective image and the one-dimensional feature vectors corresponding to each of the satellite images with coordinates, obtains the geographic coordinate signal of the satellite image with the maximum cosine distance as the latitude and longitude coordinates corresponding to the drone-perspective image, and outputs the result.