Unmanned aerial vehicle positioning method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an unmanned aerial vehicle positioning method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: respectively extracting features of a sample image through a first network layer and a second network layer of a to-be-trained model, and obtaining sample fusion features according to the extracted features; processing the sample fusion feature through a plurality of encoders connected in series of the to-be-trained model to obtain a sample target feature; each encoder comprises multiple attention layers; according to the sample target features and training labels corresponding to the sample images, training the to-be-trained model, in the training process, keeping an initial weight matrix in the multi-head attention layer unchanged, and updating a low-rank matrix in the multi-head attention layer; calling a feature extraction model to process the to-be-processed image to obtain a target feature; and searching a target image from an image library according to the target feature, and obtaining the position information of the unmanned aerial vehicle according to the position information corresponding to the target image. By adopting the method, the model training cost can be reduced.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a method, apparatus, computer equipment, readable storage medium, and program product for positioning a drone. Background Art

[0002] During power inspections, drones often encounter complex terrain, changeable weather conditions, and potential sources of electromagnetic interference. Especially in specific scenarios such as deep mountains and forests, canyons and ravines, or near high-voltage power lines, GPS (Global Positioning System) signals are easily blocked or interfered with, resulting in weak positioning capabilities of drones. Therefore, the need for autonomous positioning of drones has become increasingly important.

[0003] At present, a positioning technology based on image retrieval has been proposed. The implementation of this technology depends on a trained model. However, the cost of model training is currently high and it is difficult to meet ordinary engineering requirements, making it difficult to achieve drone positioning based on image retrieval. Summary of the Invention

[0004] Based on this, it is necessary to provide a drone positioning method, device, computer equipment, readable storage medium and computer program product that can reduce the cost of model training and thereby reduce the difficulty of drone positioning based on image retrieval to address the above technical problems.

[0005] In the first aspect, the present application provides a method for positioning a drone, which includes: obtaining a sample image, which is an image taken by a drone during an electric power inspection process; extracting features of the sample image through the first network layer and the second network layer of the model to be trained, respectively, and obtaining sample fusion features based on the extracted features; processing the sample fusion features through multiple encoders in series of the model to be trained to obtain sample target features; wherein each encoder includes a multi-head attention layer; training the model to be trained based on the sample target features and the training labels corresponding to the sample images, and obtaining a feature extraction model after the training is completed; wherein, during the training process, the initial weight matrix in the multi-head attention layer remains unchanged, and the low-rank matrix in the multi-head attention layer is updated; calling the feature extraction model to process the image to be processed to obtain target features; finding the target image from the image library based on the target features, and obtaining the location information of the drone based on the location information corresponding to the target image.

[0006] In one embodiment, the second network layer includes a first sub-network layer, a second sub-network layer and a third sub-network layer; the features of the sample image are extracted respectively through the first network layer and the second network layer of the model to be trained, and the sample fusion features are obtained according to the extracted features, including: processing the sample image through the first sub-network layer to obtain the first sample feature; processing the first sample feature through the second sub-network layer to obtain the second sample feature; processing the first sample feature through the third sub-network layer to obtain the third sample feature; fusing the second sample feature and the third sample feature with the feature extracted by the first network layer to obtain the sample fusion feature.

[0007] In one embodiment, the second sub-network layer includes a bilinear interpolation processing layer, a first depthwise separable convolution layer and a first flattening layer; the first sample feature is processed by the second sub-network layer to obtain the second sample feature, including: performing bilinear interpolation processing on the first sample feature by the bilinear interpolation processing layer to obtain the first sample feature processed by bilinear interpolation; performing depthwise separable convolution processing on the first sample feature processed by bilinear interpolation by the first depthwise separable convolution layer to obtain the first sample feature processed by depthwise separable convolution; and flattening the first sample feature processed by depthwise separable convolution by the first flattening layer to obtain the second sample feature.

[0008] In one embodiment, the third sub-network layer includes a feature extraction block, a downsampling layer, a second depthwise separable convolution layer and a second flattening layer; the first sample feature is processed by the third sub-network layer to obtain a third sample feature, including: performing feature extraction processing on the first sample feature by the feature extraction block to obtain a first sub-sample feature; downsampling the sub-sample feature by the downsampling layer to obtain a downsampled sub-sample feature; performing depthwise separable convolution processing on the downsampled sub-sample feature by the second depthwise separable convolution layer to obtain a second sub-sample feature; and flattening the second sub-sample feature by the second flattening layer to obtain a third sample feature.

[0009] In one embodiment, the feature extraction block includes a first residual block, a second residual block and a multi-layer perceptron; the feature extraction block performs feature extraction processing on the first sample feature to obtain a first sub-sample feature, including: inputting the first sample feature and the first sample feature after convolution processing into the first residual block to obtain a first residual feature; inputting the first residual feature and the first residual feature after attention processing into the second residual block to obtain the output of the second residual block; inputting the output of the second residual block into the multi-layer perceptron to obtain the output of the multi-layer perceptron, and using the output of the multi-layer perceptron as the first sub-sample feature.

[0010] In one embodiment, features of a sample image are extracted respectively by the first network layer and the second network layer of the model to be trained, and sample fusion features are obtained based on the extracted features, including: segmenting the sample image by the first network layer to obtain multiple image blocks, and performing vector conversion processing on the multiple image blocks to obtain features extracted by the first network layer; and fusing the features extracted by the first network layer and the features extracted by the second network layer to obtain sample fusion features.

[0011] In a second aspect, the present application also provides a drone positioning device, which includes:

[0012] An acquisition module is used to acquire sample images, which are images taken by the drone during the power inspection process;

[0013] A first determination module is used to extract features of the sample image through the first network layer and the second network layer of the to-be-trained model, respectively, and obtain a sample fusion feature based on the extracted features;

[0014] A second determination module is configured to process the sample fusion features through a plurality of encoders connected in series of the model to be trained to obtain the sample target features; wherein each encoder includes a multi-head attention layer;

[0015] The training module is used to train the training model based on the sample target features and the training labels corresponding to the sample images, and obtain the feature extraction model after the training is completed. During the training process, the initial weight matrix in the multi-head attention layer remains unchanged, and the low-rank matrix in the multi-head attention layer is updated;

[0016] The third determination module is used to call the feature extraction model to process the image to be processed to obtain the target features; find the target image from the image library according to the target features, and obtain the location information of the UAV according to the location information corresponding to the target image.

[0017] In one embodiment, the second network layer includes a first sub-network layer, a second sub-network layer and a third sub-network layer; the first determination module is specifically used to process the sample image through the first sub-network layer to obtain a first sample feature; process the first sample feature through the second sub-network layer to obtain a second sample feature; process the first sample feature through the third sub-network layer to obtain a third sample feature; and fuse the second sample feature and the third sample feature with the feature extracted by the first network layer to obtain a sample fusion feature.

[0018] In one embodiment, the second sub-network layer includes a bilinear interpolation processing layer and a first depthwise separable convolution layer; the first determination module is specifically used to perform bilinear interpolation processing on the first sample feature through the bilinear interpolation processing layer to obtain the first sample feature processed by bilinear interpolation; perform depthwise separable convolution processing on the first sample feature processed by bilinear interpolation through the first depthwise separable convolution layer to obtain the first sample feature processed by depthwise separable convolution; and flatten the first sample feature processed by depthwise separable convolution through the first flattening layer to obtain the second sample feature.

[0019] In one embodiment, the third sub-network layer includes a feature extraction block, a downsampling layer, a second depthwise separable convolution layer and a second flattening layer; the first determination module is specifically used to perform feature extraction processing on the first sample feature through the feature extraction block to obtain a first sub-sample feature; downsampling the sub-sample feature through the downsampling layer to obtain a downsampled sub-sample feature; performing depthwise separable convolution processing on the downsampled sub-sample feature through the second depthwise separable convolution layer to obtain a second sub-sample feature; and flattening the second sub-sample feature through the second flattening layer to obtain a third sample feature.

[0020] In one embodiment, the feature extraction block includes a first residual block, a second residual block and a multi-layer perceptron; the first determination module is specifically used to input the first sample feature and the first sample feature after convolution processing into the first residual block to obtain the first residual feature; input the first residual feature and the first residual feature after attention processing into the second residual block to obtain the output of the second residual block; input the output of the second residual block into the multi-layer perceptron to obtain the output of the multi-layer perceptron, and use the output of the multi-layer perceptron as the first sub-sample feature.

[0021] In one embodiment, the first determination module is specifically used to segment the sample image through the first network layer to obtain multiple image blocks, and perform vector conversion processing on the multiple image blocks to obtain features extracted by the first network layer; the features extracted by the first network layer and the features extracted by the second network layer are fused to obtain sample fusion features.

[0022] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0023] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0024] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect above.

[0025] The aforementioned drone positioning method, apparatus, computer device, readable storage medium, and computer program product obtain sample images captured by a drone during a power inspection. The first and second network layers of a model to be trained are then used to extract features from the sample images. Sample fusion features are then generated based on the extracted features. The sample fusion features are then processed by multiple encoders connected in series within the model to be trained to obtain sample target features. Each encoder includes a multi-head attention layer. The model to be trained is then trained based on the sample target features and the training labels corresponding to the sample images. After training, a feature extraction model is obtained. During training, the initial weight matrix in the multi-head attention layer remains unchanged, while the low-rank matrix in the multi-head attention layer is updated. The feature extraction model is then invoked to process the image to be processed, obtaining target features. Based on the target features, a target image is retrieved from an image library, and the drone's location information is obtained based on the corresponding location information of the target image. Because the initial weight matrix in the multi-head attention layer remains unchanged during training, while only the low-rank matrix in the multi-head attention layer is updated, the model training parameters and memory overhead are reduced, making model training more efficient and cost-effective, thereby reducing the difficulty of drone positioning based on image retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A diagram showing an application environment of a method for positioning a drone in one embodiment;

[0028] Figure 2 A flowchart of a method for positioning a drone according to an embodiment is shown;

[0029] Figure 3 Schematic diagram of the architecture of a model to be trained in one embodiment;

[0030] Figure 4 A schematic diagram of a plurality of encoders connected in series in one embodiment;

[0031] Figure 5 A schematic diagram of the principle of obtaining the location information of a drone in one embodiment;

[0032] Figure 6 Schematic diagram of the architecture of the first network layer and the second network layer in one embodiment;

[0033] Figure 7 Schematic diagram of the architecture of a feature extraction block in one embodiment;

[0034] Figure 8 Schematic diagram of the architecture of a model to be trained in another embodiment;

[0035] Figure 9 is a target image retrieved based on the optimized MoCo model in one embodiment;

[0036] Figure 10 is a target image retrieved based on the optimized MoCo model in another embodiment;

[0037] Figure 11 is a target image retrieved based on the MoCo model before optimization in one embodiment;

[0038] Figure 12 This is a structural block diagram of a UAV positioning device in one embodiment;

[0039] Figure 13 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0041] The complexity and diversity of the power inspection working environment place stringent demands on the positioning technology of drones, because power inspections usually cover a wide geographical range, from remote mountainous areas to densely populated urban areas. These environments are often accompanied by complex terrain, changeable weather conditions and potential sources of electromagnetic interference. Especially in specific scenarios such as deep mountains and forests, canyons and gullies, or near high-voltage power lines, GPS signals are extremely susceptible to obstruction or interference, which greatly reduces the positioning capability of drones.

[0042] As a crucial tool for power inspections, drones' operational efficiency and safety are directly impacted by their positioning accuracy. Without accurate positioning in an environment without GPS signals, drones can become lost, unable to reach inspection points, or even misjudging the environment, leading to safety incidents. Therefore, the need for autonomous positioning in drone technology is becoming increasingly important. Developing a method that can maintain high-precision positioning even in the absence of GPS signals is crucial for improving the efficiency and safety of power inspections.

[0043] Currently, image retrieval-based localization techniques have been proposed. These techniques match spatial locations by comparing drone-captured images with geotagged images in image libraries. In recent years, contrastive learning methods have shown great potential in drone image retrieval and localization due to their excellent image feature recognition capabilities. Traditional localization methods, such as visual odometry and deep learning-based feature extraction, have limited adaptability to changes in drone viewpoints and scenes. Contrastive learning, on the other hand, generates image features through unsupervised learning, eliminating the need for large-scale annotated data, offering a new approach for drone image localization. Typical frameworks, such as SimCLR (Simple Framework for Contrastive Learning of Visual Representations), MoCo (Momentum Contrast), and SWaV (Swapping Assignments between Multiple Views of the Same Image), construct positive and negative sample pairs in different ways, enabling more discriminative feature learning and laying a solid foundation for drone image localization. For example, by enhancing image viewpoint invariance through SimCLR, researchers have been able to achieve more accurate localization in drone imagery with varying viewpoints. Frameworks like MoCo adapt to new drone scenarios by building dynamic comparison libraries, enabling rapid feature library updates. Clustering-based contrastive learning methods like SWaV further enhance feature differentiation, helping to find the closest geographically located images in an image library.

[0044] The drone positioning method based on contrastive learning has significant advantages. It generates position-related feature vectors in an unsupervised manner, avoiding reliance on large amounts of labeled data. At the same time, the features generated by contrastive learning can adapt to the multi-view and multi-scale scenes in drone images, allowing images taken at different flight altitudes and angles to effectively match images in the image library, achieving relative position estimation. However, although contrastive learning has made great progress in drone positioning, the model training process is very expensive and difficult to meet general engineering requirements, making drone positioning based on image retrieval difficult to achieve. In addition, the feature extraction expressive power of the current model used for drone positioning is insufficient, resulting in low precision in the process of matching images in the image library, which in turn leads to low accuracy in drone positioning.

[0045] Based on this, it is necessary to propose effective technical means to solve the above problems. The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0046] The drone positioning method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be integrated with server 104 or placed in the cloud or on other network servers. Terminal 102 can be, but is not limited to, a drone, various personal computers, laptops, smartphones, or tablets. Server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0047] In an exemplary embodiment, Figure 2 As shown, a drone positioning method is provided. The drone positioning method can be applied to a terminal (such as a drone), a server, or a positioning system composed of a terminal and a server. The following describes the drone positioning method applied to a terminal as an example, wherein:

[0048] Step 201: Acquire a sample image, where the sample image is an image taken by a drone during a power inspection.

[0049] The sample image may be obtained from a database that stores images taken by multiple drones during power inspections. The sample image is also a remote sensing image.

[0050] Step 202: extract features of the sample image through the first network layer and the second network layer of the model to be trained, and obtain sample fusion features based on the extracted features.

[0051] Among them, such as Figure 3 As shown, the architecture of the model to be trained is an improvement of the MoCo model architecture. The model to be trained includes a first network layer, a second network layer, and multiple encoders connected in series. The first network layer and the second network layer can both be composed of multiple sub-network layers. The first network layer and the second network layer process the sample image differently.

[0052] Optionally, features of the sample image are extracted through the first network layer, and features of the sample image are extracted through the second network layer, and the features extracted by the first network layer are fused with the features extracted by the second network layer to obtain sample fusion features.

[0053] Since the first network layer and the second network layer process the sample image differently, the features extracted by the first network layer are different from the features extracted by the second network layer. Therefore, the sample fusion feature obtained by fusion of the extracted features has rich feature information.

[0054] Step 203: Process the sample fusion features through multiple encoders connected in series of the model to be trained to obtain sample target features; wherein each encoder includes a multi-head attention layer.

[0055] Among them, the multiple encoders connected in series are the optimized ViT (Vision Transformer) structure, such as Figure 4 As shown in the figure, an architecture diagram of multiple encoders in series is provided. Each encoder includes a first normalization layer (Norm), a second normalization layer, a multi-head attention layer (Multi-Head Attention), a multi-layer perceptron (MLP), a first residual connection layer and a second residual connection layer; the residual connection layer is in Figure 4 Indicated by a circle with a plus sign.

[0056] Specifically, if the encoder is the first encoder among multiple encoders connected in series, the input of the first normalization layer of the encoder is the sample fusion feature; if the encoder is the first encoder among multiple encoders not connected in series, the input of the first normalization layer of the encoder is the output of the second residual connection layer of the previous encoder; the input of the multi-head attention layer of the encoder is the output of the first normalization layer, and the input of the first residual connection layer of the encoder is the input of the first normalization layer and the output of the multi-head attention layer; the input of the second normalization layer of the encoder is the output of the first residual connection layer; the input of the multi-layer perceptron of the encoder is the output of the second normalization layer; the input of the second residual connection layer is the output of the multi-layer perceptron and the input of the second normalization layer.

[0057] Optionally, the model to be trained further includes a target residual connection layer, which is connected to the last encoder of the plurality of encoders connected in series. The input of the target residual connection layer is the output of the last encoder and the output of the first sub-network layer of the second network layer. The output of the target residual connection is a sample target feature. In other words, the sample fusion features are processed by the plurality of encoders connected in series of the model to be trained to obtain a first sample target feature. The first sample target feature and the output of the first sub-network layer of the second network layer are subjected to residual processing by the target residual connection layer of the model to be trained to obtain a second sample target feature, and the second sample target feature is used as the sample target feature.

[0058] In step 204, the model to be trained is trained according to the sample target features and the training labels corresponding to the sample images, and a feature extraction model is obtained after the training is completed; wherein, during the training process, the initial weight matrix in the multi-head attention layer remains unchanged, and the low-rank matrix in the multi-head attention layer is updated.

[0059] The training labels can be positive samples corresponding to the sample images or negative samples corresponding to the sample images. The weight matrix W in the multi-head attention layer is W=W0+BA, where W0 is the initial weight matrix and B and A are both low-rank matrices.

[0060] Optionally, a loss value is calculated based on the contrastive loss function between the target features of the samples and the training labels, and the training model is trained based on the loss value and a training stop condition. The training stop condition can be the number of iterations that have been met; the goal of the training model is to maximize the similarity between pairs of positive samples while minimizing the difference between pairs of negative samples.

[0061] The training process of the training model is actually a process of continuously updating the model parameters. In this application, the model parameters are updated by updating the low-rank matrix in the multi-head attention layer. This is equivalent to adding a LoRA (Low-Rank Adaptation) layer to ViT, such as Figure 4 As shown in the figure, adding LoRA (Low-Rank Adaptation) to ViT is a lightweight method that reduces the amount of parameter updates during model training by introducing low-rank matrix decomposition, thereby reducing computing requirements and significantly reducing training costs. It can adapt to new tasks while maintaining performance and can also better adapt to remote sensing images.

[0062] In step 205, a feature extraction model is called to process the image to be processed to obtain target features; a target image is searched from an image library based on the target features, and the location information of the UAV is obtained based on the location information corresponding to the target image.

[0063] The image library stores a plurality of images, each of which has corresponding location information. The location information may be the coordinate information of the drone when the image was taken.

[0064] Optionally, the image to be processed is input into the feature extraction model to obtain the target feature output by the feature extraction model. Similarity matching is performed between the target feature and multiple images in the image library, and the image with the greatest similarity is used as the target image; the position information corresponding to the target image is used as the position information of the UAV. The principle diagram for obtaining the position information of the UAV is as follows: Figure 5 shown.

[0065] The above-mentioned drone positioning method obtains sample images captured by a drone during a power inspection. The first and second network layers of the training model then extract features from the sample images. Sample fusion features are then generated based on the extracted features. These fusion features are then processed by multiple encoders connected in series within the training model to obtain sample target features. Each encoder includes a multi-head attention layer. The training model is then trained based on the sample target features and the training labels corresponding to the sample images. After training, a feature extraction model is obtained. During training, the initial weight matrix in the multi-head attention layer remains unchanged, while the low-rank matrix in the multi-head attention layer is updated. The feature extraction model is then invoked to process the image to obtain target features. Based on the target features, the target image is retrieved from an image library, and the drone's location information is obtained based on the corresponding location information of the target image. Since the initial weight matrix in the multi-head attention layer remains unchanged during training, while only the low-rank matrix in the multi-head attention layer is updated, this reduces model training parameter and memory overhead, making model training more efficient and cost-effective, thereby reducing the difficulty of drone positioning based on image retrieval.

[0066] In an exemplary embodiment, Figure 6 As shown, an architecture diagram of a first network layer and a second network layer is provided, wherein the second network layer includes a first sub-network layer, a second sub-network layer, and a third sub-network layer. Features of a sample image are extracted through the first and second network layers of the model to be trained, and sample fusion features are obtained based on the extracted features. This includes: processing the sample image through the first sub-network layer to obtain a first sample feature; processing the first sample feature through the second sub-network layer to obtain a second sample feature; processing the first sample feature through the third sub-network layer to obtain a third sample feature; and fusing the second and third sample features with the features extracted by the first network layer to obtain a sample fusion feature.

[0067] In an optional embodiment, the first sub-network layer may include a convolutional neural network (CNN) and a downsampling layer; processing the sample image via the first sub-network layer to obtain the first sample feature includes: extracting features from the sample image via the CNN, and then downsampling the features extracted by the CNN via the downsampling layer to obtain the first sample feature. It should be noted that the output of the first sub-network layer of the second network layer mentioned above is the first sample feature.

[0068] For example, if the size (length × width × channel) of the feature extracted by the convolutional neural network is p × p × c, then the size of the first sample feature obtained after downsampling is (p / 2) × (p / 2) × c.

[0069] In an optional embodiment, the second sub-network layer includes a bilinear interpolation processing layer (BilinearInterpolation), a first depth-wise separable convolution layer (Depth-Wise conv) and a first flattening layer (Flatten); the first sample features are processed by the second sub-network layer to obtain second sample features, including: performing bilinear interpolation processing on the first sample features by the bilinear interpolation processing layer to obtain the first sample features after bilinear interpolation processing; performing depth-wise separable convolution processing on the first sample features after bilinear interpolation processing by the first depth-wise separable convolution layer to obtain the first sample features after depth-wise separable convolution processing; and flattening the first sample features after depth-wise separable convolution processing by the first flattening layer to obtain the second sample features.

[0070] For example, by performing bilinear interpolation, depth-wise separable convolution layer processing, and flattening processing on the first sample feature in sequence, a second sample feature of N×the dimension (Embed_dim) of the embedding vector can be obtained. The second sample feature can be used as a position embedding feature.

[0071] In an optional embodiment, the third sub-network layer includes a feature extraction block, a downsampling layer, a second depthwise separable convolution layer and a second flattening layer; the first sample feature is processed by the third sub-network layer to obtain a third sample feature, including: performing feature extraction processing on the first sample feature by the feature extraction block to obtain a first sub-sample feature; downsampling the sub-sample feature by the downsampling layer to obtain the downsampled sub-sample feature; performing depthwise separable convolution processing on the downsampled sub-sample feature by the second depthwise separable convolution layer to obtain a second sub-sample feature; and flattening the second sub-sample feature by the second flattening layer to obtain a third sample feature.

[0072] For example, the size of the first sample feature is (p / 2)×(p / 2)×c. If p / 2=a, the feature extraction block performs feature extraction on the first sample feature, and the size of the first sub-sample feature obtained is a×a×c. After downsampling, the size of the first sub-sample feature obtained is (a / 2)×(a / 2)×c. The first sub-sample feature is then subjected to depthwise separable convolution and flattening in sequence to obtain a third sample feature of size 1× the dimension of the embedding vector (embed_dim). This third sample feature can be used as an additional learnable embedding feature (class embedding).

[0073] It can be understood that the second sample feature and the third sample feature are features extracted by the second network layer.

[0074] In an optional embodiment, extracting features of the sample image through the first network layer includes: segmenting the sample image through the first network layer to obtain multiple image blocks, and performing vector conversion processing on the multiple image blocks to obtain features extracted by the first network layer.

[0075] For example, the first network layer is the image block embedding layer (path embedding), which divides the sample image into N image blocks and stretches each image block to the dimension of the embedding vector (Embed_dim) in the length and width dimensions. Therefore, features of the size of N × the dimension of the embedding vector (Embed_dim) can be obtained.

[0076] After obtaining the features extracted by the first network layer and the features extracted by the second network layer, the features extracted by the first network layer are fused with the features extracted by the second network layer to obtain a sample fusion feature, including: adding the second sample feature to the feature extracted by the first network layer to obtain a first sample fusion feature; splicing the first sample fusion feature with the third sample feature to obtain a second sample fusion feature, and using the second sample fusion feature as the sample fusion feature.

[0077] For example, the size of the second sample feature is N × the dimension of the embedding vector (embed_dim); the size of the third sample feature is 1 × the dimension of the embedding vector (embed_dim); the size of the feature extracted by the first network layer is N × the dimension of the embedding vector (embed_dim), so the size of the obtained sample fusion feature is (N+1) × the dimension of the embedding vector (embed_dim).

[0078] In this embodiment, because the first sub-network (convolutional neural network) focuses more on describing local information, while ViT focuses more on integrating global features, combining the two different feature descriptions of the first sub-network and ViT achieves feature complementarity. Furthermore, the third sample feature is derived from the features extracted by the first sub-network. It describes the overall characteristics of the input image rather than the features corresponding to a specific image block, thus avoiding the tendency to over-learn specific regions.

[0079] In an exemplary embodiment, Figure 7 As shown, an architecture diagram of a feature extraction block is provided.

[0080] The feature extraction block includes a first residual block, a second residual block and a multi-layer perceptron; the feature extraction block performs feature extraction processing on the first sample feature to obtain a first sub-sample feature, including: inputting the first sample feature and the first sample feature after convolution processing into the first residual block to obtain a first residual feature; inputting the first residual feature and the first residual feature after attention processing into the second residual block to obtain the output of the second residual block; inputting the output of the second residual block into the multi-layer perceptron to obtain the output of the multi-layer perceptron, and using the output of the multi-layer perceptron as the first sub-sample feature.

[0081] Optionally, the feature extraction block further includes a first convolutional layer, a batch normalization layer (BN), a second convolutional layer, and an attention layer.

[0082] The first sample feature (size is a×a×c) is input into the first convolutional layer, and the first convolutional layer performs a convolution operation on the first sample feature to obtain the first sample feature after convolution processing; wherein, the convolution operation can use a 3×3 convolution kernel (conv 3×3), as well as grouped convolution and 2D position embedding, which can capture local spatial information and relative position information, and grouped convolution can reduce the number of parameters, making this layer more efficient while maintaining the convolution characteristics.

[0083] The first sample feature and the convolution-processed first sample feature are input into the first residual block. The first residual block performs residual processing on the first sample feature and the convolution-processed first sample feature to obtain the first residual feature. This process is equivalent to adding the output of the convolution operation directly to the features on the main path through a residual block, helping the network better retain information and thus alleviating the gradient vanishing problem.

[0084] The first residual feature is input into the batch normalization layer, which performs batch normalization on the first residual feature to obtain the batch normalized feature. Batch normalization is used to accelerate training convergence and stabilize the training process.

[0085] The batch-normalized features are fed into the second convolutional layer, which performs a convolution operation on the batch-normalized features to obtain the first residual features after convolution. This convolution operation can use a 1×1 convolution kernel (conv 1×1). Here, the 1x1 convolution is actually used to adjust the number of channels of the features for subsequent processing.

[0086] The first residual feature after convolution is input into the attention layer, which performs attention processing on the first residual feature after convolution to obtain the first residual feature after attention processing. The attention layer uses dynamic convolution (DCNv3), which can focus on key features of the input through adaptive weight distribution, enhancing the network's ability to capture complex patterns.

[0087] The first residual features and the first residual features after attention processing are input to the second residual block. The second residual block performs residual processing on the first residual features and the first residual features after attention processing to obtain the output of the second residual block. The output of the second residual block is then input to a multi-layer perceptron (MLP) to obtain the output of the MLP. This process is equivalent to passing the output of the attention layer through a residual block to the MLP. The MLP consists of two layers of 1x1 convolutions, with the number of channels adjusted to the c*MLP ratio. Nonlinear activation functions such as GELU (Gaussian Error Linear Units) are used to further enrich the feature representation. The MLP can expand and process global information, complementing the attention layer. Therefore, the feature extraction block that includes the attention layer and the MLP can better integrate global and local features, making the first sample feature more accurate, enabling faster convergence, and more accurate representation of image features.

[0088] In this embodiment, since the convolution layer performs well in capturing local spatial information, and the attention layer can better focus on global dependencies, the present application designs the above-mentioned feature extraction block, which can balance the advantages of convolution and attention mechanisms. Among them, the residual block and batch normalization can make network training more stable and avoid gradient disappearance. The multi-layer perceptron further enriches the feature representation and helps to enhance the nonlinear expression ability of the model. In this way, it can capture local details while paying attention to the global context, making the first sub-sample feature obtained more accurate.

[0089] To sum up, the architecture diagram of the model to be trained is as follows Figure 8 As shown in the figure, its essence is the optimized model of MoCo, which solves the key problems of traditional technology such as high cost of original contrast learning method and low adaptability and accuracy to remote sensing image targets.

[0090] This application also conducted an experimental comparison between the optimized MoCo model (i.e., the feature extraction model in this application) and the MoCo model before optimization, and found that the accuracy of acquiring the target image based on the MoCo model before optimization was 79.2%, while the accuracy of acquiring the target image based on the optimized MoCo model could reach 84.5%, an improvement of 5.3%.

[0091] This application also compares the actual retrieval effects of the MoCo model before and after optimization. Figure 9 and Figure 10 is the target image retrieved based on the optimized MoCo model, Figure 11 This is the target image retrieved based on the MoCo model before optimization. It can be seen intuitively that the optimized MoCo model has higher accuracy.

[0092] During training, MoCo's pre-trained model weights can be used to train on remote sensing image datasets, significantly reducing training costs from hundreds of millions of datasets to tens of thousands. Furthermore, the hardware required for training has been reduced from 16 V100 GPUs to a standard consumer graphics card. The aforementioned experimental comparisons demonstrate the high accuracy of this application's feature extraction model in drone image retrieval and positioning tasks.

[0093] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0094] Based on the same inventive concept, the present application also provides a drone positioning device for implementing the aforementioned drone positioning method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more drone positioning device embodiments provided below can be found in the above-described limitations of the drone positioning method and will not be further elaborated here.

[0095] In an exemplary embodiment, Figure 12As shown, a drone positioning device is provided. The drone positioning device 1200 includes: an acquisition module 1201, a first determination module 1202, a second determination module 1203, a training module 1204 and a third determination module 1205, wherein:

[0096] The acquisition module 1201 is used to acquire sample images, where the sample images are images taken by the drone during the power inspection process;

[0097] A first determining module 1202 is configured to extract features of the sample image through the first network layer and the second network layer of the to-be-trained model, respectively, and obtain a sample fusion feature based on the extracted features;

[0098] The second determination module 1203 is configured to process the sample fusion features through a plurality of encoders connected in series of the model to be trained to obtain the sample target features; wherein each encoder includes a multi-head attention layer;

[0099] A training module 1204 is configured to train the model to be trained based on the sample target features and the training labels corresponding to the sample images, and obtain a feature extraction model after the training is completed; wherein, during the training process, the initial weight matrix in the multi-head attention layer remains unchanged, and the low-rank matrix in the multi-head attention layer is updated;

[0100] The third determination module 1205 is used to call the feature extraction model to process the image to be processed to obtain target features; find the target image from the image library according to the target features, and obtain the location information of the drone according to the location information corresponding to the target image.

[0101] In one embodiment, the second network layer includes a first sub-network layer, a second sub-network layer and a third sub-network layer; the first determination module 1202 is specifically used to process the sample image through the first sub-network layer to obtain a first sample feature; process the first sample feature through the second sub-network layer to obtain a second sample feature; process the first sample feature through the third sub-network layer to obtain a third sample feature; and fuse the second sample feature and the third sample feature with the feature extracted by the first network layer to obtain a sample fusion feature.

[0102] In one embodiment, the second sub-network layer includes a bilinear interpolation processing layer and a first depthwise separable convolution layer; the first determination module 1202 is specifically used to perform bilinear interpolation processing on the first sample feature through the bilinear interpolation processing layer to obtain the first sample feature processed by bilinear interpolation; perform depthwise separable convolution processing on the first sample feature processed by bilinear interpolation through the first depthwise separable convolution layer to obtain the first sample feature processed by depthwise separable convolution; and flatten the first sample feature processed by depthwise separable convolution through the first flattening layer to obtain the second sample feature.

[0103] In one embodiment, the third sub-network layer includes a feature extraction block, a downsampling layer, a second depthwise separable convolution layer and a second flattening layer; the first determination module 1202 is specifically used to perform feature extraction processing on the first sample feature through the feature extraction block to obtain a first sub-sample feature; downsampling the sub-sample feature through the downsampling layer to obtain a downsampled sub-sample feature; performing depthwise separable convolution processing on the downsampled sub-sample feature through the second depthwise separable convolution layer to obtain a second sub-sample feature; and flattening the second sub-sample feature through the second flattening layer to obtain a third sample feature.

[0104] In one embodiment, the feature extraction block includes a first residual block, a second residual block and a multi-layer perceptron; the first determination module 1202 is specifically used to input the first sample feature and the first sample feature after convolution processing into the first residual block to obtain the first residual feature; input the first residual feature and the first residual feature after attention processing into the second residual block to obtain the output of the second residual block; input the output of the second residual block into the multi-layer perceptron to obtain the output of the multi-layer perceptron, and use the output of the multi-layer perceptron as the first sub-sample feature.

[0105] In one embodiment, the first determination module 1202 is specifically used to segment the sample image through the first network layer to obtain multiple image blocks, and perform vector conversion processing on the multiple image blocks to obtain features extracted by the first network layer; the features extracted by the first network layer and the features extracted by the second network layer are fused to obtain sample fusion features.

[0106] Each module in the aforementioned drone positioning device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0107] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 13As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means. The wireless means can be implemented via Wi-Fi, mobile cellular networks, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a method for positioning a drone. The display unit of the computer device is used to produce a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0108] Those skilled in the art will understand that Figure 13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0109] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of any one of the above method embodiments when executing the computer program.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above method embodiments are implemented.

[0111] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of any one of the above method embodiments.

[0112] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0113] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0114] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for positioning a drone, characterized in that: The method comprises: Acquire sample images, where the sample images are images taken by the drone during the power inspection process; Extracting features of the sample image through the first network layer and the second network layer of the model to be trained, respectively, and obtaining sample fusion features according to the extracted features; Processing the sample fusion features through multiple encoders connected in series of the model to be trained to obtain sample target features; wherein each of the encoders includes a multi-head attention layer; The model to be trained is trained according to the sample target features and the training labels corresponding to the sample images, and a feature extraction model is obtained after the training is completed; wherein, during the training process, the initial weight matrix in the multi-head attention layer remains unchanged, and the low-rank matrix in the multi-head attention layer is updated; The feature extraction model is called to process the image to be processed to obtain target features; a target image is searched from an image library according to the target features, and the position information of the UAV is obtained according to the position information corresponding to the target image.

2. The method according to claim 1, characterized in that The second network layer includes a first sub-network layer, a second sub-network layer, and a third sub-network layer; extracting features of the sample image through the first network layer and the second network layer of the to-be-trained model, respectively, and obtaining sample fusion features based on the extracted features, including: Processing the sample image through the first sub-network layer to obtain a first sample feature; Processing the first sample feature through the second sub-network layer to obtain a second sample feature; Processing the first sample feature through the third sub-network layer to obtain a third sample feature; The second sample feature and the third sample feature are fused with the feature extracted by the first network layer to obtain the sample fusion feature.

3. The method according to claim 2, characterized in that The second sub-network layer includes a bilinear interpolation processing layer, a first depth-separable convolution layer, and a first flattening layer; the processing of the first sample feature by the second sub-network layer to obtain the second sample feature includes: Performing bilinear interpolation processing on the first sample feature through the bilinear interpolation processing layer to obtain the first sample feature after bilinear interpolation processing; Performing depthwise separable convolution processing on the first sample features after the bilinear interpolation processing through the first depthwise separable convolution layer to obtain the first sample features after the depthwise separable convolution processing; The first sample features after the depthwise separable convolution processing are flattened by the first flattening layer to obtain the second sample features.

4. The method according to claim 2, characterized in that The third sub-network layer includes a feature extraction block, a downsampling layer, a second depth-wise separable convolutional layer, and a second flattening layer; the processing of the first sample feature by the third sub-network layer to obtain a third sample feature includes: Performing feature extraction processing on the first sample feature by the feature extraction block to obtain a first sub-sample feature; Performing downsampling processing on the subsample features through the downsampling layer to obtain the downsampled subsample features; Performing depth-wise separable convolution processing on the downsampled subsample features through the second depth-wise separable convolution layer to obtain second subsample features; The second sub-sample feature is flattened by the second flattening layer to obtain the third sample feature.

5. The method according to claim 4, characterized in that The feature extraction block includes a first residual block, a second residual block, and a multi-layer perceptron; the feature extraction block performs feature extraction processing on the first sample feature to obtain a first sub-sample feature, including: Inputting the first sample feature and the first sample feature after convolution processing into the first residual block to obtain a first residual feature; Inputting the first residual feature and the first residual feature after attention processing into the second residual block to obtain an output of the second residual block; The output of the second residual block is input into the multi-layer perceptron to obtain the output of the multi-layer perceptron, and the output of the multi-layer perceptron is used as the first sub-sample feature.

6. The method according to claim 2, characterized in that The extracting features of the sample image through the first network layer and the second network layer of the to-be-trained model, and obtaining sample fusion features according to the extracted features, comprises: Segmenting the sample image through the first network layer to obtain a plurality of image blocks, and performing vector conversion processing on the plurality of image blocks to obtain features extracted by the first network layer; The features extracted by the first network layer and the features extracted by the second network layer are fused to obtain the sample fusion features.

7. A UAV positioning device, characterized in that: The device comprises: An acquisition module is used to acquire sample images, where the sample images are images taken by the drone during the power inspection process; A first determining module is configured to extract features of the sample image through the first network layer and the second network layer of the to-be-trained model, respectively, and obtain a sample fusion feature based on the extracted features; A second determination module is configured to process the sample fusion features through a plurality of encoders connected in series of a model to be trained to obtain a sample target feature; wherein each of the encoders includes a multi-head attention layer; A training module is used to train the model to be trained based on the sample target features and the training labels corresponding to the sample images, and obtain a feature extraction model after the training is completed; wherein, during the training process, the initial weight matrix in the multi-head attention layer remains unchanged, and the low-rank matrix in the multi-head attention layer is updated; The third determination module is used to call the feature extraction model to process the image to be processed to obtain target features; search the target image from the image library according to the target features, and obtain the location information of the drone according to the location information corresponding to the target image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images