An image semantic information mining method based on feature dynamic offset

Abstract

The application discloses a kind of image semantic information mining methods based on feature dynamic offset.First, the image dataset required for network training is obtained;Then construct the cross-view geographic positioning network model based on the diagonal feature offset fusion strategy of weight self-adaption;Network model contains unmanned aerial vehicle view branch and satellite map view branch, two branches use shared weight learning mode;Each branch is composed of three parts: feature extraction part, feature division part and classification supervision part;The neural network model trained receives the unmanned aerial vehicle view image that needs to be positioned, and matches the corresponding satellite image for it.The application adopts the diagonal feature offset fusion strategy of weight self-adaption, can adaptively select the ring partition strategy that the coordinate is more reasonable according to the target area in image, so as to improve the coverage and expression ability of feature area, avoid the problem of insufficient feature extraction caused by target area not in visual center position.

Description

Technical Field

[0001] This invention belongs to the field of UAV geolocation based on deep learning technology, mainly targeting UAVs and satellite images, and specifically involves a semantic information mining method based on dynamic feature offset. Background Technology

[0002] Cross-view geolocation is a challenging computer vision problem because visual content can vary significantly in appearance from different perspectives, due to factors such as lighting, occlusion, and geometric deformation. To address this, several region-based methods have been proposed. These methods divide an image into several regions (e.g., using a circular partitioning strategy) and then match regions based on their appearance or structural similarity. While these methods offer low computational complexity and good global representation capabilities, they also have drawbacks, such as the region partitioning method significantly impacting the matching results, potential overlap or omissions between regions, and the inability of regions to fully represent local details in the image. Therefore, to address the problems in existing technologies, this invention proposes a semantic information mining method based on dynamic feature offset fusion. This method dynamically offsets the center point of the partitioned regions to achieve circular partitioning at various locations, aiming to fully utilize global and local information in the image to improve the accuracy of cross-view geolocation. Summary of the Invention

[0003] To address the problem of insufficient semantic information mining caused by existing ring partitioning strategies that assume the target is located at the image center, this invention designs an image semantic information mining method based on dynamic feature offset. This method dynamically offsets the center point of the ring partitioned region along the diagonal and uses adaptive weights to efficiently fuse features before and after the offset, thereby effectively mining spatial context information and alleviating the dependence of the ring partitioning strategy on the visual center position of the target region.

[0004] Implementation Steps: This invention proposes an image semantic information mining method based on dynamic feature offset, applied to cross-view geolocation. Its basic steps are as follows:

[0005] Step 1: Dataset Acquisition;

[0006] Obtain the image dataset required for network training in step 2.

[0007] Step 2: Construct a cross-view geolocation network model based on a weight-adaptive diagonal feature offset fusion strategy;

[0008] The network model consists of a UAV-view (D branch) and a satellite image-view (S branch), with both branches learning using a shared weight approach. Each branch comprises three parts: feature extraction, feature partitioning, and classification supervision. The input to the network model is a given image X. i (i∈{D,S}), where D and S represent the UAV view or satellite image view, respectively; given image X i Feature extraction is performed using the ResNet50 backbone network in the feature extraction section to obtain feature maps f. i ; the feature map f i The data is fed into the feature segmentation section to perform feature map segmentation. Then, each segment is fused to obtain the fused features. Fusion features The images are fed into the classification supervision section for matching and retrieval of images from different perspectives, thereby achieving cross-perspective geolocation.

[0009] Step 3: The trained neural network model receives the drone's view image that needs to be located and matches it with the corresponding satellite image.

[0010] Furthermore, the image dataset mentioned in step 1 is obtained as follows:

[0011] Download the publicly available University-1652 dataset. The original University-1652 dataset is a drone-based geolocation dataset that contains images of 1,652 university buildings from three platforms: drone, satellite, and ground-based perspectives.

[0012] Download the publicly available University-160k dataset. University-160k expands upon the current University-1652 dataset by adding 167,486 satellite view library interference items. These interference satellite view images are 1024×1024 in size and were obtained by cropping orthophotos of real cities and surrounding areas.

[0013] Furthermore, the specific structure of the feature extraction part is as follows: ResNet50 is used as the backbone network, and pre-trained weights of imgnet-1k are used to extract visual features. Specifically, given an input image X... i Its size is 3×512×512. After feature extraction, a feature map f with a shape of 1024×32×32 is obtained in each branch. i The corresponding operation can be represented as:

[0014] f i =F Resnet (X i ), i∈{D,S}

[0015] Where F Resnet This represents the feature extraction operation of the backbone network. The two branches share model weights; that is, the feature extraction processes of the UAV branch and the satellite branch use the same model weights, allowing images from both branches to be processed through the same layers and weights.

[0016] Furthermore, the specific structure of the feature segmentation part: a circular segmentation strategy is used to segment the feature map f extracted from the feature extraction part. i Divide the feature map f i Divide the space from the center outwards into N non-overlapping ring-shaped blocks. The single-sided thickness of each partition can be expressed as The corresponding operation is represented as follows:

[0017]

[0018] Among them, F SP This indicates a circular partitioning operation on the feature, where N represents the number of partitions obtained by the circular partitioning strategy. Given f... i The shape is 1024×32×32, and the shape of each partition block obtained by the above ring partitioning strategy can be represented as:

[0019]

[0020] Feature map f i The N partitioning blocks of different sizes obtained after feature segmentation To alleviate the dependence of the ring segmentation strategy on the target being located in the center of the image, a diagonal feature offset fusion strategy is introduced in the feature segmentation part. The diagonal feature offset fusion strategy consists of two parts: dynamic feature offset segmentation and adaptive feature fusion.

[0021] Furthermore, a dynamic feature offset partitioning strategy is proposed: to prevent the center point of the ring partitioning strategy from coinciding with the center point of the feature map, a dynamic feature offset partitioning strategy is implemented. This strategy modifies the coordinates of the center point of the ring partitioning strategy to offset the partitioned blocks. To ensure the model is applicable to scenarios where the target is not centered, this invention achieves K feature offsets by modifying the offset values. The direction and extent of each offset depend on the offset value. Specifically, offsets shift_w and shift_h along the width and height dimensions of the center point are pre-set, where shift_w represents the offset of the partition center point along the width dimension, and the element w in shift_w is defined. k >0 indicates a rightward offset along the width dimension; shift_h represents the offset of the partition center point along the height dimension, defined by the element h in shift_h. k >0 indicates a downward offset along the higher dimension. In the element (w) k hk The center point offset of the ring partitioning strategy is achieved through the combined action of sift_w and shift_h. To further ensure that the center point of the ring partitioning strategy is offset along the image diagonal, the relationship between sift_w and shift_h is constrained as follows:

[0022]

[0023] Where K represents the number of feature shifts, [w1,...,w k / 2 ,...,w k ] = [h1,...,h k / 2 ,...h k ] = [-α,...,0,...,α]. After feature offset processing, the block obtained by the ring partitioning strategy can be represented as:

[0024]

[0025] Each partition is compressed into an output feature of shape 1024×1×1 after passing through an average pooling layer.

[0026]

[0027] Where F avg This indicates the operation of compressing input features using an average pooling layer.

[0028] Furthermore, feature adaptive fusion: In order to efficiently fuse the partition blocks generated by the dynamic feature offset partitioning, an offset weight estimation module is introduced to adaptively generate the weight ratio of the ring partitioning strategy with different offsets.

[0029] The offset weight estimation module consists of three convolutional layers, three fully connected layers, and one softmax layer. Specifically, the first convolutional layer has 2048 input channels and 1024 output channels; the second convolutional layer has 1024 input channels and 512 output channels; and the third convolutional layer has 512 input channels and 1 output channel. All three convolutional layers use a 1x1 kernel. The first fully connected layer has an input size of 32. 2 Output size 512. Second fully connected layer: input size 512, output size 512. Third fully connected layer: input size 512, output size K. The offset weight estimation module uses the feature map f extracted from the backbone network. i The input is passed through a convolutional layer, a fully connected layer, and a softmax layer in sequence, and the output is the offset weight feature. The offset weight feature Ψ is used to fuse K features for dynamic offset partitioning to generate partition blocks. The operation of this module can be represented as:

[0030]

[0031]

[0032] in: F WE This indicates that the offset weight estimation module is used for adaptive weight estimation.

[0033] The partitioned blocks obtained according to different offset modes can effectively extract key spatial information and alleviate the performance degradation problem caused by the target region not being located at the visual center. Therefore, the entire process can be summarized as follows:

[0034]

[0035] Further, the specific structure of the classification supervision part is as follows: To supervise the classification of each feature partition, each partition is fed into the classifier module CF. This module predicts the geographic label for each partition and compares it with the true label. The classifier module includes a fully connected layer, a batch normalization layer, a Dropout layer, and a classification layer. The fully connected layer has an input dimension of 1024 and an output dimension of 512; the batch normalization layer has an input dimension of 512; and the classification layer has an input dimension of 512 and an output dimension of C (based on the University1652 dataset, the total number of geographic labels C = 701). The classifier module ensures that the shape of the circular partition is the same as the label Y. Then, the cross-entropy loss function is used for optimization. The entire process can be represented as:

[0036]

[0037]

[0038] Where C represents the number of geographic label categories, and CF represents the classifier module. The loss for each partition is calculated separately, and then optimized uniformly.

[0039] Furthermore, the specific method for step 3 is as follows:

[0040] First, load the weights of the cross-view geolocation network model trained in step 2 using a weight-adaptive diagonal feature offset fusion strategy, and update the parameters in the model. Second, use the UAV view image X that needs to be localized. D The input data is fed into the network model, where it passes through the feature extraction and feature segmentation parts. The classification layer is then removed in the classifier module of the feature segmentation part to obtain the UAV-view image X. DThe corresponding classification features are then derived. Simultaneously, all satellite view images from the University1652 dataset are sequentially input into the network model. The input data passes through the feature extraction and feature partitioning parts, and the classification layer is removed in the classifier module of the feature partitioning part to obtain the classification features corresponding to all satellite view images. Finally, the UAV view image X... D The classification features of the image are compared with the classification features of all satellite images. Euclidean distance is used to measure the similarity between the two viewpoint images, thereby retrieving satellite images that match the UAV viewpoint images.

[0041] The beneficial effects of this invention are as follows:

[0042] This invention employs a weighted adaptive diagonal feature offset fusion strategy, which adaptively selects a more reasonable annular partitioning strategy based on the target region in the image. This improves the coverage and expressive power of the feature region, avoiding insufficient feature extraction caused by the target region not being located at the visual center. It effectively mines spatial context information, enhances the correlation and consistency between feature regions, and improves the accuracy and robustness of matching. Attached Figure Description

[0043] Figure 1 : A schematic diagram of the method flow of the present invention;

[0044] Figure 2 Flowchart of cross-view geolocation network based on feature dynamic offset;

[0045] Figure 3 : Schematic diagram of dynamic feature offset;

[0046] Figure 4 : Schematic diagram of cross-view geolocation network structure based on feature dynamic offset. Detailed Implementation

[0047] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] Implementation steps: such as Figure 1-4 As shown, an image semantic information mining method based on feature dynamic offset is applied to cross-view geolocation. Its basic steps are as follows:

[0049] Step 1: Dataset Acquisition;

[0050] Obtain the image dataset required for network training in step 2.

[0051] Download the publicly available University-1652 dataset. The original University-1652 dataset is a drone-based geolocation dataset containing images of 1652 university buildings from three platforms: drone, satellite, and ground-based views. This dataset addresses the major challenge of cross-view localization, which is learning stable features for large viewpoint changes. Existing benchmark datasets can be helpful, but they have limited viewpoints. Therefore, we propose using drones as a third-party viewpoint. Because drone-view images serve as an intermediary, and drones can reduce tree occlusion, they are easier to match with satellite images. This dataset can be used for two tasks: drone-view target localization and drone navigation. The training set includes 701 buildings from 33 universities, and the test set includes the remaining 951 buildings from the other 39 universities. There is no overlap between the training and test sets. Drone and satellite views from this dataset are used for drone-view target localization. The query set contains 37,855 UAV view images, and the database contains 701 true matching satellite view images and 250 satellite view interference items. For each UAV view image, there is only one true matching satellite view image. The goal of the UAV geolocation task is to find the corresponding satellite view image based on the UAV view image, thereby determining the UAV's geographical location.

[0052] Download the publicly available University-160k dataset. University-160k expands upon the current University-1652 dataset by adding 167,486 satellite view distractors. These distractor satellite view images are 1024×1024 pixels in size and were obtained by cropping orthophotos of real cities and surrounding areas. The larger image size ensures higher image sharpness, while the wider field of view allows the images to include more diverse scenes, such as buildings and city roads. Compared to the previous dataset, it presents a greater challenge for feature extraction. Therefore, specifically, in the reference set of satellite images, there are 168,437 satellite images, of which only 701 are true matches; the remaining 167,736 images are satellite view distractors. In the query set, similar to University-1652, there are 37,855 drone view images.

[0053] Step 2: Construct a cross-view geolocation network model based on a weight-adaptive diagonal feature offset fusion strategy;

[0054] like Figure 2As shown, the network model comprises a UAV viewpoint (D branch) and a satellite image viewpoint (S branch), with both branches employing a shared weight learning approach. Each branch consists of three parts: a feature extraction part, a feature partitioning part, and a classification supervision part. The input to the network model is a given image X. i (i∈{D,S}), where D and S represent the UAV view or satellite image view, respectively; given image X i Feature extraction is performed using a ResNet50 backbone network to obtain feature maps f. i Next, feature map f i The data is fed into the feature segmentation section to partition the feature map. Then, each segment is fused to obtain the fused feature. Fusion features The images are fed into the classification supervision module to match and retrieve images from different perspectives, thereby achieving cross-perspective geolocation.

[0055] The specific structure of the feature extraction part: ResNet50 is used as the backbone network, and pre-trained weights from imgnet-1k are used to extract visual features. Specifically, given an input image X... i Its size is 3×512×512. After feature extraction, a feature map f with a shape of 1024×32×32 is obtained in each branch. i The corresponding operation can be represented as:

[0056] f i =F Resnet (X i ), i∈{D,S}

[0057] Where F Resnet This represents the feature extraction operation of the backbone network. The two branches share model weights; that is, the feature extraction processes of the UAV branch and the satellite branch use the same model weights, allowing images from both branches to be processed through the same layers and weights.

[0058] The specific structure of the feature partitioning part: The feature map f extracted by the feature extraction part is divided into two parts using a circular partitioning strategy. i Divide the feature map f i Divide the space from the center outwards into N=4 non-overlapping ring-shaped blocks. The single-sided thickness of each partition can be expressed as The corresponding operation is represented as follows:

[0059]

[0060] Among them, F SP This indicates a circular partitioning operation on the feature, where N represents the number of partitions obtained by the circular partitioning strategy. Given f... iThe shape is 1024×32×32, and the shape of each partition block obtained by the above ring partitioning strategy can be represented as:

[0061]

[0062] Feature map f i After feature segmentation, N=4 partition blocks of different sizes are obtained. To alleviate the dependence of the ring segmentation strategy on the target being located in the center region of the image, this invention introduces a diagonal feature offset fusion strategy in the feature segmentation part. The diagonal feature offset fusion strategy consists of two parts: dynamic feature offset segmentation and adaptive feature fusion.

[0063] Dynamic Feature Offset Partitioning: To prevent the center point of the ring partitioning strategy from coinciding with the center point of the feature map, this invention proposes a dynamic feature offset partitioning strategy. This strategy modifies the coordinates of the center point of the ring partitioning strategy to offset the partitioned blocks. To ensure the algorithm model is applicable to most scenarios where the target is not centered, this invention implements K=3 feature offsets by modifying the offset values. The direction and extent of each offset depend on the offset value. Specifically, the offsets shift_w and sfhift_h along the width and height dimensions of the center point are pre-set, where shift_w represents the offset of the partition center point along the width dimension. The element w in shift_w is defined. k >0 indicates an offset to the right along the width dimension; sfhift_h represents the offset of the partition center point along the height dimension, and the element h in sfhift_h is defined. k >0 indicates a downward offset along the higher dimension. In the element (w) k h k Under the combined effect of these factors, the center point shift of the ring partitioning strategy is achieved. To further ensure that the center point of the ring partitioning strategy shifts along the image diagonal, this invention constrains the relationship between shift_w and shift_h as follows:

[0064]

[0065] To balance computational cost and algorithm performance, K is set to 3 in this invention; and the offset is set to [w1,w2,w3] = [h1,h2,h3] = [-2,0,2], representing three offset modes: top-left offset, no offset, and bottom-right offset, respectively. Figure 3 As shown. After feature offset processing, the partitioned blocks obtained by the ring partitioning strategy can be represented as:

[0066]

[0067] Each partition is compressed into an output feature of shape 1024×1×1 after passing through an average pooling layer.

[0068]

[0069] Where F avg This indicates the operation of compressing input features using an average pooling layer.

[0070] Feature Adaptive Fusion: In order to efficiently fuse the partitioned blocks generated by the dynamic feature offset partitioning, this invention introduces an offset weight estimation module to adaptively generate the weight ratio of the ring partitioning strategy with different offsets.

[0071] The offset weight estimation module consists of three convolutional layers, three fully connected layers, and one softmax layer. Specifically, the first convolutional layer has 2048 input channels and 1024 output channels; the second convolutional layer has 1024 input channels and 512 output channels; and the third convolutional layer has 512 input channels and 1 output channel. All three convolutional layers use a 1x1 kernel. The first fully connected layer has an input size of 32. 2 Output size 512. Second fully connected layer: input size 512, output size 512. Third fully connected layer: input size 512, output size K=3. The offset weight estimation module uses the feature map f extracted from the backbone network. i The input is passed through a convolutional layer, a fully connected layer, and a softmax layer in sequence, and the output is the offset weight feature. The offset weight feature Ψ is used to fuse K=3 features to dynamically offset and divide the generated blocks. The operation of this module can be represented as:

[0072]

[0073]

[0074] in: F WE This indicates that the offset weight estimation module is used for adaptive weight estimation.

[0075] The partitioned blocks obtained according to different offset modes can effectively extract key spatial information and alleviate the performance degradation problem caused by the target region not being located at the visual center. Therefore, the entire process can be summarized as follows:

[0076]

[0077] The specific structure of the classification supervision part: To supervise the classification of each feature partition, each partition is fed into the classifier module CF. This module predicts the geographic label for each partition and compares it with the true label. The classifier module includes a fully connected layer, a batch normalization layer, a Dropout layer, and a classification layer. The fully connected layer has an input dimension of 1024 and an output dimension of 512; the batch normalization layer has an input dimension of 512; and the classification layer has an input dimension of 512 and an output dimension of C (based on the University1652 dataset, the total number of geographic labels C = 701). The classifier module ensures that the shape of the circular partition is the same as the label Y. Then, the cross-entropy loss function is used for optimization. The entire process can be represented as:

[0078]

[0079]

[0080] Where C represents the number of geographic label categories, and CF represents the classifier module. The loss for each partition is calculated separately, and then optimized uniformly.

[0081] Step 3: The trained neural network model receives the drone's view image that needs to be located and matches it with the corresponding satellite image.

[0082] First, load the weights of the cross-view geolocation network model trained in step 2 using a weight-adaptive diagonal feature offset fusion strategy, and update the parameters in the model. Second, use the UAV view image X that needs to be localized. D The input data is fed into the network model, where it passes through the feature extraction and feature segmentation parts. The classification layer is then removed in the classifier module of the feature segmentation part to obtain the UAV-view image X. D The corresponding classification features are then derived. Simultaneously, all satellite view images from the University1652 dataset are sequentially input into the network model. The input data passes through the feature extraction and feature partitioning parts, and the classification layer is removed in the classifier module of the feature partitioning part to obtain the classification features corresponding to all satellite view images. Finally, the UAV view image X... D The classification features of the image are compared with the classification features of all satellite images. Euclidean distance is used to measure the similarity between the two viewpoint images, thereby retrieving satellite images that match the UAV viewpoint images. The results obtained from our experiments are shown in Table 1. After adding the feature offset fusion strategy, the model's performance was significantly improved.

[0083] Table 1 Results of the semantic information mining method based on feature dynamic offset

[0084]

[0085] The above description, in conjunction with specific / preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. Those skilled in the art can make various substitutions or modifications to these described embodiments without departing from the inventive concept, and all such substitutions or modifications should be considered within the scope of protection of the present invention.

[0086] The parts of this invention not described in detail are well-known to those skilled in the art.

Claims

1. A method for mining image semantic information based on dynamic feature offset, characterized in that, The steps are as follows: Step 1: Dataset Acquisition; Obtain the image dataset required for network training in step 2; Step 2: Construct a cross-view geolocation network model based on a weight-adaptive diagonal feature offset fusion strategy; The network model includes the drone's perspective. Branch and satellite view The network employs a two-branch learning approach that shares weights; each branch consists of three parts: a feature extraction part, a feature partitioning part, and a classification supervision part; the input to the network model is a given image. Where D and S represent the drone view or satellite image view, respectively; given image Feature extraction is performed using the ResNet50 backbone network in the feature extraction section to obtain feature maps. ; feature map The data is fed into the feature segmentation section to perform feature map segmentation. Then, each segment is fused to obtain the fused features. ; Fusion features The images are fed into the classification supervision section to match and retrieve images from different perspectives, thereby achieving cross-perspective geolocation. A diagonal feature offset fusion strategy is introduced in the feature segmentation part; the diagonal feature offset fusion strategy consists of two parts: dynamic feature offset segmentation and adaptive feature fusion; The dynamic feature offset partitioning is as follows: To prevent the center point of the ring partitioning strategy from coinciding with the center point of the feature map, a dynamic feature offset partitioning strategy is proposed. This strategy involves modifying the coordinates of the center point of the ring partitioning strategy to offset the partitioned blocks. To ensure the model is applicable to scenarios where the target is not centered, K feature offsets are performed by modifying the offset values. The direction and extent of each offset depend on the offset value. Specifically, the offset of the center point along the width and height dimensions is pre-set. ,in This represents the offset of the partition center point along the wide dimension, defined as follows: elements in This indicates a rightward offset along the width dimension; This represents the offset of the partition center point along the higher dimension, defined as follows: elements in Indicates offset downwards along the higher dimension; in the element ( , Under the combined effect of these factors, the center point shift of the ring partitioning strategy was achieved; to further ensure that the center point of the ring partitioning strategy shifts along the image diagonal, the following steps were taken: The relationships are constrained as follows: in, Indicates the number of feature offsets. After feature offset processing, the partitioned blocks obtained by the ring partitioning strategy can be represented as: Each partition is compressed into a shape after passing through an average pooling layer. The output characteristics; in This indicates the operation of compressing input features using an average pooling layer; Step 3: The trained neural network model receives the drone's view image that needs to be located and matches it with the corresponding satellite image.

2. The image semantic information mining method based on feature dynamic offset according to claim 1, characterized in that, The image dataset mentioned in step 1 is obtained as follows: Download the publicly available University-1652 dataset. The original University-1652 dataset is a drone-based geolocation dataset that contains images of 1,652 university buildings from three platforms: drone, satellite, and ground-based perspectives. Download the publicly available University-160k dataset. University-160k expands upon the current University-1652 dataset by adding 167,486 satellite view library interference items. These interference satellite view images are 1024×1024 in size and were obtained by cropping orthophotos of real cities and surrounding areas.

3. The image semantic information mining method based on feature dynamic offset according to claim 1, characterized in that, The specific structure of the feature extraction part is as follows: ResNet50 is used as the backbone network, and pre-trained weights of imgnet-1k are used to extract visual features; specifically, given an input image... Its size is 3×512×512. After feature extraction, a feature map with a shape of 1024×32×32 is obtained in each branch. The corresponding operation can be represented as: in This represents the operation of extracting features from the backbone network; the two branches share model weights, that is, the feature extraction process of the UAV branch and the satellite branch uses the same model weights, allowing the images of the two branches to be processed through the same layers and weights.

4. The image semantic information mining method based on feature dynamic offset according to claim 1, characterized in that, The specific structure of the feature partitioning part: The feature map extracted by the feature extraction part is obtained by using a circular partitioning strategy. Divide the feature map Divided from the center outwards into Non-overlapping ring-shaped partitions The single-sided thickness of each partition block can be expressed as The corresponding operation is represented as: in, This indicates a circular partitioning operation on the features. This represents the number of partitions obtained by the ring partitioning strategy; it is known that... The shape is 1024×32×32, and the shape of each partition block obtained by the above ring partitioning strategy can be represented as: Feature map The N partitioning blocks of different sizes obtained after feature segmentation .

5. The image semantic information mining method based on feature dynamic offset according to claim 4, characterized in that, Feature Adaptive Fusion: In order to efficiently fuse the partition blocks generated by the dynamic feature offset partitioning, an offset weight estimation module is introduced to adaptively generate the weight ratio of the ring partitioning strategy with different offsets. The offset weight estimation module consists of three convolutional layers, three fully connected layers, and one... The layer structure is as follows: Specifically, the first convolutional layer has 2048 input channels and 1024 output channels; the second convolutional layer has 1024 input channels and 512 output channels; the third convolutional layer has 512 input channels and 1 output channel; all three convolutional layers use a 1x1 convolutional kernel; the first fully connected layer has an input size of... Output size 512; Second fully connected layer: input size 512, output size 512; Third fully connected layer: input size 512, output size K; Offset weight estimation module uses feature maps extracted from the backbone network. As input, it passes through a convolutional layer, a fully connected layer, and... Post-layer output offset weight features Offset weight features This module is used to fuse partition blocks generated by dynamic offset partitioning based on K features; its operation can be represented as: in: ; This indicates that the offset weight estimation module is used for adaptive weight estimation. The partitioned blocks obtained according to different offset modes can effectively extract key information in the space, alleviating the performance degradation problem caused by the target region not being located in the visual center; therefore, the entire process can be summarized as follows: 。 6. The image semantic information mining method based on feature dynamic offset according to claim 5, characterized in that, The specific structure of the classification supervision part: In order to perform classification supervision for each feature partition, each partition block is fed into the classifier module. This module predicts the geographic label for each partition and compares it with the true label. The classifier module includes a fully connected layer, a batch normalization layer, a Dropout layer, and a classification layer. The fully connected layer has an input dimension of 1024 and an output dimension of 512; the batch normalization layer has an input dimension of 512; and the classification layer has an input dimension of 512 and an output dimension of C. The classifier module ensures that the shape of the circular partitions matches the label Y. Then, the cross-entropy loss function is used for optimization. The entire process can be represented as: in Indicates the number of geographic tag categories. This represents the classifier module; the loss for each partition is calculated separately and then optimized uniformly.

7. A method for mining image semantic information based on dynamic feature offset according to any one of claims 1-6, characterized in that, The specific method for step 3 is as follows: First, load the weights of the cross-view geolocation network model trained in step 2 using a weight-adaptive diagonal feature offset fusion strategy, and update the parameters in the model; second, load the drone view image that needs to be located. The input data is fed into the network model, where it passes through the feature extraction and feature segmentation parts. The classification layer is then removed in the classifier module of the feature segmentation part to obtain the UAV-view image. The corresponding classification features are then derived. Simultaneously, all satellite view images from the University1652 dataset are sequentially input into the network model. The input data passes through the feature extraction and feature partitioning parts, and the classification layer is removed in the classifier module of the feature partitioning part to obtain the classification features corresponding to all satellite view images. Finally, the drone view images are... The classification features are compared with the classification features of all satellite images, and the similarity between the two viewpoint images is measured by Euclidean distance, so as to retrieve satellite images that match the UAV viewpoint images.

Images