Indoor positioning method based on high and low frequency sub-band image multi-scale feature extraction

Abstract

The application discloses an indoor positioning method based on high and low frequency sub-band image multi-scale feature extraction, and relates to the technical field of computer vision and indoor positioning. In the application, indoor scene images and position information of each position in a to-be-measured indoor scene are collected, the collected images are preprocessed and multi-scale decomposed to obtain high and low frequency sub-band images; a feature extraction fusion network sub-model is continuously trained to obtain a feature extraction fusion network model; reference feature vectors are obtained based on the high and low frequency sub-band images and the network model; to-be-measured images are collected in real time in the to-be-measured indoor scene and preprocessed and multi-scale decomposed to obtain high and low frequency sub-band images, and prediction feature vectors are obtained based on the corresponding high and low frequency sub-band images of the to-be-measured images and the network model; the prediction feature vectors and the reference feature vectors are subjected to similarity matching to obtain image positioning information of the to-be-measured images. The method disclosed by the application can obtain higher positioning accuracy.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and indoor positioning technology, and to an indoor positioning method based on multi-scale feature extraction of high and low frequency sub-band images. Background Technology

[0002] Monocular vision indoor localization technology has received widespread attention in recent years as an important support for realizing indoor space services by mobile robots. Existing monocular vision indoor localization methods often suffer from difficulties in multi-scale extraction of visual information from images when applied to complex indoor environments, leading to poor localization accuracy. Therefore, this application proposes an indoor localization method based on multi-scale feature extraction from high- and low-frequency sub-band images. Summary of the Invention

[0003] To address the aforementioned problems, this invention provides an indoor positioning method based on multi-scale feature extraction from high- and low-frequency sub-band images.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: An indoor positioning method based on multi-scale feature extraction from high- and low-frequency sub-band images includes the following steps: Indoor scene images and their location information at various locations within the indoor scene to be tested are acquired. These images are then preprocessed and decomposed at multiple scales to obtain high-frequency and low-frequency sub-band images. The feature extraction and fusion network sub-model is then trained to obtain a new feature extraction and fusion network model. Reference feature vectors are then obtained based on the corresponding high-frequency and low-frequency sub-band images and the feature extraction and fusion network model at each location. Next, images to be tested are acquired in real-time within the indoor scene. These acquired images are then preprocessed and decomposed at multiple scales to obtain corresponding high-frequency and low-frequency sub-band images. Predicted feature vectors for the tested images are obtained based on these high-frequency and low-frequency sub-band images and the feature extraction and fusion network model. Finally, the predicted feature vectors of the tested images are compared with the reference feature vectors to obtain the image localization information of the tested images. The feature extraction fusion network sub-model is trained based on the feature extraction fusion network, which includes a dynamic convolutional unit and a Swing Transformer module. The outputs of the dynamic convolutional unit and the Swing Transformer module are connected to the feature pyramid module, and the feature pyramid module is connected to the pooling layer. The pooling layer outputs the predicted feature vector. The dynamic convolutional unit is used to perform multi-scale feature extraction and detail enhancement on the high-frequency sub-band image, resulting in a feature map with multi-scale detail enhancement. The Swin Transformer module is used to extract features from the low-frequency sub-band image at multiple scales, resulting in a low-frequency feature map containing multi-scale global semantic information. The feature pyramid module is used to fuse the feature map with multi-scale detail enhancement output from the dynamic convolutional unit and the low-frequency feature map containing multi-scale global semantic information output from the Swin Transformer module, resulting in a fused feature map. The fused feature map contains edge information and texture detail information from the high-frequency sub-band image, as well as global structural information and main contour features from the low-frequency sub-band image. The pooling layer performs pooling operations on the fused feature map to obtain the predicted feature vector.

[0005] Preferably, the indoor scene image is preprocessed, including the following steps: The indoor scene image is converted into an RGB three-channel image, and the RGB three-channel image is scaled to adjust the resolution of the RGB three-channel image to 224×224, thus obtaining a preprocessed image.

[0006] Preferably, the preprocessed image is decomposed into multi-scale images to obtain corresponding high-frequency sub-band images and low-frequency sub-band images, specifically including the following steps: 1) Perform two-dimensional wavelet transform on the preprocessed image to obtain the low-frequency subband image LL and the high-frequency subband images LH, HL and HH; 2) Calculate the high-frequency energy of the l-th two-dimensional wavelet transform based on the high-frequency sub-band images LH, HL, and HH of the l-th two-dimensional wavelet transform; calculate the high-frequency energy of the (l+1)-th two-dimensional wavelet transform based on the high-frequency sub-band images LH, HL, and HH of the (l+1)-th two-dimensional wavelet transform. 3) Calculate the change in high-frequency energy between two adjacent two-dimensional wavelet transforms based on the high-frequency energy of the l-th two-dimensional wavelet transform and the (l+1)-th two-dimensional wavelet transform; 4) Determine whether the change in high-frequency energy between two adjacent two-dimensional wavelet transforms is less than a preset threshold; when the change in high-frequency energy between two adjacent two-dimensional wavelet transforms is less than the preset threshold, directly output the high-frequency sub-band images LH, HL and HH obtained from the last two-dimensional wavelet transform and all previous two-dimensional wavelet transform processes, as well as the low-frequency sub-band image LL. When the change in high-frequency energy between two adjacent two-dimensional wavelet transforms is greater than or equal to a preset threshold, the low-frequency sub-band image LL obtained by the current two-dimensional wavelet transform is used as the input image for the next two-dimensional wavelet transform. Steps 1) to 3) are repeated cyclically until the change in high-frequency energy between two adjacent two-dimensional wavelet transforms is less than the preset threshold. The high-frequency sub-band images LH, HL and HH obtained by the last two-dimensional wavelet transform and all previous two-dimensional wavelet transform processes, as well as the low-frequency sub-band image LL, are output.

[0007] Preferably, the dynamic convolutional unit is constructed as follows: The dynamic convolutional unit includes an input layer and a first convolutional module. The high-frequency subband image is input into the input layer, and the input layer inputs the high-frequency subband image into the first convolutional module. The first convolutional module performs a convolution operation on the high-frequency subband image to obtain feature map A. Feature map A is then input into the gating probability prediction module to obtain the execution probability of the first convolutional module. The execution probability of the first convolutional module is compared with a preset value. When the execution probability of the first convolutional module is greater than or equal to the preset value, the feature map A output by the first convolutional module is used as the final output feature map of the dynamic convolutional unit. When the execution probability of the first convolutional module is less than the preset value, a second convolutional module is added. The network structure of the second convolutional module is the same as that of the first convolutional module. The second convolutional module performs a convolution operation on the feature map A output by the first convolutional module to obtain feature map B. Feature map B is input again into the gating probability prediction module to obtain the execution probability of the second convolutional module. Then, the sum of the execution probabilities of the first and second convolutional modules is calculated. When the sum of the execution probabilities of the first and second convolutional modules is greater than or equal to a preset value, feature map B output by the second convolutional module is used as the final output feature map of the dynamic convolutional unit. When the sum of the execution probabilities of the first and second convolutional modules is less than the preset value, a third convolutional module is added. The network structure of the third convolutional module is the same as that of the first convolutional module. The third convolutional module performs a convolution operation on feature map B output by the second convolutional module to obtain feature map C. This process is repeated until the sum of the execution probabilities of all convolutional modules is greater than or equal to a preset value. At this point, no new convolutional modules are added, and the feature map output by the last added convolutional module is used as the final output feature map of the dynamic convolutional unit.

[0008] Preferably, the gated probability prediction module includes a global average pooling layer, a fully connected layer, and a sigmoid activation layer connected in sequence; wherein, in the gated probability prediction module, the global average pooling layer performs global average pooling processing on the input feature map, the fully connected layer performs fully connected mapping on the output result of the global average pooling layer, and the sigmoid activation layer performs nonlinear mapping on the output result of the fully connected layer, constraining the output value to the (0,1) interval to obtain the execution probability.

[0009] Preferably, the predicted feature vector is matched with the reference feature vector to obtain the image positioning information of the image to be tested; specifically, the following steps are included: first, the cosine similarity between the predicted feature vector and all reference feature vectors in the pre-built indoor scene feature library is calculated, the reference feature vector corresponding to the maximum cosine similarity value is taken as the reference feature vector most similar to the predicted feature vector, and the spatial location information corresponding to the reference feature vector most similar to the predicted feature vector is taken as the image positioning information.

[0010] Preferably, training the feature extraction and fusion network includes the following steps: First, the feature extraction fusion network is trained for 5 epochs using the high-frequency and low-frequency sub-band images corresponding to the training scene support set to obtain the updated network. Then, the updated network is trained for 1 epoch using the high-frequency and low-frequency sub-band images corresponding to the training scene support set. This completes the training of one training segment. After training 300 training segments, the feature extraction fusion network sub-model is obtained.

[0011] Preferably, the training scene support set and training scene query set are obtained by randomly selecting 80% of the number of preprocessed images corresponding to dataset A. These selected preprocessed images and their corresponding spatial coordinate information constitute the training scene support set, and the remaining preprocessed images corresponding to dataset A and their corresponding spatial coordinate information constitute the training scene query set.

[0012] Compared with the prior art, the beneficial technical effects of this application are as follows: In the feature extraction and fusion network constructed in this application, dynamic convolutional units are used to perform multi-scale feature extraction and detail enhancement on high-frequency sub-band images, resulting in feature maps with multi-scale detail enhancement; the Swing Transformer module is used to extract features from low-frequency sub-band images at multiple scales, resulting in low-frequency feature maps containing multi-scale global semantic information; the feature pyramid module is used to fuse the feature maps with multi-scale detail enhancement output by the dynamic convolutional units and the low-frequency feature maps containing multi-scale global semantic information output by the Swing Transformer module, resulting in a fused feature map; the fused feature map contains edge information and texture detail information from the high-frequency sub-band images, as well as global structural information and main contour features from the low-frequency sub-band images; the pooling layer performs pooling operations on the fused feature map to obtain predicted feature vectors; The dynamic convolutional unit is constructed as follows: It includes an input layer and a first convolutional module. A high-frequency subband image is input to the input layer, which then inputs the high-frequency subband image to the first convolutional module. The first convolutional module performs a convolution operation on the high-frequency subband image to obtain feature map A. Feature map A is then input to a gated probability prediction module. In this module, a global average pooling layer performs global average pooling on the input feature map A, a fully connected layer performs a fully connected mapping on the output of the global average pooling layer, and a sigmoid activation layer performs a non-linear mapping on the output of the fully connected layer to obtain the execution probability of the first convolutional module. The execution probability of the first convolutional module is compared with a preset value. When the execution probability of the first convolutional module is greater than or equal to the preset value, the feature map A output by the first convolutional module is used as the final output feature map of the dynamic convolutional unit. When the execution probability of the first convolutional module is less than the preset value, a second convolutional module is added. The network structure of the second convolutional module is the same as that of the first convolutional module. The second convolutional module performs a convolution operation on the feature map A output by the first convolutional module to obtain feature map B. Feature map B is input again into the gating probability prediction module to obtain the execution probability of the second convolutional module. Then, the sum of the execution probabilities of the first and second convolutional modules is calculated. When the sum of the execution probabilities of the first and second convolutional modules is greater than or equal to a preset value, feature map B output by the second convolutional module is used as the final output feature map of the dynamic convolutional unit. When the sum of the execution probabilities of the first and second convolutional modules is less than the preset value, a third convolutional module is added. The network structure of the third convolutional module is the same as that of the first convolutional module. The third convolutional module performs a convolution operation on feature map B output by the second convolutional module to obtain feature map C. This process is repeated until the sum of the execution probabilities of all convolutional modules is greater than or equal to a preset value. At this point, no new convolutional modules are added, and the feature map output by the last added convolutional module is used as the final output feature map of the dynamic convolutional unit.

[0013] Tests have shown that the method described in this application can achieve a high positioning accuracy. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the network structure of the feature extraction fusion network in this application; Figure 2 for Figure 1 A schematic diagram of the network structure of the dynamic convolutional unit; Figure 3 for Figure 2 A schematic diagram of the network structure of the gating probability prediction module. Detailed Implementation

[0015] An indoor positioning method based on multi-scale feature extraction from high- and low-frequency sub-band images includes the following steps: Step S1: Obtain the training scenario support set, training scenario query set, test scenario support set, and test scenario query set, which specifically includes the following steps: In this application, the dataset A is consistent with the Real dataset disclosed in the paper "BIM-PoseNet: Indoor camera localization using a 3D indoor model and deep learning from synthetic images"; wherein, dataset A contains 949 indoor scene images and their corresponding spatial coordinate information, these indoor scene images are called original indoor scene images, and each image has a resolution of 640×480 pixels. The dataset B is consistent with the Sdjzu dataset published in the paper "An improved BIM aided indoor localization method via enhancing cross-domain image retrieval based on deep learning"; dataset B contains 798 indoor scene images and their corresponding spatial coordinate information. These indoor scene images are also called original indoor scene images, and each image has a resolution of 1080×1920 pixels. Step S1-1: Preprocess the images in datasets A and B respectively to obtain preprocessed images; specifically, this includes the following steps: The images in dataset A are converted into RGB three-channel images; and the RGB three-channel images are scaled to adjust the resolution of the RGB three-channel images to 224×224, thus obtaining the preprocessed images corresponding to dataset A. The images in dataset B are converted into RGB three-channel images; and the RGB three-channel images are scaled to adjust the resolution of the RGB three-channel images to 224×224, thus obtaining the preprocessed images corresponding to dataset B. Step S1-2: Divide the preprocessed images and their corresponding spatial coordinate information corresponding to dataset A to obtain the training scene support set and the training scene query set; divide the preprocessed images and their corresponding spatial coordinate information corresponding to dataset B to obtain the test scene support set and the test scene query set; specifically including the following steps: 80% of the preprocessed images corresponding to dataset A are randomly selected. These selected preprocessed images and their corresponding spatial coordinate information constitute the training scene support set, and the remaining preprocessed images corresponding to dataset A and their corresponding spatial coordinate information constitute the training scene query set. 80% of the preprocessed images corresponding to dataset B are randomly selected. These selected preprocessed images and their corresponding spatial coordinate information constitute the test scenario support set, and the remaining preprocessed images corresponding to dataset B and their corresponding spatial coordinate information constitute the test scenario query set.

[0016] Step S2: Perform multi-scale decomposition on the preprocessed images in the training scene support set and the training scene query set respectively to obtain the multi-scale high-frequency sub-band images and low-frequency sub-band images corresponding to the training scene support set and the multi-scale high-frequency sub-band images and low-frequency sub-band images corresponding to the training scene query set respectively. Taking multi-scale decomposition of preprocessed images supporting training scenarios as an example, the specific steps include: 1) Two-dimensional wavelet transform is performed on the preprocessed images in the training scene support set to obtain low-frequency sub-band image LL and high-frequency sub-band images LH, HL and HH; wherein, when performing the l-th two-dimensional wavelet transform, the low-frequency sub-band image LL and the high-frequency sub-band images LH, HL and HH of the l-th two-dimensional wavelet transform are obtained, and this application performs at least two two-dimensional wavelet transforms; when performing the (l+1)-th two-dimensional wavelet transform, the low-frequency sub-band image LL and the high-frequency sub-band images LH, HL and HH of the (l+1)-th two-dimensional wavelet transform are obtained; in this application, the low-frequency sub-band image LL contains the overall structural information and main contour features of the preprocessed image, and the high-frequency sub-band images LH, HL and HH contain the horizontal edge information, vertical edge information and diagonal texture detail information of the image, respectively; 2) Calculate the high-frequency energy of the l-th two-dimensional wavelet transform based on the high-frequency sub-band images LH, HL and HH of the l-th two-dimensional wavelet transform; calculate the high-frequency energy of the (l+1)-th two-dimensional wavelet transform based on the high-frequency sub-band images LH, HL and HH of the (l+1)-th two-dimensional wavelet transform; wherein, the calculation method of the high-frequency energy of the l-th two-dimensional wavelet transform is as shown in Equation (1); the calculation method of the high-frequency energy of the (l+1)-th two-dimensional wavelet transform is as shown in Equation (2); (1) In equation (1), This represents the high-frequency energy of the l-th two-dimensional wavelet transform. This represents a subband containing horizontal edge information of the image obtained from the l-th two-dimensional wavelet transform. This represents the subband containing vertical edge information of the image in the l-th two-dimensional wavelet transform. This represents the subband containing diagonal texture detail information of the image in the l-th two-dimensional wavelet transform; (2) In equation (2), This represents the high-frequency energy of the (l+1)th two-dimensional wavelet transform. This represents a sub-band containing horizontal edge information of the image obtained by the (l+1)th two-dimensional wavelet transform. This represents the subband containing vertical edge information of the image in the (l+1)th two-dimensional wavelet transform. This represents the subband containing diagonal texture details of the image in the (l+1)th two-dimensional wavelet transform; 3) Calculate the change in high-frequency energy between two adjacent two-dimensional wavelet transforms based on the high-frequency energy of the l-th and (l+1)-th two-dimensional wavelet transforms; where the change in high-frequency energy S between two adjacent two-dimensional wavelet transforms is... (l) The calculation method is shown in equation (3); (3) In formula (3), S (l) This represents the high-frequency energy change between the l-th and (l+1)-th two-dimensional wavelet transforms. This represents the high-frequency energy of the l-th two-dimensional wavelet transform. This represents the high-frequency energy of the (l+1)th two-dimensional wavelet transform; 4) Determine whether the change in high-frequency energy between two consecutive two-dimensional wavelet transforms is less than a preset threshold (the preset threshold is 0.05 in this embodiment). When the change in high-frequency energy between two consecutive two-dimensional wavelet transforms is less than a preset threshold, the low-frequency sub-band image LL and the high-frequency sub-band images LH, HL and HH obtained from the last two-dimensional wavelet transform and all previous two-dimensional wavelet transform processes are directly output. When the change in high-frequency energy between two adjacent two-dimensional wavelet transforms is greater than or equal to a preset threshold, the low-frequency sub-band image LL obtained by the current two-dimensional wavelet transform is used as the input image for the next two-dimensional wavelet transform. Steps 1) to 3) are repeated in a loop until the change in high-frequency energy between two adjacent two-dimensional wavelet transforms is less than the preset threshold. The high-frequency sub-band images LH, HL and HH obtained by the last two-dimensional wavelet transform and all previous two-dimensional wavelet transform processes, as well as the low-frequency sub-band image LL, are output. The aforementioned configuration in this application is essentially an adaptive wavelet decomposition method, which achieves the purpose of multi-scale decomposition of the preprocessed images in the training scene support set, resulting in multi-scale high-frequency sub-band images LH, HL, and HH, as well as multi-scale low-frequency sub-band images LL. In this application, the method of multi-scale decomposition of the preprocessed images in the training scene query set is the same as the method of multi-scale decomposition of the preprocessed images in the training scene support set.

[0017] Step S3: Construct a feature extraction and fusion network for multi-scale feature extraction and feature fusion of visual information in high-frequency subband images and low-frequency subband images; The network structure of the feature extraction fusion network is as follows: Figure 1 As shown, it includes a dynamic convolutional unit and a Swing Transformer module. The outputs of the dynamic convolutional unit and the Swing Transformer module are connected to the feature pyramid module, and the feature pyramid module is connected to the pooling layer. The feature pyramid module outputs a fused feature map, and the pooling layer outputs a predicted feature vector. The dynamic convolutional unit is used to perform multi-scale feature extraction and detail enhancement on the high-frequency sub-band image, resulting in a feature map with multi-scale detail enhancement. The Swin Transformer module is used to extract features from the multi-scale low-frequency sub-band image LL, resulting in a low-frequency feature map containing multi-scale global semantic information. Since the Swin Transformer module can extract features from the multi-scale low-frequency sub-band image LL, this application also achieves multi-scale feature extraction of visual information in the low-frequency sub-band image LL. The feature pyramid module is used to fuse the feature map with multi-scale detail enhancement output by the dynamic convolutional unit and the low-frequency feature map containing multi-scale global semantic information output by the Swin Transformer module, resulting in a fused feature map. The fused feature map contains edge information and texture detail information in the high-frequency sub-band image, as well as global structural information and main contour features in the low-frequency sub-band image. The pooling layer performs pooling operation on the fused feature map to obtain a predicted feature vector.

[0018] In this application, the dynamic convolutional unit is constructed as follows: The dynamic convolutional unit includes an input layer and a convolutional module. A high-frequency subband image is input to the input layer, which then inputs the high-frequency subband image to the first convolutional module. The first convolutional module performs a convolution operation on the high-frequency subband image to obtain a feature map A. Feature map A is then input to a gated probability prediction module. In this module, a global average pooling layer performs global average pooling on the input feature map A, a fully connected layer performs a fully connected mapping on the output of the global average pooling layer, and a sigmoid activation layer performs a non-linear mapping on the output of the fully connected layer, constraining the output value to the (0,1) interval to obtain the execution probability of the first convolutional module. All convolutional modules in this application are used for feature extraction and detail enhancement of the input feature map. The network structure of the convolutional modules in this application is the same as the ConvolutionalNeural Networks model structure disclosed in the paper "ImageNet Classification with Deep Convolutional Neural Networks". The execution probability of the first convolutional module is compared with a preset value. When the execution probability of the first convolutional module is greater than or equal to the preset value, the feature map A output by the first convolutional module is used as the final output feature map of the dynamic convolutional unit. When the execution probability of the first convolutional module is less than the preset value, a second convolutional module is added. The network structure of the second convolutional module is the same as that of the first convolutional module. The second convolutional module performs a convolution operation on the feature map A output by the first convolutional module to obtain feature map B. Feature map B is input again into the gating probability prediction module to obtain the execution probability of the second convolutional module. Then, the sum of the execution probabilities of the first and second convolutional modules is calculated. When the sum of the execution probabilities of the first and second convolutional modules is greater than or equal to a preset value, feature map B output by the second convolutional module is used as the final output feature map of the dynamic convolutional unit. When the sum of the execution probabilities of the first and second convolutional modules is less than the preset value, a third convolutional module is added. The network structure of the third convolutional module is the same as that of the first convolutional module. The third convolutional module performs a convolution operation on feature map B output by the second convolutional module to obtain feature map C. This process is repeated until the sum of the execution probabilities of all convolutional modules is greater than or equal to a preset value. At this point, no new convolutional modules are added, and the feature map output by the last added convolutional module is used as the final output feature map of the dynamic convolutional unit.

[0019] Specifically, in this embodiment... The high-frequency subband image is input into the input layer, which then inputs the high-frequency subband image into the first convolutional module. The first convolutional module performs a convolution operation on the high-frequency subband image to obtain feature map A, which is a feature map with low-level feature representation. Feature map A is then input into the gated probability prediction module. In the gated probability prediction module, the global average pooling layer performs global average pooling processing on the input feature map, the fully connected layer performs a fully connected mapping on the output of the global average pooling layer, and the Sigmoid activation layer performs a non-linear mapping on the output of the fully connected layer, constraining the output value to the (0,1) interval to obtain the execution probability of the first convolutional module. The execution probability of the first convolutional module is compared with a preset value (in this embodiment, the preset value is 1); If the execution probability of the first convolutional module is found to be less than the preset value, then a second convolutional module is added. The second convolutional module performs a convolution operation on the feature map A output by the first convolutional module to obtain feature map B, which is a feature map with intermediate feature representation. Feature map B is input into the gated probability prediction module to obtain the execution probability of the second convolutional module. Then, the sum of the execution probabilities of the first and second convolutional modules is calculated. If the sum of the execution probabilities of the first and second convolutional modules is found to be less than the preset value, a third convolutional module is added. The third convolutional module performs a convolution operation on the feature map output by the second convolutional module to obtain feature map C, which is a feature map with high-level feature representation. The feature map C is input into the gated probability prediction module to obtain the execution probability of the third convolutional module. The execution probabilities of the first, second, and third convolutional modules are calculated. If the sum of the execution probabilities of the first, second, and third convolutional modules is found to be less than the preset value, a fourth convolutional module is added. The fourth convolutional module performs a convolution operation on the feature map C output by the third convolutional module to obtain the feature map D, which is a feature map with abstract semantic feature representation. The feature map D is input to the gating probability prediction module to obtain the execution probability of the fourth convolutional module; the execution probabilities of the first, second, third and fourth convolutional modules are calculated and summed; it is found that the sum of the execution probabilities of the first, second, third and fourth convolutional modules is greater than the preset value. Therefore, in this embodiment, the feature map D output by the fourth convolutional module is the feature map output by the dynamic convolutional unit.

[0020] In other words, in this embodiment, the network structure of the dynamic convolutional unit is as follows: Figure 2 As shown; the dynamic convolutional unit includes an input layer and four convolutional modules with the same network structure, referred to as the first convolutional module, the second convolutional module, the third convolutional module, and the fourth convolutional module; wherein, each of the first to fourth convolutional modules includes a convolutional layer (with a kernel size of 3×3), a normalization layer, and a ReLU activation layer connected in sequence; wherein, each of the first to fourth convolutional modules is used to perform feature extraction and detail enhancement on the input feature map; In this embodiment, the input layer, the first convolutional module, the second convolutional module, the third convolutional module, and the fourth convolutional module are connected sequentially; the outputs of the first, second, third, and fourth convolutional modules are all connected to the gated probability prediction module; and the network structure of the gated probability prediction module is as follows: Figure 3 As shown, it includes a global average pooling layer, a fully connected layer, and a Sigmoid activation layer connected in sequence; wherein, in the gated probability prediction module, the global average pooling layer performs global average pooling processing on the input feature map, the fully connected layer performs fully connected mapping on the output result of the global average pooling layer, and the Sigmoid activation layer performs non-linear mapping on the output result of the fully connected layer, constraining the output value to the (0,1) interval to obtain the execution probability; The above-mentioned settings in this application can achieve the effect of multi-scale feature extraction and detail enhancement of visual information in the high-frequency subband image of the input dynamic convolution unit; In this application, the network structure of the Swin Transformer module is the same as the Swin Transformer model structure disclosed in the paper "Swin Transformer: Hierarchical Vision Transformer using Shifted Window"; In this application, the network structure of the feature pyramid module is the same as the Feature Pyramid Networks network structure disclosed in the paper "Feature Pyramid Networks for Object Detection"; Step S4: Train the feature extraction fusion network based on the high-frequency and low-frequency subband images corresponding to the support set and the query set. Specifically, first train the feature extraction fusion network for 5 epochs using the high-frequency and low-frequency subband images corresponding to the support set of the training scenario to obtain the updated network; then train the updated network for 1 epoch using the high-frequency and low-frequency subband images corresponding to the support set of the training scenario; thus completing one training segment; after training 300 training segments, the feature extraction fusion network sub-model is obtained. Step S4 specifically includes the following steps: (1) Input the high-frequency subband images and low-frequency subband images corresponding to the support set of the training scene into the dynamic convolutional unit and the Swing Transformer module in the feature extraction and fusion network, respectively, and perform forward propagation to calculate the inner loop loss. Ltr Loss in the inner loop Ltr Guided by the initial network parameters, backpropagation is performed to refine the initial network parameters of the feature extraction and fusion network. θ1 Gradient updates are performed to complete one epoch of training. After 5 gradient updates (i.e., 5 epochs of training), the updated network is obtained, and its parameters are as follows: Among them, network parameters With network parameters θ1 ( θ1 The relationship between the network parameters (before gradient update) is shown in equation (4): (4) In equation (4), θ1 Indicates the initial network parameters. This indicates updating the network parameters, where α represents the inner loop learning rate. In this embodiment, α is set to 1 × 10⁻². Representing network parameters θ1 gradient operator, Indicates network parameters as θ1Feature extraction fusion network fθ Inner loop loss on the training scenario support set Dtr; In this application, the inner-loop loss is calculated in the same way as the inner-loop loss disclosed in "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks".

[0021] (2) Input the high-frequency subband images and low-frequency subband images corresponding to the training scene query set into the network parameters. In the updated network, forward propagation is performed to calculate the outer loop loss. Lval Loss in the external circulation Lval Backpropagation is performed under the guidance of the network parameters. Gradient updates are performed, which completes one epoch of training. In this application, completing the training process of 5 epochs described in step (1) followed by the training process of 1 epoch described in step (2) is considered as completing one training segment. After completing 300 training segments, this application obtains the feature extraction fusion network sub-model. The network parameters of the feature extraction fusion network sub-model are as follows: θ2 In this application, network parameters With network parameters θ2 The relationship is shown in equation (5): (5) In equation (5), β represents the outer loop learning rate, and β takes the value of 1×10⁻⁵. Representing network parameters gradient operator, Indicates network parameters as Network update Find the outer loop loss function value on the training scenario query set Dval; In this application, the outer loop loss is calculated in the same way as the outer-loop meta loss disclosed in "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks".

[0022] Step S5: Acquire indoor scene images and their location information at various locations within the indoor scene to be tested. Preprocess and multi-scale decompose the indoor scene images at each location sequentially to obtain high-frequency sub-band images and low-frequency sub-band images. Then, train the feature extraction and fusion network sub-model to obtain the feature extraction and fusion network model. Next, obtain reference feature vectors based on the corresponding high-frequency and low-frequency sub-band images of the indoor scene images at each location and the feature extraction and fusion network model. Then, acquire test images in real-time within the indoor scene to be tested. Preprocess and multi-scale decompose the acquired test images sequentially to obtain the corresponding high-frequency and low-frequency sub-band images of the test images. Obtain the predicted feature vector of the test image based on the corresponding high-frequency and low-frequency sub-band images of the test image and the feature extraction and fusion network model. Perform similarity matching between the predicted feature vector of the test image and the reference feature vector to obtain the image localization information of the test image. Specifically, this includes the following steps: First, a monocular camera is used to acquire indoor scene images at various locations in the indoor scene to be tested. These indoor scene images serve as reference images, and the position information corresponding to the indoor scene images (position information refers to precise coordinates (x, y, z)) is recorded. These acquired indoor scene images and their corresponding position information constitute an indoor scene reference library. The indoor scene images in the indoor scene reference library are preprocessed according to the steps described in step S1-1 to obtain preprocessed images. Then, the preprocessed images are divided into a reference image support set and a reference image query set according to the method described in step S1-2. Then, the preprocessed images in the reference image support set and the reference image query set are decomposed into multi-scale images according to the steps described in step S2 to obtain high-frequency sub-band images and low-frequency sub-band images corresponding to the support set and the query set. Based on the high-frequency sub-band images and low-frequency sub-band images corresponding to the support set and the query set, the feature extraction fusion network sub-model obtained in step S4 is trained for 30 training segments according to the training method described in step S4 to obtain the feature extraction fusion network model. The high-frequency sub-band images and low-frequency sub-band images of the reference image support set are input into the feature extraction fusion network model to obtain the predicted feature vectors corresponding to the reference images. These predicted feature vectors corresponding to the reference images are used as reference feature vectors. Then, the image to be tested, which is acquired in real time by a monocular camera in the indoor scene to be tested, is preprocessed according to the steps described in step S1-1 to obtain a preprocessed image. Then, the preprocessed image is decomposed into multiple scales according to the steps described in step S2 to obtain the high-frequency sub-band image and low-frequency sub-band image corresponding to the image to be tested. Then, the high-frequency sub-band image and low-frequency sub-band image are input into the feature extraction fusion network model and propagated forward to obtain the predicted feature vector of the image to be tested. Then, the predicted feature vector of the image to be tested is matched with the reference feature vector to obtain the image positioning information of the image to be tested. Specifically, the cosine similarity between the predicted feature vector of the image to be tested and all reference feature vectors is calculated first. The reference feature vector corresponding to the maximum cosine similarity value is taken as the reference feature vector most similar to the predicted feature vector of the image to be tested. The reference feature vector most similar to the predicted feature vector of the image to be tested is taken as the image positioning position. The spatial position information corresponding to the reference feature vector most similar to the predicted feature vector of the image to be tested is obtained. The spatial position information is the image positioning information of the image to be tested.

[0023] test: To verify the advantages of the indoor positioning method proposed in this application in different indoor scenarios, this application specifically adopted the same testing strategy to obtain the positioning performance of the indoor positioning method described in this application and five existing cross-scenario positioning methods. Among them, the existing indoor positioning methods used in this application include the TransferLearning method (from "A Comprehensive Survey on Transfer Learning"), the FOMAML method (from "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks"), the Reptile method (from "On First-Order Meta-Learning Algorithms"), the Meta-SGD method (from "Meta-SGD: Learning to Learn Quickly for Few-Shot Learning"), and the Prototypical Networks method (from "Prototypical Networks for Few-shot Learning"). The testing strategy adopted in this application is as follows: First, the preprocessed images in the test scene support set and the test scene query set are decomposed into multiple scales to obtain the high-frequency subband image and low-frequency subband image corresponding to the test scene support set and the high-frequency subband image and low-frequency subband image corresponding to the test scene query set, respectively. In this application, the method of decomposing the preprocessed images in the test scene support set and the test scene query set into multiple scales is the same as the method of decomposing the preprocessed images in the training scene support set into multiple scales. Then, based on the high-frequency sub-band images and low-frequency sub-band images corresponding to the training scenario support set and the high-frequency sub-band images and low-frequency sub-band images corresponding to the training scenario query set, the feature extraction fusion network sub-model obtained in step S4 is trained for 30 training segments according to the training method described in step S4, and the feature extraction fusion network test model is obtained. Then, the high-frequency subband image and low-frequency subband image corresponding to the test scenario support set are input into the feature extraction fusion network test model and propagated forward to obtain the predicted feature vector corresponding to the test scenario support set. The predicted feature vector corresponding to the test scenario support set is used as the reference feature vector. The high-frequency subband image and low-frequency subband image corresponding to the test scenario query set are input into the feature extraction fusion network test model and propagated forward to obtain the predicted feature vector corresponding to the test scenario query set. The predicted feature vector corresponding to the test scene query set is matched with the reference feature vector for similarity. Specifically, the cosine similarity between the predicted feature vector corresponding to the test scene query set and all reference feature vectors is calculated first. The reference feature vector corresponding to the maximum cosine similarity value is taken as the reference feature vector most similar to the predicted feature vector corresponding to the test scene query set. The spatial location information corresponding to the reference feature vector most similar to the predicted feature vector corresponding to the test scene query set is taken as the image positioning information.

[0024] Subsequently, this application uses the N-way K-shot evaluation method to test the localization effect. The test results are shown in Table 1. The evaluation method of N-way K-shot used in this application is the same as the N-way k-shot learning task method disclosed in the paper "Matching Networks for One Shot Learning". Here, N-way means that N candidate location categories are selected from the indoor scene feature library in each test. K-shot means that K reference samples are selected for each location category to construct the feature representation of that category. Taking 3way and 1shot in Table 1 as an example, it means that in each test, 3 candidate location categories are selected from the indoor scene feature library, and 1 reference sample is selected for each location category to construct the feature representation of that category. Table 1. Test results of positioning accuracy obtained from the above indoor positioning methods.

[0025] As can be seen from Table 1, among the existing indoor positioning methods, the Transfer-Learning method achieved better test results. Therefore, this application will focus on comparing the Transfer-Learning method with the method described in this application, as follows: Compared to the Transfer-Learning method, the method described in this application improves the localization accuracy by (78.23%−70.13%) / 70.13%×100%=11.55% in a 3-way, 1-shot task setting; Compared to the Transfer-Learning method, the method described in this application improves the localization accuracy by (88.33%−82.48%) / 82.48%×100%=7.09% in 3-way, 5-shot task settings; Compared to the Transfer-Learning method, the method described in this application improves the localization accuracy by (72.71%−66.96%) / 66.96%×100%==8.59% in a 5-way, 1-shot task setting; Compared to the Transfer-Learning method, the method described in this application improves the localization accuracy by (79.47%−74.48%) / 74.48%×100%=6.70% under 5-way, 5-shot task settings.

[0026] Clearly, as shown in Table 1, the positioning accuracy achieved by the method described in this application is significantly better than that achieved by the existing methods described above.

Claims

1. An indoor positioning method based on multi-scale feature extraction of high- and low-frequency sub-band images, characterized in that: Includes the following steps: Indoor scene images and their location information at various locations in the indoor scene to be tested are collected. The indoor scene images are then preprocessed and decomposed at multiple scales to obtain high-frequency and low-frequency sub-band images. The feature extraction fusion network sub-model is further trained to obtain the feature extraction fusion network model; a reference feature vector is obtained based on the high and low frequency sub-band images and the fusion network model; the image to be tested is acquired in real time in the indoor scene to be tested and preprocessed and decomposed at multiple scales to obtain high and low frequency sub-band images; the predicted feature vector of the image to be tested is obtained based on the corresponding high and low frequency sub-band images and the network model; the predicted feature vector is matched with the reference feature vector to obtain the image localization information of the image to be tested. The network model is trained based on a feature extraction fusion network; in the fusion network, dynamic convolutional units and Swing Transformer modules are connected to a feature pyramid module, the feature pyramid module is connected to a pooling layer, and the pooling layer outputs a predicted feature vector. The dynamic convolutional unit is used to perform multi-scale feature extraction and detail enhancement on the high-frequency sub-band image, resulting in a feature map with multi-scale detail enhancement. The Swing Transformer module is used to extract features from the low-frequency sub-band image at multiple scales, resulting in a low-frequency feature map containing multi-scale global semantic information. The feature pyramid module is used to fuse the feature map with multi-scale detail enhancement output from the dynamic convolutional unit and the low-frequency feature map containing multi-scale global semantic information output from the Swing Transformer module, resulting in a fused feature map. The fused feature map contains edge information and texture detail information from the high-frequency sub-band image, as well as global structural information and main contour features from the low-frequency sub-band image. The multi-scale decomposition includes the following steps: 1) Perform two-dimensional wavelet transform on the preprocessed image to obtain the low-frequency subband image LL and the high-frequency subband images LH, HL and HH; 2) Calculate the high-frequency energy of the l-th two-dimensional wavelet transform based on the high-frequency sub-band images LH, HL, and HH of the l-th two-dimensional wavelet transform; calculate the high-frequency energy of the (l+1)-th two-dimensional wavelet transform based on the high-frequency sub-band images LH, HL, and HH of the (l+1)-th two-dimensional wavelet transform. 3) Calculate the change in high-frequency energy between two adjacent two-dimensional wavelet transforms based on the high-frequency energy of the l-th two-dimensional wavelet transform and the (l+1)-th two-dimensional wavelet transform; 4) Determine whether the energy change is less than a preset threshold; when the energy change is less than the preset threshold, output the high-frequency subband images LH, HL and HH obtained from the last two-dimensional wavelet transform and all previous two-dimensional wavelet transform processes, as well as the low-frequency subband image LL. When the energy change is greater than or equal to a preset threshold, the low-frequency sub-band image LL obtained by the current two-dimensional wavelet transform is used as the input image for the next two-dimensional wavelet transform. Steps 1) to 3) are repeated cyclically until the calculated energy change is less than the preset threshold. The high-frequency sub-band images LH, HL and HH obtained by the last two-dimensional wavelet transform and all previous two-dimensional wavelet transform processes, as well as the low-frequency sub-band image LL, are output.

2. The indoor positioning method based on multi-scale feature extraction of high- and low-frequency sub-band images according to claim 1, characterized in that: The preprocessing of the indoor scene image includes the following steps: After converting the indoor scene image into an RGB three-channel image, the image is scaled to obtain a preprocessed image.

3. The indoor positioning method based on multi-scale feature extraction of high- and low-frequency sub-band images according to claim 1, characterized in that: The dynamic convolutional unit is constructed as follows: The dynamic convolutional unit includes an input layer and a first convolutional module. The input layer inputs the high-frequency subband image into the first convolutional module to obtain feature map A; feature map A is then input into the gating probability prediction module to obtain the execution probability of the first convolutional module. The execution probability of the first convolutional module is compared with a preset value; when the execution probability is greater than or equal to the preset value, the feature map A output by the first convolutional module is used as the final output feature map of the dynamic convolutional unit; when the execution probability is less than the preset value, a second convolutional module is added, and the second convolutional module performs a convolution operation on the feature map A to obtain the feature map B. Feature map B is input into the gated probability prediction module to obtain the execution probability of the second convolution module; the sum of the execution probabilities of the first and second convolution modules is calculated; when the sum of the execution probabilities is greater than or equal to a preset value, feature map B is used as the final output feature map of the dynamic convolution unit; when the sum of the execution probabilities is less than the preset value, a third convolution module is added, and the third convolution module performs a convolution operation on feature map B to obtain feature map C. This process is repeated until the sum of the execution probabilities of all convolutional modules is greater than or equal to a preset value. At this point, no new convolutional modules are added, and the feature map output by the last added convolutional module is used as the final output feature map of the dynamic convolutional unit.

4. The indoor positioning method based on multi-scale feature extraction of high- and low-frequency sub-band images according to claim 3, characterized in that: The gated probability prediction module consists of a global average pooling layer, a fully connected layer, and a Sigmoid activation layer connected in sequence.

5. The indoor positioning method based on multi-scale feature extraction of high- and low-frequency sub-band images according to claim 1, characterized in that: The image localization information of the image to be tested is obtained by performing similarity matching between the predicted feature vector and the reference feature vector; the steps include: first, calculating the cosine similarity between the predicted feature vector and all reference feature vectors, and then using the spatial location information corresponding to the reference feature vector that is most similar to the predicted feature vector as the image localization information.

6. The indoor positioning method based on multi-scale feature extraction of high- and low-frequency sub-band images according to claim 1, characterized in that: Training the feature extraction and fusion network includes the following steps: First, the feature extraction fusion network is trained for 5 epochs using the high-frequency and low-frequency sub-band images corresponding to the training scene support set to obtain the updated network. Then, the updated network is trained for 1 epoch using the high-frequency and low-frequency sub-band images corresponding to the training scene support set. This completes the training of one training segment. After training 300 training segments, the feature extraction fusion network sub-model is obtained.

7. The indoor positioning method based on multi-scale feature extraction of high- and low-frequency sub-band images according to claim 6, characterized in that: The training scenario support set and query set are obtained as follows: 80% of the number of preprocessed images corresponding to dataset A are randomly selected. These selected preprocessed images and their corresponding spatial coordinate information constitute the training scenario support set, and the remaining preprocessed images corresponding to dataset A and their corresponding spatial coordinate information constitute the training scenario query set.

Images