Method and device for carrying out Bluetooth navigation in virtual reality and storage medium

Abstract

The invention discloses a method and device for conducting Bluetooth navigation in virtual reality and a storage medium, and the method comprises the steps that if a request for navigating to a destination in an indoor environment is received, multi-frame image data and multi-frame Bluetooth beacon signals are collected in the indoor environment at the same time; judging whether the multi-frame image data meets an indoor environment navigation condition or not; if yes, extracting target building features from the current frame image data; extracting a plurality of target Bluetooth beacon features from the multi-frame received signal strength indication; fusing the target building feature and multiple target Bluetooth beacon features into a target fingerprint feature; detecting the real-time position of the mobile device in the indoor environment according to the target fingerprint features; a virtual route from the real-time location to the destination is displayed in the image data in a virtual reality manner. When the indoor environment changes and interference exists, the influence on the characteristics can be reduced, and the positioning accuracy can be improved, so that the accuracy of AR navigation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Bluetooth, and in particular to a method, device and storage medium for performing Bluetooth navigation in virtual reality. Background Art

[0002] In large indoor places such as shopping malls, fingerprint technology based on the Bluetooth protocol is usually used for positioning and navigation to facilitate users to find their destinations.

[0003] In the offline stage, Bluetooth base stations (Beacons) are deployed in various target areas indoors to collect RSSI (Received Signal Strength Indication) data of Bluetooth beacon signals at different locations to form a fingerprint library.

[0004] During the online stage, the user's device scans the surrounding Bluetooth beacon signals and matches the real-time collected RSSI data with the fingerprint library to determine the user's location.

[0005] However, the indoor environment changes rapidly and there are many interferences, which makes the online indoor environment and the offline indoor environment have certain differences, reducing the accuracy of positioning and thus reducing the accuracy of navigation. Summary of the invention

[0006] In view of this, the present invention provides a method, device and storage medium for performing Bluetooth navigation in virtual reality, so as to improve the accuracy of positioning and navigation based on fingerprint technology of Bluetooth protocol.

[0007] A first aspect of the present invention provides a method for performing Bluetooth navigation in virtual reality, which is applied to a mobile device, and the method comprises:

[0008] If a request for navigating to a destination in an indoor environment is received, a plurality of frames of image data and a plurality of frames of Bluetooth beacon signals are simultaneously collected in the indoor environment; the Bluetooth beacon signals have a received signal strength indication;

[0009] Determine whether the image data of the multiple frames meet the conditions for navigation in the indoor environment; if so, extract the target building features from the image data of the current frame;

[0010] Extracting multiple target Bluetooth beacon features from the received signal strength indications of multiple frames;

[0011] Merging the target building feature with multiple target Bluetooth beacon features into a target fingerprint feature;

[0012] Detecting the real-time location of the mobile device in the indoor environment according to the target fingerprint feature;

[0013] A virtual route from the real-time location to the destination is displayed in the image data in a virtual reality manner.

[0014] A second aspect of the present invention provides a device for performing Bluetooth navigation in virtual reality, which is applied to a mobile device, and the device comprises:

[0015] An environmental data acquisition module, for simultaneously acquiring multiple frames of image data and multiple frames of Bluetooth beacon signals in the indoor environment if a request for navigating to a destination in the indoor environment is received; the Bluetooth beacon signal has a received signal strength indication;

[0016] A navigation condition judgment module, used to judge whether the image data of multiple frames meet the conditions for navigation in the indoor environment; if so, calling the building feature extraction module;

[0017] A building feature extraction module, used to extract target building features from the image data of the current frame;

[0018] A Bluetooth beacon feature extraction module, used to extract multiple target Bluetooth beacon features from the received signal strength indication of multiple frames;

[0019] A fingerprint feature fusion module, used to fuse the target building feature with multiple target Bluetooth beacon features into a target fingerprint feature;

[0020] A real-time position detection module, used to detect the real-time position of the mobile device in the indoor environment according to the target fingerprint feature;

[0021] A virtual route display module is used to display a virtual route from the real-time location to the destination in the image data in a virtual reality manner.

[0022] A third aspect of the present invention provides a mobile device, the mobile device comprising:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for Bluetooth navigation in virtual reality as described in the first aspect above.

[0026] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for Bluetooth navigation in virtual reality as described in the first aspect above.

[0027] A fifth aspect of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for Bluetooth navigation in virtual reality as described in the first aspect above.

[0028] In this embodiment, if a request to navigate to a destination in an indoor environment is received, multiple frames of image data and multiple frames of Bluetooth beacon signals are collected simultaneously in the indoor environment; the Bluetooth beacon signal has a received signal strength indication; it is determined whether the multiple frames of image data meet the conditions for navigation in the indoor environment; if so, the target building features are extracted from the current frame image data; multiple target Bluetooth beacon features are extracted from the multiple frames of received signal strength indication; the target building features and multiple target Bluetooth beacon features are merged into a target fingerprint feature; the real-time position of the mobile device in the indoor environment is detected based on the target fingerprint feature; and a virtual route from the real-time position to the destination is displayed in the image data in a virtual reality manner. This embodiment reuses the features of the building in the image data to enhance the features of the RSSI, increase the amount of information of the features, reduce the impact on the features when the indoor environment changes and there is interference, improve the accuracy of positioning, and thus improve the accuracy of AR navigation.

[0029] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 This is a flow chart of a method for performing Bluetooth navigation in virtual reality provided by Embodiment 1 of the present invention.

[0032] Figure 2 It is a structural diagram of a building detection network provided in Example 1 of the present invention.

[0033] Figure 3 It is a structural schematic diagram of a convolution module provided in Example 1 of the present invention.

[0034] Figure 4 It is a structural diagram of a Bluetooth beacon detection network provided in Example 1 of the present invention.

[0035] Figure 5 It is a structural diagram of a multimodal fusion network provided in Example 1 of the present invention.

[0036] Figure 6 This is an example diagram of AR navigation provided by Embodiment 1 of the present invention;

[0037] Figure 7 It is a structural schematic diagram of a device for performing Bluetooth navigation in virtual reality provided by Embodiment 2 of the present invention.

[0038] Figure 8 It is a structural diagram of a mobile device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can cover sequential implementations other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] Embodiment 1

[0042] See also Figure 1 , shows a flow chart of a method for performing Bluetooth navigation in virtual reality provided by Embodiment 1 of the present invention, the method can be executed by a device for performing Bluetooth navigation in virtual reality, the device for performing Bluetooth navigation in virtual reality can be implemented in the form of hardware and / or software, and the device for performing Bluetooth navigation in virtual reality can be applied to mobile devices, such as mobile phones, smart watches, smart glasses, etc. Figure 1 As shown, the method includes:

[0043] Step 101: If a request for navigating to a destination in an indoor environment is received, a plurality of frames of image data and a plurality of frames of Bluetooth beacon signals are simultaneously collected in the indoor environment.

[0044] In actual applications, mobile devices are equipped with sensors such as cameras, Bluetooth modules, gyroscopes, etc. Operating systems such as Android, iOS, and Harmony can be installed in mobile devices. Various applications and their applets can be installed in these operating systems, such as shopping mall applets, map applications, games, social applications, shopping applications, and so on.

[0045] When the user arrives at an indoor environment such as a shopping mall, he or she starts the application or its mini-program on the mobile terminal and selects the destination of the indoor environment, such as a store, toilet, elevator, etc., thereby triggering a request to navigate to the destination in the indoor environment. At this time, the application or its mini-program can generate prompt information to guide the user to stabilize the mobile device, reduce jitter, and face the camera towards the building.

[0046] Then, the application or its applet can simultaneously turn on the camera, Bluetooth module and gyroscope, call the camera to collect multiple frames of image data, call the Bluetooth module to collect multiple frames of Bluetooth beacon signals, and call the gyroscope to collect multiple frames of angular velocity signals.

[0047] Each frame of the Bluetooth beacon signal has a received signal strength indication RSSI.

[0048] Step 102 , determining whether the multi-frame image data meets the conditions for navigation in an indoor environment; if so, executing step 103 .

[0049] In this embodiment, an indoor navigation service based on AR (Augmented Reality) is provided to users, but the computing power of mobile devices is relatively limited. Factors such as blurring of image data due to jitter and fewer building features due to improper holding of users will affect the effect of navigation in indoor environments. Therefore, the information of the image data itself or external information can be used to determine whether multi-frame image data meets the conditions for navigation in indoor environments, thereby ensuring user experience while reducing the consumption of mobile device resources.

[0050] When the conditions for navigation in an indoor environment are met, an AR-based indoor navigation service is provided based on multi-frame image data.

[0051] When the conditions for navigating in an indoor environment are not met, the AR-based indoor navigation service may be suspended, prompting the user to adjust the mobile device so as to meet the conditions for navigating in the indoor environment.

[0052] In a specific implementation, the angular velocity signal recorded when each frame of image data is collected can be queried, and the angular velocity signal can be processed by Kalman filtering to obtain the posture of the mobile device.

[0053] The posture of the mobile device can be represented by a quaternion, and the rotation can be represented by four parameters.

[0054] Taking the Extended Kalman Filter (EKF) as an example, the steps of EKF include the following two stages:

[0055] 1. Prediction stage: Use the posture (estimated value) of the previous moment and the system model to predict the posture of the current moment.

[0056] 2. Update phase: Use the gyroscope's angular velocity signal (measured value) at the current moment to update the attitude (estimated value).

[0057] Among them, the system model for calculating the posture of moving the device can be expressed as: k =f(x k-1 ,u k )+w k , z k =h(x k )+v k .

[0058] Among them, x k is the posture at time k, f(x k-1 , u k ) is the state transfer function, which describes the posture x at the previous moment k-1 To the current moment posture x k The change relationship of u k is the angular velocity signal of the gyroscope, w k is process noise, representing the random error in the state transfer process, z k is the angular velocity signal (measured value) of the gyroscope at time k.

[0059] h(x k ) is the measurement function, which describes the current posture x k The angular velocity signal (measured value) z of the gyroscope k The corresponding relationship between k is the measurement noise, which represents the random error in the gyroscope measurement process.

[0060] Furthermore, the EKF processing of the angular velocity signal includes the following steps:

[0061] S1. Initialization: Initialize the state estimate x0 and the covariance matrix P0.

[0062] S2, prediction stage: using the state transfer function f(x k-1 ,u k ) Calculate the predicted state x hat {k|k-1}; using the Jacobian matrix F of the state transfer function k Calculate the prediction covariance matrix P hat {k|k-1}.

[0063] S3, Update phase: Calculate Kalman gain K k ; Using the measurement function h(x k ) and the measured value z k Calculate measurement residuals; update state estimate x hat {k|k}; Update the covariance matrix P{k|k}.

[0064] Calculate the difference between the current posture and the previous posture to get the current deviation.

[0065] The posture at the current moment is compared with a preset range, and the movement amplitude at the current moment is compared with a preset first threshold, wherein the range represents the posture of the mobile phone as the camera is facing horizontally and in a forward shooting state, not shooting from above or below. In this way, the image data contains more features of buildings (such as roads, shops, etc.). The first threshold is used to divide the jitter state of the mobile device. When it is less than or equal to the first threshold, the jitter state of the mobile device is lower and the image data is clearer. When it is greater than the first threshold, the jitter state of the mobile device is higher and the image data is blurred.

[0066] If the current posture is within a preset range, and the offset amplitude is less than or equal to a preset first threshold, it is determined that the current frame image data meets the conditions for navigation in an indoor environment.

[0067] If the current posture is outside a preset range, and / or the offset amplitude is greater than a preset first threshold, it is determined that the current frame image data does not meet the conditions for navigation in an indoor environment.

[0068] Step 103: extract target building features from the current frame image data.

[0069] In practical applications, deep learning technology or machine learning technology can be used to extract the features of surrounding buildings from the current frame image data and record them as target building features.

[0070] In indoor environments such as shopping malls, the degree of commercialization is high and the decoration of individual buildings (such as stores) varies greatly, making the characteristics of the buildings useful as auxiliary anchor points for positioning.

[0071] In a specific implementation, a building detection network can be loaded for operation, wherein the building detection network is trained using a semantic segmentation model so that the building detection network has the ability to extract target building features from image data.

[0072] During semantic segmentation training, the building detection network is used as the encoder and a decoder is configured for it. The encoder is responsible for extracting the target building features from the image data, and the decoder is responsible for identifying the semantics of each pixel (building or non-building) based on the target building. The encoder (i.e., the building detection network) and the decoder are updated based on the loss value between the image data and the semantics (such as cross entropy, Dice loss, etc.). When the training is completed, the decoder is discarded and the encoder (i.e., the building detection network) is retained.

[0073] like Figure 2 As shown, the building detection network includes a first convolution module ConvModule_1, a second convolution module ConvModule_2, a third convolution module ConvModule_3 and a spatial pyramid pooling layer (Spatial PyramidPooling, SPP).

[0074] Furthermore, if Figure 3 As shown, the first convolution module ConvModule_1, the second convolution module ConvModule_2 and the third convolution module ConvModule_3 respectively include a convolutional layer (Convolutional Layer, Conv), a batch normalization layer (Batch Normalization, BN) and a linear rectification layer (Rectified Linear Unit, ReLU). The convolution layer can provide convolution operations, the batch normalization layer can provide normalization operations, and the linear rectification layer can provide activation operations.

[0075] The current frame image data is input into the first convolution module ConvModule_1 for processing (ie, the convolution operation, the normalization operation and the activation operation are performed in sequence), and the first candidate building feature is extracted from the current frame image data.

[0076] The first candidate building feature is input into the second convolution module ConvModule_2 for processing (i.e., convolution operation, normalization operation and activation operation are performed in sequence), and the second candidate building feature is extracted from the first candidate building feature.

[0077] The Concatnate function is used to concatenate the first candidate building feature and the second candidate building feature into a third candidate building feature.

[0078] The third candidate building feature is input into the third convolution module ConvModule_3 for processing (ie, the convolution operation, the normalization operation and the activation operation are performed in sequence), and the fourth candidate building feature is extracted from the third candidate building feature.

[0079] The fourth candidate building feature is input into the spatial pyramid pooling layer SPP to perform a pooling operation to obtain the target building feature.

[0080] Step 104: extract multiple target Bluetooth beacon features from the multi-frame received signal strength indication.

[0081] In practical applications, deep learning technology or machine learning technology can be used to extract features in different dimensions from multi-frame received signal strength indications to obtain multiple target Bluetooth beacon features.

[0082] Multiple RSSI target Bluetooth beacon features can increase the information content of the features and improve the differentiation of RSSI features, so that the RSSI target Bluetooth beacon features can be used as the main anchor point for positioning.

[0083] In one embodiment of the present invention, the target Bluetooth beacon features include target beacon classification features and target beacon enhancement features. Therefore, in this embodiment, step 104 may include the following steps:

[0084] Step 1041: Load the Bluetooth beacon detection network.

[0085] In this embodiment, a Bluetooth beacon detection network can be loaded and run, wherein the Bluetooth beacon detection network is trained using a classification mode so that the Bluetooth beacon detection network has the ability to extract multiple target Bluetooth beacon features from multi-frame received signal strength indications.

[0086] Among them, Figure 4 As shown, the Bluetooth beacon detection network includes a first encoder Encoder_1, a second encoder Encoder_2 and a decoder Decoder. The first encoder Encoder_1 is responsible for extracting features from multi-frame received signal strength indications from the perspective of multi-classification of base stations. The second encoder Encoder_2 is responsible for extracting features from multi-frame received signal strength indications from the perspective of binary classification of base station ranges, thereby enhancing the features of the first encoder Encoder_1. The decoder Decoder is responsible for integrating the features of the first encoder Encoder_1 with the features of the second encoder Encoder_2.

[0087] During classification training, the first fully connected layer is cascaded after the second encoder Encoder_2, and the second fully connected layer is cascaded after the decoder Decoder to form a beacon classification network.

[0088] The first fully connected layer is responsible for performing binary classification on the features of the second encoder Encoder_2 (i.e., whether the base station to which the Bluetooth beacon signal belongs is within a range formed by a specified radius centered on the mobile device), and generates a first loss value (such as cross entropy, etc.) based on the classification result of the first fully connected layer.

[0089] The second fully connected layer is responsible for multi-classification of the features in the decoder (i.e., detecting the base station to which the Bluetooth beacon signal belongs), and generating a second loss value (such as cross entropy, etc.) based on the classification results of the second fully connected layer.

[0090] The first loss value and the second loss value are linearly fused into a third loss value, and the beacon classification network is updated according to the third loss value. When the training is completed, the first fully connected layer and the second fully connected layer are discarded, and the first encoder Encoder_1, the second encoder Encoder_2 and the decoder Decoder are retained.

[0091] Step 1042: Input the multi-frame received signal strength indication into the first encoder and encode it into the target beacon classification feature.

[0092] In this embodiment, if Figure 4 As shown, starting from the current moment, the RSSI of T (T is a positive integer) frames of Bluetooth beacon signals are screened and input into the first encoder Encoder_1 to be encoded as target beacon classification features, and the target beacon classification features are used to detect the base station to which the Bluetooth beacon signal belongs.

[0093] In the specific implementation, Figure 4 As shown, the first encoder Encoder_1 includes a first time delay neural network TDNN_1, a fourth convolution module ConvModule_4 and a fifth convolution module ConvModule_5.

[0094] Among them, the first time delay neural network TDNN_1 belongs to the time delay neural network (TDNN), which introduces multiple delayed RSSIs at each time step. These delayed RSSIs are combined together to form a new feature, which can capture the correlation of RSSI at different time points.

[0095] like Figure 3 As shown, the fourth convolution module ConvModule_4 and the fifth convolution module ConvModule_5 each include a convolution layer Conv, a batch normalization layer BN and a linear rectification layer ReLU in sequence.

[0096] Then, the multi-frame received signal strength indication RSSI is input into the first time delay neural network TDNN_1 and converted into a three-dimensional first candidate beacon classification feature.

[0097] The first candidate beacon classification feature is input into the fourth convolution module ConvModule_4 for processing (ie, the convolution operation, the normalization operation and the activation operation are performed in sequence), and the second candidate beacon classification feature is extracted from the first candidate beacon classification feature.

[0098] The second candidate beacon classification features are input into the fifth convolution module ConvModule_5 for processing (ie, convolution operation, normalization operation and activation operation are performed in sequence), and the target beacon classification features are extracted from the second candidate beacon classification features.

[0099] Step 1043: Input the multi-frame received signal strength indication into the second encoder and encode it into the target beacon range feature.

[0100] In this embodiment, if Figure 4 As shown, starting from the current moment, the RSSI of T (T is a positive integer) frames of Bluetooth beacon signals are filtered out and input into the second encoder Encoder_2 to be encoded as a target beacon range feature. The target beacon range feature is used to detect whether the base station to which the Bluetooth beacon signal belongs is within a range formed by a specified radius centered on the mobile device.

[0101] Generally, the RSSI of a Bluetooth beacon signal with a closer distance and a stronger signal is used for positioning. However, in a real indoor environment, there is interference from wireless signals such as WiFi (Wireless Fidelity), which causes the RSSI of the Bluetooth beacon signal to fluctuate. In this embodiment, a set of features is constructed based on whether the RSSI of the Bluetooth beacon signal is within the surrounding range. This set of features is relatively simple and has strong stability, and can achieve the effect of activation (i.e., enhancing the features of the RSSI of the Bluetooth beacon signal in the surrounding range and suppressing the features of the RSSI of the Bluetooth beacon signal in the distant range), and the features of the RSSI of the original Bluetooth beacon signal used for classification are enhanced to enhance robustness.

[0102] In the specific implementation, Figure 4 As shown, the second encoder Encoder_2 includes a second time delay neural network TDNN_2, a sixth convolution module ConvModule_6 and a seventh convolution module ConvModule_7.

[0103] Among them, Figure 3 As shown, the sixth convolution module ConvModule_6 and the seventh convolution module ConvModule_7 each include a convolution layer Conv, a batch normalization layer BN and a linear rectification layer ReLU in sequence.

[0104] Then, the multi-frame received signal strength indication RSSI is input into the second time delay neural network TDNN_2 and converted into a three-dimensional first candidate beacon range feature.

[0105] The first candidate beacon range feature is input into the sixth convolution module ConvModule_6 for processing (ie, convolution operation, normalization operation and activation operation are performed in sequence), and the second candidate beacon range feature is extracted from the first candidate beacon range feature.

[0106] The second candidate beacon range feature is input into the seventh convolution module ConvModule_7 for processing (i.e., convolution operation, normalization operation and activation operation are performed in sequence), and the target beacon range feature is extracted from the second candidate beacon range feature.

[0107] Step 1044: concatenate the target beacon classification feature and the target beacon range feature into a beacon fusion feature.

[0108] In this embodiment, the Concatnate function can be used to concatenate the target beacon classification feature and the target beacon range feature into a beacon fusion feature.

[0109] Step 1045: Input the beacon fusion feature into the decoder and decode it into the target beacon enhanced feature.

[0110] In this embodiment, if Figure 4 As shown, the beacon fusion features are input into the decoder Encoder for decoding to obtain the target beacon enhanced features.

[0111] In the specific implementation, Figure 4 As shown, the decoder Encoder includes the eighth convolution module ConvModule_8 and a multi-head attention module (Multi Head Attention, MHA).

[0112] Among them, Figure 3 As shown, the eighth convolution module ConvModule_8 includes a convolution layer Conv, a batch normalization layer BN and a linear rectification layer ReLU in sequence.

[0113] Then, the beacon fusion features are input into the eighth convolution module ConvModule_8 for processing (i.e., convolution operation, normalization operation and activation operation are performed in sequence), and candidate beacon enhancement features are extracted from the beacon fusion features.

[0114] The candidate beacon enhancement features are input into the multi-head attention module MHA to fuse the context information of the candidate beacon enhancement features to extract the target beacon enhancement features.

[0115] Step 105: Merge the target building features with multiple target Bluetooth beacon features into a target fingerprint feature.

[0116] In practical applications, deep learning technology or machine learning technology can be used to fuse the target building features with multiple target Bluetooth beacon features to obtain the target fingerprint features.

[0117] In the specific implementation, a multimodal fusion network can be loaded for operation. The multimodal fusion network is trained using a classification model, so that the multimodal fusion network has the ability to fuse target building features with multiple target Bluetooth beacon features into target fingerprint features.

[0118] During classification training, a fully connected layer is cascaded after the multimodal fusion network to form a fingerprint classification network. The fully connected layer is responsible for multi-classification of the features of the multimodal fusion network (i.e., detecting the category of the target fingerprint features). A loss value (such as cross entropy) is generated based on the classification result of the fully connected layer, and the fingerprint classification network is updated based on the loss value. When the training is completed, the fully connected layer is discarded and the multimodal fusion network is retained.

[0119] When constructing the fingerprint library offline, image data can be collected at different angles at the same position, or panoramic image data can be collected using equipment such as a fisheye camera, and image data of multiple areas can be cropped from the panoramic image data, that is, multiple groups of image data can be formed at the same position. This embodiment decouples the building detection network, the Bluetooth beacon detection network, and the multimodal fusion network, that is, the structures are separated and trained independently, so that the building detection network can independently output the features of each group of image data, and the features of the RSSI output by the Bluetooth beacon detection network can be fused with the features of any group of image data in the multimodal fusion network into a fingerprint, thereby improving the flexibility of fingerprint modeling.

[0120] The feature engineering of image data and the feature engineering of RSSI can be executed in parallel. The execution frequency can be adjusted according to the confidence of image data and RSSI for positioning, which can effectively improve the real-time performance.

[0121] In indoor environments such as shopping malls, if a building (such as a store) is renovated, the fingerprint database can be updated by re-collecting image data near the building, effectively reducing the maintenance cost of the fingerprint database.

[0122] Among them, Figure 5 As shown, the multimodal fusion network includes a first self-attention module Self-Atteintion_1 and a second self-attention module Self-Atteintion_2.

[0123] Then, the Concatnate function can be used to concatenate the target building feature E1 and the target beacon classification feature E2 into the first candidate fingerprint feature.

[0124] The first candidate fingerprint feature is input into the first self-attention module Self-Atteintion_1 to extract the second candidate fingerprint feature.

[0125] The Add function can be used to add the second candidate fingerprint feature and the target beacon enhanced feature E3 to obtain the third candidate fingerprint feature.

[0126] The third candidate fingerprint feature is input into the second self-attention module Self-Atteintion_2 to extract the target fingerprint feature.

[0127] On the one hand, RSSI is a long time series, and the self-attention mechanism can capture long-distance dependencies to better model the global information in multimodal data (i.e., the characteristics of the building and the characteristics of RSSI).

[0128] On the other hand, buildings of different structures block Bluetooth beacon signals. Therefore, the characteristics of buildings (i.e., target building characteristics) can not only serve as auxiliary anchor points for positioning, but also enhance the characteristics of RSSI (i.e., multiple target Bluetooth beacon characteristics).

[0129] The self-attention mechanism is used to capture the dependency between buildings and RSSI, and dynamically classify the weights of building features and RSSI features to fuse high-quality features and enhance the robustness of the system.

[0130] Step 106: Detect the real-time position of the mobile device in the indoor environment based on the target fingerprint feature.

[0131] In this embodiment, the target fingerprint feature may be used for positioning to detect the real-time position of the mobile device in the indoor environment.

[0132] In a specific implementation, a fingerprint library configured for an indoor environment is loaded; the fingerprint library has reference fingerprint features formulated at various reference positions in the indoor environment, and the method of constructing the reference fingerprint features is the same as the method of constructing the target fingerprint features.

[0133] At this time, the similarity between the target fingerprint feature and each reference fingerprint feature is calculated, such as cosine similarity.

[0134] The similarities of the reference fingerprint features are compared. If a reference fingerprint feature has the highest similarity, the reference position corresponding to the reference fingerprint feature is marked as the real-time position of the mobile device in the indoor environment.

[0135] Step 107: Display a virtual route from the real-time location to the destination in the image data in a virtual reality manner.

[0136] When determining the real-time location of the user in an indoor environment, the A* algorithm, Dijkstra algorithm and other algorithms can be used to plan the optimal path from the real-time location to the destination, and the SLAM (simultaneous localization and mapping) technology can be used to align the path with the coordinates of the real environment (i.e., the content of the image data). Figure 6 As shown, according to the camera's viewing angle and the user's position, the path is projected onto the plane of the image data, and the path is represented by geometric figures 601 (such as arrows, line segments), etc., and supplemented by dynamic effects (such as flashing arrows, etc.) and navigation prompt information 602 (such as "turn left", "go straight", etc.), thereby generating a virtual route and realizing AR navigation.

[0137] In a specific implementation, the offset amplitude may be greater than a preset second threshold for comparison, wherein the second threshold is smaller than the first threshold.

[0138] If the offset amplitude is greater than a preset second threshold, it means that the mobile device has a certain jitter, which has a certain impact on the AR navigation. Then, the difference between the current posture and the previous posture is calculated to obtain the offset direction.

[0139] Load the view window View in the current frame image data.

[0140] In the view window, the switching animation of the virtual route is played along the offset direction in a virtual reality manner to achieve a smooth transition of the virtual route and improve the user experience.

[0141] If the switching animation is finished, a virtual route from the real-time position to the destination is displayed in a virtual reality manner in the view window.

[0142] In this embodiment, if a request to navigate to a destination in an indoor environment is received, multiple frames of image data and multiple frames of Bluetooth beacon signals are collected simultaneously in the indoor environment; the Bluetooth beacon signal has a received signal strength indication; it is determined whether the multiple frames of image data meet the conditions for navigation in the indoor environment; if so, the target building features are extracted from the current frame image data; multiple target Bluetooth beacon features are extracted from the multiple frames of received signal strength indication; the target building features and multiple target Bluetooth beacon features are merged into a target fingerprint feature; the real-time position of the mobile device in the indoor environment is detected based on the target fingerprint feature; and a virtual route from the real-time position to the destination is displayed in the image data in a virtual reality manner. This embodiment reuses the features of the building in the image data to enhance the features of the RSSI, increase the amount of information of the features, reduce the impact on the features when the indoor environment changes and there is interference, improve the accuracy of positioning, and thus improve the accuracy of AR navigation.

[0143] Embodiment 2

[0144] See also Figure 7, shows a schematic diagram of the structure of a device for performing Bluetooth navigation in virtual reality provided by Embodiment 2 of the present invention. Figure 7 As shown, applied to a mobile device, the device includes:

[0145] The environment data acquisition module 701 is used to simultaneously acquire multiple frames of image data and multiple frames of Bluetooth beacon signals in the indoor environment if a request for navigating to a destination in the indoor environment is received; the Bluetooth beacon signal has a received signal strength indication;

[0146] The navigation condition judgment module 702 is used to judge whether the image data of the plurality of frames meets the conditions for navigation in the indoor environment; if so, the building feature extraction module 703 is called;

[0147] Building feature extraction module 703, used to extract target building features from the image data of the current frame;

[0148] A Bluetooth beacon feature extraction module 704 is used to extract multiple target Bluetooth beacon features from the received signal strength indication of multiple frames;

[0149] The fingerprint feature fusion module 705 is used to fuse the target building feature with the multiple target Bluetooth beacon features into a target fingerprint feature;

[0150] A real-time location detection module 706, configured to detect the real-time location of the mobile device in the indoor environment according to the target fingerprint feature;

[0151] The virtual route display module 707 is used to display a virtual route from the real-time location to the destination in the image data in a virtual reality manner.

[0152] In one embodiment of the present invention, the navigation condition determination module 702 is further used to:

[0153] Querying the angular velocity signal recorded when collecting the image data of each frame;

[0154] Performing Kalman filtering on the angular velocity signal to obtain the posture of the mobile device;

[0155] Calculate and collect the difference between the current posture and the previous posture to obtain the offset amplitude;

[0156] If the currently acquired posture is within a preset range, and the offset amplitude is less than or equal to a preset first threshold, it is determined that the image data of the current frame meets the conditions for navigation in the indoor environment;

[0157] If the currently acquired posture is outside a preset range, and / or the offset amplitude is greater than a preset first threshold, it is determined that the image data of the current frame does not meet the conditions for navigation in the indoor environment;

[0158] The virtual route display module 707 is also used for:

[0159] If the offset amplitude is greater than a preset second threshold, the difference between the current posture and the previous posture is calculated to obtain the offset direction; the second threshold is less than the first threshold;

[0160] Loading the view window with the image data of the current frame;

[0161] Playing a switching animation of a virtual route along the offset direction in a virtual reality manner in the viewing window;

[0162] If the switching animation ends, a virtual route from the real-time location to the destination is displayed in the view window in a virtual reality manner.

[0163] In one embodiment of the present invention, the building feature extraction module 703 is further used for:

[0164] Loading a building detection network; the building detection network includes a first convolution module, a second convolution module, a third convolution module and a spatial pyramid pooling layer; the first convolution module, the second convolution module and the third convolution module each include a convolution layer, a batch normalization layer and a linear rectification layer in sequence;

[0165] Inputting the image data of the current frame into the first convolution module to extract the first candidate building feature;

[0166] Inputting the first candidate building feature into the second convolution module to extract the second candidate building feature;

[0167] splicing the first candidate building feature and the second candidate building feature into a third candidate building feature;

[0168] Inputting the third candidate building feature into the third convolution module to extract a fourth candidate building feature;

[0169] The fourth candidate building feature is input into the spatial pyramid pooling layer to perform a pooling operation to obtain a target building feature.

[0170] In one embodiment of the present invention, the target Bluetooth beacon characteristics include target beacon classification characteristics and target beacon enhancement characteristics;

[0171] The Bluetooth beacon feature extraction module 704 is also used for:

[0172] Loading a Bluetooth beacon detection network; the Bluetooth beacon detection network includes a first encoder, a second encoder and a decoder;

[0173] Inputting the received signal strength indication of multiple frames into the first encoder and encoding them into a target beacon classification feature; the target beacon classification feature is used to detect the base station to which the Bluetooth beacon signal belongs;

[0174] Inputting the received signal strength indication of multiple frames into the second encoder and encoding them into a target beacon range feature; the target beacon range feature is used to detect whether the base station to which the Bluetooth beacon signal belongs is within a range formed by a specified radius with the mobile device as the center;

[0175] splicing the target beacon classification feature and the target beacon range feature into a beacon fusion feature;

[0176] The beacon fusion feature is input into the decoder and decoded into a target beacon enhanced feature.

[0177] In one embodiment of the present invention, the first encoder includes a first time-delay neural network, a fourth convolution module and a fifth convolution module, and the second encoder includes a second time-delay neural network, a sixth convolution module and a seventh convolution module; the fourth convolution module, the fifth convolution module, the sixth convolution module and the seventh convolution module all include a convolution layer, a batch normalization layer and a linear rectification layer in sequence;

[0178] The Bluetooth beacon feature extraction module 704 is also used for:

[0179] Inputting the received signal strength indications of multiple frames into the first time-delay neural network and converting them into three-dimensional first candidate beacon classification features;

[0180] Inputting the first candidate beacon classification feature into the fourth convolution module to extract the second candidate beacon classification feature;

[0181] Inputting the second candidate beacon classification feature into the fifth convolution module to extract the target beacon classification feature;

[0182] The Bluetooth beacon feature extraction module 704 is also used for:

[0183] Inputting the received signal strength indication of multiple frames into the second time-delay neural network and converting it into a three-dimensional first candidate beacon range feature;

[0184] Inputting the first candidate beacon range feature into the sixth convolution module to extract the second candidate beacon range feature;

[0185] The second candidate beacon range feature is input into the seventh convolution module to extract the target beacon range feature.

[0186] In one embodiment of the present invention, the decoder includes an eighth convolution module and a multi-head attention module; the eighth convolution module includes a convolution layer, a batch normalization layer and a linear rectification layer in sequence;

[0187] The Bluetooth beacon feature extraction module 704 is also used for:

[0188] Inputting the beacon fusion features into the eighth convolution module to extract candidate beacon enhancement features;

[0189] The candidate beacon enhancement features are input into the multi-head attention module to extract the target beacon enhancement features.

[0190] In one embodiment of the present invention, the fingerprint feature fusion module 705 is further used for:

[0191] Loading a multimodal fusion network; the multimodal fusion network includes a first self-attention module and a second self-attention module;

[0192] splicing the target building feature and the target beacon classification feature into a first candidate fingerprint feature;

[0193] Inputting the first candidate fingerprint feature into the first self-attention module to extract a second candidate fingerprint feature;

[0194] Adding the second candidate fingerprint feature to the target beacon enhanced feature to obtain a third candidate fingerprint feature;

[0195] The third candidate fingerprint feature is input into the second self-attention module to extract the target fingerprint feature.

[0196] In one embodiment of the present invention, the real-time location detection module 706 is further used for:

[0197] Loading a fingerprint library configured for the indoor environment; the fingerprint library has reference fingerprint features established at various reference positions in the indoor environment;

[0198] Calculating the similarity between the target fingerprint feature and each of the reference fingerprint features;

[0199] If the similarity of a certain reference fingerprint feature is the highest, the reference position corresponding to the reference fingerprint feature is marked as the real-time position of the mobile device in the indoor environment.

[0200] The device for performing Bluetooth navigation in virtual reality provided by the embodiment of the present invention can execute the method for performing Bluetooth navigation in virtual reality provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the method for performing Bluetooth navigation in virtual reality.

[0201] Embodiment 3

[0202] See also Figure 8 , shows a schematic diagram of the structure of a mobile device provided by an embodiment of the present invention. The mobile device is intended to represent various forms of digital computers, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0203] like Figure 8 As shown, the mobile device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the mobile device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0204] A number of components in the mobile device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the mobile device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0205] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for performing Bluetooth navigation in virtual reality.

[0206] In some embodiments, the method for performing Bluetooth navigation in virtual reality may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the mobile device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for performing Bluetooth navigation in virtual reality described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the method for performing Bluetooth navigation in virtual reality in any other appropriate manner (e.g., by means of firmware).

[0207] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0208] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0209] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0210] To provide interaction with a user, the systems and techniques described herein may be implemented on a mobile device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the mobile device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0211] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0212] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0213] Embodiment 4

[0214] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for performing Bluetooth navigation in virtual reality as provided in any embodiment of the present invention is implemented.

[0215] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet using an Internet service provider).

[0216] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0217] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for performing Bluetooth navigation in virtual reality, characterized in that: Applied to a mobile device, the method comprises: If a request for navigating to a destination in an indoor environment is received, a plurality of frames of image data and a plurality of frames of Bluetooth beacon signals are simultaneously collected in the indoor environment; the Bluetooth beacon signals have a received signal strength indication; Determine whether the image data of the multiple frames meet the conditions for navigation in the indoor environment; if so, extract the target building features from the image data of the current frame; Extracting multiple target Bluetooth beacon features from the received signal strength indications of multiple frames; Merging the target building feature with multiple target Bluetooth beacon features into a target fingerprint feature; Detecting the real-time location of the mobile device in the indoor environment according to the target fingerprint feature; A virtual route from the real-time location to the destination is displayed in the image data in a virtual reality manner.

2. The method according to claim 1, characterized in that The determining whether the multiple frames of image data satisfy the conditions for navigation in the indoor environment includes: Querying the angular velocity signal recorded when collecting the image data of each frame; Performing Kalman filtering on the angular velocity signal to obtain the posture of the mobile device; Calculate and collect the difference between the current posture and the previous posture to obtain the offset amplitude; If the currently acquired posture is within a preset range, and the offset amplitude is less than or equal to a preset first threshold, it is determined that the image data of the current frame meets the conditions for navigation in the indoor environment; If the currently acquired posture is outside a preset range, and / or the offset amplitude is greater than a preset first threshold, it is determined that the image data of the current frame does not meet the conditions for navigation in the indoor environment; The step of displaying a virtual route from the real-time location to the destination in the image data in a virtual reality manner comprises: If the offset amplitude is greater than a preset second threshold, the difference between the current posture and the previous posture is calculated to obtain the offset direction; the second threshold is less than the first threshold; Loading the view window with the image data of the current frame; Playing a switching animation of a virtual route along the offset direction in a virtual reality manner in the viewing window; If the switching animation ends, a virtual route from the real-time location to the destination is displayed in the view window in a virtual reality manner.

3. The method according to claim 1, characterized in that The step of extracting target building features from the image data of the current frame includes: Loading a building detection network; the building detection network includes a first convolution module, a second convolution module, a third convolution module and a spatial pyramid pooling layer; the first convolution module, the second convolution module and the third convolution module each include a convolution layer, a batch normalization layer and a linear rectification layer in sequence; Inputting the image data of the current frame into the first convolution module to extract the first candidate building feature; Inputting the first candidate building feature into the second convolution module to extract the second candidate building feature; splicing the first candidate building feature and the second candidate building feature into a third candidate building feature; Inputting the third candidate building feature into the third convolution module to extract a fourth candidate building feature; The fourth candidate building feature is input into the spatial pyramid pooling layer to perform a pooling operation to obtain a target building feature.

4. The method according to claim 3, characterized in that The target Bluetooth beacon features include target beacon classification features and target beacon enhancement features; The extracting multiple target Bluetooth beacon features from the received signal strength indication of multiple frames includes: Loading a Bluetooth beacon detection network; the Bluetooth beacon detection network includes a first encoder, a second encoder and a decoder; Inputting the received signal strength indication of multiple frames into the first encoder and encoding them into a target beacon classification feature; the target beacon classification feature is used to detect the base station to which the Bluetooth beacon signal belongs; Inputting the received signal strength indication of multiple frames into the second encoder and encoding them into a target beacon range feature; the target beacon range feature is used to detect whether the base station to which the Bluetooth beacon signal belongs is within a range formed by a specified radius with the mobile device as the center; splicing the target beacon classification feature and the target beacon range feature into a beacon fusion feature; The beacon fusion feature is input into the decoder and decoded into a target beacon enhanced feature.

5. The method according to claim 4, characterized in that The first encoder includes a first time-delay neural network, a fourth convolution module and a fifth convolution module, and the second encoder includes a second time-delay neural network, a sixth convolution module and a seventh convolution module; the fourth convolution module, the fifth convolution module, the sixth convolution module and the seventh convolution module all include a convolution layer, a batch normalization layer and a linear rectification layer in sequence; The step of inputting the received signal strength indication of multiple frames into the first encoder and encoding them into target beacon classification features comprises: Inputting the received signal strength indications of multiple frames into the first time-delay neural network and converting them into three-dimensional first candidate beacon classification features; Inputting the first candidate beacon classification feature into the fourth convolution module to extract the second candidate beacon classification feature; Inputting the second candidate beacon classification feature into the fifth convolution module to extract the target beacon classification feature; The step of inputting the received signal strength indication of multiple frames into the second encoder and encoding them into a target beacon range feature comprises: Inputting the received signal strength indication of multiple frames into the second time-delay neural network and converting it into a three-dimensional first candidate beacon range feature; Inputting the first candidate beacon range feature into the sixth convolution module to extract the second candidate beacon range feature; The second candidate beacon range feature is input into the seventh convolution module to extract the target beacon range feature.

6. The method according to claim 5, characterized in that The decoder includes an eighth convolution module and a multi-head attention module; the eighth convolution module includes a convolution layer, a batch normalization layer and a linear rectification layer in sequence; The step of inputting the beacon fusion feature into the decoder and decoding it into a target beacon enhancement feature comprises: Inputting the beacon fusion features into the eighth convolution module to extract candidate beacon enhancement features; The candidate beacon enhancement features are input into the multi-head attention module to extract the target beacon enhancement features.

7. The method according to claim 4, characterized in that The step of fusing the target building feature with the multiple target Bluetooth beacon features into a target fingerprint feature includes: Loading a multimodal fusion network; the multimodal fusion network includes a first self-attention module and a second self-attention module; splicing the target building feature and the target beacon classification feature into a first candidate fingerprint feature; Inputting the first candidate fingerprint feature into the first self-attention module to extract a second candidate fingerprint feature; Adding the second candidate fingerprint feature to the target beacon enhanced feature to obtain a third candidate fingerprint feature; The third candidate fingerprint feature is input into the second self-attention module to extract the target fingerprint feature.

8. The method according to any one of claims 1 to 7, characterized in that The detecting the real-time position of the mobile device in the indoor environment according to the target fingerprint feature includes: Loading a fingerprint library configured for the indoor environment; the fingerprint library has reference fingerprint features established at various reference positions in the indoor environment; Calculating the similarity between the target fingerprint feature and each of the reference fingerprint features; If the similarity of a certain reference fingerprint feature is the highest, the reference position corresponding to the reference fingerprint feature is marked as the real-time position of the mobile device in the indoor environment.

9. A mobile device, characterized in that: The mobile device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for performing Bluetooth navigation in virtual reality according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for performing Bluetooth navigation in virtual reality according to any one of claims 1 to 8 is implemented.

Images