A museum intelligent tour guide method and system

Abstract

The application discloses a kind of museum intelligent tour method and system, belong to indoor high-precision positioning and intelligent tour technical field.System includes mobile positioning terminal, UWB positioning base station, bluetooth AOA positioning base station.Mobile positioning terminal integrates UWB, bluetooth AOA, IMU sensor, to realize multi-source fusion with EKF extended Kalman filter;To UWB observation do multi-mode NLOS detection and suppression, bluetooth AOA angle uses distance adaptive weighting, and online compensates IMU zero offset, solves museum metal, glass environment under positioning jump drift, angle error amplification, inertial error cumulative divergence problem.Meanwhile, position-direction-separation wall-stay time four-dimensional joint fence determination is adopted, to eliminate the explanation mis-trigger caused by separation wall, back to, passing by.All positioning and trigger logic is completed in terminal local, without edge server, positioning accuracy is high, robustness is good, deployment cost is low, applicable to complex museum venue.

Description

Technical Field

[0001] This invention relates to the field of indoor high-precision positioning and intelligent museum navigation technology, and belongs to the field of intelligent museum navigation method and system. Background Technology

[0002] The user experience of smart tour guide devices in museums is highly dependent on indoor positioning accuracy. The interior of military museum exhibition halls contains numerous steel weapons, metal exhibits, and thick glass display cases, making wireless signals highly susceptible to reflection, absorption, and blockage, resulting in severe non-line-of-sight (NLOS) interference.

[0003] Existing technologies have many shortcomings: pure UWB positioning suffers from the NLOS effect in dense metal-glass environments, resulting in positive ranging offset and positioning coordinate jumps and drifts; pure Bluetooth AOA positioning angle error amplifies linearly with distance, and there is a blind spot below the base station; pure IMU dead reckoning lacks external absolute observation correction, and the position error of low-cost MEMS devices diverges rapidly in a short time; traditional geofencing relies solely on RSSI or distance thresholds for triggering, and cannot obtain information on visitor posture and wall occlusion, leading to frequent triggering when passing through walls, when facing away from exhibits, or when passing by unintentionally.

[0004] Existing UWB+IMU combination solutions lack Bluetooth AOA angle redundancy, and there is no backup observation source when UWB is completely blocked; explanation triggers still rely solely on distance judgment, and false triggers cannot be eliminated; many solutions rely on edge servers for positioning calculations, which are complex to deploy and costly to small and medium-sized museums. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A museum intelligent navigation method is based on a mobile terminal, several UWB base stations, and several Bluetooth AOA base stations. The mobile terminal includes a UWB ranging module, a Bluetooth AOA direction finding module, an IMU inertial sensing module, and a navigation processing system. The navigation processing system includes an NLOS detection and suppression module, an EKF fusion filter, an intelligent fence determination module, an audio playback control module, and a distance adaptive weighting unit. The method includes the following steps:

[0007] S1: The UWB ranging module communicates bidirectionally with the UWB positioning base station to obtain UWB distance observation data;

[0008] S2: The Bluetooth AOA direction finding module continuously transmits Bluetooth signals to the AOA positioning base station and then receives Bluetooth AOA angle data sent by the AOA base station based on the Bluetooth signals;

[0009] S3: The IMU inertial sensing module acquires the raw inertial sensing data of the IMU;

[0010] S4: The EKF fusion filter performs prediction stage processing on the raw IMU inertial sensing data to obtain the predicted state, predicted covariance matrix and predicted position.

[0011] S5: The NLOS detection and suppression module performs detection and suppression processing on the original UWB ranging observations in conjunction with the predicted location and UWB base station coordinates to obtain the effective UWB distance observations;

[0012] S6: The distance adaptive weighting unit performs adaptive weighting processing on the Bluetooth AOA angle data in conjunction with the predicted location Bluetooth and AOA base station coordinates;

[0013] S7: The EKF fusion filter performs an update stage on the predicted state, predicted covariance matrix, UWB effective range observation set, and weighted AOA angle observations to obtain the fused 3D position and pose, and the updated IMU zero-bias estimate; IMU zero-bias estimation closed-loop feedback prediction stage:

[0014] S8: The intelligent fence determination module performs a four-dimensional joint determination of position, orientation, wall separation, and dwell time based on the fusion of three-dimensional position and user yaw attitude, and triggers the audio output module to explain the exhibits.

[0015] In S1, the specific steps include:

[0016] S11: The UWB ranging module communicates with the venue's UWB positioning base station;

[0017] S12: The UWB ranging module performs TWR bidirectional ranging; it outputs the raw UWB ranging observation and sends it to the NLOS detection and suppression module;

[0018] In S2, the specific steps include:

[0019] S21: The Bluetooth AOA direction finding module continuously transmits Bluetooth signals with CTE and receives azimuth and elevation data packets sent by the AOA positioning base station;

[0020] S22: The Bluetooth AOA direction finding module analyzes azimuth and elevation data packets and outputs Bluetooth AOA angle observations, inputting the distance adaptive weighting unit.

[0021] S3 includes the following steps:

[0022] S31: Simulated physical motion signal acquired by MEMS sensor;

[0023] S32: Analog-to-digital conversion, outputting raw IMU inertial sensing data, which includes three-axis angular velocity and three-axis acceleration.

[0024] The predicted state vector is defined as follows:

[0025] x = [p] x py, pz, v x , vy , vz ,φ, θ, ψ, bωx, bωy, bωz, bax, bay, baz ]ᵀ

[0026] Where p x , py, pz represent three-dimensional positions; v x , vy, vz represent three-dimensional velocities; φ, θ, ψ represent attitude Euler angles (roll, pitch, yaw); bωx, bωy, bωz represent three-axis zero bias of the gyroscope; bax, bay, baz represent three-axis zero bias of the accelerometer.

[0027] S4 includes the following steps:

[0028] S41: Obtain the estimated three-axis zero bias values ​​of the gyroscope and the zero bias value of the accelerometer through the IMU gyroscope and accelerometer of the previous closed-loop feedback;

[0029] S42: The three-axis angular velocity is subtracted from the zero-bias three-axis gyroscope, and the attitude is updated by quaternion differential equations and converted into Euler angles;

[0030] S43: The three-axis accelerometer is deducted for three-axis zero bias. The coordinate system is transformed to the navigation coordinate system through the attitude rotation matrix R(b→n), the gravity component is removed, and the three-dimensional velocity and three-dimensional position are updated by integration.

[0031] S44: Perform covariance matrix prediction, specifically using the formula P(k|k−1) = F·P(k−1|k−1)·Fᵀ + Q(k); where F is the state transition Jacobian, its value is obtained by substituting the amount of the current IMU raw inertial sensing data processed by S42 and S43 into the Jacobian of the kinematic equations—specifically, the gyroscope three-axis angular velocity ω (after deducting the estimated zero bias b̂_ω and used for attitude update) determines the coupling of the attitude channel to the attitude itself and the instantaneous value of the rotation matrix R(b→n) in F, and the accelerometer three-axis specific force f (after deducting the estimated zero bias b̂_a and rotating through R(b→n), subtracting gravity, and integrating for velocity and position update) determines the coupling of velocity to attitude and position to velocity in F; P(k−1|k−1) is the value of the previous EKF update stage (after UWB distance observation and AOA). The posterior covariance matrix output after angle observation correction is used as the starting point for this prediction; Q(k) is the process noise covariance, the value of which is calibrated according to the noise characteristics of the IMU device—the gyroscope angle random walk (ARW) corresponds to the noise term of the attitude-zero bias channel in Q, the accelerometer velocity random walk (VRW) corresponds to the noise term of the velocity-zero bias channel in Q, and the random walk noise of the gyroscope and accelerometer zero bias corresponds to the noise term of the zero bias diagonal block in Q; the three are propagated through F·P·Fᵀ + Q to obtain the current prediction covariance matrix P(k|k−1), which is used to calculate the Kalman gain in the subsequent S7 update stage.

[0032] In S5, the following steps are included:

[0033] S51: Set a sliding window for the original UWB ranging observations, calculate the ranging variance, and mark NLOS if the variance is greater than the threshold.

[0034] S52: Calculate the theoretical predicted distance from the predicted location and the coordinates of the UWB base station, and obtain the residual by subtracting the actual measured distance. If the residual is greater than the residual threshold, mark it as NLOS.

[0035] S53: If the ranging of a certain UWB base station does not match the geometric consistency test of other UWB base stations, it is judged as NLOS;

[0036] S54: For the raw UWB ranging observations marked with NLOS, multiply their observation noise covariance R_uwb,i by an amplification factor α (α>1, typically 10~100) to reduce their fusion weight;

[0037] S55: If the observation noise covariance R_uwb,i of the fusion weight exceeds the rejection threshold T_reject, the original UWB ranging observation is directly rejected; the output UWB effective distance observation is entered into the EKF fusion filter.

[0038] S56: When there are fewer than 3 UWB base stations with available LOS, automatically switch to AOA angle-dominant mode.

[0039] In S6, the following steps are included:

[0040] S61: Calculate the estimated tag-to-base station distance d based on the predicted location and Bluetooth AOA base station coordinate parameters;

[0041] S62: The angular observation noise covariance is calculated according to R_aoa = R0·(1 + λ·d_(tag-bs)). The larger the distance d, the larger the noise covariance and the lower the observation weight. Thus, the UWB ranging constrains the long-distance error amplification of AOA, realizing adaptive complementarity between near and far fields.

[0042] S63: Bluetooth AOA raw angle observation outputs AOA through angle observation noise covariance processing and inputs to EKF fusion filter.

[0043] In S7, the following steps are included:

[0044] S71: The effective distance observation set of UWB is updated after constructing Jacobian H_uwb,i through the observation equation z_uwb,i = ‖p − p_bs,i‖ + v_uwb,i;

[0045] S72: Azimuth update in Bluetooth AOA angle observation: α = arctan((p_y − p_bs,y) / (p x − p_bs,x)) + v_α、

[0046] Pitch angle update in Bluetooth AOA angle observation: β = arctan((p_z − p_bs,z) / √((p x −p_bs,x)²+ (p_y−p_bs,y)²)) + v_β

[0047] S73: Performs Kalman gain calculation, state correction, and covariance update; estimates gyroscope and accelerometer bias online, with each estimate based on UWB / AOA absolute observations and compensation for bias, continuously correcting IMU cumulative errors to prevent long-term divergence; outputs fused 3D position and attitude, and outputs updated IMU bias estimates, including closed-loop feedback prediction stage processing for IMU bias estimation; fused 3D position output is sent to the attitude calculation module and smart fence determination module.

[0048] S8 specifically includes the following steps.

[0049] S81: Position Determination: Calculate the horizontal distance d_j from exhibit j based on the EKF position p̂ = √((p̂) x When d_j < R_j (R_j is the radius of the exhibit fence, which is 1~5 meters depending on the exhibit type), it is considered a candidate.

[0050] S82: Partition wall suppression (line of sight detection): On the pre-stored 2D wall map, perform intersection detection on the line connecting p̂ to the exhibit position and each wall line segment. If there is an intersection, it is judged as a partition wall scene and the trigger is suppressed.

[0051] S83: Orientation Determination: Calculate the yaw angle ψ from the IMU, and then calculate the angle Δθ_j between the orientation and the exhibit direction: Δθ_j = |ψ −arctan((y_j−p̂_y) / (x_j−p̂) x When Δθ_j < θ_th (typically 60°), it is judged as facing the exhibit;

[0052] S84: Dwell timer, the preceding conditions must be met for more than 2-3 seconds;

[0053] S85: When all four conditions are met, an explanation trigger command carrying the exhibit ID is sent to the audio playback control module; The audio playback control module receives the trigger command, retrieves the corresponding exhibit explanation audio material from the guide content storage module, decodes and outputs the audio drive signal to the audio output module, and the speaker plays the explanation voice.

[0054] A museum intelligent tour guide system includes a mobile terminal, several UWB base stations, and several Bluetooth AOA base stations; the mobile terminal includes a UWB ranging module, a Bluetooth AOA direction finding module, an IMU inertial sensing module, and a tour guide processing system; the tour guide processing system includes an NLOS detection and suppression module, an EKF fusion filter, an intelligent fence determination module, an audio playback control module, and a distance adaptive weighting unit;

[0055] The UWB ranging module communicates bidirectionally with the UWB positioning base station to obtain UWB distance observation data.

[0056] The Bluetooth AOA direction finding module continuously transmits Bluetooth signals to the AOA positioning base station and then receives Bluetooth AOA angle data sent by the AOA base station based on the Bluetooth signals.

[0057] The IMU inertial sensing module acquires the raw inertial sensing data of the IMU;

[0058] The EKF fusion filter performs prediction stage processing on the raw IMU inertial sensing data to obtain the predicted state, predicted covariance matrix, and predicted position.

[0059] The NLOS detection and suppression module performs detection and suppression processing on the raw UWB ranging observations in conjunction with the predicted location and UWB base station coordinates to obtain the effective UWB distance observations;

[0060] The distance adaptive weighting unit performs adaptive weighting processing on the Bluetooth AOA angle data in conjunction with the predicted location Bluetooth and AOA base station coordinates;

[0061] The EKF fusion filter performs an update phase on the predicted state, predicted covariance matrix, UWB effective range observation set, and weighted AOA angle observations to obtain the fused 3D position and pose, and the updated IMU zero-bias estimate; the IMU zero-bias estimate closed-loop feedback prediction phase:

[0062] The intelligent fence determination module performs a four-dimensional joint determination of position, orientation, wall separation, and dwell time based on the fusion of three-dimensional position and user yaw attitude, and triggers the audio output module to explain the exhibits.

[0063] The beneficial effects of this invention are:

[0064] Suppressing NLOS-induced positioning drift and jumps: Through a triple NLOS detection mechanism, adaptive weighting and elimination of UWB non-line-of-sight observations are achieved, combined with IMU high-frequency prediction smoothing output. In museum environments with abundant metal and glass, positioning accuracy reaches 0.3-0.5 decimeter level, eliminating coordinate jump and drift phenomena.

[0065] Multi-sensor complementary advantages: UWB provides high-precision ranging constraints, while Bluetooth AOA amplifies long-distance errors; when UWB encounters full NLOS obstruction, Bluetooth AOA provides angle observation redundancy; in AOA blind spots, UWB takes over positioning; multiple sources serve as backups for each other, avoiding direct positioning loss due to the failure of a single sensor.

[0066] Suppressing IMU cumulative error: The EKF filter incorporates the zero bias of the gyroscope and accelerometer into the state vector, and relies on online estimation of absolute observations from UWB and AOA to compensate for the zero bias of the IMU, thus solving the problem of error divergence in long-term dead reckoning of low-cost MEMS IMUs.

[0067] Significantly reduce the rate of false triggers during guided tours: A four-dimensional joint determination based on location, orientation, partition wall, and dwell time is adopted. By detecting wall intersections, triggers triggered by partition walls are filtered out. IMU yaw angle identification of the visitor's orientation is used to filter triggers triggered by visitors facing away from exhibits. Dwell time is increased to filter false triggers triggered by passing by, greatly improving the visitor experience.

[0068] Low local computing deployment cost: All positioning fusion and fence determination logic run locally on the handheld guide device, eliminating the need to deploy edge servers, reducing hardware investment in venues, and enabling small and medium-sized museums to quickly deploy the technology. Attached Figure Description

[0069] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the detailed embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0070] Figure 1 This is a schematic diagram of the overall system deployment of the present invention;

[0071] Figure 2 This is a block diagram of the internal structure of the anti-mobility positioning terminal of the present invention;

[0072] Figure 3 This is a flowchart of the EKF multi-source fusion localization algorithm of the present invention;

[0073] Figure 4 This is a flowchart of the four-dimensional joint determination process for the intelligent geofencing of this invention. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] In this embodiment, see Figure 2 The mobile positioning terminal is an electronic guide device that tourists hold or wear. It is equipped with a UWB ranging module 201, a Bluetooth AOA direction finding module 202, an IMU inertial sensing module 203, a guide processing system 204, a power management module 205, a human-computer interaction module 206, and an audio output module 207. The UWB ranging module 201, Bluetooth AOA direction finding module 202, and IMU module 203 are connected to the guide processing system 204 via SPI / Bluetooth / I2C buses, respectively. The power management module 205 supplies power to all modules. The human-computer interaction module 206 and the audio output module 207 communicate bidirectionally with the guide processing system 204.

[0076] Among them, the UWB ranging module 201 uses a UWB radio frequency chip (such as the DW3000 series), which is connected to the tour guide processing system 204 via the SPI interface. It performs TWR bidirectional ranging or TDOA ranging with the UWB base station in the museum and outputs the distance observation value from the tag to each base station.

[0077] Among them, the Bluetooth AOA direction finding module 202 uses a Bluetooth chip (such as nRF52811) that supports Bluetooth 5.1 direction finding, transmits Bluetooth signals with CTE for AOA base station to measure angles, and receives azimuth and elevation angles sent by the base station via Bluetooth link for observation.

[0078] Among them, the IMU inertial sensing module 203 uses a six-axis or nine-axis MEMS IMU chip (such as ICM-42688 or BNO085) to output three-axis angular velocity, three-axis acceleration (and optional magnetic field) at 100Hz~200Hz for dead reckoning and attitude estimation.

[0079] The tour guide processing system 204 uses an embedded microprocessor (such as ARM Cortex-M4F / Cortex-A53), and its internal functional sub-modules include: NLOS detection and suppression module 204a, EKF fusion filter 204b, attitude calculation module 204c, intelligent fence determination module 204d, tour guide content storage module 204e, and audio playback control module 204f.

[0080] The power management module 205 uses a rechargeable lithium battery and a power management IC to power each module.

[0081] The human-computer interaction module 206 uses a button / touchscreen / voice interface to select the navigation mode and adjust the volume.

[0082] The audio output module 207 uses a speaker or headphone jack to play narration audio.

[0083] See Figure 1 This embodiment takes a military museum exhibition hall as an example. The exhibition hall contains steel weapons exhibits and thick glass display cases, and the metal and glass cause wireless signal reflection and obstruction. Several UWB positioning base stations 301 and several Bluetooth AOA positioning base stations 301 are deployed on the ceiling of the exhibition hall; the spatial coordinates of all base stations are calibrated; the base station coordinates, the two-dimensional wall map of the exhibition hall, the coordinates of the exhibits, the fence parameters, and the audio and video content of the explanation are pre-imported into the mobile positioning terminal's guide content storage module to generate a pre-configured dataset; the UWB positioning base stations and AOA positioning base stations can be co-located or separately deployed to achieve complementary coverage: UWB focuses on ranging, and AOA focuses on angle measurement; UWB is densified in areas with dense metal, and AOA is used to provide angle constraints in open areas; visitors use handheld mobile positioning terminals (smart guide devices) 200 for viewing;

[0084] See Figure 3 The workflow of the system of the present invention is as follows:

[0085] S1: The UWB ranging module communicates bidirectionally with the UWB positioning base station to obtain UWB distance observation data;

[0086] In S1, the specific steps include:

[0087] S11: The UWB ranging module communicates with the venue's UWB positioning base station;

[0088] S12: The UWB ranging module performs TWR bidirectional ranging; it outputs the raw UWB ranging observation and sends it to the NLOS detection and suppression module;

[0089] S2: The Bluetooth AOA direction finding module continuously transmits Bluetooth signals to the AOA positioning base station and then receives Bluetooth AOA angle data sent by the AOA base station based on the Bluetooth signals;

[0090] In S2, the specific steps include:

[0091] S21: The Bluetooth AOA direction finding module continuously transmits Bluetooth signals with CTE and receives azimuth and elevation data packets sent by the AOA positioning base station;

[0092] S22: The Bluetooth AOA direction finding module parses azimuth and elevation angle data packets and outputs Bluetooth AOA angle observations, which are then input to the distance adaptive weighting unit.

[0093] S3: The IMU inertial sensing module acquires the raw inertial sensing data of the IMU; S3 includes the following steps:

[0094] S31: Simulated physical motion signal acquired by MEMS sensor;

[0095] S32: Analog-to-digital conversion, outputting raw IMU inertial sensing data, which includes three-axis angular velocity and three-axis acceleration;

[0096] S4: The EKF fusion filter performs prediction processing on the raw inertial sensing data of the IMU. The predicted state vector is obtained through attitude update in S42 and velocity update in S43. The predicted position is obtained by extracting the three-dimensional position updated by integration in S43. The predicted covariance matrix is ​​obtained through covariance matrix prediction in S44.

[0097] The predicted state vector is defined as follows:

[0098] x = [p] x py, pz, v x , vy , vz ,φ, θ, ψ, bωx, bωy, bωz, bax, bay, baz ]ᵀ

[0099] Where p x , py, pz represent three-dimensional positions; v x, vy, vz represent three-dimensional velocities; φ, θ, ψ represent attitude Euler angles (roll, pitch, yaw); bωx, bωy, bωz represent three-axis zero bias of the gyroscope; bax, bay, baz represent three-axis zero bias of the accelerometer.

[0100] S4 includes the following steps:

[0101] S41: Obtain the estimated three-axis zero bias values ​​of the gyroscope and the zero bias value of the accelerometer through the IMU gyroscope and accelerometer of the previous closed-loop feedback;

[0102] S42: The three-axis angular velocity is subtracted from the zero-bias three-axis gyroscope, and the attitude is updated by quaternion differential equations and converted into Euler angles;

[0103] S43: The three-axis accelerometer is deducted for three-axis zero bias. The coordinate system is transformed to the navigation coordinate system through the attitude rotation matrix R(b→n), the gravity component is removed, and the three-dimensional velocity and three-dimensional position are updated by integration.

[0104] S44: Perform covariance matrix prediction, specifically using the formula P(k|k−1) = F·P(k−1|k−1)·Fᵀ + Q(k); where F is the state transition Jacobian, its value is obtained by substituting the amount of the current IMU raw inertial sensing data processed by S42 and S43 into the Jacobian of the kinematic equations—specifically, the gyroscope three-axis angular velocity ω (after deducting the estimated zero bias b̂_ω and used for attitude update) determines the coupling of the attitude channel to the attitude itself and the instantaneous value of the rotation matrix R(b→n) in F, and the accelerometer three-axis specific force f (after deducting the estimated zero bias b̂_a and rotating through R(b→n), subtracting gravity, and integrating for velocity and position update) determines the coupling of velocity to attitude and position to velocity in F; P(k−1|k−1) is the value of the previous EKF update stage (after UWB distance observation and AOA). The posterior covariance matrix output after angle observation correction is used as the starting point for this prediction; Q(k) is the process noise covariance, the value of which is calibrated according to the noise characteristics of the IMU device—the gyroscope angle random walk (ARW) corresponds to the noise term of the attitude-zero bias channel in Q, the accelerometer velocity random walk (VRW) corresponds to the noise term of the velocity-zero bias channel in Q, and the random walk noise of the gyroscope and accelerometer zero bias corresponds to the noise term of the zero bias diagonal block in Q; the three are propagated through F·P·Fᵀ + Q to obtain the current prediction covariance matrix P(k|k−1), which is used to calculate the Kalman gain in the subsequent S7 update stage.

[0105] S5: The NLOS detection and suppression module performs detection and suppression processing on the raw UWB ranging observations in conjunction with the predicted location and UWB base station coordinates to obtain the effective UWB distance observations;

[0106] In S5, the following steps are included:

[0107] S51: Set a sliding window for the original UWB ranging observations, calculate the ranging variance, and mark NLOS if the variance is greater than the threshold.

[0108] S52: Calculate the theoretical predicted distance from the predicted location and the coordinates of the UWB base station, and obtain the residual by subtracting the actual measured distance. If the residual is greater than the residual threshold, mark it as NLOS.

[0109] S53: If the ranging of a certain UWB base station does not match the geometric consistency test of other UWB base stations, it is judged as NLOS;

[0110] S54: For the raw UWB ranging observations marked with NLOS, multiply their observation noise covariance R_uwb,i by an amplification factor α (α>1, typically 10~100) to reduce their fusion weight;

[0111] S55: If the observation noise covariance R_uwb,i of the fusion weight exceeds the rejection threshold T_reject, the original UWB ranging observation is directly rejected; the output UWB effective distance observation is entered into the EKF fusion filter.

[0112] S56: When there are fewer than 3 UWB base stations with available LOS, automatically switch to AOA angle-dominant mode.

[0113] S6: The distance adaptive weighting unit performs adaptive weighting processing on the Bluetooth AOA angle data in conjunction with the predicted location Bluetooth and AOA base station coordinates;

[0114] S61: Calculate the estimated tag-to-base station distance d based on the predicted location and Bluetooth AOA base station coordinate parameters;

[0115] S62: The angular observation noise covariance is calculated according to R_aoa = R0·(1 + λ·d_(tag-bs)). The larger the distance d, the larger the noise covariance and the lower the observation weight. Thus, the UWB ranging constrains the long-distance error amplification of AOA, realizing adaptive complementarity between near and far fields.

[0116] S63: Bluetooth AOA raw angle observation is processed by angle observation noise covariance and output as AOA angle observation (azimuth, elevation and corresponding observation noise covariance) input to EKF fusion filter.

[0117] S7: The EKF fusion filter performs an update stage on the predicted state, predicted covariance matrix, UWB effective range observation set, and weighted AOA angle observations to obtain the fused 3D position and pose, and the updated IMU zero-bias estimate; IMU zero-bias estimation closed-loop feedback prediction stage:

[0118] In S7, the following steps are included:

[0119] S71: The effective UWB range observation set is expressed by the observation equation z_uwb,i = ‖p − p_bs,i‖ + v_uwb,i, where z_uwb,i is the range observation value of the i-th UWB base station, p = [p x Let p_y, p_z be the 3D location predicted by EKF, p_bs,i = [x_bs,i, y_bs,i, z_bs,i] be the known spatial coordinates of the i-th UWB base station, ‖·‖ be the Euclidean norm, and v_uwb,i be the distance observation noise that follows a zero-mean Gaussian distribution with a covariance of R_uwb,i. After constructing the Jacobian matrix H_uwb,i of the state vector using the observation equation, the update is performed.

[0120] S72: Azimuth update in Bluetooth AOA angle observation: α = arctan((py − p_bs,y) / (p x − pbs,x)) + vα, where α is the azimuth observation value, py, p x Let p_bs,y and pbs,x be the two-dimensional location coordinates predicted by EKF, respectively. Let vα be the azimuth observation noise, which follows a zero-mean Gaussian distribution with a covariance of R_aoa,α.

[0121] Pitch angle update in Bluetooth AOA angle observation: β = arctan((pz − pbs,z) / √((p x −pbs,x)² +(p_y−p_bs,y)²)) + vβ, where β is the elevation angle observation value, pz is the z-axis position coordinate predicted by EKF, pbs,z is the z-axis coordinate of the AOA base station, and vβ is the elevation angle observation noise that follows a zero-mean Gaussian distribution with a covariance of R_aoa,β;

[0122] S73: Perform Kalman gain calculation, specifically the formula K(k) = P(k|k−1)·Hᵀ·(H·P(k|k−1)·Hᵀ + R)⁻¹, where K(k) is the Kalman gain matrix, H is the observation Jacobian matrix (H_uwb,i for UWB distance observations, H_aoa for AOA angle observations), and R is the observation noise covariance matrix (R_uwb,i for UWB, R_aoa for AOA); perform state correction, specifically the formula x(k|k) = x(k|k−1) + K(k)·(z − h(x(k|k−1))), where x(k|k) is the updated posterior state estimate, x(k|k−1) is the predicted state, z is the actual observation vector, and h(·) is the nonlinear observation function; perform covariance update, specifically the formula P(k|k) = (I − K(k)·H)·P(k|k−1), where P(k|k) is the updated posterior covariance matrix and I is the identity matrix; online estimation of gyroscope and accelerometer biases, each time estimated and compensated online by UWB / AOA absolute observations, continuously correcting IMU cumulative errors to prevent long-term divergence; output fused 3D position and attitude, output updated IMU bias estimates, where the IMU bias estimation closed-loop feedback prediction stage is processed; fused 3D position output is sent to the attitude calculation module and the smart fence determination module.

[0123] In S8, the smart fence determination module performs a four-dimensional joint determination of position, orientation, wall separation, and dwell time based on the fusion of three-dimensional position and user yaw attitude, and triggers the audio output module to explain the exhibits;

[0124] See Figure 4 In S8, the specific steps include:

[0125] S81: Position Determination: Calculate the horizontal distance d_j from exhibit j based on the EKF position p̂ = √((p̂) x When d_j < R_j (R_j is the radius of the exhibit fence, which is 1~5 meters depending on the exhibit type), it is considered a candidate.

[0126] S82: Partition wall suppression (line of sight detection): On the pre-stored 2D wall map, perform intersection detection on the line connecting p̂ to the exhibit position and each wall line segment. If there is an intersection, it is judged as a partition wall scene and the trigger is suppressed.

[0127] S83: Orientation Determination: Calculate the yaw angle ψ from the IMU, and then calculate the angle Δθ_j between the orientation and the exhibit direction: Δθ_j = |ψ −arctan((y_j−p̂_y) / (x_j−p̂) x))|, where ψ is the yaw angle output by the IMU attitude calculation, i.e., the angle between the user's facing direction and the x-axis of the navigation coordinate system, Δθ_j is the angle between the user's facing direction and the direction of exhibit j, x_j and y_j are the two-dimensional coordinates of exhibit j, p̂ x p̂_y represents the user's current 2D position coordinates output by EKF fusion, arctan is the arctangent function used to calculate the direction angle from the user's position to the exhibit, θ_th is the orientation determination threshold, typically 60°, and |·| is the absolute value operation; when Δθ_j < θ_th (typically 60°), it is determined to be facing the exhibit;

[0128] S84: Dwell timer, the preceding conditions must be met for more than 2-3 seconds;

[0129] S85: If all four conditions are met, an explanation trigger command carrying the exhibit ID is sent to the audio playback control module. The audio playback control module receives the trigger command, retrieves the corresponding exhibit explanation audio material from the guide content storage module, decodes and outputs an audio drive signal to the audio output module, and the speaker plays the explanation audio.

[0130] The system continuously iterates through S4-S8, updating its position and posture in real time. The narration pauses when a visitor leaves the exhibit's enclosure; it switches to a new exhibit area if all four conditions are met. The audio guide is returned to its charging station, and its location trajectory log can be exported for museum use in visitor behavior analysis and exhibition layout optimization.

[0131] In this embodiment, all data processing, calculation, and judgment are completed locally on the mobile positioning terminal without the need for real-time server involvement; the positioning accuracy is 0.3-0.5m; it eliminates false triggering due to walls, back-to-back orientation, or passing by; the sensors are redundant, and local signal interference will not cause positioning failure.

[0132] Example data

[0133] (1) Example of raw inertial sensing data from IMU

[0134] Taking the ICM-42688 six-axis MEMS IMU as an example, with a sampling rate of 200Hz, the typical raw output when a visitor walks in the exhibition hall (walking speed of about 1.0~1.2 m / s) is as follows:

[0135] Standing still [0.01, 0.02, 0.01] [0.02, -0.03, 9.81] Normal walking [0.15~2.50, -0.30~0.80, -1.20~1.50] [0.10~3.50, -0.50~2.00, 8.50~11.20]

[0136] The zero bias values ​​estimated by EKF closed-loop feedback are: gyroscope zero bias b̂_ω ≈ [0.05, -0.03, 0.02] ° / s, accelerometer zero bias b̂_a ≈ [0.02, -0.01, 0.03] m / s². In the process noise covariance Q(k), the gyroscope angle random walk ARW = 0.0028 ° / s / √Hz, the accelerometer velocity random walk VRW = 200 μg / √Hz, and the standard deviation of the zero bias random walk noise is 0.001 ° / s (gyroscope) and 0.0005 m / s² (accelerometer).

[0137] (2) Examples of UWB ranging and NLOS detection

[0138] Four UWB base stations are deployed in the exhibition hall, with coordinates BS1(0,0,3), BS2(10,0,3), BS3(10,8,3), and BS4(0,8,3). When a visitor is located at (5.2, 3.1, 1.2), the ranging data from each base station is as follows:

[0139] BS1 5.83 5.81 0.02 0.008 LOS BS2 5.42 5.40 0.02 0.010 LOS BS3 6.31 6.28 0.03 0.012 LOS BS4 6.75 4.92 1.83 0.180 NLOS (residual > 0.3m and variance > 0.05m²)

[0140] BS4 was marked as NLOS due to a 1.83m positive offset caused by obstruction from steel exhibits. Multiplying its observation noise covariance R_uwb,4 by a magnification factor α=50 (reducing from 0.01 to a weight of 0.0002) effectively suppressed the contamination of the fusion results by NLOS observations. The final effective UWB distance observation set is {BS1, BS2, BS3}.

[0141] (3) Bluetooth AOA Angle and Adaptive Weighted Example

[0142] At the same time, the AOA base station is located at (5.0, 0, 3), with a transmit azimuth angle α = 78.5° and an elevation angle β = -22.3°. The estimated tag-base station distance d = 3.12m. The angular observation noise covariance is calculated as R_aoa = R0·(1 + λ·d), where R0 = 0.5°² and λ = 0.08, resulting in R_aoa,α = R_aoa,β = 0.5×(1+0.08×3.12) = 0.625°². When the tourist moves away to d = 8m, R_aoa increases to 0.82°², and the weight automatically decreases, achieving near-field and far-field adaptation.

[0143] (4) Comparison of EKF fusion results

[0144] Position error (m) 0.85 (cumulative) 0.42 0.31 Attitude error (°) 1.8 0.9 0.6 Zero bias convergence Non-convergence Partial convergence Convergence (b̂_ω stabilizes to ±0.01° / s)

[0145] After the three sources are integrated, the positioning accuracy is 0.3~0.5m, which meets the triggering requirements of the exhibit fence with a radius of 1~5m.

[0146] (5) Example of four-dimensional judgment of smart fence

[0147] The exhibit "a certain fighter jet" is located at coordinates (5.0, 3.0), and the radius of the enclosure is R_j = 3m.

[0148] Triggering scenario: The visitor is located at (5.2, 3.1) and facing the exhibit.

[0149] S81 Position Determination: d_j = √((5.2-5.0)²+(3.1-3.0)²) = 0.22m < 3m → Enter Candidate; S82 Wall Suppression: p̂ has no intersection with the wall line segment → No suppression; S83 Orientation Determination: Yaw angle ψ = 30°, exhibit orientation arctan((3.0-3.1) / (5.0-5.2)) = arctan(0.5) ≈ 26.6°, Δθ_j = |30°-26.6°| = 3.4° < 60° → Facing Exhibit; S84 Dwell Timer: Condition continuously meets 2.5s > 2s → Trigger; S85: Output Exhibit ID, play J-15 explanation audio.

[0150] Suppressed scenario: Visitors are located at (7.0, 2.5), and exhibits are behind a partition wall.

[0151] S81 Position Determination: d_j = √((7.0-5.0)²+(2.5-3.0)²) = 2.06m < 4m → Enter Candidate; S82 Partition Wall Suppression: p̂ intersects the line connecting to the exhibit and the wall segment (6.0, 2.0)-(6.0, 4.0) → Suppression trigger, no explanation played.

[0152] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A smart museum navigation method, characterized in that, Based on a mobile terminal, several UWB base stations, and several Bluetooth AOA base stations; the mobile terminal includes a UWB ranging module, a Bluetooth AOA direction finding module, an IMU inertial sensing module, and a navigation processing system; the navigation processing system includes an NLOS detection and suppression module, an EKF fusion filter, a smart fence determination module, an audio playback control module, and a distance adaptive weighting unit; including the following steps: S1: The UWB ranging module communicates bidirectionally with the UWB positioning base station to obtain UWB distance observation data; S2: The Bluetooth AOA direction finding module continuously transmits Bluetooth signals to the AOA positioning base station and then receives Bluetooth AOA angle data sent by the AOA base station based on the Bluetooth signals; S3: The IMU inertial sensing module acquires the raw inertial sensing data of the IMU; S4: The EKF fusion filter performs prediction stage processing on the raw inertial sensing data of the IMU. The predicted state vector is obtained through attitude update in S42 and velocity update in S43. The predicted position is obtained by extracting the three-dimensional position updated by integration in S43. The predicted covariance matrix is ​​obtained by predicting the covariance matrix in S44. S5: The NLOS detection and suppression module performs detection and suppression processing on the original UWB ranging observations in conjunction with the predicted location and UWB base station coordinates to obtain the effective UWB distance observations; S6: The distance adaptive weighting unit performs adaptive weighting processing on the Bluetooth AOA angle data in conjunction with the predicted location Bluetooth and AOA base station coordinates; S7: The EKF fusion filter performs an update stage on the predicted state, predicted covariance matrix, UWB effective range observation set, and weighted AOA angle observations to obtain the fused 3D position and pose, and the updated IMU zero-bias estimate; IMU zero-bias estimate closed-loop feedback prediction stage: S8: The intelligent fence determination module performs a four-dimensional joint determination of position, orientation, wall separation, and dwell time based on the fusion of three-dimensional position and user yaw attitude, and triggers the audio output module to explain the exhibits.

2. The intelligent museum navigation method according to claim 1, characterized in that: In S1, the specific steps include: S11: The UWB ranging module communicates with the venue's UWB positioning base station; S12: The UWB ranging module performs TWR bidirectional ranging; outputs the raw UWB ranging observation and sends it to the NLOS detection and suppression module.

3. The intelligent museum navigation method according to claim 1, characterized in that: In S2, the specific steps include: S21: The Bluetooth AOA direction finding module continuously transmits Bluetooth signals with CTE and receives azimuth and elevation data packets sent by the AOA positioning base station; S22: The Bluetooth AOA direction finding module analyzes azimuth and elevation data packets and outputs Bluetooth AOA angle observations, inputting the distance adaptive weighting unit.

4. The intelligent museum navigation method according to claim 1, characterized in that: S3 includes the following steps: S31: Simulated physical motion signal acquired by MEMS sensor; S32: Analog-to-digital conversion, outputting raw IMU inertial sensing data, which includes three-axis angular velocity and three-axis acceleration.

5. The intelligent museum navigation method according to claim 1, characterized in that: in, The predicted state vector is defined as: x = [ p x ,py,pz,v x ,vy ,vz ,φ ,θ ,ψ , bωx ,bωy ,bωz ,bax ,bay ,baz ]ᵀ Where p x , py, pz represent three-dimensional positions; v x , vy, vz represent three-dimensional velocities; φ, θ, ψ represent attitude Euler angles (roll, pitch, yaw); bωx, bωy, bωz represent three-axis zero bias of the gyroscope; bax, bay, baz represent three-axis zero bias of the accelerometer. S4 includes the following steps: S41: Obtain the estimated three-axis zero bias values ​​of the gyroscope and the zero bias value of the accelerometer through the IMU gyroscope and accelerometer of the previous closed-loop feedback; S42: The three-axis angular velocity is subtracted from the zero-bias three-axis gyroscope, and the attitude is updated by quaternion differential equations and converted into Euler angles; S43: The three-axis accelerometer is deducted for three-axis zero bias. The coordinate system is transformed to the navigation coordinate system through the attitude rotation matrix R(b→n), the gravity component is removed, and the three-dimensional velocity and three-dimensional position are updated by integration. S44: Perform covariance matrix prediction, specifically using the formula P(k|k−1) = F·P(k−1|k−1)·Fᵀ + Q(k); where F is the state transition Jacobian, its value is obtained by substituting the values ​​of the current IMU raw inertial sensing data processed by S42 and S43 into the Jacobian of the kinematic equations—specifically, the gyroscope three-axis angular velocity ω (after deducting the estimated zero bias b̂_ω and used for attitude update) determines the coupling of the attitude channel to the attitude itself and the instantaneous value of the rotation matrix R(b→n) in F; the accelerometer three-axis specific force f (after deducting the estimated zero bias b̂_a and rotating through R(b→n), subtracting gravity, and integrating for velocity and position update) determines the coupling of velocity to attitude and position to velocity in F; P(k−1|k−1) is the value of the previous EKF update stage (after UWB distance observation and AOA). The posterior covariance matrix output after angle observation correction is used as the starting point for this prediction; Q(k) is the process noise covariance, the value of which is calibrated according to the noise characteristics of the IMU device—the gyroscope angle random walk (ARW) corresponds to the noise term of the attitude-zero bias channel in Q, the accelerometer velocity random walk (VRW) corresponds to the noise term of the velocity-zero bias channel in Q, and the random walk noise of the gyroscope and accelerometer zero bias corresponds to the noise term of the zero bias diagonal block in Q; the three are propagated through F·P·Fᵀ + Q to obtain the current prediction covariance matrix P(k|k−1), which is used to calculate the Kalman gain in the subsequent S7 update stage.

6. The intelligent museum navigation method according to claim 1, characterized in that: In S5, the following steps are included: S51: Set a sliding window for the original UWB ranging observations, calculate the ranging variance, and mark NLOS if the variance is greater than the threshold. S52: Calculate the theoretical predicted distance from the predicted location and the coordinates of the UWB base station, and obtain the residual by subtracting the actual measured distance. If the residual is greater than the residual threshold, mark it as NLOS. S53: If the ranging of a certain UWB base station does not match the geometric consistency test of other UWB base stations, it is judged as NLOS; S54: For the raw UWB ranging observations marked with NLOS, multiply their observation noise covariance R_uwb,i by an amplification factor α (α>1, typically 10~100) to reduce their fusion weight; S55: If the observation noise covariance R_uwb,i of the fusion weight exceeds the rejection threshold T_reject, the original UWB ranging observation is directly rejected; the effective UWB range observation is output and enters the EKF fusion filter; S56: When there are fewer than 3 UWB base stations with available LOS, automatically switch to AOA angle-dominant mode.

7. The intelligent museum navigation method according to claim 1, characterized in that: In S6, the following steps are included: S61: Calculate the estimated tag-to-base station distance d based on the predicted location and Bluetooth AOA base station coordinate parameters; S62: The angular observation noise covariance is calculated according to R_aoa = R0·(1 + λ·d_(tag-bs)). The larger the distance d, the larger the noise covariance and the lower the observation weight. Thus, UWB ranging constrains the long-distance error amplification of AOA, realizing adaptive complementarity between near and far fields. S63: Bluetooth AOA raw angle observation outputs AOA through angle observation noise covariance processing and inputs to EKF fusion filter.

8. The intelligent museum navigation method according to claim 1, characterized in that: In S7, the following steps are included: S71: The effective UWB range observation set is expressed by the observation equation z_uwb,i = ‖p − p_bs,i‖ + v_uwb,i, where z_uwb,i is the range observation value of the i-th UWB base station, p = [p x Let p_y, p_z be the 3D location predicted by EKF, p_bs,i = [x_bs,i, y_bs,i, z_bs,i] be the known spatial coordinates of the i-th UWB base station, ‖·‖ be the Euclidean norm, and v_uwb,i be the distance observation noise that follows a zero-mean Gaussian distribution with a covariance of R_uwb,i. After constructing the Jacobian matrix H_uwb,i of the state vector using the observation equation, the update is performed. S72: Azimuth update in Bluetooth AOA angle observation: α = arctan((p_y − p_bs,y) / (p x − p_bs,x))+ v_α, where α is the azimuth observation value, p_y, p x Let p_bs,y and p_bs,x be the two-dimensional location coordinates predicted by EKF, respectively. Let v_α be the azimuth observation noise, which follows a zero-mean Gaussian distribution with a covariance of R_aoa,α. Pitch angle update in Bluetooth AOA angle observation: β = arctan((p_z − p_bs,z) / √((p x −p_bs,x)² +(p_y−p_bs,y)²)) + v_β, where β is the observed pitch angle, p_z is the z-axis position coordinate predicted by EKF, p_bs,z is the z-axis coordinate of the AOA base station, and v_β is the pitch angle observation noise that follows a zero-mean Gaussian distribution with a covariance of R_aoa,β; S73: Perform Kalman gain calculation, specifically the formula K(k) = P(k|k−1)·Hᵀ·(H·P(k|k−1)·Hᵀ +R)⁻¹, where K(k) is the Kalman gain matrix, H is the observation Jacobian matrix (H_uwb,i for UWB distance observations, H_aoa for AOA angle observations), and R is the observation noise covariance matrix (R_uwb,i for UWB, R_aoa for AOA); perform state correction, specifically the formula x(k|k) = x(k|k−1) + K(k)·(z − h(x(k|k−1))), where x(k|k) is the updated posterior state estimate, x(k|k−1) is the predicted state, z is the actual observation vector, and h(·) is the nonlinear observation function; perform covariance update, specifically the formula P(k|k) = (I − K(k)·H)·P(k|k−1), where P(k|k) is the updated posterior covariance matrix and I is the identity matrix; online estimation of gyroscope and accelerometer biases, each time estimated and compensated online by UWB / AOA absolute observations, continuously correcting IMU cumulative errors to prevent long-term divergence; output fused 3D position and attitude, output updated IMU bias estimates, where the IMU bias estimation closed-loop feedback prediction stage is processed; fused 3D position output is sent to the attitude calculation module and the smart fence determination module.

9. The intelligent museum navigation method according to claim 1, characterized in that: S8 specifically includes the following steps. S81: Position Determination: Calculate the horizontal distance d_j from exhibit j based on the EKF position p̂ = √((p̂) x When d_j < R_j (R_j is the radius of the exhibit fence, which is 1~5 meters depending on the exhibit type), it is considered a candidate. S82: Partition wall suppression: On the pre-stored 2D wall map, perform intersection detection on the line connecting p̂ to the exhibit location and each wall line segment. If there is an intersection, it is judged as a partition wall scene and the trigger is suppressed. S83: Orientation Determination: Calculate the yaw angle ψ from the IMU, and then calculate the angle Δθ_j between the orientation and the exhibit direction: Δθ_j = |ψ − arctan((y_j−p̂_y) / (x_j−p̂) x ))|, where ψ is the yaw angle output by the IMU attitude calculation, i.e., the angle between the user's facing direction and the x-axis of the navigation coordinate system, Δθ_j is the angle between the user's facing direction and the direction of exhibit j, x_j and y_j are the two-dimensional coordinates of exhibit j, p̂ x p̂_y represents the user's current 2D position coordinates output by EKF fusion, arctan is the arctangent function used to calculate the direction angle from the user's position to the exhibit, θ_th is the orientation determination threshold, typically 60°, and |·| is the absolute value operation; when Δθ_j < θ_th (typically 60°), it is determined to be facing the exhibit; S84: Dwell timer, the preceding conditions must be met for more than 2-3 seconds; S85: If all four conditions are met, output a narration trigger command carrying the exhibit ID and send it to the audio playback control module; The audio playback control module receives the trigger command, retrieves the corresponding exhibit explanation audio material from the guide content storage module, decodes and outputs the audio drive signal to the audio output module, and the speaker plays the explanation voice.

10. The intelligent museum navigation system according to claim 1, characterized in that: It includes a mobile terminal, several UWB base stations, and several Bluetooth AOA base stations; the mobile terminal includes a UWB ranging module, a Bluetooth AOA direction finding module, an IMU inertial sensing module, and a navigation processing system; the navigation processing system includes an NLOS detection and suppression module, an EKF fusion filter, a smart fence determination module, an audio playback control module, and a distance adaptive weighting unit; The UWB ranging module communicates bidirectionally with the UWB positioning base station to obtain UWB distance observation data. The Bluetooth AOA direction finding module continuously transmits Bluetooth signals to the AOA positioning base station and then receives Bluetooth AOA angle data sent by the AOA base station based on the Bluetooth signals. The IMU inertial sensing module acquires the raw inertial sensing data of the IMU; The EKF fusion filter performs prediction stage processing on the raw IMU inertial sensing data to obtain the predicted state, predicted covariance matrix, and predicted position. The NLOS detection and suppression module performs detection and suppression processing on the raw UWB ranging observations in conjunction with the predicted location and UWB base station coordinates to obtain the effective UWB distance observations; The distance adaptive weighting unit performs adaptive weighting processing on the Bluetooth AOA angle data in conjunction with the predicted location Bluetooth and AOA base station coordinates; The EKF fusion filter performs an update phase on the predicted state, predicted covariance matrix, UWB effective range observation set, and weighted AOA angle observations to obtain the fused 3D position and pose, and the updated IMU zero-bias estimate; the IMU zero-bias estimate closed-loop feedback prediction phase: The intelligent fence determination module performs a four-dimensional joint determination of position, orientation, wall separation, and dwell time based on the fusion of three-dimensional position and user yaw attitude, and triggers the audio output module to explain the exhibits.

Images