A scenic spot mixed reality experience system and method combined with a shared unmanned aerial vehicle

Abstract

This invention relates to the field of drone technology, specifically to a mixed reality (MR) experience system and method for scenic areas that integrates shared drones. The system includes a shared drone experience cabin, a head-mounted MR device, a control handle, a drone dispatch system, and a cloud server. After renting a drone, tourists wear the MR device and select a preset flight path. The drone flies along the path and transmits images in real time. The MR device overlays 3D virtual animations to provide historical and cultural explanations. When the drone reaches a key point, the control handle is temporarily opened, allowing tourists to take photos from multiple angles and switch display modes within an electronic fence. This invention integrates shared drones with MR technology, providing a first-person, immersive aerial experience, balancing operational freedom and guidance, supporting personalized narrative tours, and is suitable for the digital display and science education of large outdoor cultural heritage scenic areas.

Description

Technical Field

[0001] This invention relates to the field of drone technology, and more specifically, to a mixed reality experience system and method for scenic spots that combines shared drones. Background Technology

[0002] Currently, the application of shared drones in scenic areas has developed to a certain extent. Existing technologies mainly fall into two categories: the first focuses on the shooting methods of shared drones, such as using drones for 3D real-time image transmission (Zhang Yaqiang, 2016) or aerial photography control in public areas (Ning Zhixiong, 2024); the second focuses on the operation and management of shared drones in scenic areas, including management platforms, service methods, and control algorithms (Wu Zhenghui et al., 2018; Ma Dongyue, 2019; Wu Ying et al., 2023). On the other hand, the application of mixed reality (MR) technology in the cultural tourism field is mostly manifested in self-guided tour systems or virtual-real scene operation platforms (Wu Junxiao et al., 2021; Ji Kuibin et al., 2019).

[0003] However, the aforementioned existing technologies have the following shortcomings: Large outdoor cultural heritage sites often restrict or prohibit drone aerial photography for protection purposes, and drones and accessories are heavy and inconvenient for tourists to carry on long-distance hikes.

[0004] Existing shared drone solutions mainly focus on shooting functions or process management, with little consideration given to how to use drones to enhance tourists' in-depth experience of scenic spots' unique landscapes and historical culture.

[0005] The combination of MR technology and shared drones is rarely seen in cultural heritage sites, lacking a systematic solution that can provide both an aerial perspective and an immersive explanation of history and culture.

[0006] In existing technologies, there is a lack of reasonable balance between tourists' freedom of operation of drones and their guiding role in the tour, which can easily lead to loss of control or fragmented experience. Summary of the Invention

[0007] The purpose of this invention is to provide a mixed reality experience and method for scenic spots that combines shared drones, in order to solve the above-mentioned problems.

[0008] To achieve the above objectives, the embodiments of this application provide the following technical solutions: On one hand, this application provides a mixed reality experience system for scenic spots that combines shared drones. The system includes: a shared drone experience cabin for tourists to rent drones; a head-mounted MR device and a control handle, worn on the tourist's head and held in hand for receiving real-time image transmissions from the drone, displaying 3D modeled virtual animations, and controlling the drone's flight speed and viewing angle; a drone scheduling module for assessing weather and congestion levels based on preset routes selected by tourists, allocating available drones, and granting flight permissions; and a cloud server for storing full-flight video recordings and photos taken by the user. During the flight of the drone along the preset route, the head-mounted MR device provides real-time explanations of the region's historical and cultural information based on the shooting scenes transmitted back by the drone and combined with virtual animations created by 3D modeling. When the drone flies to a special node in the route, the system temporarily grants the operating permissions of the control handle, allowing tourists to take multi-angle photos of the node within the airspace limited by the electronic fence, and switch between different display modes built into the MR device to achieve multiple switching or overlay of virtual scenes and real-time captured images.

[0009] Optionally, the preset route includes multiple narrative threads, which are selected from at least one of the following: a storyline about ordinary people's lives, a storyline about architectural art, or a storyline about historical events.

[0010] Optionally, the built-in display modes of the head-mounted MR device include a virtual scene display mode, a real-time captured image display mode, and a virtual scene and real-time captured image overlay display mode.

[0011] Optionally, the deposit for the shared drone experience cabin is automatically refunded after the user returns the head-mounted MR device and control handle and it is found to be undamaged; the user can download flight videos and photos from the cloud server via a mobile app.

[0012] Optionally, when the drone flies to a featured node in the flight path, the system temporarily grants the operator's control to allow tourists to take multi-angle photos of the node within the airspace defined by the electronic fence, including: The drone scheduling module reads the original displacement signal of the joystick at a preset sampling frequency, performs a first-order low-pass filter on the original displacement signal to eliminate the high-frequency jitter component caused by human physiological tremors, and obtains a smooth displacement signal. The drone scheduling module determines the offset of the smooth displacement signal based on a preset static dead zone threshold. If the offset is lower than the static dead zone threshold, the effective joystick command is forcibly set to zero. At the same time, the module dynamically expands or shrinks the range of the current effective dead zone threshold according to the fluctuation frequency and amplitude characteristics of the joystick displacement within the historical time window. The UAV scheduling module calculates the angle between the flight direction vector indicated by the effective joystick command and the current heading vector of the UAV. The effective command is a smooth displacement signal with an offset greater than or equal to the static dead zone threshold. When the included angle is less than the preset direction change threshold and the duration does not exceed the preset duration, the effective joystick command is mapped to the pointing deflection command of the airborne gimbal camera to maintain the fuselage heading unchanged; when the included angle is greater than or equal to the direction change threshold and the duration exceeds the preset duration, the UAV fuselage heading adjustment operation is performed.

[0013] Optionally, the UAV's sub-control module incorporates a first-order inertial ramp virtual response model, which is used to represent the response process of the target flight rate indicated by the effective joystick command as a nonlinear gradual acceleration and deceleration. The damping coefficient of the first-order inertial ramp model is set to 0.15-0.25, so that the rate of change of the drone's speed when performing acceleration or deceleration is lower than the drone's default rate of change, thereby suppressing the sudden stop or ejection sensation caused by the joystick being pushed or pulled abruptly.

[0014] Optionally, during the remote control of the drone by the drone handle, the drone scheduling module in the background constructs a joystick data observation queue that continuously covers a preset number of frames. The observation queue records the joystick displacement vector magnitude and direction angle data that change over time, and calculates the fluctuation variance value of the joystick displacement data sequence. If the fluctuation variance value is higher than the preset activity threshold, it is determined to be in an active control state, and the drone maintains the current flight response logic based on joystick commands; If the fluctuation variance value is continuously lower than the preset silent threshold and the duration does not reach the first preset duration, it is determined to be a transition correction state, and the drone only performs linear deceleration and gliding without triggering emergency braking. If the fluctuation variance value remains below the preset silence threshold and the duration reaches or exceeds the first preset duration, a stay intention confirmation signal is triggered. In response to the hovering intention confirmation signal, the flight control system forcibly switches to the high-precision hovering hold mode, ignoring the slight drift signal of the joystick within the mid-position dead zone, and locks the current viewpoint of the gimbal; When the UAV scheduling module detects that the joystick displacement signal clearly exceeds the dynamic dead zone threshold and continues for a second preset duration, it exits the high-precision hovering mode and resumes normal flight response to valid joystick commands.

[0015] Optionally, the UAV scheduling module monitors the rate of change and directional change characteristics of the effective joystick command sequence in real time. If the joystick command is detected to switch instantaneously from the first extreme direction pose to the extreme direction pose opposite to the first extreme direction within a preset very short time period, it is determined to be a handle collision or non-subjective accidental touch event. The flight control system directly discards the joystick command data packet of that frame, maintains the flight state of the previous effective command unchanged, and sends a transient tactile vibration command to the handle operation terminal to indicate to the operator that the current operation is invalid. The head-mounted MR device generates a dual-ring virtual instrument interface at the edge of its field of view. The position of the inner ring cursor is updated in real time based on the original displacement signal of the joystick, which is used to display the operator's actual physical operation trajectory. The position of the outer ring cursor is updated in real time based on the effective flight command data after smoothing displacement signal, separation of direction intent and inertial correction, which is used to display the smooth flight trajectory finally executed by the UAV.

[0016] On the other hand, embodiments of this application provide a method for a mixed reality experience in scenic areas that combines shared drones, the method comprising: Visitors can rent drones by scanning the QR code on the shared drone experience cabin; After confirming the rental request, the shared drone experience cabin will grant space access to the experience cabin. Users wear head-mounted MR devices and control handles inside the experience cabin to activate a scenic area mixed reality experience system that combines shared drones. After the system detects that the device is activated, it will enter the operation tutorial and provide the user with preset flight routes that are available in this scenic area; Users can choose routes based on their personal preferences or system suggestions. Then, the drone scheduling module allocates currently available drones and assesses the weather conditions and congestion level of the current route. Once the flight conditions are met, the drones are granted flight permissions. After the flight path is confirmed, the drone takes off automatically. While maintaining the overall direction of the flight path, the user can control the drone's flight speed and viewing angle using the control handle according to their personal needs. During the flight, the head-mounted MR device explains the historical and cultural information of the target area in real time based on the shooting scene transmitted back by the drone, combined with the virtual animation of 3D modeling. It can shoot at any time during the flight. After the drone flies to a pre-set special node in the flight path, the operation of the control handle is granted for a limited time. Users can take pictures of this node from multiple angles within the airspace according to the electronic fence set by the system, and can switch between different display modes built into the MR device to switch or overlay virtual scenes and displayed images. After the flight route is completed, the drone automatically returns to the drone bay for charging, and the video recording of the entire flight and the photos taken by the user are uploaded to the cloud server. After removing the head-mounted MR device and control handle, the user returns them to their original positions and exits the experience chamber. Once the system detects that the device is not damaged, the deposit is refunded. After the shared drone experience ends, users can download flight videos and photos via a mobile app.

[0017] Optionally, the flight speed is divided into 1-5 levels, and the drone defaults to speed level 3 after takeoff; Users can adjust the posture of the head-mounted MR device by rotating their heads, thereby adjusting the viewing angle; During the flight, the head-mounted MR device overlays a 3D model of the area onto the captured real-world scene, and explains the distribution of attractions, roads, and historical context of the scenic area in real time through the highlighted display and flashing animation of the 3D model. Secondly, when the drone flies to a specific node, the system loads 3D reconstruction models from different periods onto the captured scene and provides audio commentary. During this process, the user can use a joystick or gestures to disassemble, enlarge, or reduce the 3D reconstruction model, and the head-mounted MR device will explain the structural information of the corresponding local area.

[0018] The beneficial effects of this invention are as follows: The mixed reality experience system for scenic spots that combines shared drones with head-mounted MR devices provided by this invention allows tourists to obtain real-time aerial images from a first-person perspective, while simultaneously overlaying virtual animations with 3D modeling (such as site restoration, structural disassembly, etc.), achieving immersive explanations of history and culture from multiple times and angles, significantly enhancing tourists' understanding and perception of cultural heritage.

[0019] Secondly, the system presets flight paths and grants limited access to the control handles at key nodes. Combined with electronic fences to restrict airspace, this not only prevents users from excessive or dangerously operating the drone, but also gives it the right to take photos and switch perspectives at key locations, enhancing the sense of participation and immersion.

[0020] Secondly, by adopting a shared drone operation model, tourists do not need to carry their own drones and accessories. They can rent them on demand in the scenic area's experience cabin, which effectively reduces the burden of hiking and improves the convenience of the tour. At the same time, the system's built-in preset routes can include a variety of narrative threads (such as the line of ordinary people's lives, the line of architectural art, and the line of historical events). Users can choose different routes according to their interests, making the tour experience more personalized and story-based, and enhancing the attractiveness of cultural tourism.

[0021] Secondly, the system features a modular design, including a shared drone experience cabin, a head-mounted MR device, a control handle, a drone scheduling module, and a cloud server. The clear structure facilitates rapid deployment and content updates in various large-scale outdoor cultural heritage scenic areas.

[0022] In summary, this invention effectively overcomes the shortcomings of existing shared drone solutions, such as neglecting cultural experience, inconvenience in carrying, and poor operational guidance. It can be widely applied to the digital display and tourism services of large-scale outdoor cultural heritage scenic spots.

[0023] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of a mixed reality experience system for scenic spots that combines shared drones, as described in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0027] It should be noted that similar reference numerals or letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. Example

[0028] like Figure 1 As shown, this embodiment provides a mixed reality experience system for scenic spots that combines shared drones. The system includes: The shared drone experience cabin is used for tourists to rent drones. After the rental ends, the damage to the head-mounted MR device and the control handle is checked. If no damage is found, the deposit is automatically refunded. The head-mounted MR device and operating handle are worn on the head and held by the user for receiving real-time image transmission from the drone, displaying virtual animations of 3D modeling, and controlling the drone's flight speed and viewing angle. The built-in display modes of the head-mounted MR device include a virtual scene display mode, a real-time captured image display mode, and a virtual scene and real-time captured image overlay display mode. The drone dispatch module is used to assess the weather and congestion level based on the preset routes selected by tourists, allocate available drones, and grant flight permissions. The cloud server is used to store the entire flight video recording and photos taken by the user. The user can download the flight video and photos from the cloud server through a mobile APP. During the flight of the drone along the preset route, the head-mounted MR device explains the historical and cultural information of the area in real time based on the shooting scene transmitted back by the drone and combined with the virtual animation of 3D modeling. The preset route includes multiple narrative clues, which are selected from at least one of the following: the line of ordinary people's life, the line of architectural art, or the line of historical events. When the drone flies to a featured node in the flight path, the system grants the operator access to the control handle for a limited time, allowing tourists to take multi-angle photos of the node within the airspace defined by the electronic fence, and switch between different display modes built into the MR device to achieve multiple switching or overlay of virtual scenes and real-time captured images.

[0029] The mixed reality experience system for scenic spots that combines shared drones with head-mounted MR devices provided in this embodiment allows tourists to obtain real-time aerial images from a first-person perspective, while simultaneously overlaying virtual animations with 3D modeling (such as site restoration and structural disassembly), achieving immersive explanations of history and culture from multiple times and angles, significantly enhancing tourists' understanding and perception of cultural heritage.

[0030] Secondly, the system presets flight paths and grants limited access to the control handles at key nodes. Combined with electronic fences to restrict airspace, this not only prevents users from excessive or dangerously operating the drone, but also gives it the right to take photos and switch perspectives at key locations, enhancing the sense of participation and immersion.

[0031] Secondly, by adopting a shared drone operation model, tourists do not need to carry their own drones and accessories. They can rent them on demand in the scenic area's experience cabin, which effectively reduces the burden of hiking and improves the convenience of the tour. At the same time, the system's built-in preset routes can include a variety of narrative threads (such as the line of ordinary people's lives, the line of architectural art, and the line of historical events). Users can choose different routes according to their interests, making the tour experience more personalized and story-based, and enhancing the attractiveness of cultural tourism.

[0032] Secondly, the system features a modular design, including a shared drone experience cabin, a head-mounted MR device, a control handle, a drone scheduling module, and a cloud server. The clear structure facilitates rapid deployment and content updates in various large-scale outdoor cultural heritage scenic areas.

[0033] In summary, the mixed reality experience system for scenic spots that combines shared drones as described in this embodiment effectively overcomes the shortcomings of existing shared drone solutions, such as neglecting cultural experience, inconvenience in carrying, and poor operational guidance. It can be widely used in the digital display and tourism services of large-scale outdoor cultural heritage scenic spots. Example

[0034] This embodiment uses the example of tourists taking free time to take photos at a scenic spot's ancient building to illustrate the entire process from joystick input to the drone's smooth response.

[0035] Phase 1: Signal Input and Physical Layer Cleanup.

[0036] Step S101: Visitors enter the shared drone experience cabin, put on the head-mounted mixed reality device, and hold the control handle. After the drone arrives at the preset special node airspace and is granted control permissions, the drone scheduling module begins to read the Hall sensor voltage values ​​of the left and right joysticks of the handle in real time at a frequency of 100Hz, obtaining the raw displacement signals representing lateral roll and longitudinal pitch.

[0037] Step S102: The drone scheduling module performs first-order low-pass filtering on the original displacement signal. In this embodiment, the time constant of the filter is optimized to significantly attenuate physiological hand tremor signals with frequencies higher than 15Hz. For example, the sawtooth wave generated by the natural micro-tremor of a tourist's fingers is smoothed into a continuous voltage change curve without obvious spikes, thus obtaining a "smooth displacement signal".

[0038] Step S103: The drone scheduling module, based on a preset static dead zone threshold (the preset static dead zone threshold is 5% of the total joystick travel), determines the offset of the smooth displacement signal. If the offset is lower than the static dead zone threshold, the effective joystick command is forcibly reset to zero. Simultaneously, based on the fluctuation frequency and amplitude characteristics of the joystick displacement within the historical time window, the current effective dead zone threshold range is dynamically expanded or reduced. Specifically, if the offset of the smooth displacement signal caused by the user's natural finger placement is less than 5%, the system forcibly resets the effective joystick command in that direction to zero. Furthermore, the system monitors the operation history within the past 2 seconds. If it detects high-frequency back-and-forth shaking of the joystick within a 10% amplitude range, the system determines that the user is in a "framing hesitation state" and automatically expands the dead zone threshold temporarily to 12%, thereby completely eliminating the ghost drift phenomenon caused by finger fatigue during drone hovering.

[0039] Phase Two: Semantic separation and smoothing correction of direction and rate intentions.

[0040] Step S104: The drone scheduling module performs vector decomposition on the purified joystick valid commands and executes "directional intent and fuselage turning separation determination". The drone scheduling module calculates the angle between the current joystick push direction and the current drone nose direction. For example, if a tourist wants to slightly adjust the framing, a slight push of the joystick to the right produces an angle of approximately 20 degrees. The valid command is indicated by a smooth displacement signal with an offset greater than or equal to the static dead zone threshold; If the 20-degree angle is less than the preset 30-degree directional change threshold, and the duration of the action is only 0.2 seconds (not exceeding the preset 0.3-second confirmation duration), the system determines that the action is a "gimbal fine-tuning intention" rather than a "body turning intention." In this case, the system only maps the joystick command to the right yaw command of the onboard gimbal camera, while the drone's body heading remains unchanged, effectively avoiding dizziness caused by repeated body rotation due to frequent fine-tuning by tourists.

[0041] Conversely, if the user continues to push the joystick to the right and holds it for more than 0.3 seconds, the system will then confirm that the user intends to change the viewing position and execute a slow yaw rotation of the drone's body.

[0042] Step S105: The system synchronously executes "rate intention inertial correction". A first-order inertial ramp virtual response model is built into the UAV's sub-control module, with a damping coefficient set to 0.2. When a tourist, due to tension, instantly pushes the pitch stick to the bottom, the model intervenes: the target flight speed command is 2 meters per second, but the actual speed output by the UAV motors does not jump, but rather increases slowly according to the rule of catching up with the target value by 20% in each control cycle.

[0043] In this embodiment, the drone's actual flight speed will gradually reach 2 meters per second after approximately 1.5 seconds, with extremely low acceleration during this period. Similarly, when the user instantly releases the joystick to return to the center, the drone does not brake suddenly, but rather smoothly decelerates to a hovering position. This setting ensures that the image transmitted back to the MR glasses maintains a consistently smooth, cinematic camera movement, eliminating the visual discomfort caused by "launching" or "sudden stops."

[0044] Phase 3: Precise identification of hovering intentions and hovering lock.

[0045] Step 106: During the remote control of the drone by the drone handle, the drone scheduling module in the background constructs a joystick data observation queue that continuously covers a preset number of frames (maintaining a short observation window with a duration of 200 milliseconds and continuously calculating the fluctuation variance of the joystick displacement data). The observation queue records the joystick displacement vector magnitude and direction angle data that change over time, and calculates the fluctuation variance value of the joystick displacement data sequence. If the fluctuation variance value is higher than the preset activity threshold, it is determined to be in an active control state, and the drone maintains the current flight response logic based on joystick commands; If the fluctuation variance value is continuously lower than the preset silent threshold and the duration does not reach the first preset duration, it is determined to be a transition correction state, and the drone only performs linear deceleration and gliding without triggering emergency braking. If the fluctuation variance value remains below the preset silence threshold and the duration reaches or exceeds the first preset duration, a stay intention confirmation signal is triggered. When a visitor is drawn to a particular scene and their finger naturally stops moving, the variance value within the observation window rapidly decreases and remains below the preset silence threshold. For the first 0.4 seconds, the system determines this to be a "transitional correction state," and the drone only performs linear deceleration gliding. When this silence state lasts for 1.0 second, the system triggers a "stay intention confirmation signal."

[0046] Step S107: In response to the hovering intention confirmation signal, the flight control system forcibly switches to high-precision hovering mode. In this mode, even if the tourist's fingers experience slight tremors within the dead zone due to physiological fatigue, the flight control system will classify such signals as invalid mis-touches and discard them directly. It ignores minor drift signals from the joystick within the mid-range dead zone and locks the gimbal's current viewing angle. At this time, the drone's position in three-dimensional space is locked within centimeter-level precision, and the gimbal's direction is also locked, providing the tourist with an extremely stable shooting platform.

[0047] When the drone scheduling module detects that the joystick displacement signal clearly exceeds the dynamic dead zone threshold and continues for a second preset duration, it exits the high-precision hovering mode and resumes normal flight response to valid joystick commands. Specifically, this can be achieved by actively and significantly pushing the joystick (the offset exceeds the dynamic dead zone threshold and continues for 0.3 seconds) after the tourist has finished enjoying the view. The system then exits the high-precision hovering mode and seamlessly returns to the inertial correction flight state described in step 105.

[0048] Phase 4: Preventing accidental touches and providing visual intent mapping feedback.

[0049] Step S109: The UAV scheduling module monitors the rate of change and directional abrupt change characteristics of the valid joystick command sequence in real time. If, within a preset very short time period, the joystick command is detected to instantly switch from the first extreme directional pose to the opposite extreme directional pose, it is determined to be a handle collision or non-subjective accidental touch event. The flight control system directly discards the joystick command data packet of that frame, maintaining the flight state of the previous valid command frame unchanged. Simultaneously, a transient tactile vibration command is sent to the handle operation terminal to indicate to the operator that the current operation is invalid. Specifically, during flight, if the flight control system detects a sudden change in the joystick command from "full forward range" to "full backward range" within a very short time of 0.2 seconds (e.g., the signal characteristics of a handle accidentally dropped or snatched by a child), the system determines it to be a handle collision or non-subjective accidental touch event. The system directly discards the joystick command data packet of that frame, and the UAV maintains the flight state of the previous valid command frame unchanged. At the same time, the handle's built-in motor emits a short 10-millisecond vibration as tactile rejection feedback indicating invalid operation.

[0050] Step 110: In order to bridge the gap between tourists' psychological expectations and the actual drone response, a dual-ring virtual instrument interface is generated at the edge of the field of view of the head-mounted MR device. The position of the inner ring cursor is updated in real time based on the original displacement signal of the joystick, which is used to display the actual physical operation trajectory of the operator. The position of the outer ring cursor is updated in real time based on the effective flight command data after smoothing displacement signal, directional intention separation and inertial correction processing, which is used to display the smooth flight trajectory finally executed by the drone.

[0051] For example, tourists can see the inner red dot drawing chaotic small circles due to their shaky hands, while the outer green dot, representing the drone's actual flight path, moves slowly along a straight and elegant line. This visual mapping not only eliminates tourists' confusion about latency but also intuitively demonstrates the system's auxiliary correction effect, significantly enhancing control confidence and immersive experience. Example

[0052] This embodiment provides a method for creating a mixed reality experience in a scenic area using shared drones, the method comprising: Step S1: Tourists rent drones by scanning the QR code on the shared drone experience cabin; Step S2: After confirming the rental request, the shared drone experience cabin will grant space access to the experience cabin. Step S3: The user wears a head-mounted MR device and a control handle in the experience cabin to activate the scenic area mixed reality experience system combined with shared drones; Step S4: After the system detects that the device is activated, it will enter the operation tutorial and provide the user with the preset flight routes that can be flown in this scenic area; Step S5: The user selects a flight route according to personal preference or system suggestions. Then, the drone scheduling module allocates currently available drones and, after assessing the weather conditions and congestion level of the current flight route to ensure that the flight conditions are met, grants the drones flight permissions. Step S6: After the flight path is confirmed, the drone takes off automatically. Under the premise that the overall direction of the flight path is kept in the preset position, the user can control the flight speed and observation angle of the drone through the control handle according to personal needs. During the flight, the head-mounted MR device explains the historical and cultural information of the target area in real time based on the shooting scene transmitted back by the drone and combined with the virtual animation of 3D modeling. It can shoot at any time during the flight. Step S7: After the drone flies to the preset special node in the flight path, the operation of the control handle is opened for a limited time. The user can take pictures of this node from multiple angles within the airspace according to the electronic fence set by the system, and can switch between different display modes built into the MR device to switch or overlay virtual scenes and displayed images. Step S8: After the flight route is completed, the drone automatically returns to the drone bay for charging, and the video recording of the entire flight and the photos taken by the user are uploaded to the cloud server. Step S9: The user removes the head-mounted MR device and control handle, puts them back in their original positions, and exits the experience chamber. After the system detects that the device is not damaged, the deposit is refunded. Step S10: The shared drone experience ends. Users can download flight videos and photos via the mobile app.

[0053] Secondly, in this embodiment, the flight speed is divided into 1-5 levels, and the drone defaults to speed level 3 after takeoff; Users can adjust the posture of the head-mounted MR device by rotating their heads, thereby adjusting the viewing angle; During the flight, the head-mounted MR device overlays a 3D model of the area onto the captured real-world scene, and explains the distribution of attractions, roads, and historical context of the scenic area in real time through the highlighted display and flashing animation of the 3D model. Secondly, when the drone flies to a specific node, the system loads 3D reconstruction models from different periods onto the captured scene and provides audio commentary. During this process, the user can use a joystick or gestures to disassemble, enlarge, or reduce the 3D reconstruction model, and the head-mounted MR device will explain the structural information of the corresponding local area.

[0054] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A mixed reality experience system for scenic spots that combines shared drones, characterized in that, The system includes: A shared drone rental cabin is available for tourists to rent drones. The head-mounted MR device and control handle are worn on the head and held by the user. They are used to receive real-time images from the drone, display virtual animations of 3D modeling, and control the drone's flight speed and viewing angle. The drone dispatch module is used to assess the weather and congestion level based on the preset routes selected by tourists, allocate available drones, and grant flight permissions. Cloud servers are used to store video recordings of the entire flight and photos taken by the user; During the flight of the drone along the preset route, the head-mounted MR device explains the regional historical and cultural information in real time based on the shooting scene transmitted back by the drone and combined with the virtual animation of 3D modeling. When the drone flies to a featured node in the flight path, the system grants the operator access to the control handle for a limited time, allowing tourists to take multi-angle photos of the node within the airspace defined by the electronic fence, and switch between different display modes built into the MR device to achieve multiple switching or overlay of virtual scenes and real-time captured images.

2. The scenic area mixed reality experience system combining shared drones as described in claim 1, characterized in that, The preset route includes multiple narrative threads, which are selected from at least one of the following: a storyline about ordinary people's lives, a storyline about architectural art, or a storyline about historical events.

3. The scenic area mixed reality experience system combining shared drones as described in claim 2, characterized in that, The built-in display modes of the head-mounted MR device include a virtual scene display mode, a real-time captured image display mode, and a virtual scene and real-time captured image overlay display mode.

4. The scenic area mixed reality experience system combining shared drones as described in claim 3, characterized in that, The deposit for the shared drone experience cabin is automatically refunded after the user returns the head-mounted MR device and control handle and it is found to be undamaged; the user can download flight videos and photos from the cloud server via a mobile app.

5. The scenic area mixed reality experience system combining shared drones as described in claim 1, characterized in that, When the drone flies to a featured node on the flight path, the system temporarily grants the operator's control to allow tourists to take multi-angle photos of the node within the airspace defined by the electronic fence, including: The drone scheduling module reads the original displacement signal of the joystick at a preset sampling frequency, performs a first-order low-pass filter on the original displacement signal to eliminate the high-frequency jitter component caused by human physiological tremors, and obtains a smooth displacement signal. The drone scheduling module determines the offset of the smooth displacement signal based on a preset static dead zone threshold. If the offset is lower than the static dead zone threshold, the effective joystick command is forcibly set to zero. At the same time, the module dynamically expands or shrinks the range of the current effective dead zone threshold according to the fluctuation frequency and amplitude characteristics of the joystick displacement within the historical time window. The UAV scheduling module calculates the angle between the flight direction vector indicated by the effective joystick command and the current heading vector of the UAV. The effective command is a smooth displacement signal with an offset greater than or equal to the static dead zone threshold. When the included angle is less than the preset direction change threshold and the duration does not exceed the preset duration, the effective joystick command is mapped to the pointing deflection command of the airborne gimbal camera to maintain the fuselage heading unchanged; when the included angle is greater than or equal to the direction change threshold and the duration exceeds the preset duration, the UAV fuselage heading adjustment operation is performed.

6. The scenic area mixed reality experience system combining shared drones as described in claim 5, characterized in that, The UAV's sub-control module has a built-in first-order inertial ramp virtual response model, which is used to represent the response process of the target flight rate indicated by the effective joystick command as a nonlinear gradual acceleration and deceleration. The damping coefficient of the first-order inertial ramp model is set to 0.15-0.25, so that the rate of change of the drone's speed when performing acceleration or deceleration is lower than the drone's default rate of change, thereby suppressing the sudden stop or ejection sensation caused by the joystick being pushed or pulled abruptly.

7. The scenic area mixed reality experience system combining shared drones as described in claim 6, characterized in that, During the remote control of the drone by the drone handle, the drone scheduling module in the background constructs a joystick data observation queue that continuously covers a preset number of frames. The observation queue records the joystick displacement vector magnitude and direction angle data that change over time, and calculates the fluctuation variance value of the joystick displacement data sequence. If the fluctuation variance value is higher than the preset activity threshold, it is determined to be in an active control state, and the drone maintains the current flight response logic based on joystick commands; If the fluctuation variance value is continuously lower than the preset silent threshold and the duration does not reach the first preset duration, it is determined to be a transition correction state, and the drone only performs linear deceleration and gliding without triggering emergency braking. If the fluctuation variance value remains below the preset silence threshold and the duration reaches or exceeds the first preset duration, a stay intention confirmation signal is triggered. In response to the hovering intention confirmation signal, the flight control system forcibly switches to the high-precision hovering hold mode, ignoring the slight drift signal of the joystick within the mid-position dead zone, and locks the current viewpoint of the gimbal; When the UAV scheduling module detects that the joystick displacement signal clearly exceeds the dynamic dead zone threshold and continues for a second preset duration, it exits the high-precision hovering mode and resumes normal flight response to valid joystick commands.

8. The scenic area mixed reality experience system combining shared drones according to claim 7, characterized in that, The UAV scheduling module monitors the rate of change and directional change characteristics of the valid joystick command sequence in real time. If the joystick command is detected to switch instantaneously from the first extreme direction pose to the opposite extreme direction pose within a preset very short time period, it is determined to be a handle collision or non-subjective accidental touch event. The flight control system directly discards the joystick command data packet of that frame, maintains the flight state of the previous valid command frame unchanged, and sends a transient tactile vibration command to the handle operation terminal to indicate to the operator that the current operation is invalid. The head-mounted MR device generates a dual-ring virtual instrument interface at the edge of its field of view. The position of the inner ring cursor is updated in real time based on the original displacement signal of the joystick, which is used to display the operator's actual physical operation trajectory. The position of the outer ring cursor is updated in real time based on the effective flight command data after smoothing displacement signal, separation of direction intent and inertial correction, which is used to display the smooth flight trajectory finally executed by the UAV.

9. A method for creating a mixed reality experience in scenic areas using shared drones, characterized in that, The method includes: Visitors can rent drones by scanning the QR code on the shared drone experience cabin; After confirming the rental request, the shared drone experience cabin will grant space access to the experience cabin. Users wear head-mounted MR devices and control handles inside the experience cabin to activate a scenic area mixed reality experience system that combines shared drones. After the system detects that the device is activated, it will enter the operation tutorial and provide the user with preset flight routes that are available in this scenic area; Users can choose routes based on their personal preferences or system suggestions. Then, the drone scheduling module allocates currently available drones and assesses the weather conditions and congestion level of the current route. Once the flight conditions are met, the drones are granted flight permissions. After the flight path is confirmed, the drone takes off automatically. While maintaining the overall direction of the flight path, the user can control the drone's flight speed and viewing angle using the control handle according to their personal needs. During the flight, the head-mounted MR device explains the historical and cultural information of the target area in real time based on the shooting scene transmitted back by the drone, combined with the virtual animation of 3D modeling. It can shoot at any time during the flight. After the drone flies to a pre-set special node in the flight path, the operation of the control handle is granted for a limited time. Users can take pictures of this node from multiple angles within the airspace according to the electronic fence set by the system, and can switch between different display modes built into the MR device to switch or overlay virtual scenes and displayed images. After the flight route is completed, the drone automatically returns to the drone bay for charging, and the video recording of the entire flight and the photos taken by the user are uploaded to the cloud server. After removing the head-mounted MR device and control handle, the user returns them to their original positions and exits the experience chamber. Once the system detects that the device is not damaged, the deposit is refunded. After the shared drone experience ends, users can download flight videos and photos via a mobile app.

10. The method for providing a mixed reality experience in scenic areas by combining shared drones according to claim 9, characterized in that, The flight speed is divided into 1-5 levels, and the drone defaults to speed level 3 after takeoff. Users can adjust the posture of the head-mounted MR device by rotating their heads, thereby adjusting the viewing angle; During the flight, the head-mounted MR device overlays a 3D model of the scenic area onto the captured real-world scene, and explains the distribution of attractions, roads, and historical context of the scenic area in real time through the highlighted display and flashing animation of the 3D model. Secondly, when the drone flies to a specific node, the system loads 3D reconstruction models from different periods onto the captured scene and provides audio commentary. During this process, the user can use a joystick or gestures to disassemble, enlarge, or reduce the 3D reconstruction model, and the head-mounted MR device will explain the structural information of the corresponding local area.

Images