Enhanced functionality for unmanned aerial vehicles including audio synchronization and geo-fencing

Abstract

The function of an unmanned aerial vehicle (UAV) may depend on location. In some embodiments, the digital audio communication channel is synchronized and multiplexed with a high definition video stream captured by a user-controlled remote UAV. The accompanying audio stream is used to provide supplemental and rich information to end users who are controlling the action of the remote UAV. The system can be used for providing virtualized tourism experience at hot places such as historical residence sites, wild animal protection areas and landmark tourism destinations. In some embodiments, a virtual three-dimensional geometry having a boundary surface typically referred to as a geo-fence and a concomitant control system restricts the flight path of a remotely driven UAV to an internal geometry.

Description

BACKGROUND

[0001] Quadcopters UAVs, commonly known as drones, are often equipped with high definition video cameras capable of transmitting captured digital video in real time to a remote control device. A system and method for transmitting real-time video streams over a wide-area-network (WAN) to a remote end user who can manipulate UAV controls including movement of the drone and orientation and focus of the attached imaging device is also described in PCT / US2021 / 032011. See International Application PCT / US2021 / 032011, filed May 12, 2021, the entirety of which is incorporated herein by reference.

[0002] Quadcopters are increasingly popular for both commercial and private use. As the frequency and volume of drone activity increases, airspace congestion is growing exponentially, leading to various adverse consequences such as increased risk of mid-air collisions and collateral ground damage to property and people, and increased noise disturbance to humans and wildlife. To mitigate these adverse consequences, national aviation authorities have enacted laws governing permissible airspace for drones. These restrictions are primarily focused on areas with high traffic of aircraft and their surroundings, noise sensitive areas such as national parks, and above large gatherings of people (i.e. public stadiums). As most drones are now equipped with imaging devices (cameras, etc.), aviation authorities have also restricted drone airspace within or around areas of national interest such as military bases, government buildings, utility plants, nuclear power plants, and dams. Furthermore, legal challenges involving invasion of privacy and copyright infringement are further driving drone airspace restrictions. SUMMARY

[0003] This summary introduces some concepts in a simplified form that will be further described in the detailed description below. This summary neither identifies key or essential features of the claimed subject matter nor limits its scope.

[0004] Various enhancements can be provided for unmanned aerial vehicles (UAVs), such as functions that rely on the location of the UAV, synchronization of audio data from other sources with real-time video streams from the UAV, or takeover of remote control of the motion of the UAV by another entity or system.

[0005] In some embodiments, a digital audio communication channel is synchronized and multiplexed with a video stream captured by a user-controlled remote unmanned aerial vehicle and transmitted by the UAV to provide a virtualized travel experience. The audio data provided by the digital audio communication channel can be provided by a live commentary by a person, or generated by a computing device, or selected by a computing device from recorded audio data based on the location of the UAV or based on information about a target within the live video stream from the UAV. The accompanying audio stream serves to provide supplemental and enriching information to the end user controlling the remote UAV actions. The system can be used to provide virtualized travel experiences at popular locations such as historical settlements, wildlife preserves, and iconic tourist destinations. A person narrating the live video stream from the UAV can take over control of the UAV from the end user.

[0006] In some embodiments, a three-dimensional spatial region defines a permissible airspace for navigating an unmanned aerial vehicle. The location of the UAV can be an input to the control system of the UAV and can be used to keep the UAV within the permissible airspace. In some embodiments, a virtual three-dimensional geometric shape having a bounding surface, commonly referred to as a geo-fence, and an accompanying control system restricts the flight path of a remotely controlled unmanned aerial vehicle ("UAV") to the interior geometry of the geo-fence. The control system can take over control of the UAV from the end user to ensure that the UAV remains within the interior geometry of the geo-fence.

[0007] Thus, in one aspect, a computing device is used in a system having a remotely controlled unmanned aerial vehicle. The unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device from the wireless network. The remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate control data for the unmanned aerial vehicle. The remote controller device transmits a video transmission based on the real-time imaging data over the wide area network. The computing device used in such a system includes a processing system that includes processing circuitry and a memory storing computer program instructions. The computer program instructions configure the computing device to track spatial coordinates of a geographic location of the unmanned aerial vehicle and generate information based on the geographic location of the unmanned aerial vehicle location for transmission to the end user computing device.

[0008] In an aspect, a method is for a system having a remotely controlled unmanned aerial vehicle. The unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network. The remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate control data for the unmanned aerial vehicle. The remote controller device transmits the real-time imaging data as a video transmission over the wide area network. The method used in such a system includes tracking spatial coordinates of a geographic location of the unmanned aerial vehicle and generating information based on the geographic location of the unmanned aerial vehicle location that is to be sent to the end user computing device.

[0009] In an aspect, a system includes an unmanned aerial vehicle, a remote controller device, an end user computing device, and an audio guide computing device. The unmanned aerial vehicle and the remote controller are in communication over a wireless network. The remote controller device, the end user computing device, and the audio guide computing device are in communication over a wide area network. The unmanned aerial vehicle includes an imaging sensor and a wireless device. The wireless device is connected to transmit real-time imaging data from the imaging sensor over the wireless network and to receive control data over the wireless network. The remote controller device includes a wireless device that is tuned to receive real-time imaging data from the unmanned aerial vehicle over the wireless network. The remote controller device is configured to transmit control data to the unmanned aerial vehicle over the wireless network and to transmit a video transmission based on the received real-time image data over the wide area network. The end user computing device includes one or more rendering devices. The end user computing device is configured to send instructions to the remote controller device to generate control data that is sent to the unmanned aerial vehicle. The audio guide computing device transmits supplemental audio data. The end user computing device receives the video transmission and the supplemental audio data that are synchronized and multiplexed over a communication channel of the wide area network and renders the received video transmission and the supplemental audio data through the one or more rendering devices.

[0010] In an aspect, a method includes transmitting, by an unmanned aerial vehicle, real-time imaging data from an imaging sensor of the unmanned aerial vehicle over a wireless network; receiving, by the unmanned aerial vehicle, control data over the wireless network; receiving, by a remote controller device, the real-time imaging data from the wireless network; transmitting, by the remote controller device, the control data over the wireless network; sending, by an end user computing device, instructions to the remote controller device over a wide area network to generate control data for the unmanned aerial vehicle; providing, by an audio guide computing device, supplemental audio data; synchronizing and multiplexing a video transmission based on the real-time imaging data and the supplemental audio data over a communication channel of the wide area network for transmission to the end user computing device over the communication channel; and rendering the video transmission and the supplemental audio data through one or more rendering devices of the end user computing device.

[0011] In one aspect, an audio guide computing device is used in a system having a remotely controlled unmanned aerial vehicle. The unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network. The remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate control data for the unmanned aerial vehicle. The remote controller device transmits a video transmission based on the real-time imaging data over the wide area network. For this system, the audio guide computing device includes a processing system that includes processing circuitry and a memory storing computer program instructions. The computer program instructions configure the audio guide computing device to provide supplemental audio data and synchronize and multiplex the supplemental audio data with the video transmission over a communication channel of the wide area network for transmission to the end user computing device.

[0012] In one aspect, a method is provided for use in a system having a remotely controlled unmanned aerial vehicle. The unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network. The remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate control data for the unmanned aerial vehicle. The remote controller device transmits a video transmission based on the real-time imaging data over the wide area network. The method used in such a system includes providing supplemental audio data and synchronizing and multiplexing the supplemental audio data with the video transmission over a communication channel of the wide area network for transmission to the end user computing device.

[0013] In an aspect, a system includes an unmanned aerial vehicle, a remote controller device, an end user computing device, and a geo-fence computing device. The unmanned aerial vehicle and the remote controller device communicate over a wireless network. The remote controller device, the end user computing device, and the geo-fence computing device communicate over a wide area network. The unmanned aerial vehicle includes an imaging sensor and a wireless device. The wireless device is connected to transmit real-time imaging data from the imaging sensor over the wireless network and to receive control data over the wireless network. The remote controller device includes a wireless device tuned to receive real-time imaging data from the unmanned aerial vehicle over the wireless network. The remote controller device transmits control data to the unmanned aerial vehicle over the wireless network and transmits a received video transmission based on the real-time imaging data over the wide area network. The end user computing device includes one or more rendering devices. The end user computing device is configured to send instructions to the remote controller device to generate control data for transmission to the unmanned aerial vehicle. The geo-fence computing device creates a representation of a three-dimensional spatial region based on a union of polygonal prisms to define an interior space, an exterior space, and a boundary between the interior space and the exterior space as a geo-fence and tracks spatial coordinates of a geographic location of the unmanned aerial vehicle. The geo-fence computing device determines a spatial relationship between the geographic location of the unmanned aerial vehicle location and the geo-fence.

[0014] In an aspect, a method includes: transmitting, by an unmanned aerial vehicle, real-time imaging data from an imaging sensor of the unmanned aerial vehicle over a wireless network; receiving, by the unmanned aerial vehicle, control data over the wireless network over the wireless network; receiving, by a remote controller device, the real-time imaging data from the wireless network; transmitting, by the remote controller device, the control data over the wireless network; sending, by an end user computing device, instructions to the remote controller device over a wide area network to generate control data for the unmanned aerial vehicle; creating a representation of a three-dimensional spatial region based on a union of polygonal prisms to define an interior space, an exterior space, and a boundary between the interior space and the exterior space as a geo-fence; tracking spatial coordinates of a geographic location of the unmanned aerial vehicle; and determining a spatial relationship between the geographic location of the unmanned aerial vehicle location and the geo-fence.

[0015] In an aspect, a geofence computing device is for a system having a remotely controlled unmanned aerial vehicle. The unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network. The remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate control data for the unmanned aerial vehicle. The remote controller device transmits a video transmission based on the real-time imaging data over the wide area network. The geofence computing device includes a processing system that includes processing circuitry and a memory storing computer program instructions. The computer program instructions configure the geofence computing device to create a representation of a three-dimensional spatial region based on a union of polygonal prisms to define an interior space, an exterior space, and a boundary between the interior space and the exterior space as a geofence. The geofence computing device also tracks spatial coordinates of a geographic location of the unmanned aerial vehicle. The geofence computing device also determines a spatial relationship between the geographic location of the unmanned aerial vehicle location and the geofence.

[0016] In an aspect, a method is provided for a system having a remotely controlled unmanned aerial vehicle. The unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network. The remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate control data for the unmanned aerial vehicle. The remote controller device transmits a video transmission based on the real-time imaging data over the wide area network. The method includes creating a representation of a three-dimensional spatial region based on a union of polygonal prisms to define an interior space, an exterior space, and a boundary between the interior space and the exterior space as a geofence. The method includes tracking spatial coordinates of a geographic location of the unmanned aerial vehicle. The method includes determining a spatial relationship between the geographic location of the unmanned aerial vehicle location and the geofence.

[0017] In one aspect, a geofencing computing device is provided for a system having a remotely controlled unmanned aerial vehicle (UAV). The UAV includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor via a wireless network and receives control data from a remote controller device via the same wireless network. The remote controller device is connected to a wide area network (WAN) and receives instructions from an end-user computing device to generate control data for the UAV. The remote controller device transmits the real-time imaging data as video transmission via the WAN. The geofencing computing device includes a processing system comprising processing circuitry and a memory storing computer program instructions. The computer program instructions configure the geofencing computing device to create a representation of a three-dimensional spatial region, which defines an internal space, an external space, and the boundary between the internal and external spaces as a geofence. Based on the three-dimensional coordinates of the remote controller device's geographic location and an elevation map of the geographic terrain of the geofence region, a minimum ground clearance value for each point within the geofence is determined through viewpoint analysis. The geofencing computing device tracks the three-dimensional spatial coordinates of the UAV's geographic location. The geofencing computing device determines the spatial relationship between the UAV's geographic location and the geofence.

[0018] In one aspect, a method is provided for a system having a remotely controlled unmanned aerial vehicle (UAV). The UAV includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor via a wireless network and receives control data from a remote controller device via the same wireless network. The remote controller device is connected to a wide area network (WAN) and receives instructions from an end-user computing device to generate control data for the UAV. The remote controller device transmits the real-time imaging data as video transmission via the WAN. The method includes creating a representation of a three-dimensional spatial region that defines an internal space, an external space, and the boundary between the internal and external spaces as a geofence. Based on 1) the three-dimensional coordinates of the remote controller device's geographic location and 2) an elevation map of the geographic terrain of the geofence region, a minimum ground clearance value for each point within the geofence is determined through viewpoint analysis. The method includes tracking the three-dimensional spatial coordinates of the UAV's geographic location. The method also includes determining the spatial relationship between the UAV's geographic location and the geofence.

[0019] In one aspect, a computing device is provided for a system having a remotely controlled unmanned aerial vehicle (UAV). The UAV includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor via a wireless network and receives control data from a remote controller device via the same wireless network. The remote controller device is connected to a wide area network (WAN) and receives instructions from an end-user computing device to generate control data for the UAV. The remote controller device transmits video based on the real-time imaging data via the WAN. The computing device includes a processing system comprising processing circuitry and a memory storing computer program instructions. The computer program instructions configure the computing device to generate text describing the content of the video transmission based on a large language model. The computing device uses a text-to-speech engine to generate audio data corresponding to the generated text. Supplemental audio data is synchronized with and multiplexed to the video transmission over a communication channel of the WAN before being transmitted to the end-user computing device.

[0020] In one aspect, a method is provided for a system having a remotely controlled unmanned aerial vehicle (UAV). The UAV includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor via a wireless network and receives control data from a remote controller device via the same wireless network. The remote controller device is connected to a wide area network (WAN) and receives instructions from an end-user computing device to generate control data for the UAV. The remote controller device transmits video based on the real-time imaging data via the WAN. The method includes generating text describing the content of the video transmission using a large language model. The method also includes generating audio data corresponding to the generated text using a text-to-speech engine. Finally, the method includes synchronizing and multiplexing the audio data with the video transmission over a communication channel on the WAN for transmission to the end-user computing device.

[0021] Any of the foregoing aspects may include one or more of the following features: The unmanned aerial vehicle is a quadcopter. A low-Earth orbit satellite provides a direct, wireless link from the remote controller device to the unmanned aerial vehicle. A cellular tower provides a direct wireless link between the remote controller device and the unmanned aerial vehicle. The remote controller device is configured to act as a bridge between a wireless network and a wide area network.

[0022] Any of the foregoing aspects may include one or more of the following features: One or more supplementary audio guide computing devices are connected to a wide area network, wherein each audio guide computing device provides its own supplementary audio data for synchronization and multiplexing with video transmissions from the unmanned aerial vehicle. The supplementary audio data includes live audio data of the audio guide narration captured by the microphone of the audio guide computing device. The corresponding supplementary audio data transmitted by one or more supplementary audio guide computing devices contains data from local files on the supplementary audio guide computing devices. The transmission of the supplementary audio data is triggered by the geographic location of the unmanned aerial vehicle. A remote controller device includes an audio guide computing device.

[0023] Any of the foregoing aspects may include one or more of the following features: The artificial intelligence system is configured to use real-time imaging data to identify image features of a target in the location of the unmanned aerial vehicle (UAV). The artificial intelligence system is configured to acquire control of the UAV to follow an object having the identified image features. The artificial intelligence system is configured to acquire control of the UAV and imaging sensors to focus on the target having the identified image features. The artificial intelligence system is configured to embed graphic overlays on video transmission based on the target having the identified image features. The artificial intelligence system is configured to trigger the transmission of audio information associated with the target having the identified image features.

[0024] Any of the foregoing aspects may include one or more of the following features. The artificial intelligence system or audio-guided computing device may include a large language model configured to generate text describing the content of the video transmission. A text-to-speech engine generates audio corresponding to the generated text, wherein supplementary audio data includes the generated audio.

[0025] Any of the foregoing aspects may include one or more of the following features: Generating notifications to end-user computing devices based on the determined spatial relationship between the geographic location of the unmanned aerial vehicle and the geofence.

[0026] In any of the foregoing, the minimum ground clearance of each point within the geofence can be determined by view analysis based on 1) the three-dimensional coordinates of the geographic location of the remote controller device and 2) an elevation map of the geographic terrain of the geofence area.

[0027] Any of the foregoing aspects may include one or more of the following features. Determining spatial relationships includes determining whether the UAV's geographic location is in external space or near a geofence. Generating notifications includes storing minimum threshold times and minimum threshold distances, and waking up a notification based on either (i) the distance from the UAV to the geofence is less than or equal to the minimum threshold distance or (ii) the predicted time for the UAV to reach the geofence is less than or equal to the minimum threshold time. The minimum threshold times and minimum threshold distances used to wake up the notifications are values ​​calculated based on the UAV's kinematic dynamics. The minimum threshold distance is based on the UAV's maximum deceleration in the direction perpendicular to the geofence. The minimum threshold time is based on the UAV's maximum deceleration in the direction perpendicular to the geofence.

[0028] Any of the foregoing aspects may include one or more of the following features. The notification includes a graphical representation superimposed on a video transmission from the remote controller device to the end-user computing device.

[0029] Any of the foregoing aspects may include one or more of the following features: Causing the remote controller device to send control data to the unmanned aerial vehicle to decelerate in a direction away from the geofence, thereby avoiding navigation into outer space. Triggering the remote controller upon waking a notification.

[0030] Any of the foregoing aspects may be implemented as a process performed by a computer system, any individual component of the computer system, the computer system, or any individual component of the computer system, or include an article of writing computer memory storing computer program code that, when processed by one or more computer processing systems, configures the one or more computer processing systems to provide the computer system or any individual component of the computer system, or to implement the method.

[0031] The following detailed description refers to the accompanying drawings, which form part of this application, and illustrate specific exemplary embodiments by way of illustration. Other embodiments can be implemented without departing from the scope of this disclosure. Attached Figure Description

[0032] Referring now to the accompanying drawings, which are not necessarily drawn to scale. Some embodiments may include fewer or more components than those shown in the figures.

[0033] Figure 1 A schematic diagram of a virtual tour system that provides audio and video narration for a remote destination, controlled by an end-user drone.

[0034] Figure 2A block diagram of an example circuit implemented as a computing device is shown, which can perform various operations according to some example embodiments described herein.

[0035] Figure 3 A schematic diagram of a virtual tour system that provides audio and video narration for a remote destination, controlled by an end-user drone.

[0036] Figure 4 The geometric interpretation of voxels or rectangular prisms used to create the interior regions is shown.

[0037] Figure 5 The geometry of multiple voxels is shown as an example of the permissible airspace for UAV navigation above the terrain.

[0038] Figure 6 An example image is shown showing the overlay of graphics on the drone as it approaches the boundary voxel edge.

[0039] Figure 7 A color-coded topographic map is shown, which illustrates an example geofence.

[0040] Figure 8 This is a sample output image of the addViewshedPath function, where the orange line represents the minimum AGL generated in that radial direction. Detailed Implementation

[0041] It can provide various enhancements to unmanned aerial vehicles (UAVs), such as location-dependent functions, synchronization of audio data from other sources with real-time video streams from the UAV, or remote control of the UAV's movement by another entity or system.

[0042] The first thing to address is audio synchronization.

[0043] Quadcopter UAVs (also known as drones) can be equipped with cameras capable of transmitting captured digital video in real time to a remote controller device. As described in PCT / US2021 / 032011, the real-time video stream can be transmitted from the UAV to a remote end-user via a wide area network (WAN), which is incorporated herein by reference. The camera can be a high-definition camera. The remote end-user can manipulate the UAV controls, including the movement of the drone and the orientation and focusing of the connected imaging device. A digital audio communication channel is synchronized with and multiplexed to the video stream captured by the user-controlled remote UAV. An accompanying audio stream is used to provide supplementary and enriching information to the end-user controlling the remote UAV's actions. This system can be used to provide virtualized tourism experiences at popular locations such as historic settlements, wildlife sanctuaries, and iconic tourist destinations.

[0044] By providing one or more accompanying audio guides with real-time narration, end users can be guided to a specific region of interest, given supplemental information about the UAV's flight location, and able to engage in informational dialogue with one or more audio guide experts who also have access to the UAV's video transmission. Furthermore, the accompanying audio guides can temporarily control the UAV's movement and its imaging device to effectively direct the end user's attention to the region of interest. In another embodiment, stored audio files can be transmitted while the UAV is flying near the region of interest, or the audio transmission can be triggered by an AI image recognition system after identifying relevant image features in the video transmitted by the UAV. This invention is not limited to audio data, as other media data can be generated or provided based on the UAV's location.

[0045] Figure 1 An example system is described, depicting an audio-visual narrated virtual tour of a remote destination navigated by an end-user-controlled drone. 100 is an unmanned aerial vehicle (UAV) equipped with a camera (preferably high-definition) and a device for transmitting the captured video data via a wireless network.

[0046] 101 is a remote controller device that communicates with the UAV (100) and acts as a bridge between the UAV (100) and the wide area network (102). 102 is the wide area network, commonly referred to as the Internet. The management system 103 is a network-in-network component that manages, supplies, and stores the necessary data and credentials used for the operation of the system. For example, as described below, the management system 103 may include an AI system 106 or may exchange data with the AI ​​system 106 for identifying digital sound and image features in the data transmitted by the UAV.

[0047] 104 is an end-user computing device, such as a PC or mobile device, connected to the wide area network 102. This computing device 104 executes a software application that sends user control information to the UAV 100. The end-user computing device 104 can access audio communication channels and receive video streams from the UAV 100, all through an agent provided by the remote controller device 101. Multiple end-user computing devices may be provided.

[0048] 105 is an audio guide computing device connected to a wide area network 102 via a network. This computing device executes a software application that sends supplemental audio data to an end-user computing device 104, a management system 103, and a remote controller device 101. The audio guide computing device also receives data from the management system 103 via the WAN 102. In some embodiments, multiple interconnected audio guide computing devices 105 may be used.

[0049] One type of unmanned aerial vehicle (UAV) is the quadcopter drone. The use of drones by businesses and consumers has become commonplace. Most drones are equipped with digital imaging devices that generate high-definition digital images and videos. These digital images can be stored on the drone's onboard storage unit and, in most implementations, can be transmitted back to a remote controller via a wireless network link. Commercial entities can use these drones and their imaging capabilities in hazardous or physically challenging environments, such as investigating spreading forest fires, inspecting gas pipeline leaks, or proactively monitoring and tracking wildlife poachers.

[0050] In some implementations, the remote controller device acts as a bridge between the wireless network and the wide area network (WAN) to which the drone is connected. In this case, real-time image data transmitted by the drone is transmitted by the remote controller to a remote end-user device via the WAN proxy. The end-user can view the real-time video data from the remote device and send control information (such as flight path or changes in camera orientation and focus) back to the drone bridged by the remote controller.

[0051] An embodiment with an audio wizard extends the video experience transmitted by the drone by adding an audio communication channel over the WAN. This communication channel can be accessed by a remote controller device, an end-user device, and one or more other computing devices primarily used for broadcasting supplemental audio data that is synchronized with and multiplexed to the video stream transmitted by the drone.

[0052] When the computing device is used by an audio guide, this configuration enables virtual tour scenarios for end users, who can assist and educate them at various high-value points of interest along the drone's flight path. In this configuration, the end user can also communicate with a person operating a remote control unit, which can 1) participate as an audio guide during flight, 2) act as a remote pilot to regain control of the drone in emergencies, and 3) act as a visual observer to convey local information such as proximity to other aircraft, flocks of birds intruding into the drone's airspace, or weather fronts moving into the area.

[0053] In the same configuration, in some implementations, the device used by the audio guide provides priority access to drone control. This allows the audio guide to remotely control the drone's movement and its imaging devices, such as camera orientation and lens magnification. One use case for this configuration is when the audio guide has detected fast-moving animals or highly camouflaged animals in dense vegetation and wishes to guide the video experience to these high-value areas of interest.

[0054] In some implementations, audio guidance communication is replaced by background-relevant, pre-recorded audio files stored on the audio guidance device. Each of the multiple pre-recorded audio files corresponds to information related to a specific point of interest within the geofence boundary of the drone's flight path. When the drone's flight path geographically approaches the point of interest, the audio guidance device transmits the geographically matched pre-recorded audio stream to the end-user device via the WAN.

[0055] Furthermore, these points of interest can be determined during flight by an AI-based image processing system. This AI-based image processing system uses a pre-trained object detection model with specified target categories (e.g., various animals). The system processes each captured video frame to map image features to target categories with a certain confidence factor. If the confidence factor exceeds a predefined threshold, the system generates a notification that the image features match a specific target category. In cases where the image signature matches multiple target categories, the system selects the target category with the highest confidence to identify the target. An example of this algorithm written in Python using OpenCV (an open-source computer vision and machine learning software library) and the YOLO library for object detection is shown below:

[0056]

[0057]

[0058] After the AI ​​image processing system identifies image features with target categories in the drone's streaming video transmission, the AI ​​system can 1) navigate the drone to the identified region of interest, 2) control the drone's camera to focus on the identified region of interest, 3) embed graphic overlays in the video transmission to assist in identifying the region of interest, and 4) trigger the sending of a background-related pre-recorded audio stream to the end user.

[0059] For example, in drone video transmission, the AI ​​system can identify a group of sleeping lions hidden in dense bushes. The AI ​​system guides the drone to hover over the sleeping lions, focuses the camera on the pride, highlights the pride graphically in the video transmission to help end users identify it, and sends a pre-recorded audio stream describing the lions' sleep cycle.

[0060] In some configurations, the human audio guidance narration provided by the audio guidance computing device is replaced by a large language model (LLM) combined with a text-to-speech engine. In some implementations, the LLM-based narration is triggered by the aforementioned AI system for recognizing image features in drone streaming video transmissions. LLM narration and text-to-speech generation can be implemented on a computing device supporting AI system 106, a computing device supporting audio guidance computing device 105, or other computing devices.

[0061] To enable the provision of Large Language Model (LLM) commentaries based on an AI-based object classification model, the following Python code demonstrates an implementation using the PYTTSx3 text-to-speech library, OpenAI's LLM, the OpenCV machine learning library, and the YOLO library for object detection:

[0062]

[0063]

[0064] In all of the above configurations, the end user receives a virtual tour experience with video narration.

[0065] Given that audio and video streams can be created from a variety of sources, any of the aforementioned devices (remote control units, AI systems, audio wizard computing devices, LLM-generated narration and text-to-speech systems, and end-user devices), or dedicated devices in public or private cloud infrastructure components connected to a WAN, such as... Figure 1 The central system management server (103) described herein can be used to multiplex various streams into a single file for local storage. The archived file digitally preserves the end-user's audio and video experience, which can be downloaded over the WAN to the end-user's device for offline playback and / or downloaded (shared) to other devices approved by the end-user.

[0066] Drone service providers offer drones for remote navigation in tourist destinations, wildlife reserves, historical sites, and other locations. The following components are used to provide supplementary audio accompaniment synchronized with the UAV video transmission:

[0067] 1. A drone equipped with an imaging sensor (e.g., a high-definition camera) and wireless devices, the wireless devices being used to transmit imaging data captured by the drone and to receive control data used for directional imaging sensor and drone navigation information.

[0068] 2. Remote Controller Devices: For remote control devices that communicate with drones using wireless devices operating in unlicensed frequency bands (ISM) (e.g., 2.4 GHz and 5.8 GHz frequencies), most countries have regulations limiting the effective isotropic radiated power (EIRP) of transmitted wireless signals to minimize interference with other wireless devices operating in the same ISM band. EIRP restrictions mean that the effective spatial range of wireless transmission and reception from the remote controller to the drone is limited; that is, signal noise saturates the communication link, rendering communication and control ineffective. To extend the communication spatial range, the remote controller can connect to a wide area network (WAN) and can support network bridging between the wireless network and the WAN. If the drone's wireless device uses another frequency band with less stringent EIRP restrictions, the drone can connect directly to the WAN, for example, via low-Earth orbit satellites or remote cellular towers.

[0069] 3. End-user computing device: The end user can use a computing device (such as a PC or mobile device) connected to the WAN to execute a software application that sends control information to the drone, can access audio communication channels, and can receive video stream data from the drone. All network data traffic is bridged by the aforementioned remote controller device.

[0070] 4. Remote Audio Guide Computing Device (one or more): One or more computing devices connected to the WAN that can access the drone's video transmission via an application, remotely control the drone's movement and camera, and access communication channels connected to end-user computing devices and remote controllers.

[0071] The geofencing issue for UAVs will now be addressed.

[0072] The permitted airspace or internal geometry of a drone can be defined in three-dimensional space and bounded by a virtual two-dimensional extent, referred to as a virtual geofence. In embodiments involving a three-dimensional geometry consistent with the generally defined geometry of controlled airspace, and for other simply connected convex boundaries, the internal geometry can be approximated by the union of multiple polygonal prisms. One specific implementation of these polygonal prisms is a rectangular prism, commonly referred to as a voxel. The simple geometry of the voxel is useful when determining which polygonal prism contains the drone. If the drone's location is inside one of the polygonal prisms, it is considered to be within the permitted airspace; similarly, if the drone is not located within one of the polygonal prisms, it is considered to be in the restricted external area.

[0073] In an embodiment that introduces a geofence, the system and method 1) generate a virtualized geofence airspace defined as the range of the union of multiple polygonal prisms, 2) acquire flight control of the UAV to ensure that it is kept within the internal geometry (i.e., the permissible airspace), and 3) provide a graphical notification triggered by the UAV approaching the external geometry (i.e., the restricted airspace).

[0074] As UAV airspace becomes increasingly congested, national aviation authorities are implementing UAV airspace restrictions in order to 1) mitigate collisions with other aircraft and damage to people, wildlife and property; 2) reduce noise pollution in sensitive areas, such as near eagle nests; and 3) avoid flying over or around areas of significant national interest, such as military bases.

[0075] Other reasons for imposing airspace restrictions on drones include avoiding collisions with physical obstacles such as trees, mountains, and towers; reducing privacy violations caused by UAV image capture; and assisting UAV remote pilots in navigating near areas of interest, such as iconic waterfalls or wildlife drinking pools.

[0076] Figure 3 An example system is shown for a virtual tour of a remote destination with audio-visual narration, controlled by an end-user drone. 300 is an unmanned aerial vehicle equipped with a camera (preferably a high-definition camera) and means for transmitting the captured video data via a wireless network.

[0077] 301 is a remote controller device that communicates with the UAV (300) and acts as a bridge between the UAV (300) and the wide area network (302). 302 is the wide area network, commonly referred to as the Internet. The management system 303 is a network-in-network component that manages, supplies, and stores the necessary data and credentials used for the operation of the system. The management system may include or may exchange data with a geofencing system 306, as described below. The geofencing system is used to 1) store representations of external, internal, and boundary areas for navigation of the UAV (300), 2) track the spatial coordinates of the unmanned aerial vehicle, 3) issue a notification if the UAV's location is in the outer space or near a geofence, 4) generate graphic warnings superimposed on video transmitted by the UAV, and 5) send control information to the UAV to avoid navigation through the geofence.

[0078] 304 is an end-user computing device, such as a PC or mobile device, connected to the wide area network 302. This end-user computing device executes a software application that sends user control information to the UAV 300. The end-user computing device can access audio communication channels and receive video streams from the UAV 300, all of which is achieved through a remote controller device 301 acting as an agent. Multiple end-user computing devices may be present.

[0079] 305 is an audio guide computing device connected to a wide area network 302. This audio guide computing device executes a software application that sends supplementary audio data to an end-user computing device 304, a management system 303, and a remote controller device 301. The audio guide computing device also receives data from the management system 303 via WAN 302. In some embodiments, multiple interconnected audio guide computing devices 305 may be used.

[0080] Figure 4 The geometric interpretation of the voxels or rectangular prisms used to create the interior region is shown. 400 represents a voxel with a top voxel base plane (401) and a bottom voxel base plane (402). 402 is a point represented by a pair of tuples, i.e., latitude and longitude coordinates, which is the projection of the voxel vertex onto the two-dimensional surface.

[0081] Figure 5 The geometry of multiple voxels is shown as an example of permissible airspace for UAV navigation above the terrain. 500 is the side of the boundary voxel, with no other adjacent voxels on its side, and represents the area of ​​the geofence.

[0082] Figure 6 An example graphic (600) is shown for overlaying on the UAV (300) when the UAV (300) is close to the side of the boundary voxel.

[0083] One method for constructing a set of voxels to define the internal geometry is to: 1) generate a process sampling map that projects the internal geometry of a 3D convex shape onto the latitude and longitude of the Earth. This projection map generates a set of tuples, namely {(lat 1, long 1), (lat 2, long 2), etc.}, where each tuple can be associated with the minimum and maximum above-ground level (AGL) of a virtual geofence. Voxels are created by associating the tuples with their three nearest neighbors, where the minimum AGL is defined as the average of the minimum AGLs of the selected point and its three nearest neighbors. Similarly, the top datum plane of the voxels is created in a similar manner, except that it is calculated by averaging the maximum AGL of the four points. In spherical coordinates, the voxel vertices are defined as follows:

[0084] Bottom Voxel Base={(latl,longl,ave_min_agl),(lat2,long2,ave_min_agl),(lat3,long3,ave_min_agl),(lat4,long4,ave_min_alg)}

[0085] TopVoxel Base={(latl,longl,ave_max_agl),(lat2,long2,ave_max_agl),(lat3,long3,ave_max_agl),(lat4,long4,ave_max_agl)}

[0086] To determine whether a drone is within permitted airspace, the system checks if the drone's spatial coordinates are contained within one of the defined voxels. If the drone's coordinates are not within any voxel, the drone is considered to be flying outside permitted airspace. To define the virtual geofence, the concept of boundary voxel flanks is introduced. A boundary voxel flank is a flank that is not adjacent to another voxel. For example, in the current construction, the top and bottom base planes of each voxel are boundary voxel flanks.

[0087] Using the drone's current speed and direction, a method can be used to predict when the drone will travel into the external geometric space, i.e., the confined airspace, which means the drone plans to cross the boundary voxel side. If located at A = (LAT... a LONG a ALT a A drone with velocity vector V(A) approaches the side of a boundary voxel defined by four vertices:

[0088] P1 = (Lat1, Long1, Alt1); P2 = (Lat2, Long2, Alt1); P3 = (Lat1, Long1, Alt2); and P4

[0089] =(Lat2,Long2,Alt2)

[0090] A method can be used to calculate the distance a drone travels to a geofence and predict the time it takes to reach it. The first step is a preliminary process of calculating the distance from the drone (point A) to the side of the boundary voxel (as defined above by P1, P2, P3, and P4):

[0091] 1. Two vectors can be formed from these four coplanar points; for example... and

[0092] 2. The product of these two vectors generates the normal vector. That is, a vector perpendicular to the plane formed by these three points.

[0093] 3. Define a new vector This vector is a vector from the drone's position to one of the points defining the boundary, i.e. The distance between the drone's position and the given plane is a vector. At the unit normal vector The projection length on the vector is calculated by the dot product of the vectors.

[0094] The predicted time to reach the virtual geofence can be calculated by dividing the normal component of the velocity vector (the component of the velocity vector perpendicular to the boundary voxel side) by the calculated normal distance. Because the drone may have a positive velocity component that could intercept up to three additional boundary voxel sides, the prediction time calculation needs to be performed on those boundary voxel sides where the drone has a positive velocity component in the direction of the boundary's normal vector. The predicted time to reach the virtual geofence is the minimum predicted time associated with reaching any single boundary voxel side.

[0095] A boundary threshold time can be defined and used as a benchmark for triggering alarm notifications. For example, an alarm can be generated if the predicted time to reach the virtual geofence is less than the defined threshold time. In another implementation, the boundary threshold time can be dynamically calculated based on the drone speed perpendicular to the side of the boundary voxel, the drone's maximum deceleration, and the pilot's response time.

[0096] To avoid crossing a geofence, the drone pilot must change the drone's speed perpendicular to the side of the boundary voxel to zero (stopping it from moving towards the boundary) or a negative value (changing direction away from the boundary). For example, if the normal distance from the drone to the side of the boundary voxel is D... drone Its velocity perpendicular to the side of the boundary voxel is V. drone And the maximum drone deceleration is Deacc drone Then notify the threshold time T threshold and the time T to reach the virtual geofence geofence yes:

[0097] T threshold =V drone / Deacc drone +Driver reaction time

[0098] T geofence =V drone / D done ,

[0099] If T threshold ~T geofence The system will issue a notification warning the remote driver that they are about to cross the geofence unless immediate action is taken.

[0100] In some embodiments, the notification method is based on a threshold distance, using the following formula to convert the threshold time T... threshold Defined as distance:

[0101] D threshold =V 2drone / (2*Deacc drone) )

[0102] If D threshold ~D drone If this occurs, a system notification will be generated, warning of an impending airspace violation unless corrective action is taken immediately.

[0103] To notify drivers that they are about to cross a virtual geofence, an alarm can take the form of a loud warning signal sent to the driver's remote control device and / or a graphic image of a virtual grid or "cage" superimposed on the video displayed on the driver's remote control device.

[0104] To eliminate the variability of pilot reaction time in response to intrusions into restricted airspace, the system can control the drone's navigation to change its direction away from the geofence or reduce its speed, thereby allowing it to come to a smooth stop at the geofence and await control from a remote drone pilot to safely leave the geofence.

[0105] The upper limit of drone altitude is typically stipulated by government aviation authorities as a fixed maximum altitude above ground (AGL). Therefore, in practice, the top voxel datum AGL is usually set to have this maximum altitude. The lower voxel datum AGL is determined by line-of-sight analysis between the drone and the remote controller unit. To maintain continuous communication, the wireless transmission link between the drone and the remote controller unit must be free of obstacles, such as physical obstructions due to terrain changes or disruptive signal interference due to terrain reflections and Fresnel zone-based calculations.

[0106] The lower voxel datum plane (AGL) is determined through view domain analysis based on 1) the location of the remote controller unit (x0, y0, h) and 2) a topographic map of the entire geofence area providing an elevation map H(x, y). The output of this analysis is a minimum AGL defined for each point within the geofence.

[0107] In the code below, frm = (x0, y0) is the location of the remote controller unit; to = (x, y) is the boundary of the geofence in a specific angular direction; dem is a matrix containing the terrain elevation map H(x, y) between frm and to; CLEARANCE is the minimum ground offset to avoid obstacles such as trees and buildings; RADIO_ELEV is h, the height of the remote control unit.

[0108]

[0109]

[0110] `output_array` is a sequence of points along the radial direction between the remote controller and the sampled geofence boundary. For example, in... Figure 7 In the topographic map 700, the endpoints of the red line 702 applied to the topographic map 700 correspond to the edges of the remote controller location and the geofence boundary. Figure 8 This is a graph 800 showing the output of the addViewshedPath function, where the orange line 802 represents the minimum AGL generated for that radial direction. To create a complete lower AGL map of the geofence's interior, addViewshedPath needs to be called with additional samples of the geofence boundaries.

[0111] Example Management System

[0112] Figure 1 and Figure 3 The various components, namely the management system 103, remote controller device 101, UAV 100, AI system 106, audio guide computing device, end-user computing device 104, management system 303, remote controller device 301, UAV 300, geofencing system 306, audio guide computing device 305, and end-user computing device 304, can be implemented by one or more computing devices, processing systems, or servers, such as... Figure 2 The device 200 is shown in the figure.

[0113] like Figure 2 As shown, device 200 may include processing circuitry 202, memory 204, and communication hardware 206, which will be described in more detail below. Although the various components are only... Figure 2 The device is shown as being connected to processing circuitry 202, but it should be understood that device 200 may also include a bus for transmitting information between any combination of the various components of device 200. Figure 2 (Not explicitly shown in the text). Device 200 can be configured to execute the above combination using computer program instructions. Figure 1 and Figures 3 to 6 And the various operations described above.

[0114] Processing circuitry 202 (and / or a coprocessor or auxiliary processor or any other processor otherwise associated with the processor) can communicate with memory 204 via a bus for transferring information between components of the device. Processing circuitry 202 can be implemented in several different ways, for example, it may include one or more processing devices configured to execute independently. Furthermore, the processor may include one or more processors configured in series via a bus to enable independent execution of software instructions, pipelined processes, and / or multithreaded processes. The term "processor" can be understood to include a single-core processor, a multi-core processor, multiple processors of device 200, a remote or "cloud" processor, or any combination thereof.

[0115] Processing circuitry 202 may be configured to execute software instructions stored in memory 204 or accessible to the processor (e.g., software instructions stored on a separate storage device (not shown)). In some cases, the processor may be configured to execute hard-coded functions. Thus, whether configured by hardware or software methods, or by a combination of hardware and software, processing circuitry 202 represents an entity (e.g., physically contained in circuitry) capable of performing the operations described herein when appropriately configured. Alternatively, as another example, when processing circuitry 202 is implemented as an executor of software instructions, the software instructions may specifically configure processing circuitry 202 to perform the algorithms and / or operations described herein upon execution of the software instructions.

[0116] Memory 204 is non-transitory and may include, for example, one or more volatile and / or non-volatile memories. In other words, memory 204 may be, for example, an electronic storage device (e.g., a computer-readable storage medium). Memory 204 may be configured to store information, data, content, applications, software instructions, etc., to enable the device to perform various functions according to the exemplary embodiments contemplated herein.

[0117] Communication hardware 206 can be any device, such as an apparatus or circuit implemented in hardware or a combination of hardware and software, configured to receive data from or send data to a network and / or any other device, circuit, or module communicating with device 200. In this regard, communication hardware 206 may include, for example, a network interface for enabling communication with wired or wireless communication networks. For example, communication hardware 206 may include one or more network interface cards, antennas, buses, switches, routers, modems, and supporting hardware and / or software, or any other apparatus suitable for enabling communication over a network. Furthermore, communication hardware 206 may include processing circuitry for transmitting these signals to the network or for processing received signals from the network.

[0118] Communication hardware 206 may be configured to provide output to a user and, in some embodiments, receive indications of user input. Communication hardware 206 may include a user interface, such as a display, and may also include components for managing the use of the user interface, such as a web browser, mobile application, dedicated user equipment, etc. In some embodiments, communication hardware 206 may include a keyboard, mouse, touchscreen, touch area, softkeys, microphone, display, speaker, or other input device, output device, or presentation device. Communication hardware 206 may utilize processing circuitry 202 to control one or more functions of one or more of these user interface elements via software instructions (e.g., application software and / or system software, such as firmware) stored in memory accessible to processing circuitry 202 (e.g., memory 204).

[0119] in conclusion

[0120] Benefiting from the teachings in the foregoing description and the associated drawings, those skilled in the art will conceive of many modifications and other embodiments of the invention set forth herein. Therefore, it should be understood that the invention is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Furthermore, although the foregoing description and associated drawings have described exemplary embodiments in the context of certain example combinations of elements and / or functions, it should be understood that alternative embodiments may provide different combinations of elements and / or functions without departing from the scope of the appended claims. In this regard, for example, combinations of elements and / or functions different from those explicitly described above, as set forth in some of the appended claims, are also contemplated. Although specific terminology is used herein, it is used only in a general and descriptive sense and not for limiting purposes.

[0121] What is being protected is:

Claims

1. A system comprising: an unmanned aerial vehicle comprising an imaging sensor and a wireless device, wherein the wireless device is connected to transmit real-time imaging data from the imaging sensor over a wireless network, and wherein the wireless device is connected to receive control data from the wireless network; a remote controller device comprising a wireless device tuned to receive real-time imaging data from the unmanned aerial vehicle over the wireless network; wherein the remote controller device is further configured to transmit the control data to the unmanned aerial vehicle over the wireless network, and to transmit a video transmission based on the received real-time image data over a wide area network; an end user computing device connected to the wide area network and comprising one or more rendering devices, wherein the end user computing device is configured to transmit instructions to the remote controller device to generate the control data for transmission to the unmanned aerial vehicle; and an audio guide computing device connected to the wide area network and configured to transmit supplemental audio data, wherein the audio guide computing device is configured to provide the supplemental audio data; and wherein the end user computing device is configured to receive the video transmission and the supplemental audio data synchronized and multiplexed over a communication channel of the wide area network, and to render the received video transmission and supplemental audio data through the one or more rendering devices.

2. The system of claim 1, wherein, the unmanned aerial vehicle is a quadcopter drone.

3. The system of claim 1, further comprising one or more additional audio guide computing devices connected to the wide area network, wherein each audio guide computing device provides respective supplemental audio data to be synchronized and multiplexed with the video transmission from the unmanned aerial vehicle.

4. The system of claim 3, wherein, the respective supplemental audio data transmitted by the one or more additional audio guide computing devices comprises data from local files on the additional audio guide computing devices, and wherein the transmission of the supplemental audio data is triggered by a geographic location of the unmanned aerial vehicle.

5. The system of claim 4, wherein, the remote controller device incorporates the audio guide computing device.

6. The system of claim 1, wherein, the supplemental audio data is based on data related to a geographic location of the unmanned aerial vehicle.

7. The system of claim 1, further comprising a low earth orbit satellite providing a direct wireless link from the remote controller device to the unmanned aerial vehicle.

8. The system of claim 1, further comprising a cellular tower providing a direct wireless link between the remote controller device and the unmanned aerial vehicle.

9. The system of claim 1, wherein, the remote controller device is configured to function as a bridge between the wireless network and the wide area network.

10. The system of claim 1, further comprising an artificial intelligence system configured to: identify image features of a target in a location of the unmanned aerial vehicle using the real-time imaging data; and taking control of the unmanned aerial vehicle and the image sensor to focus on a target having the identified image feature.

11. The system of claim 1, further comprising an artificial intelligence system configured to: identify, using the real-time imaging data, an image feature of a target in a location of the unmanned aerial vehicle; and embed, based on the target having the identified image feature, a graphical overlay on the video transmission.

12. The system of claim 1, further comprising an artificial intelligence system configured to: identify, using the real-time imaging data, an image feature of a target in a location of the unmanned aerial vehicle; and trigger transmission of audio information associated with the target having the identified image feature.

13. The system of claim 10, wherein, The artificial intelligence system is further configured to embed, based on the target having the identified image feature, a graphical overlay on the video transmission.

14. The system of claim 13, wherein, The artificial intelligence system is further configured to trigger transmission of audio information associated with the target having the identified image feature.

15. The system of claim 10, wherein, The artificial intelligence system is further configured to trigger transmission of audio information associated with the target having the identified image feature.

16. A method comprising: transmitting, by an unmanned aerial vehicle, real-time imaging data from an imaging sensor of the unmanned aerial vehicle over a wireless network; receiving, by the unmanned aerial vehicle, control data from the wireless network over the wireless network; receiving, by a remote controller device, the real-time imaging data from the wireless network; transmitting, by the remote controller device, the control data over the wireless network; transmitting, by an end user computing device, instructions over a wide area network to the remote controller device to generate the control data for the unmanned aerial vehicle; providing, by an audio guide computing device, supplemental audio data; synchronizing and multiplexing, over a communication channel of the wide area network, a video transmission based on the real-time imaging data and the supplemental audio data for transmission to the end user computing device over the communication channel; and presenting, by one or more presentation devices of the end user computing device, the video transmission and the supplemental audio data. The unmanned aerial vehicle is a quadcopter drone.

17. The method of claim 16, wherein, 18. The method of claim 1, further comprising: providing, by one or more additional audio guide computing devices, a plurality of supplemental audio data streams to be synchronized and multiplexed with the video transmission over the communication channel.

19. The method of claim 16, further comprising triggering transmission of the supplemental audio data by a geographic location of the unmanned aerial vehicle. The remote controller device comprises the audio guide computing device.

20. The method of claim 16, wherein, 21. The method of claim 16, wherein the supplemental audio data is based on data related to a geographic location of the unmanned aerial vehicle. The wireless network comprises low earth orbit satellites providing a direct wireless link from the remote controller device to the unmanned aerial vehicle.

22. The method of claim 16, wherein, ​ 23. The method of claim 16, wherein, The wireless network includes a cellular tower that provides a direct wireless link between the remote controller device and the unmanned aerial vehicle.

24. The method of claim 16, wherein, The remote controller device is configured to act as a bridge between the wireless network and a wide area network.

25. The method of claim 16, further comprising, identifying, using the real-time imaging data, an image feature of a target in a location of the unmanned aerial vehicle; and taking control of the unmanned aerial vehicle and the image sensor to focus on the target having the identified image feature.

26. The method of claim 16, further comprising: identifying, using the real-time imaging data, an image feature of a target in a location of the unmanned aerial vehicle; and embedding a graphical overlay on the video transmission based on the target having the identified image feature.

27. The method of claim 16, further comprising: identifying, using the real-time imaging data, an image feature of a target in a location of the unmanned aerial vehicle; and triggering transmission of audio information associated with the target having the identified image feature.

28. The method of claim 25, further comprising: embedding a graphical overlay on the video transmission based on the target having the identified image feature.

29. The method of claim 29, further comprising: triggering transmission of audio information associated with the target having the identified image feature.

30. The method of claim 25, further comprising: triggering transmission of audio information associated with the target having the identified image feature.

31. The system of claim 1, further comprising: a large language model configured to generate text describing content of the video transmission; and a text-to-speech engine configured to generate audio corresponding to the generated text, wherein the supplemental audio data includes the generated audio.

32. The method of claim 16, further comprising: generating, using a large language model, text describing content of the video transmission; and generating, using a text-to-speech engine, audio corresponding to the generated text, wherein the supplemental audio data includes the generated audio.

33. An audio guide computing device for use in a system having a remotely controlled unmanned aerial vehicle, wherein the unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device transmits a video transmission based on the real-time imaging data over the wide area network, the audio guide computing device comprising a processing system including processing circuitry and memory storing computer program instructions that configure the audio guide computing device to provide supplemental audio data; and ​ synchronizing and multiplexing the supplemental audio data with the video transmission on a communication channel of the wide area network for transmission to the end user computing device.

34. A method for use in a system having a remotely controlled unmanned aerial vehicle, wherein, The unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device transmits a video transmission based on the real-time imaging data over the wide area network, the method comprising: providing supplemental audio data; and synchronizing and multiplexing the supplemental audio data with the video transmission on a communication channel of the wide area network for transmission to the end user computing device.

35. A computing device for use in a system having a remotely controlled unmanned aerial vehicle, wherein the unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device transmits a video transmission based on the real-time imaging data over the wide area network, the computing device comprising a processing system including processing circuitry and a memory storing computer program instructions that configure the computing device to: track spatial coordinates of a geographic location of the unmanned aerial vehicle; and generate information based on the geographic location of the unmanned aerial vehicle location for transmission to the end user computing device.

36. A method for use in a system having a remotely controlled unmanned aerial vehicle, wherein the unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device transmits a video transmission based on the real-time imaging data over the wide area network, the method comprising: tracking spatial coordinates of a geographic location of the unmanned aerial vehicle; and generating information based on the geographic location of the unmanned aerial vehicle location for transmission to the end user computing device.

37. A system comprising: an unmanned aerial vehicle including an imaging sensor and a wireless device, wherein the wireless device is connected to transmit real-time imaging data from the imaging sensor over a wireless network, and wherein the wireless device is connected to receive control data over the wireless network; ​ a remote controller device comprising a wireless device tuned to receive the real-time imaging data from the unmanned aerial vehicle over the wireless network; wherein the remote controller device is further configured to transmit the control data to the unmanned aerial vehicle over the wireless network, and to transmit a video transmission based on the received real-time imaging data over the wide area network; a terminal user computing device connected to the wide area network and comprising one or more rendering devices, wherein the terminal user computing device is configured to transmit instructions to the remote controller device to generate the control data for transmission to the unmanned aerial vehicle; and a geo-fence computing device connected to the wide area network and configured to: create a representation of a three-dimensional spatial region based on a union of polygonal prisms to define an interior space, an exterior space, and a boundary between the interior space and the exterior space as a geo-fence; track spatial coordinates of a geographic location of the unmanned aerial vehicle; and determine a spatial relationship between the geographic location of the unmanned aerial vehicle location and the geo-fence.

38. The system of claim 37, wherein, the unmanned aerial vehicle is a quadcopter drone.

39. The system of claim 37, wherein, the geo-fence computing device is further configured to generate a notification to the terminal user computing device based on the determined spatial relationship between the geographic location of the unmanned aerial vehicle and the geo-fence.

40. The system of claim 39, wherein determining the spatial relationship comprises determining whether the geographic location of the unmanned aerial vehicle is in the exterior space or proximate to the geo-fence.

41. The system of claim 39, wherein to generate the notification, the geo-fence computing device is configured to: store a minimum threshold time and a minimum threshold distance; raise a notification based on either of: (i) a distance of the unmanned aerial vehicle to the geo-fence being less than or equal to the minimum threshold distance; or (ii) a predicted time of the unmanned aerial vehicle reaching the geo-fence being less than or equal to the minimum threshold time.

42. The system of claim 41, wherein, the minimum threshold time and the minimum threshold distance used to raise the notification are values calculated based on kinematic dynamics of the unmanned aerial vehicle.

43. The system of claim 41, wherein, the minimum threshold distance is based on a maximum deceleration of the unmanned aerial vehicle in a direction perpendicular to the geo-fence.

44. The system of claim 41, wherein, the minimum threshold time is based on a maximum deceleration of the unmanned aerial vehicle in a direction perpendicular to the geo-fence.

45. The system of claim 39, wherein the notification comprises a graphical representation overlaid on the video transmission transmitted from the remote controller device to the terminal user computing device.

46. The system of claim 39, wherein, the geo-fence computing device, upon raising the notification, causes the remote controller device to transmit control data to the unmanned aerial vehicle to decelerate in a direction away from the geo-fence to avoid navigating into the exterior space.

47. The system of claim 37, wherein, The geo-fencing computing device causes the remote controller device to send control data to the unmanned aerial vehicle to decelerate in a direction away from the geo-fence to avoid navigating into the outside space.

48. A method comprising: sending, by an unmanned aerial vehicle, real-time imaging data from an imaging sensor of the unmanned aerial vehicle over a wireless network; receiving, by the unmanned aerial vehicle, control data from the wireless network over the wireless network; receiving, by a remote controller device, the real-time imaging data from the wireless network; sending, by the remote controller device, the control data over the wireless network; sending, by an end-user computing device, instructions to the remote controller device over a wide area network to generate the control data for the unmanned aerial vehicle; creating a representation of a three-dimensional spatial region based on a union of polygonal prisms to define an inside space, an outside space, and a boundary between the inside space and the outside space as a geo-fence; tracking spatial coordinates of a geo-location of the unmanned aerial vehicle; and determining a spatial relationship between the geo-location of the unmanned aerial vehicle location and the geo-fence.

49. The method of claim 48, wherein, The unmanned aerial vehicle is a quadcopter drone.

50. The method of claim 48, further comprising: generating a notification to the end-user computing device based on the determined spatial relationship between the geo-location of the unmanned aerial vehicle location and the geo-fence.

51. The method of claim 50, wherein determining the spatial relationship comprises determining whether the geo-location of the unmanned aerial vehicle is in the outside space or is close to the geo-fence.

52. The method of claim 50, wherein generating the notification comprises: storing a minimum threshold time and a minimum threshold distance; arousing the notification based on either of: (i) a distance of the unmanned aerial vehicle to the geo-fence being less than or equal to the minimum threshold distance; or (ii) a predicted time of the unmanned aerial vehicle reaching the geo-fence being less than or equal to the minimum threshold time.

53. The method of claim 52, wherein, The minimum threshold time and the minimum threshold distance used to arouse the notification are values calculated based on kinematic dynamics of the unmanned aerial vehicle.

54. The method of claim 52, wherein, The minimum threshold distance is based on a maximum deceleration of the unmanned aerial vehicle in a direction perpendicular to the geo-fence.

55. The method of claim 52, wherein, The minimum threshold time is based on a maximum deceleration of the unmanned aerial vehicle in a direction perpendicular to the geo-fence.

56. The method of claim 50, wherein, The notification comprises a graphical representation superimposed on the video transmission sent from the remote controller device to the end-user computing device.

57. The method of claim 50, further comprising, after arousing the notification, causing the remote controller device to send control data to the unmanned aerial vehicle to decelerate in a direction away from the geo-fence to avoid navigating into the outside space.

58. The system of claim 48, further comprising causing the remote controller device to send control data to the unmanned aerial vehicle to decelerate in a direction away from the geo-fence to avoid navigating into the outside space.

59. A geo-fencing computing device for use in a system having a remotely controlled unmanned aerial vehicle, wherein the unmanned aerial vehicle includes an imaging sensor and a wireless device that sends real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end-user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device sends real-time imaging data as a video transmission over the wide area network, the geo-fencing computing device comprising a processing system including processing circuitry and memory storing computer program instructions that configure the geo-fencing computing device to: creating a representation of a three-dimensional spatial region based on a union of polygonal prisms to define an interior space, an exterior space, and a boundary between the interior space and the exterior space as a geofence; and track spatial coordinates of a geographic location of the unmanned aerial vehicle; and determine a spatial relationship between the geographic location of the unmanned aerial vehicle location and the geo-fence.

60. A method for use in a system having a remotely controlled unmanned aerial vehicle, wherein the unmanned aerial vehicle includes an imaging sensor and a wireless device that sends real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end-user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device sends the real-time imaging data as a video transmission over the wide area network, the method comprising: creating a representation of a three-dimensional spatial region to define an inside space, an outside space, and a boundary between the inside space and the outside space as a geo-fence; and tracking spatial coordinates of a geographic location of the unmanned aerial vehicle; and determining a spatial relationship between the geographic location of the unmanned aerial vehicle location and the geo-fence.

61. A geo-fencing computing device for use in a system having a remotely controlled unmanned aerial vehicle, wherein the unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over a wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end-user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device transmits the real-time imaging data as a video transmission over the wide area network, the geo-fencing computing device comprising a processing system including processing circuitry and memory storing computer program instructions that configure the geo-fencing computing device to: creating a representation of a three-dimensional spatial area defining an interior space, an exterior space, and a boundary between the interior space and the exterior space as a geofence, wherein determine a minimum ground clearance value for each point in the interior of the geo-fence based on three-dimensional coordinates of a geographic location of the remote controller device and an elevation map of a geographic terrain of a geo-fenced area through a viewshed analysis; and track three-dimensional spatial coordinates of a geographic location of the unmanned aerial vehicle; and determine a spatial relationship between the geographic location of the unmanned aerial vehicle position and the geo-fence. the unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over a wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end-user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device transmits the real-time imaging data as a video transmission over the wide area network, the method comprising: creating a representation of a three-dimensional spatial area that defines an interior space, an exterior space, and a boundary between the interior space and the exterior space as a geo-fence, wherein a minimum ground clearance value for each point in the interior of the geo-fence is determined based on 1) three-dimensional coordinates of a geographic location of the remote controller device and 2) an elevation map of a geographic terrain of a geo-fenced area through a viewshed analysis; and 62. A method for use in a system having a remotely controlled unmanned aerial vehicle, wherein, tracking three-dimensional spatial coordinates of a geographic location of the unmanned aerial vehicle; and determining a spatial relationship between the geographic location of the unmanned aerial vehicle position and the geo-fence. ​ ​ 63. A computing device for use in a system having a remotely controlled unmanned aerial vehicle, wherein the unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device transmits a video transmission based on the real-time imaging data over the wide area network, the computing device comprising a processing system including processing circuitry and a memory storing computer program instructions that configure the computing device to: generate text describing content of the video transmission based on a large language model; generate audio data corresponding to the generated text using a text-to-speech engine; and synchronize and multiplex the supplemental audio data with the video transmission on a communication channel of the wide area network for transmission to the end user computing device.

64. A method for use in a system having a remotely controlled unmanned aerial vehicle, wherein the unmanned aerial vehicle includes an imaging sensor and a wireless device that transmits real-time imaging data from the imaging sensor over a wireless network and receives control data from a remote controller device over the wireless network, wherein the remote controller device is connected to a wide area network and receives instructions from an end user computing device to generate the control data for the unmanned aerial vehicle, wherein the remote controller device transmits a video transmission based on the real-time imaging data over the wide area network, the method comprising: generating text describing content of the video transmission using a large language model; generating audio data corresponding to the generated text using a text-to-speech engine; and synchronizing and multiplexing the audio data with the video transmission on a communication channel of the wide area network for transmission to an end user computing device. ​ ​ ​ ​ ​ ​ ​ ​

Images