Method for managing a 3D flight path and related system
By generating and observing the 3D flight path of UAVs using virtual reality devices, the problem of obstacles that cannot be considered in traditional methods is solved, enabling precise control and intuitive operation of UAVs, which is suitable for precision tasks in modern cities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SZ DJI TECH CO LTD
- Filing Date
- 2016-12-01
- Publication Date
- 2026-06-16
AI Technical Summary
Traditional UAV flight path planning methods cannot take into account third-dimensional obstacles, resulting in complex and inaccurate operations that are difficult to meet the needs of precision missions in modern cities and lack an intuitive user experience.
By generating and observing the 3D flight path of UAVs through virtual reality devices, receiving environmental information and generating 3D paths, and combining action items and obstacle avoidance algorithms, an intuitive path planning and verification tool is provided.
It enables precise control and intuitive path planning for UAVs, improving ease of operation and user experience, and is suitable for precision and challenging tasks.
Smart Images
Figure CN115238018B_ABST
Abstract
Description
Technical Field
[0001] This technology generally relates to methods and related systems for generating, analyzing, and validating three-dimensional (3D) flight paths of mobile platforms such as unmanned aerial vehicles (UAVs). Background Technology
[0002] Traditionally, UAV flight paths are planned based on certain location points (waypoints) marked on a two-dimensional (2D) map. This approach can be inaccurate because it fails to account for the third dimension (e.g., altitude) of objects such as buildings, structures, and other obstacles that may exist along the UAV's flight path. This approach also fails to meet the need for precise UAV control during sophisticated missions, such as driving a UAV through a modern city with tall buildings to deliver small items. Furthermore, it takes considerable time and practice for UAV operators to become familiar with path planning tasks using traditional methods. Traditional methods do not provide UAV operators with an intuitive user experience when operating UAVs. Therefore, an improved method and system are needed for generating or planning 3D flight paths for UAVs. Summary of the Invention
[0003] The following overview, provided for the reader's convenience, outlines some representative embodiments of the disclosed technology. In general, this technology provides improved methods and related systems that enable operators to generate, analyze, and verify 3D flight paths of UAVs in a simple, easy-to-learn, and intuitive manner. More specifically, this technology enables operators to create and observe 3D flight paths of UAVs via virtual reality devices. For example, this technology allows operators to observe and verify the generated 3D flight path from a first-person perspective via virtual reality devices. By doing so, the operator can verify whether the generated 3D flight path is exactly what he / she wants to perform a certain task (e.g., filming a movie or taking pictures of a target person or object). Furthermore, this technology enables operators to generate accurate 3D flight paths and precisely control the UAV to perform sophisticated or challenging tasks. Examples of challenging tasks include delivering a package to an east-facing window on a floor of a building, capturing images of an actor's face standing in a specific location, and photographing a moving target from a specific perspective.
[0004] Representative embodiments of this technology include methods and related systems for generating 3D paths for a UAV. The method includes receiving a set of 3D information (e.g., a set of geographic information or coordinates of objects) associated with an environment (e.g., the location where the UAV operates in the real world, such as a defined space in a city area, structure, or building, or an outdoor area) or a virtual reality environment (e.g., based on objects generated in the real world). The method also includes receiving multiple virtual locations in the virtual reality environment. In some embodiments, the method may receive physical locations in the real-world environment and then convert them into virtual locations in the virtual reality environment. This conversion can be accomplished by: (1) user input (e.g., the user inputs the coordinates of a specific location), (2) user selection from a recommendation list (e.g., the system provides a list of candidate locations from which the user can select), or (3) retrieving data from a storage device (e.g., locations of paths previously traveled by the UAV, locations frequently flown to by the UAV, and / or locations generated based on logs or historical files associated with the operator or UAV). For each virtual or physical location, the system receives one or more corresponding action items. Representative actions include performing predetermined tasks at various locations, such as camera aiming, stabilizing the UAV, acquiring images of a specific size or format, acquiring information associated with each location (e.g., whether the UAV can see objects / individuals at each location; acquiring / measuring virtual / real environment information at each location), configuring the UAV's components (e.g., adjusting the power output of the UAV's power supply components), etc.
[0005] The system then receives multiple virtual locations (e.g., locations or coordinates in a virtual reality environment) corresponding to multiple locations. In some embodiments, the system may receive physical locations and then generate corresponding virtual locations. Once the virtual locations are determined, the system then generates a 3D path (e.g., a 3D trajectory) based on this set of 3D information. For example, the 3D path may be based on the requirement that it maintains a certain distance from any object described by the set of 3D information. The 3D path is also based on the multiple virtual locations (e.g., based on the requirement that the 3D path passes through all virtual locations) and the multiple action items (e.g., the action item could be a UAV flying around a target, and in this case, the 3D path includes a path around the target). Details of the virtual reality environment will be discussed in the following specific embodiments.
[0006] The system then generates a set of images associated with the 3D path based on at least one of the 3D information set, the plurality of virtual locations, and the plurality of actions. For example, the generated images could be a set of images observed from a first-person perspective by a UAV in a virtual reality environment. The system then visually presents the set of images to the operator. In a particular embodiment, the set of images can be presented via a virtual reality device. Therefore, the system enables the operator to observe the proposed 3D flight path in an intuitive manner.
[0007] In certain embodiments, the system enables an operator to manually or automatically adjust the generated 3D path based on user settings. For example, an operator can create additional locations (e.g., input via virtual reality devices, keypads, touchscreens, joysticks, and / or other suitable devices) to include them in the 3D path within the virtual environment.
[0008] In certain embodiments, the system may include an image component coupled to the UAV, configured to acquire images based on predetermined actions. In some embodiments, the image component may include a color-sensing camera that acquires color images (e.g., those with red, green, and blue (RGB) pixels). In other embodiments, the image acquisition component may be a camera that acquires various other types of images (e.g., a thermal / infrared camera or a night vision camera).
[0009] Some embodiments of this technology can be implemented as a method for configuring a system for planning flight paths or routes for a UAV. The method may include programming a computer-readable medium with instructions that, when executed, receive a set of 3D information associated with a virtual reality environment (or, in some embodiments, a real-world environment) and receive multiple virtual locations in the virtual reality environment (or physical locations in the real-world environment, which will be converted into virtual locations). For each virtual location, the instructions may include receiving one or more corresponding actions. The instructions may generate a 3D path based on the set of 3D information, the multiple virtual locations, and the multiple actions. The instructions may also generate a set of images associated with the 3D path based on the set of 3D information, the multiple virtual locations, and the multiple actions. The instructions may visually present the set of images to an operator. In a particular embodiment, the images are presented via a virtual reality device. In some embodiments, the instructions may adjust the 3D path in response to receiving instructions from the operator via a virtual reality device. Methods and systems according to embodiments of this technology may include any one of the foregoing elements or any combination of the foregoing elements. Attached Figure Description
[0010] Figure 1A This is a block diagram illustrating a system configured according to a representative embodiment of the present technology.
[0011] Figure 1B This is a block diagram illustrating a system configured according to a representative embodiment of the present technology.
[0012] Figure 2 This is a partial schematic diagram of a UAV configured according to a representative embodiment of the present technology.
[0013] Figure 3A This is a partial schematic diagram showing 3D paths and action items generated according to a representative embodiment of the present technology.
[0014] Figure 3B This is a partial schematic diagram illustrating a 3D path generated and avoiding obstacles according to a representative embodiment of the present technology.
[0015] Figure 4A and Figure 4B This is a partial schematic diagram illustrating an image created by an image component of a UAV according to a representative embodiment of the present technology.
[0016] Figure 5 This is a flowchart illustrating an embodiment of a method for generating a 3D flight path according to a representative embodiment of the present technology. Detailed Implementation
[0017] 1. Overview
[0018] This technology generally relates to methods and related systems for generating, analyzing, and validating 3D flight paths of a UAV. A representative system configured according to this technology generates a 3D flight path in a virtual reality environment (e.g., created based on a real-world environment by measuring the dimensions of tangible / physical objects therein and then generating a virtual dataset corresponding to said physical / physical objects) based at least in part on (1) the locations the UAV will traverse and (2) the actions the UAV will perform at each location. In a particular embodiment, the locations may include real-world locations and virtual reality locations. A real-world location may be a set of coordinates corresponding to a real-world environment, and a virtual reality location may be a set of coordinates corresponding to a virtual reality environment. In a particular embodiment, the actions may include tasks to be performed by the UAV (e.g., rotating to face a different direction) or tasks performed by components of the UAV (e.g., a camera). Representative examples of actions include (1) aligning the UAV's image component with the target; (2) positioning the UAV's image component at a horizontal level; (3) maintaining the UAV's image component's viewpoint; (4) aiming the UAV's image component at the target; (5) acquiring images associated with the target through the UAV's image component; (6) commanding the UAV to fly around the target; and / or (7) commanding the UAV to rotate about an axis.
[0019] The system then generates a set of images associated with the 3D flight path. In a particular embodiment, this set of images includes virtual reality images (based on position and corresponding actions) that will be captured as the UAV flies along the 3D flight path in a virtual reality environment. The flight path or multiple sections of the flight path can be generated based on a shortest distance algorithm or other suitable algorithms, and the expected durability of the UAV, among other factors, can be taken into account. The system then presents the set of images to the operator, providing the operator with an intuitive experience of how the environment appears to make the UAV fly along the generated 3D flight path in the corresponding real-world environment. The system provides the operator with the opportunity to view the generated 3D flight path by examining the set of images. In a particular embodiment, the operator can adjust the 3D flight path by adding / removing additional / existing positions or actions to the existing 3D flight path via a virtual reality device.
[0020] In some embodiments, the system enables an operator to adjust the 3D flight path in real time. For example, the UAV can take off and fly based on the generated 3D flight path. Real-world images captured by an imaging component coupled to the UAV can be transmitted to the system and then presented to the UAV operator. The operator can then adjust the 3D flight path (the portion not yet in flight) within the virtual reality environment. With this setup, the system allows the operator to simultaneously monitor and precisely control the UAV to perform sophisticated, precise, and / or challenging tasks.
[0021] In certain embodiments, the system may generate a 3D flight path based at least in part on one or more rules provided by the operator. For example, these rules may be associated with various factors, such as the minimum / maximum distance between the UAV and obstacles or targets, algorithms for obstacle avoidance (e.g., distance-based, UAV-time-based, obstacle-based, etc.), user preferences, and / or other appropriate factors.
[0022] In some embodiments, an operator may provide location or action items to the system via a virtual reality device. In some embodiments, the operator may provide such information through one or more gestures. For example, an operator wearing a virtual reality device on his / her arm may orient his / her arm toward a certain direction in the virtual reality environment to indicate that the operator wants the UAV to face or move in that direction. As another example, an operator wearing a virtual reality device in front of his / her eyes may blink his / her eyes at a specific location in the virtual reality environment to instruct the system to add that specific location to the 3D flight path. In one example, the operator may input location information via an input device or controller. In yet another example, an operator wearing a virtual reality device on his / her hand may use specific gestures (e.g., gestures associated with the game of rock-paper-scissors) to indicate specific action items.
[0023] Unlike traditional systems, aspects of this technology relate to enabling operators to generate, analyze, and validate 3D flight paths for UAVs suitable for precision, high-accuracy, and / or other challenging UAV flight missions. Furthermore, aspects of this technology can improve the ease of flight path planning and provide a better and more intuitive user experience than traditional methods. For clarity, several details are omitted in the following description, which are used to describe structures or processes well-known and frequently associated with UAVs and corresponding systems and subsystems but may unnecessarily confuse with some important aspects of the disclosed technology. Moreover, although the following disclosure illustrates several embodiments of different aspects of this technology, some other embodiments may have different configurations or different components than those described in this section. Therefore, the technology may have other embodiments with additional elements or without the following reference figures 1 to 12. Figure 5 Several of the components described.
[0024] Figure 1 is provided. Figure 5 The accompanying drawings illustrate representative embodiments of the disclosed technology. Unless otherwise specified, the drawings are not intended to limit the scope of the claims in this application.
[0025] Many embodiments of the present technology described below can take the form of computer or controller executable instructions, including routines executed by a programmable computer or controller. Those skilled in the art will recognize that the present technology can be implemented on computer or controller systems other than those shown and described below. The present technology can be embodied in a dedicated computer or data processor specifically programmed, configured, or constructed to execute one or more computer executable instructions described below. Therefore, the terms “computer” and “controller” as commonly used herein refer to any suitable data processor and may include internet devices and handheld devices (including PDAs, wearable computers, cellular or mobile phones, multiprocessor systems, processor-based or programmable consumer electronics, network computers, minicomputers, programmable computer chips, etc.). Information processed by these computers and controllers can be presented on any suitable display medium, including CRT displays or LCDs. Instructions for performing computer or controller executable tasks can be stored in or on any suitable computer-readable medium, including hardware, firmware, or a combination of hardware and firmware. Instructions can be contained in any suitable storage device, including, for example, flash drives, USB devices, or other suitable media. In specific embodiments, the term “component” can be hardware, firmware, or a set of instructions stored in a computer-readable medium.
[0026] 2. Representative Implementation Examples
[0027] Figure 1A This is a block diagram illustrating a system 100a configured according to a representative embodiment of the present technology. In some embodiments, system 100a may be or may include means having a computer-readable medium for storing information / instructions associated with the components of system 100a. Figure 1A As shown, system 100a includes a processor 101, a storage component 102, a virtual reality component 103, a flight path generation component 105, a flight path analysis component 107, a flight path verification component 109, and a communication module 110. As shown, the processor 101 is coupled to and configured to control other components of system 100a. The storage component 102 is configured to permanently or temporarily store information generated by system 100a (e.g., data related to the virtual reality environment and / or the generated 3D path). In a particular embodiment, the storage component 102 may include a disk drive, hard disk, flash drive, memory, etc.
[0028] like Figure 1A As shown, communication component 110 is configured to send signals to UAV 11 and receive signals from UAV 11. Figure 1AAs shown, UAV 11 includes: a UAV controller 111 configured to control UAV 11; a UAV power supply 113 configured to provide power to UAV 11; a UAV communication component 115 configured to communicate with communication component 110; a UAV sensor 117 configured to measure or detect information associated with UAV 11; and a UAV image component 119 configured to acquire images of the exterior of UAV 11. In a particular embodiment, UAV image component 119 may be a camera that acquires two-dimensional images using red, green, and blue (RGB) pixels. UAV image component 119 may include an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge-Coupled Device) image sensor. Reference is made below. Figure 3A , Figure 4A and Figure 4B Further examples of two-dimensional images are described. The acquired images can be transferred to and stored in storage component 102. In some embodiments, UAV image component 119 can be a thermal imaging camera, a night vision camera, or any other suitable device capable of acquiring images.
[0029] like Figure 1A As shown, the virtual reality component 103 can be used as an interface between the operator 12 and the system 100a. The virtual reality component 103 is also configured to generate / maintain a virtual reality environment corresponding to a real-world environment. In a particular embodiment, the virtual reality component 103 may further include (1) a virtual reality engine configured to generate the virtual reality environment and (2) a virtual reality device / controller configured to interact with a user. For example, the virtual reality engine may be a set of computer-readable instructions or software applications that can (1) process acquired location information associated with physical objects in a real-world environment; and (2) thus generate a virtual reality environment containing virtual objects corresponding to physical objects in the real-world environment. In some embodiments, the virtual reality environment may be generated based on a set of geographic information (e.g., a set of coordinates, lines, or shapes associated with one or more objects in a specific area of a real-world environment). In some embodiments, the virtual reality environment may be based on, for example, 3ds data available from Autodesk Inc. in the UK. Software or other suitable 3D modeling applications can be used to generate virtual reality environments. Embodiments of virtual reality devices include wearable virtual reality devices, tablets, touchscreens, displays, etc. In certain embodiments, wearable virtual reality devices may include headsets, helmets, goggles, virtual reality glasses, gloves, sleeves, handheld devices, etc.
[0030] Flight path generation component 105 is configured to generate a 3D path based at least in part on one or more (virtual or physical) locations provided by operator 12 or suggested by system 100a. In some embodiments, the locations may be provided as virtual reality locations in a virtual reality environment (e.g., these virtual locations may be identified by operator 12 via virtual reality component 103). In some embodiments, the locations may be provided as real-world locations in a real-world environment (e.g., in the form of real-world coordinates). In such embodiments, virtual reality component 103 may transform the provided real-world locations into corresponding virtual locations. For example, system 100a may first determine the relationship between the coordinate systems used in the real-world environment and the virtual reality environment. Once this relationship is determined, system 100a may then transform the provided locations into corresponding virtual locations based on the real-world locations (or, in other embodiments, the reverse).
[0031] The flight path generation component 105 is also configured to generate a 3D path based at least in part on one or more actions corresponding to the provided / suggested location. In certain embodiments, actions include performing a predetermined task at a specific location. In some embodiments, for example, actions may involve UAV movement, such as guiding the UAV to fly around a target or commanding the UAV to rotate about an axis. In some embodiments, actions may involve actions performed by components of the UAV. In such embodiments, actions may include, for example, aligning the UAV's imaging component with a target; positioning the UAV's imaging component in a horizontal position; maintaining the UAV's imaging component's field of view; aiming the UAV's imaging component at the target; acquiring images associated with the target through the UAV's imaging component; acquiring a set of information through the UAV's sensors; and / or commanding the UAV's communication component to transmit the set of information to a remote device (e.g., a smartphone under the control of operator 12). This information may include UAV information measured by the UAV's sensors or images acquired by the UAV's imaging component.
[0032] When generating a 3D flight path, the flight path generation component 105 also considers objects, targets, or obstacles in the virtual reality environment. In certain embodiments, objects, targets, or obstacles in the virtual reality environment can be identified as groups of 3D information (e.g., in formats such as coordinates, lines, shapes, etc.). The flight path generation component 105 can generate the 3D flight path based on one or more predetermined rules. In some embodiments, these rules may include physical rules, such as the 3D flight path not passing through tangible objects in the virtual reality environment or through the ground of the virtual reality environment. In some embodiments, the rules may relate to the maneuverability of the UAV, such as the UAV's minimum turning radius, the UAV's maximum / minimum speed, and / or the UAV's maximum / minimum acceleration.
[0033] After generating the 3D flight path, the flight path analysis component 107 can then analyze the generated 3D flight path and perform a simulation in which the UAV flies along the generated 3D flight path in a virtual reality environment. In a particular embodiment, the simulation includes generating a set of virtual reality images, which are images that the UAV can capture at each provided location along the 3D flight path in the virtual reality environment. The flight path analysis component 107 then visually presents the set of images to the operator 12. In a particular embodiment, the set of images can be visually presented to the operator 12 via the virtual reality component 103. By doing so, system 100a enables the operator 12 to experience the 3D flight path from a first-person perspective. By doing so, the operator 12 can have a clear and intuitive feeling or understanding of how the UAV will move in the real-world environment. At the same time, the operator 12 can view and verify whether the UAV can perform actions as he / she expects (e.g., to photograph a target from a specific perspective).
[0034] In the illustrated embodiment, the flight path verification component 109 is configured to further verify the generated 3D flight path to ensure that the 3D flight path meets certain predetermined requirements. These requirements may be set by the operator 12 (e.g., based on the operator's preferences or skill level when operating the UAV) and / or from a third-party entity (e.g., government regulations prohibiting UAVs from flying in specific areas). By verifying the generated 3D flight path, system 100a can provide operator 12 with a safe and feasible 3D flight path.
[0035] System 100a also enables operator 12 to adjust the generated 3D flight path. In certain embodiments, operator 12 can add / remove attached / existing positions or adjust the curvature of the generated 3D flight path via virtual reality component 103. In some embodiments, operator 12 can manually adjust the 3D flight path (e.g., via virtual reality device or input device). In some embodiments, operator 12 can automatically adjust the 3D flight path (e.g., based on user preferences, where the system has learned these preferences by learning from previous adjustments made to the generated 3D flight path by the operator during previous tasks / projects). By doing so, system 100a enables operator 12 to precisely control UAV 11 to accomplish the desired task.
[0036] Figure 1B This is a block diagram illustrating a system 100b configured according to a representative embodiment of the present technology. System 100b includes a 3D flight control system 10 and a UAV 13. Figure 1BAs shown, compared to system 100a described in Figure 1, the 3D flight control system 10 includes an additional input component 104 configured to receive user input from operator 12. User input may include: (1) positions or actions to be included in the 3D flight path, or (2) one or more rules or requirements to be considered and followed when generating the 3D flight path. In a particular embodiment, the additional input component 104 may be a keypad, touchscreen, joystick, keyboard, or any other suitable device.
[0037] like Figure 1B As shown, with Figure 1A Compared to UAV 11 described herein, UAV 13 further includes a UAV storage component 112 and a UAV image analysis component 114. The UAV image analysis component 114 is configured to compare (1) a set of images simulated in a virtual reality environment by the flight path analysis component 107 and (2) images actually acquired by the UAV component 119 when UAV 13 is flying in a real-world environment. In some embodiments, if the UAV image analysis component 114 identifies a difference between these two sets of images, it will notify the operator 12. In some embodiments, if the UAV image analysis component 114 identifies a difference between these two sets of images, it will notify the UAV controller 111, which will then adjust UAV 13 (or a component thereof) accordingly to minimize the difference. For example, the simulated image at position X may include the face of a target person located at the center of the simulated image. When UAV 13 flies to position X, UAV image analysis component 114 may detect that the image actually acquired by UAV image component 119 does not include the target person's face (e.g., it may only include the target person's body). UAV image analysis component 114 can then instruct UAV controller 111 to rotate or move UAV 13 accordingly so that the target person's face can be displayed at the center of the actually acquired image. References will follow below. Figure 4A and Figure 4B Examples of the acquired images are discussed further.
[0038] Figure 2This is a partial schematic diagram of a UAV 20 configured according to a representative embodiment of the present technology. The UAV 20 may include a fuselage 210, which may correspondingly include a central portion and one or more external portions. In a specific embodiment, the fuselage 210 may include four external portions (e.g., arms) spaced apart from each other as they extend away from the central portion. In other embodiments, the fuselage 210 may include other numbers of external portions. In any of these embodiments, a single external portion may support components of the propulsion system that drives the UAV 20. For example, a single arm may support a corresponding individual motor that drives a corresponding propeller 206.
[0039] The fuselage 210 may carry a payload 204, such as an imaging device. In a particular embodiment, the imaging device may include an imaging camera (e.g., a camera configured to capture video data, still data, or both). The imaging camera may be sensitive to wavelengths in any of a variety of suitable bands, including visible, ultraviolet, infrared, or combinations thereof. In another embodiment, the payload 204 may include other types of sensors, other types of cargo (e.g., parcels or other deliverables), or both. In most of these embodiments, a gimbal 202 is used to support the payload 204 relative to the fuselage 210, allowing the payload to be positioned independently of the fuselage 210. Thus, for example, when the payload 204 includes an imaging device, the imaging device may be moved relative to the fuselage 210 to track a target. More specifically, for example, the imaging device may be rotated at an angle relative to the fuselage 210 (or relative to another reference plane, such as a horizontal plane). When the UAV 20 is not in flight, the landing gear may support the UAV 20 in a position that protects the payload 204.
[0040] In a representative embodiment, UAV 20 includes a controller 208 carried by UAV 20. Controller 208 may include an onboard computer-readable medium 203 that executes instructions commanding actions of UAV 20, including but not limited to the operation of the propulsion system and imaging equipment. Onboard computer-readable medium 203 may be removed from UAV 20.
[0041] Figure 3A This is a partial schematic diagram illustrating a generated 3D path 301 and action items according to a representative embodiment of the present technology. In the illustrated embodiment, the generated 3D path 301 passes through positions A, B, C, and D. In the illustrated embodiment, the action items include an image 303 of the operator 30 being captured by the UAV image component 119 at position A. The action item may also specify a particular format for the image 303 to be captured. For example, the action item may request a format that can be based on a horizontal angle (e.g., Figure 3A Angle A in the middle h), vertical angle (e.g., Figure 3A Angle A in the middle v ) or diagonal angle ( Figure 3A Angle A in the middle d The image 303 is captured from a specific angle measured. More specifically, the angle of view of the image camera 119 determines how the image 303 looks and the position of the operator 30 within the image 303 (e.g., the operator 30 may be located in the center of the image 303 and occupy half or a quarter of the total image area of the image 303).
[0042] Figure 3B This is a partial schematic diagram illustrating a generated 3D path 303 for avoiding a first obstacle 31 and a second obstacle 33 according to a representative embodiment of the present technology. The shape of the obstacle can be one of the factors to be considered when generating the 3D path 303. In the illustrated embodiment, for example, the generated 3D path 303 passes through positions A, B, C, and D. Flying along the 3D path 303, the UAV can avoid the first obstacle 31 and the second obstacle 33. The first obstacle 31 is a "slender" obstacle with a low length-to-height ratio. In the illustrated embodiment, the UAV flying along the 3D path 303 avoids the first obstacle 31 by flying around it. As shown, the second obstacle 33 is a "wide" obstacle with a high length-to-height ratio. The UAV flying along the 3D path 303 avoids the second obstacle 33 by flying over it. In some embodiments, when generating 3D path 303, the operator can set his / her own rules (e.g., maintain a distance of 10 meters between the UAV and a “narrow” obstacle; or fly 15 meters above a “wide” obstacle).
[0043] Figure 4A and Figure 4B This is a partial schematic diagram showing images 41 and 43 created by the image component of a UAV according to a representative embodiment of the present technology. Figure 4A Image 41 shown represents an image captured before the system made adjustments based on the action item. Figure 4B Image 43 shown represents the adjusted image after adjustments based on the action item. Figure 4AIn the image 400, image 41 may include a specific region 401, a target person 403, and a background item 405. In the illustrated embodiment, the action item associated with image 400 may be "positioning the face of the target person 403 at the center of the specific region 401" and "making the face of the target person 403 occupy more than 50% of the specific region 401". Based on the action item, the system can adjust image 41 to become image 43 (e.g., by changing the planned 3D flight path) to meet the requirements specified in the action item. Using a similar technique, the system can enable the operator to precisely control the UAV or its components to perform other specific tasks by setting parameters for corresponding action items.
[0044] Figure 5 This is a flowchart illustrating an embodiment of a method 500 for generating a 3D flight path according to a representative embodiment of the present technology. Method 500 may be initiated upon a request from an operator. At block 501, the method receives a set of 3D information associated with a virtual reality environment. For example, this information may be obtained from sources such as Autodesk Inc. in the UK. The software or other suitable 3D modeling application is used to generate the 3D information. Box 503 includes receiving multiple virtual locations in the virtual reality environment. In some embodiments, method 500 may include receiving physical locations (in the real world environment) and converting them into virtual locations in the virtual reality environment. The physical or virtual locations may be provided in a 3D coordinate format (e.g., a table of points). At box 505, for each virtual location, the system receives at least one corresponding action. In a particular embodiment, the action may include (1) aligning the UAV's image component with a target; (2) positioning the UAV's image component at a horizontal level; (3) maintaining the viewpoint of the UAV's image component; (4) aiming the UAV's image component at the target; (5) acquiring an image associated with the target through the UAV's image component; (6) commanding the UAV to fly around the target; and / or (7) commanding the UAV to rotate about an axis.
[0045] At block 507, the system generates a 3D path based on the 3D information set, the plurality of virtual locations, and the plurality of actions. Method 500 then continues at block 509 to generate an image set associated with the 3D path based on the 3D information set, the plurality of virtual locations, and the plurality of actions. In some embodiments, this image set may be generated by a virtual reality system. At block 511, the system visually presents the image set to the operator. In a particular embodiment, the image set is visually presented to the operator via a virtual reality device. Method 500 then returns to await further instructions. In some embodiments, the system may also adjust the 3D path when instructions are received from the operator via the virtual reality device.
[0046] As can be understood from the foregoing, specific embodiments of the present technology have been described herein for illustrative purposes; however, various modifications may be made without departing from the present technology. For example, specific embodiments have been described in the context of UAVs. In other embodiments, the present technology may be implemented by other suitable mobile devices, such as unmanned ground vehicles (UGVs), unmanned surface vehicles (USVs), or robots.
[0047] Furthermore, while advantages associated with certain embodiments of the present technology have been described in the context of these embodiments, other embodiments may also exhibit such advantages, and not all embodiments need to demonstrate advantages falling within the scope of the present technology. Accordingly, this disclosure and related technologies may cover other embodiments not expressly shown or described herein.
[0048] At least a portion of the disclosure in this patent document contains copyrighted material. The copyright holder does not object to any faxed copying of the patent document or patent disclosure as it appears in the patent documents or records of the Patent and Trademark Office, but otherwise reserves all copyright.
Claims
1. A method for generating a three-dimensional 3D path for a movable platform, the method comprising: Receive groups of 3D information associated with the virtual reality environment; Receive multiple virtual locations in the virtual reality environment; Receive multiple action items, wherein each of the virtual locations corresponds to one or more of the action items, wherein the action items include tasks to be performed by the mobile platform and / or tasks to be performed by components of the mobile platform; A 3D path is generated in the virtual reality environment based on the multiple virtual locations; Based on the 3D information set, the plurality of virtual locations, and the plurality of actions, an image set associated with the 3D path is generated, the image set including virtual reality images to be captured as the mobile platform flies along the 3D path; and The image set is presented to the operator visually.
2. The method of claim 1, further comprising generating the image set through a flight simulation process.
3. The method of claim 1, wherein the image set is visually presented via a virtual reality device, and wherein the image set is visually presented to the operator based on an order determined by the 3D path.
4. The method of claim 1, further comprising adjusting the 3D path in response to receiving an instruction from the operator via a virtual reality device.
5. The method of claim 1, wherein the movable platform includes an image component, and wherein the action item includes: (1) establishing or maintaining the viewpoint of the image component, (2) aligning the image component with the target, (3) aiming the image component at the target, (4) acquiring an image associated with the target via the image component, or (5) aiming the image component at the target; and instructing the movable platform to move around the target.
6. The method according to claim 1, wherein the plurality of virtual locations correspond to a plurality of physical locations.
7. The method according to any one of claims 1-6, further comprising: (1) The 3D path is generated based at least in part on an obstacle avoidance algorithm; (2) The 3D path is generated based at least in part on a shortest distance algorithm; (3) The 3D path is generated based at least in part on the expected travel time of the mobile platform; (4) The 3D path is generated based at least in part on the shape of obstacles in the environment; or (5) The 3D path is generated based at least in part on input from the operator.
8. The method according to any one of claims 1-7, further comprising determining the plurality of virtual locations based at least in part on input from the operator.
9. A system for controlling a mobile platform, the system comprising: processor; A storage component, coupled to the processor and configured to store groups of 3D information associated with a virtual reality environment; An input component, coupled to the processor and configured to receive a plurality of virtual locations and a plurality of action items in the virtual reality environment, wherein each virtual location corresponds to one or more of the action items, wherein the action items include tasks to be performed by a mobile platform and / or tasks to be performed by components of the mobile platform; A flight path generation component, coupled to the processor and configured to generate 3D paths in the virtual reality environment based on the plurality of virtual locations; and A flight path analysis component, coupled to the processor and configured to generate an image set associated with the 3D path based on the 3D information set, the plurality of virtual locations, and the plurality of action items, the image set including virtual reality images to be acquired as the mobile platform flies along the 3D path; The image set is presented to the operator visually via a virtual reality component.
10. A method for configuring a mobile platform controller, comprising: Programming a computer-readable medium using instructions that, when executed, perform the following operations: Receive groups of 3D information associated with the virtual reality environment; Receive multiple virtual locations in the virtual reality environment; Receive multiple action items, wherein each of the virtual locations corresponds to one or more of the action items, wherein the action items include tasks to be performed by the mobile platform and / or tasks to be performed by components of the mobile platform; A 3D path is generated in the virtual reality environment based on the multiple virtual locations; Based on the 3D information set, the plurality of virtual locations, and the plurality of actions, an image set associated with the 3D path is generated, the image set including virtual reality images to be captured as the mobile platform flies along the 3D path; and The image set is presented to the operator visually.