Systems and methods for monitoring using visual markers

Through visual sensors to detect visual marks in the environment, calculate the position of UAV in the three-dimensional coordinate system, solving the problem of difficulty in navigation of UAV in weak signal environments, and achieving the effect of autonomous navigation and monitoring.

CN112859899BActive Publication Date: 2024-11-01SZ DJI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110067820.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2014-10-31
Publication Date
2024-11-01
Estimated Expiration
2034-10-31

AI Technical Summary

Technical Problem

Existing unmanned aerial vehicles (UAVs) have difficulty navigating in weak signal environments, especially when the Global Positioning System (GPS) signals are unreliable, making it difficult to achieve autonomous navigation and positioning.

Method used

Vision sensors are used to detect visual marks in the environment. By calculating the shape and size of the visual marks, the processor is used to determine the position of the UAV in the three-dimensional coordinate system, and avoid obstacles through the visual sensors and sensors to achieve autonomous navigation and monitoring.

Benefits of technology

In an environment where GPS signals are unreliable, UAV can navigate and monitor independently, improving navigation reliability and accuracy, and effectively positioning and obstacle avoidance in indoor environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112859899B_ABST
    Figure CN112859899B_ABST
Patent Text Reader

Abstract

Systems and methods for locating an unmanned aerial vehicle (UAV) in an environment are provided. The UAV may be able to discern visual markers and patterns in the environment. The UAV may distinguish visual markers from other features in the environment. Visual markers may be analyzed with visual sensors to determine the location of the UAV in the environment. The visual markers may encode instructions to the UAV.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Aircraft such as unmanned aerial vehicles (UAVs) can be used to perform surveillance, reconnaissance, and exploration missions for military and civilian applications. Such aircraft can carry a payload configured to perform a specific function.

[0002] When navigating in an environment, a typical UAV may communicate with external systems. The UAV may operate in an environment where navigation signals (such as signals from a global positioning system (GPS) signal) are weak. This may pose challenges to the autonomous navigation of the UAV. Summary of the invention

[0003] It is desirable to provide a method for navigating an unmanned aerial vehicle (UAV) in an environment without relying on the signal strength and reliability of navigation sensors such as global positioning software (GPS) signals. Systems and methods for locating and navigating a UAV in an environment are provided herein. The systems and methods also provide the ability to transmit directions to the UAV for monitoring and navigation purposes. Markings can be provided in the environment to assist in UAV navigation. The markings can be visual markings, which can allow the UAV to obtain its orientation and / or transmit instructions to the UAV.

[0004] In one aspect of the present disclosure, a method for performing environmental monitoring includes positioning an unmanned aerial vehicle (UAV) within the environment, wherein the positioning is determined by: using one or more visual sensors of the UAV to detect one or more visual markers having a visual pattern and attached to one or more surfaces within the environment; and using one or more sensors of the UAV to collect visual images and / or audio data within the environment, thereby performing the environmental monitoring.

[0005] The environment may be an indoor environment. The environment may be an environment in which a global positioning system (GPS) signal is unreliable. Positioning of the UAV may include determining the position of the UAV in three-dimensional space.

[0006] In some embodiments, the UAV may weigh less than 1000 g and have a maximum dimension not exceeding 100 cm.

[0007] The vertices of the shapes in the visual pattern can be detected and used to calculate the position relative to one or more visual markers. The one or more visual markers can include a band with a visual pattern including squares. The visual pattern can include squares of different colors. The vertices of the squares can be detected and used to calculate the size of the portion of the visual pattern.

[0008] The method may also include determining, with the aid of one or more processors, a position of the UAV in a three-dimensional coordinate system based on the visual pattern. The three-dimensional coordinate system may be a global coordinate system. The three-dimensional coordinate system may be a local coordinate system.

[0009] The method may also include recognizing the visual pattern and implementing a response by the UAV based on the recognized visual pattern.

[0010] In some cases, positioning may include calculating the rotation and position of the UAV relative to the visual marker using (1) one or more intrinsic characteristics of the one or more visual sensors onboard the UAV, (2) the known location of the visual marker, and (3) a set of points in an image of the visual marker taken by the visual sensor onboard the UAV.

[0011] At least one of the one or more visual sensors may be a camera. The UAV may monitor within the environment while moving autonomously. Positioning of the UAV may occur with the aid of one or more processors onboard the UAV.

[0012] In some embodiments, the method may further include detecting an obstacle with the aid of one or more sensors onboard the UAV; and adjusting the flight direction of the UAV to avoid the obstacle.

[0013] The position can be determined with the aid of infrared detectors or radar detectors.

[0014] In one aspect of the present disclosure, a method for locating an unmanned aerial vehicle (UAV) may include: while the UAV is flying, capturing an image including a visual marker with the aid of one or more visual sensors of the UAV, the visual marker having a visual pattern and attached to a surface within the environment; calculating the size of at least a portion of the visual pattern of the visual marker in the image; and determining, with the aid of one or more processors, a position of the UAV within a three-dimensional coordinate system based on the size of the portion of the visual pattern.

[0015] The environment may be an indoor environment. The environment may be an environment where a global positioning system (GPS) signal is unreliable.

[0016] In some embodiments, the UAV may weigh less than 1000 g and have a maximum dimension not exceeding 100 cm.

[0017] The dimension may be between vertices of shapes in the visual pattern.

[0018] The vertices of the shapes in the visual pattern can be detected and used to calculate the position relative to one or more visual markers. The one or more visual markers can include a band with a visual pattern including squares. The visual pattern can include squares of different colors. The vertices of the squares can be detected and used to calculate the size of the portion of the visual pattern.

[0019] The method may also include determining, with the aid of one or more processors, a position of the UAV in a three-dimensional coordinate system based on the visual pattern. The three-dimensional coordinate system may be a global coordinate system. The three-dimensional coordinate system may be a local coordinate system.

[0020] The method may also include recognizing the visual pattern and implementing a response by the UAV based on the recognized visual pattern.

[0021] In some cases, positioning can include calculating the rotation and position of the UAV relative to the visual marker using (1) one or more inherent characteristics of the one or more visual sensors onboard the UAV, (2) the known location of the visual marker, and (3) a set of points in an image of the visual marker taken by the visual sensor onboard the UAV. The dimensions can be used to calculate the distance of the UAV from the visual marker. The dimensions can be used to calculate the angle of the UAV relative to the visual marker.

[0022] At least one of the one or more visual sensors may be a camera. When the one or more visual sensors capture the image, the UAV may be autonomously patrolling the environment. One or more processors may be onboard the UAV.

[0023] In some embodiments, the method may further include detecting an obstacle with the aid of one or more sensors onboard the UAV; and adjusting the flight direction of the UAV to avoid the obstacle.

[0024] The method may also include detecting obstacles with the aid of one or more sensors onboard the UAV; and adjusting the flight direction of the UAV to avoid the obstacle.

[0025] The position can be determined with the aid of infrared detectors or radar detectors.

[0026] In one aspect of the present disclosure, a system for locating an unmanned aerial vehicle (UAV) within an environment may include: one or more visual sensors of the UAV, configured to capture an image including a visual marker having a visual pattern and attached to a surface within the environment while the UAV is flying; and one or more processors, configured individually or collectively to: (1) calculate a size of at least a portion of the visual pattern of the visual marker in the image; and (2) determine a position of the UAV within a three-dimensional coordinate system based on the size of the portion of the visual pattern.

[0027] The environment may be an indoor environment. The environment may be an environment in which GPS signals are unreliable. The UAV may weigh less than 1000 g and have a maximum dimension not exceeding 100 cm.

[0028] The marker size may be between vertices of a shape in the visual pattern. The visual marker may be a band having a visual pattern including squares. The visual pattern may include squares of different colors. The vertices of the squares may be detected and used to calculate the size of the portion of the visual pattern. The three-dimensional coordinate system may be a three-dimensional global coordinate system. The three-dimensional coordinate system may be a three-dimensional local coordinate system. The visual pattern may be discerned, and a response may be implemented by the UAV based on the discerned visual pattern.

[0029] The one or more processors may be configured to calculate the rotation and position of the UAV relative to the visual marker using one or more intrinsic characteristics of the one or more visual sensors onboard the UAV, the known position of the visual marker, and a set of points in an image of the visual marker taken by the visual sensors onboard the UAV.

[0030] At least one of the one or more visual sensors may be a camera. The dimensions may be used to calculate the distance of the UAV from the visual marker. The dimensions may be used to calculate the angle of the UAV relative to the visual marker.

[0031] When the one or more visual sensors capture the image, the UAV can autonomously patrol the environment. The one or more processors can be onboard the UAV. The UAV can also be configured to detect obstacles with the help of one or more sensors onboard the UAV; and adjust the flight direction of the UAV to avoid the obstacles.

[0032] The position can be determined with the aid of infrared detectors or radar detectors.

[0033] In another aspect of the present disclosure, a method for locating an unmanned aerial vehicle (UAV) within an environment may include: while the UAV is flying, capturing an image including a visual marker with the aid of one or more visual sensors of the UAV, the visual marker having a unique visual pattern within the environment and attached to a surface within the environment; based on the unique visual pattern of the visual marker in the image, identifying and distinguishing the visual marker from a plurality of different markers, wherein the identified visual marker has a known position that is different from the positions of the plurality of different markers; and determining, with the aid of one or more processors, the position of the UAV within a three-dimensional coordinate system based on the known position of the identified visual marker.

[0034] The environment may be an indoor environment. The environment may be an environment in which GPS signals are unreliable. The UAV may weigh less than 1000 g and have a maximum dimension not exceeding 100 cm.

[0035] The vertices of the shapes in the visual pattern may be detected and used to calculate the position relative to the one or more visual markers. The visual marker may be a band with a unique visual pattern including squares. The unique visual pattern may include squares of different colors. The unique visual pattern may include an AprilTag. The unique visual pattern may include a QR code. The unique visual pattern may include a barcode.

[0036] The method may also include calculating the size of at least a portion of the unique visual pattern of the visual marker in the image; and determining, with the aid of the one or more processors, the position of the UAV within the three-dimensional coordinate system based on the size of the portion of the unique visual pattern.

[0037] The three-dimensional coordinate system may be a three-dimensional global coordinate system or a three-dimensional local coordinate system.

[0038] The method may also include recognizing the visual pattern and implementing a response by the UAV based on the recognized visual pattern.

[0039] The method may also include positioning the UAV within the environment by calculating the rotation and position of the UAV relative to the visual marker using one or more intrinsic characteristics of the one or more visual sensors onboard the UAV, the known position of the visual marker, and a set of points in an image of the visual marker taken by the visual sensor onboard the UAV.

[0040] The one or more processors may be onboard the UAV. The UAV may autonomously patrol the environment while the one or more visual sensors capture the images. The method may also include detecting obstacles with the aid of one or more sensors onboard the UAV; and adjusting the flight direction of the UAV to avoid the obstacles.

[0041] The position can be determined with the aid of infrared detectors or radar detectors.

[0042] In another aspect of the present disclosure, a system for locating an unmanned aerial vehicle (UAV) within an environment may include: one or more visual sensors of the UAV, configured to capture images including visual markers while the UAV is flying, the visual markers having a unique visual pattern within the environment and attached to surfaces within the environment; and one or more processors, configured individually or collectively to: identify and distinguish the visual marker from a plurality of different markers based on the unique visual pattern of the visual marker in the image, wherein the identified visual marker has a known position that is different from the positions of the plurality of different markers; and determine the position of the UAV within a three-dimensional coordinate system based on the known position of the identified visual marker.

[0043] The environment may be an indoor environment. The environment may be an environment where GPS signals are unreliable. The UAV may weigh less than 1000 g and have a maximum dimension not exceeding 1000 cm.

[0044] The vertices of the shapes in the visual pattern may be detected and used to calculate the position relative to the one or more visual markers. The visual marker may be a band with a unique visual pattern including squares. The unique visual pattern may include squares of different colors. The unique visual pattern may include an AprilTag. The unique visual pattern may include a QR code. The unique visual pattern may include a barcode.

[0045] The one or more processors may be configured to calculate a size of at least a portion of the unique visual pattern of the visual marker in the image; and determine a position of the UAV within the three-dimensional coordinate system based on the size of the portion of the unique visual pattern.

[0046] The three-dimensional coordinate system may be a three-dimensional global coordinate system or a three-dimensional local coordinate system.

[0047] The visual pattern may be discerned, and a response may be implemented by the UAV based on the discerned visual pattern.

[0048] The one or more processors may be configured to calculate the rotation and position of the UAV relative to the visual marker using one or more intrinsic characteristics of the one or more visual sensors onboard the UAV, the known position of the visual marker, and a set of points in an image of the visual marker taken by the visual sensors onboard the UAV.

[0049] At least one of the one or more visual sensors may be a camera. When the one or more visual sensors capture the image, the UAV may patrol the environment autonomously. One or more processors may be onboard the UAV. Positioning may be determined with the aid of an infrared detector or a radar detector.

[0050] In another aspect of the present disclosure, a method for locating an unmanned aerial vehicle (UAV) within an environment may include: while the UAV is flying, capturing an image including a visual marker with the aid of one or more visual sensors of the UAV, the visual marker having a visual pattern; with the aid of one or more processors, discerning and distinguishing the visual pattern from a plurality of different patterns, wherein the discerned visual pattern triggers a response from the UAV, the response being different from responses triggered by the plurality of different patterns; and implementing a response by the UAV to the discerned visual pattern of the visual marker.

[0051] The environment may be an indoor environment. The environment may be an environment where GPS signals are unreliable. The UAV may weigh less than 1000 g and have a maximum dimension not exceeding 100 cm.

[0052] The vertices of the shapes in the visual pattern may be detected and used to calculate the position relative to the one or more visual markers. The visual marker may be a band with a unique visual pattern including squares. The unique visual pattern may include squares of different colors. The unique visual pattern may include an AprilTag. The unique visual pattern may include a QR code. The unique visual pattern may include a barcode.

[0053] The response may be independent of the flight of the UAV. The response may include adjusting the position of the payload of the UAV. The payload may be a camera onboard the UAV. The response may include capturing an image of a selected subject. The response may occur within a predetermined amount of time after the visual pattern is discerned and distinguished. The response may occur within a predetermined distance after the visual pattern is discerned and distinguished. The response may include changing the attitude of the UAV. The response may include causing the UAV to fly in a specific direction and / or at a specific speed. The response may include causing the UAV to fly up or down stairs. The response may affect the flight of the UAV. The response may include causing the flight of the UAV to follow a set of preset instructions. In some cases, the visual marker is a band, and the response includes causing the flight of the UAV to follow the band.

[0054] The method may also include calculating the size of at least a portion of the unique visual pattern of the visual marker in the image; and determining, with the aid of the one or more processors, the position of the UAV within the three-dimensional coordinate system based on the size of the portion of the unique visual pattern.

[0055] The three-dimensional coordinate system may be a three-dimensional global coordinate system. The three-dimensional coordinate system may be a three-dimensional local coordinate system. The local coordinates of the UAV may be calculated by using an N-point perspective (PNP) algorithm.

[0056] The method may also include positioning the UAV by calculating a rotation and position of the UAV relative to the visual marker using one or more intrinsic characteristics of the one or more visual sensors onboard the UAV, a known position of a visual marker, and a set of points in an image of the visual marker taken by the visual sensor onboard the UAV. At least one of the one or more visual sensors may be a camera.

[0057] The one or more processors may be onboard the UAV. The UAV may autonomously patrol the environment while the one or more visual sensors capture the images. The method may also include detecting obstacles with the aid of one or more sensors onboard the UAV; and adjusting the flight direction of the UAV to avoid the obstacles. The positioning may also be determined with the aid of an infrared detector or a radar detector.

[0058] In another aspect of the present invention, a system for locating an unmanned aerial vehicle (UAV) within an environment may include: one or more visual sensors of the UAV, which are configured to capture images including a visual marker while the UAV is flying, the visual marker having a visual pattern; and one or more processors, which are configured individually or collectively to: (1) with the aid of the one or more processors, discern and distinguish the visual pattern from a plurality of different patterns, wherein the discerned visual pattern triggers a response from the UAV, the response being different from responses triggered by the plurality of different patterns; and (2) implementing a response by the UAV to the discerned visual pattern of the visual marker.

[0059] The environment may be an indoor environment. The environment may be an environment in which GPS signals are unreliable. The UAV may weigh less than 1000 g and have a maximum dimension not exceeding 100 cm.

[0060] The vertices of the shapes in the visual pattern may be detected and used to calculate the position relative to the one or more visual markers. The visual marker may be a band with a unique visual pattern including squares. The unique visual pattern may include squares of different colors. The unique visual pattern may include an AprilTag. The unique visual pattern may include a QR code. The unique visual pattern may include a barcode.

[0061] The response may be independent of the flight of the UAV. The response may include adjusting the position of the payload of the UAV. The payload may be a camera onboard the UAV. The response may include capturing an image of a selected subject. The response may occur within a predetermined amount of time after the visual pattern is discerned and distinguished. The response may occur within a predetermined distance after the visual pattern is discerned and distinguished. The response may include changing the attitude of the UAV. The response may include causing the UAV to fly in a specific direction and / or at a specific speed. The response may include causing the UAV to fly up or down stairs. The response may affect the flight of the UAV. The response may include causing the flight of the UAV to follow a set of preset instructions. In some cases, the visual marker is a band, and the response includes causing the flight of the UAV to follow the band.

[0062] The system may also include calculating a size of at least a portion of the visual pattern of the visual marker in the image; and determining, with the aid of the one or more processors, a position of the UAV within the three-dimensional coordinate system based on the size of the portion of the visual pattern.

[0063] The three-dimensional coordinate system may be a three-dimensional global coordinate system. The three-dimensional coordinate system may be a three-dimensional local coordinate system. The local coordinates of the UAV may be calculated by using an N-point perspective (PNP) algorithm.

[0064] The one or more processors are configured to calculate the rotation and position of the UAV relative to the visual marker using one or more inherent characteristics of the one or more visual sensors onboard the UAV, the known position of the visual marker, and a set of points in an image of the visual marker taken by the visual sensor onboard the UAV. At least one of the one or more visual sensors can be a camera.

[0065] When the one or more visual sensors capture the image, the UAV may autonomously patrol the environment. The one or more processors may be onboard the UAV. The system may also include detecting obstacles with the aid of one or more sensors onboard the UAV; and adjusting the flight direction of the UAV to avoid the obstacles. The positioning may also be determined with the aid of an infrared detector or a radar detector.

[0066] In another aspect of the present disclosure, a method for locating an unmanned aerial vehicle (UAV) within an environment may include: providing a plurality of visual markers within the environment, wherein each of the visual markers within the environment has a visual pattern that is unique to the environment; capturing an image including a detected visual marker having a visual pattern discerned from the plurality of visual markers by means of one or more visual sensors of the UAV while the UAV is flying within the environment; and implementing, by means of one or more processors, a navigation response by the UAV to the detected visual marker having the discerned visual pattern.

[0067] The navigation response may include determining the position of the UAV in a three-dimensional coordinate system. The navigation response may include causing the UAV to fly according to a set of preset flight commands. The navigation response may include causing the UAV to fly to a predetermined position relative to the detected visual marker. The navigation response may include causing the UAV to fly to a position from which the one or more visual sensors of the UAV are configured to capture an image of a visual marker different from the detected visual marker. The environment may be an indoor environment. The environment may be an environment where a global positioning system (GPS) signal is unreliable. The UAV may weigh less than 1000g and have a maximum dimension of no more than 100cm.

[0068] The vertices of the shapes in the visual pattern may be detected and used to calculate the position relative to the one or more visual markers. The visual marker may be a band with a unique visual pattern including squares. The unique visual pattern may include squares of different colors. The unique visual pattern may include an AprilTag. The unique visual pattern may include a QR code. The unique visual pattern may include a barcode.

[0069] The response may be independent of the flight of the UAV. The response may include adjusting the position of the payload of the UAV. The payload may be a camera onboard the UAV. The response may include capturing an image of a selected subject. The response may occur within a predetermined amount of time after the visual pattern is discerned and distinguished. The response may occur within a predetermined distance after the visual pattern is discerned and distinguished. The response may include changing the attitude of the UAV. The response may include causing the UAV to fly in a specific direction and / or at a specific speed. The response may include causing the UAV to fly up or down stairs. The response may affect the flight of the UAV. The response may include causing the flight of the UAV to follow a set of preset instructions. In some cases, the visual marker is a band, and the response includes causing the flight of the UAV to follow the band.

[0070] The method may also include calculating the size of at least a portion of the visual pattern of the visual marker in the image; and determining, with the aid of the one or more processors, the position of the UAV within the three-dimensional coordinate system based on the size of the portion of the visual pattern.

[0071] The three-dimensional coordinate system may be a three-dimensional global coordinate system. The three-dimensional coordinate system may be a three-dimensional local coordinate system. The local coordinates of the UAV may be calculated by using an N-point perspective (PNP) algorithm.

[0072] The method may also include positioning the UAV within the environment by calculating the rotation and position of the UAV relative to the visual marker using one or more intrinsic characteristics of the one or more visual sensors onboard the UAV, the known position of the visual marker, and a set of points in an image of the visual marker taken by the visual sensor onboard the UAV.

[0073] At least one of the one or more visual sensors may be a camera. When the one or more visual sensors capture the image, the UAV may autonomously patrol the environment. The one or more processors may be onboard the UAV. The method may also include detecting obstacles with the aid of one or more sensors onboard the UAV; and adjusting the flight direction of the UAV to avoid the obstacle. The positioning may also be determined with the aid of an infrared detector or a radar detector.

[0074] In another aspect of the present disclosure, a method for locating an unmanned aerial vehicle (UAV) within an environment may include: providing a continuous visual marker within the environment, wherein the continuous visual marker provides a path within the environment; while the UAV is flying within the environment, capturing an image including at least a portion of the continuous visual marker having a visual pattern by means of one or more visual sensors of the UAV; and implementing a flight response by the UAV to keep the continuous visual marker within the line of sight of the UAV.

[0075] The continuous marking may be a belt. The environment may be an indoor environment. The environment may be an environment where GPS signals are unreliable. The UAV may weigh less than 1000 g and have a maximum dimension not exceeding 100 cm.

[0076] The vertices of the shapes in the visual pattern may be detected and used to calculate the position relative to the one or more visual markers. The visual marker may be a band with a unique visual pattern including squares. The unique visual pattern may include squares of different colors. The unique visual pattern may include an AprilTag. The unique visual pattern may include a QR code. The unique visual pattern may include a barcode.

[0077] The response may be independent of the flight of the UAV. The response may include adjusting the position of the payload of the UAV. The payload may be a camera onboard the UAV. The response may include capturing an image of a selected subject. The response may occur within a predetermined amount of time after the visual pattern is discerned and distinguished. The response may occur within a predetermined distance after the visual pattern is discerned and distinguished. The response may include changing the attitude of the UAV. The response may include causing the UAV to fly in a specific direction and / or at a specific speed. The response may include causing the UAV to fly up or down stairs. The response may affect the flight of the UAV. The response may include causing the flight of the UAV to follow a set of preset instructions. In some cases, the visual marker is a band, and the response includes causing the flight of the UAV to follow the band.

[0078] When the one or more visual sensors capture the image, the UAV can autonomously patrol the environment. The method can also include detecting obstacles with the help of one or more sensors onboard the UAV; and adjusting the flight direction of the UAV to avoid the obstacle. Positioning can also be determined with the help of an infrared detector or a radar detector.

[0079] Other objects and features of the present invention will become apparent by examining the specification, claims and drawings.

[0080] Incorporation by reference

[0081] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The novel features of the present invention are particularly set forth in the appended claims. The features and advantages of the present invention will be better understood by reference to the following detailed description and accompanying drawings which set forth illustrative embodiments in which the principles of the invention are utilized; in the accompanying drawings:

[0083] Figure 1 A UAV configured for surveillance in an environment is shown.

[0084] Figure 2 Examples of visual markers are shown.

[0085] Figure 3 Examples of visually marked vertices that may be detected by a UAV are shown.

[0086] Figure 4 A UAV is shown with onboard components.

[0087] Figure 5 The UAV is shown detecting visual markers from two vertical positions.

[0088] Figure 6 The UAV is shown detecting visual markers from two horizontal positions.

[0089] Figure 7 The UAV is shown detecting and avoiding obstacles by determining its position based on the detection of visual markers.

[0090] Figure 8 The UAV is shown eliminating false detection events of visual markers.

[0091] Fig. 9 Visual indicia are shown that include flags to communicate instructions to the UAV.

[0092] Fig.10 The UAV is shown following a visual marker tape along a path.

[0093] Fig.11 The UAV is shown responding to position instructions from multiple visual markers.

[0094] Fig.12 The UAV is shown interpreting the environment based on information encoded on visual markers.

[0095] Fig.13 The UAV is shown adjusting the position of the payload in response to instructions encoded by visual markers.

[0096] Fig.14 An unmanned aerial vehicle according to an embodiment of the present invention is shown.

[0097] Fig.15 A movable object including a carrier and a payload according to an embodiment of the present invention is shown.

[0098] Fig.16 is a schematic diagram of a system for controlling a movable object according to an embodiment of the present invention, the schematic diagram being illustrated by a block diagram. DETAILED DESCRIPTION

[0099] The system, apparatus and method of the present invention provide a positioning method for automatically assisting an unmanned aerial vehicle (UAV) to navigate in an environment. The description of UAV is applicable to any other type of unmanned vehicle or any other type of movable object.

[0100] UAV can be provided for patrolling environment, to collect information about the environment or one or more subjects in the environment. UAV can follow a defined path through the environment, or UAV can follow an unstructured path. UAV can autonomously or semi-autonomously follow the subject of interest and collect sensory data about the subject of interest. Sensory data can be visual data, audio data, position data and / or movement data. UAV located in the monitoring environment can receive commands from an off-board control center through a communication network. The communication network can be wired or wireless. In some cases, UAV can receive communication from a pattern recognition mechanism, which interprets the instructions encoded on the visual markers placed in the environment.

[0101] The UAV can collect visual information in the environment. Visual sensors can be used to collect visual information. In some cases, the visual sensor can be a camera. The visual sensor can also be used to detect visual markers to determine the position of the UAV in the environment. In some cases, one visual sensor can be used to detect both visual markers and collect visual information in the environment. In other cases, the UAV can have at least two visual sensors so that the first visual sensor is configured to detect visual markers and the second visual sensor is configured to collect visual information in the environment. The visual sensor can be located in the fuselage of the UAV, or the visual sensor can be an external load attached to the UAV. The UAV can also use an audio sensor (e.g., a microphone) to collect audio information in the environment. The collected information can be collected and stored on a memory storage device located onboard or off the UAV. The collected information can be monitored by a user in real time. Alternatively, the collected information can be stored on a memory storage device and reviewed by a user after collection. In some cases, the UAV can send an audio or visual signal as a warning or indication that the UAV has entered or left a specified location.

[0102] Visual markers detectable by one or more visual sensors of the UAV may be distributed in the environment. Visual markers may or may not be visually distinguishable from each other. Visual markers may be located at known locations and may be used to determine the UAV location. Visual markers may also be used to assist the UAV in navigating in the environment and / or performing tasks within the environment.

[0103] The UAV may be configured to perform surveillance tasks in an environment to perform safety assessments. The UAV may collect information in the environment using a plurality of sensors that communicate with one or more processors onboard or offboard the UAV. The processor may interpret the collected information to determine events that may indicate a dangerous or unsafe condition. When a dangerous or unsafe condition is detected by the processor, the processor may instruct the UAV to generate an alarm.

[0104] Figure 1An example of an unmanned aerial vehicle (UAV) 101 is shown. The UAV 101 may have one or more sensors. The UAV 101 may include one or more visual sensors 102, such as an image sensor. For example, the image sensor may be a monocular camera, a stereo vision camera, a radar, a sonar, or an infrared camera. The UAV 101 may also include other sensors that may be used to determine the position of the UAV, such as a global positioning system (GPS) sensor, an inertial sensor (e.g., accelerometer, gyroscope, magnetometer) that may be used as part of an inertial measurement unit (IMU) or separated therefrom, a laser radar, an ultrasonic sensor, an acoustic sensor, a WiFi sensor. The UAV may have sensors onboard the UAV that collect information directly from the environment without contacting additional components outside the UAV for additional information or processing. For example, a sensor that collects data directly in the environment may be a visual sensor or an audio sensor. Alternatively, the UAV may have sensors onboard the UAV but in contact with one or more components outside the UAV to collect data about the environment. For example, a sensor that contacts a component outside the UAV to collect data about the environment may be a GPS sensor or another sensor that relies on a connection to another device, such as a satellite, tower, router, server, or other external device. Various examples of sensors may include, but are not limited to, a position sensor (e.g., a global positioning system (GPS) sensor, a mobile device transmitter that supports position triangulation), a visual sensor (e.g., an imaging device capable of detecting visible light, infrared light, or ultraviolet light, such as a camera), a proximity or range sensor (e.g., an ultrasonic sensor, a lidar, a time-of-flight camera, or a depth camera), an inertial sensor (e.g., an accelerometer, a gyroscope, an inertial measurement unit (IMU)), an altitude sensor, a posture sensor (e.g., a compass), a pressure sensor (e.g., a barometer), an audio sensor (e.g., a microphone), or a field sensor (e.g., a magnetometer, an electromagnetic sensor). Any suitable number and combination of sensors may be used, such as one, two, three, four, five, or more sensors. Optionally, data may be received from sensors of different types (e.g., two, three, four, five, or more types). Different types of sensors may measure different types of signals or information (e.g., position, orientation, velocity, acceleration, proximity, pressure, etc.) and / or utilize different types of measurement techniques to acquire data. For example, sensors may include any suitable combination of active sensors (e.g., sensors that generate and measure energy from their own energy source) and passive sensors (e.g., sensors that detect available energy).As another example, some sensors may generate absolute measurements based on a global coordinate system (e.g., position data provided by a GPS sensor, attitude data provided by a compass or magnetometer), while other sensors may generate relative measurements based on a local coordinate system (e.g., relative angular velocity provided by a gyroscope; relative translational acceleration provided by an accelerometer; relative attitude information provided by a visual sensor; relative distance information provided by an ultrasonic sensor, a lidar, or a time-of-flight camera). Sensors onboard or offboard the UAV may collect information such as the location of the UAV, the location of other objects, the orientation of the UAV 101, or environmental information. A single sensor may be able to collect a complete set of information in an environment, or a group of sensors may work together to collect a complete set of information in an environment. Sensors may be used for mapping locations, navigation between locations, detection of obstacles, or detection of targets. Sensors may be used for monitoring an environment or a subject 103 of interest.

[0105] Any description of UAV 101 herein may be applicable to any type of movable object. The description of UAV may be applicable to any type of unmanned movable object (e.g., it may travel through the sky, land, water, or space). UAV may be able to respond to commands from a remote controller. The remote controller may not be connected to the UAV, and the remote controller may communicate with the UAV wirelessly from a distance. In some cases, the UAV may be able to operate autonomously or semi-autonomously. The UAV may be able to follow a set of pre-programmed instructions. In some cases, the UAV may be able to operate semi-autonomously in response to one or more commands from the remote controller, and otherwise operate autonomously. For example, one or more commands from the remote controller may initiate a series of autonomous or semi-autonomous actions performed by the UAV according to one or more parameters.

[0106] The UAV 101 may be an aircraft. The UAV may have one or more power units that allow the UAV to move around in the air. The one or more power units may enable the UAV to move about one or more, two or more, three or more, four or more, five or more, six or more degrees of freedom. In some cases, the UAV may be able to rotate around one, two, three or more rotation axes. The rotation axes may be orthogonal to each other. The rotation axes may remain orthogonal to each other throughout the flight of the UAV. The rotation axes may include a pitch axis, a roll axis and / or a yaw axis. The UAV may be able to move along one or more dimensions. For example, the UAV may be able to move upward due to the lift generated by one or more rotors. In some cases, the UAV may be able to move along a Z axis (which may be upward relative to the orientation of the UAV), an X axis and / or a Y axis (which may be lateral). The UAV may be able to move along one, two or three axes that may be orthogonal to each other.

[0107] UAV 101 can be a rotorcraft. In some cases, the UAV can be a multi-rotor aircraft that can include multiple rotors. The multiple rotors can be able to rotate so as to generate lift for the UAV. The rotor can be a power unit that can enable the UAV to move freely in the air. The rotor can rotate at the same rate and / or can generate equal amounts of lift or thrust. The rotor can optionally rotate at different rates, which can generate unequal amounts of lift or thrust and / or allow the UAV to rotate. In some cases, one, two, three, four, five, six, seven, eight, nine, ten or more rotors can be provided on the UAV. The rotors can be arranged so that their rotation axes are parallel to each other. In some cases, the rotors can have rotation axes at any angle relative to each other, which can affect the movement of the UAV.

[0108] The UAV shown may have multiple rotors. The rotors may be connected to the fuselage of the UAV, which may include a control unit, one or more sensors, a processor, and a power supply. The sensors may include visual sensors and / or other sensors that may collect information about the UAV environment. The information from the sensors may be used to determine the position of the UAV. The rotors may be connected to the fuselage via one or more arms or extensions that may be separated from the central portion of the fuselage. For example, one or more arms may extend radially from the central fuselage of the UAV and may have rotors at or near the ends of the arms.

[0109] By maintaining and / or adjusting the output of one or more power units of the UAV, the vertical position and / or speed of the UAV can be controlled. For example, increasing the rotation speed of one or more rotors of the UAV can assist the UAV to increase altitude or increase altitude at a faster rate. Increasing the rotation speed of the one or more rotors can increase the thrust of the rotor. Reducing the rotation speed of one or more rotors of the UAV can assist the UAV to reduce altitude or reduce altitude at a faster rate. Reducing the rotation speed of the one or more rotors can reduce the thrust of the one or more rotors. When the UAV takes off, the output that can be provided to the power unit can be increased from its previous landed state. When the UAV lands, the output provided to the power unit can be reduced from its previous flight state. The UAV can be configured to take off and / or land in a substantially vertical manner.

[0110] By maintaining and / or adjusting the output of one or more power units of the UAV, the lateral position and / or speed of the UAV can be controlled. The height of the UAV and the rotation speed of one or more rotors of the UAV can affect the lateral movement of the UAV. For example, the UAV can be tilted in a particular direction to move in that direction, and the speed of the rotors of the UAV can affect the speed and / or movement trajectory of the lateral movement. The lateral position and / or speed of the UAV can be controlled by changing or maintaining the rotation speed of one or more rotors of the UAV.

[0111] The UAV 101 may have a small size. The UAV may be capable of being carried and / or carried by a human. The UAV may be capable of being carried by a human in one hand.

[0112] UAV 101 can have the maximum dimension (for example, length, width, height, diagonal, diameter) of no more than 100cm. In some cases, the maximum dimension can be less than or equal to 1mm, 5mm, 1cm, 3cm, 5cm, 10cm, 12cm, 15cm, 20cm, 25cm, 30cm, 35cm, 40cm, 45cm, 50cm, 55cm, 60cm, 65cm, 70cm, 75cm, 80cm, 85cm, 90cm, 95cm, 100cm, 110cm, 120cm, 130cm, 140cm, 150cm, 160cm, 170cm, 180cm, 190cm, 200cm, 220cm, 250cm or 300cm. Optionally, the maximum dimension of UAV can be greater than or equal to any one of the values ​​described herein. UAV can have the maximum dimension falling in the range between any two of the values ​​described herein.

[0113] The UAV 101 can be lightweight. For example, the UAV can weigh less than or equal to 1 mg, 5 mg, 10 mg, 50 mg, 100 mg, 500 mg, 1 g, 2 g, 3 g, 5 g, 7 g, 10 g, 12 g, 15 g, 20 g, 25 g, 30 g, 35 g, 40 g, 45 g, 50 g, 60 g, 70 g, 80 g, 90 g, 100 g, 120 g, 150 g, 200 g, 250 g, 300 g, 350 g, 400 g, 450 g, 500 g, 600 g, 700 g. The UAV may have a weight greater than or equal to any of the values ​​described herein. The UAV may have a weight within the range of any two of the values ​​described herein.

[0114] UAV 101 may include visual sensors, such as monocular cameras, stereo vision cameras, radars, sonars, or infrared cameras. UAV 101 may also include sensors, such as GPS, IMU, laser radar, or any other type of sensor described elsewhere herein. Sensors onboard the UAV may collect information, such as the position of the UAV, the position of other objects, the orientation of the UAV 101, or environmental information. A single sensor may be able to fully determine any of the above parameters, or a group of sensors may work together to determine one of the listed parameters. Sensors may be used for mapping of locations, navigation between locations, detection of obstacles, detection of targets, or monitoring of objects of interest or environments.

[0115] The sensor may be located on or off the UAV. The onboard sensor may be located on the fuselage of the UAV 101. The sensor may be attached to the exterior of the fuselage of the UAV 101 and / or the sensor may be attached to the interior of the fuselage of the UAV 101. The sensor may be located at the center of a single area on the fuselage. Alternatively, the sensor may be located at different locations on the fuselage. The sensor may be permanently or removably attached to the UAV 101. The UAV 101 may have a carrier that may be configured to carry a payload. The sensor may be attached to the carrier.

[0116] A sensor may be characterized by one or more sensor parameters. A sensor parameter may be an intrinsic parameter or an extrinsic parameter. An intrinsic parameter may relate to the internal configuration of the sensor. Examples of intrinsic parameters may include focal length, scale factor, radial distortion coefficient, and tangential distortion coefficient. An intrinsic parameter may be any parameter that depends on the hardware configuration, and in some cases, the intrinsic parameter may be set by the factory settings of the sensor. An extrinsic parameter may relate to the spatial relationship between any two or more sensors. Each sensor may have a relative coordinate system that is independent of other sensors onboard the UAV. External characteristics may be important for sensor fusion (combining data from sensors at different locations on the UAV). Sensor fusion may involve the process of transforming the relative coordinates of a given sensor to match the reference system of another sensor.

[0117] The UAV can be configured to monitor in a space or environment. The space can be an outdoor space or an indoor space or a combination of an indoor space and an outdoor space. The UAV can automatically or semi-automatically locate itself in the space. The UAV can locate itself with a position sensor (e.g., a GPS sensor). Alternatively, the UAV can locate itself by interpreting the image of a visual marker. The visual marker can be an alternative positioning mechanism for a positioning sensor in an environment where the positioning sensor is unreliable. The visual marker can be used in combination with one or more types of positioning sensors. The visual marker can have a positioning marker. The visual marker can adhere or be attached to one or more surfaces in a space or environment.

[0118] The data about the flight of the UAV can be related to the data captured by one or more sensors of the UAV for monitoring. For example, the position (e.g., latitude, longitude, altitude or any other coordinate system) of the UAV, the attitude (e.g., about the angle of the roll axis, pitch axis and / or yaw axis) of the UAV can be stored. Information about the UAV such as position and attitude can be related to timing data. For example, a clock can be provided on the UAV machine or on an external device, which can record time for UAV positioning. Time data can be associated with the corresponding position and / or attitude of the UAV. This time-based data can be related to the data captured by one or more sensors of the UAV (such as image data captured by the camera onboard the UAV). Optionally, camera data (e.g., the attitude and / or timing information of the camera relative to the UAV) can be related to other data. In some embodiments, the UAV can be able to freely rotate at least about 45 degrees, 90 degrees, 180 degrees, 270 degrees, 360 degrees or 720 degrees around any of the axes described herein. The payload (e.g., camera) may be able to freely rotate at least about 45, 90, 180, 270, 360, or 720 degrees relative to the UAV about any of the axes described herein. During monitoring, this data may be used to determine the location of the UAV and to understand the location of the area captured by the sensor (e.g., the field of view captured by the camera).

[0119] Figure 2 Examples of positioning markers are shown. Positioning markers may exist in indoor environments, such as in buildings. The interior of a building may have a weak GPS signal or no GPS signal. Positioning markers may be adhered to surfaces in the environment. Throughout the environment, there may be one or more positioning markers and / or continuous bands of positioning markers. One or more positioning markers and / or continuous bands of positioning markers may be adhered to one or more surfaces in a room, corridor, alcove, closet or other surroundings. The environment may be an indoor environment. Positioning markers may have discrete shapes. For example, the positioning marker may be square, rectangular, circular, triangular, hexagonal or have any other shape. Positioning markers may be adhered to ceilings, walls, support beams, objects or floors in indoor environments. Positioning markers may be continuous bands. Positioning markers may be a series of unconnected markers with uniform or non-uniform sizes. Positioning markers may be black and white, or positioning markers may be in multiple colors. Positioning markers may include multiple geometric shapes with identifiable vertices. The multiple geometric shapes may include visual patterns. The geometric shapes may be triangles, squares, pentagons, hexagons or any other regular or irregular polygons. Positioning marker visual patterns may include at least one type of polygon. The positioning marks may include a repeating pattern, a checkerboard, repeating polygons in rows and / or columns, or an array of polygons. The positioning marks may have shapes that alternate in color in a consistent repeating pattern. Figure 2 In the example shown in , the markers include a set of squares. Figure 2 The example notation in shows black and white squares, alternatively the squares may be other colors.

[0120] In an alternative embodiment, the marker may not be a visual marker. The marker may be detected by another sensory interaction. In some cases, the marker may be an audio marker. When the UAV leaves or enters a specified location, the marker may provide an audio or visual warning. The marker may emit an acoustic interference within a frequency range that can be detected by the UAV. The frequency range of the acoustic interference emitted may be in a frequency range outside the frequency range detected by humans. The UAV may determine its position by analyzing the distortion in the acoustic interference. The acoustic interference may instruct the UAV to move to perform a task. In another case, the marker may emit an RFID, radio or WiFi signal that may be detected by the UAV and processed by the UAV to determine the position. In some embodiments, the marker may emit a detectable substance, such as a gas. The detectable substance may be safe for inhalation and consumption by humans and other animals. The UAV may determine its position relative to the marker using the concentration of the gas at the location where the UAV detects the gas and the diffusion characteristics of the gas.

[0121] Different markers at different locations may be distinguishable from each other. For example, different markers may emit different signals. In one example, the sound emitted by a first audio marker may be different from the sound emitted by a second audio marker. The sounds may have different frequencies, harmonics, or patterns. Thus, by identifying the sound patterns, the UAV may be able to distinguish whether the UAV is near a first audio marker at a first location or a second audio marker at a second location. Similarly, different radios may emit different signals, or different gas markers may emit gases of different compositions. The UAV may be able to determine its location based on the identified markers and the known locations of the identified markers.

[0122] Any description herein of visual marks or positioning marks may apply to any other type of mark, including non-visual marks, and vice versa.

[0123] The positioning mark can be made of paper or plastic material. The positioning mark can be a screen display (e.g., a liquid crystal display (LCD), a touch screen, an LED screen, an OLED screen, or a plasma screen display). Alternatively, the positioning mark can be projected onto the surface from a projector installed in the environment. The positioning mark can be a screen display or a projector that communicates with a computer unit. The positioning mark can be static (e.g., unchanged) or dynamic (e.g., variable). The computer unit can change the pattern on the positioning mark. Changing the pattern can change the interaction of the UAV with the positioning mark. The positioning mark can be adhered to the surface with glue, resin, magnets, or hardware components (e.g., screws, fasteners, Velcro, rivets, nails, or snaps). The positioning mark can be permanently or removably attached to the surface.

[0124] In the environment, the visual marker can be uniform. The environment can contain only one type of visual marker, wherein the type of visual marker can refer to a specific pattern on the visual marker. The difference in the pattern on the visual marker can be the difference in the shape type in the pattern, the number of shapes in the pattern, the shape color in the pattern, the difference in the size ratio of the shape in the pattern and / or the difference in the coded pattern. In some cases, the environment can contain more than one type of visual marker. Different visual markers in the environment can be recognized by different UAVs. The processor onboard or outside the UAV can be configured to distinguish between different visual markers. In an example, different UAVs can be UAVs with different missions, with different functions, or belonging to different companies. In some embodiments, the UAV can be configured to only recognize and follow one type of visual marker.

[0125] For example, the first UAV may be configured to recognize a first type of visual marker, but not a second type of visual marker. The second UAV may be configured to recognize a second type of visual marker, but not a first type of visual marker. For example, the first visual marker type may include alternating black squares and white squares. The second visual marker type may include alternating red circles and green circles. The first UAV may only recognize black squares and white square patterns as visual markers, while ignoring red circle and green circle patterns. Similarly, the second UAV may only recognize red circle and green circle patterns as visual markers, while ignoring black square and white square patterns.

[0126] The UAV can recognize static patterns. For example, the UAV can recognize a still image as a visual marker. The still image can be changed from a first still image to a second still image. For example, the visual marker can be implemented as a screen that can show a first still image and then change to a second still image. In some cases, the UAV can recognize both still images as markers, or different UAVs can recognize still images as markers (for example, the first UAV can recognize the first still image as a marker, and the second UAV can recognize the second still image as a marker). In some cases, the UAV can recognize a visual marker that can be changed. The actual change or sequence of the change can be part of what allows the UAV to recognize the visual marker. The UAV can recognize the change of the visual marker over time to distinguish the visual marker. For example, the visual marker can be implemented as a screen that can show a video sequence.

[0127] Visual markers can be interpreted by UAV to determine the location or position of the UAV in the environment. In the case of unreliable global positioning system (GPS) signals, visual markers can be used to locate the UAV in the environment. In an example, GPS signals may be unreliable in indoor environments and in remote locations. UAVs may be able to determine their position locally or globally in the environment. Local position may refer to a position relative to an object, a landmark, or a known location in the environment. In an example, a local position may be the cardinal direction limit of the environment (e.g., the north end of a building) or the distance above an object or landmark and an angle (e.g., an azimuth of 30°, a posture of 45°, and a radial distance of 10 feet from a landmark). Alternatively, UAVs may determine their global position in the environment. Global position may be a position in space, such as longitude and latitude. UAVs may monitor in the environment. In addition, visual markers may convey instructions to UAVs. Instructions may be commands related to UAV position, UAV mission, UAV sensor, information processing, and / or to interactions with people, objects, or vehicles in the environment. UAVs may execute the commands within a specified time interval after receiving the commands. The UAV can execute the command after traveling a specified distance from the marked distance of the conveyed command. The command may involve the motion of controlling the UAV. In an example, the instruction may be to increase or decrease the height, change the directional heading, land on the surface, turn on or off the sensor, capture data with the sensor, transmit sensor data to an off-board server or processor, or interact with another vehicle or object in the environment. The instruction may cause a response from the UAV. In an example, the response may be to adjust the position of the load, the exchange of the load, capture the image of the selected subject, make the UAV fly or advance in a specific direction with a specific speed or acceleration and / or height, or make the UAV fly up or down the stairs. The instruction for switching the load may include the instruction to pick up the load, unload the load, and / or add an additional load. The instruction may specify a specific location to bring the load. In some cases, the instruction may sound an alarm provided on the UAV machine.

[0128] The visual markers may be detected by visual sensors onboard the UAV. Figure 4A block diagram of possible components that may be present onboard a UAV 400 is shown. The UAV 400 may have one or more sensors 401. The one or more sensors may include a visual sensor, such as a monocular camera. The one or more sensors may be in electronic or network communication with a processor 402 via a wired or wireless connection. The processor may be configured to analyze sensor data. The analysis may include using the sensor data to determine or calculate the position of the UAV. The UAV may also include an onboard memory storage device 403. The memory storage device may be in electronic or network communication with the processor and / or one or more sensors via a wired or wireless connection. The memory storage device may have stored information about the environment. The stored information about the environment may be related to the geographic location of the environment in a global coordinate system, the layout of the environment, the size of one or more spaces in the environment, the features in the environment, and / or the location and size of objects in the environment. In an example, the stored information may be a blueprint, a map, or a floor plan. The stored information may also include the location of a visual marker. The location of the marker may be related to a blueprint, a map, or a floor plan. The information stored on the memory storage device may be updated periodically or updated when changes occur in the environment. The information stored on the memory storage device can be updated by a device that communicates with the memory storage device wirelessly or through a wired connection. In an alternative embodiment, the UAV can be provided with information about the environment from a storage device outside the UAV. The UAV 400 can also include a power source 404 onboard the UAV, such as a battery. The UAV can be configured to carry a load 405, which can be an additional sensor, cargo, or a backup battery. The load 405 can be rotated and translated to achieve multiple orientations relative to the UAV 400.

[0129] The visual sensor can convey instructions to the UAV through a pattern recognition mechanism. Pattern recognition can be performed by a processor on the UAV. The processor can be configured to recognize a pattern and can determine that the pattern conveys a command for a function or action. In some cases, the pattern can be a QR code, AprilTag, or barcode. The pattern can include any shape, image, icon, letter, symbol, number, or pattern. The pattern can be a one-dimensional pattern, a two-dimensional pattern, or a three-dimensional pattern. For example, the pattern can include ridges or protrusions that can protrude and be recognized by the UAV.

[0130] The UAV can know the position of each visual marker or visual marker band. The position of the visual marker can be stored in a memory device on or off the UAV. Each visual marker or visual marker band can have a unique position. The UAV can be configured to locate a specific visual marker and know the position of the visual marker.

[0131] The UAV may be configured to locate the vertices of a polygon or multiple polygons on the positioning marker. Figure 3 Vertices that can be detected by the UAV are shown. The UAV can detect the vertices of a square or other polygon and calculate the size of the visual pattern. The size of the visual pattern can be interpreted to determine the position of the UAV. The position of the UAV can be determined in a three-dimensional coordinate system. The three-dimensional coordinate system can be a global coordinate system or a relative or local coordinate system.

[0132] The UAV may use a camera to detect a visual pattern and capture an image of the visual pattern. The UAV may capture an image of a portion of the visual pattern or the entire visual pattern. The UAV may have one or more processors configured to process an image of a visual pattern. The one or more processors may be on or off the UAV. The processor may communicate with one or more memory storage devices. The image of the visual pattern may be stored on the memory storage device. The UAV may use the one or more processors to determine the distance between two or more vertices. The UAV may use the one or more processors to determine the angle between one or more vertices. The distance and angle between vertices may include the size of a visual marker. The size of the visual marker may be used as an input to calculate the distance between the UAV and the marker and the angle of the UAV relative to the marker.

[0133] The UAV's perception of the size of the visual marker can be a function of the UAV's position relative to the visual marker. The perceived size of the visual marker can vary with the distance between the UAV and the visual marker. The size of the marker can be described as the distance between the detected vertices of the polygon on the marker. Based on the apparent size of the marker relative to the known marker size, the UAV can determine its height or its vertical distance from the marker. When the vertical distance between the UAV and the marker is relatively large, the marker may appear smaller than its actual size. Figure 5 An example of a change in the visual perception of a marker as a function of the vertical position (altitude) of the UAV is shown in FIG. 5 . At a first position 501, the UAV is relatively far away vertically from the visual marker. The UAV may be vertically close to the visual marker so that the visual marker appears relatively large at a second position 502. The change in apparent marker size as a function of the vertical distance from the marker may be known and may be stored in a memory storage device onboard or offboard the UAV. The change in apparent marker size as a function of the vertical distance from the marker may be determined by calibration or by a known mathematical relationship.

[0134] The UAV can detect distortion in the shape of the marker as a function of the offset from the marker or the horizontal displacement from the marker. The distortion can be measured as the angular difference between the coordinates of the detected vertices of the marker polygon. In the case where the polygon is a square, the UAV can expect the square to have four vertices at 90° angles relative to each other. When the UAV deviates from a marker containing a square or other polygon, the angles between the vertices may be distorted. In the case where the UAV deviates from a marker containing a square, the shape of the square may be distorted so that it appears as a diamond to the UAV. Similar distortions may occur in other shapes. Figure 6 An example of distortion detection is described in 601 , where the UAV may approach a marker horizontally. The angle of the vertices may vary as a function of the horizontal distance from the marker. When the UAV is a certain distance from the marker, the angle between the vertices may be distorted. In the first case 601 , when the UAV is a horizontal distance away from the marker, the angle between the vertices may appear shortened to the UAV. When the UAV approaches the marker horizontally so that the UAV is directly above the marker, the angle between the vertices may stretch to a true undistorted angle 602.

[0135] The perceived distortion of the visual marker can be analyzed to determine the horizontal and vertical position of the UAV relative to the marker. Based on the visual marker, one or more processors can determine the position or location of the UAV in a three-dimensional coordinate system. While the UAV is flying, the one or more processors can determine the position or location of the UAV in a three-dimensional coordinate system based on the visual marker. The position of the marker in three-dimensional space can be known, so determining the position of the UAV relative to the marker can result in a determination of the position of the UAV in three-dimensional space.

[0136] Radar or infrared detectors can also assist in determining the position of the UAV in three-dimensional space. In the case of radar detection, the UAV can emit a radar signal or a marker can emit a radar signal. The UAV can interpret the emitted or reflected radar signal to determine the distance between it and the surface or obstacle. The UAV can use the detection of the surface or obstacle and compare it to the stored plan or blueprint to determine the position of the UAV in the environment. Similarly, the UAV can include an infrared detector to detect surfaces or obstacles in the environment. The UAV can compare the detected obstacles and / or surfaces with the known environment layout to determine its position in the environment.

[0137] The position of the UAV in three-dimensional space may be a position in global or local three-dimensional space. The UAV may capture an image of a visual marker using a visual sensor onboard the UAV. The visual sensor onboard the UAV may be a visual sensor for environmental monitoring, or the visual sensor may be a separate visual sensor configured to detect a visual marker. The visual sensor may be a monocular camera, a stereoscopic vision camera, a radar, a sonar, or an infrared camera. In some cases, the visual sensor may capture an image. The image may be stored on a memory storage device onboard or offboard the UAV. The visual sensor image may be analyzed by a processor onboard or offboard the UAV. The processor may analyze the visual sensor image to determine the external characteristics of the visual sensor. In the case where the visual sensor is a camera, the external characteristics of the camera may be the rotation (R) of the camera relative to the visual marker and the position (t) of the camera relative to the visual marker. The external characteristics may be calculated using an N-point perspective (n-point perspective) algorithm, which may solve the position of the camera based on the correspondence from a three-dimensional point to a two-dimensional point. In this case, the three-dimensional point may be a known position of a vertex of a visual marker. The two-dimensional point may be a point corresponding to a vertex of a visual marker in one or more camera images. The camera image may represent a two-dimensional projection of a three-dimensional visual marker. The algorithm may also include variables corresponding to intrinsic characteristics of the camera. Intrinsic characteristics (K) may be parameters specific to the camera design or components. Intrinsic characteristics may change or drift and may be pre-calibrated or recalibrated before the UAV's surveillance mission begins. Intrinsic characteristics of a camera may be focal length, principal point, and / or lens distortion. The following equations are used to make the known positions (X) of the image vertices in three-dimensional space i ) and the vertex position (x) in the camera image i ), the camera position defined by R and t can be solved as a function of 3D points, 2D points, and the camera intrinsic properties (K).

[0138]

[0139] The n-point perspective algorithm can determine the position of a camera or other visual sensor onboard the UAV. The position of the camera onboard the UAV relative to the center of mass of the UAV and / or relative to other sensors or components onboard the UAV can be known. The position of the camera relative to other components or sensors onboard the UAV can be combined with the determined position of the camera to determine the position of the components or sensors onboard the UAV.

[0140] Determination of the UAV position based on determination of the camera onboard the UAV can be used for obstacle avoidance. Visual markers can be attached to obstacles, such as walls, columns, doors, or other objects. The UAV can automatically identify obstacles, or the visual markers can have specific patterns that convey to the UAV that the visual markers are attached to the obstacles. The UAV can capture and analyze the visual markers to determine the distance to the obstacle. In response to the determined distance, the UAV can adjust its heading and / or altitude to avoid the detected obstacle. The UAV can collect a series of images as it approaches an obstacle to determine the change in relative position between the UAV and the obstacle when it changes its heading and / or altitude. Figure 7 701. An example of an obstacle detection and avoidance event using a visual marker is shown in FIG. At a first position 701, a UAV 703 may detect a visual marker 704. The visual marker 704 may have a pattern that may indicate to the UAV 703 that the visual marker 704 is attached to an obstacle 705. The UAV may capture an image of the visual marker, and the image may be analyzed to determine the position of the UAV relative to the position of the visual marker. The UAV may adjust its position to reach a second position 702. After adjusting the position, the UAV 703 may capture a second image of the visual marker 704 to determine the new position and confirm that the new position will avoid a collision or interaction with the obstacle 705. In the event that the second position is not sufficient to avoid the obstacle, the UAV may change position again and capture another image to confirm that the UAV is on a trajectory to avoid the obstacle. The process of capturing images and determining the position of the UAV may be repeated until the obstacle is successfully avoided.

[0141] In an alternative embodiment, the UAV can automatically detect and avoid obstacles. The UAV can use visual sensors such as monocular cameras, stereo vision cameras, radars, sonars, or infrared cameras to detect obstacles. The processor onboard or offboard the UAV can be configured to interpret data from the image sensor in order to detect obstacles. The UAV can automatically adjust its route to avoid detected obstacles. In some cases, the UAV may have prior knowledge of the location of obstacles. In some cases, these obstacles may be permanent building features, such as walls, beams, or columns. The UAV can know these permanent building features from an environmental map or floor plan stored on a memory storage device onboard or offboard the UAV. The UAV can automatically adjust its route to avoid permanent building features that may be obstacles. The UAV can follow a continuous band of visual markers or a series of continuous discrete visual markers. When following the visual markers, the UAV may encounter obstacles. The UAV can avoid the obstacles by re-enacting the UAV path to bypass the obstacles and then return to the path of the visual markers. In some cases, the visual markers can notify the UAV that an obstacle is blocking a path along a visual marker strip or series of visual markers, and the visual markers can instruct the UAV to move to avoid the obstacle and continue along the path.

[0142] The UAV can detect visual markers in the environment. The environment may have other objects and patterns that the UAV can distinguish from the markers. The UAV can be configured to exclude false detection events. A false detection event may occur when the UAV detects a pattern or object with a visual structure similar to a visual marker that the UAV may be interested in. The UAV can be configured to analyze the detected pattern or object to determine whether it is a visual marker of interest or a false visual marker that is mistakenly identified. In the environment, the UAV can attempt to identify the visual marker by scanning areas in the environment with the color and / or shape of the expected visual marker. The method of scanning the expected shape and color pattern or a combination thereof can produce a set of detection areas. The UAV can further analyze the detected areas to exclude false detection events. Figure 8 An example of a UAV 801 in an environment is shown. The UAV 801 is configured to recognize visual markers that include a repeating pattern of a set of shapes (e.g., squares) with a given one or more colors. The UAV can detect a set of possible visual markers 802, 803, and 804 in real time. The UAV can further analyze the detected possible visual markers 802, 803, and 804 to exclude false positive events. The further analysis can include processing the exact color, pattern, shape, and / or one or more sizes of the relative shapes in the visual marker. Through further analysis, the UAV can determine that 802 and 804 are not visual markers of interest and therefore represent false positive events.

[0143] When the UAV detects a visual marker, the UAV may determine its position based on one or more images of the visual marker, and / or receive instructions encoded by the visual marker. The instructions may trigger a navigation response performed by the UAV. The visual marker may include a difference pattern that can be distinguished by the UAV. When the UAV is flying, the UAV may detect a visual marker with a visual difference pattern and may use a visual sensor to capture an image of the visual marker. The image of the visual pattern may be interpreted by a processor onboard or offboard the UAV. The processor may distinguish and discern a visual pattern from a plurality of possible visual patterns, and determine that the discerned visual pattern triggers a response from the UAV that is different from the response triggered by a plurality of other possible visual patterns. The processor may initiate a response triggered by the detected visual pattern.

[0144] The visual marker including the differential pattern may have similar elements to the visual marker without the differential pattern. In addition to having a pattern that can convey a command or a piece of information to the UAV, the visual marker including the differential pattern may also have colored shapes in a repeating pattern. The UAV may know the location of the visual marker including the differential pattern in the environment. In some cases, the differential pattern may be an Apriltag, a QR code, or a barcode. Fig. 9 An example of a visual marker having a distinctive pattern is shown. The UAV can distinguish the visual marker including the distinctive pattern from a sign that looks similar to the visual marker including the distinctive pattern to avoid false detection events.

[0145] The difference pattern can instruct the UAV to perform a task or perform a navigation response. The task or navigation response can be to travel along a path from one place to another, move a sensor, change the attitude of the UAV, fly in a specified or specific direction, fly up or down stairs, adjust the position of the load, fly according to a preset flight command, turn on or off a sensor, guide the sensor in a specified direction, or capture an object. The instruction can include a specified time interval that the UAV should wait before performing the task or navigation response. The instruction can include a predetermined distance that the UAV should be from a visual marker before performing the task or navigation response. The difference pattern can instruct the UAV to follow a visual marker belt along the path or a series of discrete visual markers along the path. When the UAV follows a belt or a series of discrete visual markers along the path, the difference pattern can instruct the UAV to be a specified distance from the belt. The path can be indoors, outdoors, or in both indoor and outdoor environments. Fig.10An example of a visual marker band 1001 that can be followed by a UAV 1002 is shown. The visual marker band 1001 can be adhered to a surface so that the UAV 1002 can follow the band 1001 without colliding or encountering an obstacle. The UAV can be configured to remain at a distance from the visual marker attachment when following the mark. In an example, the attachment distance can be at least 1cm, 5cm, 10cm, 20cm, 30cm, 40cm, 50cm, 60cm, 70cm, 80cm, 90cm or 100cm. The UAV can remain in an attachment orientation relative to the visual marker when following the visual marker. For example, the UAV can remain at a height flush with the visual marker (e.g., approximately 90 degrees). Alternatively, the UAV can fly below or above the visual marker by a predetermined degree or within a predetermined degree range (e.g., within 30 degrees -60 degrees). The UAV can continuously determine its position by analyzing the image of the visual marker 1001 as it follows the marker to detect and avoid obstacles detected by the visual sensors or known from a map of the environment. Fig.10 In the case shown in , the visual marker tape 1001 can guide the UAV 1002 up the stairs. In another case, the differential pattern can instruct the UAV to follow a path by instructing the UAV to move in a specified direction and distance. The differential pattern can instruct the UAV to move at an angle in three-dimensional space. The differential pattern can instruct the UAV to move a specified distance. Through analysis of speed and time sensors that can be located on or outside the UAV, the UAV can calculate the distance traveled. Fig.11 Examples are shown in which the one or more visual indicia comprising a distinctive pattern instructs the UAV to move along a path by providing the UAV with instructions to move in discrete directions and distances. Fig.11 1104. In the example of FIG. 1105, UAV 1101 may detect a first visual marker having a distinctive pattern 1102. Distinctive pattern 1102 may instruct the UAV to move a specified distance 1103 upward and to the right. The next visual marker having a distinctive pattern 1104 detected by the UAV may instruct the UAV 1101 to move upward at a 30° angle 1105. The next visual marker having a distinctive pattern 1106 detected by the UAV may instruct the UAV 1101 to move a specified distance 1107 horizontally.

[0146] In another example, the differential pattern may inform the UAV about the layout of the environment. The differential pattern may inform the UAV about the presence of features in the environment. The features may be openings, aisles, corners, obstacles, slopes, inclines, or objects. Fig.12 In the example shown in , a visual marker having a distinctive pattern 1201 can notify the UAV 1202 that the environment has features such as aisle opening 1203.

[0147] The difference pattern can instruct the UAV to perform a surveillance mission. The surveillance mission can involve moving the sensor to capture data about the subject of interest. Fig.13 In the example shown in FIG. 1 , at a first position 1301, a UAV 1302 may have a sensor for collecting surveillance data in an environment, for example, a camera 1303 configured to collect surveillance data in an environment, the camera having a field of view 1304 directed away from a subject of interest 1305. The UAV 1302 may detect a visual marker having a differential pattern 1306 using a first visual sensor configured to detect the visual marker. The differential pattern 1306 may instruct the UAV 1302 to move or rotate the camera 1303 to a second position 1302 so that the camera's field of view 1304 includes the subject of interest 1305. The differential pattern may also instruct the UAV to perform a specific surveillance task on the subject of interest 1305. The surveillance task may include collecting data about the subject of interest 1305 from one or more sensors and storing or transmitting the data to an off-board computer system.

[0148] UAVs can be used for monitoring an environment. For example, a UAV can patrol an environment. When performing monitoring of an environment, the UAV can use one or more sensors to collect information about the environment. The UAV can follow a predetermined or regular path for patrolling the environment. Alternatively, the UAV can follow a random flight path. The UAV can detect and / or follow subjects of interest, such as people, animals, or vehicles. The UAV can respond to one or more detected events within the environment and conduct further investigation.

[0149] As discussed above, the UAV can patrol the environment by collecting one or more images within the environment. The UAV can send the collected images to an external device. The images can be sent outside the UAV machine. In some cases, the images can be stored in one or more memory units outside the UAV machine. The images can be displayed on a display device away from the UAV. For example, a security monitor can show images streamed from the UAV. A human viewer can view a live feed from the UAV patrolling the area.

[0150] In some embodiments, the environment may be an indoor environment. An indoor environment may include the interior of a building or a group of buildings. An indoor environment may include a multi-story building. The UAV may traverse a multi-story building by going up or down one or more stairs or by directly flying up or down a stairwell. The UAV may also be able to traverse a multi-story building by using an elevator or by flying up or down an elevator shaft. The UAV may also be able to traverse a multi-story building by exiting from a window on one floor and entering through a window on another floor. Some examples of indoor buildings may include a residence, an office building, a shopping mall, a hospital, a school, or any other type of building. The UAV may be used to patrol the environment.

[0151] Visual markers as described herein can assist UAVs in patrolling an environment. Visual markers can orient UAVs within an environment. UAVs may be able to determine the UAV position relative to various parts of an environment. For example, UAVs may be able to determine which floor the UAV is on. UAVs may also be able to tell which room or corridor they are in. UAVs may be able to determine their orientation (e.g., relative to cardinal directions or other types of directions). UAVs may be able to determine their position relative to other rooms or corridors of an environment. UAVs may be able to determine their local or global coordinates. UAVs may be able to determine their coordinates to an accuracy level within approximately 50m, 30m, 20m, 15m, 10m, 7m, 5m, 3m, 2m, 1.5m, 1m, 50cm, 30cm, 10cm, 5cm, or 1cm.

[0152] The visual marker may optionally provide the UAV with instructions for navigating within the environment. For example, the visual marker may be distributed within the environment. Multiple visual markers may be distributed throughout the environment. The visual marker may be positioned so that it is within each other's sight, or it may not be positioned so. During patrol, the UAV may encounter a first visual marker. The UAV may discern and respond to instructions displayed using the visual marker. As described elsewhere herein, the instructions may be encoded into the visual pattern of the visual marker. The first visual marker may guide the UAV to fly in a manner that will bring the UAV to the second visual marker. When the UAV reaches the second visual marker, the UAV may discern and respond to the instructions in the second visual marker. The second visual marker may guide the UAV to fly in a manner that will bring the UAV to the third visual marker, and so on. This may provide point-to-point navigation for the UAV. In some cases, point-to-point navigation may guide the UAV to circulate within the environment to patrol the environment regularly. In some cases, point-to-point navigation may allow the UAV to determine its position relative to the visual marker. The UAV may determine its relative position to a marker without having to determine its global position (eg, global coordinates). The UAV may be able to patrol an area without calculating or determining its global position.

[0153] In some cases, the visual marker can be changed, or the flight instructions associated with the visual marker can be changed. For example, the visual marker can be displayed on a screen that can dynamically change the visual marker. In another case, the visual marker can be static, but a new flight instruction associated with the visual marker can be associated with the visual marker. The new flight instruction can be downloaded to the UAV. In some cases, the UAV can have multiple sets of flight instructions associated with a specific visual marker, and the associated flight instructions can be selected based on other parameters. For example, the UAV can randomly, based on the time indicated by the UAV's clock (for example, if the time is between 3 pm and 6 pm, the UAV will follow the first set of instructions, and between 6 pm and midnight, the UAV will follow the second set of instructions), based on sensed environmental conditions (for example, based on temperature, light, detected motion, detected sound) or any other factor, from multiple flight instructions associated with the visual marker, select a set of flight instructions. Different groups of flight paths can guide the UAV to the same visual marker or different visual markers. For example, in a first scenario, the UAV can be guided from a first visual marker to a second visual marker via a first flight path. In a second scenario, the UAV may be directed from the first visual marker to the second visual marker via a second flight path. Alternatively, in the second scenario, the UAV may be directed from the first visual marker to a third visual marker. This may allow the UAV to follow different flight paths according to different flight instructions. This may provide some variation or unpredictability to the UAV patrol path, which may provide improved patrol effectiveness in some cases. This may also allow the UAV patrol path to change in a manner that may adapt to different detected scenarios.

[0154] The use of visual markers for point-to-point navigation may allow the UAV to perform surveillance or patrolling in a manner that does not require input from a remote user. The UAV may be able to fly and navigate in an environment using visual markers without receiving any signals from external devices (e.g., remote controls, satellites, towers, routers). Sensors onboard the UAV (such as visual sensors) may detect sufficient information to allow the UAV to navigate.

[0155] As discussed above, various types of instructions can be visually encoded into visual markers. Flight instructions can be encoded into visual markers or associated with visual markers. Other instructions can be included, such as instructions for the operation or movement of sensors, instructions for charging the UAV, or instructions for data to be transmitted. Examples of instructions about sensors can include instructions for operating a carrier that can move the sensor. Visual markers can enable the UAV to redirect sensors (such as cameras) about one, two, three or more axes. Visual markers can directly control sensors (e.g., turn on sensors, turn off sensors, zoom, change sensor modes). This can assist UAV security patrols and allow UAVs to perform data collection using one or more sensors without instructions from external sources or devices. Sensor instructions or other instructions can be provided individually or in combination with flight instructions from a single visual marker.

[0156] With the aid of the visual markers, the UAV may be able to patrol the environment autonomously or semi-autonomously. The UAV may be able to patrol the environment autonomously without human intervention. Visual markers may assist the UAV in navigating the environment without input from a human operator. The UAV may determine how to fly without the need for humans to manually guide the flight of the UAV. Visual markers may allow the UAV to orient itself without using GPS or other position sensors that require external communication. The UAV may be able to orient itself within the environment without any communication with an external device. The UAV may be able to fly within the environment without communication with an external device. The UAV may be able to monitor the environment without communication with an external device.

[0157] In some embodiments, a human operator may communicate with the UAV via a remote control, or may not communicate. In some embodiments, the UAV may operate autonomously without receiving any input from a human via a remote control. In some cases, a human may be able to intervene using a remote control. For example, if a human witnesses an event through a live feed captured by the UAV, the human may be able to intervene using a remote control. In the absence of human intervention, the UAV may be able to continue its patrol without input from a human operator.

[0158] The systems, devices and methods described herein can be applicable to numerous movable objects. As mentioned above, any description of aircraft (such as UAV) herein can be applicable to and used for any movable object. Any description of aircraft herein can be specifically applicable to UAV. The movable object of the present invention can be configured to move in any suitable environment, such as in the air (e.g., an attached wing aircraft, a rotary wing aircraft, or an aircraft having neither an attached wing nor a rotary wing), in water (e.g., a ship or a submarine), on the ground (e.g., a motor vehicle, such as a car, a truck, a bus, a van, a motorcycle, a bicycle; a movable structure or frame, such as a rod, a fishing rod; or a train), underground (e.g., a subway), in space (e.g., a space plane, a satellite or a probe) or any combination of these environments. The movable object can be a vehicle, such as a vehicle described elsewhere herein. In some embodiments, the movable object can be carried by a living body such as a human or an animal, or take off from a living body. Suitable animals can include birds, dogs, cats, horses, cattle, sheep, pigs, dolphins, rodents or insects.

[0159] The movable object may be able to move freely about six degrees of freedom (e.g., three translational degrees of freedom and three rotational degrees of freedom) within the environment. Alternatively, the movement of the movable object may be constrained about one or more degrees of freedom, such as constrained by a predetermined path, trajectory or orientation. The movement may be actuated by any suitable actuating mechanism such as an engine or motor. The actuating mechanism of the movable object may be powered by any suitable energy source, such as electrical energy, magnetic energy, solar energy, wind energy, gravitational energy, chemical energy, nuclear energy or any suitable combination thereof. The movable object may be self-propelled via a power system as described elsewhere herein. The power system may optionally rely on energy to operate, such as electrical energy, magnetic energy, solar energy, wind energy, gravitational energy, chemical energy, nuclear energy or any suitable combination thereof. Alternatively, the movable object may be carried by a living being.

[0160] In some cases, the movable object can be an aircraft. For example, the aircraft can be an attached wing aircraft (e.g., airplane, glider), a rotary wing aircraft (e.g., helicopter, gyroplane), an aircraft with attached wings and rotary wings, or an aircraft without attached wings and rotary wings (e.g., airship, hot air balloon). The aircraft can be self-propelled, such as self-propelled in the air. Self-propelled aircraft can utilize a power system, such as a power system including one or more engines, motors, wheels, axles, magnets, rotors, propellers, blades, nozzles, or any suitable combination thereof. In some cases, a power system can be used to enable a movable object to take off from a surface, land on a surface, maintain its current position and / or orientation (e.g., hover), change orientation, and / or change position.

[0161] The movable object may be remotely controlled by a user or locally controlled by an occupant in or on the movable object. The movable object may be remotely controlled via an occupant in a separate vehicle. In some embodiments, the movable object is an unmanned movable object, such as a UAV. An unmanned movable object, such as a UAV, may not have an occupant on the movable object. The movable object may be controlled by a human or autonomous control system (e.g., a computer control system) or any suitable combination thereof. The movable object may be an autonomous or semi-autonomous robot, such as a robot configured with artificial intelligence.

[0162] The movable object may have any suitable size and / or dimensions. In some embodiments, the movable object may have a size and / or dimensions that allow a human occupant to be within or on a vehicle. Alternatively, the movable object may have a size and / or dimensions that are smaller than the size and / or dimensions that allow a human occupant to be within or on a vehicle. The movable object may have a size and / or dimensions that are suitable for being carried or carried by a human. Alternatively, the movable object may be larger than a size and / or dimensions that are suitable for being carried or carried by a human. In some cases, the movable object may have a maximum dimension (e.g., length, width, height, diameter, diagonal) that is less than or equal to approximately: 2cm, 5cm, 10cm, 50cm, 1m, 2m, 5m, or 10m. The maximum dimension may be greater than or equal to approximately: 2cm, 5cm, 10cm, 50cm, 1m, 2m, 5m, or 10m. For example, the distance between the axes of the relative rotors of the movable object may be less than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. Alternatively, the distance between the axes of the relative rotors may be greater than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m.

[0163] In some embodiments, the movable object may have a volume of less than 100 cm x 100 cm x 100 cm, less than 50 cm x 50 cm x 30 cm, or less than 5 cm x 5 cm x 3 cm. The total volume of the movable object may be less than or equal to about: 1 cm 3 , 2cm 3 , 5cm 3 , 10cm 3 , 20cm 3 、30cm 3 、40cm 3 , 50cm 3 、60cm 3 、70cm 3 、80cm 3 、90cm 3 、100cm3 、150cm 3 , 200cm 3 、300cm 3 , 500cm 3 、750cm 3 , 1000cm 3 、5000cm 3 、10,000cm 3 、100,000cm 3 , 1m 3 or 10m 3 Conversely, the total volume of the movable object may be greater than or equal to approximately: 1 cm 3 , 2cm 3 , 5cm 3 , 10cm 3 , 20cm 3 、30cm 3 、40cm 3 , 50cm 3 、60cm 3 、70cm 3 、80cm 3 、90cm 3 、100cm 3 、150cm 3 , 200cm 3 、300cm 3 , 500cm 3 、750cm 3 , 1000cm 3 、5000cm 3 、10,000cm 3 、100,000cm 3 , 1m 3 or 10m 3 .

[0164] In some embodiments, a movable object may have a footprint (which may refer to the cross-sectional area enclosed by the movable object) less than or equal to about: 32,000 cm 2 、20,000cm 2 、10,000cm 2 、1,000cm 2 , 500cm 2 、100cm 2 , 50cm 2 , 10cm 2 or 5cm 2 Conversely, the footprint may be greater than or equal to approximately: 32,000 cm 2 、20,000cm2 、10,000cm 2 、1,000cm 2 , 500cm 2 、100cm 2 , 50cm 2 , 10cm 2 or 5cm 2 .

[0165] In some cases, the movable object may weigh no more than 1000 kg. The weight of the movable object may be less than or equal to about: 1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60 kg, 50 kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10 kg, 9 kg, 8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1 kg, 0.05 kg, or 0.01 kg. Conversely, the weight may be greater than or equal to approximately: 1000kg, 750kg, 500kg, 200kg, 150kg, 100kg, 80kg, 70kg, 60kg, 50kg, 45kg, 40kg, 35kg, 30kg, 25kg, 20kg, 15kg, 12kg, 10kg, 9kg, 8kg, 7kg, 6kg, 5kg, 4kg, 3kg, 2kg, 1kg, 0.5kg, 0.1kg, 0.05kg or 0.01kg.

[0166] In some embodiments, the movable object may be smaller than the load carried by the movable object. As further described elsewhere herein, the load may include a load and / or a carrier. In some examples, the ratio of the movable object weight to the load weight may be greater than, less than, or equal to about 1:1. In some cases, the ratio of the movable object weight to the load weight may be greater than, less than, or equal to about 1:1. Alternatively, the ratio of the carrier weight to the load weight may be greater than, less than, or equal to about 1:1. When desired, the ratio of the movable object weight to the load weight may be less than or equal to: 1:2, 1:3, 1:4, 1:5, 1:10 or even less. Conversely, the ratio of the movable object weight to the load weight may also be greater than or equal to: 2:1, 3:1, 4:1, 5:1, 10:1 or even greater.

[0167] In some embodiments, the movable object can have low energy consumption. For example, the movable object can use less than about: 5W / h, 4W / h, 3W / h, 2W / h, 1W / h or less. In some cases, the carrier of the movable object can have low energy consumption. For example, the carrier can use less than about: 5W / h, 4W / h, 3W / h, 2W / h, 1W / h or less. Optionally, the load of the movable object can have low energy consumption, such as less than about: 5W / h, 4W / h, 3W / h, 2W / h, 1W / h or less.

[0168] Fig.14 An unmanned aerial vehicle (UAV) 1400 according to an embodiment of the present invention is illustrated. The UAV may be an example of a movable object as described herein. The UAV 1400 may include a power system having four rotors 1402, 1404, 1406, and 1408. Any number of rotors (e.g., one, two, three, four, five, six, or more) may be provided. The rotors, rotor assemblies, or other power systems of the unmanned aerial vehicle may enable the unmanned aerial vehicle to hover / maintain position, change orientation, and / or change position. The distance between the axes of the relative rotors may be any suitable length 1410. For example, the length 1410 may be less than or equal to 2m, or less than or equal to 5m. In some embodiments, the length 1410 may be in the range of from 40cm to 1m, from 10cm to 2m, or from 5cm to 5m. Any description of the UAV herein may be applicable to movable objects, such as movable objects of different types, and vice versa. The UAV may use an assisted takeoff system or method as described herein.

[0169] In some embodiments, the movable object may be configured to carry a load. The load may include one or more of passengers, cargo, equipment, instruments, etc. The load may be provided in a housing. The housing may be separate from the housing of the movable object, or may be part of the housing of the movable object. Alternatively, the load may be provided with a housing, while the movable object does not have a housing. Alternatively, part of the load or the entire load may be provided without a housing. The load may be rigidly attached relative to the movable object. Optionally, the load may be movable relative to the movable object (e.g., translatable or rotatable relative to the movable object). As described elsewhere herein, the load may include a load and / or a carrier.

[0170] In some embodiments, the movement of the movable object, carrier and load relative to the attachment reference system (e.g., the surrounding environment) and / or relative to each other can be controlled by a terminal. The terminal can be a remote control device at a position away from the movable object, carrier and / or load. The terminal can be placed on a support platform or attached to a support platform. Alternatively, the terminal can be a handheld or wearable device. For example, the terminal can include a smart phone, a tablet computer, a laptop computer, a computer, glasses, gloves, a helmet, a microphone, or a suitable combination thereof. The terminal can include a user interface, such as a keyboard, a mouse, a joystick, a touch screen, or a display. Any suitable user input can be used to interact with the terminal, such as manually input commands, voice control, gesture control, or position control (e.g., via movement, position, or tilt of the terminal).

[0171] The terminal can be used to control any suitable state of the movable object, carrier and / or load. For example, the terminal can be used to control the position and / or orientation of the movable object, carrier and / or load relative to the attached reference and / or relative to each other. In some embodiments, the terminal can be used to control individual elements of the movable object, carrier and / or load, such as an actuating component of the carrier, a sensor of the load, or a transmitter of the load. The terminal may include a wireless communication device suitable for communicating with one or more of the movable object, carrier or load.

[0172] The terminal may include a suitable display unit for viewing information of the movable object, carrier and / or load. For example, the terminal may be configured to display information of the movable object, carrier and / or load regarding position, translational velocity, translational acceleration, orientation, angular velocity, angular acceleration, or any suitable combination thereof. In some embodiments, the terminal may display information provided by the load, such as data provided by the functional load (e.g., images recorded by a camera or other image capture device).

[0173] Optionally, the same terminal can simultaneously control a movable object, a carrier and / or a load or the state of the movable object, carrier and / or load, and receive and / or display information from the movable object, carrier and / or load. For example, a terminal can control the positioning of a load relative to an environment, while displaying image data captured by the load, or information about the position of the load. Alternatively, different terminals can be used for different functions. For example, a first terminal can control the movement or state of a movable object, a carrier and / or a load, while a second terminal can receive and / or display information from a movable object, a carrier and / or a load. For example, a first terminal can be used to control the positioning of a load relative to an environment, while a second terminal displays image data captured by the load. Various communication modes can be utilized between a movable object and an integrated terminal that simultaneously controls the movable object and receives data, or between a movable object and multiple terminals that simultaneously control the movable object and receive data. For example, at least two different communication modes can be formed between a movable object and a terminal that simultaneously controls the movable object and receives data from the movable object.

[0174] Fig.15 A movable object 1500 including a carrier 1502 and a payload 1504 is illustrated according to an embodiment. Although the movable object 1500 is depicted as an aircraft, this depiction is not intended to be limiting, and any suitable type of movable object may be used as described herein before. It will be appreciated by those skilled in the art that any of the embodiments described herein in the context of an aircraft system may be applicable to any suitable movable object (e.g., a UAV). In some cases, a payload 1504 may be provided on the movable object 1500 without the carrier 1502. The movable object 1500 may include a power mechanism 1506, a sensing system 1508, and a communication system 1510.

[0175] As described above, power mechanism 1506 may include one or more of a rotor, a propeller, a blade, an engine, a motor, a wheel, an axle, a magnet, or a nozzle. The movable object may have one or more, two or more, three or more, or four or more power mechanisms. The power mechanisms may all be of the same type. Alternatively, one or more power mechanisms may be different types of power mechanisms. Power mechanism 1506 may be mounted on movable object 1500 using any suitable means, such as a support element (e.g., a drive shaft) as described elsewhere herein. Power mechanism 1506 may be mounted on any suitable portion of movable object 1500, such as the top, bottom, front, back, side, or a suitable combination thereof.

[0176] In some embodiments, the power mechanism 1506 can enable the movable object 1500 to take off vertically from a surface or land vertically on a surface without any horizontal movement of the movable object 1500 (e.g., without traveling along a runway). Optionally, the power mechanism 1506 can be operable to allow the movable object 1500 to hover in the air at a specified position and / or orientation. One or more of the power mechanisms 1500 can be controlled independently of other power mechanisms. Alternatively, the power mechanism 1500 can be configured to be controlled simultaneously. For example, the movable object 1500 can have a plurality of horizontally oriented rotors, which can provide lift and / or thrust to the movable object. The plurality of horizontally oriented rotors can be actuated to provide vertical takeoff, vertical landing, and hovering capabilities to the movable object 1400. In some embodiments, one or more of the horizontally oriented rotors can rotate in a clockwise direction, while one or more of the horizontal rotors can rotate in a counterclockwise direction. For example, the number of clockwise rotors can be equal to the number of counterclockwise rotors. The rotation rate of each of the horizontally-oriented rotors can be independently varied to control the lift and / or thrust generated by each rotor and thereby adjust the spatial placement, velocity, and / or acceleration of movable object 1500 (e.g., with respect to up to three translational degrees of freedom and up to three rotational degrees of freedom).

[0177] The sensing system 1508 may include one or more sensors that can sense the spatial placement, speed, and / or acceleration of the movable object 1500 (e.g., with respect to up to three translational degrees of freedom and up to three rotational degrees of freedom). The one or more sensors may include a global positioning system (GPS) sensor, a motion sensor, an inertial sensor, a proximity sensor, or an image sensor. The sensed data provided by the sensing system 1508 can be used to control the spatial placement, speed, and / or orientation of the movable object 1500 (e.g., using a suitable processing unit and / or control module, as described below). Alternatively, the sensing system 1508 can be used to provide data about the environment surrounding the movable object, such as weather conditions, proximity to potential obstacles, location of geographic features, location of man-made structures, etc.

[0178] The communication system 1510 supports communication with a terminal 1512 having a communication system 1514 via a wireless signal 1516. The communication system 1510, the communication system 1514 may include any number of transmitters, receivers and / or transceivers suitable for wireless communication. The communication may be a one-way communication, so that data can only be transmitted in one direction. For example, a one-way communication may only involve the movable object 1500 transmitting data to the terminal 1512, or vice versa. Data may be transmitted from one or more transmitters of the communication system 1510 to one or more receivers of the communication system 1512, or vice versa. Alternatively, the communication may be a two-way communication, so that data can be transmitted in both directions between the movable object 1500 and the terminal 1512. The two-way communication may involve transmitting data from one or more transmitters of the communication system 1510 to one or more receivers of the communication system 1514, and vice versa.

[0179] In some embodiments, the terminal 1512 can provide control data to one or more of the movable object 1500, the carrier 1502, and the load 1504, and receive information from one or more of the movable object 1500, the carrier 1502, and the load 1504 (e.g., position and / or motion information of the movable object, the carrier, or the load; data sensed by the load, such as image data captured by the load camera). In some cases, the control data from the terminal may include instructions for the relative position, movement, actuation, or control of the movable object, the carrier, and / or the load. For example, the control data may result in modification of the position and / or orientation of the movable object (e.g., via control of the power mechanism 1506), or movement of the load relative to the movable object (e.g., via control of the carrier 1502). The control data from the terminal may result in control of the load, such as control of the operation of a camera or other image capture device (e.g., taking still or moving pictures, zooming in or out, turning on or off, switching imaging modes, changing image resolution, changing focus, changing depth of field, changing exposure time, changing viewing angle or field of view). In some cases, the communication from the movable object, carrier and / or load may include information from one or more sensors (e.g., of the sensing system 1508 or of the load 1504). The communication may include sensed information from one or more different types of sensors (e.g., GPS sensors, motion sensors, inertial sensors, proximity sensors, or image sensors). Such information may be about the positioning (e.g., position, orientation), movement, or acceleration of the movable object, carrier, and / or load. Such information from the load may include data captured by the load or the sensed state of the load. The control data provided and transmitted by the terminal 1512 may be configured to control the state of one or more of the movable object 1500, the carrier 1502, or the load 1504. Alternatively or in combination, the carrier 1502 and the load 1504 may also each include a communication module configured to communicate with the terminal 1512 so that the terminal can independently communicate with and control each of the movable object 1500, the carrier 1502, and the load 1504.

[0180] In some embodiments, the movable object 1500 may be configured to communicate with another remote device in addition to or in place of the terminal 1512. The terminal 1512 may also be configured to communicate with another remote device as well as the movable object 1500. For example, the movable object 1500 and / or the terminal 1512 may communicate with another movable object or a carrier or load of another movable object. When necessary, the remote device may be a second terminal or other computing device (e.g., a computer, laptop computer, tablet computer, smart phone or other mobile device). The remote device may be configured to transmit data to the movable object 1500, receive data from the movable object 1500, transmit data to the terminal 1512 and / or receive data from the terminal 1512. Optionally, the remote device may be connected to the Internet or other telecommunications network so that data received from the movable object 1500 and / or the terminal 1512 may be uploaded to a website or server.

[0181] Fig.16 1 is a schematic diagram illustrating a system 1600 for controlling a movable object according to an embodiment by way of a block diagram. The system 1600 can be used in combination with any suitable embodiment of the systems, devices, and methods disclosed herein. The system 1600 may include a sensing module 1602, a processing unit 1604, a non-transitory computer-readable medium 1606, a control module 1608, and a communication module 1610.

[0182] The sensing module 1602 can utilize different types of sensors that collect information related to the movable object in different ways. Different types of sensors can sense different types of signals or signals from different sources. For example, the sensor may include an inertial sensor, a GPS sensor, a proximity sensor (e.g., a lidar), or a visual / image sensor (e.g., a camera). The sensing module 1602 can be operably coupled to a processing unit 1604 having multiple processors. In some embodiments, the sensing module can be operably coupled to a transmission module 1612 (e.g., a Wi-Fi image transmission module), which is configured to directly transmit sensing data to a suitable external device or system. For example, the transmission module 1612 can be used to transmit an image captured by a camera of the sensing module 1602 to a remote terminal.

[0183] The processing unit 1604 may have one or more processors, such as a programmable processor (e.g., a central processing unit (CPU)). The processing unit 1604 may be operably coupled to a non-transitory computer-readable medium 1606. The non-transitory computer-readable medium 1606 may store logic, code, and / or program instructions that can be executed by the processing unit 1604 to perform one or more steps. The non-transitory computer-readable medium may include one or more memory units (e.g., removable media or external storage devices, such as an SD card or a random access memory (RAM)). In some embodiments, data from the sensing module 1602 may be directly transmitted to and stored in the memory unit of the non-transitory computer-readable medium 1606. The memory unit of the non-transitory computer-readable medium 1606 may store logic, code, and / or program instructions that can be executed by the processing unit 1604 to perform any suitable embodiment of the method described herein. For example, the processing unit 1604 may be configured to execute instructions so that one or more processors of the processing unit 1604 analyze the sensing data generated by the sensing module. The memory unit may store sensing data from the sensing module to be processed by the processing unit 1604. In some embodiments, the memory unit of the non-transitory computer-readable medium 1606 may be used to store processing results generated by the processing unit 1604.

[0184] In some embodiments, the processing unit 1604 can be operably coupled to a control module 1608, which is configured to control the state of the movable object. For example, the control module 1608 can be configured to control the power mechanism of the movable object to adjust the spatial layout, speed and / or acceleration of the movable object with respect to six degrees of freedom. Alternatively or in combination, the control module 1608 can control the state of one or more of the carrier, the load, or the sensing module.

[0185] The processing unit 1604 may be operably coupled to a communication module 1510, which is configured to transmit and / or receive data from one or more external devices (e.g., a terminal, a display device, or other remote controller). Any suitable communication means may be used, such as wired communication or wireless communication. For example, the communication module 1610 may utilize one or more of a local area network (LAN), a wide area network (WAN), infrared, radio, WiFi, a point-to-point (P2P) network, a telecommunications network, a cloud communication, etc. Alternatively, a relay station such as a tower, a satellite, or a mobile station may be used. Wireless communication may rely on proximity or be independent of proximity. In some embodiments, communication may or may not require line of sight. The communication module 1610 may transmit and / or receive one or more of the sensing data from the sensing module 1602, the processing results generated by the processing unit 1604, predetermined control data, user commands from a terminal or remote controller, etc.

[0186] The components of system 1600 may be arranged in any suitable configuration. For example, one or more of the components of system 1600 may be located on a movable object, a carrier, a load, a terminal, a sensing system, or an additional external device that communicates with one or more of the above. Fig.16 A single processing unit 1604 and a single non-transitory computer-readable medium 1606 are depicted, but those skilled in the art will appreciate that this is not intended to be limiting, and that the system 1600 may include multiple processing units and / or non-transitory computer-readable media. In some embodiments, one or more of the multiple processing units and / or non-transitory computer-readable media may be located at different locations, such as on a movable object, a carrier, a payload, a terminal, a sensing module, an additional external device in communication with one or more of the foregoing, or a suitable combination thereof, such that any suitable aspects of the processing and / or memory functions performed by the system 1600 may occur at one or more of the foregoing locations.

[0187] Although preferred embodiments of the present invention have been shown and described herein, it is obvious to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will now appreciate many variations, changes and substitutions without departing from the present invention. It should be understood that various alternatives to the embodiments of the present invention described herein may be adopted in the process of practicing the present invention. The appended claims are intended to define the scope of the present invention, and therefore cover methods and structures within the scope of these claims and their equivalents.

Claims

1. A method for locating a movable object in an environment, the method comprising: capturing, by means of one or more visual sensors of the movable object, an image including a visual marker having a unique visual pattern within the environment while the movable object is moving; discerning and distinguishing the visual marker from a plurality of different markers based on the unique visual pattern of the visual marker in the image, wherein the discerning visual marker has a known position that is different from positions of the plurality of different markers; as well as With the aid of one or more processors, the position of the movable object in a three-dimensional coordinate system is determined based on the known position of the identified visual marker, the distance between the movable object and the identified visual marker, and the angle of the movable object relative to the identified visual marker. The method according to claim 1 , wherein the environment is an indoor environment. The method of claim 1 , wherein the environment is one in which global positioning system signals are unreliable.

4. The method of claim 1, wherein the movable object weighs less than 1000 g and has a maximum dimension not exceeding 100 cm.

5. The method of claim 1, wherein vertices of shapes in the visual pattern are detected and used to calculate a position relative to the one or more visual markers.

6. The method of claim 1, wherein the visual indicia is a band having a unique visual pattern, the unique visual pattern comprising a square.

7. The method of claim 1, wherein the unique visual pattern comprises squares of different colors.

8. The method of claim 1, wherein the unique visual pattern comprises an AprilTag.

9. The method of claim 1, wherein the unique visual pattern comprises a QR code.

10. The method of claim 1, wherein the unique visual pattern comprises a bar code.

11. The method of claim 1 , further comprising calculating a size of at least a portion of the unique visual pattern of the visual marker in the image; and With the one or more processors, a distance of the movable object from the discerned visual marker and an angle of the movable object relative to the discerned visual marker are determined based on the size of the portion of the unique visual pattern.

12. The method of claim 1, wherein the three-dimensional coordinate system is a three-dimensional global coordinate system.

13. The method of claim 1, wherein the three-dimensional coordinate system is a three-dimensional local coordinate system.

14. The method of claim 1, further comprising recognizing the visual pattern and implementing a response by the movable object based on the recognized visual pattern.

15. The method according to claim 1, further comprising: The movable object is positioned within the environment by calculating the rotation and position of the movable object relative to the visual marker using one or more intrinsic characteristics of the one or more visual sensors onboard the movable object, the known position of the visual marker, and a set of points in an image of the visual marker taken by the visual sensor onboard the movable object.

16. The method of claim 15, wherein at least one of the one or more visual sensors is a camera.

17. The method of claim 15, wherein the movable object autonomously patrols the environment while the one or more visual sensors capture the images.

18. The method of claim 1, wherein the one or more processors are onboard the movable object.

19. The method of claim 1, further comprising: detecting obstacles with the aid of one or more sensors onboard the movable object; and adjusting the moving direction of the movable object to avoid the obstacle.

20. The method of claim 1, wherein the positioning is also determined by means of an infrared detector or a radar detector.

21. A system for locating a movable object within an environment, the system comprising: one or more visual sensors of the movable object configured to capture images including a visual marker having a unique visual pattern within the environment while the movable object is moving; as well as One or more processors that are individually or collectively configured to: (1) discerning and distinguishing the visual marker from a plurality of different markers based on the unique visual pattern of the visual marker in the image, wherein the discerned visual marker has a known position that is different from the positions of the plurality of different markers; as well as (2) Determining a position of the movable object in a three-dimensional coordinate system based on the known position of the identified visual marker, a distance between the movable object and the identified visual marker, and an angle of the movable object relative to the identified visual marker.

22. The system of claim 21, wherein the environment is an indoor environment; or, wherein the environment is one in which global positioning system signals are unreliable; or, wherein the movable object weighs less than 1000 g and has a maximum dimension not exceeding 1000 cm; or, wherein vertices of shapes in the visual pattern are detected and used to calculate positioning relative to the one or more visual markers; or, wherein the visual indicia is a band having a unique visual pattern comprising a square; or, wherein the unique visual pattern comprises squares of different colors; Alternatively, wherein the unique visual pattern comprises an AprilTag; Alternatively, wherein the unique visual pattern comprises a QR code; Alternatively, wherein the unique visual pattern comprises a barcode; Alternatively, wherein the one or more processors are further configured to calculate a size of at least a portion of the unique visual pattern of the visual marker in the image; and determining a distance of the movable object from the discerned visual marker and an angle of the movable object relative to the discerned visual marker based on the size of the portion of the unique visual pattern; Alternatively, the three-dimensional coordinate system is a three-dimensional global coordinate system; Alternatively, the three-dimensional coordinate system is a three-dimensional local coordinate system; or, wherein the visual pattern is discerned and a response is effected by the movable object based on the discerned visual pattern; Alternatively, wherein the one or more processors are configured to calculate the rotation and position of the movable object relative to the visual marker using one or more intrinsic characteristics of the one or more visual sensors onboard the movable object, the known position of the visual marker, and a set of points in an image of the visual marker taken by the visual sensor onboard the movable object; or, wherein the one or more processors are onboard the movable object; Alternatively, the movable object is further configured to detect obstacles with the aid of one or more sensors onboard the movable object; and adjust the moving direction of the movable object to avoid the obstacles; Alternatively, the positioning is also determined by means of an infrared detector or a radar detector.