Contour scanning using unmanned aerial vehicles

Abstract

Techniques and arrangements for configuring a UAV to scan a three dimensional (3D) scanning target are directed.SOLUTION: A plurality of contour paths spaced apart from each other in at least one axis associated with the object to be scanned are determined, and the UAV determines a plurality of image-acquisition locations for each contour path. The image-acquisition position is a position at which an image of the surface of the object is to be acquired, and navigating along the plurality of contour paths based on the determined velocity while acquiring the image of the surface of the object based on the image-acquisition position.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This disclosure relates to the technical field of unmanned aerial vehicles.

Background Art

[0002] An unmanned aerial vehicle (UAV), sometimes referred to as a "drone", typically includes one or more cameras for acquiring images of an object during flight. For example, such a UAV can be used to acquire images from a vantage point that would otherwise be difficult to reach. Conventionally, UAVs have been controlled by a ground pilot to acquire images of a desired target. However, acquiring an image of the surface of a target object at a desired distance from the surface, as well as at a desired resolution and accuracy of coverage, can be a tedious task for a human pilot.

Summary of the Invention

[0003] Some implementations include a UAV that can determine a plurality of contour paths spaced apart from each other along at least one axis associated with a scan target. For example, each contour path may be spaced apart from the surface of the scan target based on a selected distance. The UAV may determine a plurality of image acquisition positions for each contour path. The image acquisition positions may indicate positions where an image of the surface of the scan target is to be acquired. The UAV may navigate along the plurality of contour paths based on a determined speed while acquiring an image of the surface of the scan target based on the image acquisition positions.

Brief Description of the Drawings

[0004] The detailed description is shown with reference to the accompanying drawings. In the figures, the leftmost digit of a reference number identifies the figure in which the reference number first appears. The use of the same reference number in different figures indicates similar or identical items or features. [Figure 1] FIG. shows an exemplary system including an unmanned aerial vehicle (UAV) capable of acquiring images according to some implementations. [Figure 2] This figure shows examples of determining the contour path and waypoints of a slice to be scanned using several implementation methods. [Figure 3A] This figure shows examples of determining the contour path of the scanned object using several implementation methods. [Figure 3B] This figure shows examples of determining the contour path of the scanned object using several implementation methods. [Figure 4A] This figure shows examples of determining the contour path of the scanned object using several implementation methods. [Figure 4B] This figure shows examples of determining the contour path of the scanned object using several implementation methods. [Figure 5] This flowchart illustrates an example process for scanning targets using several implementation configurations. [Figure 6] This flowchart illustrates an example process for generating a 3D model of a scan target using several implementation methods. [Figure 7] This figure shows an example configuration of the upper part of a UAV using several implementation methods. [Figure 8] This figure shows examples of the bottom side configuration of a UAV in several implementation forms. [Figure 9] This figure shows excerpts of components from an example UAV in several implementation forms. [Figure 10] This figure shows an example configuration of a controller that includes computing devices in several implementation forms. [Figure 11] This figure shows excerpts of controller components as examples of several implementation forms. [Modes for carrying out the invention]

[0005] Some implementations of this specification concern techniques and arrangements for configuring a UAV to scan a three-dimensional (3D) scan target, such as a structure, building, bridge, pipeline, equipment, site, landscape, geographical feature, or any other object, which may be designated as a scan target by an operator. For example, a UAV may be configured to autonomously scan a scan target based on an initial indication of the scan target. The scan may include acquiring a series of images of the surface of the scan target indicated at a desired distance from the surface. The UAV may autonomously acquire images along a series of slices of the scan target in a complete and repeatable manner. The UAV can acquire scan targets having complex geometric shapes, such as scan targets containing recesses, uneven and sloped surfaces and features, openings, asymmetrical geometric shapes, etc. In some cases, the acquired images may be used to generate a 3D model of the scan target or for any of a variety of other end uses.

[0006] As one example, the UAV may be configured to traverse a path corresponding to multiple virtual contours around the surface being scanned at a selected distance (sometimes referred to herein as the baseline distance) from the surface being scanned. The UAV may determine the virtual contours by dividing the surface being scanned into a series of slices acquired across the axes of the surface being scanned. As one example, the UAV may associate a coordinate system (such as an x, y, z coordinate system) with the surface being scanned and divide the surface being scanned into multiple slices across at least one axis of the coordinate system. The UAV may determine a contour for each slice based on a selected distance and detected location of the surface of the surface being scanned (also known as the zero contour). Furthermore, the UAV may determine waypoints and gaze points for each contour. Each waypoint may be an image acquisition location where the UAV acquires an image of the surface of the surface being scanned. The UAV may acquire images of the object being scanned in an orderly and continuous manner by moving along multiple contours determined for the object being scanned, such as by acquiring an image along the first contour of the first slice, then moving to the next adjacent slice and acquiring an image along the contour determined for that slice, and continuing to move to the next adjacent slice until all desired portions of the object being scanned have been acquired.

[0007] Each slice may be equal to a two-dimensional (2D) contour in a plane at a spatial position of that particular slice. The UAV may traverse its contour as a path, pointing its camera at the target to acquire an image of the target surface corresponding to the slice's position. For example, the contour may essentially follow the shape of the target surface at the spatial position of that slice, forming a path for the UAV to traverse at a selected distance from the target surface while acquiring an image of the target surface. The UAV may select a pattern for traversing the contour that allows the UAV to move optimally at a constant speed while acquiring images at regular intervals.

[0008] The distance between slices may be determined based at least on the field of view of the camera mounted on the UAV used to acquire the image, the selected distance of the travel path from the surface to be scanned, and the desired level of side overlap (sidelap) between adjacent slices of the acquired image. The camera distance from the surface to be scanned may be selected at least on the capabilities of the camera on the UAV and the desired level of detail of the acquired image. For example, the desired level of detail, such as resolution, sharpness, and exposure time, may depend at least in part on the intended use of the acquired image.

[0009] As will be further explained below, depending in part to the configuration of the scan target, the UAV may be configured to traverse the virtual contour determined around the scan target using one of a variety of patterns. As one example, the UAV may move from the contour in one slice to the contour in the next slice using a lawnmower pattern, or move in the opposite direction in each alternating slice using a reciprocating pattern. As another example, the UAV may perform a trajectory of the scan target, moving up / down or left / right (depending on which axis the slice is acquired) to the next adjacent slice as it reaches the endpoint of the first slice and moves to the starting point of the next adjacent slice. For example, the UAV may traverse the slices as multiple trajectories (e.g., circular, elliptical, etc.) around the scan target by making an initial trajectory at the location of the first slice, moving to the location of the next slice, and making another trajectory of the scan target along the contour determined for that slice. As yet another example, the UAV may traverse the slices as a series of adjacent vertical columns, for example, moving vertically along the length of the object being scanned in the first vertical slice of the object, then moving to an adjacent slice, and moving along that slice as another vertical column.

[0010] Furthermore, in some examples, depending on the configuration of the object being scanned, additional slices may be determined along a second axis (e.g., perpendicular to the first axis) to obtain additional images of the object being scanned. For example, suppose a first series of slices of the object are obtained by slicing the object across the z-axis to obtain images of the sides of the object being scanned. The object being scanned may be further sliced ​​across the x-axis or y-axis to obtain images of the top and / or bottom of the object being scanned, for example. Furthermore, additional slices may be obtained across a third axis or any combination of axes as needed. In addition, while the examples described herein describe several optimal slicing techniques and slicing patterns, numerous variations in slicing a particular object and selecting possible patterns for traversing the contours determined for the slices will be apparent to those skilled in the art who are interested in the disclosure herein.

[0011] In some implementations of this specification, the UAV may determine a structured, continuous path that can be traversed at a constant speed, etc., while acquiring images at regular intervals and at regularly spaced intervals. The speed may, in some cases, be determined based on an acceptable threshold amount of subject blur in the captured images, which may also depend in part on the intended use of the captured images and the level of detail required. For example, the further away the image is acquired from the surface being scanned, the faster the UAV can move while acquiring continuous images of the surface as it moves along the contour traverse path. On the other hand, if detailed close-up images of the surface are desired, the UAV may decelerate or stop to acquire images in order to eliminate subject blur.

[0012] For illustrative purposes, several exemplary implementations for configuring a UAV to autonomously scan a target and acquire images of that target are described. However, the implementations described herein are not limited to the specific examples provided and may be extended to other types of targets, other types of vehicles, other types of flight path planning techniques, other types of scanning, other types of image acquisition, etc., as will be apparent to those skilled in the art in light of the disclosure herein.

[0013] Figure 1 shows an exemplary system 100 that includes an unmanned aerial vehicle (UAV) 102 capable of acquiring images according to several implementation configurations. In this example, the UAV 102 can communicate with a controller 104. The controller 104 may optionally include a computing device. As one example, the controller 104 may include a mobile device such as a mobile phone, tablet computing device, wearable device, or laptop computer that can communicate wirelessly with the UAV 102, either directly or indirectly via another device such as a network.

[0014] One or more processors (not shown in Figure 1) mounted on the UAV102 may be configured with instructions that can be executed to receive images from at least one camera mounted on the UAV102. In the example shown, the UAV102 includes multiple cameras, such as a first camera 106 and a plurality of second cameras 108. For example, the first camera 106 may include a lens with a longer focal length and a higher resolution image sensor than the second cameras 108. The second cameras 108 may have a shorter focal length, a wider field of view (FOV), and a lower resolution image sensor than the first camera 106. The first camera 106 may be mounted on a gimbal 110 to enable it to be directed at a desired object, such as a scan target 111 in this example, or other scan targets. In some cases, the second cameras 108 may provide images that the UAV102 can use for various purposes, such as navigation, positioning, distance determination, stereo imaging, obstacle avoidance, and tracking.

[0015] The UAV 102 can communicate with the controller 104 via wireless communication or the like. The controller 104 may be controlled by the user 112 and may be configured for bidirectional communication with the UAV 102 by any of the various types of wireless communication technologies. In several examples, the controller 104 may communicate via various types of wireless protocols and frequencies, such as via a Wi-Fi network, a Bluetooth® wireless link, cellular radio, direct ISM band communication, or any other suitable wireless communication. For example, 900 MHz, 2.4 GHz, and 5.8 GHz are the most common wireless frequencies used for bidirectional communication with the UAV, but the implementations described herein are not limited to any particular type of communication, frequency, or protocol.

[0016] Additionally, or alternatively, the UAV 102 may communicate with one or more networks 113 via any of the aforementioned types of wireless communication, or any other type of wireless communication technology. As one example, the network 113 may include a wireless access point, a cellular wireless tower or other cellular transceiver, a short-range wireless transceiver, etc. to enable the UAV 102 to connect with one or more networks 113 and communicate via the one or more networks 113.

[0017] The UAV 102 includes a body 114 and one or more propulsion devices 116. In this example, a first set of the second cameras 108 is attached to the upper side of the body 114, and a second set is attached to the lower side of the body 114. Further, the first camera 106 may include a fixed-focus lens, or alternatively, an optically zoomable lens. The gimbal 110 enables the first camera 106 to be focused on a target without the need to rotate the UAV 102 to directly face the desired target. The UAV 102 may acquire an image by the first camera 106 and may transmit at least a part of the acquired image to the controller 104 as an image 118. Further, in some implementations, at least a part of the image acquired by the second camera 108 may be transmitted as part of the image 118. In some examples, the transmitted image 118 may be a lower-resolution image compared to the resolution of the image acquired by the UAV 102 to enable faster wireless transmission to the controller 104 or the like.

[0018] In some cases, the UAV 102 may use images acquired by the UAV 102 to generate 3D model information to be mounted on the UAV 102, and at least a portion of the 3D model information may be transmitted to the controller 104 as model information 120. The controller 104 may receive images 118 (which may include video or still images) from the UAV 102 via a wireless communication link. Based on the received images 118, the user 112 can use the display 124 associated with the controller 104 to view the field of view acquired by the UAV 102 and / or view a 3D model of the object to be scanned 111 which may be at least partially generated by the UAV 102 and provided as 3D model information 120.

[0019] In some examples, user 112 may use controller 104 to select a scan target 111 in order to instruct UAV 102 to perform a scan of the scan target 111. Furthermore, user 112 may use controller 104 to issue other conventional commands to UAV 102, such as "take off," "land," or "follow," via one or more virtual controls presented in a graphical user interface (GUI) included in controller 104, and / or one or more physical controls such as joysticks, buttons, etc. (not shown in Figure 1). Thus, controller 104 may allow user 112 to make manual control inputs in order to manually control UAV 102 and / or to instruct UAV 102 to operate autonomously. An exemplary controller 104 is shown and described further below with respect to Figures 10 and 11.

[0020] In some examples, the controller 104 and / or the UAV 102 may be able to communicate with one or more service computing devices 126, or other suitable computing devices such as other user computing devices, via one or more networks 113. The one or more networks 113 can include wide area networks (WANs) such as the Internet, local area networks (LANs) such as intranets, wireless networks such as cellular networks or other wireless communications, local wireless networks such as Wi-Fi, short-range wireless communications such as BLUETOOTH (registered trademark), wired networks including fiber optic and Ethernet, any combination thereof, or any other suitable communication network or other communication technology, including any suitable network or other communication technology.

[0021] In some cases, the service computing device 126 may be located remotely from the controller 104 and / or UAV 102, such as at a cloud computing location, data center, or server farm. The service computing device 126 may include a management program 128 that can be run to communicate with the controller 104 and / or UAV 102, such as to receive images 132 to be stored in the database 130. Images 132 may include the aforementioned images 118 and / or a complete set of images of the scan target 111 acquired by the UAV 102 during the scan of the scan target 111. For example, a complete set of images at a specified resolution may be uploaded to the service computing device 126 over time, such as after the UAV 102 has completed scanning the scan target 111. Furthermore, the database 130 may include 3D model information 134. For example, the 3D model information may include 3D model information 120 transmitted by the UAV 102. Furthermore, or alternatively, the management program 128 may run one or more 3D modeling programs using the image 132 received from the UAV 102 and the received associated location information to generate a high-resolution 3D model of the scan target 111 that can be textured using the acquired image.

[0022] In addition, the management program 128 may perform other functions to manage the images 132 and other information received from the controller 104 and / or the UAV 102. In some cases, the service computing device 126 may include a web application (not shown in Figure 1) that allows a user 112 associated with the controller 104 to access the images 132, 3D model information 134, and / or other information related to the UAV 102 in the database 130.

[0023] In some examples, one or more processors onboard the UAV102 may be configured by program code or other executable instructions to perform the autonomous operations described herein. For example, one or more processors may control and navigate the UAV102 along an intended flight path while also performing other operations described herein, such as generating or accessing an initial lower-resolution model of the object to be scanned 111, determining multiple slices of the object to be scanned 111 based on the initial model, determining the contour of each slice, and determining waypoints for acquiring images of the surface of the object to be scanned 111. The UAV102 may also determine a scan plan for navigating the UAV102 along multiple contour paths, such as at a constant speed, to acquire regularly spaced images at regular intervals, while also avoiding obstacles.

[0024] The initial model of the scan target 111 may be updated in real time during the scan based on additional image information acquired by the UAV 102 during the scan. Furthermore, or alternatively, a controller 104 or service computing device 126 located remotely from the UAV 102 and communicating with the UAV 102 may provide instructions to the processor on board the UAV 102, for example, to assist or manage one or more of the operations described above. For example, instead of having the UAV create the initial model of the scan target using distance determination, the initial model may be received by the UAV 102 from the controller 104 or service computing device 126.

[0025] To initiate a scan, user 112 may initially indicate the scan target 111 or a portion of the scan target 111 by making one or more inputs to a user interface presented to a controller 104 or other computing device that can provide input information to the UAV 102. As one example, user 112 may manually navigate the UAV 102 to acquire an image of the scan target, or user 112 may indicate the scan target by creating a polygon or other 2D shape on the image of the scan target presented in the user interface. Alternatively, the user may specify a boundary volume around the image of the scan target in the user interface, specify a boundary area based on three or more reference points (e.g., "pillars"), or specify the scan target to the UAV 102 using one of several other techniques. For example, the user may employ a technique for specifying the scan target that is determined to be most effective, depending on the shape of the scan target, the desired type of scan, the portion of the target to be scanned, etc.

[0026] Based on the received indication of the scan target 111, the UAV 102 may correlate the indication of the scan target 111 to one or more positions in space, based on the global coordinate system, correlation with the navigation coordinate system, or by various other techniques. For example, the UAV 102 may autonomously perform an initial rough scan of the scan target 111 by performing an initial scan using, for example, a range sensor to determine the position of the scan target's surface in 3D space. As one example, the range sensor may be provided by one or more of the second cameras 108, each containing an image sensor. For example, two or more arrays of the second cameras 108 may be configured for stereoscopic imaging and used to determine the distance from the UAV 102 to various points on the surface of the scan target 111. The second camera 108 may be used by the UAV 102 to enable autonomous navigation and may provide one or more low-resolution images (e.g., compared to images received from the high-resolution image sensor of the first camera 106) that can be used to determine the distance from the UAV 102 to the surface of the object to be scanned 111. If the second camera 108 includes one or more stereo imaging pairs, the UAV 102 may use the input from the second camera 108 to determine the parallax between two related images of the same surface to determine the relative distance from the UAV 102 to one or more points on the surface of the object to be scanned 111. Alternatively, images from one or more monocular images of the second camera 108 taken at different angles may be used to determine the position of the surface of the object to be scanned in 3D space, such as relative to a known position of the UAV 102, and / or various other distance sensing and 3D reconstruction techniques may be used.

[0027] As one example, the UAV 102 may merge range images from the stereo pair and / or wide baseline multi-view stereo (MVS) pair of the second camera 108 into a volumetric signed distance function (SDF) that can provide a surface position estimate containing multiple points. For example, an occupancy map may be determined first, and then a complete SDF model containing the positions of multiple points in 3D space may be determined for the scan target 111. The multiple points may represent the positions of various points on the surface 136 of the scan target 111.

[0028] As an alternative, the UAV 102 may sample surface 136 of the object to be scanned 111 with respect to its contours on surface 136, and extend these zero contours based on a function of the normal of each zero contour, which may include a smoothing step to determine the contour path at a selected distance from the zero contours (i.e., the surface of the object to be scanned 111). Spline fitting may be one technique for this alternative technique. Furthermore, performing the initial sampling at a greater distance from surface 136 of the object to be scanned 111 may be more efficient as it quickly generates a lower-resolution 3D model of the object to be scanned 111 overall, but this may limit the detail of the contours of smaller surfaces of the object to be scanned 111, which may result in insufficient image acquisition during detailed scanning of the object to be scanned 111, such as when inclined surfaces are present.

[0029] As mentioned above, the initial SDF model may typically be a low-resolution 3D model containing multiple points in 3D space indicating the locations of some points on the surface 136 of the object to be scanned 111, and may also typically include the locations of major edges 138 of the object to be scanned 111 in 3D space. The SDF model may be referred to as a lower-resolution 3D model with lower accuracy because the object to be scanned 111 has not yet been imaged or has not been scanned from sufficiently diverse viewpoints and / or sufficiently close distances for higher accuracy. For example, as UAV 102 continues to fuse stereo pair range images into the model, the closer UAV 102 flies to the surface, the more accurate the estimation of the surface location and surface shape in the SDF model becomes.

[0030] An initial scan may be performed by UAV102 to generate a low-resolution 3D model of the object to be scanned. During the initial scan, UAV102 may autonomously image one or more surfaces of the object to be scanned 111. During the initial scan, or later when performing a full scan of the object to be scanned 111, UAV102 may fly in close proximity to the object to be scanned 111, acquiring additional images and dynamically improving the completeness and / or resolution of the lower-resolution 3D model in real time. For example, the lower-resolution 3D model may include a set of points in 3D space, the positions of which correspond to surfaces of the object to be scanned 111 determined based on distance measurements.

[0031] UAV102 may use a lower-resolution 3D model to generate a scan plan for performing a scan of the object to be scanned 111 (i.e., acquiring high-resolution images). For example, the scan may include acquiring a series of images of the surface 136 of the object to be scanned 111 from a selected distance between UAV102 and the object to be scanned 111 (e.g., a selected ground resolution) to acquire images of a desired resolution, detail, overlap, etc. For example, determining the scan plan may include slicing the object to be scanned 111 into a number of slices 140 along a selected axis, the centerline of the object to be scanned, the body of the object to be scanned, etc., as will be further described below. In the example shown, assume that UAV102 has divided the object to be scanned 111 into a number of slices 140(1) to 140(14) acquired along the Z axis of a coordinate system 142 determined for the object to be scanned 111, as will be further described below.

[0032] Furthermore, the UAV 102 may select a distance D from the surface 136 of the scan target 111 to use as the baseline distance. In some cases, distance D may be a predetermined default value specified for a particular type of scan being performed, based on the desired level of detail of the image to be acquired, the intended use of the scan results, etc. Alternatively, distance D may be specified by the user 112.

[0033] In addition, the UAV 102 may determine the overlap rate O and the side overlap rate S for the images acquired during the scan. For example, the overlap rate O may be the overlap rate of consecutive images acquired in the direction of the UAV's movement when acquiring images during the scan. The side overlap (also known as lateral overlap or side overlap) may refer to the amount of overlap between images of adjacent contour slices 140, i.e., the amount of side overlap between the image acquired along the second slice 140(2) and the image acquired along the first slice 140(1) adjacent to the second slice 140(2). In some cases, the overlap and side overlap may be predetermined default values ​​for performing a particular type of scan, and / or may be specified by the user 112, etc.

[0034] Based on the camera FOV of the camera used to acquire images during scanning (e.g., the FOV of the first camera 106 in the example of Figure 1), the UAV 102 may determine a waypoint baseline Bw from the distance D and overlap O. For example, the waypoint baseline Bw may be the distance along the slice contour that the UAV 102 travels between the location where one image is acquired and the location where the next image is acquired (taking into account, for example, the FOV, the distance D from the camera to the surface, and the percentage of overlap O). For example, the surface area covered by each image may be calculated based on the FOV and distance D. Furthermore, to ensure that a specified overlap of the two images is achieved, the rotation of the UAV 102 (e.g., around the Z axis in the example of Figure 1) may be tracked and controlled as it moves from the waypoint where one image is acquired to the next waypoint where the next image is acquired. For example, the gimbal 110 may be controlled to counteract the rotation of the UAV 102 by pointing camera 106 towards the next point of interest while traversing the contour path.

[0035] The UAV 102 may determine the slice baseline Bs from the distance D and the sidelap S. For example, the slice baseline distance Bs may be the distance between a slice and an adjacent slice. For example, the area of ​​the surface covered by the FOV can be determined based on the distance D from the camera 106 to the surface 136, and then the UAV 102 may determine the spacing between slices 140 (i.e., the slice baseline distance Bs) based on a specified percentage of the sidelap S.

[0036] After the slice baselines Bs are determined, since the slice baselines Bs specify the acceptable distance between adjacent slices, the UAV 102 may determine the number of slices 140 to apply to the object being scanned 111. As one example, the UAV may associate a (X, Y, Z) coordinate system 142 with the object being scanned 111. Various techniques may be used to determine how to associate the coordinate system 142 with the object being scanned 111. As one example, the orientation of the coordinate system 142 may be based at least partially on the detected shape of the object being scanned 111 determined from the SDF model. For example, in the example of Figure 1, since the object being scanned 111 is an elongated structure extending vertically upwards, the UAV 102 may associate one of the axes, in this case the Z-axis, with the longest vertical portion of the object being scanned 111 (e.g., major edge 138), and attempt to associate at least one of the other axes, namely the X-axis and / or Y-axis, with other detected major edges 144 and / or other prominent structures of the object being scanned 111. As an alternative, UAV102 may fit a bounding ellipse to the scan target, associating one axis of the coordinate system, such as the Z-axis in this example, with the major axis of the ellipse and one of the X or Y axes with the minor axis of the ellipse. As yet another alternative, UAV102 may fit a bounding rectangle to the scan target, associating one axis of coordinate system 142 with the major axis of the bounding rectangle and another axis of coordinate system 142 with the minor axis of the bounding rectangle. As yet another alternative, the user may decide how to associate coordinate system 142 with the scan target. As yet another alternative, UAV102 may employ a machine learning model to determine the optimal assignment of the X, Y, and Z axes to the scan target. For example, the machine learning model may be trained on several different structures associated with the coordinate system in order to train the machine learning model to assign the axes of the coordinate system to various shapes and structures.

[0037] After the coordinate system 142 is associated with the scan target 111, for each axis A of the (X, Y, Z) coordinate system 142, the computing device may sample SDF slices along the selected axis A in a continuous order (e.g., ascending or descending) in steps of Bs such that A = floor + n*Bs. After the scan target model is divided into multiple slices 140 separated by a distance Bs from each other along the selected axis A, for each slice, the UAV 102 may determine the contour D along that slice. The contour may be represented as a set of 1D paths (e.g., from one waypoint to an adjacent waypoint) in a 2D plane corresponding to a particular slice.

[0038] The UAV 102 may further determine the position of waypoints based on the waypoint baseline Bw for each contour step from one waypoint along the contour path in the 2D plane of each slice to the next adjacent waypoint. Each waypoint is an image acquisition position where the first camera can be operated to acquire an image of the surface of the object to be scanned 111. Furthermore, for each selected waypoint in the slice, the UAV may select a fixation point on the surface of the object to be scanned 111 as a function of the SDF gradient and the slice plane. For example, the fixation point may be a point on the surface of the object to be scanned 111 to which the first camera 106 is focused to acquire an image of the surface when the UAV 102 is positioned at the corresponding selected waypoint. The rotation of the gimbal 110 and the rotation of the UAV 102 may be controlled so that the first camera is pointed at the specified fixation point relative to the current waypoint when the UAV 102 moves from one waypoint to the next.

[0039] In addition, UAV102 may order the waypoints within each slice in an optimal manner. As one example, UAV102 may run a traveling salesman problem algorithm between the average positions of the waypoints. This will form an ordered set of slices containing an ordered set of contours containing an ordered set of waypoints. For each of the three axes X, Y, and Z of coordinate system 142, this process may be repeated for each of the other two axes to form an ordered set of slices containing an ordered set of contours containing an ordered set of waypoints.

[0040] In some examples, UAV102 may apply a preferred axis ordering, such as Z, X, and Y, prioritizing the removal of waypoints from the second and third axes based on some proximity to the incremental coverage metric, while maintaining all waypoints for image acquisition on the first selected axis. For example, if the selected axis provides complete coverage of all areas to be scanned, scanning along the other two axes may not be performed at all. However, if the first selected axis provides only partial coverage, UAV102 may select one of the remaining axes that provides coverage for the areas remaining uncovered by the slice of the first selected axis. Following the selection of the axis and slice to be traversed, UAV102 may select a starting position based on minimizing unnecessary travel distances during the scan, such as starting from one end of the first selected axis. Depending on the scan target and the slice configuration, UAV102 may perform a trajectory pattern of a continuous trajectory of the scan target, a lawnmower pattern, a column pattern, or a combination of these patterns. UAV102 may traverse one of the selected axes sequentially contour by contour, such as starting from a first end of the axis and moving to the other end. For example, in the example shown, UAV102 may start from slice 140(1) and move sequentially upward slice by slice, e.g., 140(2), 140(3), etc., or conversely, start from slice 140(14) and move sequentially downward slice by slice, e.g., 140(13), 140(12), etc.

[0041] In addition, the UAV102 may be configured to move at a selected constant speed when traversing a contour path. For example, the traverse speed of the UAV102 may be determined based on a selected distance D, the camera's exposure time setting, the current lighting conditions, and the amount of subject blur acceptable for the intended use of the acquired image. Subject blur may be determined based on the distance (in pixels) that a point on the surface being scanned by the camera's image sensor moves between the time the exposure begins and the time the exposure ends. For example, an acceptable threshold level of subject blur may be established depending on the desired sharpness of the image. Based on an acceptable threshold level of subject blur, the capabilities of the first camera 106 (determined, for example, based on a camera model empirically determined in advance for the first camera 106, or provided by the camera manufacturer), the current lighting conditions, for example, determined by a light sensor associated with the first camera 106, and the distance D, the UAV 102 may determine a maximum traverse speed for traversing the contour path of the slice, and may traverse the contour path at a constant speed, for example based on the maximum traverse speed, while acquiring images at waypoints at regular intervals, without requiring deceleration or stopping at each waypoint.

[0042] In some examples, the speeds on different sides of the object being scanned may differ. For example, if one side of the object being scanned 111 is in shade while another side is in bright sunlight, the UAV 102 may move at a slower constant speed on the shaded side and at a faster constant speed on the sunlit side. Alternatively, in another example, the UAV 102 may determine a maximum constant speed for the portion of the object being scanned 111 with the lowest light level and use that speed as a constant image acquisition speed for the entire scan. Many other variations will be apparent to those skilled in the art who are interested in the disclosures herein.

[0043] During the scan of the object to be scanned 111, such as when traversing contour paths (not shown in Figure 1) corresponding to slices 140(1) to 140(14), UAV 102 may identify additional contours or other surfaces of the object to be scanned. For example, while UAV 102 is performing a scan, UAV 102 may replan the scan plan in real time. For example, UAV 102 may maintain the originally determined slices and join the current contour to the new contour if the current contour and the new contour are on approximately the same plane (e.g., within a difference of less than the size of slice baseline Bs). In another example, if a previously undiscovered surface is found, UAV 102 may interrupt the current scan and perform an investigation by flying around the object and at a greater distance to generate a mesh 3D model of the object that includes the new area, thereby obtaining better visualization and effective time estimation and determining a new scan plan that includes additional slices.

[0044] In some examples, a coverage index may be employed to ensure that the UAV 102 performs a scan of all desired areas of the object to be scanned 111. As one example, an initial coverage mesh may be created from waypoints of multiple contour paths of multiple slices. The UAV 102 may determine incomplete areas and fill these areas by taking images based on a comparison of the initial coverage mesh with a 3D model determined and updated during the scan based on integrating the locations of additional points into the 3D model by taking images using a second camera 108 during the scan. For each incomplete area identified based on the comparison, the UAV 102 may determine the surface normal at that location and move to that location to take one or more images of the surface at a distance D from the surface.

[0045] In some examples, the fixation point may be determined based on the gradient of the SDF model. If the Z slice is located near the ground, only the gradient of the SDF in the slice plane may be used, thereby maintaining the level of the gimbal 110 relative to that slice and preventing a situation where the gradient would otherwise cause the camera to point towards the ground. In addition, for slices that can be traversed using a lawnmower pattern, the gimbal 110 may be controlled so that the camera 106 is tilted along the contour to acquire a surface with a relatively small incline relative to the distance as the UAV 102 moves in the opposite direction in each slice of the sequence, traversing back and forth through the rows of the lawnmower pattern. A similar effect may be achieved in the case of a trajectory pattern by reversing the direction of the trajectory in alternating slices that include a forward-distorted fixation point.

[0046] Based on the scan plan determined for traversing the slices, UAV102 may traverse the contour slice by slice in the orderly manner described above, and possibly at a constant rate at which images are acquired at a constant rate. Furthermore, in some examples, while UAV102 is acquiring images using the first camera 106, UAV102 may update the 3D model using images from the second camera 108 to improve the accuracy of the 3D model. UAV102 may navigate autonomously and acquire images of the object being scanned based on the scan plan. For example, based on the scan plan, seamless images may be acquired while UAV102 flies continuously from one contour to the next, avoiding collisions with any obstacles that may be present.

[0047] During a scan of the object to be scanned based on a scan plan, UAV102 may use 3D reconstruction techniques to generate a higher-resolution version of the 3D model of the object to be scanned in real time, at least partially based on newly detected distances to points on the surface, etc. In the examples herein, “real time” may include “near real time,” and may mean performing the mentioned processing or other operations without excessive delay as processing capacity becomes available, for example, while UAV102 is still flying after acquiring one or more images, and / or while UAV102 is performing a scan, moving between waypoints, etc. The actual amount of time for real-time processing in this specification may vary based on the processing capacity of the onboard processor and other components of UAV102. For example, UAV102 in this specification may perform any of several different processing operations in real time, such as updating the 3D model or updating the scan plan.

[0048] In some examples, scanning and 3D reconstruction may be performed iteratively, such as by adding further points or other information representing the surface of the object to be scanned to the 3D model while scanning the object to be scanned 111 according to the initial scan plan, and dynamically updating the initial scan plan based on the information added to the 3D model. Thus, the quality of surface information in the 3D model may continue to improve as the UAV 102 navigates the scan plan and additional surfaces may be discovered. In some cases, the scan plan may be dynamically updated to cover new points that were not previously included or to avoid obstacles that were not previously identified, as the accuracy, coverage, etc., of the 3D model improves iteratively.

[0049] For example, after UAV102 has completed scanning the object to be scanned, or has completed at least a portion of the scan, higher resolution 3D models and images acquired during the scan may be exported from UAV102 to service computing device 126 via network 113, etc. In some examples, while scanning the object to be scanned 111 is in progress, acquired images and / or 3D model information may be transmitted wirelessly to controller 104 and / or service computing device 126. In some cases, the images may be correlated with points on the 3D model to enable the creation of a textured 3D model of the object to be scanned, for example, to view the object to be scanned 111, to perform a high-resolution inspection of the object to be scanned 111, or to perform any of the various other observations, computer graphics, or computer modeling operations that can be performed using such high-resolution 3D models and high-resolution images.

[0050] Figure 2 shows an example 200 of determining the contour paths and waypoints of slices of the scan target 202 according to several implementations. In this example, it is assumed that the scan target 202 has a star-shaped cross section, as shown in the figure. Furthermore, it is assumed that the cross section represents the XY plane with the Z axis pointing upward, as shown by coordinate system 204, and represents slice 206 acquired across the Z axis, similar to slice 140 described above with respect to Figure 1. Of course, in other examples, slice 206 may be acquired across the X axis or the Y axis.

[0051] In some cases, each slice may be used to determine contours sampled along a 2D plane of the SDF model at the slice's location, for example, by keeping one axis of the coordinate system constant. For example, a 2D contour may be determined at the zero contour, i.e., at a desired distance from the surface being scanned. Slices may be acquired in any plane and may be aligned to two of the axes by keeping a third axis constant to simplify calculations. For example, keeping Z=1 allows obtaining a cross-section at Z in the XY plane. Multiple slices may be determined by acquiring slices at specified intervals (e.g., based on the slice baseline distance Bs). For each slice, the contour may be sampled at a distance D from the surface to acquire a 1D contour path. Waypoints may then be sampled along the contour path at intervals related to the specified overlap, i.e., the waypoint baseline distance Bw. The UAV102 may move along the contour path while acquiring images at regular intervals, typically looking inward along the normal of the signed distance or some derived information of the contour shape. In some cases, traverse patterns, such as lawnmowers, tracks, or pillars, may correspond to specific instances of the scene shape and selected slice dimensions.

[0052] As described above, when acquiring images for a scan, the UAV 102 may determine a distance D from the surface 207 of the scan target 202 to be used as the baseline distance between the first camera 106 and the surface 207. In some cases, the distance D may be a predetermined default value specified for a particular type of scan being performed, based on the desired level of detail of the acquired images, the intended use of the scan results, etc. Furthermore, or alternatively, the distance D may be specified by user 112 when the user instructs the UAV 102 to perform a scan of a specified scan target 202. Similarly, user 112 may specify overlap O and side wrap S for the scan, and / or O and S may be default values ​​for a particular type of scan, a particular type of scan target, etc.

[0053] The UAV 102 may determine a contour path 208 for traversing the surface 207 relative to the surface 207 based on distance D. In addition, the UAV 102 may determine a waypoint baseline Bw based on distance D used to acquire images for scanning, desired overlap O, and the FOV of the first camera 106. For example, the waypoint baseline Bw may be the distance along the contour path 208 of the slice to which the UAV 102 travels, between, for example, the position where the first image is acquired at the first waypoint 210(1) and the position where the next image is acquired at the second waypoint 210(2). For example, the surface area covered by each image acquisition may be calculated based on the FOV and distance D. Furthermore, by spacing the waypoints evenly according to the waypoint baseline Bw, in some examples the UAV 102 may be configured to traverse the contour path 208 at a constant speed while acquiring images of the surface 207 at regular intervals. This arrangement optimizes the operation of the UAV102 while acquiring images, while also ensuring complete and orderly coverage of the scan target 202.

[0054] Furthermore, the positions of the fixation points 212 on the surface 207 of the scan target 202 may be determined. Each fixation point 212 may correspond to one of the waypoints 210, or it may be the focal point of the first camera 106 on the surface 207 of the scan target 202 when acquiring an image at the corresponding waypoint 210. For example, the first fixation point 212(1) may correspond to the first waypoint 210(1), the second fixation point 212(2) may correspond to the second waypoint 210(2), the third fixation point 212(3) may correspond to the third waypoint 210(3), and so on. In some cases, the waypoint baseline distance Bw may be used to determine the distance between fixation points (not shown in Figure 2). Then, in some cases, the positions of the waypoints 210 may be determined based on the fixation points 212.

[0055] Furthermore, to ensure that a specified overlap of two images is achieved, the rotation of the UAV 102, for example around the Z-axis, may be tracked and controlled, such as when moving from one waypoint 210 where an image is acquired to the next waypoint 210 where the next image is acquired. For example, the gimbal 110 of the first camera 106 may be controlled so that the camera is reliably pointed at the correct fixation point 222 on the surface 207 when the UAV 102 reaches the waypoint 210 corresponding to the correct fixation point 222. For example, the gimbal may be controlled by the UAV 102 to rotate the camera 106 to counteract the rotation of the UAV 102's body that may occur when traversing the contour path 208 from one waypoint 210 to the next waypoint 210. Thus, based on the waypoint baseline distance Bw, the UAV 102 may determine a plurality of spaced waypoints 210 along the contour path 208. As mentioned above, this process may be repeated for each slice to be scanned. Following the determination of waypoints, a scan pattern for efficiently traversing the waypoints may be determined.

[0056] Figures 3A and 3B illustrate examples of determining the contour path of a scan target according to several implementations. Figure 3A shows example 300 of determining slices of the scan target 302 according to several implementations. In this example, it is assumed that the UAV 102 associates a coordinate system 304 with the scan target 302, aligning the Y-axis of the coordinate system 304 with the longest major edge 306 of the scan target 302 and the X-axis with another edge 308. Furthermore, the UAV 102 slices the scan target 302 across the Y-axis to generate multiple slices spaced by a slice baseline distance Bs, determining multiple corresponding contour paths 310 spaced at a distance D from the surface of the scan target 302. Furthermore, it is assumed that the UAV 102 has determined that the contour paths 310 will result in coverage of all surfaces to be scanned. Thus, the UAV 102 may determine the location of a waypoint for each of the contours 310, or it may select a pattern for performing the scan, such as a lawnmower pattern. For example, to scan the target object 302, the lawnmower pattern may be executed by flying back and forth along adjacent contour paths 310 in succession, starting at the left or right end of a contour path 310 and reaching the opposite end.

[0057] Figure 3B shows an example 320 in which slices of the scan target 322 are determined according to several implementations. In this example, the scan target 322 is cubic, and the UAV 102 aligns the coordinate system 322 with the horizontal and vertical edges of the cube. Furthermore, the UAV 102 determines a plurality of first slices along the Z-axis to determine a plurality of corresponding first contour paths 326, which are spaced at a distance D from the side 328 of the scan target 322 and spaced apart from each other by a scan baseline distance Bs, corresponding to slices along the Z-axis.

[0058] In addition, the UAV 102 determines a second set of slices acquired across the Y-axis to determine a second set of corresponding contour paths 330, spaced at a distance D from the top surface 332 of the scan target 322 and separated from each other by a scan baseline distance Bs. The UAV 102 may determine a set of waypoints for each contour 326, 330. Furthermore, if the waypoints overlap in the surface covering of the scan target 322, the UAV 102 may remove some of those waypoints to prevent overlapping image acquisition on the same surface of the scan target 322. For example, in this example, a Y-axis waypoint for acquiring an image of the side surface 328 of the scan target 322 may be removed from the scan plan performed to scan the scan target 322. Furthermore, when determining a scan plan for scanning the object to be scanned 322, the UAV may execute a trajectory pattern to scan the side surface 328 by traversing a first set of contour paths 326, such as starting from the bottom or top of the object to be scanned 322, and then apply a lawnmower pattern to scan the top surface 332 of the object to be scanned by traversing a second set of contour paths 330. In addition, several examples of determining contour paths for various types of objects to be scanned are described herein, but many variations will be apparent to those skilled in the art who are interested in the disclosure herein.

[0059] Figures 4A and 4B show examples of determining the contour path of a scan target according to several implementations. Figure 4A shows example 400 of determining the contour path of a column perpendicular to the scan target 402 according to several implementations. In this example, it is assumed that UAV 102 associates coordinate system 404 with the scan target 402 and aligns the Z axis of coordinate system 404 with the longest major edge 406 of the scan target 402. Furthermore, UAV 102 slices the scan target 402 across the Y axis to determine multiple slices in the XZ plane, each obtaining the contour path 408 of multiple vertical columns. The contour paths 408 are spaced apart from each other by a distance less than or equal to the slice baseline distance Bs. After UAV102 has determined the position of waypoints along the contour path 408, UAV102 may traverse the contour paths 408 of a vertical column by starting from the bottom of the first contour path 408, moving upward, then moving downward in the opposite direction along the opposite side of the contour path 408, moving to the next adjacent contour path 408, and repeating until all contour paths 408 have been traversed.

[0060] Figure 4B shows an example 420 of determining multiple contour paths for a slice of the scan target 422 according to several implementations. In this example, it is assumed that the scan target 422 has a triangular cross section, as shown in the figure. Furthermore, it is assumed that the cross section represents the XY plane and represents a slice 426 acquired across the Z axis, with the Z axis pointing upward, as shown by coordinate system 424. In this example, it is assumed that the UAV 102 is instructed to perform three scans of the scan target 422 at three different distances D1, D2, and D3 from the surface 428 of the scan target 422. Thus, the UAV 102 may determine three contour paths corresponding to the three different distances: a first contour path 430 located at a distance D1 from the surface 428, a second contour path 432 located at a distance D2 from the surface 428, and a third contour path 434 located at a distance D3 from the surface 428. As one example, UAV102 may traverse all contour paths at distance D3, then all contour paths at distance D2, and then all contour paths at distance D1. As another example, UAV102 may traverse three contour paths at D3, D2, and D1 for each slice 426 before moving the Z-axis up or down to the next adjacent slice. In some examples, the speed at which UAV102 traverses each contour path 430, 432, and 434 may differ. For example, the traverse speed of contour path 434 may be faster than the traverse speed of contour path 432, and the traverse speed of contour path 432 may be faster than the traverse speed of contour path 430, for example, to reduce the amount of subject blur that may occur in images acquired on contour path 430 which is closer to surface 428, and to enable the acquisition of more detailed images.

[0061] Figures 5 and 6 include flowcharts illustrating exemplary processes according to several implementations. These processes are shown as a collection of blocks in a logical flowchart, representing a set of actions that may be performed in part or in whole in hardware, software, or a combination thereof. In a software context, a block may represent a computer-executable instruction stored in one or more computer-readable media that programs a processor to perform the enumerated actions when executed by one or more processors. Typically, a computer-executable instruction includes routines, programs, objects, components, data structures, etc., that perform a particular function or implement a particular data type. The order in which the blocks are described should not be construed as a restriction. Any number of described blocks can be combined in any order and / or in parallel to perform a process or an alternative process, and not all blocks need to be executed. For illustrative purposes, processes are described with reference to the environments, systems, and devices described in the examples herein, but processes may be performed in a wide variety of other environments, systems, and devices.

[0062] Figure 5 is a flowchart illustrating an exemplary process 500 for scanning an object according to several implementation configurations. In some examples, at least part of process 500 may be performed by the UAV 102, such as by executing a scan program and a vehicle control program. Alternatively, in some examples, at least part of process 500 may be performed by a computing device located remotely from the UAV 102, such as a controller 104 and / or a service computing device 126.

[0063] In 502, UAV102 may receive indications to be scanned. In some examples, UAV102 may receive indications to be scanned from controller 104 based on one or more user inputs made via controller 104. Alternatively, in another example, UAV102 may receive indications to be scanned from service computing device 126 or from any other computing device that can communicate with UAV102.

[0064] In 504, the UAV 102 may determine the distance D, overlap O, and sidelap S to be used for the requested scan. In some cases, the distance D, overlap O, and / or sidelap S may be predetermined default values ​​specified for a particular type of scan being performed, based on the desired level of detail of the images to be acquired, the intended use of the scan results, etc. In addition, or alternatively, one or more of these values ​​may be specified by the user 112.

[0065] In 506, the UAV 102 may access a model containing multiple points in 3D space representing the location of one or more surfaces of the indicated object to be scanned. As one example, the UAV 102 may generate a 3D SDF model based on acquiring multiple initial images of the object to be scanned in response to a received indication of the object to be scanned, as further described below with respect to Figure 6. In another example, the UAV 102 may receive an existing 3D model of the object to be scanned from the controller 104, the service computing device 126, or another remote computing device.

[0066] In 508, the UAV 102 may determine the waypoint baseline distance Bw and the slice baseline distance Bs for the scan. For example, the UAV 102 may determine the waypoint baseline distance Bw based on the FOV, distance D, and overlap O of the first camera 106. The waypoint baseline Bw may be the distance along the contour of the slice to which the UAV 102 moves between the position where one image is acquired and the position where the next image is acquired, while maintaining the desired overlap. Similarly, the UAV may determine the slice baseline distance Bs based on the FOV, distance D, and side wrap S of the first camera 106. For example, the slice baseline distance Bs may be the distance between a slice and an adjacent slice.

[0067] In 510, UAV102 may associate a coordinate system with the object being scanned. For example, UAV102 may associate one of the axes of the coordinate system with the longest edge or element of the object being scanned, and attempt to associate the other axes with other parts of the object being scanned, such as other major edges.

[0068] In 512, for each axis A of the (X, Y, Z) coordinate system, the UAV102 may determine one or more slices that cross axis A and are spaced apart on axis A by a slice baseline distance Bs. For example, if the operation is performed on a 3D SDF model, the slices may be determined for the SDF model along each of the three axes of the coordinate system.

[0069] In 514, the UAV102 may determine a contour at a distance D for each slice. As one example, the contour may be determined as a set of multiple 1D paths in the 2D plane of each slice. The multiple 1D paths together may include a contour path in 3D space located at a distance D from the surface being scanned.

[0070] In 516, for each contour, the UAV102 may move along the contour and determine the positions of waypoints spaced at intervals of less than or equal to the waypoint baseline distance Bw.

[0071] In 518, for each waypoint, UAV102 may determine the fixation point that the camera will focus on when UAV102 is at each waypoint. As one example, the fixation point may be determined for each waypoint as a function of the SDF gradient of the slice plane.

[0072] In block 520, UAV102 may order the contours within each slice. For example, UAV102 may use the Traveling Salesperson Problem (TSP) algorithm to order the contours based on the average position of each contour. By repeating blocks 512-520 for each axis of the coordinate system, an ordered set of slices can be obtained for each axis of the coordinate system, which contains an ordered set of contours containing an ordered set of waypoints.

[0073] In 522, UAV102 may determine a scan plan for performing the scan. As one example, UAV102 may use a preferred axis ordering, such as selecting the Z axis, then the X axis, and then the Y axis. Alternatively, as another example, UAV102 may select the axis containing the most slices. UAV102 may then remove slices and / or waypoints from the other two axes based on proximity index, incremental coverage index, etc. For example, if an area of ​​the surface to be scanned is already covered by waypoints on the selected axis, the waypoints on the other two axes in that area may be removed or otherwise not used.

[0074] In 524, UAV102 may determine an efficient pattern for executing the scan plan. For example, if the contour path encompasses the object to be scanned, UAV102 may first attempt to apply a trajectory pattern to traverse the contour path. Alternatively, for contour paths to which a trajectory pattern cannot be applied, UAV102 may attempt to apply a lawnmower pattern or other efficient pattern that allows movement from one contour path to adjacent contour paths of adjacent slices, following the traverse of the first contour path.

[0075] In 526, the UAV 102 may determine the speed at which it traverses each contour path in the scan plan. As one example, the UAV 102 may be configured to traverse at least a portion of the contour path at a constant speed in order to acquire images of the surface to be scanned at regular intervals in time and / or space. For example, the UAV 102 may determine the traverse speed based on a selected distance D, the camera exposure time setting, the current lighting conditions on the surface to be scanned, and the amount of subject blur that is acceptable for the intended use of the acquired image. In some cases, an acceptable threshold level of subject blur may be established depending on the desired sharpness of the image. Based on the acceptable threshold level of subject blur, the capabilities of the first camera 106, for example, the current lighting conditions determined by the light sensor associated with the first camera 106, and the distance D, the UAV 102 may determine a constant traverse speed for traversing at least a portion of the contour path in the scan plan.

[0076] In some examples, the speeds in different parts of the scan plan may differ. For example, if one part of the scan is in shade while another part is brightly lit, the UAV 102 may move at a slower constant speed in the shaded area and at a faster constant speed with respect to the brightly lit area. Alternatively, in another example, the UAV 102 may determine a maximum constant speed for the part of the scan that has the lowest light level and use that speed as a constant image acquisition speed for the entire scan plan. Many other variations will be apparent to those skilled in the art who are interested in the disclosures herein.

[0077] In 528, the UAV102 may perform the scan plan by moving at a determined constant speed as far as possible and acquiring images at regular intervals in time and / or space, which is at least partially made possible by the continuous structured nature of the contour path.

[0078] Figure 6 is a flowchart illustrating an exemplary process 600 for generating a 3D model of a scan target according to several implementation configurations. In some examples, as will be further described below, at least part of the process 600 may be performed by the UAV 102, such as by executing a scan program and a vehicle control program. Alternatively, in some examples, at least part of the process 600 may be performed by a computing device located remotely from the UAV 102, such as a controller 104 and / or a service computing device 126.

[0079] In 602, UAV102 may receive indications to be scanned. For example, UAV102 may receive indications to be scanned from controller 104 based on one or more user inputs made via controller 104. Alternatively, as another example, UAV102 may receive indications to be scanned from service computing device 126 or from any other computing device that can communicate with UAV102.

[0080] In 604, UAV102 may determine one or more orientations for acquiring images of the surface corresponding to the indicated scan target. For example, if a boundary area or boundary volume is used to specify the scan target, UAV102 may determine one or more positions and fields of view for acquiring images of the indicated scan target based on correlating the boundary area or boundary volume with a real-world location in order to acquire images of any surface within the boundary area or boundary volume.

[0081] In 606, the UAV 102 may control its propulsion mechanism 116 to fly to one or more determined attitudes in order to acquire images in each attitude. As one example, the UAV may acquire images simultaneously with two or more of the second cameras 108 so that the second camera 108 functions as a stereoscopic rangefinder and allows for the determination of distances to various points on a surface being scanned.

[0082] In 608, UAV102 may determine the distance from the UAV to the imaged surface based on, for example, determining the disparity between points in the relevant images. For example, UAV102 may employ multi-view stereo analysis of multiple images to determine the distance to each point on the surface being scanned.

[0083] In 610, UAV102 may use images from camera 108 to generate a 3D SDF model that includes the locations of multiple points in 3D space. The 3D model may initially be a coarse, low-resolution model, and its accuracy may be improved as additional images of the scanned object are acquired by UAV102. For example, a lower-resolution 3D SDF model may include multiple points, and the locations of those points in 3D space may be determined based on image analysis, such as determining the parallax or other difference between two or more images of the points acquired using multiple image sensors on UAV102.

[0084] In 612, the UAV 102 may transmit information about a lower-resolution 3D model to a remote computing device. For example, model information relating to a 3D SDF model may be transmitted from the UAV 102 to a remote controller 104, a service computing device 126, or other computing device that can communicate with the UAV 102. Furthermore, as previously stated, the 3D SDF model may be used to determine a scan plan for scanning the object to be scanned, as described above with respect to Figure 5. In addition, while a signed distance function has been described as one technique for determining the position of a point on the surface of the object to be scanned in 3D space, other techniques will be apparent to those skilled in the art who are interested in the disclosure herein.

[0085] The illustrative processes described herein are merely examples of processes provided for illustrative purposes. Many other variations will be apparent to those skilled in the art in light of the disclosure herein. Furthermore, while the disclosure herein provides several examples of suitable frameworks, architectures, and environments for performing the processes, the implementations herein are not limited to the specific examples shown and described. Moreover, this disclosure provides various illustrative implementations as described and shown in the drawings. However, this disclosure is not limited to the implementations described and shown herein and can be extended to other implementations as known to or to those skilled in the art.

[0086] Figure 7 shows an upper exemplary configuration of the UAV102 according to several implementations. In this example, the UAV102 includes a propulsion system 116 comprising four motors 704 and propellers 706. In this example, the UAV102 is shown as a quadcopter drone, but the implementations described herein are not limited to such quadcopter drones.

[0087] The UAV 102 includes a number of second cameras 108 mounted on the body 114 of the UAV 102, which may optionally be used as navigation cameras. The UAV 102 further includes a targetable first camera 106 which may include a higher resolution image sensor than the image sensors of the wider-angle cameras 108. Optionally, the first camera 106 includes a fixed focal length lens. Otherwise, the first camera 106 may include a mechanically controllable optically zoomable lens. The first camera 106 is mounted on a gimbal 110, which enables targeting of the first camera 106 within a hemispherical area of ​​approximately 180 degrees, supporting stable, low-shake image acquisition and object tracking. For example, the first camera 106 may be used to acquire high-resolution images of a target object, provide object-tracking video, or for various other operations.

[0088] In this example, three second cameras 108 are spaced apart on the upper side 708 of the UAV 102, providing a wide field of view and covered by a fisheye lens, each supporting stereoscopic computer vision. In addition to the wider-angle cameras 108 on the upper side 708 of the UAV 102, the wider-angle cameras 108 on the bottom side, described below, can be precisely calibrated relative to each other after being mounted on the body 114 of the UAV 102. As a result of the calibration, for each pixel of the image acquired by each wider-angle camera, the precise corresponding three-dimensional (3D) orientation relative to the virtual sphere surrounding the UAV can be predetermined. In some cases, six wider-angle cameras 108 with sufficiently wide FOVs (e.g., 180-degree FOV, 200-degree FOV, etc.) are employed and positioned on the body 114 of the UAV 102 to cover the entire spherical space around the UAV 102.

[0089] Figure 8 shows an exemplary configuration of the bottom side of the UAV102 according to several implementation forms. This perspective view shows three additional second cameras 108 located on the bottom side 802 of the UAV102. The second cameras 108 on the bottom side 802 may also be covered by fisheye lenses, each providing a wide field of view and supporting stereoscopic computer vision. This array of second cameras 108 (e.g., three on the top side and three on the bottom side of the UAV102) may enable high-resolution localization as well as visual inertial odometry (VIO) for obstacle detection and avoidance. For example, the array of second cameras 108 may be used to provide image information that can be used to scan the surrounding area to acquire range data and generate a range map showing the distance to objects detected within the FOV of the second cameras 108, for use during autonomous navigation of the UAV102 or for determining the distance to a surface from the UAV102.

[0090] The UAV102 may include a battery pack 810 attached to the bottom side 802 of the UAV102, along with conductive contacts 812 that enable battery charging. In addition to an internal processing unit that includes one or more processors and computer-readable media (not shown in Figure 8), the UAV102 also includes various other electronic and mechanical components. For example, the UAV102 may include a hardware configuration as described below with respect to Figure 6.

[0091] Figure 9 shows excerpts of exemplary UAV102 components according to several implementation configurations. In the examples herein, UAV102 may be referred to as a “drone” and may be implemented as any type of UAV capable of controlled flight without a human pilot on board. For example, UAV102 may be autonomously controlled by one or more onboard processors 902 running one or more executable programs. Furthermore, or alternatively, UAV102 may be controlled via a remote controller, such as via a remotely located controller 104, which is operated by a human pilot and / or controlled by an executable program running on or in conjunction with the controller 104.

[0092] In the example shown, the UAV102 includes one or more processors 902 and one or more computer-readable media 904. For example, one or more 902 may execute software, executable instructions, etc., to control flight, navigation, image acquisition, and other functions of the UAV102. Each processor 902 may be a single processing unit or multiple processing units, and may include one or more computing units or multiple processing cores. The processor 902 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, graphics processing units, state machines, logic circuits, and / or any device that manipulates signals based on operation instructions. For example, the processor 902 may be one or more hardware processors and / or logic circuits of any suitable type that are specifically programmed or configured to perform the algorithms and processes described herein. The processor 902 may be configured to fetch and execute computer-readable instructions stored in the computer-readable media 904, which can be programmed to perform the functions described herein.

[0093] The computer-readable medium 904 may include volatile and non-volatile memory and / or removable and non-removable media implemented in any type of technology for storing information such as computer-readable instructions, data structures, program modules, or other executable code and data. Such computer-readable medium 904 may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, optical storage, solid-state storage, magnetic storage, or any other media that can be used to store desired information and can be accessed by a computing device. Depending on the configuration, the computer-readable medium 904, as described herein, may be a type of computer-readable medium and / or a tangible non-temporary medium, insofar as the non-temporary computer-readable medium excludes media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0094] The computer-readable medium 904 may be used to store any number of functional elements that can be executed by the processor 902. In many implementations, these functional elements include instructions or programs that are executable by the processor 902 and, when executed, specifically configure one or more processors 902 to perform at least some of the above actions caused by the UAV 102. The functional elements stored in the computer-readable medium 904 may include a vehicle control program 906 that can be executed to control the first camera 106 to point it at a target and acquire an image of the target, in addition to controlling the autonomous navigation of the UAV 102. The functional elements further include a scan program 908 that can be executed by one or more processors to perform at least some of the scanning functions of the object to be scanned as described herein. In addition, the functional elements may include a web application 909 that can be accessed by the controller 104 for use in controlling the UAV 102. For example, the web application 909 may run in a browser on a user device connected to the controller 104, allowing the user to receive images of the UAV 102's current viewpoint, view the scanning progress of the UAV 102 while scanning the target object, and receive scanned images, 3D models, etc.

[0095] In addition, the computer-readable medium 904 may store data used to perform the navigation and scanning operations described herein. Therefore, the computer-readable medium 404 may, at least temporarily, store acquired images 910, sensor data 912, one or more 3D models 914, and one or more scan plans 916. In addition, the computer-readable medium may store navigation / tracking information 918 that can be used to navigate the UAV 102 according to one or more instructions and to provide information relating to one or more objects for imaging. Furthermore, the UAV 102 may include many other logic, program, and physical components, and the logic, program, and physical components described herein are merely examples relevant to this description.

[0096] To assist navigation, UAV102 may include a Global Navigation Satellite System (GNSS) receiver or other satellite positioning system receiver 920 mounted on UAV102. The GNSS receiver 920 may be capable of receiving signals from one or more GNSS satellites, such as the Global Navigation Satellite System (GPS), Russia's Global Navigation Satellite System (GLONASS), China's Beidou Satellite Navigation System (BDS), the European Union's Galileo system, Japan's Quasi-Zenith Satellite System (QZSS), and India's Regional Navigation Satellite System (IRNSS).

[0097] The UAV 102 may further include an inertial measurement unit (IMU) 922. In some embodiments, the IMU 922 may be configured to detect linear acceleration and gravity using one or more accelerometers, and rotational velocity using one or more gyroscopes. As one example, the IMU 922 may be self-contained, having a 3-axis gyroscope, a 3-axis accelerometer, and an embedded processor for processing inputs from the gyroscope and accelerometer to provide outputs such as acceleration and attitude. For example, the IMU 922 may measure and report velocity, acceleration, orientation, and gravity in the UAV 102, for example, by using a combination of gyroscopes and accelerometers. In addition, the UAV 102 may include other sensors 924, such as a magnetometer, barometer, proximity sensor, lidar, radar, ultrasonic, or any of the various other types of sensors known in the art.

[0098] Furthermore, the UAV 102 may include one or more communication interfaces 926, one or more flight controllers 928, one or more propulsion devices 116, and an image acquisition system 930. The image acquisition system 930 may include a second camera 108, a first camera 106, and one or more stabilization and tracking devices 934, such as the gimbal 110 described above.

[0099] In addition, the UAV102 may include an image transmission system 936, an input / output (I / O) device 938, and a power system 940. Components included in the UAV102 may communicate with at least one or more processors 902 via one or more communication buses, signal lines (not shown), etc.

[0100] The UAV102 may contain more or fewer components than those shown in the example in Figure 9, may combine two or more components as functional units, or may have different configurations or arrangements of components. Some of the various components of the illustrative UAV102 shown in Figure 9 may be implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0101] The flight controller 928 may include a combination of hardware and / or software configured to receive input data (e.g., sensor data, image data, generated trajectory, or other instructions) from the vehicle control program 906, interpret the data and / or instructions, and output control signals to the propulsion system 116 of the UAV 102. Alternatively, or further, the flight controller 928 may be configured to receive control commands generated by another component or device (e.g., processor 902 and / or controller 104), interpret those control commands, and generate control signals to the propulsion system 116 of the UAV 102. In some implementations, the aforementioned vehicle control program 906 of the UAV 102 may include any one or more of the flight controller 928 and / or other components of the UAV 102. Alternatively, the flight controller 928 may exist as a separate component from the vehicle control program 906.

[0102] The communication interface 926 may enable the transmission and reception of communication signals, for example, via a radio frequency (RF) transceiver. In some implementations, the communication interface 926 may include an RF circuit (not shown in Figure 9). In such implementations, the RF circuit may convert electrical signals to / from electromagnetic signals and communicate with communication networks and other communication devices via electromagnetic signals. The RF circuit may include, but is not limited to, an antenna system, an RF transceiver, one or more amplifiers, tuners, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identification module (SIM) card, memory, and other known circuits for performing these functions. The RF circuit may facilitate the transmission and reception of data over communication networks (including public, private, local, and wide-area networks). For example, communication may traverse a network of networks such as a wide-area network (WAN), a local area network (LAN), or the Internet.

[0103] The communication interface 926 may include one or more interfaces and hardware components to enable communication with various other devices via one or more networks. For example, the communication interface 926 may enable communication via one or more of the following, as further enumerated elsewhere in this specification: the Internet, cable networks, cellular networks, wireless networks (e.g., Wi-Fi), wired networks (e.g., optical fiber and Ethernet), and short-range wireless communication such as BLUETOOTH®. For example, 900 MHz, 2.4 GHz, and 5.8 GHz are the most common radio frequencies used for communication with UAVs, but the implementations described herein are not limited to any particular frequency.

[0104] The input / output (I / O) devices 938 may include physical buttons (e.g., push buttons, rocker buttons), LEDs, dials, displays, touchscreen displays, speakers, etc., which can be used to interact with certain functions of the UAV 102 or otherwise operate those functions. The UAV 102 also includes a power system 940 for supplying power to various components. The power system 940 may include a power management system, one or more power sources (e.g., batteries, alternating current), a charging system, a failure detection circuit, a power converter or transformer, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the computerized device.

[0105] In some embodiments, similar to an aircraft, the UAV 102 may utilize fixed wings or other aerodynamic surfaces along with one or more propulsion devices 116 to achieve lift and navigation. Alternatively, in other embodiments, similar to a helicopter, the UAV 102 may directly use one or more propulsion devices 116 to counteract gravity and achieve lift and navigation. Lift generated by thrust (as in the case of a helicopter) can offer advantages in some implementations because it allows for more controlled motion along all axes compared to a UAV that uses a fixed aerodynamic surface for lift.

[0106] The UAV102 shown in Figures 1 and 7-9 is an example provided for illustrative purposes only. A UAV102 conforming to this disclosure may include more or fewer components than those shown. For example, while a quadcopter is shown, a UAV102 is not limited to any particular UAV configuration and may include a hexacopter, octocopter, fixed-wing aircraft, or any other type of independently maneuverable aircraft, as will be obvious to a person skilled in the art who has an interest in the disclosure herein. Furthermore, while techniques for controlling the navigation of an autonomous UAV102 to perform a scan of the object to be scanned are described herein, the techniques described may similarly be applied to guided navigation by other types of vehicles (e.g., spacecraft, land vehicles, ships, underwater vessels, etc.).

[0107] Figure 10 shows an exemplary configuration of a controller 104 that includes a computing device 1001 according to several implementation forms. The controller 104 may control the UAV 102 and present a graphical user interface (GUI) 1002 for viewing images 118 received from the UAV 102. The controller 104 may include a touch screen 1004 that can correspond to the display 124 described above with respect to Figure 1. The touch screen 1004 may provide virtual controls and status indicators for controlling and viewing the status of the UAV 102. For example, a camera settings virtual control 1008 may allow the user to control the resolution and other settings of at least a first camera 106 on the UAV 102. Furthermore, a battery charge level indicator 1010 may indicate the current state of the battery on the UAV 102. A signal strength indicator 1012 may indicate the current signal strength of the communication signal with the UAV 102. A settings virtual control 1014 may allow the user to control the settings of the controller 104. Furthermore, the map virtual control 1016 may allow the user to view the location of the UAV 102 on a map. The home virtual control 1018 may allow the user to return to the home screen of the user interface 1002. The recording virtual control 1020 may allow the user to control the start or stop of recording of the scene currently within the field of view of the first camera 106. The skill virtual control 1022 may allow the user to control the skill settings of the UAV 102. The manual virtual control 1024 may allow the user to switch between manually piloting the UAV 102 or allowing the UAV 102 to pilot itself autonomously.

[0108] In addition, the user interface may display an image 1026 of the UAV 102's current field of view (e.g., a live video image). In this example, the touch screen 1004 is part of the computing device 1001, such as a smartphone, a tablet computing device, or other computing device that can be attached to the controller 104 using a controller accessory 1034. The controller 104 may further include a controller body 1036 that includes several physical controls that can be used to manually control the UAV 102, such as a left joystick 1038, a right joystick 1040, a home button 1042, a take-off / landing button 1044, an LED status indicator 1046 that shows the status of the controller 104, and other physical controls not shown in this figure. In some examples, a Wi-Fi antenna may be included in the controller accessory 1034 so that the controller 104 can provide communication range extension capability for communication with the UAV 102 at distances longer than that the computing device 1001 alone can achieve.

[0109] In some cases, the computing device 1001 (or another computing device located remotely from the UAV 102) may run an application on the processor of the computing device 1001. As one example, the application may include a browser that runs a web application supplied to the computing device 1001 by the UAV 102, or which may otherwise be supplied. For example, the web application (or another application running on the computing device 1001) may provide the user interface 1002 described above, and may provide other functions described herein with respect to the computing device 1001, such as enabling communication with the UAV 102 and enabling remote control of the UAV 102. Furthermore, in some cases, the application may enable wireless connectivity of the computing device 1001 to the controller 104 via BLUETOOTH® radio, Wi-Fi, etc.

[0110] In some implementations, some of the processing that would normally be performed by the UAV 102 (e.g., image processing and control functions) may instead be performed by an application running on the processor of a computing device 1001 located remotely from the UAV 102. Furthermore, in some embodiments, the processing load may be split between the processor on the UAV 102 and the processor on the computing device 1001, for example, to reduce processing time. Many other variations will be apparent to those skilled in the art who are interested in the disclosures herein.

[0111] In the example shown, assume that the user is using the controller 104 to select the scan target 111 to indicate to the UAV 102 that the scan target 111 should be scanned, as described above with respect to Figure 1. Thus, the user may select an acquired image of the desired scan target 111, or manipulate the user interface 1002 to draw a polygon or other 2D shape 1050 around the scan target or a part of the scan target 111 to specify the scan target 111 to the UAV 102. In this way, the user may specify an area in the user interface 1002 to indicate the scan target 111 to the UAV 102. For example, the user may use a finger 1052 or the like to control at least three reference points, i.e., handles 1054, to draw a polygonal perimeter (i.e., boundary) around the scan target 111. The UAV 102 may be configured to examine the scan target 111 within the specified perimeter, and in some examples, a volume of prism may be presented around the detected scan target corresponding to the specified perimeter. For example, the user may adjust the distance to the surface and other scan parameters before the UAV102 starts scanning.

[0112] Furthermore, while a polygon is shown in the example in Figure 8, in other examples, any other technique may be used to create a perimeter around or on the scan target. For example, the user may draw a circle, ellipse, irregular line, etc., on or around the scan target in the user interface 1002. Alternatively, in other examples, the user interface 1002 may allow manipulation of the boundary volume rather than the boundary area to indicate the scan target 111 to the UAV 102. In yet another example, the user may tap or otherwise select the image of the scan target 111 in the user interface 1002 to indicate the scan target to the UAV 102. In some embodiments, the UAV 102 may employ a machine learning model to recognize the scan target 111 based on indications received via the user interface 1002. In another example, the user may provide the UAV 102 with specified latitude and longitude coordinates of the scan target 111, and the UAV 102 may navigate itself to the specified coordinates to obtain an image of the scan target 111. Many other variations will be obvious to those skilled in the art who have an interest in the disclosures herein.

[0113] Figure 11 shows excerpts of components of an exemplary controller 104 according to several implementation configurations. In this example, the controller 104 may include components such as at least one processor 1102, one or more computer-readable media 1104, one or more communication interfaces 1106, and one or more input / output (I / O) devices 1108. For example, at least some of the processor 1102, computer-readable media, communication interfaces 1106, and I / O devices 1108 may be provided by a computing device 1001 (not shown separately in Figure 11) that is connected to the controller 104 or otherwise included together with the controller 104.

[0114] Each processor 1102 itself may comprise one or more processors or processing cores. For example, processor 1102 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operation instructions. In some cases, processor 1102 may be one or more hardware processors and / or logic circuits of any suitable type that are specifically programmed or configured to perform the algorithms and processes described herein. Processor 1102 may be configured to fetch and execute computer-readable processor-executable instructions stored in computer-readable medium 1104.

[0115] Depending on the configuration of the controller 104, the computer-readable medium 1104 may be an example of a tangible non-temporary computer storage medium and may include volatile and non-volatile memory and / or removable and non-removable media implemented in any type of technology for storing information such as computer-readable processor-executable instructions, data structures, program modules, or other data. The computer-readable medium 1104 may include, but is not limited to, RAM, ROM, EEPROM, flash memory, solid-state storage, magnetic disk storage, optical storage, and / or other computer-readable medium technologies. Furthermore, the controller 104 may, in some cases, access external storage such as a storage system, storage array, network-attached storage, storage area network, cloud storage, or any other medium that can be used to store information and can be accessed by the processor 1102 directly or via another computing device or network. Thus, the computer-readable medium 1104 may be a computer storage medium capable of storing instructions, modules, or components that can be executed by the processor 1102. Furthermore, non-temporary computer-readable media, when mentioned, exclude media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

[0116] The computer-readable medium 1104 may be used to store and maintain any number of functional elements that can be executed by the processor 1102. In some implementations, these functional elements include instructions or programs that are executable by the processor 1102 and, when executed, implement operational logic for performing the actions and services attributed to the controller 104 described above. The functional elements of the controller 104 stored in the computer-readable medium 1104 may include a user application 1108 that allows a user to remotely control the UAV 102 using the controller, as described above. In some examples, the user application 1108 may access a web application on the UAV 102, while in other examples, the user application 1108 may be a standalone controller application.

[0117] In addition, the computer-readable medium 1104 may store data, data structures, etc., used by the functional elements. For example, the computer-readable medium 1104 may at least temporarily store images 118 received from the UAV 102, and in some examples, it may store 3D model information 120 received from the UAV 102. Depending on the type of controller 104, the computer-readable medium 1104 may optionally include other functional elements and data, such as other programs and data 1110, which may include applications, programs, drivers, and data used or generated by the functional elements. Furthermore, the controller 104 may include many other logic, programs, and physical components, and the logic, programs, and physical components described herein are merely examples relevant to the description herein.

[0118] The communication interface 1106 may include one or more interfaces and hardware components to enable communication with various other devices, such as via the network 113 or directly. For example, the communication interface 1106 may enable communication via one or more of the following, as further described elsewhere in this specification: the Internet, cable networks, cellular networks, wireless networks (e.g., Wi-Fi), wired networks, one-way or two-way wireless transmission, and short-range communications such as BLUETOOTH®.

[0119] In addition, the controller 104 may include a display 124 which may include the touch screen 1004 described above with respect to Figure 10. The controller 104 may further include several other I / O devices 1112 as described above with respect to Figure 10.

[0120] Paragraph 1. An unmanned aerial vehicle (UAV) comprising a first camera mounted on the UAV and one or more processors configured to perform actions by executable instructions, the action comprising: determining a surface to be scanned; determining a distance from the surface to be scanned as a selected distance associated with acquiring an image; determining a maximum speed for traversing at least the first portion of the surface based at least on illumination associated with at least the first portion of the surface and based on the selected distance; and navigating the UAV at a speed based at least on the determined maximum speed relative to the at least first portion of the surface to be scanned based on the selected distance, while acquiring an image of at least the first portion of the surface to be scanned.

[0121] The UAV described in paragraph 1, wherein the operation further includes taking an image of a first portion of a surface to be scanned while flying at a speed below the maximum speed, determining that the illumination associated with a second portion of the surface to be scanned is different from the illumination associated with the first portion of the surface, and determining a maximum speed for traversing the second portion of the surface that is different from the maximum speed determined for traversing the first portion of the surface, based at least on the illumination associated with the second portion of the surface and on a selected distance.

[0122] Section 3. The operation for determining the maximum speed of the UAV described in Section 1 is further based on the threshold level of subject blur determined to be acceptable for the acquired image.

[0123] The operation of the UAV in paragraph 1 comprises determining a plurality of contour paths spaced apart from each other along at least one axis associated with the object to be scanned, each contour path being spaced apart from the surface of the object to be scanned based on a selected distance, and determining a plurality of image acquisition positions for each contour path, each image acquisition position indicating a position where each image of the surface of the object to be scanned should be acquired, and navigating the UAV at a speed based on a determined maximum speed relative to at least the first portion of the surface of the object to be scanned based on a selected distance, while acquiring images of at least the first portion of the surface of the object to be scanned, or navigating the UAV along one or more of the contour paths at a determined speed while acquiring images of the surface of the object to be scanned based on the image acquisition positions.

[0124] The UAV according to paragraph 5, wherein the operation comprises associating a coordinate system with the scan object based on at least the configuration of the scan object, wherein the coordinate system includes at least one axis, and further determining a plurality of spaced contour paths along at least one axis based on each contour of the surface of the scan object.

[0125] The UAV described in paragraph 6. The operation comprises receiving an indication of a scan target from a computing device; causing the UAV to assume one or more positions based on the indication of a scan target to point one or more fields of view of one or more cameras mounted on the UAV; acquiring at least one image from one or more positions using one or more second cameras; and determining a 3D model based on the distance to one or more surfaces of the scan target determined based on at least one image, wherein the 3D model includes a plurality of points corresponding to the surfaces of the scan target.

[0126] Paragraph 7. A method comprising determining a plurality of contour paths spaced apart from one another along at least one axis associated with a scan target, each contour path being spaced apart from the surface of the scan target based on a selected distance; determining a plurality of image acquisition positions for each contour path, each image acquisition position indicating a position where each image of the surface of the scan target should be acquired; and navigating the UAV along the plurality of contour paths at a determined speed while acquiring images of the surface of the scan target based on the image acquisition positions.

[0127] The method of the

[0128] Section 9. Determining the maximum speed is further based on the threshold level of subject blur determined to be acceptable for the acquired image, as described in Section 8.

[0129] Paragraph 10. The method of Paragraph 7, further comprising determining changes in illumination associated with at least a portion of the surface being scanned, and changing the speed of the UAV at least in part based on determining the changes in illumination.

[0130] The method of the paragraph 11. The method of the paragraph 7, further comprising determining overlap of images of a surface and determining the distance between adjacent image acquisition positions of each contour path, based at least on the overlap, selected distance and the field of view of the camera used to acquire the images of the surface.

[0131] Clause 12. The method of Clause 7, further comprising determining sidelaps between images of a surface and determining distances between adjacent contour paths based at least on the sidelaps, a selected distance, and the field of view of a camera used to acquire images of the surface.

[0132] The method of paragraph 13. The method of paragraph 7, further comprising associating a coordinate system with the scan object based at least on the configuration of the scan object, wherein the coordinate system includes at least one axis, and the coordinate system is associated with the scan object based at least one of aligning at least one axis with at least one of the longest edge or longest element of the scan object.

[0133] Paragraph 14. The method of Paragraph 7, further comprising navigating a UAV along at least a portion of a plurality of contour paths based on a constant velocity while acquiring images of a surface based on regular intervals.

[0134] Paragraph 15. An unmanned aerial vehicle (UAV) comprising a UAV body including a propulsion mechanism, a camera mounted on the UAV body, and one or more processors configured to perform actions by executable instructions, wherein the action includes receiving an indication of a target to scan; determining a plurality of contour paths spaced apart from each other along at least one axis associated with the target to scan, each contour path being spaced apart from the surface of the target to scan based on a selected distance; determining a plurality of image acquisition positions for each contour path, each image acquisition position indicating a position where an image of the surface of the target to scan should be acquired; determining a maximum speed for traversing the plurality of image acquisition positions of the plurality of contour paths, the maximum speed being at least partially based on a selected distance and illumination associated with the surface; and operating the propulsion mechanism to navigate the UAV along the plurality of contour paths at a speed at least partially based on the maximum speed, while acquiring an image of the surface of the target to scan using the camera, based on the image acquisition positions of each contour path.

[0135] The operation of the UAV described in paragraph 15 further includes determining overlap of surface images and determining the distance between adjacent image acquisition locations of each contour path, based at least on the overlap, selected distance and the field of view of the camera used to acquire the surface images.

[0136] Paragraph 17. The operation of the UAV described in Paragraph 15 further includes determining the sidelap between images of a surface and determining the distance between adjacent contour paths based at least on the sidelap, a selected distance, and the field of view of the camera used to acquire the images of the surface.

[0137] Paragraph 18. The operation of the UAV described in Paragraph 15 further includes determining a change in illumination associated with at least a portion of the surface being scanned, and changing the speed of the UAV based at least in part on determining the change in illumination.

[0138] Paragraph 19. The UAV described in Paragraph 15, whose operation further includes, in response to receiving an indication of a scan target, determining the locations of a plurality of points on the surface of the scan target relative to the UAV, based on one or more images of the scan target, and associating a coordinate system with the scan target, at least based on the configuration of the scan target determined from the plurality of points.

[0139] Paragraph 20. The operation of the UAV described in Paragraph 19 further includes associating a selected axis of the coordinate system with at least one of the longest edges or longest elements to be scanned.

[0140] The various instructions, processes, and techniques described herein may be considered in the general context of computer executable instructions, such as computer programs and applications stored on computer-readable media and executed by the processor herein. Generally, the terms "program" and "application" may be used interchangeably and may include instructions, routines, modules, objects, components, data structures, executable code, etc., for performing a particular task or implementing a particular data type. These programs, applications, etc., may be executed as native code or downloaded and executed on a virtual machine or other just-in-time compilation execution environment. Typically, the functionality of programs and applications may be combined or distributed as desired in various implementations. Implementations of these programs, applications, and techniques may be stored on computer storage media or transmitted via some form of communication medium.

[0141] While this subject matter is described in a language specific to structural features and / or methodological behavior, it should be understood that the subject matter is not necessarily limited to the specific features or behaviors defined in the attached claims. Rather, specific features and behaviors are disclosed as exemplary forms that implement the claims.

Claims

1. An unmanned aerial vehicle (UAV), The camera attached to the aforementioned UAV, One or more processors consisting of the following executable instructions and Determine the surface to be scanned. The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition. A first maximum speed for traversing the first portion of the surface is determined, based at least on the selected distance and the illumination associated with the first portion of the surface. A second maximum speed for traversing the second portion of the surface is determined, based at least on the selected distance and the illumination associated with the second portion of the surface. Based on the second maximum velocity, a constant velocity is determined for traversing the first portion of the surface and the second portion of the surface, and Navigate the UAV to at least the first portion and the second portion of the surface based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. Equipped with, The illumination associated with the second portion of the surface is lower in illumination than the illumination associated with the first portion of the surface. The aforementioned one or more processors are further comprised of the following executable instructions: The positions of multiple points on the surface of the object to be scanned relative to the UAV are determined based on one or more images of the object to be scanned, and A coordinate system is associated with the scan target based at least on the relative positions of the plurality of points on the surface of the scan target with respect to the UAV. Unmanned aerial vehicle (UAV).

2. An unmanned aerial vehicle (UAV), The camera attached to the aforementioned UAV, One or more processors consisting of the following executable instructions and Determine the surface to be scanned. The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition. A first maximum speed for traversing the first portion of the surface is determined, based at least on the selected distance and the illumination associated with the first portion of the surface. A second maximum speed for traversing the second portion of the surface is determined, based at least on the selected distance and the illumination associated with the second portion of the surface. Based on the second maximum velocity, a constant velocity is determined for traversing the first portion of the surface and the second portion of the surface, and Navigate the UAV to at least the first portion and the second portion of the surface based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. Equipped with, The illumination associated with the second portion of the surface is lower in illumination than the illumination associated with the first portion of the surface. The aforementioned one or more processors are further comprised of the following executable instructions: A plurality of contour paths, spaced apart from each other along at least one axis of the coordinate system associated with the scan target, are determined by virtually dividing the scan target into a series of slices along at least one axis of the coordinate system, and Navigating the UAV along one or more of the plurality of contour paths, thereby navigating the UAV with respect to at least the first portion of the surface and the second portion of the surface. The aforementioned one or more processors are further comprised of the following executable instructions: Determine multiple image acquisition positions corresponding to each contour path. Each image acquisition position indicates the position where an image of the surface to be scanned is acquired. The plurality of image acquisition positions are determined at least in part based on the constant speed, the selection distance, and the camera's field of view. Unmanned aerial vehicle (UAV).

3. An unmanned aerial vehicle (UAV), The camera attached to the aforementioned UAV, One or more processors consisting of the following executable instructions and Determine the surface to be scanned. The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition. A first maximum speed for traversing the first portion of the surface is determined, based at least on the selected distance and the illumination associated with the first portion of the surface. A second maximum speed for traversing the second portion of the surface is determined, based at least on the selected distance and the illumination associated with the second portion of the surface. Based on the second maximum velocity, a constant velocity is determined for traversing the first portion of the surface and the second portion of the surface, and Navigate the UAV to at least the first portion and the second portion of the surface based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. Equipped with, The illumination associated with the second portion of the surface is lower in illumination than the illumination associated with the first portion of the surface. The aforementioned one or more processors are further comprised of the following executable instructions: The system receives the indication of the item to be scanned via the network. In response to receiving the indication of the object to be scanned, the positions of multiple points on the surface of the object to be scanned relative to the UAV are determined based on one or more images of the object to be scanned, and Based on the distance from the UAV to one or more surfaces of the scan target determined based on the one or more images, a 3D model of at least a portion of the scan target is determined. Unmanned aerial vehicle (UAV).

4. Determining the surface to be scanned, The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition, Determining a first maximum speed for traversing the first portion of the surface, based at least on the selected distance and the illumination associated with the first portion of the surface, Determining a second maximum speed for traversing the second portion of the surface, based at least on the selected distance and the illumination associated with the second portion of the surface, Based on the second maximum speed, a constant speed for traversing the first portion of the surface and the second portion of the surface is determined, Navigating the UAV with respect to at least the first portion and the second portion of the surface, based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. The position of a plurality of points on the surface of the object to be scanned relative to the UAV is determined based on one or more images of the object to be scanned. Associating a coordinate system with the scan target based at least on the relative positions of the plurality of points on the surface of the scan target with respect to the UAV. It has, The illumination associated with the second portion of the surface is lower than the illumination associated with the first portion of the surface. method.

5. Determining the surface to be scanned, The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition, Determining a first maximum speed for traversing the first portion of the surface, based at least on the selected distance and the illumination associated with the first portion of the surface, Determining a second maximum speed for traversing the second portion of the surface, based at least on the selected distance and the illumination associated with the second portion of the surface, Based on the second maximum speed, a constant speed for traversing the first portion of the surface and the second portion of the surface is determined, Navigating the UAV with respect to at least the first portion and the second portion of the surface, based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface, The plurality of contour paths, spaced apart from each other along at least one axis of the coordinate system associated with the scan target, are determined by virtually dividing the scan target into a series of slices along at least one axis of the coordinate system. Navigating the UAV along one or more of the plurality of contour paths to navigate the UAV with respect to at least the first portion of the surface and the second portion of the surface, Determine multiple image acquisition positions corresponding to each contour path. It has, The illumination associated with the second portion of the surface is lower in illumination than the illumination associated with the first portion of the surface. Each image acquisition position indicates the position where an image of the surface to be scanned is acquired. The plurality of image acquisition positions are determined at least in part based on the constant speed, the selection distance, and the camera's field of view. method.

6. Determining the surface to be scanned, The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition, Determining a first maximum speed for traversing the first portion of the surface, based at least on the selected distance and the illumination associated with the first portion of the surface, Determining a second maximum speed for traversing the second portion of the surface, based at least on the selected distance and the illumination associated with the second portion of the surface, Based on the second maximum speed, a constant speed for traversing the first portion of the surface and the second portion of the surface is determined, Navigating the UAV with respect to at least the first portion and the second portion of the surface, based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface, Receiving the indication to be scanned via the network, In response to receiving the indication of the object to be scanned, the positions of multiple points on the surface of the object to be scanned relative to the UAV are determined based on one or more images of the object to be scanned. Based on the distance from the UAV to one or more surfaces of the scan target determined based on the one or more images, a 3D model of at least a portion of the scan target is determined. It has, The illumination associated with the second portion of the surface is lower than the illumination associated with the first portion of the surface. method.

7. An unmanned aerial vehicle (UAV), The UAV body, which includes the propulsion mechanism, The camera attached to the UAV body, One or more processors mounted on the UAV body, consisting of the following executable instructions, Determine the surface to be scanned. The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition. A first maximum speed for traversing the first portion of the surface is determined, based at least on the selected distance and the illumination associated with the first portion of the surface. A second maximum speed for traversing the second portion of the surface is determined, based at least on the selected distance and the illumination associated with the second portion of the surface. Based on the second maximum velocity, a constant velocity is determined for traversing the first portion of the surface and the second portion of the surface, and The propulsion mechanism is controlled to navigate the UAV with respect to at least the first portion and the second portion of the surface, based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. Equipped with, The illumination associated with the second portion of the surface is lower in illumination than the illumination associated with the first portion of the surface. The aforementioned one or more processors are further comprised of the following executable instructions: The positions of multiple points on the surface of the object to be scanned relative to the UAV are determined based on one or more images of the object to be scanned, and A coordinate system is associated with the scan target based at least on the relative positions of the plurality of points on the surface of the scan target with respect to the UAV. Unmanned aerial vehicle (UAV).

8. An unmanned aerial vehicle (UAV), The UAV body, which includes the propulsion mechanism, The camera attached to the UAV body, One or more processors mounted on the UAV body, consisting of the following executable instructions, Determine the surface to be scanned. The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition. A first maximum speed for traversing the first portion of the surface is determined, based at least on the selected distance and the illumination associated with the first portion of the surface. A second maximum speed for traversing the second portion of the surface is determined, based at least on the selected distance and the illumination associated with the second portion of the surface. Based on the second maximum velocity, a constant velocity is determined for traversing the first portion of the surface and the second portion of the surface, and The propulsion mechanism is controlled to navigate the UAV with respect to at least the first portion and the second portion of the surface, based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. Equipped with, The illumination associated with the second portion of the surface is lower in illumination than the illumination associated with the first portion of the surface. The aforementioned one or more processors are further comprised of the following executable instructions: A plurality of contour paths, spaced apart from each other along at least one axis of the coordinate system associated with the scan target, are determined by virtually dividing the scan target into a series of slices along at least one axis of the coordinate system, and The propulsion mechanism is controlled to navigate the UAV along one or more of the plurality of contour paths, so as to navigate the UAV with respect to at least the first portion of the surface and the second portion of the surface. The aforementioned one or more processors are further comprised of the following executable instructions: Determine multiple image acquisition positions corresponding to each contour path. Each image acquisition position indicates the position where an image of the surface to be scanned is acquired. The plurality of image acquisition positions are determined at least in part based on the constant speed, the selection distance, and the camera's field of view. Unmanned aerial vehicle (UAV).

9. An unmanned aerial vehicle (UAV), The UAV body, which includes the propulsion mechanism, The camera attached to the UAV body, One or more processors mounted on the UAV body, consisting of the following executable instructions, Determine the surface to be scanned. The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition. A first maximum speed for traversing the first portion of the surface is determined, based at least on the selected distance and the illumination associated with the first portion of the surface. A second maximum speed for traversing the second portion of the surface is determined, based at least on the selected distance and the illumination associated with the second portion of the surface. Based on the second maximum velocity, a constant velocity is determined for traversing the first portion of the surface and the second portion of the surface, and The propulsion mechanism is controlled to navigate the UAV with respect to at least the first portion and the second portion of the surface, based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. Equipped with, The illumination associated with the second portion of the surface is lower in illumination than the illumination associated with the first portion of the surface. The aforementioned one or more processors are further comprised of the following executable instructions: The system receives the indication of the item to be scanned via the network. In response to receiving the indication of the object to be scanned, the positions of multiple points on the surface of the object to be scanned relative to the UAV are determined based on one or more images of the object to be scanned, and Based on the distance from the UAV to one or more surfaces of the scan target determined based on the one or more images, a 3D model of at least a portion of the scan target is determined. Unmanned aerial vehicle (UAV).

10. An unmanned aerial vehicle (UAV), The camera attached to the aforementioned UAV, One or more processors consisting of the following executable instructions and Determine the surface to be scanned. The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition. A first maximum speed for traversing the first portion of the surface is determined, based at least on the selected distance and the illumination associated with the first portion of the surface. A second maximum speed for traversing the second portion of the surface is determined, based at least on the selected distance and the illumination associated with the second portion of the surface. Based on the second maximum velocity, a constant velocity is determined for traversing the first portion of the surface and the second portion of the surface, and Navigate the UAV to at least the first portion and the second portion of the surface based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. Equipped with, The illumination associated with the second portion of the surface is lower than the illumination associated with the first portion of the surface. Unmanned aerial vehicle (UAV).

11. The one or more processors are further configured with executable instructions as follows: A plurality of contour paths, spaced apart from each other along at least one axis of the coordinate system associated with the scan target, are determined by virtually dividing the scan target into a series of slices along at least one axis of the coordinate system, and Navigating the UAV along one or more of the plurality of contour paths, thereby navigating the UAV with respect to at least the first portion of the surface and the second portion of the surface. The UAV according to claim 10.

12. The one or more processors are further configured with executable instructions as follows: Determine the change in illumination associated with at least one of the first portion and the second portion of the surface to be scanned, Based at least in part on the determination of the changes in lighting, a new constant speed of the UAV is determined. The UAV according to claim 10.

13. The one or more processors are further configured with executable instructions as follows: The constant speed is determined based in part on the acceptable threshold level of subject blur in the acquired image. The UAV according to claim 10.

14. The one or more processors are further configured with executable instructions as follows: Select a pattern for traversing contours, enabling optimal movement at a constant speed while acquiring images at regular intervals. The UAV according to claim 10.

15. Determining the surface to be scanned, The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition, Determining a first maximum speed for traversing the first portion of the surface, based at least on the selected distance and the illumination associated with the first portion of the surface, Determining a second maximum speed for traversing the second portion of the surface, based at least on the selected distance and the illumination associated with the second portion of the surface, Based on the second maximum speed, a constant speed for traversing the first portion of the surface and the second portion of the surface is determined, Navigating the UAV with respect to at least the first portion and the second portion of the surface, based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. It has, The illumination associated with the second portion of the surface is lower than the illumination associated with the first portion of the surface. method.

16. A plurality of contour paths spaced apart from each other along at least one axis of the coordinate system associated with the scan target are determined by virtually dividing the scan target into a series of slices along at least one axis of the coordinate system, Navigating the UAV along one or more of the plurality of contour paths, thereby navigating the UAV with respect to at least the first portion of the surface and the second portion of the surface. The method according to claim 15, further comprising:

17. Determining changes in illumination associated with at least one of the first portion and the second portion of the surface to be scanned, Based at least in part on the determination of the changes in lighting, a new constant speed of the UAV is determined. The method according to claim 15, further comprising:

18. Determining the constant speed based in part on the threshold level of acceptable subject blur in the acquired image. The method according to claim 15, further comprising:

19. Selecting a pattern for traversing a contour that enables optimal movement at a constant speed while acquiring images at regular intervals. The method according to claim 15, further comprising:

20. An unmanned aerial vehicle (UAV), The UAV body, which includes the propulsion mechanism, The camera attached to the UAV body, One or more processors mounted on the UAV body, consisting of the following executable instructions, Determine the surface to be scanned. The distance from the surface of the object to be scanned is determined as the selection distance associated with image acquisition. A first maximum speed for traversing the first portion of the surface is determined, based at least on the selected distance and the illumination associated with the first portion of the surface. A second maximum speed for traversing the second portion of the surface is determined, based at least on the selected distance and the illumination associated with the second portion of the surface. Based on the second maximum velocity, a constant velocity is determined for traversing the first portion of the surface and the second portion of the surface, and The propulsion mechanism is controlled to navigate the UAV with respect to at least the first portion and the second portion of the surface, based on the selected distance and the constant speed, while acquiring images of at least the first portion and the second portion of the surface. Equipped with, The illumination associated with the second portion of the surface is lower than the illumination associated with the first portion of the surface. Unmanned aerial vehicle (UAV).

21. The one or more processors are further configured with executable instructions as follows: A plurality of contour paths, spaced apart from each other along at least one axis of the coordinate system associated with the scan target, are determined by virtually dividing the scan target into a series of slices along at least one axis of the coordinate system, and The propulsion mechanism is controlled to navigate the UAV along one or more of the plurality of contour paths, so as to navigate the UAV with respect to at least the first portion of the surface and the second portion of the surface. The UAV according to claim 20.

22. The one or more processors are further configured with executable instructions as follows: Determine the change in illumination associated with at least one of the first portion and the second portion of the surface to be scanned, Based at least in part on the determination of the changes in lighting, a new constant speed of the UAV is determined. The UAV according to claim 20.

23. The one or more processors are further configured with executable instructions as follows: Select a pattern for traversing contours, enabling optimal movement at a constant speed while acquiring images at regular intervals. The UAV according to claim 20.

Images