Pose estimation of a mobile autonomous device for a partial period of a complete sensor scan

Abstract

An apparatus for determining a current posture of a mobile autonomous device is disclosed. In an embodiment, the apparatus may include: an interface circuit for receiving detection and ranging data output by a light detection and ranging (LIDAR) sensor, the sensor performing a scan every time period T and continuously providing detection and ranging data of D degrees as a plurality of data units, each of the plurality of data units covering a portion of the D degree scan. The apparatus may further include a posture estimation circuit coupled to the interface circuit for determining and providing the current posture of the mobile autonomous device every partial time period t independently of the time when the LIDAR sensor actually completes each scan. In an embodiment, the apparatus may be disposed in the mobile autonomous device.

Description

Technical Field

[0001] Embodiments relate generally to the field of mobile autonomous devices, and in particular to pose estimation for such devices at partial time periods of a complete sensor scan. Background Art

[0002] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.

[0003] Mobile autonomous devices such as robots, self-driving cars or unmanned aerial vehicles (UAVs) need to determine their current pose (position and orientation) in real time. This process (commonly called localization) is usually performed by pose estimation algorithms operating on odometry data obtained by sensors.

[0004] In most designs, pose is estimated based on sensor data obtained via light detection and ranging (LIDAR) sensors, cameras, and / or other types of sensors. For LIDAR-based designs, there is usually a laser scanner in the LIDAR device that rotates at a specific frequency (usually 5 to 40 Hz) to obtain a complete or near-complete view of the environment. Typically, the device updates the pose estimate each time the device receives a new set of scanned data. Therefore, the pose estimation frequency is limited by the rotation rate of the laser scanner. When a higher frequency of pose estimation is required, a higher-end LIDAR device must be selected, which usually increases the cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The embodiments will be easily understood by the following detailed description in conjunction with the accompanying drawings. To facilitate this description, the same reference numerals indicate the same structural elements. The embodiments are shown in the figures of the accompanying drawings by way of example and not by way of limitation.

[0006] Figure 1 Examples of pose estimation input and pose estimation cycles using segments of LIDAR sensor data, which are portions of a complete cycle of data, are shown in accordance with various embodiments of the present disclosure.

[0007] Figure 2 shows the contents of an example mobile buffer for storing sensor data and

[0008] Figure 3 A comparison of partial pose estimation determination with and without LIDAR driver support is shown in accordance with various embodiments.

[0009] Figure 4An overview of the operational flow of a process for determining an estimated pose of an autonomous device in real time using portions of a complete cycle of data is shown in accordance with various embodiments.

[0010] Figure 5 An overview of the operational flow of a process for determining an estimated pose of an autonomous device using portions of a complete cycle of data with a delay is shown in accordance with various embodiments.

[0011] Figure 6 The figure illustrates a block diagram of a computer device suitable for implementing the present disclosure according to various embodiments.

[0012] Figure 7 An example computer-readable storage medium having instructions configured to implement Figure 4 and Figure 5 aspects of the process shown in . DETAILED DESCRIPTION

[0013] In an embodiment, an apparatus for determining a current pose of a mobile autonomous device may be provided. The apparatus may include an interface circuit for receiving detection and ranging data output by a light detection and ranging (LIDAR) sensor that nominally scans per time period T and provides D degrees of detection and ranging data in a continuous plurality of quanta, each quantum covering a portion of the D degree scan. The apparatus may further include a pose estimation circuit coupled to the interface circuit for determining and providing the current pose of the mobile autonomous device per portion of time period t independently of when the LIDAR sensor actually completes each scan. In an embodiment, the apparatus may be disposed in the mobile autonomous device.

[0014] In embodiments, a pose determination device may significantly increase the frequency with which a mobile autonomous device can perform pose estimation.

[0015] In embodiments, the mobile autonomous device may be a robot, an autonomous or computer-assisted driving vehicle, an unmanned aerial vehicle (UAV) or "drone," or the like.

[0016] The following specific embodiments refer to the accompanying drawings. The same reference numerals may be used in different figures to identify the same or similar elements. In the following description, specific details such as specific structures, architectures, interfaces, technologies, etc. are described for the purpose of explanation rather than limitation, so as to provide a comprehensive understanding of various aspects of various embodiments. However, it will be apparent to those skilled in the art who benefit from this disclosure that various aspects of the claimed embodiments may be implemented in other examples that are separated from these specific details. In some instances, descriptions of well-known devices, circuits, and methods are omitted to avoid obscuring the descriptions of the embodiments due to unnecessary details.

[0017] Each operation can be described as a plurality of discrete actions or operations in sequence in a manner that is most helpful in understanding the claimed subject matter. However, the order of description should not be interpreted as implying that these operations must be dependent on the order. Specifically, these operations may not be performed in the order presented. The described operations may be performed in an order different from the described embodiments. In additional embodiments, various additional operations may be performed or the described operations may be omitted.

[0018] As used herein (including in the claims), the term "circuit" may refer to, may be part of, or may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and / or memory (shared, dedicated, or group) executing one or more software or firmware programs, a combinational logic circuit, and / or other suitable hardware components that provide the described functionality. In some embodiments, the circuit can be implemented in one or more software or firmware modules, or the functions associated with the circuit can be implemented by one or more software or firmware modules. In some embodiments, the circuit may include logic that is at least partially operable in hardware.

[0019] In the following description, reference is made to the accompanying drawings forming a part of this document, wherein the same reference numerals throughout the drawings indicate the same parts, and wherein practicable embodiments are shown by way of illustration. It should be understood that other embodiments may be utilized, and structural or logical changes may be made without departing from the scope of the present disclosure. Therefore, the following detailed description should not be understood in a limiting sense, and the scope of the embodiments is defined by the appended claims and their equivalents.

[0020] The operations of the various methods may be described in turn as a plurality of discrete actions or operations in a manner that is most helpful in understanding the claimed subject matter. However, the order of description should not be interpreted as implying that the operations necessarily depend on the order. Specifically, the operations may not be performed in the order presented. The operations described may be performed in an order different from that of the described embodiments. In additional embodiments, various additional operations may be performed and / or the operations described may be omitted, split, or combined.

[0021] For purposes of this disclosure, the phrase "A and / or B" means (A), (B), or (A and B). For purposes of this disclosure, the phrase "A, B, and / or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C).

[0022] The specification may use the phrases "in an embodiment" or "in embodiments," which may each refer to one or more of the same or different embodiments. Furthermore, the terms "comprising," "including," "having," etc. as used with respect to the embodiments of the present disclosure are synonymous.

[0023] Moreover, it should be noted that the embodiment can be described as a process depicted as a flow chart, a flow chart, a data flow chart, a structure diagram or a block diagram. Although the flow chart can describe the operation as a sequential process, many of these operations can be performed in parallel, concurrently or simultaneously. In addition, the order of the operation can be rearranged. The process may be terminated when its operation is completed, but it may also have additional steps that are not included in (multiple) drawings. The process may correspond to a method, a function, a program, a subroutine, a subprogram, etc. When the process corresponds to a function, the termination of the process may correspond to the function returning to the calling function and / or the main function. In addition, the process can be implemented by hardware, software, firmware, middleware, microcode, hardware description language or any combination thereof. When implemented with software, firmware, middleware or microcode, the program code or code segment that performs the necessary tasks can be stored in a machine or computer readable medium. The code segment can represent any combination of a program, a function, a subroutine, a program, a routine, a subroutine, a module, a program code, a software package, a class, or an instruction, a data structure, a program state, etc.

[0024] As used below (including in the claims), the term "circuitry" may refer to, may be part of, or may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group, including one or more processing elements (PEs) of a dataflow machine) and / or memory (shared, dedicated, or group) that executes one or more software or firmware programs, a programmable combinatorial logic circuit (e.g., a field programmable gate array (FPGA)), and / or other suitable hardware components that provide the described functionality. In some embodiments, a circuit may implement one or more software or firmware modules, or the functionality associated with a circuit may be implemented by one or more software or firmware modules.

[0025] As used below, including in the claims, the term "memory" may refer to one or more hardware devices for storing data, including random access memory (RAM), magnetic RAM, magnetic core memory, read-only memory (ROM), magnetic disk storage media, optical storage media, flash memory devices, and / or other machine-readable media for storing data. The term "computer-readable medium" may include, but is not limited to, memories, portable or fixed storage devices, optical storage devices, wireless channels, and various other media capable of storing, containing, or carrying (multiple) instructions and / or data.

[0026] As mentioned, typically, a LIDAR device sends data from its laser scanner to a host processor in real time via some type of serial communication protocol while the scanner is rotating. The LIDAR device then sends a synchronization signal to the host processor when the scanner rotates through a predefined angle (e.g., 360°). The host processor then knows that the scanning cycle has ended, and it can perform pose estimation using the scanned data in that cycle. In an embodiment, the host processor does not need to wait for the synchronization signal to determine the estimated pose. In fact, in an embodiment, the host processor can perform pose estimation each time a programmable segment of the scanned data representing a portion of a complete scan is received. For example, for a standard 360° scanning cycle, in an embodiment, pose estimation can be performed after a segment of scan data of each 90° has been received, thereby increasing the frequency of pose estimation by four times. In an embodiment, to ensure that the estimator still has a complete view of the environment, as in conventional solutions, the host processor can retain scan data for a segment to perform several estimates and recombine the stored data with the newly received segment.

[0027] In an embodiment, the system cost can be reduced. For some use cases, pose estimation is often required. For example, if you want to move the robot quickly, assuming a speed of 2m / s (which is the fast walking speed of a human), then pose estimation is performed at 5-10Hz, which is a typical rotation speed of a low-end LIDAR, which may be too slow to avoid obstacles and perform motor control because the robot will move up to 20-40cm in each scanning cycle. In order to provide pose estimation that keeps up with the speed of the robot, either a high-end LIDAR (with a rotation rate of, for example, 40Hz) needs to be selected, or various embodiments need to be implemented to obtain high-frequency pose estimation with a low-end LIDAR device.

[0028] In addition, in an embodiment, the robustness of posture estimation can be enhanced. Most posture estimation schemes are iterative, which means that the algorithm uses the posture estimation of the previous cycle as the initial value of the posture estimation of the current cycle. If the ground truth posture of the current cycle is too far away from the initial value, the algorithm may fail. Because in an embodiment, the period of posture estimation can be greatly shortened, this may result in smaller differences between consecutive postures of adjacent cycles, thereby reducing the possibility of posture estimation failure.

[0029] Figure 1 An example LIDAR device implemented according to various embodiments is shown, which acquires scan data at each angle. As described above, in an embodiment, pose estimation can be called whenever a small amount of scan data has been obtained, which is different from waiting for a full cycle of scan data in conventional solutions. Figure 1It shows how the scan data can be reassembled to ensure that the pose estimation algorithm always has a complete view.

[0030] refer to Figure 1 , it can be seen that the pose estimation input 120 includes 360-degree scanning data. In addition, Figure 1 In the example of , pose estimation inputs 120 are provided to a pose estimation algorithm in a host processor (not shown) every second, for a total of 360 pose estimates 125 per cycle, increasing the pose estimation frequency 360 times. This is accomplished by using each of the pose estimation inputs 120 for a new 1 degree of scan data (essentially one scan data point) in conjunction with the immediately preceding 359 degrees of scan data. Thus, for example, pose estimation input #3 uses the data points from 2° to 359° of the immediately preceding scan, as well as the data points from 0° and 1° of the currently ongoing scan, the latter of which are the newly acquired scan data points. In an embodiment, this is facilitated by maintaining a motion buffer that stores the full 360° of scan data on a rolling basis. Figure 1 The process is as follows: Figure 2 described.

[0031] Note that Figure 1 In the example of , it is assumed that the laser scanner senses 360 times per cycle from 0° to 359°. Note that this is an illustrative example, and in practice, the range and interval of the angles may be different. Thus, for example, the scanner may scan every 0.25° from 0° to 270°, or may not even be fixed. In embodiments, several variations of the angle intervals and ranges may be used. For example, the inventors successfully implemented an embodiment using an RPLidar A2 LIDAR device that scanned 400 times per 360° cycle at uneven angle intervals.

[0032] It should also be noted here that Figure 1 , pose estimation is shown as being called for each single scan data point. This is also an illustrative example and not a requirement. In practice, this may not be practical in some cases due to computational resource limitations or other reasons. Thus, for example, in an embodiment implemented on an RPLidar A2 device, pose estimation is called whenever a segment of 32 new scan data points is obtained (out of a total of 400 scans per cycle). Thus, in this example implementation, the rate of pose estimation is increased by a factor of 12.5 (400 / 32) rather than 400.

[0033] Figure 2 The buffer zone 200 is shown in accordance with various embodiments. Figure 2, shows the contents of buffer 200 during three consecutive scan cycles: T0 210, T1 220, and T3 230. In this example, buffer 200 obtains scan data in portions of 1 / 4 of a scan cycle, or for a sensor that takes a period of time T to complete a full scan, the buffer obtains data increments every portion of time period t, where t is T / 4. Thus, Figure 1 Taking 360 scans per cycle as an example, each segment includes 90 degrees of scanning data.

[0034] As a result, the buffer obtains a new data segment four times during a scan cycle, and thus always maintains four segments of data: the new segment and the three immediately preceding segments. Thus, the four segments of data are input to a pose estimation algorithm, which can run in a processor or in a hardware accelerator such as an FPGA, as described more fully below.

[0035] Continue to refer Figure 2 , to the right of the graphical representation of the buffer contents over time, there are three columns, Tpose 250, tpose 260, and accuracy 270, each of which refers to a pose estimate, and accuracy 270 indicating whether tpose estimate 260 is more accurate than or as accurate as the Tpose estimate. According to various embodiments, Tpose 250 provides a pose estimate obtained by calculating the pose only once per scan cycle, while tpose 260 provides a pose estimate in which the pose is estimated after each partial segment of the scan data is obtained.

[0036] Continue to refer Figure 2 , the buffer contents for the first scan cycle T0 are shown at 210. Each row shows the contents of the moving buffer 200 at the beginning of its time period. Note that each consecutive row in the buffer shows the contents of the next t interval, and therefore the time domain distance between any two consecutive rows in the buffer 200 is a fraction of time t. Since t=T / 4, four configurations of the buffer are shown for each scan cycle. Initially, at 211, where t=0, no data is stored in the moving buffer 200. At 212, where time t=1, the first segment enters scan T0, as shown on the far left of the row. At rows 213 and 214, at times t=2 and t=3, respectively, the second and third data segments enter the buffer, such that at 214 (the last configuration of scan T0), there are three data segments stored. Note that since the data segments are not available until some time after acquisition, all of the data from scan T0 is available only during the first segment of scan T1 (representing the buffer contents at the beginning of scan cycle T1).

[0037] Therefore, at time t=4, at the beginning of scan cycle T1, as shown in row 221, all four segments of the scan data are now available for calculating the estimated posture. Here, T-pose 250 shows P0, the first full scan cycle posture, and t-pose based on the partial data segment shows P11, the first posture according to the embodiments of the present invention. Note that for ease of explanation, the t-pose value has a "hat" on "P". Since P0 and P11 use the same data, their precision is equal, as indicated by the "=" sign in the precision 270 column. Therefore, posture P0 is the same as partial posture P11 because they use the same input data set.

[0038] Continue to refer Figure 2 , at rows 222 to 224, the next three partial data segments of scan T1 enter the mobile buffer 200, so that at row 224, at the beginning of the last partial time interval of scan cycle T1, the mobile buffer 200 has three data segments of T1 and the last data segment of scan T0. In addition, at each of rows 222-224, although the T posture 250 does not change, the t posture changes. Therefore, at each of rows 222, 223 and 224, t postures P12, P13 and P14 are calculated based on a mixture of T0 and T1 data segments, as shown in the figure. Since the data used by each of these t postures is newer than T posture 260P0, each of them has a higher accuracy than P0, as shown in the accuracy column 270, where each of the t postures P12, P13 and P14 is shown as having a higher accuracy than P0, indicated by the ">" symbol.

[0039] Therefore, at time t=8, at the beginning of scan cycle T2, as shown in row 231, all four data segments of the T1 scan are now available for calculating the estimated posture. Here, T Posture 250 shows P1, the second full scan cycle posture, and T Posture shows P21, the first posture using the entire data of the T1 scan. As described above, since P1 and P21 use the same data, their accuracy is approximately equal, as indicated by the "~" symbol in the Accuracy 270 column. However, it is noted that they are not exactly equal, because the output of the posture estimation also depends on the output of the previous cycle, so in an embodiment, the robustness of the posture estimation can be enhanced. Therefore, even if posture P21 and posture P1 are based on the same data, posture P21 is more accurate than posture P1 because posture P21 is also based on the previous posture P14, while posture P1 is only based on the previous posture P0.

[0040] Further reference Figure 2, at rows 232 to 234 representing times t=9 to t=11, the next three partial data segments of scan T2 enter the moving buffer 200, so that at row 234, at the beginning of the last partial time interval of scan cycle T2, the buffer has three data segments of T2 and the last data segment of scan T1. As described above, at each of rows 232-234, although T posture 250 does not change, the t posture changes. Therefore, at each of rows 232, 233 and 234, t postures P22, P23 and P24 are calculated based on a mixture of T1 and T2 data segments, as shown. Since these t postures each use newer data than T posture 260P1, each of them has a higher accuracy than P1, as shown in the accuracy column 270, where each of t postures P22, P23 and P24 is shown as having a higher accuracy than P0, indicated by the ">" symbol.

[0041] It can be understood that there is a relationship between the scanning rate and the number of partial data segments to be stored in the mobile buffer 200. Assuming that the LIDAR device rotates R times per second and scans S times per second, S / R scanning data will be generated in each scanning cycle. Further assuming that the number P of data is packaged together for transmission between the LIDAR device and the host, then in an embodiment, segments can be kept in a buffer.

[0042] As mentioned above, in Figure 2 The embodiment shown in obtains partial data during several intervals during each scanning cycle. Therefore, the host processor executing the pose estimation algorithm operates independently of the scanning cycle synchronization signal that the LIDAR sensor can output, and can calculate pose estimates at intervals of partial time t based on its own defined data segments. Note that this is possible in embodiments in which a driver of the LIDAR device can be accessed and there is an instruction for the LIDAR device to provide partial data segments to the host processor. Sometimes it may not be possible to change the data output parameters of the LIDAR device. However, in alternative embodiments, it is possible to speed up the pose estimation frequency even if the driver of the LIDAR device is not accessed, as described below with reference to Figure 3 described.

[0043] Figure 3A comparison of the timing of scan data provided by a LIDAR device and the corresponding timing of pose estimation based on the data is shown in three scenarios. The three scenarios include a full cycle 315, a segmentation 330 with driver support, and a segmentation 350 without driver support, where, as described above, the full cycle 315 is a standard LIDAR full cycle for data pose estimation. For the purpose of comparing the timing of scan data transmission and pose estimation in the two illustrated embodiments 330 and 350 and for ease of description, the information of the full cycle 315 is provided.

[0044] refer to Figure 3 , in a full cycle 315, a complete scan of data must be collected before pose estimation can occur. Figure 3 In , the term "SLAM" is used for pose estimation, referring to "simultaneous localization and mapping algorithm". Thus, pose estimate P1 340 for scan cycle D1 is short-lived into scan cycle D2, as shown. Similarly, pose estimate P2 342 for scan cycle D2 is short-lived into scan cycle D32, also as shown. This delay is characteristic of the finite time it takes to perform SLAM calculations after receiving data input to the calculations, and is substantially the same in each of the scenarios 315, 330, and 350.

[0045] Further reference Figure 3 , the second scenario (partitioning with drive support) is Figure 1 and Figure 2 As expected, there are four SLAM calculations per scan cycle. For example, as shown during scan cycle D2, P1.4, P2.1, P2.2, and P2.3. SLAM P1.4 operates on the four partial data segments obtained during scan cycle D1 (i.e., D1.1, D1.2, D1.3, and D1.4), which are similar to those shown in FIG. Figure 2 The four partial data segments of the scanning cycle T0 shown in row 221 of SLAM P2.1. At the partial time t after P1.4, SLAM P2.1 operates on the mixed set of D1 and D2 data (i.e., D1.2, D1.3, D1.4, and D2) as described above. Figure 3As can be seen in , each SLAM calculation ends at a certain time after the last segment of its data is obtained. Therefore, in scene 330, P1.4 is completed at a certain time t2 after the end of data segment D1.4 at t1, and SLAM P2.1 is completed at a certain time after the end of data segment D2.1. In this example, the delay (t2-t1) of the SLAM calculation is less than the time interval t331 of a partial time period. In addition, P1.4 is equivalent to P1, but as mentioned above, P1.4 is different from P1 because the output of SLAM also depends on the output of the previous cycle, which is why, as mentioned above, each embodiment can enhance robustness.

[0046] Now go to Figure 3 Scenario 350, i.e., the "segmentation without driver support" scenario, although in this scenario, the host processor receives the scan data of the entire cycle each time through buffering and delaying calculations, it can be pretended that the scan data appears sequentially within the host processor, so a fast scan (FastScan) can be performed on these segments in partial segments to optimize scan data processing. Note that for convenience, the embodiments of this article may be referred to as "FastScan". In addition, in the embodiments of scenario 350, the segments can be specified essentially arbitrarily, limited only by the computing power and capacity of the host computer when performing SLAM calculations.

[0047] Continue to refer Figure 3 , as shown in scenario 350, because the fast scan must wait to receive all of the D1 data before it can process the various segments into which the D1 data is divided, the fast scan calculations P1.1, P1.2, P1.3, and P1.4 are all performed sequentially during data period D2. As shown, this introduces an additional delay 360 on top of scenario 330, which outputs the estimated posture P1.4 a short time t2 after the end of scan period D1 at t1, while in scenario 350, posture P1.4 is output at time t3, where (t3-t2)=delay 360. In an embodiment, the additional delay 360 can be strictly less than the period 365 of one scan period, which is acceptable in most applications.

[0048] Described next Figure 4 and Figure 5 An overview of the operational flow of a process for pose estimation of an autonomous device according to embodiments of scenarios 330 and 350 is shown, respectively. Such an autonomous device may be a robot, an autonomous or computer-assisted driving vehicle, an unmanned aerial vehicle (UAV) or "drone," etc. These figures are described next.

[0049] Figure 4An overview of the operational flow of a process for determining a posture of an autonomous device using portions of a complete cycle of data is shown in accordance with various embodiments. Figure 4 The process can be done by Figure 6 4. As shown in the figure, process 400 may include operations performed at blocks 410-460. Process 400 may start at block 410. At block 410, a new segment of LIDAR sensor data for a portion of time period t may be obtained. Process 400 may proceed from block 410 to block 420, where a new segment of LIDAR sensor data for a portion of time period t may be obtained. Figure 6 The LIDAR data buffer 655 provided in the computer device 600 or as described above Figure 2 Buffer 200 shown in FIG. 1 ) clears the oldest partial segments of sensor data.

[0050] Continue to refer Figure 4 , process 400 may proceed from block 420 to block 430, where the new segmentation of data may be combined with previously received segments of LIDAR sensor data for estimating a pose for (partial) time period t. Process 400 may proceed from block 430 to block 440, where the LIDAR sensor data for pose estimation at time period t (which, as mentioned, includes the most recently received segmentation and the segments received sufficiently earlier to include sensor data for time T) may be transformed into a point cloud. Process 400 may proceed from block 440 to block 450, where the current pose of the autonomous device at (partial) time period t is estimated by matching the point cloud with a reference map. Process 400 may proceed from block 450 to block 460, where the estimated current pose of the autonomous device at time period t may be provided to various clients (such as motion controllers, etc.). At block 460, process 400 may return to block 410 and receive the next portion of data segments, or if not, process 400 may terminate.

[0051] Figure 5 An overview of the operational flow of a process for performing pose estimation on an autonomous device using multiple portions of a complete cycle of data is shown in accordance with an embodiment in which a host processor receives data for one complete scan cycle at a time, such as in the example described above. Figure 3 In scene 350. Figure 5 The process can be done by Figure 65. The process 500 may include operations performed at blocks 510-550. The process 500 may start at block 510. At block 410, LIDAR sensor data for a complete scan cycle of scan cycle T may be obtained. From block 510, the process 500 may proceed to block 520, where the data may be buffered and divided into a plurality of partial segments, each partial segment representing data for a portion of time t, t<t.

[0052] Continue to refer Figure 5 , process 500 may proceed from box 520 to box 530, in which the partial data segment at time tk may be combined with the sequence of immediately preceding partial data segments tk-1, tk-2, ..., tk-n to obtain sensor data for estimating the pose at partial time period tk. Process 500 may proceed from box 530 to box 540, in which the LIDAR sensor data for pose estimation at time period tk (which, as mentioned, includes the most recently received segment at time tk and the segments tk-1, tk-2, ..., tk-n received sufficiently early to include sensor data at time T) may be transformed into a point cloud. Process 500 may proceed from box 540 to box 550, in which the estimated current pose of the autonomous device at time period tk may be determined by matching the point cloud with a reference map, and the estimated pose may be provided to various clients (such as motion controllers, etc.). As described above in combination with Figure 3 As described in the scenario 350 of FIG. 3 , this determination of the estimated posture has a certain delay relative to the end of the time period tk. Process 500 may return to frame 510 from frame 550 and receive data for the next full cycle, or if not, process 500 may terminate.

[0053] Figure 6 The figure shows a block diagram of a computer device suitable for implementing the present disclosure according to various embodiments. Figure 6 Before, some details of example SLAM calculations performed by the example SLAM program according to various embodiments are first provided. The example calculations are described above. Figure 4Specific implementation of process 400. In an embodiment, seven tasks can be performed in pose estimation. (The division of tasks is arbitrary and for illustration purposes only). First, the LIDAR device can scan the environment of the autonomous device and then send data to the host computer. In an embodiment, the data represents a partial segmentation of a complete scan cycle. According to one embodiment, the portion of the complete scan cycle contained in the segmentation can be selected and modified. Second, the LIDAR driver on the host computer can pre-process the data and publish it to the SLAM program. Third, the SLAM program can use a moving buffer to regroup the scan data, and can also decide how much previously received data to clear from the buffer so as to always keep the scan data of the complete cycle. Fourth, the SLAM program can receive the scan data (angle + range) and transform it into a point cloud, which includes the coordinates of all scanned points from the perspective of the autonomous device. Fifth, the SLAM program can try to find the pose when the point cloud best matches the map. Sixth, in an optional task, the SLAM program can update the map using the point cloud and the estimated pose (if there is a pre-built map, this task can be skipped). Finally, and seventhly, the SLAM program can publish the estimated pose and updated map to any interested clients.

[0054] Of all the tasks described above, the fifth item (scan matching) may be the most computationally intensive. Therefore, in an embodiment, it may be performed by an FPGA, while the other tasks (except the first item) may be performed by software running on a CPU (e.g., an ARM CPU provided in a system on chip (SoC)). In such embodiments, whenever the software obtains a data cycle including a new segment of scan data, it may transform the data into a point cloud and copy the point cloud to the FPGA via a shared memory. The process may then call the scan matching logic in the FPGA, for example, by writing a signal to a predefined address of the shared memory. In such embodiments, the FPGA may perform scan matching, write back the estimated posture, and notify the software program. In some embodiments, the graph may be pre-built and static (not updated after each posture estimation or any posture estimation). In such embodiments, the graph may be copied to the FPGA only when the process starts.

[0055] Thus, a host processor may include an interface circuit for receiving detection and ranging data output by a light detection and ranging (LIDAR) sensor that nominally scans every time period T and provides D degrees of detection and ranging data in a succession of quanta, each quantum covering a portion of the D degree scan.

[0056] The host processor device may further include a pose estimation circuit coupled to the interface circuit for determining and providing a current pose of the mobile autonomous device at each partial time period t independent of when the LIDAR sensor actually completes each scan.

[0057] In an embodiment, the host processor and LIDAR sensor equipment may be provided on a mobile autonomous device.

[0058] It is important to note here that in this example implementation, the more the embodiment increases the scanning frequency (i.e., reduces the number of data points per segment), the more work the FPGA has to do, so the fifth task is repeated multiple times per scanning cycle (finding the pose when the point cloud best matches the graph).

[0059] Given these task descriptions, now refer to Figure 6 , which illustrates a block diagram of a computer device suitable for implementing the present disclosure according to various embodiments. As shown, the computer device 600 may include a SoC 610 and a system memory 650. The SoC 610 may include one or more processors 615, a memory 613, and a hardware accelerator (such as an FPGA 620). According to various embodiments, a FastScan core including scan matching logic may be provided to the FPGA 620 to perform pose estimation.

[0060] The system memory 650 may include any known volatile or non-volatile memory and may include a LIDAR driver 651. A FastScan module 653 may also be provided to the system memory 650, which manages the various tasks described above in addition to the intensive calculations implemented in the FPGA 620. The system memory 650 may also include a LIDAR data buffer 655, which may be used as described above in conjunction with Figure 2 It should be noted that in alternative embodiments, unlike the hybrid embodiment shown, the entire fast scan calculation can be implemented in hardware or software.

[0061] In addition, as described above, the computer device 600 may include an input port 630 (e.g., a universal serial bus (USB) port), which may be communicatively connected to the LIDAR device 640 to receive sensor data to be buffered and operated on for performing pose estimation. In alternative embodiments, other serial or parallel ports may be used instead. The elements of the computer device 600 may be coupled to each other via a system bus 612, which may represent one or more buses. In the case of multiple buses, they may be bridged by one or more bus bridges (not shown).

[0062] Each of these elements can perform its conventional functions as known in the art. Specifically, system memory 650 can be used to store working copies and permanent copies of executable code of programming instructions for LIDAR driver 651 and fast scan 653, which can be configured to practice and can include Figure 4 The course 400 and Figure 5 The system memory 650 may also include programming instructions that implement an operating system and one or more other applications. The programming instructions may include assembler instructions supported by the processor(s) 615 or a high-level language (such as, for example, C) that can be compiled into such instructions.

[0063] The executable code of programming instructions or the bitstream for configuring the hardware accelerator 620 may be placed into the system memory 650 and / or the SoC memory 613 in the factory or in the field, for example, by a distribution medium (not shown) such as a compact disk (CD), a USB storage device, etc., or by an input port 630 (from a distributed server (not shown)). The number, capabilities, and / or capacity of these elements may vary depending on the intended use of the example computer device 600 (i.e., in which mobile autonomous device the example computer device 600 is embedded). In other respects, the composition of these elements is known and accordingly will not be further described.

[0064] Figure 7 The figures illustrate an example computer-readable storage medium having instructions according to various embodiments, which are configured to implement and / or practice the above-described Figure 4 The course 400 and Figure 5 As shown, the computer-readable storage medium 702 may include executable code of a plurality of programming instructions or bitstreams 704. The executable code of the programming instructions (or bitstream) 704 may be configured to enable a device (e.g., the computer device 600) to perform, respectively, in response to the execution of the executable code / programming instructions (or the operation of the FPGA 620). Figure 4 The course 400 and Figure 5 In an alternative embodiment, the executable code / programming instructions / bitstream 704 may alternatively be provided on a plurality of non-transitory computer-readable storage media 702. In an embodiment, the computer-readable storage media 702 may be non-transitory. In yet other embodiments, the executable code / programming instructions 704 may be encoded in a transient computer-readable medium such as a signal.

[0065] Note that the disclosed fast scanning embodiment has been implemented in a robotic platform with RPLidar A2 (a low-end LIDAR product from Slamtec). It increases the frequency of pose estimation by 12.5 times (from 10Hz to 125Hz), thereby enhancing the robustness of pose estimation. It should also be noted that a robot or other autonomous device is more likely to get lost (e.g., its pose cannot be tracked by a SLAM algorithm) when rotating rather than moving forward. To address this issue, the maximum angular velocity at which the robot can rotate without getting lost has been tested for two different robotic platforms in several different scenarios. It has been found that using fast scanning, the robot can rotate about twice as fast.

[0066] Note further that in alternative embodiments, a standard computer (e.g., a desktop computer or a The Next Unit of Computing (NUC) is used to implement the fast scanning embodiment. The fast scanning embodiment as disclosed has also been used in a robotic platform with RPLidar A2. In this example implementation, the frequency of the pose estimation cycle is increased by a factor of 7 to approximately 70 Hz compared to the 125 Hz of the FPGA due to the more limited computing power.

[0067] In an embodiment, programming instructions to implement the fast scan of the present disclosure may be included in a device driver / SDK from a LIDAR manufacturer.

[0068] Illustrative examples of the techniques disclosed herein are provided below. Embodiments of these techniques may include any one or more of the examples described below and any combination thereof.

[0069] Example

[0070] Example 1 may include an apparatus for determining a current pose of a mobile autonomous device, comprising: an interface circuit for receiving detection and ranging data output by a light detection and ranging (LIDAR) sensor that nominally scans per time period T and provides D degrees of detection and ranging data in a continuous plurality of quanta, each quantum covering a portion of the D degree scan; and a pose estimation circuit coupled to the interface circuit for determining and providing the current pose of the mobile autonomous device for each partial time period t independently of when the LIDAR sensor actually completes each scan. The apparatus may be disposed on the mobile autonomous device.

[0071] Example 2 may include the apparatus of Example 1 and / or other examples herein, wherein each data quantum covers d degrees, and wherein D is divisible by d.

[0072] Example 3 may include the apparatus of Example 1 and / or other examples herein, wherein D is equal to 360 degrees.

[0073] Example 4 may include the apparatus of Example 1 and / or other examples herein, wherein T is divisible by t.

[0074] Example 5 may include the apparatus of Example 3 and / or other examples herein, wherein d is 90 degrees and t is ¼ of T.

[0075] Example 6 may include the apparatus of Example 1 and / or other examples herein, wherein d is 1 degree and t is 1 / 360 of T.

[0076] Example 7 may include the apparatus of Example 1 and / or other examples herein, further comprising a buffer for storing detection and ranging data of at least one scan.

[0077] Example 8 may include the apparatus of any of Examples 1-7 and / or other examples herein, wherein the pose estimation circuit accesses the buffer every time period t to obtain detection and ranging data for one scan.

[0078] Example 9 may include the apparatus of Example 7 and / or other examples herein, further comprising a driver accessing the buffer and providing detection and ranging data of one scan to the pose estimation circuit every time period t.

[0079] Example 10 may include the apparatus of Example 9 and / or other examples herein, wherein the LIDAR sensor issues a synchronization signal to indicate when it has actually completed a scan, and wherein the driver provides detection and ranging data for one scan to the pose estimation circuit every time period t, independent of the synchronization signal.

[0080] Example 11 may include the apparatus of Example 1 and / or other examples herein, wherein, to determine and provide a current pose, the pose estimation circuit transforms the detection and ranging data into a point cloud and determines the current pose by matching the point cloud with a reference map.

[0081] Example 12 may include the apparatus of Example 11 and / or other examples herein, wherein the pose estimation circuit includes a processor and a detection and ranging data transformation program operated by the processor to transform the detection and ranging data into a point cloud.

[0082] Example 13 may include the apparatus of Example 12 and / or other examples herein, wherein the pose estimation circuit includes a hardware accelerator for determining the current pose by matching the point cloud to a reference map.

[0083] Example 14 may include the apparatus of Example 13 and / or other examples herein, wherein the hardware accelerator includes an FPGA.

[0084] Example 15 may include the apparatus of any of Examples 1-6 and / or any other examples herein, wherein the mobile autonomous device is a robot.

[0085] Example 16 may include the apparatus of any of Examples 1-6 and / or any other examples herein, wherein the mobile autonomous device is an autonomous self-driving car.

[0086] Example 17 may include the apparatus of any of Examples 1-6 and / or any other examples herein, wherein the mobile autonomous device is an unmanned aerial vehicle (UAV).

[0087] Example 18 may include a method for determining a current posture of the mobile autonomous device performed by a computer device on a mobile autonomous device, the method comprising: receiving, by the computer device, detection and ranging data output by a light detection and ranging (LIDAR) sensor that nominally scans per time period T and provides D degrees of detection and ranging data in a continuous plurality of quanta, each quantum covering a portion of the D degree scan; and determining, by the computer device, providing the current posture of the mobile autonomous device for each partial time period t independently of when the LIDAR sensor actually completes each scan.

[0088] Example 19 may include the method of Example 18 and / or other examples herein, wherein each data quantum covers d degrees, where D is divisible by d, and where T is divisible by t.

[0089] Example 20 may include the method of Example 18 and / or other examples herein, wherein D is equal to 360 degrees and d is 90 degrees.

[0090] Example 21 may include the method of any of Examples 18-20 and / or other examples herein, further comprising storing, by a computer device, detection and ranging data of at least one scan in a buffer.

[0091] Example 22 may include the method of Example 21 and / or other examples herein, further comprising accessing, by a computer device, the buffer every time period t to obtain detection and ranging data for one scan.

[0092] Example 23 may include the method of Example 18 and / or other examples herein, wherein determining and providing the current posture includes transforming the detection and ranging data into a point cloud by a computer device, and determining the current posture by matching the point cloud with a reference map.

[0093] Example 24 may include the method of Example 23 and / or other examples herein, wherein the computer device includes a hardware accelerator, and wherein determining the current pose includes matching the point cloud to a reference map via the hardware accelerator.

[0094] Example 25 may include the method of any of Examples 18-20 and / or any other examples herein, wherein the mobile autonomous device is a robot.

[0095] Example 26 may include the method of any of Examples 18-20 and / or any other examples herein, wherein the mobile autonomous device is an autonomous self-driving car.

[0096] Example 27 may include the method of any of Examples 18-20 and / or any other examples herein, wherein the mobile autonomous device is an unmanned aerial vehicle (UAV).

[0097] Example 28 may include the methods of Examples 18-20 and / or other examples herein, wherein the LIDAR sensor emits a synchronization signal to indicate when it has actually completed a scan, and wherein the computer device determines and provides the current pose of the mobile autonomous device for each partial time period t independently of the synchronization signal.

[0098] Example 29 may include the method of Example 24 and / or other examples herein, wherein the hardware accelerator includes an FPGA.

[0099] Example 30 may include one or more non-transitory computer-readable storage media including a plurality of instructions that, in response to being executed, cause a computing device to: receive, by the computer device, detection and ranging data output by a light detection and ranging (LIDAR) sensor that nominally scans per time period T and provides D degrees of detection and ranging data in a continuous plurality of quanta, each quantum covering a portion of the D degree scan; and determine, by the computer device, and provide a current pose of the mobile autonomous device for each partial time period t, independent of when the LIDAR sensor actually completes each scan.

[0100] Example 31 may include one or more non-transitory computer-readable storage media of Example 30 and / or other examples herein, wherein each data quantum covers d degrees, where D is divisible by d, and where T is divisible by t.

[0101] Example 32 may include one or more non-transitory computer-readable storage media of Example 30 and / or other examples herein, wherein D is equal to 360 degrees and d is 90 degrees.

[0102] Example 33 may include one or more non-transitory computer-readable storage media of any of Examples 30-32 and / or other examples herein, further comprising storing, by the computing device, detection and ranging data of at least one scan in a buffer.

[0103] Example 34 may include one or more non-transitory computer-readable storage media of Example 33 and / or other examples herein, further comprising accessing, by a computing device, the buffer in each time period t to obtain detection and ranging data for one scan.

[0104] Example 35 may include one or more non-transitory computer-readable storage media of Example 30 and / or other examples herein, wherein determining and providing the current posture includes: transforming the detection and ranging data into a point cloud by a computing device, and determining the current posture by matching the point cloud with a reference map.

[0105] Example 36 may include one or more non-transitory computer-readable storage media of Example 35 and / or other examples herein, wherein the computing device includes a hardware accelerator, and wherein determining the current pose includes matching the point cloud to a reference map via the hardware accelerator.

[0106] Example 37 may include one or more non-transitory computer-readable storage media of Example 36 and / or other examples herein, wherein the hardware accelerator includes an FPGA.

[0107] Example 38 may include the one or more non-transitory computer-readable storage media of any of Examples 30-32 and / or any other examples herein, wherein the mobile autonomous device is a robot.

[0108] Example 39 may include the one or more non-transitory computer-readable storage media of any of Examples 30-32 and / or any other examples herein, wherein the mobile autonomous device is an autonomous self-driving car.

[0109] Example 40 may include the one or more non-transitory computer-readable storage media of any of Examples 30-32 and / or any other examples herein, wherein the mobile autonomous device is an unmanned aerial vehicle (UAV).

[0110] Example 41 may include one or more non-transitory computer-readable storage media of Examples 30-32 and / or other examples herein, wherein the LIDAR sensor emits a synchronization signal to indicate when it has actually completed a scan, and wherein the computing device determines and provides the current pose of the mobile autonomous device for each partial time period t independently of the synchronization signal.

[0111] Example 42 may include an apparatus for computing disposed in a mobile autonomous device, comprising: means for receiving detection and ranging data output by a light detection and ranging (LIDAR) sensor that nominally scans per time period T and provides D degrees of detection and ranging data in a continuous plurality of quanta, each quantum covering a portion of the D degree scan; and means for determining and providing a current pose of the mobile autonomous device per partial time period t, independent of when the LIDAR sensor actually completes each scan, the means being coupled to the means for receiving.

[0112] Example 43 may include an apparatus for computing of Example 41 and / or other examples herein, wherein each data quantum covers d degrees, and wherein D is divisible by d.

[0113] Example 44 may include the apparatus for computing of Example 42 and / or other examples herein, wherein D is equal to 360 degrees.

[0114] Example 45 may include the apparatus for computing of Example 43 and / or other examples herein, wherein T is divisible by t.

[0115] Example 46 may include the apparatus for calculating of Example 44 and / or other examples herein, wherein d is 1 degree and t is 1 / 4 of T.

[0116] Example 47 may include the apparatus for calculating of Example 44 and / or other examples herein, wherein d is 1 degree and t is 1 / 360 of T.

[0117] Example 48 may include the device for computing of Examples 42-48 and / or other examples herein, further comprising a memory device for storing detection and ranging data of at least one scan.

[0118] Example 49 may include the apparatus for computing of Example 48 and / or other examples herein, wherein the means for determining and providing is for accessing the memory device for each time period t to obtain detection and ranging data for one scan.

[0119] Example 50 may include the apparatus for computing of Example 49 and / or other examples herein, further comprising means for accessing, for accessing a memory device, and providing detection and ranging data of one scan per time period t to the means for determining and providing.

[0120] Example 51 may include an apparatus for computing of Example 50 and / or other examples herein, wherein the LIDAR sensor sends a synchronization signal to indicate when it has actually completed a scan, and wherein the means for accessing provides detection and ranging data for one scan to the means for determining and providing independently of the synchronization signal every time period t.

[0121] Example 52 may include an apparatus for computing of Example 42 and / or other examples herein, wherein, to determine and provide a current posture, the apparatus for determining and providing transforms the detection and ranging data into a point cloud, and determines the current posture by matching the point cloud with a reference map.

[0122] Example 53 may include an apparatus for computing of Example 42 and / or other examples herein, wherein the means for determining and providing comprises a processing device, and a detection and ranging data transformation program executed by the processing device to transform the detection and ranging data into a point cloud.

[0123] Example 54 may include the apparatus for computing of Example 53 and / or other examples herein, wherein the means for determining and providing includes a hardware acceleration means for determining the current pose by matching the point cloud with a reference map.

[0124] Example 55 may include the apparatus for computing of Example 54 and / or other examples herein, wherein the hardware acceleration device includes an FPGA.

[0125] Example 56 may include an apparatus for computing of any of Examples 42-47 and / or any other examples herein, wherein the mobile autonomous device is a robot.

[0126] Example 57 may include the apparatus for computing of any of Examples 42-47 and / or any other examples herein, wherein the mobile autonomous device is an autonomous self-driving car.

[0127] Example 58 may include an apparatus for computing of any of Examples 42-47 and / or any other examples herein, wherein the mobile autonomous device is an unmanned aerial vehicle (UAV).

Claims

1. A device for determining a current posture of a mobile autonomous device, the device comprising: an interface circuit for receiving detection and ranging data output by a light detection and ranging LIDAR sensor, the light detection and ranging LIDAR sensor scanning every time period T and continuously providing detection and ranging data of D degrees as a plurality of data units, each of the plurality of data units covering a portion of the D degree scan; as well as a pose estimation circuit coupled to the interface circuit for determining and providing the current pose of the mobile autonomous device for each partial time period t independently of the time when the LIDAR sensor actually completes each scan, The device is arranged on the mobile autonomous device.

2. The apparatus of claim 1, wherein each data unit covers d degrees, and wherein D is divisible by d.

3. The device of any one of claims 1 or 2, wherein D is equal to 360 degrees.

4. The device of any one of claims 1 or 2, wherein T is divisible by t.

5. The device of claim 3, wherein d is 90 degrees and t is 1 / 4 of T.

6. The apparatus of claim 3, wherein d is 1 degree and t is 1 / 360 of T.

7. The apparatus as claimed in any one of claims 1, 2, 5, and 6, further comprising a buffer for storing detection and ranging data of at least one scan.

8. The apparatus of claim 7, wherein the posture estimation circuit accesses the buffer every time period t to obtain detection and ranging data of one scan.

9. The apparatus of claim 8, further comprising a driver that accesses the buffer every time period t and provides detection and ranging data of one scan to the pose estimation circuit.

10. The apparatus of claim 9, wherein the LIDAR sensor sends out a synchronization signal to indicate when it has actually completed a scan, and wherein the driver provides detection and ranging data for one scan to the pose estimation circuit every time period t independently of the synchronization signal.

11. The apparatus of claim 1, wherein to determine and provide the current pose, the pose estimation circuit transforms the detection and ranging data into a point cloud and determines the current pose by matching the point cloud with a reference map.

12. The apparatus of claim 11, wherein the pose estimation circuit comprises a processor and a detection and odometry data conversion program operated by the processor to convert the detection and odometry data into the point cloud.

13. The apparatus of claim 12, wherein the pose estimation circuit comprises a hardware accelerator for determining the current pose by matching the point cloud with the reference map.

14. The apparatus of claim 13, wherein the hardware accelerator comprises a FPGA.

15. The device of any one of claims 1, 2, 5, and 6, wherein the mobile autonomous device is a robot.

16. The device of any one of claims 1, 2, 5, and 6, wherein the mobile autonomous device is an autonomous self-driving car.

17. The device of any one of claims 1, 2, 5, and 6, wherein the mobile autonomous device is an unmanned aerial vehicle (UAV).

18. A method for execution by a computer device on a mobile autonomous device for determining a current posture of the mobile autonomous device, the method comprising: receiving, by the computer device, detection and ranging data output by a light detection and ranging LIDAR sensor, the light detection and ranging LIDAR sensor scanning every time period T and continuously providing D degrees of detection and ranging data as a set of data units, each data unit in the set of data units covering a portion of the D degree scan; as well as The current pose of the mobile autonomous device is determined and provided by the computer device for each partial period t independently of the time when each scan is actually completed by the LIDAR sensor.

19. The method of claim 18, wherein each data unit covers a degree d, wherein D is divisible by d, and wherein T is divisible by t.

20. The method of claim 18, wherein D is equal to 360 degrees and d is 90 degrees.

21. The method of claim 18, further comprising storing, by the computer device, detection and ranging data of at least one scan in a buffer, and accessing, by the computer device, the buffer every time period t to obtain detection and ranging data of one scan.

22. The method of claim 18, wherein determining and providing the current pose comprises transforming, by the computer device, the detection and ranging data into a point cloud, and determining the current pose by matching the point cloud with a reference map.

23. The method of claim 22, wherein the computer device comprises a hardware accelerator, and wherein determining the current pose comprises matching the point cloud to the reference map via the hardware accelerator.

24. The method of claim 18, wherein the mobile autonomous device is at least one of a robot, an autonomous self-driving car, or an unmanned aerial vehicle (UAV).

25. A machine-readable medium comprising code which, when executed, causes a machine to perform the method of any one of claims 18 to 24.

Images