Self-position estimation device, automatic driving control device, and self-position estimation method

Abstract

To provide a computing device that can appropriately process point cloud data within a certain time period. [Solution] The self-position estimation device is a device for estimating the self-position of a vehicle, and comprises: a first calculation unit that estimates the self-position of the vehicle based on point cloud data showing positional information around the vehicle and a point cloud map representing the environment in which the vehicle is traveling in a point cloud; and a second calculation unit that corrects the self-position based on the vehicle's speed and angular velocity and the self-position estimated by the first calculation unit, wherein the first calculation unit and the second calculation unit are activated separately by triggers caused by external events.

Description

[Technical Field]

[0001] The present disclosure relates to a self-localization apparatus, an automatic driving control apparatus, and a self-localization method. [Background Art]

[0002] In autonomous driving of automobiles, robots, and the like, a technique for estimating self-position using point cloud data acquired from LiDAR or the like is utilized. Since the amount of computation for processing a large amount of point cloud data in real time is enormous, improvements in arithmetic processing performance have been achieved by parallel computation or the like (for example, Patent Document 1). [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] International Publication No. 2019 / 078292 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] However, in autonomous driving of automobiles, robots, and the like, a delay in processing point cloud data leads to a delay in driving judgment, so it is necessary to appropriately process point cloud data within a fixed period of time.

[0005] An object of the present disclosure is to provide a self-localization apparatus, an automatic driving control apparatus, and a self-localization method that can appropriately process data within a fixed period of time. [Means for Solving the Problem]

[0006] In order to solve the above problem, the present disclosure proposes the following means. A self-position estimation device according to a first aspect of the present disclosure is a device for estimating the self-position of a vehicle, comprising: a first calculation unit that estimates the self-position of the vehicle based on point cloud data indicating positional information around the vehicle and a point cloud map representing the environment in which the vehicle is traveling in a point cloud; and a second calculation unit that corrects the self-position based on the vehicle's speed and angular velocity and the self-position estimated by the first calculation unit, wherein the first calculation unit and the second calculation unit are activated separately by triggers caused by external events. [Effects of the Invention]

[0007] The self-position estimation device, automatic driving control device, and self-position estimation method of this disclosure can appropriately process point cloud data within a certain period of time. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram of the arithmetic system according to the first embodiment. [Figure 2] This is a block diagram of the arithmetic processor of the same computing system. [Figure 3] This is a block diagram of the arithmetic unit of the same arithmetic processor. [Figure 4] This diagram shows the threads executed by the same processing unit. [Figure 5] This is a timing chart of the processor elements of the same arithmetic unit. [Figure 6] This is a block diagram of the integer arithmetic processor of the same computing system. [Figure 7] This is a timing chart showing the operation of the calculation circuit of the calculation system. [Figure 8] This is a block diagram of a self-position estimation device according to the second embodiment. [Figure 9] This is a block diagram of the external memory. [Figure 10] This is a flowchart showing the operation of the self-localization device. [Figure 11]It is a diagram showing pipeline operations of a first arithmetic unit and a second arithmetic unit. [Figure 12] It is a block diagram of a vehicle equipped with an automatic driving control apparatus 3000 according to a third embodiment. DETAILED DESCRIPTION OF EMBODIMENTS

[0009] (First Embodiment) An arithmetic system 1000 according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 7.

[0010] [Arithmetic System 1000] FIG. 1 is a block diagram of the arithmetic system 1000. The arithmetic system 1000 is a system that processes three-dimensional data acquired from LiDAR or the like in real time. The arithmetic system 1000 includes a CPU 700, an external memory 600, and an arithmetic circuit 100. The CPU 700, the external memory 600, and the arithmetic circuit 100 are connected via an external bus EB.

[0011] The CPU 700 includes one or more processors, and is a host that controls the entire arithmetic system 1000. The functions of the CPU 700 are realized by a program that controls the arithmetic system 1000.

[0012] The external memory 600 includes a volatile memory such as DRAM and a memory controller. The external memory 600 stores programs, three-dimensional data, intermediate operation data, and the like.

[0013] The arithmetic circuit 100 is an arithmetic circuit that processes three-dimensional data in real time. The arithmetic circuit 100 includes an arithmetic processor 200 and an integer arithmetic processor 300 that assists the arithmetic processor 200. Note that the arithmetic circuit 100 does not need to include the integer arithmetic processor 300.

[0014] FIG. 2 is a block diagram of the arithmetic processor 200. The arithmetic processor 200 is a SIMT (Single Instruction, Multiple Threads) processor including a floating-point arithmetic circuit specialized for multithreaded software. The arithmetic processor 200 differs from conventional processors in its operation: it is activated by a trigger caused by an external event, and after completing processing of a plurality of assigned threads, it waits until being activated next. The arithmetic processor 200 includes a management unit 210, an arithmetic unit 220, a memory 250, a DMAC 260, and a bus interface 270. The arithmetic processor 200 illustrated in FIG. 2 includes two arithmetic units 220 (a first arithmetic unit 220-1 and a second arithmetic unit 220-2). Note that the number of arithmetic units 220 included in the arithmetic processor 200 is not limited thereto.

[0015] The plurality of arithmetic units 220 included in the arithmetic processor 200 may have different operating frequencies or operating voltages from each other.

[0016] The management unit 210 dispatches workloads to the arithmetic units 220 based on a trigger caused by an external event (such as an instruction from the CPU 700, a timer event, an interrupt or synchronization signal from an external device such as a sensor). Specifically, the management unit 210 dispatches Workgroups (WG), which are groups of a plurality of threads, to the arithmetic units 220. The management unit 210 notifies the CPU 700 of the operating status of the arithmetic unit 220.

[0017] The management unit 210 may manage rules that associate external events, the arithmetic units 220, and workloads to be dispatched (such as program counters associated with Workgroups) using a rewritable rule table, and perform workload dispatch based on this rule table. The rule table may have an initial value as hardware of the management unit 210, may be rewritten by the CPU 700 before or during execution of a workload by the arithmetic processor 200, or may be rewritten by the arithmetic processor 200 itself.

[0018] Figure 3 is a block diagram of the arithmetic unit 220. The Computing Unit (CU) 220 is a circuit that executes multithreading. The Computing Unit 220 includes a thread management unit 221, a control register 222, a load / store unit 223, a branch management unit 224, and a processor element (PE) 230.

[0019] The thread management unit 221 manages the threads executed by the arithmetic unit 220. The thread management unit 221 assigns threads to the processor element 230 and executes them in response to requests for thread execution from the management unit 210 based on triggers caused by external events.

[0020] The control register 222 stores variables used to control the arithmetic unit 220. The control register 222 also contains the program counter for threads executed on the arithmetic unit 220.

[0021] The load / store unit 223 centrally manages all memory access instructions issued by the arithmetic unit 220. The load / store unit 223 also manages the memory access status of threads executed by the arithmetic unit 220.

[0022] The branch management unit 224 manages the control flow context. When a branch instruction results in different control flows for some threads, the branch management unit 224 autonomously manages the branching (divergence) and recorvergence (realignment) of the control flow context. The control flow context (hereinafter also referred to as "context") includes the program counter and the number of function calls (call stack depth). As described later, one or more threads (hereinafter also referred to as "thread group") that share a single context are executed indivisibly. When a branch instruction causes a portion of the thread group to branch off and have different contexts, the branch management unit 224 separately manages the contexts of the branched thread group. Furthermore, the branch management unit 224 integrates the contexts at the realignment point where the contexts of the branched thread group become the same. Because threads sharing the same context are executed simultaneously as much as possible, even searches with many branches and complex algorithms can be executed efficiently.

[0023] The processor element (PE) 230 is an arithmetic unit that includes floating-point arithmetic circuits such as a single-precision floating-point exponentiation and addition unit, a single-precision floating-point division / square root unit, and other single-precision floating-point data processing units. The arithmetic unit 220 illustrated in Figures 3 and 4 has eight processor elements 230. However, the number of processor elements 230 in the arithmetic unit 220 is not limited to this.

[0024] Figure 4 shows the threads executed by the arithmetic unit 220. The processing unit 220 manages Workgroups (WGs), which are groups of multiple threads. The processing unit 220 manages multiple Wavefronts, which are obtained by dividing a Workgroup, as execution units. The Workgroup exemplified in Figure 4 has m+1 Wavefronts (m≧0).

[0025] A Wavefront (WF, thread set) is a set of threads that run in parallel on the arithmetic unit 220. Each thread belonging to a Wavefront corresponds to one or more contexts (including program counters) that correspond to the Wavefront. The program counters of each control flow context are updated independently. When a thread belonging to a Wavefront is executed, one context corresponding to the thread being executed is selected, and one or more threads (thread groups) that share the selected context are executed indivisibly. A group of threads dispatched to multiple processor elements 230 in the same clock cycle is called a Wave. The Wavefront illustrated in Figure 4 has n+1 Waves (n≧0).

[0026] Figure 5 is a timing chart for the processor element 230. The thread management unit 221 assigns multiple waves belonging to the wavefront to multiple processor elements 230. The multiple processor elements 230 execute the same instruction(s) identified by the context n+1 times consecutively over multiple clock cycles, thereby separating and executing the same instruction for the assigned waves over different clock cycles over time. As shown in Figure 5, the same instruction for threads belonging to the same wave (Thread[0] and Thread[1]) is executed in the same clock cycle. The same instruction for threads belonging to different waves (Thread[0] and Thread[8]) is executed in different clock cycles.

[0027] For example, suppose the arithmetic unit 220 has eight processor elements 230 and handles eight wavefronts (m=7) that assign eight waves (n=7) to the eight processor elements 230. In this case, the Workgroup (WG) contains a total of 512 threads.

[0028] The arithmetic processor 200 may have multiple arithmetic units 220, each comprising a different number of processor elements 230.

[0029] The thread management unit 221 selects an executable Wavefront from the Workgroup and chooses one context corresponding to the selected Wavefront as the target for execution. The thread management unit 221 fetches and decodes the instruction indicated by the program counter contained in the selected context, and then dispatches the threads that share the selected context to the processor element 230. When the execution of one instruction in the Wavefront is completed, the branch management unit 224 distributes and merges the control flow contexts of the threads in the Wavefront and notifies the thread management unit 221 of the next program counter of the Wavefront.

[0030] Memory 250 is a rewritable memory such as volatile memory composed of, for example, SRAM (Static RAM). Memory 250 may also include a register file. Multiple arithmetic units 220 may partially share each area within memory 250, or they may use it exclusively. By sharing a portion of memory 250, multiple arithmetic units 220 can transfer data to each other at high speed without going through external memory 600. Memory 250 may also have a cache function.

[0031] The DMAC260 is connected to the external bus EB via the bus interface 270 and performs data transfer between the external memory 600 and memory 250, etc.

[0032] The bus interface 270 relays access from the CPU 700 to the management unit 210 and other devices. The bus interface 270 also relays access from the management unit 210 and DMAC 260 to the CPU 700 and external devices such as external memory 600.

[0033] Figure 6 is a block diagram of the integer arithmetic processor 300. The integer arithmetic processor 300 is a SIMT (Single Instruction, Multiple Threads) processor specialized for multithreaded software that does not require floating-point arithmetic. The integer arithmetic processor 300 comprises a management unit 210, integer arithmetic units 320, memory 250, DMAC 260, and bus interface 270. The integer arithmetic processor 300 illustrated in Figure 6 has four integer arithmetic units 320 (first integer arithmetic unit 320-1, second integer arithmetic unit 320-2, third integer arithmetic unit 320-3, and fourth integer arithmetic unit 320-4). However, the number of integer arithmetic units 320 in the integer arithmetic processor 300 is not limited to this.

[0034] The integer arithmetic unit 320 has the same configuration as the arithmetic unit 220, but instead of multiple processor elements 230, it has multiple integer processor elements. Each integer processor element is an arithmetic unit that includes an integer arithmetic circuit.

[0035] The integer arithmetic processor 300 is capable of executing at least a portion of the instruction set that the arithmetic processor 200 can execute. The integer arithmetic processor 300 reduces the workload of the arithmetic processor 200 by executing threads that do not require floating-point arithmetic on behalf of the arithmetic processor 200.

[0036] The integer arithmetic processor 300 should ideally have a higher degree of parallelism (such as the number of integer processor elements) than the arithmetic processor 200, so that it can perform many integer operations.

[0037] Next, the operation of the arithmetic circuit 100 will be explained. Figure 7 is a timing chart showing the operation of the arithmetic circuit 100.

[0038] The management unit 210 is activated by external events such as timer events, interrupts from external devices like sensors, or synchronization signals, and dispatches workloads to the arithmetic units 220. The management unit 210 can activate the first arithmetic unit 220-1 and the second arithmetic unit 220-2 with different triggers. For example, the management unit 210 dispatches multiple threads to the first arithmetic unit 220-1 to perform process A based on a first trigger G1 that occurs with period T1. The management unit 210 dispatches multiple threads to the second arithmetic unit 220-2 to perform process B based on a second trigger G2 that occurs with a different period T2 than period T1. By assigning processes with different periods, processing amounts, and importance to separate arithmetic units 220, the management unit 210 can prevent one process from affecting another and ensure that processing is completed within a certain time. Note that period T2 may be the same as period T1. Note that when the trigger is generated by an interrupt from an external device such as a sensor or a synchronization signal, the trigger occurs aperiodically rather than periodically.

[0039] Since the management unit 210 can be activated by triggers caused by external events such as sensors and timers, it can start processing on the arithmetic unit 220 without requiring intervention from the CPU 700, thus enabling the construction of a highly responsive system. Furthermore, even when the processing load on the CPU 700 is high, it can start processing on the arithmetic unit 220 more appropriately at predetermined intervals. In addition, the management unit 210 can definitively dispatch predetermined workloads to predetermined arithmetic units 220 based on pre-set rules. Therefore, predetermined processing can be executed more reliably on the same arithmetic unit 220 while maintaining the same punctuality, without being affected by the system load status or operating conditions such as frequency at any given time.

[0040] By assigning operations that do not require floating-point arithmetic to the integer arithmetic unit 320 of the integer arithmetic processor 300, the arithmetic unit 220 of the arithmetic processor 200 can efficiently perform floating-point arithmetic.

[0041] According to the arithmetic system 1000 of this embodiment, in addition to the CPU 700, it is equipped with an arithmetic circuit 100 that processes three-dimensional data, etc., so that it can process three-dimensional data etc. appropriately within a certain time. The arithmetic system 1000 can configure the number of arithmetic units 220 and the number of processor elements 230, etc., according to the three-dimensional data to be processed.

[0042] Although the first embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope of the gist of the present invention. Furthermore, the components shown in the above-described embodiment and the following modifications can be combined as appropriate.

[0043] (Second embodiment) A self-position estimation device 2000 according to the second embodiment of the present invention will be described with reference to Figures 8 to 11. In the following description, components that are common to those already described will be denoted by the same reference numerals, and redundant descriptions will be omitted.

[0044] Figure 8 is a block diagram of the self-position estimation device 2000. The self-position estimation device 2000 is a device that estimates its own position using three-dimensional data acquired from LiDAR or the like. The self-position estimation device 2000 is mounted on vehicles or robots. The self-position estimation device 2000 comprises a computing system 1000, a LiDAR 810, a camera 820, a radar 830, a GNSS 840, and an IMU 850. At least one of the LiDAR 810, camera 820, radar 830, GNSS 840, and IMU 850 is also called a sensor. The self-position estimation device 2000 estimates vehicle information such as position, attitude, and speed on a map by integrating sensor, point cloud map, and vehicle information. The self-position estimation device 2000 may include a timer that generates timer events at predetermined intervals.

[0045] The LiDAR810 is a sensor that detects the distance to surrounding objects by irradiating a laser beam around a vehicle and measuring the time it takes for the laser beam to reflect off surrounding objects and return. Based on the direction from which the laser beam was irradiated and the measured distance, highly accurate positional information about surrounding objects can be obtained. The process of converting the data acquired by the LiDAR810, consisting of direction and distance, into a data format that represents position in three-dimensional spatial coordinates may be performed by the processing unit of the LiDAR810 or by the device that receives the data from the LiDAR810. The individual positional information obtained in this way represents a point in three-dimensional spatial coordinates relative to the LiDAR810, and this collection of point data is called point cloud data. Each point in the point cloud data may be represented based on three numerical values ​​corresponding to the X, Y, and Z axes, for example. Furthermore, each point may include values ​​such as the reflectivity of the laser beam acquired by the LiDAR810 and a timestamp indicating the measurement time.

[0046] Camera 820 is a sensor that uses an image sensor made of semiconductors or similar materials to periodically photograph the area around the vehicle and acquire image data. The image sensor integrates millions to tens of millions of light-receiving elements, depending on the number of pixels, such as 2K or 4K, to acquire high-resolution images at once. It has the characteristic of high resolution in the direction perpendicular to the optical axis and can also acquire color information. The acquired image data can be used for detecting and recognizing objects around the vehicle.

[0047] The Radar 830 is a sensor that detects the distance to an object by emitting radio waves in the millimeter wave band and measuring the time it takes for the radio waves to reflect off the object and return. By utilizing the Doppler effect of radio waves, it can be used to detect the presence and speed of distant objects. By combining multiple radio waves, it can also generate a low-resolution two-dimensional image.

[0048] The GNSS840 is a sensor that receives signals from multiple GNSS satellites to calculate global positional information about a vehicle's latitude and longitude. The acquired positional information can be used, for example, as initial values ​​in self-position estimation processing.

[0049] The IMU850 is a sensor that detects the linear acceleration and angular velocity of a vehicle around its axis of rotation using an accelerometer and angular velocity meter. The acquired acceleration and angular velocity information can be used to estimate the vehicle's motion state, etc.

[0050] The self-position estimation device 2000 may have other sensors, such as ultrasonic sensors. Furthermore, the self-position estimation device 2000 may not have some of the aforementioned sensors. The self-position estimation device 2000 may also be equipped with and use in combination multiple LiDAR 810s to acquire point cloud data representing the vehicle's external environment, or it may use the outputs of sensors with different characteristics. The self-position estimation device 2000 may further include a speed sensor to detect the vehicle's speed.

[0051] Figure 9 is a block diagram of the external memory 600. External memory 600 stores point cloud map data 610, initial pose data 620, and point cloud data 630. These data are transferred to the memory 250 of the arithmetic processor 200 or the memory 250 of the integer arithmetic processor 300 as needed.

[0052] Next, the operation of the self-position estimation device 2000 will be described. Figure 10 is a flowchart of the operation of the self-position estimation device 2000. Note that Figure 10 shows the relationships between processes and does not show the dependencies regarding the triggers for initiating processes. For example, step S130 may start without waiting for the completion of the process in step S120.

[0053] In step S110, the integer arithmetic processor 300 generates point cloud data 630 by filtering the data acquired from the LiDAR 810. The integer arithmetic processor 300 generates point cloud data 630 triggered by the acquisition of data from the LiDAR 810. Since the amount of data in the point cloud data 630 generated by the filtering in step S100 is reduced compared to the data acquired from the LiDAR 810, the computing resources of the entire arithmetic circuit 100 can be effectively utilized in subsequent steps, and the self-localization process can be prevented from falling into a local optimum. The integer arithmetic processor 300 may perform other processing on the data acquired from the LiDAR 810. As an example, the integer arithmetic processor 300 may perform statistical processing such as trimming to limit the processing range, such as removing error point clouds caused by sensor noise, insects, or raindrops, removing less important distant points, and removing data corresponding to the vehicle itself, correcting sensor distortion due to the movement of the vehicle, adjusting the timing between multiple sensors, thinning or adding points, interpolation between point clouds, rearranging the order of data, converting bit precision, and normalizing values. The integer arithmetic processor 300 stores the generated point cloud data 630 in external memory 600. In the process of self-localization using three-dimensional data acquired from LiDAR 810, etc., the amount of data and processing required tends to be greater in the upstream (start) stages of processing. Conversely, the required calculation precision tends to be higher in the downstream stages of processing. Since the integer arithmetic processor 300 is a processor mainly composed of integer arithmetic circuits, it can be constructed with fewer circuits and less power compared to a processor mainly composed of floating-point arithmetic circuits. Therefore, it is easy to incorporate more arithmetic circuits, and thus upstream processing such as filtering can be processed efficiently using more arithmetic circuits.

[0054] In step S120, the first arithmetic unit 220-1 of the arithmetic processor 200 estimates its own position and orientation on the map based on the point cloud map 610 and point cloud data 630. For example, the position may be represented by a combination of three values ​​indicating the magnitude on each of the three coordinate axes in three-dimensional space, namely the X, Y, and Z axes. The orientation may be represented, for example, by a three-dimensional rotation matrix as a quaternion, i.e., by four values: X, Y, Z, and W.

[0055] Specifically, the first processing unit 220-1 constructs a K-dimensional tree in 3D space from the point cloud data 630, uses the constructed K-dimensional tree to search for neighboring regions in the point cloud map 610 for each point in the point cloud data 630, and estimates its own position (its own position and orientation) at a given time using NDT (Normal Distributions Transform) scan matching from the point cloud data 630 and the neighboring regions. By using a K-dimensional tree, point cloud data represented as a sparse data structure in 3D space can be efficiently searched. Furthermore, NDT scan matching can reduce the amount of computation while ensuring positional accuracy in the 3D direction, and can also increase robustness to suppress falling into local optima. High computational performance can be obtained by having each thread of the first processing unit 220 execute the point-by-point processing of each point in the point cloud in parallel, and by efficiently performing matrix operations etc. with a large number of PE230s.

[0056] In step S130, the second arithmetic unit 220-2 of the arithmetic processor 200 estimates a highly reliable position, attitude, and velocity from its own position, attitude, and velocity, as well as acceleration and angular velocity.

[0057] Specifically, the second arithmetic unit 220-2 corrects the self-position information estimated in step S120 using an Extended Kalman Filter (EKF) based on the vehicle's speed and angular velocity calculated using sensors mounted on the vehicle, and the self-position information (position and attitude) estimated in step S120. By using the Extended Kalman Filter, the reliability can be calculated along with the correction process based on the model, enabling higher accuracy in self-position estimation and evaluation of the estimation results. If the reliability of the self-position estimation is determined to be low, it becomes possible to notify the system of an abnormal state. The floating-point calculation precision and high parallel processing performance of the second arithmetic unit 220 enable more appropriate high-precision self-position estimation and reliability determination at short time intervals. The second arithmetic unit 220-2 can perform processing based on self-position reliability monitoring, which is important in autonomous driving, on a regular and timely basis. In step S130, the second arithmetic unit 220-2 may use other sensor outputs, etc., to estimate its own position. For example, the second processing unit 220-2 may use a combination of images acquired by the camera 820 and the output of the GNSS 840.

[0058] Steps S110, S120, and S130 are pipelined in different processing units.

[0059] Figure 11 shows the pipeline operation of the first arithmetic unit 220-1 and the second arithmetic unit 220-2. The first arithmetic unit 220-1 executes NDT scan matching processing triggered by the completion of point cloud data 630 generation by the integer arithmetic processor 300. The first arithmetic unit 220-1 is activated by a non-periodic trigger caused by an event from an external device that includes uncertain factors (data acquisition from LiDAR 810 and completion of point cloud data 630 generation). The second arithmetic unit 220-2 executes extended Kalman filter (EKF) processing at a period of, for example, N frames / second. The second arithmetic unit 220-2 is activated by a periodic trigger caused by a timer event. That is, the second arithmetic unit 220-2 can process the extended Kalman filter (EKF) periodically without waiting for the completion of NDT scan matching processing in the first arithmetic unit 220-1. In this way, the extended Kalman filter, which generally requires a shorter processing period than NDT scan matching and for which timeliness is more important, can be processed appropriately and periodically.

[0060] The second arithmetic unit 220-2 executes an extended Kalman filter (EKF) using the results of the most recent NDT scan matching that is completed when it is started. In the operation illustrated in Figure 11, the second arithmetic unit 220-2 performs the second extended Kalman filter (referred to as "EKF2" in the figure) using the results of the first completed NDT scan matching (referred to as "NDT1" in the figure). When the second arithmetic unit 220-2 performs the third extended Kalman filter (referred to as "EKF3" in the figure), the second NDT scan matching (referred to as "NDT2" in the figure) is not yet complete. Therefore, the second arithmetic unit 220-2 performs the third extended Kalman filter (referred to as "EKF3" in the figure) using the results of the first completed NDT scan matching (referred to as "NDT1" in the figure).

[0061] The second processing unit 220-2 can reliably start the extended Kalman filter (EKF) regardless of the processing status of the first processing unit 220-1, and can complete the execution of the extended Kalman filter (EKF) at a period of N frames / second. Furthermore, since the second processing unit 220-2 can be activated by external events such as timers, it can reliably start the extended Kalman filter (EKF) even when the CPU 700 is under heavy load. Therefore, the estimation of the self-position and the determination of its reliability, which are important for autonomous vehicles, can be performed more appropriately within a defined period. In addition, even for diverse vehicles with different configurations of the installed LiDAR 810, or with different numbers and frequencies of acquired data, the same self-position estimation device 2000 can be used to estimate the self-position within a defined period, regardless of the processing load of the LiDAR 810.

[0062] According to the self-localization device 2000 of this embodiment, in addition to the CPU 700, it is equipped with a calculation circuit 100 for processing point cloud data, etc., so that point cloud data etc. can be processed appropriately within a certain time. The calculation system 1000 can be configured with the number of calculation units 220 and the number of processor elements 230, etc., according to the point cloud data to be processed.

[0063] Although a second embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design modifications and the like are also included within the scope of the gist of the present invention. Furthermore, the components shown in the above-described embodiment and the following modifications can be combined as appropriate.

[0064] Furthermore, the operating frequency of the first arithmetic unit 220-1, which performs the NDT scan matching process, may be higher than the operating frequency of the second arithmetic unit 220-2, which performs the extended Kalman filter (EKF) process. This allows for increased processing power on the NDT scan matching side, which generally has a higher processing load than the extended Kalman filter, while suppressing the increase in power consumption of the second arithmetic unit 220-2.

[0065] Furthermore, the operating frequency and power supply voltage of the first arithmetic unit 220-1, which performs NDT scan matching, and the operating frequency and power supply voltage of the second arithmetic unit 220-2, which performs extended Kalman filter (EKF) processing, may be changed independently during each execution. This allows for appropriate processing of three-dimensional data within a certain time for various processing loads while suppressing increased power consumption.

[0066] Furthermore, if the management unit 210 detects, for example, through a diagnosis using an error detection mechanism with ECC (Error Correction Code) in memory 250, that the second arithmetic unit 220-2, which performs the extended Kalman filter (EKF) processing, is functionally faulty, it may transfer the extended Kalman filter (EKF) processing to the first arithmetic unit 220-1 and have it perform the processing there. This allows for the continuation of minimal self-localization, albeit for a short period of time.

[0067] Furthermore, the first arithmetic unit 220-1, which performs NDT scan matching processing, and the second arithmetic unit 220-2, which performs extended Kalman filter (EKF) processing, may mutually perform functional safety diagnoses. For example, the arithmetic unit 220 may be diagnosed as being functionally sound by periodically accessing memory 250 and control register 222. In addition, the integer arithmetic unit 320 and the arithmetic unit 220 may mutually perform functional safety diagnoses.

[0068] For example, the system may diagnose whether the arithmetic unit 220 is functioning properly by periodically accessing the control register 222 of the arithmetic unit 220 or by collecting error information such as ECC when accessing the memory 250.

[0069] Furthermore, the point cloud data generated by the integer arithmetic processor 300 based on input from the LiDAR 810 may be used not only for self-position estimation in the self-position estimation device 2000, but also for recognition processing of objects around the vehicle using a neural network, thereby increasing the overall processing power of the system.

[0070] Furthermore, the arithmetic processor 200 may be capable of executing a general-purpose instruction set that can represent an operating system. This allows the boot process of the arithmetic circuit 100 to be performed without the intervention of the CPU 700. In addition, by enabling the execution of a general-purpose instruction set, a variety of development environments such as open-source software can be used, thereby improving development efficiency.

[0071] (Third embodiment) An automatic driving control device 3000 according to the third embodiment of the present invention will be described with reference to Figure 12. In the following description, components that are common to those already described will be denoted by the same reference numerals, and redundant descriptions will be omitted.

[0072] Figure 12 is a block diagram of a vehicle equipped with the automatic driving control system 3000. A vehicle equipped with the autonomous driving control device 3000 is an autonomous driving vehicle capable of autonomous driving at autonomous driving level 4 or higher. Here, autonomous driving level 4 is a driving level in which the control device, rather than the driver, can be the primary driver of the driving operation within a limited operational design domain. The operational design domain is defined as the driving environment conditions that are prerequisites for the operation of the autonomous driving system. The vehicle of this embodiment can also exchange information and data with an external management center, etc., via a communication unit (not shown). This information may include various types of information such as the vehicle's location, surrounding conditions including video, the vehicle's operation history, and system information such as the software version. This data may also include video, audio, log data, and software.

[0073] A vehicle equipped with an automatic driving control device 3000 has sensors including a LiDAR 810, a camera 820, a radar 830, a GNSS 840, and an IMU 850. The vehicle also includes the automatic driving control device 3000, a vehicle control device 3100, a drive control device 3200, a braking control device 3300, a steering control device 3400, and a vehicle sensor 3500. The automatic driving control device 3000 has a computing system 1000 that functions as a self-position estimation device 2000.

[0074] The vehicle may also include other control devices such as ECUs (Electronic Control Units), sensors, actuators, mechanical components, and HMIs (Human Machine Interfaces), which are not shown in the diagram. These devices may communicate with each other via various wired and wireless communication methods, such as Ethernet, GSML (Gigabit Multimedia Serial Link), CAN (Controller Area Network), and LIN (Local Interconnect Network). The vehicle may also communicate with the outside world via wired or wireless communication.

[0075] The automatic driving control device 3000 may include, in addition to the computing system 1000 which functions as a self-position estimation device 2000, a recognition device that recognizes the surrounding environment based on a neural network and a planning device that determines a vehicle control plan. The self-position estimation device 2000, which acquires information from each sensor, outputs the self-position estimation result as a calculation result. The recognition device recognizes the surrounding environment based on images acquired by the camera 820 and the outputs of each sensor. For example, the recognition device recognizes the status of traffic lights, road shape, and the presence or absence of parked vehicles. The planning device determines how to control the vehicle based on the self-position estimation result of the self-position estimation device 2000 and the recognition result of the recognition device, and instructs the vehicle control device 3100 on the determined control content. Examples of instructions include commands for controlling steering angle, accelerator and brake amounts, and the timing of each control. It is desirable for the automatic driving control device 3000 to give instructions to the vehicle control device 3100 at predetermined short time cycles for safe driving. In particular, for autonomous vehicles capable of high-speed or high-precision autonomous driving, such as those intended for autonomous driving level 4 or higher, it is desirable to issue control instructions at intervals of at least less than 1 second, and depending on the conditions of the operational design domain, at intervals of, for example, less than 0.1 seconds.

[0076] The vehicle control device 3100 controls the drive control device 3200, the braking control device 3300, and the steering control device 3400 in accordance with instructions from the automatic driving control device 3000 or from the vehicle driver.

[0077] The vehicle control device 3100 can absorb differences in various vehicle control methods by converting abstract commands from the automatic driving control device 3000 into concrete instructions. The vehicle control device 3100 may also transmit vehicle-related information to the automatic driving control device 3000 as needed. Examples of vehicle-related information include tire pressure, braking or acceleration performance, steering angle, fuel level, and the status of lighting and lamps. The automatic driving control device 3000 can use this information to achieve automatic driving control with higher precision. The vehicle control device 3100 includes, for example, an ECU (Electronic Control Unit).

[0078] The drive control device 3200 controls the vehicle's speed and acceleration characteristics. For example, if the vehicle is an electric vehicle, the drive control device 3200 may control the rotational speed and acceleration of the motor that rotates the drive wheels to increase. For example, if the vehicle is an engine vehicle with an internal combustion engine, the drive control device 3200 may control the rotational speed and acceleration of the drive shaft connected to the engine via the transmission by directly or indirectly controlling the amount of fuel injected into the engine. The speed and acceleration characteristics may be directly controlled by the vehicle's motors and actuators, or indirectly controlled by the position of the vehicle's accelerator pedal.

[0079] The braking control device 3300 controls the vehicle's speed and deceleration characteristics. For example, if the vehicle is an electric vehicle, the braking control device 3300 may control the rotational speed and acceleration of the motor that rotates the drive wheels to reduce them. For example, if the vehicle is an engine-powered vehicle, the braking control device 3300 may control the brake actuator that decelerates the rotation of the wheels. The speed and deceleration characteristics may be controlled directly by the vehicle's motors and actuators, or indirectly by the position applied to the vehicle's brake pedal.

[0080] The steering control device 3400 controls the left-right direction of travel characteristics of the vehicle. For example, the steering control device 3400 may control the direction and rotational angular velocity of the steering wheels by controlling the rotation of the vehicle's steering shaft, thereby controlling the left-right motion of the vehicle with respect to the direction of travel.

[0081] The vehicle control device 3100 may also receive state information from the drive control device 3200, the braking control device 3300, and the steering control device 3400, and use that information to perform feedback control to the drive control device 3200, the braking control device 3300, and the steering control device 3400.

[0082] The vehicle sensor 3500 includes a speed sensor for detecting the vehicle's speed. For example, the vehicle sensor 3500 can detect the vehicle's speed by measuring the rotational speed of the vehicle's wheels. The automatic driving control device 3000 can use the speed data acquired from the vehicle sensor 3500 to estimate the vehicle's motion state, etc. Alternatively, the automatic driving control device 3000 may receive the speed data acquired by the vehicle sensor 3500 via the vehicle control device 3100. In addition to the speed sensor, the vehicle sensor 3500 may also include other sensors such as a temperature sensor.

[0083] According to the automatic driving control device 3000 of this embodiment, in addition to the CPU 700, it is equipped with a calculation circuit 100 that processes point cloud data, etc., so that point cloud data etc can be processed appropriately within a certain time, and autonomous driving assuming an autonomous driving level of 4 or higher can be appropriately performed.

[0084] Although a third embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design modifications and the like are also included within the scope of the gist of the present invention. Furthermore, the components shown in the above-described embodiment and the following modifications can be combined as appropriate. [Explanation of Symbols]

[0085] 3000 Automated Driving Control System 2000 Self-position estimation device 1000 computing systems 100 Arithmetic circuit 200 arithmetic processors 210 Management Units 220 Computing Units (CUs) 220-1 First Processing Unit 220-2 Second Processing Unit 221 Thread Management Unit 222 Control Registers 223 Load Store Unit 224 Branch Management Unit 230 Processor Elements (PEs) 250 memory 270 Bus Interface 300 integer arithmetic processors 320 integer arithmetic units 320-1 First Integer Arithmetic Unit 320-2 Second Integer Arithmetic Unit 320-3 Third integer arithmetic unit 320-4 Fourth Integer Unit 600 External memory 610-point cloud map (Map Data) 620 Initial pose data 630 point cloud data 810 LiDAR 820 Camera 830 Radar 840 GNSS 850 IMU EB external bus G1 First Trigger G2 Second Trigger

Claims

1. A device for estimating the vehicle's own position, A first calculation unit estimates the vehicle's own position based on point cloud data showing positional information around the vehicle and a point cloud map representing the environment in which the vehicle travels. A second calculation unit corrects the vehicle's position based on the vehicle's speed and angular velocity, and the vehicle's position estimated by the first calculation unit. Equipped with, The first and second arithmetic units are activated separately by triggers resulting from external events. Self-location estimation device.

2. The first arithmetic unit and the second arithmetic unit are activated by different triggers. The self-position estimation device according to claim 1.

3. When the second arithmetic unit is activated by the trigger and the first arithmetic unit is estimating its own position, it uses the position that has already been estimated by the first arithmetic unit. The self-position estimation device according to claim 1.

4. The self-position used by the second calculation unit is the most recent self-position that has already been estimated by the first calculation unit. The self-position estimation device according to claim 3.

5. The first calculation unit is activated by a non-periodic trigger that occurs upon completion of the generation of the point cloud data. The second arithmetic unit is activated by the periodic trigger caused by a timer event. The self-position estimation device according to claim 1.

6. The first arithmetic unit and the second arithmetic unit include a floating-point arithmetic circuit. The self-position estimation device according to claim 1.

7. The system further comprises an integer arithmetic unit and an integer arithmetic processor that generates the point cloud data based on data acquired from a sensor, The first arithmetic unit, the second arithmetic unit, and the integer arithmetic unit are each activated separately by a trigger caused by the external event. The self-position estimation device according to claim 6.

8. The integer arithmetic unit is activated by a non-periodic trigger that occurs when the acquisition of the data from the sensor is completed. The first calculation unit is activated by a non-periodic trigger that occurs upon completion of the generation of the point cloud data. The second arithmetic unit is activated by the periodic trigger caused by a timer event. The self-position estimation device according to claim 7.

9. A self-position estimation device according to any one of claims 1 to 8, Automatic driving control system.

10. A method for estimating the vehicle's own position, A first calculation step for estimating the vehicle's own position based on point cloud data showing positional information around the vehicle and a point cloud map representing the environment in which the vehicle is traveling. A second calculation step corrects the self-position based on the vehicle's speed and angular velocity and the self-position estimated by the first calculation step. Equipped with, The first calculation step and the second calculation step are triggered separately by an external event. Self-localization method.

11. The first calculation step and the second calculation step are activated by different triggers. The self-localization method according to claim 10.

12. When the second calculation step is initiated by the trigger and the first calculation step is estimating its own position, it uses the self-position that has already been estimated by the first calculation step. The self-localization method according to claim 10.

13. The self-position used in the second calculation step is the most recent self-position that has already been estimated by the first calculation step. The self-localization method according to claim 12.

Images