Automatic map generation device and transport vehicle system

Abstract

To provide a map generation function which does not depend on a vehicle model / the number of vehicles, and a shape of travel environment, further, to eliminate the need of environment measurement of each vehicle model for map creation, and to reduce man hours of the map creation.SOLUTION: According to the present invention, a map automatic generation device includes a three-dimensional environment measurement part for acquiring three-dimensional information of travel environment of transport vehicles, a vehicular map generation part for generating a vehicular map for the transport vehicles for each of the plurality of transport vehicles on the basis of the three-dimensional information, vehicle grade information showing dimensions of the transport vehicles, and performance of an on-vehicle sensor mounted on the transport vehicles and / or information on a mounting position, and a map distribution part for transmitting the generated vehicular map for each of the plurality of transport vehicles.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a map automatic generation device and a transport vehicle system.

Background Art

[0002] With the labor shortage due to the declining birthrate and aging population and the expansion of the e-commerce market, the reduction of labor and the improvement of work efficiency in logistics warehouses and factories have become issues. To solve these issues, the introduction of transport vehicles that can operate without human intervention and transport vehicle systems that efficiently operate multiple transport vehicles has been promoted.

[0003] Examples of transport vehicles include cart-type vehicles such as AMRs (Autonomous Mobile Robots) and forklifts. The transport vehicle system transmits a signal indicating a target task (such as loading and unloading) to these transport vehicles. Then, each transport vehicle automatically travels to the destination and executes the task at the destination.

[0004] Existing transport vehicle systems generally limit the range within which transport vehicles can move from a safety perspective, and the working ranges of humans and transport vehicles are also different. On the other hand, with the development of autonomous movement technology in the robotics field, even if the movement lines of transport vehicles and humans overlap, the transport vehicles themselves can determine the need to avoid or detour and select flexible actions according to the surrounding situation. Therefore, in the future, it is expected that the form of a transport vehicle system will be such that the working area is not limited for transport vehicles, the entire site is set as the movable range, and multiple transport vehicles and workers share the entire site.

[0005] In response to the aforementioned trends, in recent years, technologies for operating and managing multiple types (multiple vendors) of transport vehicles have attracted attention in transport vehicle systems. For example, for AMRs, forklifts, etc., there are operation and traffic control technologies that allow them to share the same driving environment, even though their tasks may differ. These control technologies for multiple vehicle types are also called Robot Operations or RobOps, and are attracting attention as a technology whose market is expected to expand further in the future.

[0006] A transport vehicle system targeting multiple vehicle types requires a different viewpoint map for each vehicle in its self-position calculation using on-board sensors. Conventionally, on-board sensors installed in each vehicle type created a map representing the environmental (external) shape for each individual vehicle. However, as the number of vehicles and vehicle types increases, the effort required for map creation increases.

[0007] Furthermore, from the perspective of reducing introduction and operating costs, it is necessary to keep the cost of on-board sensors per transport vehicle as low as possible. Therefore, on-board sensors are often low-spec models with limited measurement range or distance performance. Consequently, in complex driving environments such as warehouses or factories, where shelves are present all around and goods are placed on the road surface, the field of view or detection range of on-board sensors alone is insufficient, resulting in blind spots where distance measurement is impossible.

[0008] To address the aforementioned problems, for example, Patent Document 1 proposes a technique for creating a local grid map for a transport vehicle from information obtained from fixed sensors installed in the driving environment, and combining it with a grid map of the entire driving environment acquired by on-board sensors. By adopting the configuration of Patent Document 1, it is possible to perform self-position estimation and route planning to the destination, taking into account obstacle information in areas that cannot be measured by on-board sensors alone. In addition, even if the shape of the surrounding environment changes, such as the intrusion of workers or the rearrangement of shelves or cargo, these changes can be captured by fixed sensors and reflected in the map information. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] International Publication No. 2019 / 054209 [Overview of the project] [Problems that the invention aims to solve]

[0010] However, Patent Document 1 has difficulty in handling environments with multiple vehicle types. For example, if the mounting position of the on-board sensors or the performance of the sensors themselves differ for each target vehicle, even if a map is created in the same driving environment, a map with the same shape may not be obtained, and scale errors and orientation errors (distortion) may occur in the map. In such situations, it is difficult to accurately create a map for each vehicle by processing the information obtained from fixed sensors while taking into account the differences in map shape for each vehicle.

[0011] Furthermore, the technology described in Patent Document 1 requires the placement of many fixed sensors depending on the driving environment or the task being performed, and considering the optimal placement and number of sensors takes a considerable amount of time, which presents new challenges. Therefore, the present invention aims to provide a map generation function that is independent of vehicle type, number of vehicles, and the shape of the driving environment, and furthermore, to eliminate the need for environmental measurements for each vehicle type for map creation, thereby reducing the man-hours required for map generation. [Means for solving the problem]

[0012] The present invention provides an automatic map generation device comprising: a 3D environment measurement unit that acquires 3D information of the driving environment of a transport vehicle; a vehicle map generation unit that generates a vehicle map for each of the transport vehicles based on the 3D information, vehicle class information indicating the dimensions of the transport vehicle, and information regarding the performance and / or mounting position of on-board sensors mounted on the transport vehicle; and a map distribution unit that transmits the generated vehicle map to each of the transport vehicles. The vehicle map generation unit extracts the area measurable by the on-board sensor from the 3D information and generates a self-position estimation map for each of the multiple transport vehicles with a first resolution; extracts the area where the transport vehicles may interfere from the 3D information and generates a route planning map for each of the multiple transport vehicles with a second resolution finer than the first resolution; calculates the reliability of a portion of the vehicle map based on the distribution of wall point clouds obtained from the 3D information; and graphs the calculated reliability for each angle viewed from the transport vehicle. It is characterized by the following. Other means will be described within the descriptions of embodiments for carrying out the invention.

Advantages of the Invention

[0013] According to the present invention, it is possible to provide a map generation function that does not depend on the vehicle type, the number of vehicles, and the shape of the driving environment. Further, it is possible to eliminate the need for environmental measurement for each vehicle type for map creation and reduce the man-hours for map generation.

Brief Description of the Drawings

[0014] [Figure 1] It is a side view of a bogie-type transport vehicle. [Figure 2] It is a side view of a towing vehicle and a forklift. [Figure 3] It is a diagram showing the driving environment of Example 1. [Figure 4] It is a diagram showing a state where an in-vehicle sensor measures the surrounding environment. [Figure 5] It is a functional block diagram of a map automatic generation device and an in-vehicle controller. [Figure 6] It is a flowchart of a processing procedure executed by the map automatic generation device. [Figure 7] It is a diagram for explaining a three-dimensional point cloud. [Figure 8] It is a diagram for explaining principal component analysis. [Figure 9] It is a diagram showing an example of a method for approximating a road surface point cloud with a plane. [Figure 10] It is a conceptual diagram of the processing in step S607. [Figure 11] It is a conceptual diagram of the processing in step S608. [Figure 12] It is a conceptual diagram of the processing in step S609. [Figure 13] It is a diagram for comparing a map for self-position estimation and a map for route planning. [Figure 14] It is a diagram for explaining Example 2. [Figure 15] It is a conceptual diagram of step S607 in Example 3. [Figure 16] It is a diagram for explaining reliability. [Figure 17] It is an example of a graph of reliability. [Modes for carrying out the invention]

[0015] (terminology, etc.) The main terms used in Examples 1-3 below have the following meanings: A transport vehicle is a vehicle that autonomously and unmanned transports goods on the travel surface (floor) inside warehouses, factories, etc. The transport vehicle estimates its own position and plans its travel route. In this embodiment, an on-board sensor is a sensor mounted on a transport vehicle at a predetermined height relative to the road surface. The on-board sensor emits a light ray into a planar area at that height parallel to the road surface, and by measuring the time it takes for the light ray to reflect off a wall or other surface and return, it acquires the position of the wall or other surface at that height as a single layer of point cloud data. The height at which the on-board sensor is mounted varies depending on the type of transport vehicle, etc.

[0016] A master sensor is a sensor that emits a light ray into the entire three-dimensional space inside a warehouse, factory, etc., and measures the time it takes for the light ray to reflect off walls and other surfaces and return, thereby acquiring the positions of walls and other surfaces in three-dimensional space as a multi-layered point cloud. The portion of the point cloud acquired by the master sensor that is cut (sliced) into layers of a predetermined height can be used by various types of transport vehicles. The master sensor may be fixed or mobile, or it may be mounted on a transport vehicle. The number of master sensors is usually "1".

[0017] A master map is a multi-layered point cloud covering all regions of three-dimensional space, acquired by a master sensor; in other words, it is a three-dimensional map before slicing. The master map contains redundant point clouds from the perspective of a particular transport vehicle. A vehicle map is a portion of a sliced ​​master map that is usable only for specific types of transport vehicles. A vehicle map has a single-layer (or layer with limited thickness) point cloud that is usable only for specific types of transport vehicles. The vehicle map includes the self-position estimation map and route planning map described below.

[0018] The self-position estimation map is a vehicle map used by transport vehicles to estimate their own position, that is, a comparison map for the point clouds acquired by the vehicle's onboard sensors. A self-position estimation map is created for each type of transport vehicle and contains a single layer of point clouds at the height of its onboard sensors. In other words, the self-position estimation map is a vehicle map obtained by extracting the area measurable by the onboard sensors from the 3D information of the master map.

[0019] A route planning map is a vehicle map used by transport vehicles to plan their travel routes. Transport vehicles must travel (transport) in a way that avoids interference between themselves and their cargo with other objects such as walls and shelves. Therefore, a route planning map is created for each type of transport vehicle and has a single-layer point cloud with a thickness between the height of the lowest part of the transport vehicle (the road surface) and the height of the highest part. The positions of the transport vehicle's onboard sensors are included in this thickness. The maximum dimensions of the anticipated cargo are also included in this thickness. In short, a route planning map is a vehicle map obtained by extracting areas where transport vehicles may interfere from the 3D information of a master map. Map matching is the process of comparing point cloud data acquired by on-board sensors while a transport vehicle is in motion with point cloud data previously acquired by master sensors and prepared on a vehicle map.

[0020] <Example 1> (Transport vehicle) Transport vehicles include trolley-type (low-floor) transport vehicles, towing vehicles, and forklifts. Figure 1 is a side view of the trolley-type transport vehicle 100. The trolley-type transport vehicle 100 includes a vehicle frame 101 equipped with a bumper 102, a transfer device 103 that can move freely up, down, left, and right relative to the vehicle frame 101, and a cargo handling member 104 that supports the load 112. The vehicle frame 101 has a transfer motor 105 inside, which drives the transfer device 103. The trolley-type transport vehicle 100 includes drive wheels 106 and driven wheels 107. The vehicle frame 101 has a travel motor 108 and an encoder 109 inside. The travel motor 108 drives the drive wheels 106. The encoder 109 measures the rotational speed of the drive wheels 106.

[0021] The vehicle frame 101 houses an on-board controller 120. The on-board controller 120 is a computer that performs calculations to enable autonomous driving of the trolley-type transport vehicle 100, and comprises a processor such as a CPU (Central Processing Unit) and memory for storing programs and data. Specifically, the on-board controller 120 performs the following: driving path planning, self-position estimation, and control command value generation, which calculates command values ​​for the driving motor 108.

[0022] Figure 2 is a side view of the towing vehicle 200 and the forklift 203. This embodiment applies to multiple vehicle types. The transport vehicle in this embodiment may be, for example, a towing vehicle 200 with a transport body 202 attached to the rear of a powered vehicle 201, as shown in the upper part of Figure 2, or a forklift 203, as shown in the lower part of Figure 2. Operation may be a push method, where the transport body 202 is pushed, rather than a pull method like towing, or a push-pull method, where both are used. In addition, multiple vehicles may be coupled together.

[0023] (Vehicle-mounted sensors) In Example 1, each type of transport vehicle is equipped with a LiDAR (Light Detection And Ranging) sensor as an on-board sensor. LiDAR is a sensor that measures the distance to objects within the irradiation range while changing the irradiation angle of the laser beam. LiDAR acquires point cloud information indicating the position of an object from the optical axis angle at the time of laser irradiation and the distance to the object. Example 1 describes a 2D-LiDAR with only one layer of laser light in the vertical direction as a representative example, but the LiDAR configuration is not limited to 2D-LiDAR; it may also be a 3D-LiDAR capable of irradiating multiple layers. Note that LiDAR is sometimes called a ToF sensor (Time of Flight Sensor).

[0024] The on-board sensor 111 in Figure 1 and the on-board sensors 211 and 221 in Figure 2 show on-board sensors installed on transport vehicles with various driving modes. In Embodiment 1, one on-board sensor is mounted on one transport vehicle, but in this embodiment, multiple on-board sensors may be mounted on one transport vehicle.

[0025] (Driving environment (warehouse / factory)) Figure 3 shows the driving environment 300 for the transport vehicle group (trolley-type transport vehicle 100, towing vehicle 200, and forklift 203) assumed in Example 1. In a warehouse or factory, there are narrow and narrow paths enclosed by shelves 301, etc. Goods 302 may be stored on the shelves 301. Pallets 303 may be temporarily placed on the path, and goods 302 may be stacked on the pallets 303. There are also charging units 304 for each transport vehicle, conveyors 305 for transporting goods from other areas, etc. Note that the driving environment also includes places such as attractions, farms, and plant cultivation greenhouses.

[0026] (Driving environment (LiDAR illumination surface)) Figure 4 shows how on-board sensors 111 and 221 mounted on a trolley-type transport vehicle 100 and a forklift 203 measure the driving environment 300. Regions 401 and 402 indicate the laser light irradiation areas of each transport vehicle. The irradiation surface differs depending on the mounting position of the on-board sensor. Therefore, when realizing self-position estimation by map matching, it is necessary to individually generate maps in advance that represent the environmental shape that each on-board sensor will measure.

[0027] Furthermore, since the pallet 303, charging unit 304, etc., are low-profile obstacles placed on the road surface, they may fall outside the range of the on-board sensors depending on their mounting position. This does not affect the accuracy of self-position estimation by map matching. However, in route planning to the destination, if a map that also represents these obstacles is not used, the transport vehicle will mistakenly judge areas with obstacles as drivable areas and interfere with the obstacles. In Example 1, a transport vehicle system operating in such a driving environment provides an automatic map generation function that is independent of the vehicle type, number of vehicles, and shape of the driving environment.

[0028] (Automatic map generation device and transport vehicle system) Figure 5 is a functional block diagram of the automatic map generation device 501 and the in-vehicle controller 120. The solid arrows in Figure 5 represent the flow of data. The automatic map generation device 501 includes a vehicle model information management unit 511, an in-vehicle sensor information management unit 512, a master sensor 513, a 3D environment measurement unit 514, a 3D point cloud information management unit 515, a vehicle map generation unit 516, a map distribution unit 519, and a communication device 530. The following describes the internal functions of the automatic map generation device 501, which is the main focus of this embodiment. The automatic map generation device 501 and the transport vehicle (reference numerals 100, 200, or 203) constitute the transport vehicle system 500. In Figure 5, reference numeral 500 is used to denote the combined configuration of the in-vehicle controller 120 and the automatic map generation device 501 (the description of the entire transport vehicle is omitted).

[0029] (Vehicle Model Information Management Department) The vehicle model information management unit 511 stores basic information about the transport vehicle (a trolley-type transport vehicle in Example 1) 100. Specifically, it stores dimensional values ​​related to the vehicle class (vehicle specifications), such as vehicle width, vehicle length, vehicle height, and wheelbase. If the transport vehicle is a forklift 203, the vehicle height when the mast is raised is also stored, as the mast is raised and lowered. If the transport vehicle is loaded with cargo, the vehicle width, vehicle length, and vehicle height are stored, including the maximum possible size of the cargo that may occur during operation.

[0030] (Vehicle Sensor Information Management Department) The in-vehicle sensor information management unit 512 stores performance information and mounting position information for the in-vehicle sensor (LiDAR) 111. The performance information includes the maximum irradiation distance of the in-vehicle sensor, the horizontal resolution of the laser beam, the distance measurement range, and the distance measurement error. The mounting position information is the reference coordinates of the transport vehicle, for example, the mounting position (including the height from the road surface) and orientation (orientation of the in-vehicle sensor) of the in-vehicle sensor 111 as seen from the center of the rear wheels.

[0031] (Master sensor) The master sensor 513 is a multi-layer, wide-angle LiDAR and may be equipped with a high-period IMU (Inertial Measurement Unit) as needed. If the master sensor 513 is a multi-layer, wide-angle 3D-LiDAR, three-dimensional information of the driving environment 300 can be easily collected. In order to acquire point cloud information with the highest possible density, it is desirable that the master sensor 513 has better performance than the on-board LiDAR installed in the transport vehicle.

[0032] (Type of automatic map generation device) The automatic map generation device 501, including the master sensor 513, may be mounted on a specific transport vehicle 100, as shown by reference numeral 501A in Figure 7. Furthermore, the automatic map generation device 501, including the master sensor 513, may also be in the form of a handheld measuring unit that can be held by an operator 702, for example, as shown by reference numeral 501B in Figure 7.

[0033] (3D Environmental Measurement Unit) The 3D environment measurement unit 514 takes the point cloud information acquired by the 3D-LiDAR, which constitutes the master sensor 513, during driving, and the acceleration and angular velocity (angular acceleration) obtained from the IMU as input, and acquires 3D point cloud information of the driving environment 300 using SLAM (Simultaneous Localization And Mapping). Since the acquisition of 3D point clouds by SLAM is well known, an explanation is omitted.

[0034] (3D point cloud information management department) The 3D point cloud information management unit 515 stores the 3D point cloud information acquired by the 3D environment measurement unit 514. The 3D point cloud management unit 515 may perform downsampling using a voxel grid, for example, as needed, to reduce the amount of data. Since the downsampling technique is also publicly known, a detailed explanation is omitted. The 3D point cloud management unit 515 may also delete in advance parts that are clearly unnecessary for map generation, such as point clouds of the ceiling.

[0035] (Vehicle map generation unit) The vehicle map generation unit 516 includes a self-position estimation map generation unit 517 and a route planning map generation unit 518. The self-position estimation map generation unit 517 pre-generates a self-position estimation map that represents obstacles that are likely to be measured by the on-board sensors as grid information when performing self-position estimation by map matching. The route planning map generation unit 518 pre-generates a route planning map that represents obstacles that are likely to interfere with the body of the transport vehicle as grid information. These processes are the main focus of this embodiment, and the details of the processes will be described later.

[0036] (Map Distribution Department) The map distribution unit 519 transmits the self-position estimation map and route planning map generated by the vehicle map generation unit 516 to the on-board controller 120 of the transport vehicle via the communication device 530.

[0037] The above describes the configuration and internal processing of the automatic map generation device 501. The following describes the functions of the in-vehicle controller 120, which enables autonomous driving using the maps acquired from the automatic map generation device 501.

[0038] (Destination input section) The destination input unit 521 receives a route planning map distributed by the automatic map generation device 501 and input of the task currently being performed by the transport vehicle or the destination for task execution. For example, the destination input unit 521 may also accept direct input of the destination location by an operator via an external device (such as a mobile display) that displays the status of the transport vehicle and the route planning map. Although not shown in Figure 5, the destination input unit 521 may, for example, receive the operational status of tasks pre-set by a control server that manages multiple transport vehicles, and automatically calculate the destination based on that operational status.

[0039] (Route Planning Department) The route planning unit 522 takes a route planning map and the task currently being performed by the transport vehicle or the destination for task execution as input and plans a route to the destination. The route planning map represents the environment in a grid and stores the probability of an obstacle existing in each grid cell in association with that cell (see Figure 13). This configuration is also called an occupied grid map. There are many known route planning methods for grid maps, such as Dijkstra's algorithm and A* method, so their explanation is omitted here. The planned route may consist of continuous coordinate values ​​in the map coordinate system of the route planning map, functions representing curves, etc.

[0040] (Self-position estimation unit) The self-position estimation unit 523 takes point cloud information acquired by the on-board sensor 111 and a self-position estimation map acquired by the automatic map generation device 501 as input and uses map matching to acquire the self-position and orientation of the trolley-type transport vehicle 100. When the point cloud acquired by the on-board sensor 111 is viewed as a figure, both the point cloud and the self-position estimation map are similar. Map matching is a technique that utilizes this property and performs translation, rotation, and scaling on the point cloud to match (determine agreement) the obstacle information on the self-position estimation map with the point cloud. However, since it is difficult to achieve an error-free match, the self-position estimation unit 523 may define a certain evaluation value and explore the self-position and orientation by solving the operation that minimizes the evaluation value. The self-position estimation unit 523 may, for example, use the Adaptive Monte Carlo Localization method utilizing a particle filter.

[0041] (Control command generation unit) The control command generation unit 524 takes the target travel position of the target path planned by the travel path planning unit 522 and the current position output by the self-position estimation unit 523 as input, and generates command values ​​for the travel motor 108 of the transport vehicle 100 according to the path following algorithm so that it passes through the target travel position, and transmits them to the drive unit 525. As there are known methods for path following algorithms such as the Pure Pursuit method and the Dynamic Window Approach method, a description is omitted here.

[0042] (Drive unit and encoder) The drive unit 525 converts the command value input from the control command generation unit 524 into a predetermined current value and inputs the converted current value to the travel motor 108 of the transport vehicle 100. The travel motor 108 transmits power to the drive wheels 106. The encoder 109 provided on the drive wheels 106 acquires the rotational speed of the travel motor 108 in a time series and estimates the current speed of the vehicle by taking into account the radius of the drive wheels 106. The encoder 109 feeds back the current speed as a response value from the travel motor 108 to the control command generation unit 524 and the self-position estimation unit 523. The above is a description of the functions implemented within the on-board controller 120 mounted on the transport vehicle 100.

[0043] (Processing procedure) Figure 6 is a flowchart of the processing procedure performed by the automatic map generation device 501. Figures 7 to 12 will be referred to as appropriate during the explanation of Figure 6. In step S601, the 3D environmental measurement unit 514 performs 3D point cloud measurement using the master sensor 513. The details of the process here are as described above in the explanation of the 3D environmental measurement unit 514. Now, let's move on to Figure 7.

[0044] Figure 7 illustrates the three-dimensional point cloud. The three-dimensional point cloud in Figure 7 is the point cloud obtained when measuring the shape of the entire driving environment 300. Point clouds 711, 712, 713, and 714 are parts of these point clouds and were acquired by the master sensor 513. Point cloud 711 is the distance measurement points on the driving surface. Point cloud 712 is the distance measurement points on the shelves and loads. Point cloud 713 is the measurement points on the pallet 303 placed on the driving surface. Point cloud 714 is the measurement points on the charging unit 304. The explanation returns to Figure 6.

[0045] In step S602, the vehicle map generation unit 516 extracts a point cloud of the road surface from the acquired 3D point cloud information. The "vehicle map generation unit 516," which is the main operator in steps S602 to S606 in Figure 6, may be either the self-position estimation map generation unit 517 or the route planning map generation unit 518. Furthermore, the vehicle map generation unit 516 as the main operator may have configurations other than the self-position estimation map generation unit 517 and the route planning map 518 (not shown) from the vehicle map generation unit 516 in Figure 5. Embodiment 1 uses principal component analysis as the point cloud extraction means for the road surface. Now, let's move on to Figure 8.

[0046] Figure 8 illustrates principal component analysis. The vehicle map generation unit 516 assigns a sphere 802 with a radius of 0.5m to an arbitrary point in the 3D point cloud acquired by the master sensor 513, and performs principal component analysis on the point cloud within the sphere 802. In other words, the vehicle map generation unit 516 obtains the second principal component (the vertical vector of the first principal component) as a result of the principal component analysis. The "second principal component" is the normal vector of the plane that best fits the point cloud within that sphere.

[0047] The vehicle map generation unit 516 obtains normal vectors 803 whose second principal component is in the Z-axis direction from points on the road surface, such as point cloud 711. The vehicle map generation unit 516 obtains normal vectors 804, 805, and 806 that are close to the horizontal direction as the second principal components of obstacles, such as point clouds 712, 713, and 714. Based on this, the vehicle map generation unit 516 can determine whether an arbitrary measurement point is on the road surface or on the wall surface of an obstacle by calculating, for example, the angle of the normal vectors with respect to the XY plane. Note that the circles (lightly shaded) representing point cloud 711 etc. in Figure 8 and the black dots (●) representing point cloud 711 in Figure 7 are essentially the same. In Figure 8, circles are used to clarify that they are the basis for determining the normal vectors. The explanation returns to Figure 6.

[0048] In step S603, the vehicle map generation unit 516 calculates the plane equation of the road surface from only the point cloud determined to be the point cloud constituting the road surface (hereinafter also referred to as the road surface point cloud). Now, let's move on to Figure 9.

[0049] Figure 9 shows an example of a method for approximating a road surface point cloud with a plane. In Example 1, the vehicle map generation unit 516 divides the entire road surface point cloud into a 0.5m × 0.5m grid (see dashed line 901), and uses the point clouds present in each grid to derive the plane equation when the point cloud is approximated by a plane. Note that the circles (○) representing point cloud 711 in Figure 9 and the black dots (●) representing point cloud 711 in Figure 7 are essentially the same. In Figure 9, circles (○) are used to indicate that they serve as the basis for determining the plane equation of the road surface.

[0050] One well-known method for approximating a plane is RANSAC (RANdom SAmple Consensus), a robust estimation technique. RANSAC randomly selects more than the number of points required to derive the plane equation (at least 3 points in Example 1) from the input point cloud set, and derives the plane equation using the least squares method. RANSAC randomly changes the input point cloud to derive several candidate plane equations, then calculates the variance for the entire point cloud when actually fitting it to the entire input point cloud, and uses the plane equation with the smallest variance as the final solution. RANSAC has the advantage of being less susceptible to noise compared to the simple least squares method.

[0051] Specific examples of noise include points that protrude excessively in the Z-axis direction compared to the surrounding point cloud, caused by the distance measurement error of the master sensor 513. The thick solid frame 903 in Figure 9 shows the result of superimposing the plane equation obtained from plane estimation on the point cloud 711 located within an arbitrary grid 902 onto the road surface. The explanation returns to Figure 6.

[0052] In step S604, the vehicle map generation unit 516 extracts wall point clouds. Here, the vehicle map generation unit 516 extracts point clouds other than the road surface point cloud from the processing in step S602 as wall point clouds. The wall point cloud includes point clouds on the vertical planes of objects other than actual walls (shelves, cargo, etc.).

[0053] In step S605, the vehicle map generation unit 516 acquires vehicle class information. Here, the vehicle map generation unit 516 first sets the target vehicle. In Embodiment 1, as an example, a trolley-type transport vehicle 100 is used as the target vehicle. Subsequently, the vehicle map generation unit 516 acquires vehicle class information (dimensions, etc.) corresponding to the target vehicle from the information stored in the vehicle model information management unit 511 in Figure 5.

[0054] In step S606, the vehicle map generation unit 516 acquires on-board sensor performance information and on-board sensor mounting location information. Here again, the vehicle map generation unit 516 acquires information corresponding to the target vehicle set in step S605 from the information stored in the on-board sensor information management unit 512 in Figure 5.

[0055] The following processes are executed by the self-position estimation map generation unit 517 and the route planning map generation unit 518. The flowchart in Figure 6 divides the process into two after step S606, but either flow may be executed first. The two flows may also be executed in parallel. In Example 1, the process of generating the self-position estimation map is performed first.

[0056] In step S607, the self-position estimation map generation unit 517 extracts a point cloud for self-position estimation. Now, let's move on to Figure 10.

[0057] Figure 10 is a conceptual diagram of the process in step S607. Here, the self-position estimation map generation unit 517 uses the plane equation representing the 0.5m square road surface obtained in step S603 as a reference, and extracts a point cloud from the on-board sensor performance information acquired in step S606 that the on-board LiDAR may measure distance from. Specifically, the self-position estimation map generation unit 517 calculates a normal vector 1001 perpendicular to the road surface 903 based on the plane equation of the road surface. The starting point of the normal vector 1001 is the center point of the road surface 903. The magnitude (length) of the normal vector 1001 is an arbitrary value greater than the ceiling height of the driving environment 300.

[0058] The self-position estimation map generation unit 517 sets a horizontal circle 1003 centered on a point corresponding to the height 1002 of the on-board sensor 111 on the normal vector 1001. The radius 1004 of the circle 1003 is based on the distance measurement performance of the on-board sensor 111. Specifically, if the maximum distance measurement distance of the LiDAR is 10m, the radius of the circle 1003 will be 10m. However, as the radius of the circle 1003 increases, it becomes more susceptible to the effects of estimation errors in the plane equation and mounting errors of the sensor in the search process described later. This may cause the accuracy of the generated map to deteriorate. Therefore, even if the maximum distance measurement performance is 30m, for example, the self-position estimation map generation unit 517 may set the radius of the circle 1003 to a value less than or equal to the maximum distance measurement performance (for example, 10m).

[0059] In Example 1, the self-position estimation map generation unit 517 divides the circle 1003 into N equal parts (see reference numeral 1005 in Figure 10) and sets up a square pyramid 1007 with a height of 1006 in each of the N divided parts of the circle. If the horizontal resolution (N) is, for example, 360, then 360 square pyramids with a height of 1 degree each are set. The height 1006 of the square pyramids is set to a value of, for example, 1 degree, taking into account the mounting error of the on-board sensor 111 and the estimation error of the road surface.

[0060] Next, the self-position estimation map generation unit 517 determines whether or not a wall point cloud exists inside the set square pyramid 1007. The self-position estimation map generation unit 517 may use existing algorithms for determining whether a point is inside or outside a polygon (such as the Crossing Number Algorithm or Winding Number Algorithm).

[0061] The self-position estimation map generation unit 517 repeatedly performs this process for each of the square pyramids 1007 that are shifted by 1 degree horizontally. The self-position estimation map generation unit 517 also performs the above process for the entire plane constituting the road surface obtained in step S603 (see reference numeral 901 in Figure 9). Note that in the point cloud of Figure 10, ● is located inside the square pyramid 1007. ○ is not located inside the square pyramid 1007. The explanation returns to Figure 6.

[0062] In step S608, the self-position estimation map generation unit 517 generates a self-position estimation map. Now, let's move on to Figure 11.

[0063] Figure 11 is a conceptual diagram of the process in step S608. The point cloud 1101 shown in the upper part of Figure 11 is a projection onto the XY plane of the point cloud estimated to be measured by the on-board sensor 111 obtained in step S607. The 3x3 grid 1102 is part of the self-position estimation map 1301, and the grid size 1103, which indicates its resolution, can be set arbitrarily. In Example 1, the grid size 1103 is, for example, 0.05m x 0.05m. Here, the self-position estimation map generation unit 517 determines whether or not the point cloud projected onto the XY plane exists within the grid.

[0064] In Example 1, the self-position estimation map generation unit 517 determines that a grid is an obstacle area if at least one point exists within that grid. Based on this determination, the self-position estimation map generation unit 517 assigns a "1" (black) gradient to grids where point clouds exist and a "0" (white) gradient to grids where point clouds do not exist. The lower part of Figure 11 shows the grid map generated after the above processing. The planar size of the entire map is assumed to encompass the entire driving environment. The map obtained here is transmitted to the map distribution unit 519 as the self-position estimation map 1301. The explanation returns to Figure 6.

[0065] In step S609, the route planning map generation unit 518 extracts vehicle interference point clouds. Now, let's move on to Figure 12.

[0066] Figure 12 is a conceptual diagram of the process in step S609. In this case, if the target vehicle is a forklift 203, the mast section 1201 of the forklift 203 will rise and fall, and the maximum height of the transport vehicle will change compared to the normal state (empty and with the mast lowered). First, the route planning map generation unit 518 sets the vehicle height 1203 of the target vehicle, taking into account the rising and falling of the mast section 1201 and the size of the transported load 1202, based on the vehicle size information acquired in step S605.

[0067] Next, the route planning map generation unit 518 sets a normal vector 1204 perpendicular to the road surface 903 (plane equation) shown in Figure 10, similar to step S607. Then, the route planning map generation unit 518 sets a cylinder 1205. The height of the cylinder 1205 is equal to the vehicle height 1203. The radius of the cylinder 1205 is the maximum distance measurement distance of the on-board LiDAR, which is the same value as in step S607 (10m). The central axis of the cylinder is the normal vector 1204. Finally, the route planning map generation unit 518 determines whether or not a wall point cloud exists within the cylinder 1205. The above process is performed on all planes (see reference numeral 901 in Figure 9) that constitute the road surface obtained in step S603. The explanation returns to Figure 6.

[0068] In step S610, the route planning map generation unit 518 generates a route planning map. The content of step S610 is the same as that of the self-position estimation map generation (step S608). Now, let's move on to Figure 13.

[0069] Figure 13 compares the self-position estimation map and the route planning map. The upper part of Figure 13 is the self-position estimation map 1301 generated by the self-position estimation map generation unit 517. The lower part of Figure 13 is the route planning map 1302 generated by the route planning map generation unit 518. Compared to the self-position estimation map 1301, the route planning map 1302 can represent obstacles that cannot be measured by the on-board LiDAR alone. The explanation returns to Figure 6.

[0070] In step S611, the map distribution unit 519 determines whether both the self-position estimation map 1301 and the route planning map 1302 have been generated. For example, if steps S607 and S609 are being executed in parallel, it is possible that one of them is still being processed due to the difference in computational load, so the map distribution unit 519 waits until both steps S608 and S610 are completed.

[0071] If both the self-position estimation map 1301 and the route planning map 1302 have been generated (step S611 "YES"), the map distribution unit 519 proceeds to step S612; otherwise (step S611 "NO"), it returns to immediately after step S606. The processing in steps S607 to S610 is repeated for all types of transport vehicles in operation.

[0072] In step S612, the map distribution unit 519 transmits the self-position estimation map 1301 and the route planning map 1302 to the on-board controller 120 of the transport vehicle via the communication device 530. The map distribution unit 519 transmits one copy each of the self-position estimation map 1301 and route planning map 1302 for each transport vehicle in operation.

[0073] (Effects of Example 1) By implementing Example 1, it is possible to provide an automatic map generation function that is independent of vehicle type, number of vehicles, and the shape of the driving environment. Furthermore, it eliminates the need for environmental measurements for creating maps for each vehicle type, thereby reducing the man-hours required for map generation.

[0074] <Example 2> Figure 14 illustrates Embodiment 2. The configuration of the automatic map generation device 501 and the in-vehicle controller 120 in Embodiment 2 is the same as in Embodiment 1 (Figure 5). The difference between the two lies in the processing of the route planning map generation unit 518, and the processing of the route planning map generation unit 518 will be described in detail below.

[0075] (Route planning map generation unit) In Example 2, in the route planning map generation process (step S610 in Figure 6), the route planning map generation unit 518 may generate a map with a different resolution (grid size) than the self-position estimation map. In conventional SLAM methods using on-board LiDAR, the resolution of the self-position estimation map and the route planning map were generally determined by the horizontal resolution of the laser light. This is because if SLAM is also performed to create a map with finer subdivisions, the map generation will fail if the performance of the on-board LiDAR is insufficient.

[0076] Unlike self-localization, the actual processing of route planning does not require input from the on-board LiDAR. On the other hand, the finer the resolution of the map, the more precisely the map can represent the shapes of surrounding obstacles, allowing for the calculation of a route that takes both efficiency and safety into better consideration. Therefore, the route planning map generation unit 518 may, for example, take into account the computing power of the on-board controller and generate a route planning map with a finer grid size compared to the self-localization map.

[0077] In this embodiment, since the vehicle map is generated using the master sensor 513 instead of the on-board sensor (LiDAR), it is possible to generate a high-resolution map that would be impossible with the on-board LiDAR alone. Therefore, in Embodiment 2, the route planning map generation unit 518 sets a grid resolution of 0.01m in the route planning map generation process (step S610), for example. The route planning map generation unit 518 sets the grid resolution taking into account not only the horizontal resolution of the master sensor 513, but also the computational performance of the on-board controller 120 and the algorithm load of the driving route planning unit 522. Note that the processing content of Embodiment 2 is the same as that of Embodiment 1, except that the set value of the grid size is different.

[0078] The upper part of Figure 14 shows the self-localization map 1401, and the lower part of Figure 14 shows the path planning map 1402. The self-localization map 1401 is generated with the same first resolution (grid size 0.05m) as in Example 1 (reference numeral 1301 in Figure 13). The path planning map 1402 is generated with a second resolution (grid size 0.01m) different from the first resolution, and therefore, compared to the self-localization map 1401, it can represent the shapes of surrounding obstacles in more detail.

[0079] (Effects of Example 2) By executing Example 2, it is possible to generate a map that does not depend on the performance of the vehicle's onboard sensors. For example, in route planning, by generating a route planning map with a finer grid size, it is possible to calculate a route that takes both efficiency and safety into consideration.

[0080] <Example 3> Embodiment 3 will be described with reference to Figures 15, 16, and 17. The configuration of the automatic map generation device 501 and the in-vehicle controller 120 in Embodiment 3 is the same as in Embodiment 1 (Figure 5). The difference between the two lies in the processing of the self-position estimation map generation unit 517, the route planning map generation unit 518, the driving route planning unit 522, and the self-position estimation unit 523, and these processes will be described in detail below.

[0081] Figure 15 is a conceptual diagram of the point cloud estimation process for self-localization in Example 3 (step S607 in Figure 6). In Example 3, the process of calculating a normal vector 1001 perpendicular to the estimated road surface 903, calculating a square pyramid 1007 starting from the installation height 1002 of the on-board LiDAR, and then determining whether or not a wall point cloud exists inside the square pyramid 1007 is the same as in Example 1.

[0082] In this invention, the wall point cloud measured by the on-board sensor (LiDAR) is estimated based on the 3D point cloud obtained from the master sensor 513. Therefore, it is affected by the driving road surface 903 estimated from the 3D point cloud and the mounting position and attitude errors of the on-board sensor.

[0083] Specifically, as shown in the lower left of Figure 15, if the wall surface of the driving environment is flat and free from irregularities, even if there is some uncertainty in the road surface estimation and the mounting position of the on-board sensor, the difference in the horizontal values ​​(X and Y directions) of the extracted wall point cloud is small. On the other hand, as shown in the lower right of Figure 15, if there are irregularities on the wall surface, the horizontal values ​​1503 of the extracted wall point cloud change significantly depending on the range of the square pyramid 1007. This results in a difference between the point cloud on the vehicle map and the point cloud actually measured by the on-board sensor (LiDAR), which worsens the accuracy of map matching in the self-position estimation unit 523. Furthermore, the accuracy of route planning in the driving route planning unit 522 also deteriorates. Therefore, Example 3 determines whether the extracted road surface point cloud is a reliable point or not.

[0084] Specifically, the self-position estimation map generation unit 517 and the path planning map generation unit 518 perform plane fitting (calculation of a plane equation) on the point cloud extracted inside the square pyramid 1007, similar to step S603. The results of the plane fitting by RANSAC are estimated planes 1501 and 1502 in Figure 15. Since these are superimposed on the XZ plane, the planes are drawn like line segments. Estimated planes 1501 and 1502 are both planes with similar shapes, but the distance of each point cloud to the fitted plane (see reference numeral 1503 as an example) is different. Specifically, the variance values ​​of each point cloud with respect to the fitted plane (variance of the length of arrow 1503) are different. Therefore, the self-position estimation map generation unit 517 and the path planning map generation unit 518 calculate the confidence level of the points according to the variance value. The smaller the variance value, the higher the confidence level. The confidence level is, for example, the reciprocal of the variance value.

[0085] Figure 16 is a diagram illustrating the reliability level. The reliability level is represented as a brightness value in the self-localization map (or path planning map) 1601. Each grid cell in the self-localization map (or path planning map) 1601 in Figure 16 represents a different brightness value depending on the reliability level. The self-localization map generation unit 517 and the path planning map generation unit 518, for example, lower the brightness value (make it darker) as the reliability level increases, and increase the brightness value (make it brighter) as the reliability level decreases.

[0086] The self-position estimation map (or route planning map) 1601 obtained through the above processing is transmitted to the in-vehicle controller 120 via the map distribution unit 519 and the communication device 530, similar to the first embodiment. The driving route planning unit 522 of the in-vehicle controller 120 determines the target driving speed for each coordinate of the planned route based on the reliability of the obstacles in each grid cell within the self-position estimation map 1601. The lower the reliability of a grid cell, the lower the target driving speed in that grid cell.

[0087] As an example, the route 1602 shown in Figure 16 is a route planned by the route planning unit 522 based on the route planning map 1302 (or 1402). The route planning unit 522 calculates the reliability of the grids within circles 1605 and 1606 with a radius of 10m, centered on some of the coordinate values ​​1603 and 1604 obtained by dividing the route 1602 at, for example, 0.2m intervals, for each 0.1rad sector 1607. If there are multiple grids within a sector 1607, for example, the average of their reliability values ​​is calculated.

[0088] Figure 17 is an example of a confidence graph. Figure 17 shows the confidence levels for each sector 1607 within circles 1605 and 1606, as graphed by the self-position estimation map generation unit 517 or the route planning map generation unit 518. Graph 1701 in the upper part of Figure 17 represents the confidence level within circle 1605. Graph 1702 in the lower part of Figure 17 represents the confidence level within circle 1606.

[0089] The self-position estimation unit 523 estimates its own position from the similarity relationship between the point cloud acquired by the on-board LiDAR and the self-position estimation map. Therefore, even if there are partial differences between the point cloud and the self-position estimation map, it can still perform self-position estimation with sufficient accuracy. As shown in Graph 1701, assuming that high-accuracy self-position estimation is possible even if the reliability is partially low, the driving route planning unit 522 may determine the normal speed (for example, the maximum speed) even at locations on the route with low reliability.

[0090] On the other hand, as shown in Graph 1702, if the reliability is low across the entire surrounding area, a large discrepancy may occur between the point cloud from the on-board LiDAR and the map used for self-localization estimation. Therefore, the accuracy of self-localization estimation is likely to deteriorate. In such cases, the driving route planning unit 522 can reduce the target driving speed and make it possible to drive even if the results of self-localization estimation are not stable. Alternatively, the driving route planning unit 522 or the self-localization estimation unit 523 may use a partial map update technique to update a portion of the vehicle map using the on-board LiDAR during driving (operation) for sections with low reliability.

[0091] (Effects of Example 3) By implementing Example 3, even in areas where the driving environment has significant and complex unevenness on the walls and the accuracy of the vehicle map created by the master sensor is insufficient, the reliability of each grid representing the map can be calculated, allowing for adjustment of the target speed setting during route planning. Furthermore, areas with low reliability can be prompted to update the map using the on-board sensor.

[0092] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are described in detail for the purpose of explaining the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations. [Explanation of Symbols]

[0093] 100, 200, 203 Transport vehicles (trolley-type transport vehicles, towing vehicles, forklifts) 111, 211, 221 Automotive Sensors (LiDAR) 120 In-vehicle controllers 500 Transport Vehicle System 501 Automatic Map Generation Device 513 Master Sensor 514 3D Environmental Measurement Unit 516 Vehicle Map Generation Unit 517 Self-position estimation map generation unit 518 Route Planning Map Generation Unit 519 Map Distribution Department 522 Route Planning Department 523 Self-position estimation part 1301 Map for self-localization 1302 Route planning map

Claims

1. A 3D environment measurement unit acquires 3D information of the driving environment of the transport vehicle, A vehicle map generation unit generates a vehicle map for each of the multiple transport vehicles based on the three-dimensional information, vehicle class information indicating the dimensions of the transport vehicle, and information regarding the performance and / or mounting position of on-board sensors mounted on the transport vehicle. A map distribution unit that transmits the generated vehicle map to each of the multiple transport vehicles, Equipped with, The aforementioned vehicle map generation unit is: From the three-dimensional information, the area measurable by the on-board sensor is extracted, and a self-position estimation map is generated for each of the multiple transport vehicles with a first resolution. From the three-dimensional information, regions where the transport vehicles may interfere are extracted, and a route planning map is generated for each of the transport vehicles with a second resolution finer than the first resolution. Based on the variance of the wall point cloud obtained from the aforementioned three-dimensional information, the reliability of the portion of the vehicle map is calculated. To graph the calculated reliability for each angle viewed from the transport vehicle, An automatic map generation device characterized by the following.

2. The automatic map generation device according to claim 1, Based on the aforementioned reliability, the transport vehicle determines the travel speed, A transport vehicle system characterized by comprising the following features.

Images