Position estimation device, control device, commercial vehicle, logistics support system, position estimation method and program

Abstract

Position estimation device (100A) for estimating the position of a commercial vehicle, which is equipped with a laser scanner capable of capturing point cloud data by scanning a laser beam and a camera capable of capturing an image in a predetermined direction, wherein the position estimation device (100A) comprises: a capture unit (1000) configured to capture the point cloud data and the image; an estimation unit (1004) configured to estimate the position of the commercial vehicle based on at least one of the point cloud data and the image;and an extraction unit (1001) configured to extract valid point cloud data from the point cloud data that can be used to estimate the position, wherein, in a case where a number of pieces of the valid point cloud data is equal to or greater than a predetermined number, the estimation unit (1004) estimates the position of the commercial vehicle (1) using the valid point cloud data, and in a case where a number of pieces of the valid point cloud data is less than the predetermined number, the estimation unit (1004) estimates the position of the commercial vehicle (1) using the image, wherein the extraction unit (1001) excludes point cloud data derived from a temporarily existing object from the point cloud data and extracts the remaining point cloud data as valid point cloud data.

Description

Technical field

[0001] The present disclosure relates to a position estimation device, a control device, a commercial vehicle, a logistics support system, a position estimation method and a program. State of the art

[0002] Some commercial vehicles used indoors or partially outdoors cannot use a global navigation satellite system (GNSS). Therefore, some commercial vehicles are equipped with a sensor configured to estimate the vehicle's position. List of oppositions patent literature

[0003] Patent Literature 1: JP 2019-074505 A Brief description of the invention: Technical problem

[0004] Light detection and localization (LiDAR) is a known sensor technology for self-position estimation (see patent literature 1). However, a problem arises when a laser beam is blocked by an obstacle in the work area (for example, another commercial vehicle or temporarily stored luggage).

[0005] One purpose of the present revelation is to achieve a more robust self-positioning estimate. Solution to the problem

[0006] According to one aspect of the present disclosure, a position estimation device is provided for estimating the position of a commercial vehicle, which is equipped with a laser scanner capable of capturing point cloud data by scanning a laser beam and a camera capable of capturing an image in a predetermined direction, the device comprising: a capture unit configured to capture the point cloud data and the image; an estimation unit configured to estimate the position of the commercial vehicle based on at least one of the point cloud data and the image; and an extraction unit configured to extract valid point cloud data from the point cloud data, which can be used to estimate the position from the point cloud data.In a case where the number of pieces of valid point cloud data is equal to or greater than a predetermined number, the estimation unit estimates the position of the commercial vehicle using the valid point cloud data, and in a case where the number of pieces of valid point cloud data is less than the predetermined number, the estimation unit estimates the position of the commercial vehicle using the image.

[0007] Furthermore, according to another aspect of the present disclosure, a position estimation method is provided for estimating the position of a commercial vehicle equipped with a laser scanner capable of capturing point cloud data by scanning a laser beam and a camera capable of capturing an image in a predetermined direction, the method comprising: a step of acquiring the point cloud data and the image; a step of estimating the position of the commercial vehicle based on the point cloud data and / or the image; and a step of extracting the valid point cloud data usable for estimating the position from the point cloud data.In the step of estimating the position, the position of the commercial vehicle is estimated using the valid point cloud data in a case where the number of elements of valid point cloud data is equal to or greater than a predetermined number, and the position of the commercial vehicle is estimated using the image in a case where the number of pieces of valid point cloud data is less than a predetermined number.

[0008] Furthermore, according to yet another aspect of the present disclosure, a program is provided which causes a computer in a position estimation device for estimating the position of a commercial vehicle, which is equipped with a laser scanner capable of capturing point cloud data by scanning a laser beam and a camera capable of capturing an image in a predetermined direction, to perform a step of acquiring the point cloud data and the image, a step of estimating the position of the commercial vehicle based on the point cloud data and / or the image, and a step of extracting valid point cloud data usable for estimating the position from the point cloud data.In the step of estimating the position, the position of the commercial vehicle is estimated using the valid point cloud data in a case where the number of elements of valid point cloud data is equal to or greater than a predetermined number, and the position of the commercial vehicle is estimated using the image in a case where the number of pieces of valid point cloud data is less than a predetermined number. Advantageous effects of the invention

[0009] A more robust self-position estimation can be achieved according to each of the aspects described above. List of characters Fig. Figure 1 is a view illustrating an overall configuration of a logistics support system according to one embodiment. Fig. Figure 2 is a view illustrating a functional configuration of a commercial vehicle according to one embodiment. Fig. Figure 3 is a view illustrating the processing flow of a control device according to one embodiment. Fig. Figure 4 is a view describing the processing of the control device according to one embodiment. Fig. Figure 5 is a view describing the processing of the control device according to one embodiment. Fig. Figure 6 is a view describing the processing of the control device according to one embodiment. Fig. Figure 7 is a view illustrating the processing flow of the control device according to one embodiment. Fig. Figure 8 is a view describing the processing of the control device according to one embodiment. Fig. Figure 9 is a view describing the processing of the control device according to one embodiment. Fig. Figure 10 is a view describing a function of a position estimating device according to one embodiment. Fig. Figure 11 is a view describing a function of the position estimating device according to one embodiment. Description of embodiments: First embodiment

[0010] Below, a position estimation device according to a first embodiment and a commercial vehicle incorporating this position estimation device are described with reference to Fig. 1 to Fig. 6 shown. Overall configuration of the logistics support system

[0011] Fig. Figure 1 is a view illustrating an overall configuration of a logistics support system according to the first embodiment. A in Fig. 1. Logistics support system shown 9performs the storage, delivery and relocation of goods while controlling a commercial vehicle that drives autonomously (for example, an unmanned forklift) in an indoor or semi-outdoor environment (for example, in a warehouse).

[0012] The logistics support system 9 includes a commercial vehicle 1 and a host device 2 a. In the present embodiment, the commercial vehicle 1 a vehicle that drives autonomously according to a command issued by the host device 2 is received, and is, for example, an unmanned forklift. The host device 2 gives a command to drive or to load and unload the commercial vehicle 1 off. The host device 2 receives various pieces of information sequentially from the commercial vehicle 1 and aggregates a placement position of goods in the warehouse or the position of the commercial vehicle 1.

[0013] In the following description, an XY plane of a Cartesian coordinate system is referred to as the "driving plane" of the commercial vehicle. 1 a ± Z-axis direction orthogonal to the driving plane is also referred to as the "height direction". Fig. Figure 1 illustrates an example in which the commercial vehicle 1 travels in the +X-axis direction on the driving plane.

[0014] As in Fig. As shown in 1, the commercial vehicle 1 a vehicle body 1A , a control device 10 , a laser scanner 11, a camera 12 and an internal field sensor 13.

[0015] The control device 10 controls an autonomous drive of the vehicle body 1A In particular, the control device controls 10 the vehicle body 1A, to travel along a predetermined path while performing self-position estimation. The specific configuration and modification of the control device. 10 will be described later.

[0016] The laser scanner 11 is, for example, a 2D LiDAR. In the present embodiment, the laser scanner 11 serves as a safety sensor. In other words, the laser scanner 11, which is a safety sensor, is designed to detect obstacles on the roadway and to warn the commercial vehicle. 1 to initiate an emergency stop to prevent a collision. In particular, the laser scanner 11 is located on the underside of the vehicle body. 1AThe laser scanner 11 is mounted and scans a laser beam over a predetermined scanning area, including a direction of travel. The laser beam is scanned along a plane parallel to the travel plane. The laser scanner 11 is preferably mounted at a height at which it can detect a person working in a posture lying on the travel plane. The laser scanner 11 scans the laser beam to capture point cloud data that indicates the direction and distance between the laser scanner 11 itself and an irradiation target of the laser beam.

[0017] Camera 12 is located on an upper section (+Z-direction side) of the vehicle body. 1A attached to be able to cover the area above the vehicle body 1Ato image. The camera 12 captures and collects light from outside through a lens in order to capture an image (hereinafter also referred to as the "field of view image") in a predetermined direction, which, for example, depicts a scene of the area above the vehicle body. 1A shows.

[0018] The in-field sensor 13, for example, is an inertial measurement unit (IMU) and is a sensor capable of measuring acceleration in three axes, angular velocity in three axes, and the like (hereinafter also referred to as "acceleration / angular velocity information") of the vehicle body. 1A to detect. In addition to the IMU, the internal field sensor 13 can be an encoder, a steering angle sensor, or the like, configured to detect the rotation angles of wheels. Functional configuration of a commercial vehicle

[0019] Fig. Figure 2 is a view illustrating a functional configuration of the commercial vehicle according to the first embodiment. As shown in Fig. 2 illustrates, the control device 10 of the commercial vehicle 1 a CPU 100 , a storage 101 , a mass storage device 102 , a wireless communication device 103 and a connection interface 104.

[0020] The CPU 100 is a processor that operates according to a program that has been prepared in advance to perform various functions, which will be described later. Mass storage 101 is a so-called main memory and is a mass storage area used by the CPU. 100 The processing is required. The mass storage 102is a so-called large-capacity auxiliary mass storage device, for example an HDD or an SSD. In the present embodiment, the mass storage device stores 102 Various pieces of information required for advance self-position estimation. In particular, the mass storage device stores... 102 Stock map information M1, characteristic point map information M2 and map information usable via the camera M3. The stock map information M1 is map information used to compare the point cloud data obtained from the laser scanner 11 in order to specify a self-position. The characteristic point map information M2 is map information that includes previously obtained characteristic point clouds and is used to match characteristic points of the field of view image obtained from camera 12 in order to specify the eigenposition. The map information M3 available via the camera is map information used to determine whether a self-position estimation should be performed using the field of view image. In the mass storage device 102 In the present embodiment, several map information M3s, which can be used via the camera depending on the time zone, are stored.

[0021] The wireless communication device 103 is a communication interface configured to provide wireless communication with the host device. 2 to perform ( Fig. 1).

[0022] The connection interface 104 is a connection interface configured to communicate with the control device. 10 , to communicate with the laser scanner 11, the camera 12 and the internal field sensor 13.

[0023] CPU functions 100 are described in detail. The CPU100 works according to a program to act as a data acquisition unit 1000 , extraction unit 1001 , first unit of determination 1002 , second unit of determination 1003 , unit of measurement 1004 and steering unit 1005 to function. Of these, the recording unit represents 1000 , the extraction unit 1001 , the first unit of determination 1002 , the second unit of determination 1003 and the unit of measurement 1004 the configuration of a position estimating device 100A that is configured to determine the vehicle's own position 1 to appreciate.

[0024] The recording unit 1000 captures the point cloud data of the laser scanner 11 and the field of view image of the camera 12. From the data collected by the data collection unit 1000 The extraction unit extracts data from the captured point cloud data. 1001Valid point cloud data that can be used to estimate self-position. Specific processing content of the extraction unit. 1001 will be described later. The first unit of determination 1002 Determines whether the estimation of one's own position should be carried out using the point cloud data. The second unit of determination. 1003 determines whether the estimation of self-position should be performed using the visual field image. The estimation unit 1004 estimates the position (own position) of the commercial vehicle 1 based on at least one of the point cloud data and the field-of-view image. The estimation unit 1004 selects based on the determination results of the first unit of determination 1002 and the second unit of determination 1003 the processing content of the self-positioning assessment accordingly. The steering unit 1005 causes the commercial vehicle 1, based on the estimation result of the own position by the position estimating device 100A , to travel along a predetermined path. Processing flow of a control device

[0025] Fig. Figure 3 is a view illustrating the processing sequence of the control device according to the first embodiment. Fig. 4 to Fig. Figure 6 shows views describing the processing of the control device according to the first embodiment. The in Fig. The processing sequence shown in point 3 takes place while the commercial vehicle is in motion. 1 iterated.

[0026] First, the recording unit captures data. 1000 the control device 10 various sensor data (step S01). In particular, the acquisition unit records 1000the point cloud data from the laser scanner 11, the field of view image from the camera 12 and further acquires acceleration and angular velocity information from the internal field sensor 13.

[0027] The extraction unit then extracts 1001 the tax authority 10 From the point cloud data acquired by laser scanner 11, the valid point cloud data that can be used for self-position estimation are extracted (step S02). Now the process of step S02, which is carried out by the extraction unit, is performed. 1001 is carried out with reference to Fig. 4 and Fig. 5 shown.

[0028] In Fig. 4 and Fig. 5 drives the commercial vehicle 1 in the +X direction. An origin O is an origin position of the commercial vehicle. 1 (Laser scanner 11). As in Fig. As illustrated in Figure 4, the laser scanner 11 repeatedly scans the laser beam along a predetermined scanning area Q. Additionally, point cloud data P is acquired, which indicates the direction and distance between the scanner's self-position (origin O) and the laser beam's irradiation target.

[0029] The point cloud data P are divided into point cloud data obtained by irradiating a permanently fixed object (e.g., the warehouse wall or a shelf) and point cloud data obtained by irradiating a temporarily existing object (e.g., another commercial vehicle or temporarily stored goods). When performing a self-positioning estimation, including point cloud data acquired by illuminating the temporarily existing object, the accuracy of the self-positioning estimation is reduced. This is due to the extraction unit. 1001The system selects from all point cloud data P acquired in step S01 by irradiating the temporarily existing object (hereinafter also referred to as "invalid point cloud data P2") and extracts the remaining point cloud data (hereinafter also referred to as "valid point cloud data P1"). Since the valid point cloud data P1 is point cloud data acquired by irradiating the storage wall or a shelf, it is point cloud data that can be used to estimate the self-position. In other words, the control device can 10 accurately estimate the self-position by performing a comparison with the stock map information M1 based only on the valid point cloud data P1 provided by the extraction unit 1001 were extracted. Fig. Figure 5 illustrates a result of excluding the invalid point cloud data P2 from the point cloud data P and extracting only the valid point cloud data P1.

[0030] A method for distinguishing whether each point cloud datum P is the valid point cloud datum P1 (point cloud datum derived from the warehouse wall or rack) or the invalid point cloud datum P2 (point cloud datum derived from other commercial vehicles or temporarily parked goods) is as follows. First, the extraction unit receives 1001 Administrative information provided by the host device 2 be held. The administrative information held by the host device 2 The information held includes the location of each commercial vehicle and the placement position of the goods at the present time. Next, the extraction unit closes. 1001Point cloud data derived from other commercial vehicles and point cloud data derived from temporarily placed goods, with reference to the own position estimated in the previous stage and the management information provided by the host device. 2 be received, out.

[0031] Returning to Fig. 3 determines the first unit of determination 1002 the control device 10 , whether a self-positioning estimation should be performed using the valid point cloud data. In particular, the first unit of determination is determined. 1002 , whether more than a predetermined number of valid point cloud data points remain in each of the subdivided areas within the laser beam's scan range (step S03). The process of step S03, which is carried out by the first determination unit 1002 is carried out with reference to Fig. 5 described in detail.

[0032] As in Fig. As shown in section 5, the first unit of determination is initially divided. 1002 The scanning range Q of the laser beam is divided into a multitude of regions. In the Fig. In the illustrated example 5, the sampling area Q is divided into three areas (subdivided areas R1, R2, and R3). Subdivided area R1 is an area in the direction of travel of the commercial vehicle. 1 On the left side of the scan area Q, the subdivided area R2 is an area in front of the commercial vehicle. 1 in scan area Q, and the subdivided area R3 is an area in the direction of travel of the commercial vehicle. 1 right side in scan area Q. Next, the first unit of determination is counted. 1002the number of pieces of valid point cloud data P1 that are included in each of the subdivided areas R1, R2 and R3, and determines whether more than a predetermined number of pieces of valid point cloud data P1 remain in all subdivided areas R1, R2 and R3.

[0033] Returning to Fig. 3, in a case where more than a predetermined number of pieces of valid point cloud data remain in each of the subdivided areas (step S03; JA), the estimation unit performs 1004 a self-positioning estimation using the valid point cloud data, after it has been determined that an accurate self-positioning estimation based on the valid point cloud data is possible (step S04). In particular, the estimating unit estimates 1004the own position by finding the layout (shape) of the wall or shelf extracted from the valid point cloud data captured by steps S01 to S02 from the stock map information M1 recorded in advance.

[0034] On the other hand, in a case where no more than a predetermined number of pieces of valid point cloud data remain in each of the subdivided areas (step S03; NO), the control device executes 10 a determination process using the second determination unit 1003 through, after it was determined, based on the valid point cloud data, that no accurate self-positioning estimate is possible. The second unit of determination 1003 the control device 10 Based on the visual field image, it determines whether a self-position estimation should be performed. Specifically, the second determining unit determines 1003Based on the map information M3 available via the camera (step S05), it is determined whether the previously estimated self-position belongs to an area available via the camera. The process of step S05, which is determined by the second determination unit, 1003 is carried out with reference to Fig. 6 described in detail.

[0035] As in Fig. Figure 6 illustrates that the map information M3 usable via the camera defines a camera-usable area A1 and a camera-usable area A2. The camera-usable area A2 is an area where the accuracy of self-position estimation based on the field of view is likely to deteriorate. Furthermore, the camera-usable area A1 is a different area than the camera-usable area A2 among all areas in the warehouse.

[0036] In the present embodiment, the area A2 usable by the camera is defined as an area that is likely to receive direct sunlight, based on the arrangement of the windows in the warehouse and the like. Therefore, the area A2 usable by the camera is defined as being near a side window or a roof window of the warehouse.

[0037] Furthermore, it is likely that the amount of directly received sunlight varies depending on the time of day. Therefore, the control device prepares 10 According to the present embodiment, several map information M3 usable via the camera (a morning map M3a, a day map M3b and an evening map M3b) are displayed and the map to be referenced changes depending on the time of day. In the morning map M3a, which is in Fig. As illustrated in Figure 6, the area A2 usable by the camera is defined as the vicinity of the east-facing window. In the daytime map M3b, the area A2 usable by the camera is defined as the area around the south-facing side window and the area directly below the skylight. Furthermore, in the evening map M3c, the area A2 usable by the camera is defined as the area around the west-facing window. Changing the map information to be referenced via the camera, which is dependent on the time of day (area usable via the camera), makes it possible to select a suitable method for self-position estimation, taking into account the time-of-day-dependent changing effects of direct sunlight or the like. Furthermore, “time-of-day-dependent switching” includes not only time-of-day-dependent switching (morning, day or evening) as in the present embodiment, but also switching, for example, depending on the turn of the year.

[0038] Returning to Fig. 3, in a case where the previously estimated position does not belong to the predefined area usable via the camera (step S05; JA), the estimation unit 1004 a self-position estimation using the visual field image is performed after it has been determined that an accurate self-position estimation based on the visual field image is possible (step S06). In particular, the estimating unit estimates 1004the self-position based on the characteristic points of the visual field image captured in step S01 and the previously created characteristic point map information M2 using a simultaneous localization and mapping technique (SLAM technique). Furthermore, camera 12 is positioned in such a way that it faces the area above the vehicle body. 1A As described above, since the camera is facing the direction of travel, it is unlikely that a temporary object (such as another commercial vehicle or temporarily placed goods) is included in the field of view captured by camera 12. As long as the commercial vehicle 1 Since the area A1, which can be used via the camera, is part of the camera, a sufficiently accurate self-position estimation is possible using the field of view image.

[0039] On the other hand, in a case where the previously estimated position belongs to the predefined area usable via the camera (step S05; NO), the estimation unit performs 1004 a self-position estimation by dead reckoning (autonomous navigation) using the internal field sensor 13 was performed after it was determined that an accurate self-position estimation based on the visual field image was not possible. However, errors accumulate during dead reckoning navigation with the internal field sensor 13, and it is not preferable to continue dead reckoning navigation for a predetermined distance or longer. The estimation unit 1004 It therefore first determines whether dead reckoning continues for a predetermined distance or longer (step S07). In a case where dead reckoning has not continued for a predetermined distance or longer (step S07; NO), the estimating unit performs 1004a self-position estimation based on dead reckoning navigation through (step S08). In a case where dead reckoning has continued for a predetermined distance or longer (step S07; YES), the control device stops. 10 (steering unit) 1005 ) the journey of the vehicle (step S09). Operational impact

[0040] As described above, the position estimating device estimates 100A (Control device) 10 ) according to the first embodiment, determines the position of the commercial vehicle using the valid point cloud data in a case where the number of elements of valid point cloud data is equal to or greater than a predetermined number, and estimates the position of the commercial vehicle using the field of view image in a case where the number of pieces of valid point cloud data is less than a predetermined number. In this way, a more accurate self-positioning estimation is possible using the valid point cloud data, and in the event that insufficient valid point cloud data can be acquired, it is also possible to compensate using the self-positioning estimation using the visual field image.

[0041] As described above, according to the position estimating device 100A According to the present embodiment, a more robust self-position estimation can be achieved.

[0042] Furthermore, the position estimation device 100A according to the first embodiment in each of the plurality of subdivided areas R1 to R3 (see Fig. 5) in a case where the number of elements of valid point cloud data belonging to each of the areas is equal to or greater than a predetermined number, the position of the commercial vehicle 1 estimated using valid point cloud data. If the point cloud data is focused on only a few limited regions, the possibility of a false match between the point cloud data and the stock mapping information M1 is high. Here, the configuration described above estimates the self-position using valid point cloud data that is evenly distributed across the sampling area, thus further increasing the accuracy of the self-position estimation.

[0043] The position estimating device 100A According to the first embodiment, the point cloud data derived from another commercial vehicle or goods is excluded from the captured point cloud data, and the remaining point cloud data is extracted as valid point cloud data. Point cloud data derived from a temporary object, such as another commercial vehicle or goods, can introduce noise when the point cloud data and the M1 inventory map information are reconciled during self-position estimation. Here, as described above, self-position is estimated solely based on point cloud data derived from a permanent object (a structure such as a wall or shelf), thus further increasing the accuracy of the self-position estimation.

[0044] The position estimating device 100A According to the first embodiment, the position of the commercial vehicle is estimated. 1using the field of view image in a case where the number of valid point cloud data is less than a predetermined number and the previous estimated position belongs to the predefined camera-usable area A1 (in a case where the estimated position does not belong to the camera-usable area A2). As long as the estimated position does not belong to the area usable via the camera, where an accurate self-position estimation based on the field of view becomes difficult, the self-position is estimated using the field of view, and thus it is possible to increase the accuracy of the self-position estimation through the field of view.

[0045] In a case where the previously estimated position belongs to the predefined area A2 usable via the camera (see Fig. 6) estimates the position estimating device 100Aaccording to the first embodiment, the position of the commercial vehicle 1 using the detection value of the internal field sensor 13, which is attached to the commercial vehicle 1 is mounted. Thus, even in a case where an accurate self-position estimation is difficult when either the point cloud data or the view image is used, the self-position can be estimated by dead reckoning using the acquisition value of the internal field sensor 13.

[0046] Furthermore, the control device stops 10 according to the first embodiment, the autonomous driving of the commercial vehicle 1 in a case where an obstacle in the direction of travel is detected based on the point cloud data captured by the laser scanner 11. As described above, a collision with the obstacle can be suppressed. Additionally, the commercial vehicle can 1, in which the laser scanner 11 is mounted as a safety sensor, the laser scanner 11 intended as a safety sensor is used as a sensor for self-position estimation.

[0047] Furthermore, the commercial vehicle 1 According to the first embodiment, the laser scanner 11 is attached to the underside of the vehicle body. In this way, obstacles present in the driving plane can be easily detected to increase safety. Furthermore, if the laser scanner 11 is located on the underside of the vehicle body 1AWhile a temporary object, such as another commercial vehicle or payload, is readily available, the accuracy of the self-positioning estimation using point cloud data can degrade. However, in the present embodiment, as described above, the self-positioning estimation can be compensated for by self-positioning using the field-of-view image in cases where the number of elements of valid point cloud data does not meet a predetermined threshold. Thus, accurate self-positioning can be achieved even when the laser scanner 11 is located on the underside of the vehicle body. 1A is planned.

[0048] In the commercial vehicle 1 According to the first embodiment, the camera 12 is mounted in such a way that it is able to view the area above the vehicle body. 1A to depict. This configuration reduces the frequency with which a temporary object (another commercial vehicle or goods) is included in the field of view. Therefore, the accuracy of self-position estimation using the field of view can be further increased. Second embodiment

[0049] Next, a position estimation device according to a second embodiment and a commercial vehicle incorporating this position estimation device are described with reference to Fig. 7 to Fig. 9 shown. Processing flow of a control device

[0050] Fig. Figure 7 is a view illustrating the processing flow of a control device according to the second embodiment. Fig. 8 and Fig. Figure 9 shows views describing the processing of the control device according to the second embodiment. Similar to the first embodiment, the Fig. 7. Illustrated processing sequence during the journey of the commercial vehicle 1 iterated.

[0051] In the Fig. The illustrated processing flow in step 7 shows the process from step S05 in the process flow ( Fig. 3) of the first embodiment is replaced by the process of step S05a. In particular, the second unit of determination determines 1003 According to the second embodiment, when determining whether a self-position estimation using the field-of-view image is to be performed, it is necessary to check whether the characteristic points of the field-of-view image acquired in step S01 are uniformly distributed (step S05a). The process of step S05a is described below with reference to Fig. 8 and Fig. 9 shown.

[0052] For example, in one case of a Fig. In the illustrated field-of-view image PA, the characteristic points are enclosed in fifteen subdivided image areas within sixteen subdivided image areas, which are divided into a grid shape. On the other hand, in one case of a Fig. 9 illustrated field of view image PB the characteristic points in only seven subdivided image areas included among the sixteen subdivided image areas that are divided into a grid shape. In this way, the second unit of determination is determined 1003 , whether the characteristic points of the visual field image are evenly distributed, based on the ratio of the number of divided image areas, including the characteristic points, to the total number (sixteen) divided image areas. For example, if a threshold for determining 1 / 2 is determined by the second unit of determination 1003, that the characteristic points for the visual field image PA are evenly distributed, and determined that the characteristic points for the visual field image PB are not evenly distributed.

[0053] Returning to Fig. 7, in a case where the characteristic points of the visual field image are evenly distributed (step S05a; YES) the estimation unit 1004 a self-position estimation using the visual field image is performed after it has been determined that an accurate self-position estimation based on the visual field image is possible (step S06). On the other hand, in a case where the characteristic points of the visual field image are not evenly distributed (step S05a; NO), the estimation unit 1004a self-position estimation by dead reckoning (autonomous navigation) using the internal field sensor 13 was performed after it was determined that an accurate self-position estimation based on the visual field image was not possible. Operational impact

[0054] As described above, the position estimating device estimates 100A According to the second embodiment, the position of the commercial vehicle 1 using the field of view image in a case where the characteristic points extracted from the field of view image are evenly distributed.

[0055] On the other hand, the position estimating device estimates 100A the position of the commercial vehicle 1 using the detection value of the internal field sensor 13, which is located in the commercial vehicle 1is mounted in a case where the characteristic points extracted from the field of view image are not evenly distributed.

[0056] In a case where the characteristic points reflected in the visual field image are affected, the accuracy of matching the characteristic points with the characteristic point map information M2 decreases, and the self-position may be incorrectly detected. Given this, the configuration described above can improve the self-position estimation accuracy using the visual field image. Third embodiment

[0057] Next, a position estimation device according to a third embodiment and a commercial vehicle incorporating this position estimation device are described with reference to Fig. 10 described.

[0058] Fig. Figure 10 is a view describing a function of the position estimation device according to the third embodiment.

[0059] The estimation unit 1004 the position estimating device 100A According to the present embodiment, in addition to the functions of the position estimating device, it also has 100A The following functions are available according to the first and second embodiments. In other words, the estimation unit 1004 as a sensor used for self-position estimation, and causes a smoothing filter to work with a self-position estimation result when the laser scanner 11 and the camera 12 are switched. The in Fig. Example 10 illustrates a case where, at time ta, the self-position estimation performed by laser scanner 11 is switched to the self-position estimation performed by camera 12. In this case, due to the error caused by switching the sensor type, the self-position changes stepwise before and after time ta. The estimation unit 1004 According to the present embodiment, the smoothing filter operates at the time when the sensor type is switched (i.e., when the position estimation based on the valid point cloud data and the position estimation based on the field of view image are switched), and changes the change smoothly in steps. In this way it is possible to suppress discontinuous changes in the self-position estimation result and to suppress the occurrence of unwanted abrupt steering inputs during autonomous driving. Additionally, in another embodiment, the estimation unit can be 1004 Perform the same filtering process at the time of switching between position estimation based on valid point cloud data and position estimation based on dead reckoning, or at the time of switching between position estimation using the field of view image and position estimation based on dead reckoning. Fourth embodiment

[0060] Next, a position estimation device according to a fourth embodiment and a commercial vehicle incorporating this position estimation device are described with reference to Fig. 11 described.

[0061] Fig. Figure 11 is a view describing a function of the position estimation device according to the fourth embodiment. The estimation unit 1004 the position estimating device 100A According to the first to third embodiments, it is described as performing a self-position estimation while alternately selecting the sensor type (the laser scanner 11 and the camera 12) according to the situation. On the other hand, as in Fig. Figure 11 illustrates the position estimating device 100AAccording to the fourth embodiment, a result (hereinafter also referred to as "mixed self-position estimation result C3") is obtained by mixing a self-position estimation result C1, obtained using the laser scanner 11 (point cloud data), and a self-position estimation result C2, obtained using the camera 12 (field of view image), in a predetermined ratio. The position estimation device 100A According to the present embodiment, the mixed self-positioning estimation result C3 calculated in this way gives the position of the commercial vehicle. 1 out of. In this way, it is possible to perform a self-position estimation that is more robust against unexpected disturbances than in a case where a self-position estimation is performed with only one sensor.

[0062] Furthermore, the position estimation device 100AAccording to the fourth embodiment, the mixing ratio of the self-position estimation result C1, obtained using the laser scanner 11, and the self-position estimation result C2, obtained using the camera 12, is dynamically changed according to the situation. For example, if in the estimation unit 1004 According to a modification of the present embodiment, the number of elements of valid point cloud data that are in the subdivided areas R1 to R3 (see Fig. 5) included, is large, the mixing can be done so that the weight of the self-positioning estimation result C1 increases relatively, and if the number of elements of valid point cloud data included in the subdivided areas R1 to R3 is small, the mixing can be done so that the weight of the self-positioning estimation result C2 increases relatively. Furthermore, the estimation unit can 1004, if the previous self-position is not included in the area A1 usable via the camera (see Fig. 6) the mixing is carried out in such a way that the weight of the self-position estimation result C1 increases relatively, and if the previous self-position is in the area usable via the camera A1 (see Fig. 6) is included, the mixing can be carried out in such a way that the weight of the self-position estimation result C2 increases relatively. Other modifications

[0063] Details were provided above for the position estimation device. 100A , the control device 10 and the commercial vehicle 1 as described in the first to fourth embodiments, however, the specific aspects are not limited thereto, and a variety of design changes and the like may be added within a scope that does not deviate from the core of the disclosure.

[0064] The camera 12 according to the first embodiment is mounted in such a way that it faces the area directly above the vehicle body. 1A is oriented towards, but other embodiments are not limited to this aspect. For example, according to another embodiment, the camera 12 can be mounted so that it faces an area diagonally above the vehicle body. 1A is oriented towards. Furthermore, for example, in a case where sufficient accuracy of the position estimation can be ensured even if a temporary object is included in the field of view, the camera 12 can be mounted so that it is aligned with the horizontal direction of the vehicle body. 1A is facing the camera. Furthermore, camera 12 can be an omnidirectional camera or the like.

[0065] The position estimation device according to the first embodiment was intended for the purpose of controlling the journey of the autonomously driving commercial vehicle. 1(unmanned forklift) mounted as described, but other embodiments are not limited to this aspect. In another embodiment, the position estimating device 100A be mounted on a manned forklift truck, which is driven by a person and performs a driving operation, for example to record the history of driving paths.

[0066] The position estimating device 100A According to another embodiment, the device can be provided with both the technical features of the first to third embodiments (alternatively a selection function) and the technical features of the fourth embodiment (the function of mixing in a predetermined ratio). In this case, the position estimation device can be... 100A It may be configured to switch which function is activated, depending on the operator's actions or similar.

[0067] In the embodiment described above, the sequences of various processes of the position estimation device are 100A and the control device 10 The program is stored on a computer-readable recording medium, with each of the processes described above being performed by a computer that reads and executes this program. Examples of computer-readable recording media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, and semiconductor storage devices. Furthermore, this computer program can be delivered to the computer via a communication circuit, and the computer receiving this delivery can execute the program.

[0068] The program can be a program for implementing some of the functions described above. Additionally, the functions described above can be implemented in combination with a program already stored on the computer system, a so-called differential file (differential program).

[0069] Certain embodiments of the disclosure have been described above, but all these embodiments are merely illustrative and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the core of the invention. These embodiments and modifications are included in the scope and core of the invention and are also included in the scope of the invention as described in the claims and equivalents thereof. Notes

[0070] The position estimation device, the control device, the commercial vehicle, the logistics support system, the position estimation method and the program described in each embodiment are understood, for example, as follows. (1) According to a first point of view, the position estimating device 100A to estimate the position of the commercial vehicle 1 provided, which is equipped with the laser scanner 11, which is capable of capturing the point cloud data by scanning the laser beam, and the camera 12, which is capable of capturing the image in the predetermined direction, wherein the position estimation device 100A This includes: the recording unit 1000 , which is configured to capture the point cloud data and the image; where the estimation unit 1004 is configured to determine the position of the commercial vehicle 1to estimate based on at least one of the point cloud data and the image; and the extraction unit 1001 It is configured to extract the valid point cloud data from the point cloud data that can be used to estimate the position. If a number of pieces of valid point cloud data is equal to or greater than a predetermined number, the estimation unit 1004 The estimating unit estimates the position of the commercial vehicle using the valid point cloud data, and in a case where the number of pieces of valid point cloud data is less than the predetermined number, the estimating unit estimates the position of the commercial vehicle using the image. (2) In the position estimating device 100A From a second perspective, the estimating unit estimates 1004 for each of the multitude of areas formed by dividing the scanning area of ​​the laser beam, the position of the commercial vehicle 1using the valid point cloud data in a case where the number of pieces of valid point cloud data belonging to each area is equal to or greater than the predetermined number. (3) In the position estimation device 100A According to a third point of view, the extraction unit 1001 The point cloud data derived from a temporarily existing object is extracted from the point cloud data, and the remaining point cloud data is extracted as valid point cloud data. (4) In the position estimating device 100A According to a fourth perspective, the estimating unit estimates 1004 In a case where the number of pieces of valid point cloud data is less than the predetermined number and a previous estimated position belongs to the predefined area A1 usable via the camera, the position of the commercial vehicle 1 using the image. (5) In the position estimating device 100A According to a fifth point, the unit of estimation changes 1004 The area A1 usable via the camera depends on the time of day. (6) In the position estimating device 100A According to a sixth consideration, in a case where the number of pieces of valid point cloud data is less than the predetermined number and the previously estimated position does not belong to the predefined area A1 usable via the camera, the estimation unit estimates 1004 the position of the commercial vehicle 1 using the detection value of the internal field sensor 13, which is located in the commercial vehicle 1 is mounted. (7) In the position estimating device 100AAccording to a seventh consideration, in a case where the number of pieces of valid point cloud data is less than the predetermined number and characteristic points extracted from the image are evenly distributed, the estimation unit estimates 1004 the position of the commercial vehicle 1 using the image. (8) In the position estimating device 100A According to an eighth point of view, in a case where the number of pieces of valid point cloud data is less than the predetermined number and the characteristic points extracted from the image are not uniformly distributed, the estimation unit estimates 1004 the position of the commercial vehicle 1 using the detection value of the internal field sensor 13, which is located in the commercial vehicle 1 is mounted. (9) In the position estimating device 100A According to a ninth point of view, the unit of estimation is determined1004 based on the number of subdivided image areas including the characteristic points, whether the characteristic points are evenly distributed, wherein the subdivided image areas are formed by subdividing the image into a predetermined number. (10) In the position estimating device 100A According to a tenth point of view, the estimation unit leads 1004 a process of smooth transition of a position estimation result in a case where the position estimation based on the valid point cloud data and the position estimation based on the image are switched. (11) In the position estimating device 100A According to an eleventh point of view, the estimating unit estimates 1004 the position of the commercial vehicle 1using the mixing result, in a predetermined ratio, the position estimation result based on the valid point cloud data and the position estimation result based on the image. (12) According to a twelfth point, the position estimating device 100A to estimate the position of the commercial vehicle 1 provided, which is equipped with the laser scanner 11, which is capable of capturing the point cloud data by scanning the laser beam, and the camera 12, which is capable of capturing the image in the predetermined direction, wherein the position estimation device 100A This includes: the recording unit 1000 , which is configured to capture the point cloud data and the image; where the estimation unit 1004 is configured to determine the position of the commercial vehicle 1to estimate based on at least one of the point cloud data and the image; and the extraction unit 1001 It is configured to extract valid point cloud data from the point cloud data, which can be used to estimate the position. The estimation unit 1004 estimates the position of the commercial vehicle 1 using the mixing result, in a predetermined ratio, the position estimation result based on the valid point cloud data and the position estimation result based on the image. (13) In the position estimating device 100A According to a thirteenth point of view, the unit of estimation changes 1004 the relationship dynamically according to a situation. (14) In the position estimating device 100A According to a fourteenth point of view, the estimation unit leads 1004a mixing is performed such that the weight of the position estimation result C1 based on the valid point cloud data increases relatively when a previous self-position is not included in a predetermined area A1 usable via the camera, and mixing is performed such that the weight of the position estimation result C2 based on the image increases relatively when the previous self-position is included in the area A1 usable via the camera. (15) According to a fifteenth point, a control device 10 to control the autonomous driving of the commercial vehicle 1 provided, the control device including the following: the position estimating device 100A after one of (1) to (14); and the steering unit 1005 , which is configured to cause the commercial vehicle 1 moved based on the estimated position along a predetermined path. (16) In the control device 10 According to a sixteenth point, the steering unit stops. 1005 the autonomous driving of the commercial vehicle in a case where an obstacle is detected in the direction of travel based on the point cloud data. (17) According to a seventeenth point, a commercial vehicle 1 provided, which includes the following: the control device 10 according to (15) or (16). (18) In the commercial vehicle 1 According to an eighteenth point of view, the laser scanner 11 is located on the underside of the vehicle body. 1A appropriate. (19) In the commercial vehicle 1 According to a nineteenth point of view, camera 12 is positioned in such a way that it is able to cover the area above the vehicle body. 1A to depict. (20) According to a twentieth point, the logistics support system 9 provided, including: the commercial vehicle1 according to (17) to (19); and the host device 2 , which is configured to hold the administrative information that specifies the location of each commercial vehicle and the placement position of luggage at any given time. (21) According to a twenty-first consideration, a position estimation method is used to estimate the position of the commercial vehicle. 1 provided, which is equipped with the laser scanner 11, which is capable of capturing the point cloud data by scanning the laser beam, and the camera 12, which is capable of capturing the image in the predetermined direction, wherein the method includes: a step of capturing the point cloud data and the image; a step of estimating the position of the commercial vehicle 1based on at least one of the point cloud data and the image; and a step of extracting, from the point cloud data, the valid point cloud data that can be used to estimate the position. In the position estimation step, the position of the commercial vehicle is determined. 1 The position of the commercial vehicle is estimated using the valid point cloud data in a case where the number of pieces of valid point cloud data is equal to or greater than a predetermined number, and the position of the commercial vehicle is estimated using the image in a case where the number of pieces of valid point cloud data is less than the predetermined number. (22) According to a twenty-second point, a program is provided which a computer in the position estimating device 100A to estimate the position of the commercial vehicle 1caused, which is equipped with the laser scanner 11, which is able to capture the point cloud data by scanning the laser beam and the camera 12, which is able to capture the image in the predetermined direction, to perform a step of capturing the point cloud data and the image, a step of estimating the position of the commercial vehicle 1 based on the point cloud data and / or the image, and a step of extracting, from the point cloud data, the valid point cloud data that can be used to estimate the position, wherein in the step of estimating the position the position of the commercial vehicle 1 using the valid point cloud data, in a case where the number of pieces of valid point cloud data is equal to or greater than a predetermined number, and the position of the commercial vehicle 1estimating using the image in a case where the number of pieces of valid point cloud data is less than the predetermined number. Reference symbol list 1 commercial vehicle 1A vehicle body 10 Control device 100 CPU 100A Position Estimator 1000 recording units 1001 extraction units 1002 First unit of determination 1003 Second unit of determination 1004 unit of estimation 1005 Steering unit 101 storage 102 mass storage devices 2 Host device 9 Logistics Support System QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2019074505 A

[0003]

Claims

[1] Position estimation device for estimating the position of a commercial vehicle, which is equipped with a laser scanner capable of capturing point cloud data by scanning a laser beam and a camera capable of capturing an image in a predetermined direction, wherein the position estimation device comprises: a capture unit configured to capture the point cloud data and the image; an estimation unit configured to estimate the position of the commercial vehicle based on at least one of the point cloud data and the image; and an extraction unit configured to extract valid point cloud data from the point cloud data, which can be used to estimate the position, wherein In a case where the number of pieces of valid point cloud data is equal to or greater than a predetermined number, the estimation unit estimates the position of the commercial vehicle using the valid point cloud data, and in a case where the number of pieces of valid point cloud data is less than the predetermined number, the estimation unit estimates the position of the commercial vehicle using the image. [2] Position estimation device according to claim 1, wherein the estimation unit for each of a plurality of areas formed by dividing a scanning area of ​​the laser beam estimates the position of the commercial vehicle using the valid point cloud data in a case where a number of pieces of the valid point cloud data belonging to each area is equal to or greater than the predetermined number. [3] Position estimation device according to claim 1 or 2, wherein the extraction unit excludes point cloud data derived from a temporarily existing object from the point cloud data and extracts the remaining point cloud data as valid point cloud data. [4] Position estimation device according to one of claims 1 to 3, wherein in a case where a number of pieces of the valid point cloud data is less than the predetermined number and a previous estimated position belongs to a predefined area usable via the camera, the estimation unit estimates the position of the commercial vehicle using the image. [5] Position estimation device according to claim 4, wherein the estimation unit changes the area usable via the camera depending on the time of day. [6] Position estimation device according to claim 4 or 5, wherein in a case where a number of pieces of the valid point cloud data is less than the predetermined number and the previous estimated position does not belong to the predefined area usable via the camera, the estimation unit estimates the position of the commercial vehicle using a detection value of an internal field sensor mounted in the commercial vehicle. [7] Position estimation device according to one of claims 1 to 3, wherein in a case where a number of pieces of the valid point cloud data is less than the predetermined number and characteristic points extracted from the image are evenly distributed, the estimation unit estimates the position of the commercial vehicle using the image. [8] Position estimation device according to claim 7, wherein in a case where a number of pieces of the valid point cloud data is less than the predetermined number and the characteristic points extracted from the image are not uniformly distributed, the estimation unit estimates the position of the commercial vehicle using a detection value from an internal field sensor mounted in the commercial vehicle. [9] Position estimation device according to claim 7 or 8, wherein the estimation unit determines whether the characteristic points are evenly distributed, based on the number of subdivided image areas which include the characteristic points, wherein the subdivided image areas are formed by subdividing the image into a predetermined number. [10] Position estimation device according to any one of claims 1 to 9, wherein in a case where the position estimation is switched between the valid point cloud data and the position estimation is switched between the image, the estimation unit performs a smooth transition process of a position estimation result. [11] Position estimation device according to one of claims 1 to 10, wherein the estimation unit estimates the position of the commercial vehicle using the mixing result, in a predetermined ratio, the position estimation result based on the valid point cloud data and the position estimation result based on the image. [12] Position estimation device for estimating the position of a commercial vehicle, which is equipped with a laser scanner capable of capturing point cloud data by scanning a laser beam and a camera capable of capturing an image in a predetermined direction, wherein the position estimation device comprises: a capture unit configured to capture the point cloud data and the image; an estimation unit configured to estimate the position of the commercial vehicle based on at least one of the point cloud data and the image; and an extraction unit configured to extract valid point cloud data from the point cloud data, which can be used to estimate the position, wherein The estimation unit estimates the position of the commercial vehicle using a mixed result, in a predetermined ratio, a position estimation result based on the valid point cloud data, and a position estimation result based on the image. [13] Position estimating device according to claim 11 or 12, wherein the estimating unit dynamically changes the ratio depending on a situation. [14] Position estimation device according to claim 13, wherein the estimation unit performs a mixing such that a weight of the position estimation result based on the valid point cloud data increases relatively when a previous self-position is not included in a predetermined area usable via the camera, and performs a mixing such that a weight of the position estimation result based on the image increases relatively when the previous self-position is included in the area usable via the camera. [15] Control device for controlling autonomous journeys of a commercial vehicle, the control device comprising: Position estimation device according to any one of claims 1 to 14; and a steering unit configured to cause the commercial vehicle to travel along a predetermined path based on an estimated position result. [16] Control device according to claim 15, wherein the steering unit stops the autonomous driving of the commercial vehicle in a case where an obstacle in the direction of travel is detected based on the point cloud data. [17] Commercial vehicle comprising: control device according to claim 15 or 16. [18] Commercial vehicle according to claim 17, wherein the laser scanner is attached to an underside of a vehicle body. [19] Commercial vehicle according to claim 17 or 18, wherein the camera is mounted in such a way that it can image an area above the vehicle body. [20] Logistics support system, comprehensive: Commercial vehicle according to one of claims 17 to 19; and a host device configured to hold administrative information that specifies the location of each commercial vehicle and the placement position of goods at a given time. [21] Position estimation method for estimating the position of a commercial vehicle equipped with a laser scanner capable of capturing point cloud data by scanning a laser beam and a camera capable of capturing an image in a predetermined direction, the method comprising: a step in capturing the point cloud data and the image; a step of estimating the position of the commercial vehicle based on the point cloud data and / or the image; and a step of extracting valid point cloud data from the point cloud data that can be used to estimate the position, wherein in the step of estimating the position the position of the commercial vehicle is estimated using the valid point cloud data in a case where a number of pieces of the valid point cloud data is equal to or greater than a predetermined number, and the position of the commercial vehicle is estimated using the image in a case where a number of pieces of the valid point cloud data is less than the predetermined number. [22] Program which causes a computer in a position estimation device for estimating the position of a commercial vehicle, which is equipped with a laser scanner capable of capturing point cloud data by scanning a laser beam and a camera capable of capturing an image in a predetermined direction, to execute the following: one step of capturing the point cloud data and the image, a step of estimating the position of the commercial vehicle based on the point cloud data and / or the image, and a step of extracting valid point cloud data from the point cloud data that can be used to estimate the position, wherein in the step of estimating the position the position of the commercial vehicle is estimated using the valid point cloud data in a case where a number of pieces of the valid point cloud data is equal to or greater than a predetermined number, and the position of the commercial vehicle is estimated using the image in a case where a number of pieces of the valid point cloud data is less than the predetermined number.

Images