Self-propelled transport system

Abstract

To prevent a problem occurring in transportation caused by a self-propelled vehicle.SOLUTION: A self-propelled transportation system 2 acquires space information (first space information) of a vehicle 40 by a first sensor 10 provided separately from the vehicle 40 in a space where a transportation of the vehicle 40 is performed, and also, acquires space information (second space information) of the vehicle 40 using a second sensor 20 that has a different modality from the first sensor 10. A processor 31 of the self-propelled transportation system 2 calculates a first position of the vehicle 40 in a predetermined coordinate system based on the first space information, and calculates a second position of the vehicle 40 in the predetermined coordinate system based on the second space information. Then, the processor 31 determines a deviation between the first position and the second position, and creates a control instruction for the vehicle 40 based on at least one of the first position and the second position, provided that the deviation is within an allowable range.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an autonomous transport system that makes a vehicle move autonomously for transportation.

Background Art

[0002] An autonomous transport system that makes a vehicle move autonomously for transportation is known. As a related prior art of the autonomous transport system, for example, the technology disclosed in Patent Document 1 can be cited. Patent Document 1 discloses a technology for remotely controlling a plurality of micromobilities traveling in an automatic operation area by a fixed infrastructure device. The fixed infrastructure device includes a lidar that detects targets within a detection range defined in the automatic operation area. The fixed infrastructure device uses the detection information of the targets detected by the lidar to generate a travel route of the micromobility traveling within the detection range, and transmits a control command based on the travel route to the micromobility.

[0003] In addition, as documents showing the technical level of the technical field related to the present disclosure, Patent Document 2 and Patent Document 3 can be exemplified in addition to Patent Document 1.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] In an autonomous transport system, the vehicle's position is necessary to create control instructions for the vehicle. The applicant is considering acquiring images of the vehicle using fixed cameras installed in the space through which the vehicle is transported, and calculating the vehicle's position based on those images. However, during the process of this consideration, it became clear that, due to limitations in installing fixed cameras, not all locations along the vehicle's transport route can necessarily be captured by the cameras. If there are locations where the vehicle's position cannot be calculated, problems will occur in the autonomous transport of the vehicle.

[0006] This disclosure has been made in view of the above-mentioned issues. One objective of this disclosure is to provide a self-propelled transport system that prevents malfunctions from occurring during the self-propelled transport of vehicles. [Means for solving the problem]

[0007] The self-propelled transport system of this disclosure comprises two types of sensors, namely a first sensor and a second sensor. The first sensor is a sensor that acquires spatial information of a vehicle. The second sensor is also a sensor that acquires spatial information of a vehicle. However, the second sensor is a sensor of a different modality than the first sensor. Therefore, the spatial information of the vehicle acquired by the first sensor (hereinafter referred to as the first spatial information) and the spatial information of the vehicle acquired by the second sensor (hereinafter referred to as the second spatial information) are of different types of information. The first sensor may be installed separately from the vehicle in the space in which the vehicle is transported.

[0008] The self-propelled transport system of this disclosure comprises at least one processor that processes first spatial information and second spatial information, and at least one memory that stores a plurality of instructions to be executed by the at least one processor. These plurality of instructions are configured to cause the at least one processor to perform a predetermined process. The process to be performed by the processor includes calculating a first position of the vehicle in a predetermined coordinate system based on the first spatial information, and calculating a second position of the vehicle in the same predetermined coordinate system based on the second spatial information. Furthermore, the process to be performed by the processor includes determining the difference between the first position and the second position, and creating a control instruction for the vehicle based on at least one of the first position and the second position, provided that the difference is within an acceptable range.

[0009] The processing performed by the processor may include creating control instructions based on either a first position or a second position. The processing performed by the processor may also include switching the vehicle position that forms the basis of the control instructions from the first position to the second position, or from the second position to the first position, provided that the deviation is within an acceptable range. Furthermore, if the deviation is outside an acceptable range, the processing performed by the processor may include instructing the vehicle to stop or take evasive action. [Effects of the Invention]

[0010] According to the self-propelled transport system of this disclosure, the position of a vehicle can be calculated using two types of sensors with different modalities. For creating control instructions for the vehicle, either a first position calculated based on spatial information acquired by the first sensor or a second position calculated based on spatial information acquired by the second sensor can be used. However, the condition for using either the first or second position is that the difference between the first and second positions is within an acceptable range, so that malfunctions in the vehicle's self-propelled transport due to the influence of positional deviation on control instructions are prevented. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of a self-propelled transport system according to an embodiment of the present disclosure. [Figure 2] This diagram illustrates the switching of vehicle positions used to calculate control instructions. [Figure 3] This figure shows an example of a positional deviation within the acceptable range. [Figure 4] This figure shows an example of a positional deviation outside the acceptable range. [Figure 5] This is a block diagram showing the details of the configuration of a self-propelled transport system according to an embodiment of this disclosure. [Figure 6] This is a flowchart showing the processes performed by the self-propelled transport system according to the embodiment of this disclosure. [Modes for carrying out the invention]

[0012] 1. Overview of the Self-Propelled Transport System Figure 1 shows an overview of the self-propelled transport system 2 according to the embodiment of this disclosure. The self-propelled transport system 2 is a system that transports a vehicle 40 with self-propulsion capabilities from a departure point P1 to a destination P2. The self-propelled transport system 2 can be constructed, for example, as a system for moving finished vehicles in a factory or warehouse. The self-propelled transport system 2 can also be constructed as an automated valet parking system.

[0013] The autonomous transport system 2 is equipped with two types of sensors for acquiring spatial information of the vehicle 40. The difference between the two types of sensors is the difference in modality. Specifically, the first sensor equipped with the autonomous transport system 2 is a camera 10 that acquires video of the vehicle 40 as spatial information of the vehicle 40. The second sensor equipped with the autonomous transport system 2 is a lidar 20 that acquires three-dimensional information of the vehicle 40 as spatial information of the vehicle 40.

[0014] At least one camera 10 is installed in the space where the vehicle 40 is transported. Specifically, camera 10 is positioned to provide an overview of the transport route of vehicle 40 from the departure point P1 to the destination P2. However, depending on the application of the self-propelled transport system 2, due to constraints on the installation of camera 10, it may not always be possible to install camera 10 so that the entire transport route is visible.

[0015] At least one LiDAR 20 is installed in the space where the vehicle 40 is transported. The installation requirements for LiDAR 20 are less stringent than those for camera 10. Therefore, LiDAR 20 can sometimes be installed in locations where it is difficult to install camera 10. Through the cooperation of camera 10 and LiDAR 20, spatial information of vehicle 40 is acquired along the entire transport route of vehicle 40.

[0016] The self-propelled transport system 2 is equipped with a server 30. The server 30 transmits control instructions to the vehicle 40 to enable it to move autonomously. The control instructions include instruction values ​​related to the driving, braking, and steering of the vehicle 40. The information that the server 30 uses as the basis for creating the control instructions is information about the position of the vehicle 40. The position of the vehicle 40 can be calculated from the spatial information of the vehicle 40. Therefore, the server 30 receives video as the first spatial information from the camera 10 and three-dimensional information as the second spatial information from the LiDAR 20. Based on the received spatial information, the server 30 calculates the position of the vehicle 40 and creates control instructions for the vehicle 40 based on the position of the vehicle 40.

[0017] Server 30 includes at least one processor 31 (hereinafter simply referred to as processor 31). Further, server 30 includes at least one memory 32 (hereinafter simply referred to as memory 32) communicably coupled to processor 31. Memory 32 includes a program storage area 33 and a data storage area 34. The program storage area 33 stores a plurality of instructions including an instruction for causing processor 31 to calculate the position of vehicle 40 and an instruction for causing processor 31 to create a control instruction. The data storage area 34 stores data necessary for the execution of instructions and temporarily stores data such as images acquired from camera 10 and three-dimensional information acquired from lidar 20.

[0018] 2. Details of the autonomous transport system For creating the control instruction, at least one of the position of vehicle 40 calculated from the image acquired by camera 10 (hereinafter sometimes referred to as the first vehicle position) and the position of vehicle 40 calculated from the three-dimensional information acquired by lidar 20 (hereinafter sometimes referred to as the second vehicle position) can be used. In the present embodiment, server 30 uses either the first vehicle position calculated based on the image of camera 10 or the second vehicle position calculated based on the three-dimensional information of lidar 20.

[0019] As described above, due to the constraints in installing camera 10, there may be areas on the transport route of vehicle 40 that cannot be photographed by camera 10. In the present embodiment, it is assumed that there is an area outside the shooting area of camera 10 on the transport route of vehicle 40, and lidar 20 is installed so as to scan that area. Specifically, as shown in FIG. 2, shooting areas CMR1 and CMR2 of camera 10 are set along the autonomous transport route ATR of vehicle 40. The shooting area CMR1 and the shooting area CMR2 are areas photographed by different cameras 10, and the two are not connected. A scanning area LDR of lidar 20 is set between the shooting area CMR1 and the shooting area CMR2. There is an overlap between the scanning area LDR and the shooting area CMR1, and there is also an overlap between the scanning area LDR and the shooting area CMR2.

[0020] When vehicle 40 is within the imaging area CMR1, server 30 can calculate the position of vehicle 40 from the video. When vehicle 40 moves into the scanning area LDR, server 30 can calculate the position of vehicle 40 from the three-dimensional information. Then, when vehicle 40 moves into the imaging area CMR2, server 30 can again calculate the position of vehicle 40 from the video. In this embodiment, server 30 creates control instructions based on either the first vehicle position or the second vehicle position. Therefore, between the start point L1 and the end point L2 of the overlap between imaging area CMR1 and scanning area LDR, server 30 needs to switch the position of vehicle 40 used as the basis for creating control instructions from the first vehicle position to the second vehicle position. Also, between the start point L3 and the end point L4 of the overlap between scanning area LDR and imaging area CMR2, server 30 needs to switch the position of vehicle 40 used as the basis for creating control instructions from the second vehicle position to the first vehicle position.

[0021] However, the first vehicle position calculated based on the image from camera 10 and the second vehicle position calculated based on the three-dimensional information from LiDAR 20 do not necessarily coincide. If the discrepancy between the first and second vehicle positions is large, the behavior of vehicle 40 may become unstable when switching the vehicle position which forms the basis for creating control instructions. Therefore, the server 30 is required to determine whether the positional discrepancy between the first and second vehicle positions is within an acceptable range before switching from the first to the second vehicle position, or vice versa. An acceptable positional discrepancy means a positional discrepancy within which the change in the behavior of vehicle 40 that occurs when switching vehicle positions remains within an acceptable range for vehicle 40 to move autonomously within the premises of a factory, warehouse, parking lot, etc.

[0022] An example of a method for determining whether a positional deviation is within an acceptable range will be explained using Figures 3 and 4. Figures 3 and 4 show two bounding boxes OBJ1 and OBJ2 projected onto a predetermined coordinate space. Bounding box OBJ1 shows the position of the vehicle 40 obtained from object recognition processing of the image from camera 10. Bounding box OBJ2 shows the position of the vehicle 40 obtained from object recognition processing of the three-dimensional information from LiDAR 20. Figures 3 and 4 also show the centroids CNT1 and CNT2 of each bounding box OBJ1 and OBJ2.

[0023] In the example shown in Figure 3, the centroids CNT1 and CNT2 of the two bounding boxes OBJ1 and OBJ2 are close together, whereas in the example shown in Figure 4, the two centroids CNT1 and CNT2 are far apart. Therefore, one way to determine whether the positional deviation is within an acceptable range is to use the distance between the centroids CNT1 and CNT2 of the bounding boxes OBJ1 and OBJ2. For example, if the distance between the centroids is below a threshold, the positional deviation may be determined to be within an acceptable range. On the other hand, if the distance between the centroids is greater than a threshold, the positional deviation may be determined to be outside an acceptable range.

[0024] Furthermore, in the example shown in Figure 3, the two bounding boxes OBJ1 and OBJ2 overlap, whereas in the example shown in Figure 4, the two bounding boxes OBJ1 and OBJ2 do not overlap. Therefore, another method for determining whether the positional deviation is within an acceptable range is to use the overlap rate between bounding boxes OBJ1 and OBJ2. For example, if the overlap rate is below a threshold, the positional deviation may be determined to be within an acceptable range. On the other hand, if the overlap rate is greater than the threshold, the positional deviation may be determined to be outside an acceptable range.

[0025] Figure 5 is a functional block diagram of the self-propelled transport system 2, focusing on its functions including the determination of vehicle position deviation. As shown in Figure 5, the server 30 includes a first vehicle position calculation unit 301, a second vehicle position calculation unit 302, a position deviation determination unit 303, and a control instruction creation unit 304. The memory 32 stores instructions for the processor 31 to execute the processing performed by these elements 301-304. That is, when the processor 31 executes the instructions corresponding to these elements 301-304, the processor 31 executes the processing defined in these elements 301-304.

[0026] The first vehicle position calculation unit 301 acquires video of the vehicle 40 from the infrastructure camera 10. The infrastructure camera 10 and the server 30 are connected by wired or wireless means via communication devices 15 and 35. The first vehicle position calculation unit 301 performs predetermined object recognition processing on the video acquired from the infrastructure camera 10 and recognizes the vehicle 40 included in the video. Then, the first vehicle position calculation unit 301 projects the vehicle 40 recognized from the video onto a predetermined coordinate system and calculates the first vehicle position in that coordinate system.

[0027] The second vehicle position calculation unit 302 acquires three-dimensional information of the vehicle 40 from the LiDAR 20. The LiDAR 20 and the server 30 are connected by wired or wireless means via communication devices 25 and 35. The second vehicle position calculation unit 302 applies predetermined object recognition processing to the three-dimensional information acquired from the LiDAR 20 and recognizes the vehicle 40 included in the three-dimensional information. Then, the second vehicle position calculation unit 302 projects the vehicle 40 recognized from the three-dimensional information onto the same coordinate system as the coordinate system in which the first vehicle position is represented, and calculates the second vehicle position in that coordinate system.

[0028] The position deviation determination unit 303 determines whether the deviation between the first vehicle position and the second vehicle position is within an acceptable range. For example, the method described with reference to Figures 3 and 4 may be used for this determination. The result of the position deviation determination is input to the control instruction creation unit 304.

[0029] The control instruction creation unit 304 creates control instructions to be given to the vehicle 40 based on the vehicle's position. If the vehicle 40 is within the shooting range of the camera 10 but not within the scanning range of the LiDAR 20, an image containing the vehicle 40 is obtained, but three-dimensional information containing the vehicle 40 is not. In this case, the control instruction creation unit 304 creates a control instruction based on a first vehicle position calculated from the image. Conversely, if the vehicle 40 is not within the shooting range of the camera 10 but within the scanning range of the LiDAR 20, an image containing the vehicle 40 is not obtained, but three-dimensional information containing the vehicle 40 is obtained. In this case, the control instruction creation unit 304 creates a control instruction based on a second vehicle position calculated from the three-dimensional information.

[0030] When vehicle 40 is within the shooting area of ​​camera 10 and also within the scanning area of ​​LiDAR 20, both an image containing vehicle 40 and three-dimensional information containing vehicle 40 can be obtained. In this case, the control instruction creation unit 304 performs processing according to the determination result of the position deviation determination unit 303. If the position deviation determination unit 303 determines that the position deviation is within an acceptable range, the control instruction creation unit 304 selects either the first vehicle position or the second vehicle position and creates a control instruction based on the selected vehicle position. For example, when vehicle 40 moves from the shooting area of ​​camera 10 to the scanning area of ​​LiDAR 20, the control instruction creation unit 304 may switch the vehicle position used as the basis for creating the control instruction from the first vehicle position to the second vehicle position at the time vehicle 40 enters the scanning area. Also, when vehicle 40 moves from the scanning area of ​​LiDAR 20 to the shooting area of ​​camera 10, the control instruction creation unit 304 may switch the vehicle position used as the basis for creating the control instruction from the second vehicle position to the first vehicle position at the time vehicle 40 enters the shooting area.

[0031] If the position deviation determination unit 303 determines that the position deviation is outside the acceptable range, the control instruction creation unit 304 creates an emergency control instruction. The emergency control instruction is a control instruction to prevent the behavior of the vehicle 40 from becoming unstable due to a change in the vehicle's position while a position deviation is occurring. Specifically, a stop instruction is created to instruct the vehicle 40 to stop in place. Alternatively, instead of a stop instruction, an evacuation instruction may be created to instruct the vehicle 40 to move to a nearby evacuation location.

[0032] The control instruction creation unit 304 transmits the created control instructions to the vehicle 40 via the communication device 35 of the server 30. The communication device 35 is wirelessly connected to the receiver 45 of the vehicle 40. The vehicle 40 is equipped with an actuator control unit 401. The actuator control unit 401 controls the drive actuator, brake actuator, and steering actuator according to the received control instructions.

[0033] Although not shown in Figure 5, the positional misalignment determination result by the positional misalignment determination unit 303 may also be used to determine sensor abnormalities. In other words, if both the camera 10 and the lidar 20 are functioning correctly, the positional misalignment between the first vehicle position and the second vehicle position should be within the acceptable range. Therefore, if the positional misalignment is outside the acceptable range, it can be determined that either a malfunction has occurred in at least one of the camera 10 and the lidar 20, or that there is a discrepancy in the settings of at least one of the camera 10 and the lidar 20. For example, in the case shown in Figure 2, suppose a positional misalignment occurs between the first vehicle position calculated from the image of the shooting area CMR1 and the second vehicle position calculated from the three-dimensional information of the scanning area LDR. In that case, it can be determined that an abnormality has occurred in at least one of the camera 10 that is capturing the shooting area CMR1 and the lidar 20 that is scanning the scanning area LDR.

[0034] When the vehicle 40 is stopped or instructed to take an evasive action by a control instruction issued by the control instruction creation unit 304, it is necessary to restore the self-propelled transport system 2 as quickly as possible. Therefore, if the position deviation detection unit 303 detects that a position deviation has occurred, it may be configured to contact the administrator of the self-propelled transport system 2. In this case, the administrator may be notified of which sensor has malfunctioned based on the results of the sensor abnormality detection.

[0035] The processes performed by the self-propelled transport system 2 can be represented by a flowchart as shown in Figure 6. In this flowchart, flow F100 represents the process performed in camera 10. Flow F200 represents the process performed in lidar 20. Flow F300 represents the process performed in server 30. And flow F400 represents the process performed in vehicle 40. Each flow is executed asynchronously with respect to the others at predetermined intervals.

[0036] As shown in flow F100, step S101 is executed in camera 10. In step S101, video including vehicle 40 is acquired. The acquired video is then sent to server 30.

[0037] As shown in flow F200, step S201 is executed in the lidar 20. In step S201, three-dimensional information including the vehicle 40 is acquired. The acquired three-dimensional information is then sent to the server 30.

[0038] As shown in flow F300, the server 30 first executes step S301. In step S301, it is determined whether both video and three-dimensional information have been acquired.

[0039] If only one of the video or three-dimensional information is acquired, the process proceeds from step S301 to step S302. In step S302, the vehicle position is calculated from the acquired video or three-dimensional information. Then, a control instruction is created based on the calculated vehicle position. After the execution of step S302, the process proceeds to step S303. In step S303, the control instruction created in step S302 is transmitted to the vehicle 40.

[0040] If both video and three-dimensional information are acquired, the process proceeds from step S301 to step S304. In step S304, the position of the first vehicle is calculated based on the video, and the position of the second vehicle is calculated based on the three-dimensional information. After step S304 is completed, the process proceeds to step S305. In step S305, it is determined whether the positional difference between the first and second vehicle positions is within an acceptable range.

[0041] If the positional deviation is within an acceptable range, the process proceeds from step S305 to step S306. In step S306, either the first vehicle position or the second vehicle position is selected. Then, a control instruction is created based on the selected vehicle position. After the execution of step S306, the process proceeds to step S303. In step S303, the control instruction created in step S306 is transmitted to the vehicle 40.

[0042] If the positional deviation is outside the acceptable range, the process proceeds from step S305 to step S307. In step S307, a stop instruction is created to stop the vehicle 40. After step S307 is executed, the process proceeds to step S303. In step S303, the stop instruction created in step S306 is sent to the vehicle 40 as a control instruction.

[0043] As shown in flow chart F400, in vehicle 40, step S401 is executed first. In step S401, it is determined whether a control instruction has been received. If a control instruction has been received, the process proceeds to step S402. In step S402, various actuators are controlled based on the control instruction received in step S306.

[0044] 3. Other Embodiments Vehicle 40 may be equipped with a remote operation kit to enable remote manual operation by a remote operator. If vehicle 40 is equipped with a remote operation kit, if the positional deviation is outside the acceptable range, the transport of vehicle 40 may be switched from self-propelled transport by transmission of control instructions to transport by remote manual operation.

[0045] In the areas where the imaging area CMR1 and the scanning area LDR overlap, as shown in Figure 2, and in the areas where the scanning area LDR and the imaging area CMR2 overlap, both video and three-dimensional information are acquired. In such areas, an intermediate position between the first vehicle position and the second vehicle position may be calculated, and control instructions may be created based on that intermediate position. Alternatively, the vehicle position that forms the basis for creating control instructions may be gradually switched from the first vehicle position to the second vehicle position, or from the second vehicle position to the first vehicle position.

[0046] The combination of sensors applicable to the autonomous transport system is not limited to a stationary camera and a stationary LiDAR. For example, if the first sensor is a stationary camera, the second sensor may be an on-board LiDAR. Alternatively, radar can be used as the second sensor. Radar can be stationary or on-board. If the second sensor is radar, the first sensor may be LiDAR. Alternatively, the second sensor may be an on-board GPS or a magnetic sensor installed on the road surface. In short, the first and second sensors only need to be able to acquire spatial information of the vehicle and have different sensor modalities. [Explanation of symbols]

[0047] 2...Self-propelled transport system, 10...Camera, 20...LiDAR, 30...Server, 40...Vehicle

Claims

1. An autonomous transport system that transports vehicles by driving them under its own power, A first sensor that acquires spatial information of the vehicle, A sensor for acquiring spatial information of the vehicle, comprising a second sensor having a different modality from the first sensor, A processor that processes first spatial information of the vehicle acquired by the first sensor and second spatial information of the vehicle acquired by the second sensor, The system comprises at least one memory storing a plurality of instructions executed by the at least one processor, The plurality of instructions are provided to at least one processor, Based on the first spatial information, the first position of the vehicle in a predetermined coordinate system is calculated, Calculating the second position of the vehicle in the predetermined coordinate system based on the second spatial information, The deviation between the first position and the second position is determined, Provided that the aforementioned deviation is within an acceptable range, a control instruction for the vehicle is created based on at least one of the first position and the second position. If the deviation is outside the allowable range, the system is configured to instruct the vehicle to stop or take evasive action. A self-propelled transport system characterized by the following features.

2. An autonomous transport system that transports vehicles by driving them under its own power, A first sensor is provided in the space where the vehicle is transported, separately from the vehicle, to acquire spatial information of the vehicle. A sensor for acquiring spatial information of the vehicle, comprising a second sensor having a different modality from the first sensor, A processor that processes first spatial information of the vehicle acquired by the first sensor and second spatial information of the vehicle acquired by the second sensor, The system comprises at least one memory storing a plurality of instructions executed by the at least one processor, The plurality of instructions are provided to at least one processor, Based on the first spatial information, the first position of the vehicle in a predetermined coordinate system is calculated, Calculating the second position of the vehicle in the predetermined coordinate system based on the second spatial information, The deviation between the first position and the second position is determined, Provided that the aforementioned deviation is within an acceptable range, the system is configured to create a control instruction for the vehicle based on at least one of the first position and the second position, and to perform the following actions. A self-propelled transport system characterized by the following features.

3. In the self-propelled transport system according to claim 2, The first sensor is a camera that acquires images as the first spatial information, The second sensor is a lidar that acquires three-dimensional information as the second spatial information. A self-propelled transport system characterized by the following features.

Images