Remote control device

Abstract

A remote control device, comprising: at least one processor configured to perform processing, the processing including: obtaining three-dimensional point cloud data measured using a distance measurement device; estimating at least one of a position and an orientation of a moving body in the three-dimensional point cloud data by matching a template point cloud indicating the moving body with the three-dimensional point cloud data; determining a start position at which to start matching the template point cloud with the three-dimensional point cloud data; and generating a control command for remotely controlling the moving body using at least one of the estimated position and the estimated orientation of the moving body, and transmitting the control command to the moving body.

Description

Technical Field

[0001] The present disclosure relates to a remote control device. Background Art

[0002] Techniques for detecting the position of an object in space are known. For example, Japanese Unexamined Patent Application Publication No. 11-259658 (JP 11-259658A) discloses a technique in which a two-dimensional projection image showing the side shape of a vehicle is generated from an input image obtained by a camera, the generated projection image is sequentially scanned after placing a two-dimensional model stored in a memory at the foremost point of the two-dimensional projection image, and the position of the vehicle is identified by performing matching at each scanning position. Summary of the Invention

[0003] In the related art, scanning is sequentially performed after placing a two-dimensional model stored in a memory at the foremost point of a two-dimensional projection image. Therefore, there is a problem that it takes a long period of time to complete the matching. Therefore, in a technique for detecting the position of a moving body by matching a model stored in a memory, a technique for shortening the period of time required to detect the position of the moving body is needed.

[0004] The present disclosure can be implemented in the following aspects.

[0005] A remote control device, comprising: at least one processor configured to perform processing including: acquiring three-dimensional point cloud data measured using a distance measurement device; estimating at least one of the position and orientation of a moving body in the three-dimensional point cloud data by matching a template point cloud indicating the moving body with the three-dimensional point cloud data; determining a start position at which to start matching the template point cloud with the three-dimensional point cloud data; and generating a control command for remotely controlling the moving body using at least one of the estimated position and the estimated orientation of the moving body, and transmitting the control command to the moving body.

[0006] The remote control device of this aspect can quickly detect the position of the moving body by determining the start position of the matching near the moving body. Therefore, the processing period required to detect the position of the moving body by template matching can be shortened.

[0007] In the remote control device of the above aspect, the processing may further include: determining the start position using information about the position of the moving body in the three-dimensional point cloud data.

[0008] The remote control device of this aspect can quickly detect the position of the moving body by determining the start position of the template matching using information about the position of the moving body. Therefore, the processing period required for the template matching can be shortened.

[0009] In the remote control device of the above aspect, the processing may further include: obtaining a previous matching position where a previous match between the three-dimensional point cloud data and the template point cloud has been completed, as information about the position of the moving body, and using the obtained previous matching position to determine a starting position.

[0010] With the remote control device of this aspect, the processing period required for template matching can be shortened by a simple process of using a previous matching position that is likely to be near the position of the moving body at the time of the current match.

[0011] In the remote control device of the above aspect, the processing may further include: estimating the moved position of the moving body that has moved from the previous matching position during the period from the time when the previous match was completed to the time before the current match is executed; and using the estimated moved position to determine a starting position.

[0012] The remote control device of this aspect can set the starting position of the match to be closer to the position of the moving body at the time of the current match by estimating the moved position of the moving body. Therefore, the processing period required for template matching can be further shortened.

[0013] The remote control device of the above aspect may further include: a storage device configured to store the target route of the moving body as information about the position of the moving body, and the processing may further include: determining an arbitrary position on the target route as the starting position.

[0014] The remote control device of this aspect can quickly detect the position of the moving body by setting the starting position on the target route where the moving body is likely to exist. Therefore, the processing period required for template matching can be shortened.

[0015] In the remote control device of the above aspect, the processing may further include: obtaining a previous matching position where a previous match between the three-dimensional point cloud data on the target route and the template point cloud has been completed, and using the obtained previous matching position on the target route to determine a starting position.

[0016] The remote control device of this aspect can use the previous matching position on the target route where the moving body is more likely to exist to more quickly detect the position of the moving body.

[0017] In the remote control device of the above aspect, the processing may further include: using the position of the moving body detected from the three-dimensional point cloud data by a predetermined object detection method, or the position of the moving body detected based on the difference between the point cloud data associated with the background of the three-dimensional point cloud data and the point cloud data associated with the moving body, as information about the position of the moving body, to determine a starting position.

[0018] The remote control device of this aspect can use object detection to extract the temporary position of a moving body and quickly detect the position of the moving body by a simple method. Therefore, the processing period required for template matching can be shortened.

[0019] In the remote control device of the above aspect, the processing may further include: when the positions of a plurality of moving bodies are detected, using the position of the moving body associated with the moving body identification information for identifying the moving body among the detected positions of the moving bodies to determine the start position.

[0020] Even when the temporary positions of a plurality of moving bodies are detected by object detection, the remote control device of this aspect can extract the position of the moving body to be controlled using the moving body identification information.

[0021] In the remote control device of the above aspect, the distance measuring device may be configured to communicate with at least one processor, and the distance measuring device may be configured to transmit three-dimensional point cloud data to the remote control device.

[0022] In the remote control device of the above aspect, the distance measuring device may be installed around the traveling road of the moving body.

[0023] The remote control device of the above aspect may further include a distance measuring device.

[0024] The present disclosure may also be implemented in various forms other than the remote control device. For example, the present disclosure may be implemented in the form of a remote control system, a method for moving a moving body, a method for manufacturing a moving body, a method for detecting a moving body, a moving body detection device, a method for controlling a remote control system, a method for controlling a remote control device, a computer program for implementing these control methods, or a non-transitory recording medium recording the computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Hereinafter, the features, advantages, techniques, and industrial significance of exemplary embodiments of the present invention will be described with reference to the drawings, in which like reference numerals denote like elements, and in which:

[0026] Figure 1 A schematic configuration of a remote control system is shown;

[0027] Figure 2 A vehicle traveling method is schematically shown;

[0028] Figure 3 is a flowchart showing a processing routine of vehicle traveling processing by remote control;

[0029] Figure 4 is a flowchart showing a processing routine of start position determination processing according to the first embodiment;

[0030] Figure 5 shows the estimated position of the vehicle after traveling from the previous matching position;

[0031] Figure 6 is a block diagram showing the internal functional configuration of the remote control device according to the second embodiment;

[0032] Figure 7 is a flowchart showing the start position determination process according to the second embodiment;

[0033] Figure 8 shows the template matching in the start position determination process according to the second embodiment;

[0034] Figure 9 is a top view showing an example of the overall configuration of the remote control device according to the third embodiment; and

[0035] Figure 10 is a block diagram showing the internal functional configuration of the remote control device according to the third embodiment. DETAILED DESCRIPTION

[0036] A. First Embodiment

[0037] Figure 1 shows a schematic configuration of a remote control system 500 including a remote control device 300 according to the first embodiment of the present disclosure. The remote control device 300 generates a control command for autonomously driving the vehicle 100 by remote control and transmits it to the vehicle 100. For example, the remote control system 500 is used in a factory for manufacturing the vehicle 100 that can travel by remote control.

[0038] Examples of the vehicle 100 include passenger cars, trucks, buses, and construction vehicles. The vehicle 100 is preferably a Battery Electric Vehicle (BEV). The vehicle 100 is not limited to a pure electric vehicle and may be, for example, a gasoline vehicle, a hybrid electric vehicle, or a fuel cell electric vehicle, etc. The vehicle 100 includes a vehicle communication device 190, an actuator 140, and an Electronic Control Unit (ECU) 200.

[0039] The ECU 200 is installed on the vehicle 100 and executes various types of control in the vehicle 100. The ECU 200 includes a storage device 220 (such as a hard disk drive (HDD), a solid state drive (SSD), an optical recording medium, or a semiconductor memory), a central processing unit (CPU) 210, and an interface circuit 230. The CPU 210, the storage device 220, and the interface circuit 230 are connected via an internal bus in a manner that enables two-way communication. The actuator 140 and the vehicle communication device 190 are connected to the interface circuit 230. The vehicle communication device 190 wirelessly communicates with devices outside the vehicle 100 (such as the remote control device 300) connected to the network via an access point in the factory and the like.

[0040] The read / write area of the storage device 220 stores a computer program for implementing at least a part of the functions provided in this embodiment. Functions such as the driving control unit 212 are implemented by the CPU 210 executing various computer programs stored in the memory.

[0041] The driving control unit 212 executes the driving control of the vehicle 100. "Driving control" refers to various types of control for driving the actuator 140 that executes the "traveling", "steering", and "stopping" functions of the vehicle 100, typically the adjustment of acceleration, speed, and steering angle. In this embodiment, the actuator 140 includes an actuator of a driving device for accelerating the vehicle 100, an actuator of a steering device for changing the traveling direction of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100. The driving device includes a battery, a traveling motor driven by the battery power, and a driving wheel rotated by the traveling motor. The actuator of the driving device includes the traveling motor. The actuator 140 may also include an actuator for a wiper that shakes the vehicle 100 and an actuator for an electric window that opens and closes the vehicle 100, etc.

[0042] When the driver is in the vehicle 100, the driving control unit 212 can make the vehicle 100 travel by controlling the actuator 140 in response to the driver's operation. The driving control unit 212 can also make the vehicle 100 travel by controlling the actuator 140 in response to a control command transmitted from the remote control device 300, regardless of whether the driver is in the vehicle 100.

[0043] The remote control system 500 includes one or more vehicle detectors 80 and a remote control device 300. The vehicle detector 80 is a device for measuring various types of data to be used for estimating at least one of the position of the vehicle 100 and the orientation of the vehicle 100. The vehicle detector 80 is a Light Detection And Ranging (LiDAR) sensor, which is a distance measurement device for measuring the three-dimensional point cloud data of the vehicle 100. The three-dimensional point cloud data indicates the three-dimensional positions of the point cloud. By using the LiDAR sensor, high-precision three-dimensional point cloud data can be obtained. The orientation and traveling direction of the vehicle 100 can be estimated by using the vehicle detector 80 to obtain only the position of the vehicle 100 and the changes of the vehicle 100 over time.

[0044] The vehicle detector 80 is connected to the remote control device 300 to communicate with the remote control device 300 wirelessly or via a wire. By obtaining the three-dimensional point cloud data from the vehicle detector 80, the remote control device 300 can obtain the position and orientation of the vehicle 100 relative to the target route RT in real time. In this embodiment, the position of the vehicle detector 80 is fixed near the traveling road SR, and the relative relationship between the reference coordinate system Σr and the device coordinate system of the vehicle detector 80 is known. A coordinate transformation matrix for mutually transforming the coordinate values of the reference coordinate system ∑r and the coordinate values of the device coordinate system of the vehicle detector 80 is pre-stored in the remote control device 300.

[0045] The remote control device 300 performs driving control of the vehicle 100 through remote control. During the manufacturing process in the factory, the remote control device 300 transports the vehicle 100 by, for example, making the vehicle 100 drive autonomously through remote control. The transportation of the vehicle 100 using autonomous driving through remote control is also referred to as "autonomous driving transportation". Autonomous driving transportation allows the vehicle 100 to move without using transportation devices such as cranes or conveyor belts. The remote control device 300 can monitor the surroundings of the vehicle 100.

[0046] The remote control device 300 includes a central processing unit (CPU) 310, a storage device 340, an interface circuit 350, and a remote communication device 390. The CPU 310, the storage device 340, and the interface circuit 350 are connected via an internal bus in a manner that enables two-way communication. The remote communication device 390 is connected to the interface circuit 350. The remote communication device 390 communicates with the vehicle 100 via a network or the like.

[0047] The storage device 340 is, for example, a Random Access Memory (RAM), a Read Only Memory (ROM), a HDD, or an SSD. The read / write area of the storage device 340 stores vehicle point cloud data VP, a target route RT, an actuator drive history AC, and a previous matching position BM. The target route RT is a predetermined driving route of the vehicle 100 on a driving road SR.

[0048] The vehicle point cloud data VP serves as a template point cloud for estimating at least one of the position and orientation of the vehicle 100. For example, the three-dimensional Computer-Aided Design (CAD) data of the vehicle 100 can be used as the vehicle point cloud data VP. The vehicle point cloud data VP may include information for determining the orientation of the vehicle 100. By performing template matching using the vehicle point cloud data VP, the position and orientation of the vehicle 100 in the three-dimensional point cloud data can be estimated with high accuracy.

[0049] The actuator drive history AC is a history of input values and output values of each actuator 140 of the vehicle 100. The actuator drive history AC can be, for example, a history of control command values transmitted from the remote control device 300 to the vehicle 100, or actual measurement values obtained from detectors of the vehicle 100, such as vehicle speed, steering angle, braking force, and rotation angle. As will be described later, the previous matching position BM is the coordinate value of the position where the previous template matching between the three-dimensional point cloud data and the vehicle point cloud data VP performed by the position estimation unit 318 has been completed.

[0050] The storage device 340 stores a computer program for implementing at least a part of the functions provided in this embodiment. The CPU 310 executes the computer program stored in the storage device 340 to act as the remote control unit 312, the point cloud data acquisition unit 314, the start position determination unit 316, and the position estimation unit 318. Some or all of these functions can be implemented by hardware circuits. The point cloud data acquisition unit 314 acquires the three-dimensional point cloud data measured by the vehicle detector 80.

[0051] The start position determination unit 316 determines the start position at which to start matching the vehicle point cloud data VP with the acquired three-dimensional point cloud data. From the perspective of quickly completing the template matching, the start position at which to start the matching is preferably the position of the vehicle 100 to be detected in the three-dimensional point cloud data or near the position of the vehicle 100.

[0052] In this embodiment, the start position determination unit 316 uses the information about the position of the vehicle 100 in the three-dimensional point cloud data to determine the start position of the template matching. The "information about the position of the vehicle 100 in the three-dimensional point cloud data" is data to be used for estimating the position of the vehicle 100 or a position near the position of the vehicle 100 in the three-dimensional point cloud data. The "information about the position of the vehicle 100 in the three-dimensional point cloud data" is obtained by a method different from the template matching, and it is used in the processing before performing the template matching. The "information about the position of the vehicle 100 in the three-dimensional point cloud data" is preferably small-sized data, data obtained by simple processing, etc., to improve the processing speed of the template matching.

[0053] The position estimation unit 318 estimates the position and orientation of the vehicle 100 in the acquired three-dimensional point cloud data. In this embodiment, the position estimation unit 318 estimates the position and orientation of the vehicle 100 in the three-dimensional point cloud data by performing template matching on the three-dimensional point cloud data using the vehicle point cloud data VP. For example, an Iterative Closest Point (ICP) algorithm or a Normal Distribution Transform (NDT) algorithm can be adopted to perform template matching between the vehicle point cloud data VP and the three-dimensional point cloud data.

[0054] The remote control unit 312 uses the estimated position and orientation of the vehicle 100 to generate a control command for remote control and transmits it to the vehicle 100. The control command is a command for causing the vehicle 100 to travel along the target route RT stored in the storage device 340. The control command can be generated as a command including a driving force or a braking force and a steering angle. Alternatively, the control command can be generated as a command including at least one of the position and orientation of the vehicle 100 and a future travel route. When the vehicle 100 receives a remote control request, the driving control unit 212 of the ECU 200 implements the driving control, and as a result, the vehicle 100 automatically travels.

[0055] Figure 2 Schematically shows a driving method for the vehicle 100. Figure 2 Shows how the autonomous driving transportation of the vehicle 100 is performed in the factory FC where the vehicle 100 is manufactured. The factory FC includes a first process 50 and a second process 60. The first process 50 is, for example, a place where the vehicle 100 is assembled, and the second process 60 is, for example, a place where the vehicle 100 is inspected. The first process 50 and the second process 60 are connected by a driving road SR, and the vehicle 100 can travel on the driving road SR. Any position in the factory FC is represented by the xyz coordinate values of the reference coordinate system Σr.

[0056] The remote control device 300 causes the vehicle 100 to travel along the target route RT while estimating the position and orientation of the vehicle 100 in the three-dimensional point cloud data acquired by the vehicle detector 80 at a predetermined time interval. In Figure 2 In the example of, the target route RT set on the travel road SR includes successively connected target routes RT1, RT2, and RT3. A plurality of vehicle detectors 80 are installed around the travel road SR, including vehicle detectors 80A, 80B, and 80C respectively associated with the target routes RT1, RT2, and RT3.

[0057] The remote control device 300 appropriately switches the vehicle detector 80 based on the position of the vehicle 100, estimates the position and orientation of the vehicle 100, and causes the vehicle 100 to travel along the target route RT. In Figure 2 In the example of, the vehicle 100 that has started from the first process 50 travels along the target route RT1 by remote control using the three-dimensional point cloud data acquired by the vehicle detector 80A. When the vehicle 100 has reached the target route RT2, the vehicle detector 80A is switched to the vehicle detector 80B suitable for detecting the vehicle 100 traveling on the target route RT2. When the vehicle 100 has reached the target route RT3, similarly, the vehicle detector 80B is switched to the vehicle detector 80C suitable for detecting the vehicle 100.

[0058] Figure 3 is a flowchart showing a processing routine of the travel processing of the vehicle 100 by remote control according to the present embodiment. When the vehicle 100 starts to travel on the travel road SR, this process starts. This process can be repeated each time the point cloud data acquisition unit 314 newly acquires three-dimensional point cloud data from the vehicle detector 80 that performs measurement on the vehicle 100 to be controlled.

[0059] In step S100, the point cloud data acquisition unit 314 acquires three-dimensional point cloud data from the vehicle detector 80. In step S200, the start position determination unit 316 performs start position determination processing. The "start position determination processing" is a process for determining the start position at which the vehicle point cloud data VP is to be matched with the acquired three-dimensional point cloud data.

[0060] In step S300, template matching starts from the start position determined by the start position determination process. When the matching with the vehicle point cloud data VP cannot be completed at the start position, the matching is sequentially performed near the start position. The template matching is repeated while changing the matching position until completion. In step S400, the position estimation unit 318 estimates the position and orientation of the vehicle 100 in the three-dimensional point cloud data by performing template matching using the vehicle point cloud data VP. In step S500, the remote control unit 312 uses the estimated position and orientation of the vehicle 100 to generate a control command for causing the vehicle 100 to travel along the target route RT, and transmits it to the vehicle 100.

[0061] Figure 4 is a flowchart showing an example of a processing routine of the start position determination process according to the first embodiment. In the present embodiment, depending on whether there is a previous matching result, the previous matching position BM where the previous matching between the three-dimensional point cloud data and the vehicle point cloud data VP has been completed or the target route RT of the vehicle 100 is selectively used as "information on the position of the vehicle 100 in the three-dimensional point cloud data".

[0062] In step S210, the start position determination unit 316 checks whether there is a matching result of the previous template matching. For example, it can be checked whether there is a previous matching result based on whether the previous matching result is stored in the processing history stored in the storage device 340 or whether the previous matching position BM is stored in the storage device 340.

[0063] When there is no previous matching result (S210: No), the start position determination unit 316 advances the process to step S250. In this case, the target route RT of the vehicle 100 is used as "information on the position of the vehicle 100 in the three-dimensional point cloud data". In step S250, the start position determination unit 316 acquires the target route RT stored in the storage device 340.

[0064] In step S260, the travel start position of the vehicle 100 on the target route RT is acquired. The travel start position of the vehicle 100 on the target route RT is, for example, stored in the storage device 340 in advance in association with the target route RT. In Figure 2 the example, the travel start position of the vehicle 100 is near the exit of the first process 50.

[0065] In step S270, the start position determination unit 316 determines the start position of the template matching as the start position of travel on the acquired target route RT, and terminates this process. When there is no previous matching result, it is very likely that the vehicle 100 has not yet started traveling and is still at the start position of travel. Therefore, when there is no previous matching result, the processing period of the template matching can be reduced by setting the start position of travel of the vehicle 100 on the target route RT as the start position.

[0066] When there is a previous matching result in step S210 (S210: Yes), the start position determination unit 316 advances the processing to step S220. In this case, the previous matching position BM at which the previous matching between the three-dimensional point cloud data and the vehicle point cloud data VP has been completed is used as "information on the position of the vehicle 100 in the three-dimensional point cloud data". In the present embodiment, the position after travel of the vehicle 100 at the time of executing the current matching is further estimated using the previous matching position BM, and the position after travel is used as "information on the position of the vehicle 100 in the three-dimensional point cloud data".

[0067] In step S220, the start position determination unit 316 acquires the actuator drive history AC of the vehicle 100. The acquired actuator drive history AC includes at least the drive history of the actuator 140 related to the travel of the vehicle 100 during the period from the time when the previous matching was completed to the time before the current matching is executed. In step S230, the start position determination unit 316 estimates the position after travel of the vehicle 100 during the period from the time when the previous matching was completed to the time before the current matching is executed. Specifically, the start position determination unit 316 estimates the position after travel of the vehicle 100 by estimating the travel route from the previous matching position BM using the drive history of the actuator 140. In step S240, the start position determination unit 316 determines the estimated position after travel of the vehicle 100 as the start position of the template matching, and terminates this process.

[0068] Figure 5 The estimated position after travel of the vehicle 100 from the previous matching position is shown. Figure 5 An example of the three-dimensional point cloud data PD1 acquired from the vehicle detector 80 is shown. The three-dimensional point cloud data PD1 schematically shows the position BM1 of the vehicle 100 at the time of the previous matching as an example of the previous matching position BM, and the position AM of the vehicle 100 after traveling on the travel route AR from the previous matching position.

[0069] For example, starting from the position BM1 at the time of the previous match, by estimating the travel route AR or the position AM of the vehicle 100 obtained from the actuator drive history AC, the position AM after the vehicle 100 has traveled can be obtained. In another embodiment, the start position determination unit 316 can estimate the travel route AR or the position AM of the vehicle 100 based on the output values from detectors for obtaining vehicle speed, steering angle, braking force, rotation angle, etc., instead of the actuator drive history AC. Alternatively, the current position of the vehicle 100 obtained from a Global Navigation Satellite System (GNSS) receiver or the like can be used as the position AM after the vehicle 100 has traveled.

[0070] The start position determination unit 316 determines the estimated position AM after the vehicle 100 has traveled as the start position for template matching. The position estimation unit 318 starts template matching using the vehicle point cloud data VP from the position AM. By setting the position AM after the vehicle 100 has traveled as the start position for template matching, the actual position of the vehicle 100 or its vicinity can be set as the start position. Therefore, the processing period required for template matching can be shortened. Compared with the case where template matching starts at a position far from the position AM, this can reduce the problem that the matching is completed at a position different from the actual position of the vehicle 100 due to obtaining a local solution by gradient descent or the like. Therefore, the detection accuracy of the vehicle 100 can be improved.

[0071] As described above, the remote control device 300 of the present embodiment includes: a point cloud data acquisition unit 314 configured to acquire three-dimensional point cloud data measured using the vehicle detector 80; a position estimation unit 318 configured to estimate the position and orientation of the vehicle 100 in the three-dimensional point cloud data by matching the vehicle point cloud data VP with the three-dimensional point cloud data; a start position determination unit 316 configured to determine the start position for the position estimation unit 318 to start template matching of the three-dimensional point cloud data; and a remote control unit 312 configured to generate a control command for remotely controlling the vehicle 100 using the estimated position and orientation of the vehicle 100 and transmit the control command to the vehicle 100. The remote control device 300 of the present embodiment can quickly detect the position of the vehicle 100 by setting the start position of the matching near the vehicle 100. Therefore, the processing period required for template matching can be shortened.

[0072] In the remote control device 300 of the present embodiment, the start position determination unit 316 determines the start position by using the information on the position of the vehicle 100 in the three-dimensional point cloud data acquired by the point cloud data acquisition unit 314. By setting the start position of the template matching to the position of the vehicle 100 by using the information on the position of the vehicle 100, the position of the vehicle 100 can be detected quickly. Therefore, the processing period required for the template matching can be shortened.

[0073] In the remote control device 300 of the present embodiment, the start position determination unit 316 acquires the previous matching position BM and uses the acquired previous matching position BM to determine the start position of the template matching. By using the previous matching position BM that is likely to be near the position of the vehicle 100 at the time of the current matching, the processing period required for the template matching can be shortened by simple processing.

[0074] In the remote control device 300 of the present embodiment, the start position determination unit 316 estimates the position AM of the vehicle 100 after traveling from the previous matching position BM during the period from the time when the previous matching is completed to the time before the current matching is executed, and determines the estimated position AM after traveling as the start position of the template matching. By estimating the position of the vehicle 100 after traveling, the start position of the matching can be set closer to the position of the vehicle 100 at the time of the current matching. Therefore, the processing period required for the template matching can be further shortened.

[0075] In the remote control device 300 of the present embodiment, when there is no previous matching result, the start position determination unit 316 determines the start position on the target route RT where driving starts as the start position of the template matching. By setting the start position on the target route RT where the vehicle 100 is likely to exist, the position of the vehicle 100 can be detected quickly. Therefore, the processing period required for the template matching can be shortened. In particular, when there is no previous matching result, the start position where the vehicle 100 is likely to exist is set as the start position. Therefore, the processing period required for the template matching can be further shortened.

[0076] B. Second Embodiment

[0077] Figure 6 FIG. is a block diagram showing the internal functional configuration of the remote control device 300b according to the second embodiment. The remote control device 300b of the present embodiment is different from the remote control device 300 of the first embodiment in that a CPU 310b that further serves as an object detection unit 322 is provided instead of the CPU 310, and a storage device 340b that further stores vehicle identification information VI and a trained model LM is provided instead of the storage device 340. Other configurations are the same.

[0078] The trained model LM is a machine learning model that performs object detection by using a predetermined object detection method of deep learning. The trained model LM has been sufficiently trained using a predetermined data set. For example, various methods using a Convolutional Neural Network (CNN) can be used for the trained model LM. For object detection, it is particularly preferable to adopt a method using deep learning, such as Region-Convolutional Neural Network (R-CNN), Fast R-CNN, Faster R-CNN, various versions of You Only Look Once (YOLO), or Single Shot Detector (SSD).

[0079] The object detection unit 322 uses the trained model LM to perform annotation on the three-dimensional point cloud data and detects the temporary position of the vehicle in the three-dimensional point cloud data. In this case, the term "vehicle" refers to an object with the attribute (category) of a vehicle, and may include vehicles other than the vehicle 100 to be controlled. Specifically, the object detection unit 322 performs object detection by inputting the detected three-dimensional point cloud data into the trained model LM and generates a 3D bounding box for the region identified as a vehicle. The "bounding box" refers to an object region that surrounds the vehicle by using a minimum rectangular box, and is a partial region extracted from the external region of the three-dimensional point cloud data. That is, the object detection unit 322 extracts the region occupied by the vehicle in the three-dimensional point cloud data by forming a bounding box.

[0080] The vehicle identification information VI refers to various types of information that can identify each vehicle 100. The vehicle identification information VI includes, for example, identification (ID) information assigned to each vehicle 100 (such as a Vehicle Identification Number (VIN)), specification information of the vehicle 100 (such as vehicle type, color, shape, and size), and production management information of the vehicle 100 (such as the name of the ongoing process). The vehicle identification information VI may include the size of the detected bounding box. For example, the vehicle identification information VI can be obtained from a Radio Frequency-Identification (RF-ID) tag attached to the vehicle 100 through short-range wireless communication.

[0081] Figure 7It is a flowchart showing the start position determination process according to the second embodiment. In step S600, the object detection unit 322 performs object detection using the acquired three-dimensional point cloud data and obtains the temporary position of the vehicle 100 in the three-dimensional point cloud data. More specifically, the object detection unit 322 generates the bounding box of the vehicle in the acquired three-dimensional point cloud data by inputting the three-dimensional point cloud data into the trained model LM.

[0082] In step S610, the object detection unit 322 acquires the vehicle identification information VI of the vehicle 100 to be controlled. In step S620, the object detection unit 322 extracts the bounding box associated with the vehicle 100 to be controlled using the vehicle identification information VI. For example, the object detection unit 322 extracts the bounding box that matches the size of the bounding box associated with the vehicle identification information VI of the vehicle 100 to be controlled or the size of the vehicle 100 from the generated bounding boxes. Instead of the size of the bounding box or the size of the vehicle 100, the vehicle identification information VI may also indicate the shape of part or all of the vehicle 100, or the length, width, and height of part or all of the vehicle 100. In step S630, the start position determination unit 316 determines the position of the extracted bounding box as the start position of the template matching and terminates the process.

[0083] Figure 8 It shows the template matching in the start position determination process according to the second embodiment. Figure 8 It shows an example of the three-dimensional point cloud data PD2 acquired from the vehicle detector 80. The three-dimensional point cloud data PD2 includes the three-dimensional point cloud data of the vehicle 100 to be controlled and the three-dimensional point cloud data of the vehicle 100R that is not to be controlled. The object detection unit 322 generates the bounding boxes BB1 and BB2 of the vehicles 100 and 100R recognized as vehicles by inputting the three-dimensional point cloud data PD2 into the trained model LM.

[0084] Next, the object detection unit 322 acquires the vehicle identification information VI. In this case, the vehicle identification information VI is preferably information such as the size and width of the vehicles 100 and 100R or the size of the bounding box, as information that facilitates distinguishing the vehicles 100 and 100R with different sizes. Based on the acquired vehicle identification information VI, the bounding box BB1 of the vehicle 100 is extracted from the bounding boxes BB1 and BB2. The start position determination unit 316 determines the position of the extracted bounding box BB1 as the start position of the template matching, and the position estimation unit 318 starts the template matching using the vehicle point cloud data VP from the position of the bounding box BB1 as the start position.

[0085] As described above, in the remote control device 300b of the present embodiment, the start position determination unit 316 determines the position of the vehicle 100 detected from the three-dimensional point cloud data PD2 by a predetermined object detection method using deep learning as the start position. The position of the vehicle 100 can be extracted from the three-dimensional point cloud data PD2 using a known method, which is object detection using deep learning. The start position can be set near the vehicle 100 using the position of the vehicle 100 in the three-dimensional point cloud data PD2. Therefore, the processing period required for template matching can be shortened.

[0086] In the remote control device 300b of the present embodiment, when the positions of a plurality of vehicles 100, 100R are detected, the start position determination unit 316 also uses the vehicle identification information VI for identifying the vehicle 100 to determine the start position using the position of the vehicle 100 associated with the vehicle identification information VI among the detected positions of the plurality of vehicles 100. Therefore, even when a plurality of vehicles 100, 100R are detected by the object detection method using deep learning, the position of the vehicle 100 to be controlled can be extracted.

[0087] C. Third Embodiment

[0088] Figure 9 FIG. is a top view showing an example of the overall configuration of the remote control device 300c according to the third embodiment. The remote control device 300c of the present embodiment is an automated guided vehicle also referred to as "guided mobility". As in the first embodiment, the remote control device 300c has a function of autonomously driving the vehicle 100 by remote control, and also has a function of autonomously driving the main body of the remote control device 300c on the driving road SR. That is, while the remote control device 300c autonomously travels on the driving road SR on which it is traveling, it autonomously drives the subsequent vehicle 100 by remote control to guide the subsequent vehicle 100.

[0089] The remote control device 300c includes drive wheels 142 for autonomous driving, an autonomous driving detector 70, and a vehicle detector 80. The function of the vehicle detector 80 is the same as that of the vehicle detector 80 shown in the first embodiment, and thus its description will be omitted. The autonomous driving detector 70 is a distance measuring device such as a camera or a LiDAR sensor. The data acquired by the autonomous driving detector 70 is used for simultaneous localization and mapping (SLAM) of the autonomous driving of the remote control device 300c. The camera can be a stereo camera, a monocular camera, an RGB-D camera (depth camera), etc. A time-of-flight (ToF) sensor or the like can be used instead of the camera or the LiDAR sensor. Figure 9In the example, the vehicle detector 80 is only installed on the remote control device 300c. The vehicle detector 80 can also be installed around the driving road SR.

[0090] Figure 10 FIG. is a block diagram showing the internal functional configuration of the remote control device 300c according to the third embodiment. The difference between the internal functional configuration of the remote control device 300c and the internal functional configuration of the remote control device 300 according to the first embodiment is that a CPU 310c that further serves as the SLAM unit 328 and the autonomous driving control unit 330 is provided instead of the CPU 310, and a storage device 340c that further stores the guiding vehicle route GR is provided instead of the storage device 340. Other configurations are the same. The guiding vehicle route GR is a predetermined target route on which the remote control device 300c is expected to travel.

[0091] The actuator 360 includes actuators for the driving device, steering device, and braking device for the autonomous driving of the remote control device 300c. The driving device includes a battery, a driving motor driven by the battery power, and driving wheels 142 rotated by the driving motor.

[0092] The SLAM unit 328 uses the data detected by the autonomous driving detector 70 to perform SLAM and generate a map for the autonomous driving of the remote control device 300c. The autonomous driving control unit 330 controls the actuator 360 to make the remote control device 300c drive autonomously. Specifically, the autonomous driving control unit 330 uses the map generated by the SLAM unit 328 to make the remote control device 300c drive autonomously along the guiding vehicle route GR stored in the storage device 340c.

[0093] As Figure 9 shown, when the remote control device 300c travels along the guiding vehicle route GR, the vehicle 100P is guided by the remote control of the remote control device 300c and travels on the same driving route as the remote control device 300c. In this embodiment and the first embodiment, template matching is performed on the three-dimensional point cloud data obtained from the vehicle detector 80, and the start position determination unit 316 performs start position determination processing for determining the start position of the template matching. In this embodiment and the first embodiment, the remote control device 300c for autonomous driving also uses the information about the position of the vehicle 100 to set the start position of the template matching near the vehicle 100. Therefore, the processing period required for template matching can be shortened.

[0094] D. Other Embodiments:

[0095] (D1) The second embodiment shows an example in which a bounding box of a vehicle is generated by using an object detection method of deep learning, and the position of the vehicle 100 extracted by using the generated bounding box is determined as the start position. Instead of generating the bounding box, the object detection unit 322 may also perform preprocessing for removing background point cloud data representing stationary objects from the acquired three-dimensional point cloud data. This kind of preprocessing is also called "background subtraction". In this case, the start position determination unit 316 determines the temporary position of the vehicle 100 detected based on the difference between the point cloud data associated with the background of the three-dimensional point cloud data and the point cloud data associated with the vehicle 100 to be detected as the start position of template matching. The remote control device 300 configured in this way can determine the start position through a simple process of background subtraction.

[0096] (D2) The first embodiment shows an example in which the start position determination unit 316 sets the travel start position of the vehicle 100 on the target route RT as the start position when there is no previous matching result. The start position determination unit 316 can set the travel start position of the vehicle 100 on the target route RT as the start position regardless of whether there is a previous matching result. Instead of the travel start position on the target route RT, any position on the target route RT may also be determined as the start position. In this case, the step S260 shown in Figure 4 can be omitted. With the remote control device 300 configured in this way, by setting the start position on the target route RT where the vehicle 100 is likely to appear, the processing period required for template matching can be shortened. When the vehicle 100 cannot be detected at the start position in this case, it is preferable to perform template matching along the target route RT.

[0097] The start position determination unit 316 can acquire the previous matching position BM on the target route RT of the previous time, and use the acquired previous matching position BM on the target route RT to determine the start position. With the remote control device 300 configured in this way, by using a simple method of the previous matching position BM on the target route RT where the vehicle 100 is more likely to appear, the processing period required for template matching can be shortened.

[0098] (D3)Each of the above embodiments shows an example in which the vehicle 100 is a passenger car, a truck, a bus, a construction vehicle, etc. However, the vehicle 100 is not limited thereto, and may include various motor vehicles such as two-wheel vehicles, four-wheel vehicles, trains, etc. Various moving bodies other than the vehicle 100 may be used. A "moving body" refers to an object that can move. Examples of moving bodies include electric vertical take-off and landing aircraft (so-called flying cars), ships, airplanes, robots, and maglev trains. In this case, the terms "vehicle" and "automobile" in the present disclosure may be replaced with "moving body" as appropriate, and the term "travel" may be replaced with "move" as appropriate.

[0099] (D4)The vehicle 100 only needs to have a configuration that enables the vehicle 100 to move by remote control. For example, the vehicle 100 may be in the form of a platform having the following configuration. Specifically, the vehicle 100 only needs to have a configuration that performs functions of "traveling", "steering", and "stopping" by remote control. That is, the "vehicle 100 capable of moving by remote control" does not need to have at least some of the internal components such as a driver's seat and a dashboard, at least some of the external components such as a bumper and a fender, or a body shell. In this case, the remaining components (such as the body shell) may be installed on the vehicle 100 before the vehicle 100 is shipped from the factory, or may be installed on the vehicle 100 after the vehicle 100 is shipped from the factory in a state where the remaining components (such as the body shell, etc.) are not installed on the vehicle 100.

[0100] (D5)The first embodiment shows an example in which the start position determination unit 316 further estimates the position AM after traveling of the vehicle 100 traveling from the previous matching position BM. The start position may be a position other than the position AM after traveling. For example, the previous matching position BM may be used. By using the previous matching position BM that is likely to be near the position of the vehicle 100, the processing period required for template matching can be shortened by simple processing.

[0101] (D6)The first embodiment shows an example in which the start position determination unit 316 uses the previous matching position BM to determine the start position, and an example in which the start position determination unit 316 determines an arbitrary position on the target route RT as the start position. The start position determination unit 316 may determine a position other than the previous matching position BM or a position different from the target route RT as the start position. For example, the start position determination unit 316 may determine the temporary position of the vehicle 100 obtained based on the detection result from a detector different from the vehicle detector 80 as the start position.

[0102] (D7) The third embodiment shows an example in which the object detection unit 322 uses vehicle identification information VI to extract the vehicle 100 to be controlled. For example, when a single bounding box is generated in step S600, that is, when only the bounding box of the vehicle 100 to be controlled is generated, it is not necessary to use the vehicle identification information VI. Additionally, when information other than the vehicle identification information VI (such as the position of the vehicle) can be used to extract the vehicle 100 to be controlled, the object detection unit 322 does not need to acquire the vehicle identification information VI. In this case, the steps S610 and S620 shown in Figure 7 can be omitted.

[0103] The control and its method described in the present disclosure can be implemented by a dedicated computer provided by configuring a processor and a memory to be programmed to execute one or more functions implemented by a computer program. Alternatively, the control and its method described in the present disclosure can be implemented by a dedicated computer provided by configuring a processor using one or more dedicated hardware logic circuits. Alternatively, the control and its method described in the present disclosure can be implemented by one or more dedicated computers configured by a combination of a memory and a processor, the processor being programmed to execute one or more functions, and the processor being configured by one or more hardware logic circuits. The computer program can be stored in a non-transitory tangible computer-readable storage medium as instructions to be executed by a computer.

[0104] The present disclosure is not limited to the above embodiments and can be implemented in various configurations without departing from the spirit of the present disclosure. For example, the technical features in each of the embodiments corresponding to the technical features of each aspect described in the summary of the invention can be appropriately replaced or combined to solve some or all of the above problems or achieve some or all of the above effects. When a technical feature is not described as necessary in this specification, such a feature can be omitted as appropriate.

Claims

1. A remote control device, comprising: A point cloud data acquisition unit that acquires three-dimensional point cloud data measured using a distance measurement device; A position estimation unit that estimates at least one of the position and orientation of the moving body in the three-dimensional point cloud data by matching a template point cloud indicating the moving body with the three-dimensional point cloud data; A start position determination unit that determines a start position at which the position estimation unit starts to match the template point cloud with the three-dimensional point cloud data; And A remote control unit that uses at least one of the estimated position and orientation of the moving body to generate a control command for remotely controlling the moving body and transmits the control command to the moving body, wherein The start position determination unit uses information about the driving history of an actuator provided in the moving body among information about the position for the matching and information included in the control command, and uses information about at least one of the position and orientation of the moving body for generating the control command and a target route predetermined as the travel route of the moving body by the remote control to determine the start position.

2. The remote control device according to claim 1, wherein The start position determination unit Acquires a previous matching position at which a previous match between the three-dimensional point cloud data and the template point cloud has been completed as information about the driving history of an actuator provided in the moving body, and Uses the acquired previous matching position to determine the start position.

3. The remote control device according to claim 2, wherein The start position determination unit Further estimates a post-movement position of the moving body that has moved from the previous matching position during a period from the moment when the previous match was completed to the moment before the current match is executed, and Uses the estimated post-movement position to determine the start position.

4. The remote control device according to claim 1, Further includes a memory that stores the target route, wherein The start position determination unit determines an arbitrary position on the target route as the start position.

5. The remote control device according to claim 4, wherein The start position determination unit Acquires a previous matching position on the target route at which a previous match between the three-dimensional point cloud data and the template point cloud has been completed, and Uses the acquired previous matching position on the target route to determine the start position.

6. The remote control device according to claim 1, wherein The start position determination unit uses either the position of the moving body detected from the three-dimensional point cloud data by a predetermined object detection method or the position of the moving body detected based on the difference between the point cloud data associated with the background of the three-dimensional point cloud data and the point cloud data associated with the moving body as the position and orientation of the moving body to determine the start position.

7. The remote control device according to claim 6, wherein The start position determination unit, When detecting multiple positions of the moving body, Using the mobile object identification information for identifying the mobile object, and using the position of the mobile object associated with the mobile object identification information among the detected positions of the plurality of mobile objects, to further determine the start position.

Images