Control device
By generating the modified template point cloud and matching it with the three-dimensional point cloud data, the problem of position estimation errors caused by blind spots in the three-dimensional point cloud data is solved, and the accurate position and orientation estimation in autonomous transportation vehicles is achieved, which improves processing speed.
Patent Information
- Application Number
- CN202410670908.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-10-23
- Filing Date
- 2024-05-28
- Publication Date
- 2024-12-03
AI Technical Summary
In the position or orientation estimation of autonomous transport vehicles, blind spots in the three-dimensional point cloud data make it impossible to obtain correct estimation results, especially when the field of view of the distance measuring device is obscured.
By generating modified template point clouds, eliminating defective parts in template point clouds, matching between three-dimensional point cloud data and template point clouds is performed to estimate the vehicle's position and orientation, and supplement using GPS and driving history if necessary information.
Even in the case of defective parts in the three-dimensional point cloud data, the position and orientation of the vehicle can be correctly estimated, which improves the speed and accuracy of the estimation processing.
Smart Images

Figure CN119065360A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a control device for a mobile body. Background Art
[0002] A technology for autonomously transporting a vehicle during a vehicle manufacturing process is known (for example, Japanese Unexamined Patent Application Publication (Translation of PCT Application) No. 2017-538619 (JP 2017-538619 A)). Summary of the invention
[0003] When a moving body (vehicle) moves by autonomous transportation, a process of estimating the position or orientation of the moving body is performed. The position or orientation of the moving body can be estimated by using three-dimensional point cloud data acquired by a distance measuring device such as a camera or a radar. However, in the case where a blind spot is generated in the distance measuring device, when the three-dimensional point cloud data is directly used to perform the estimation process, there is a possibility that a correct estimation result cannot be obtained.
[0004] The present disclosure can be implemented as the following aspects.
[0005] A first aspect of the present disclosure relates to a control device that generates a control command for controlling a moving body by using three-dimensional point cloud data of a moving body measured by a distance measuring device. The control device includes an estimation unit. The estimation unit is configured to, in a case where a defective portion is generated in the three-dimensional point cloud data acquired from the distance measuring device, perform matching between a substantial point cloud portion obtained by excluding a defective corresponding portion corresponding to the defective portion from a template point cloud and the three-dimensional point cloud data to estimate at least one of a position and an orientation of the moving body.
[0006] According to this aspect, even when a defective portion occurs in the three-dimensional point cloud data, at least one of the position and the orientation of the moving object can be accurately estimated.
[0007] In this aspect of the present disclosure, the control device may further include a template point cloud modification unit configured to delete the defect corresponding portion from the template point cloud to create a modified template point cloud. The estimation unit may be configured to perform matching between the three-dimensional point cloud data and the modified template point cloud to estimate at least one of the position and orientation of the moving body.
[0008] According to this aspect, since the modified template point cloud is used, the processing speed of estimation can be improved.
[0009] In this aspect of the present disclosure, the control device may further include a defect specifying unit configured to specify a defective portion of the three-dimensional point cloud data.
[0010] According to this aspect, after the defective portion is specified, the defective corresponding portion can be excluded from the template point cloud.
[0011] In an aspect of the present disclosure, in the control device, the defect specifying unit may be configured to specify a defective portion of the three-dimensional point cloud data acquired from the distance measuring device with reference to a history of the three-dimensional point cloud data.
[0012] According to this aspect, the defective portion can be correctly specified.
[0013] In the aspect of the present disclosure, in the control device, the defect specifying unit may be configured to specify the defective portion depending on the distance between the moving body and the distance measuring device by using a map registering defective portions depending on the distance between the moving body and the distance measuring device.
[0014] According to this aspect, the defective portion can be easily specified. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, wherein like numerals represent like elements, and wherein: Figure 1 is a conceptual diagram showing a configuration of a remote control system according to an embodiment; Figure 2 is a block diagram showing the configuration of a vehicle and a remote control device according to a first embodiment; Figure 3 is a flowchart showing the processing flow of remote control according to the first embodiment; Figure 4 is a diagram showing an example of a defective portion of three-dimensional point cloud data; Figure 5 is a block diagram showing the configuration of a vehicle and a remote control device according to a second embodiment; and Figure 6 is a flowchart showing a processing flow of vehicle control according to the second embodiment. DETAILED DESCRIPTION A. First Embodiment
[0016] Figure 1 is a conceptual diagram showing a configuration of a remote control system 10 according to an embodiment. The remote control system 10 includes one or more vehicles 100 as mobile bodies, a remote control device 200 that generates a control command for remotely controlling the vehicle 100 and transmits the control command to the vehicle 100, a plurality of distance measuring devices 300 that measure three-dimensional point cloud data of the vehicle 100, and a process management device 400 that manages a manufacturing process of the vehicle 100. In the first embodiment, the remote control device 200 corresponds to a “control device” according to the present disclosure.
[0017] The vehicle 100 is preferably a battery electric vehicle (BEV). The mobile body is not limited to a battery electric vehicle, and may be, for example, a gasoline vehicle, a hybrid electric vehicle, or a fuel cell electric vehicle. The mobile body is not limited to the vehicle 100, and may be, for example, an electric vertical lift (so-called flying car). An arbitrary position in the factory is represented by an xyz coordinate value in the reference coordinate system Σr.
[0018] In the present disclosure, a "mobile body" refers to a movable object. The vehicle may be a vehicle that travels by wheels or a vehicle that travels using tracks, and is, for example, a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, and a construction vehicle. In addition, in the case where the mobile body is not a vehicle, the expressions "vehicle" and "car" in the present disclosure may be appropriately replaced with "mobile body", and the expression "travel" may be appropriately replaced with "move".
[0019] The vehicle 100 is configured to travel via unmanned driving. "Unmanned driving" refers to driving that does not rely on the driving operation of the occupants. The driving operation refers to an operation related to at least one of "driving", "turning" and "stopping" of the vehicle 100. Unmanned driving is implemented by automatic or manual remote control using a device located outside the vehicle 100 or by autonomous control of the vehicle 100. The vehicle 100 that travels via unmanned driving may carry occupants who do not perform driving operations. Examples of occupants who do not perform driving operations include persons who simply sit in a seat of the vehicle 100, and persons who perform work different from driving operations (such as assembly, inspection, or operating switches, etc.) while riding in the vehicle 100. Driving performed via the driving operation of the occupants may be referred to as "manned driving".
[0020] In this specification, “remote control” includes “full remote control” in which all operations of the vehicle 100 are completely determined from the outside of the vehicle 100, and “partial remote control” in which a part of the operations of the vehicle 100 are determined from the outside of the vehicle 100. In addition, “autonomous control” includes “full autonomous control” in which the vehicle 100 autonomously controls its operation without receiving any information from an external device of the vehicle 100, and “partial autonomous control” in which the vehicle 100 autonomously controls its operation by using information received from an external device of the vehicle 100.
[0021] In the present embodiment, remote control of the vehicle 100 is performed in a factory that manufactures the vehicle 100. The factory includes a first location PL1 and a second location PL2. The first location PL1 is, for example, a location where the vehicle 100 is assembled, and the second location PL2 is, for example, a location where the vehicle 100 is inspected. The first location PL1 and the second location PL2 are connected via a travel road SR on which the vehicle 100 can travel.
[0022] The distance measuring devices 300 with each vehicle 100 as the measurement target are installed around the driving road SR. The remote control device 200 can obtain the relative position and orientation of the vehicle 100 relative to the target route TR in real time by using the three-dimensional point cloud data measured by each distance measuring device 300. As the distance measuring device 300, a camera or light detection and ranging (Light Detection And Ranging, LiDAR) can be used. Specifically, LiDAR is preferred because highly accurate three-dimensional point cloud data can be obtained. The position of each distance measuring device 300 is fixed, and the relative relationship between the reference coordinate system Σr and the device coordinate system of each distance measuring device 300 is known. The coordinate transformation matrix for mutually transforming the coordinate values in the reference coordinate system Σr of each distance measuring device 300 with the coordinate values in the device coordinate system is pre-stored in the remote control device 200.
[0023] When the three-dimensional point cloud data of the vehicle 100 as the control target is measured by the distance measuring device 300, depending on the positional relationship between the distance measuring device 300 and the vehicle as the control target or the positional relationship between the vehicle as the control target and other vehicles, a blind spot may be generated within the field of view of the distance measuring device 300. In the case of the presence of such a blind spot, a defective portion is generated in the three-dimensional point cloud data, and there is a possibility that the position and orientation of the vehicle as the control target cannot be correctly estimated. As described below, in the present disclosure, even in the case where a defective portion is generated in the three-dimensional point cloud data, the position and orientation of the vehicle 100 can be correctly estimated.
[0024] The remote control device 200 generates a control command for causing the vehicle 100 to travel along the target route TR, and transmits the control command to the vehicle 100. The vehicle 100 travels in response to the received control command. Therefore, the remote control system 10 can move the vehicle 100 from the first location PL1 to the second location PL2 by remote control operation without using a transport device such as a crane or a conveyor belt.
[0025] Figure 21 is a block diagram showing the configuration of the vehicle 100 and the remote control device 200. The vehicle 100 includes a vehicle control device 110 that controls various units of the vehicle 100, an actuator group 120 driven under the control of the vehicle control device 110, a communication device 130 that communicates with the remote control device 200 via wireless communication, and a GPS receiver 140 that acquires position information of the vehicle 100. In the present embodiment, the actuator group 120 includes an actuator of a driving device for accelerating the vehicle 100, an actuator of a steering device for changing the driving direction of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100. The driving device includes a battery, a driving motor driven by the power of the battery, and a driving wheel rotated by the driving motor. The actuator of the driving device includes a driving motor. The actuator group 120 may also include an actuator for shaking a windshield wiper of the vehicle 100 or an actuator for opening and closing an electric window of the vehicle 100.
[0026] The vehicle control device 110 is configured by a computer including a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected in a bidirectionally communicable manner with each other via the internal bus 114. The actuator group 120, the communication device 130, and the GPS receiver 140 are connected to the input / output interface 113.
[0027] In the present embodiment, the processor 111 functions as the vehicle controller 115 and the position information acquisition unit 116 by executing the program PG1 pre-stored in the memory 112. The vehicle controller 115 controls the actuator group 120. In a case where a driver is riding in the vehicle 100, the vehicle controller 115 can cause the vehicle 100 to travel by controlling the actuator group 120 in response to the driver's operation. The vehicle controller 115 can also cause the vehicle 100 to travel by controlling the actuator group 120 in response to a control command transmitted from the remote control device 200, regardless of whether the driver is riding in the vehicle 100. The position information acquisition unit 116 acquires position information indicating the current position of the vehicle 100 by using the GPS receiver 140. Note that the position information acquisition unit 116 and the GPS receiver 140 are not always required.
[0028] The remote control device 200 is configured by a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are connected in a bidirectionally communicable manner with each other via the internal bus 204. A communication device 205 for communicating with the vehicle 100, the distance measurement device 300, and the process management device 400 via wireless communication is connected to the input / output interface 203.
[0029] In this embodiment, the processor 201 functions as a three-dimensional point cloud data acquisition unit 210, a defect specification unit 220, a template point cloud modification unit 230, an estimation unit 240, and a remote control command generation unit 250 by executing a program PG2 pre-stored in the memory 202.
[0030] The three-dimensional point cloud data acquisition unit 210 acquires three-dimensional point cloud data of the vehicle 100 as a control target measured by the distance measurement device 300. The three-dimensional point cloud data is data indicating the three-dimensional position of the point cloud detected by the distance measurement device 300.
[0031] The defect specifying unit 220 specifies a defective portion in the three-dimensional point cloud data. The specifying method will be described below.
[0032] The template point cloud modification unit 230 deletes the defect corresponding portion corresponding to the defect portion from the template point cloud TP to create a modified template point cloud. The template point cloud TP is pre-stored in the memory 202.
[0033] The estimation unit 240 performs matching between the three-dimensional point cloud data of the vehicle 100 as a control target and the modified template point cloud to estimate the position and orientation of the vehicle 100. In the case where the three-dimensional point cloud data is not available, the estimation unit 240 may estimate the position and orientation of the vehicle 100 by using the driving history of the vehicle 100 or the position information detected by the GPS receiver 140 mounted on the vehicle 100. The estimation unit 240 may estimate only one of the position and orientation of the vehicle 100. In this case, the other of the position and orientation of the vehicle 100 is determined by using the driving history of the vehicle 100, etc.
[0034] The position and orientation of the vehicle are also referred to as “vehicle position information.” In the present embodiment, the position and orientation of the vehicle 100 in the reference coordinate system of the factory are included in the vehicle position information.
[0035] The remote control command generation unit 250 generates a control command for remote control by using the estimated position and orientation of the vehicle 100, and transmits the generated control command to the vehicle 100. The control command is a command to make the vehicle 100 travel along the target route TR stored in the memory 202. The control command may be generated as a command including a driving force or a braking force and a steering angle. Alternatively, a control command including at least one of the position and orientation of the vehicle 100 and the future travel route may also be generated.
[0036] In the present embodiment, the control command includes the acceleration and the steering angle of the vehicle 100 as parameters. In other embodiments, the control command may include the speed of the vehicle 100 as a parameter instead of or in addition to the acceleration of the vehicle 100.
[0037] The process management device 400 performs management of the entire manufacturing process of the vehicle 100 in the factory. For example, when one vehicle 100 starts traveling along the target route TR, individual information indicating the identification number or model of the vehicle 100 is transmitted from the process management device 400 to the remote control device 200. The position of the vehicle 100 detected by the remote control device 200 is also transmitted to the process management device 400. In addition, the function of the process management device 400 can be implemented in the same device as the remote control device 200.
[0038] Remote control device 200 is also referred to as a “server”, and distance measurement device 300 is also referred to as an “external sensor”. Control commands are also referred to as “travel control signals”, target route TR is also referred to as a “reference route”, and the reference coordinate system is also referred to as a “global coordinate system”.
[0039] Figure 3 1 is a flowchart showing the processing flow of the remote control according to the first embodiment. The processing of the remote control device 200 is performed in a constant cycle. Alternatively, each time three-dimensional point cloud data is newly acquired from the distance measurement device 300 responsible for the measurement of the vehicle 100 as the control target, the three-dimensional point cloud data acquisition unit 210 may execute Figure 3 The three-dimensional point cloud data acquisition unit 210 may perform pre-processing of removing background point cloud data indicating a stationary object from the newly acquired three-dimensional point cloud data.
[0040] In step S10 , the defect specifying unit 220 determines whether there is a defective portion in the three-dimensional point cloud data.
[0041] Figure 4 is a diagram showing an example of a defective portion of three-dimensional point cloud data. Figure 4 The upper part of shows the three-dimensional point cloud data PGa in which there is no defective portion, and Figure 4 The lower part of PGc shows the three-dimensional point cloud data PGc in which the defective part DF exists. The defective part DF may be generated due to the fact that the field of view of the distance measuring device 300 is partially blocked by other vehicles or stationary objects.
[0042] As a method of determining whether or not the defective portion DF exists in step S10 , for example, any of the following methods may be used. Method A1 for determining whether a defect exists
[0043] In the distribution of the point cloud in the three-dimensional point cloud data, the size of the part where the density of the point cloud is equal to or less than the density reference value is obtained, and when the size is equal to or greater than the size threshold, it is determined that there is a defective part DF. In addition, the "density reference value" can be zero. Method A2 for determining whether a defect exists
[0044] In the three-dimensional point cloud data, a bounding box is determined to surround the dense point cloud part, and the size of the part in the bounding box where the point cloud density is equal to or less than the density reference value is obtained. When the size is equal to or greater than the size threshold, it is determined that a defective part DF exists. Method A3 for determining whether a defect exists
[0045] The presence or absence of the defective portion DF is determined depending on the distance between the vehicle 100 and the distance measuring device 300. As the distance between the vehicle 100 and the distance measuring device 300, the distance to the reference position of the bounding box surrounding the portion where the point cloud is dense can be used. In the case where the driving environment of the vehicle 100 is almost constant, the defective portion DF caused by the stationary object is generated at a constant location depending on the distance between the vehicle 100 and the distance measuring device 300, so that it can be determined in advance whether the defective portion DF exists depending on the distance.
[0046] In determination methods A1 and A2, the process of obtaining the size of the portion where the density of the point cloud is equal to or less than the density reference value may be performed only for the portion of the point cloud where the vehicle 100 is estimated to exist. The position of the portion of the point cloud where the vehicle 100 is estimated to exist is determined depending on the model of the vehicle 100, and is stored as vehicle information VI for each type. In determination method A2, the bounding box may be divided into a plurality of voxels (micro-boxes) to calculate the density of the point cloud for each voxel, and when the size of the portion where the voxels having the density of the point cloud equal to or less than the density reference value are clustered is equal to or greater than the size threshold, it may be determined that a defective portion DF exists.
[0047] In the case where there is no defective portion DF, the process proceeds to step S50 , and the estimation unit 240 performs matching between the template point cloud TP and the three-dimensional point cloud data to estimate the position and orientation of the vehicle 100 .
[0048] On the other hand, in the case where there is a defective part DF, the process proceeds from step S10 to step S20, and the defect specifying unit 220 specifies the defective part DF of the three-dimensional point cloud data. As a specifying method of the defective part DF, for example, any of the following methods can be used. Method for specifying defective parts B1
[0049] When it is determined by the determination method A1 or A2 that a defective portion DF exists, a portion where the point cloud continues with a density equal to or less than the density reference value is specified as a defective portion DF. In this case, the defective portion can be specified as a voxel cluster. Defective part designation method B2
[0050] With reference to the history of three-dimensional point cloud data on the same vehicle as the control target, the defective part DF of the three-dimensional point cloud data PGc of this time is designated. For example, by performing a match between representative three-dimensional point cloud data in the history of three-dimensional point cloud data and the three-dimensional point cloud data PGc of this time, a part that does not exist in the three-dimensional point cloud data PGc of this time can be designated as the defective part DF. As "representative three-dimensional point cloud data", the most recent three-dimensional point cloud data or the data at the current time estimated from the most recent three-dimensional point cloud data can be used. In addition, in order to increase the processing speed of matching, pre-processing of sparse point clouds can be performed on the two three-dimensional point cloud data as matching targets. Method for specifying defective parts B3
[0051] In the case where it is determined by the determination method A3 that the defective portion DF exists, the defective portion DF is specified depending on the distance between the vehicle 100 and the distance measuring device 300 by using the defective portion map DM in which the defective portion DF depending on the distance between the vehicle 100 and the distance measuring device 300 is registered. The defective portion map DM is registered in advance in the memory 202. In the case where the driving environment of the vehicle 100 is almost constant, the defective portion DF caused by the stationary object is generated at a constant place depending on the distance between the vehicle 100 and the distance measuring device 300, so the defective portion map DM can be created and registered in advance.
[0052] In step S30, the template point cloud modification unit 230 deletes the portion corresponding to the defective portion DF from the template point cloud TP to create a modified template point cloud. The modified template point cloud is template data representing a substantial point cloud portion obtained by excluding the defective corresponding portion corresponding to the defective portion DF from the template point cloud TP. In step S40, the estimation unit 240 performs matching between the modified template point cloud and the three-dimensional point cloud data to estimate the position and orientation of the vehicle 100.
[0053] Instead of creating a modified template point cloud, the estimation process in step S40 may be performed by excluding the defect corresponding portion corresponding to the defect portion DF from the template point cloud TP. In this case, in the template point cloud TP, the points contained in the defect corresponding portion are excluded from the estimation process routine. In terms of matching between the substantial point cloud portion obtained by excluding the defect corresponding portion corresponding to the defect portion DF from the template point cloud TP and the three-dimensional point cloud data, the process is the same as the process of steps S30 and S40. Note that in the case of creating a modified template point cloud and performing matching, the processing speed of the estimation can be increased.
[0054] In step S60, the remote control command generation unit 250 generates a control command value by using the position and orientation of the vehicle 100 estimated in step S40 or S50, and transmits the generated control command value to the vehicle 100. Details of step S60 are as follows.
[0055] In step S60, the remote control command generation unit 250 first determines the target position to which the vehicle 100 should go next by using the vehicle position information including the position and orientation of the vehicle 100 and the target route TR. The remote control command generation unit 250 determines the target position on the target route TR in front of the current position of the vehicle 100, and generates a control command value for driving the vehicle 100 toward the target position. In the present embodiment, the control command value includes the acceleration and steering angle of the vehicle 100 as parameters. The remote control command generation unit 250 calculates the driving speed of the vehicle 100 according to the transition of the position of the vehicle 100, and compares the calculated driving speed with the target speed. As a whole, the remote control command generation unit 250 determines the acceleration so that the vehicle 100 accelerates when the driving speed is lower than the target speed, and determines the acceleration so that the vehicle 100 decelerates when the driving speed is higher than the target speed. The remote control command generation unit 250 determines the steering angle and acceleration so that the vehicle 100 does not deviate from the target route TR when the vehicle 100 is on the target route TR, and determines the steering angle and acceleration so that the vehicle 100 returns to the target route TR when the vehicle 100 is not on the target route TR (in other words, when the vehicle 100 deviates from the target route TR). In other embodiments, the control command value may include the speed of the vehicle 100 as a parameter instead of or in addition to the acceleration of the vehicle 100. The generated control command value is transmitted from the remote control device 200 to the vehicle 100.
[0056] The control process via the processor 111 of the vehicle 100 includes step S70 and step S80. In step S70, the vehicle controller 115 waits until a control command value is acquired from the remote control device 200. When the control command value is acquired, the process proceeds to step S80, and the vehicle controller 115 controls the actuator group 120 in response to the acquired control command value. With the remote control system 10 according to the present embodiment, the vehicle 100 can be driven via remote control, and the vehicle 100 can be moved along the target route TR without using a transport device such as a crane or a conveyor belt.
[0057] As described above, in the first embodiment, since the template matching is performed by excluding the defect corresponding portion corresponding to the defective portion of the three-dimensional point cloud data from the template point cloud, at least one of the position and orientation of the vehicle 100 can be correctly estimated even when a defective portion is generated in the three-dimensional point cloud data. Figure 3 According to the processing flow, the defect specifying unit 220 updates the defective part in response to the movement of the vehicle 100 so that the defective part can be updated to an appropriate state.
[0058] In the above-described embodiment, the vehicle 100 may have a configuration that can be moved by remote control, and may be a configuration of a platform including, for example, a configuration described below. Specifically, the vehicle 100 may include at least a vehicle controller 115 and a communication device 150 so as to exhibit three functions of "driving", "turning" and "stopping" by remote control. That is, the vehicle 100 that can be moved by remote control does not need to be equipped with at least a portion of internal components such as a driver's seat and an instrument panel, nor does it need to be equipped with at least a portion of external components such as a bumper and a rearview mirror, and does not need to be equipped with a body shell. In this case, the remaining components (such as a body shell, etc.) may be assembled to the vehicle 100 when the vehicle 100 is shipped from the factory, or the remaining components (such as a body shell, etc.) may be assembled to the vehicle 100 after the vehicle 100 is shipped from the factory without being assembled on the vehicle 100. For the form of the platform, the position determination may be performed in the same manner as the vehicle 100 in each of the above-described embodiments. B. Second Embodiment
[0059] Figure 5 1 is a block diagram showing the configuration of the vehicle 100 and the remote control device 200 according to the second embodiment. Figure 2 The configuration of the first embodiment shown in FIG. 1 is different mainly in the following three points. (1) The functions of the three-dimensional point cloud data acquisition unit 121 , the defect specifying unit 122 , the template point cloud modification unit 123 , the estimation unit 124 , and the control command generation unit 125 are added to the functions of the processor 111 of the vehicle 100 . (2) The memory 112 of the vehicle 100 stores the template point cloud TP, the vehicle information VI, the target route TR, and the defective portion map DM. (3) The functions of the three-dimensional point cloud data acquisition unit 210 , the defect specifying unit 220 , the template point cloud modification unit 230 , the estimation unit 240 , and the remote control command generation unit 250 are omitted from the functions of the processor 201 of the remote control device 200 .
[0060] Since the functions of the three-dimensional point cloud data acquisition unit 121, the defect specification unit 122, the template point cloud modification unit 123, the estimation unit 124 and the control command generation unit 125 are basically the same as those of the three-dimensional point cloud data acquisition unit 210, the defect specification unit 220, the template point cloud modification unit 230, the estimation unit 240 and the remote control command generation unit 250, their description will be omitted.
[0061] In the second embodiment, the vehicle 100 performs a process of performing template matching by excluding a defect corresponding portion corresponding to a defect portion of the three-dimensional point cloud data from the template point cloud to estimate at least one of the position and orientation of the vehicle 100. That is, in the second embodiment, the vehicle control device 110 of the vehicle 100 corresponds to the “control device” according to the present disclosure.
[0062] Figure 6 is a flowchart showing a processing flow of vehicle control according to the second embodiment. Figure 6 Steps S110 to S160 correspond to Figure 3 Note that in step S160, a control command value is created by the control command generation unit 125 of the vehicle 100 so that the control of the actuator group 120 of the vehicle 100 is performed without transmitting and receiving the control command value between the remote control device 200 and the vehicle 100.
[0063] Before the vehicle 100 starts traveling along the target route TR, the template point cloud TP, the vehicle information VI, the target route TR, and the defective portion map DM are stored in the memory 112 of the vehicle 100. These data may be provided from the remote control device 200 or the process management device 400, or may be written into the memory 112 of the vehicle 100 by using another means.
[0064] As described above, also in the second embodiment, as in the first embodiment, template matching is performed by excluding the defective portion corresponding to the defective portion of the three-dimensional point cloud data from the template point cloud, so that at least one of the position and orientation of the vehicle 100 can be accurately estimated even when the defective portion is generated in the three-dimensional point cloud data. C. Other Embodiments
[0065] In the above embodiments, “server 200” refers to remote control device 200, and “external sensor” refers to distance measuring device 300. Furthermore, “travel control signal” refers to control command, “reference route” refers to target route TR, and “global coordinate system” refers to reference coordinate system Σr.
[0066] (C1) In each of the above-described embodiments, the external sensor is a light detection and ranging (LiDAR). In contrast, the external sensor does not need to be a LiDAR, and may be, for example, a camera. For example, in the case where the external sensor is a camera, the server 200 detects the external shape of the vehicle 100 based on the captured image, calculates the coordinates of the positioning points of the vehicle 100 in the captured image coordinate system (i.e., the local coordinate system), and obtains the position of the vehicle 100 by transforming the calculated coordinates into coordinates in the global coordinate system. In addition, the external shape of the vehicle 100 included in the captured image can be detected, for example, by inputting the captured image into a detection model using artificial intelligence. The detection model is prepared, for example, inside the remote control system 10 or outside the remote control system 10, and is pre-stored in the memory of the server 200. Examples of detection models include trained machine learning models that have been trained to implement any semantic segmentation and instance segmentation. As a machine learning model, for example, a convolutional neural network (CNN) that has been trained by supervised learning using a training data set can be used. The training data set, for example, has a plurality of training images including the vehicle 100 and a label indicating whether each region in the training image indicates a region of the vehicle 100 or indicates a region outside the vehicle 100. When the CNN is trained, the parameters of the CNN are preferably updated by back propagation (error back propagation method) to reduce the error between the output result of the detection model and the label. The server 200 can perform estimation based on the orientation of the motion vector of the vehicle 100 calculated according to the position change of the feature point of the vehicle 100 between the frames of the captured image by using, for example, an optical flow method to obtain the orientation of the vehicle 100.
[0067] (C2) In the first embodiment, the server 200 performs processing from acquiring vehicle position information including the position and orientation of the vehicle 100 to generating a travel control signal. In contrast, the vehicle 100 may perform at least a portion of the processing from acquiring the vehicle position information to generating a travel control signal. For example, the following forms (1) to (3) may be used.
[0068] (1) The server 200 may acquire vehicle position information, determine a target position to which the vehicle 100 should go next, and generate a route from the current position of the vehicle 100 indicated by the acquired vehicle position information to the target position. The server 200 may generate a route to a target position between the current position and the destination, or may generate a route to the destination. The server 200 may transmit the generated route to the vehicle 100. The vehicle 100 may generate a travel control signal for causing the vehicle 100 to travel on the route received from the server 200, and control an actuator of the vehicle 100 by using the generated travel control signal.
[0069] (2) The server 200 may acquire vehicle position information and transmit the acquired vehicle position information to the vehicle 100. The vehicle 100 may determine a target position to which the vehicle 100 should go next, generate a route from the current position of the vehicle 100 indicated by the received vehicle position information to the target position, generate a travel control signal for causing the vehicle 100 to travel on the generated route, and control an actuator of the vehicle 100 by using the generated travel control signal.
[0070] (3) In the above forms (1) and (2), an internal sensor may be installed on the vehicle 100, and the detection result output from the internal sensor may be used for at least one of the generation of the route and the generation of the driving control signal. The internal sensor is a sensor installed on the vehicle 100. Specifically, the internal sensor may include, for example, a camera, a LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, and a gyroscope sensor. For example, in the above form (1), the server 200 may obtain the detection result of the internal sensor and reflect the detection result of the internal sensor in the route when generating the route. In the above form (1), the vehicle 100 may obtain the detection result of the internal sensor and reflect the detection result of the internal sensor in the driving control signal when generating the driving control signal. In the above form (2), the vehicle 100 may obtain the detection result of the internal sensor and reflect the detection result of the internal sensor in the route when generating the route. In the above form (2), the vehicle 100 may obtain the detection result of the internal sensor and reflect the detection result of the internal sensor in the driving control signal when generating the driving control signal.
[0071] (C3) In each of the above embodiments, an internal sensor may be installed on the vehicle 100, and the detection result output from the internal sensor may be used for at least one of the generation of the route and the generation of the travel control signal. For example, the vehicle 100 may obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating the route. For example, the vehicle 100 may obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the travel control signal when generating the travel control signal.
[0072] (C4) In the first embodiment, the server 200 automatically generates a travel control signal to be transmitted to the vehicle 100. In contrast, the server 200 may generate a travel control signal to be transmitted to the vehicle 100 in response to an operation of an external operator located outside the vehicle 100. For example, the external operator may operate an operating device including a display that displays a captured image output from an external sensor, a steering wheel, an accelerator pedal, and a brake pedal for remotely operating the vehicle 100, and a communication device that communicates with the server 200 via wired communication or wireless communication, and the server 200 may generate a travel control signal in response to the operation applied to the operating device.
[0073] (C5) In each of the above-mentioned embodiments, for example, the vehicle 100 only needs to have a configuration capable of moving by unmanned driving, and may have the form of a platform including the configuration described below. Specifically, the vehicle 100 only needs to include at least a control device for controlling the driving of the vehicle 100 and an actuator of the vehicle 100 so as to exhibit the three functions of "driving", "turning" and "stopping" by unmanned driving. In the case where the vehicle 100 obtains information of unmanned driving from the outside, the vehicle 100 only needs to further include a communication device. That is, the vehicle 100 capable of moving by unmanned driving does not need to be equipped with at least a part of the internal parts such as the driver's seat and the dashboard, nor does it need to be equipped with at least a part of the external parts such as the bumper and the rearview mirror, and does not need to be equipped with a body shell. In this case, the remaining parts (such as the body shell, etc.) can be assembled to the vehicle 100 when the vehicle 100 is shipped from the factory, or the remaining parts (such as the body shell, etc.) can be assembled to the vehicle 100 after the vehicle 100 is shipped from the factory without the remaining parts (such as the body shell, etc.) being assembled on the vehicle 100. Each component can be assembled from any direction such as the upper side, lower side, front side, rear side, right side or left side of the vehicle 100, and can also be assembled from the same direction or different directions. As for the form of the platform, the position can be determined in the same manner as the vehicle 100 in the first embodiment.
[0074] (C6) The vehicle 100 can be manufactured by combining a plurality of modules. A module refers to a unit composed of a plurality of components assembled depending on the part or function of the vehicle 100. For example, the platform of the vehicle 100 can be manufactured by combining a front module constituting the front of the platform, a center module constituting the center of the platform, and a rear module constituting the rear of the platform. In addition, the number of modules constituting the platform is not limited to 3, and can be less than 2 or more than 4. In addition to or instead of the components constituting the platform, the components constituting the parts of the vehicle 100 that are different from the platform can be modularized. In addition, the various modules can include any external components (such as a bumper or grille) or any internal components (such as a seat or a console). In addition, in addition to the vehicle 100, any aspect of a mobile body can be manufactured by combining modules. Such a module can be manufactured by joining the components via welding or fasteners, etc., or can be manufactured by molding at least a portion of the components constituting the module into one component as a whole by casting. The molding method of molding a component (especially a relatively large component) as a whole is also called giga casting or mega casting. For example, the front module, the center module, and the rear module may be manufactured by using one-piece casting.
[0075] (C7) Transportation of the vehicle 100 using unmanned driving to make the vehicle 100 travel is also referred to as "autonomous transportation". A configuration for implementing autonomous transportation is also referred to as a "vehicle remote-controlled autonomous driving transportation system". A production method for producing the vehicle 100 by using autonomous transportation is also referred to as "autonomous production". In autonomous production, for example, at a factory that manufactures the vehicle 100, at least a portion of the transportation of the vehicle 100 is implemented by autonomous transportation.
[0076] The present disclosure is not limited to the above-described embodiments, and can be implemented in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features of each form described in the Summary of the Invention section may be appropriately replaced or combined to solve some or all of the above-described purposes, or to achieve some or all of the above-described effects. In the case where technical features are not described as essential features in this specification, these features may be deleted as appropriate.
Claims
1. A control device that generates a control command for controlling a moving body by using three-dimensional point cloud data of the moving body measured by a distance measuring device, characterized in that: The control device includes an estimation unit, which is configured to perform matching between a substantial point cloud portion obtained by excluding a defect corresponding portion corresponding to the defective portion from a template point cloud and the three-dimensional point cloud data in a case where a defective portion is generated in the three-dimensional point cloud data acquired from the distance measuring device, so as to estimate at least one of the position and orientation of the moving body.
2. The control device according to claim 1, characterized in that: The method further comprises a template point cloud modification unit, wherein the template point cloud modification unit is configured to delete the defect corresponding portion from the template point cloud to create a modified template point cloud. The estimation unit is configured to perform matching between the three-dimensional point cloud data and the modified template point cloud to estimate at least one of the position and orientation of the moving object.
3. The control device according to claim 1, characterized in that: It also includes a defect specifying unit configured to specify the defective portion of the three-dimensional point cloud data.
4. The control device according to claim 3, characterized in that: The defect specifying unit is configured to specify the defective portion of the three-dimensional point cloud data acquired from the distance measuring device with reference to the history of the three-dimensional point cloud data.
5. The control device according to claim 3, characterized in that: The defect specifying unit is configured to specify the defective portion depending on the distance between the moving body and the distance measuring device by using a map in which the defective portion depending on the distance between the moving body and the distance measuring device is registered.
