Control device
The vehicle position and orientation are estimated by combining point cloud data of multiple distance measurement devices, and the problem of estimating accuracy and processing speed trade-offs in the prior art is solved, and high accuracy and rapid control are achieved in the transmission of autonomous vehicle driving.
Patent Information
- Application Number
- CN202410696195.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-10-23
- Filing Date
- 2024-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
In the process of autonomous driving and transmission of vehicles, the estimation accuracy of the position and orientation of the movable body is not sufficiently improved, and there is a trade-off relationship between the estimation accuracy and the processing speed, making it difficult to prioritize one according to the situation.
Using a combined point cloud data of multiple distance measurement devices, combined point cloud data is generated through point cloud combination unit, used to estimate the position and orientation of the movable body, and select a suitable estimation processing method according to the allowable processing time and accuracy requirements. Improve estimation accuracy or processing speed.
By combining point cloud data, the estimation accuracy of the position and orientation of the movable body is improved, and the estimated accuracy or processing speed is given priority according to the vehicle's driving condition, thereby achieving stable driving control.
Smart Images

Figure CN119065282A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a control device for a movable body. Background Art
[0002] A technology for transmitting autonomous driving of a vehicle during the manufacturing process of the vehicle is known (Japanese Unexamined Patent Application Publication (Translation of PCT Application) No. 2017-538619). Summary of the invention
[0003] When a movable body such as a vehicle moves by autonomous driving transmission, a process of estimating the position and orientation of the movable body is performed. The position and orientation of the movable body can be estimated using three-dimensional point cloud data acquired by a distance measuring device such as a camera and a radar. In the estimation process, short-cycle calculations are required for the stability of the driving control of the movable body. However, in JP2017-538619A, research for improving the estimation accuracy of the position and orientation of the movable body has not been fully conducted. In addition, in the estimation of the position and orientation of the movable body, the estimation accuracy and the processing speed have a trade-off relationship, and therefore, it is desirable to give priority to one of the estimation accuracy and the processing speed according to the situation.
[0004] The present disclosure provides a control device that improves the estimation accuracy of the position and orientation of a movable body.
[0005] A control device according to a first aspect of the present disclosure is configured to generate a control command for controlling a movable body using measurement results of a plurality of distance measuring devices. The control device includes a point cloud combining unit configured to create combined point cloud data by combining two or more pieces of three-dimensional point cloud data obtained by two or more distance measuring devices among the plurality of distance measuring devices. The control device includes an estimating unit configured to perform a first estimation process of estimating at least one of a position and an orientation of the movable body using the combined point cloud data. With this control device, since combined point cloud data obtained by combining two or more pieces of three-dimensional point cloud data is used, the estimation accuracy of the position and orientation of the movable body can be improved.
[0006] In the control device according to the first aspect of the present disclosure, the estimation unit may be configured to: execute the first estimation process when an allowable processing time of the estimation process is equal to or longer than a time reference value. With this control device, when the allowable processing time of the first estimation process is long, the estimation accuracy can be improved using the combined point cloud data.
[0007] In the control device according to the first aspect of the present disclosure, the estimation unit may be configured to: when the allowable processing time is shorter than the time reference value, perform a second estimation process of estimating at least one of the position and orientation of the movable body using a single piece of three-dimensional point cloud data. With this control device, when the allowable processing time of the estimation processing is short, the processing speed can be increased using a single piece of three-dimensional point cloud data.
[0008] In the control device according to the first aspect of the present disclosure, the estimation unit may be configured to execute the first estimation process when a required accuracy of the control of the movable body is equal to or higher than an accuracy reference value. With this control device, when the required accuracy of the control of the movable body is high, it is possible to improve the estimation accuracy using the combined point cloud data.
[0009] In the control device according to the first aspect of the present disclosure, the estimation unit can be configured to: select and execute one of a first estimation process and a second estimation process of estimating at least one of the position and orientation of the movable body using a single three-dimensional point cloud data according to the driving condition of the movable body. With this control device, it is possible to give priority to one of the estimation accuracy and the processing speed according to the travel condition of the movable body. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which like symbols represent like elements, and in which: Figure 1 is a conceptual diagram showing the configuration of a remote control system in the embodiment; Figure 2 is a block diagram showing the configuration of a vehicle and a remote control device in the first embodiment; Figure 3 is a flowchart showing the processing flow of remote control in the first embodiment; Figure 4 is an explanatory diagram showing an example of a change in the number of pieces of three-dimensional point cloud data to be used; Figure 5 is a block diagram showing the configuration of a vehicle and a remote control device in a second embodiment; and Figure 6 : is a flowchart showing the processing flow of vehicle control in the second embodiment. DETAILED DESCRIPTION A. First Embodiment
[0011] Figure 1is a conceptual diagram showing the configuration of the remote control system 10 in the embodiment. The remote control system 10 includes one or more vehicles 100 as movable bodies, a remote control device 200 that generates a control command for remote control of each vehicle 100 and transmits the control command to the vehicle 100, a plurality of distance measuring devices 300 that measure three-dimensional point cloud data about the vehicle 100, and a process management device 400 that manages the manufacturing process of the vehicle 100. In the first embodiment, the remote control device 200 is an example of a “control device” in the present disclosure.
[0012] The vehicle 100 is preferably a battery electric vehicle (BEV). The movable body is not limited to a battery electric vehicle, and may be, for example, a gasoline vehicle, a hybrid electric vehicle, or a fuel cell electric vehicle. The movable body is not limited to the vehicle 100, and may be, for example, an electric vertical lift (so-called flying vehicle).
[0013] In the present disclosure, a "movable body" refers to a physical body that can move. The vehicle may be a vehicle that travels using wheels, or may be a vehicle that performs track travel, and is, for example, a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, etc. In the case where the movable body is not a vehicle, the expression "vehicle" and the expression "car" in the present disclosure may be replaced with "movable body" as appropriate, and the expression "travel" may be replaced with "move" as appropriate.
[0014] The vehicle 100 is configured to be able to travel by unmanned driving. "Unmanned driving" means driving that does not rely on the driving operation of the occupants. The driving operation refers to an operation related to at least one of "moving", "turning" and "stopping" of the vehicle 100. Unmanned driving is implemented by automatic or manual remote control using a device positioned outside the vehicle 100 or by autonomous control of the vehicle 100. The vehicle 100 that travels by unmanned driving may be equipped with occupants who do not perform driving operations. Examples of occupants who do not perform driving operations include persons who only sit on seats of the vehicle 100, and persons who perform work different from driving operations (exemplified by installing, checking or operating switches) in the vehicle 100. Driving that relies on the driving operation of the occupants is sometimes referred to as "manned driving".
[0015] In this specification, “remote control” is “full remote control” in which all actions of the vehicle 100 are completely determined from the outside of the vehicle 100, and “partial remote control” in which some of the actions of the vehicle 100 are determined from the outside of the vehicle 100. In addition, “autonomous control” includes “full autonomous control” in which the vehicle 100 autonomously controls its actions without receiving information from a device outside the vehicle 100 at all, and “partial autonomous control” in which the vehicle 100 autonomously controls its actions using information received from a device outside the vehicle 100.
[0016] In the present embodiment, remote control of the vehicle 100 is performed in a factory that manufactures the vehicle 100. The factory includes a first location PL1 and a second location PL2. The first location PL1 is, for example, a location where assembly of the vehicle 100 is performed, and the second location PL2 is, for example, a location where inspection of the vehicle 100 is performed. The first location PL1 and the second location PL2 are connected by a travel road SR along which the vehicle 100 can travel. An arbitrary position in the factory is expressed as x, y, and z coordinate values in a reference coordinate system ∑r.
[0017] A plurality of distance measuring devices 300 using the vehicle 100 as a measurement object are installed around the travel road SR. The remote control device 200 can use the three-dimensional point cloud data measured by each distance measuring device 300 to acquire the position and orientation of the vehicle 100 relative to the target route TR in real time. As the distance measuring device 300, a camera and light detection and ranging (Light Detection And Ranging, LiDAR) can be used. In particular, LiDAR is preferred because highly accurate three-dimensional point cloud data can be obtained. Preferably, the plurality of distance measuring devices 300 are arranged so that when the vehicle 100 exists at any position on the target route TR, two or more distance measuring devices 300 can always perform measurement on the vehicle 100. The position of each distance measuring device 300 is fixed, and the relative relationship between the reference coordinate system ∑r and the device coordinate system of each distance measuring device 300 is known in advance. The coordinate conversion matrix for mutual conversion between the coordinate values in the reference coordinate system ∑r and the coordinate values in the device coordinate system of each distance measuring device 300 is pre-stored in the remote control device 200.
[0018] The remote control device 200 generates a control command for causing the vehicle 100 to travel along the target route TR, and transmits the control command to the vehicle 100. The vehicle 100 travels according to the received control command. Therefore, in the remote control system 10, the vehicle 100 can be moved from the first location PL1 to the second location PL2 by remote control without using a conveying device such as a crane or a conveyor.
[0019] Figure 2 1 is a block diagram showing the configuration of the vehicle 100 and the remote control device 200. The vehicle 100 includes: a vehicle control device 110 for controlling components of the vehicle 100, an actuator group 120 driven under the control of the vehicle control device 110, a communication device 130 for communicating with the remote control device 200 through wireless communication, and a GPS receiver 140 for acquiring position information about the vehicle 100. In the present embodiment, the actuator group 120 includes an actuator of a driving device for accelerating the vehicle 100, an actuator of a steering device for changing the moving direction of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100. The driving device includes: a battery, a driving motor driven by the power of the battery, and a driving wheel rotated by the driving motor. The actuator of the driving device includes a driving motor. The actuator group 120 may also include an actuator for swinging a windshield wiper of the vehicle 100, an actuator for opening and closing an electric window of the vehicle 100, and the like.
[0020] The vehicle control device 110 is composed of a computer including a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected in a bidirectional communication manner via the internal bus 114. The input / output interface 113 is connected to the actuator group 120, the communication device 130, and the GPS receiver 140.
[0021] In the present embodiment, the processor 111 functions as a vehicle control unit 115 and a position information acquisition unit 116 by executing a program PG1 pre-stored in the memory 112. The vehicle control unit 115 controls the actuator group 120. When the driver rides in the vehicle 100, the vehicle control unit 115 can make the vehicle 100 travel by controlling the actuator group 120 according to the driver's operation. In addition, regardless of whether the driver rides in the vehicle 100, the vehicle control unit 115 can make the vehicle 100 travel by controlling the actuator group 120 according to a control command transmitted from the remote control device 200. The position information acquisition unit 116 uses the GPS receiver 140 to acquire position information indicating the current location of the vehicle 100. The position information acquisition unit 116 and the GPS receiver 140 may not be included.
[0022] The remote control device 200 is composed of a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are connected in a bidirectional communication manner via the internal bus 204. The input / output interface 203 is connected to a communication device 205 for communicating with the vehicle 100, the distance measurement device 300, and the process management device 400 via wireless communication.
[0023] In the present embodiment, the processor 201 functions as a three-dimensional point cloud data acquisition unit 210 , a point cloud combination unit 220 , an estimation unit 230 , and a remote control command generation unit 240 by executing a program PG2 pre-stored in the memory 202 .
[0024] The three-dimensional point cloud data acquisition unit 210 acquires three-dimensional point cloud data measured by the distance measurement device 300. The three-dimensional point cloud data is data indicating the three-dimensional position of the point cloud detected by the distance measurement device 300.
[0025] The point cloud combining unit 220 selects two or more distance measuring devices 300 from the plurality of distance measuring devices 300 included in the remote control system 10, and creates combined point cloud data by combining two or more pieces of three-dimensional point cloud data obtained from the two or more distance measuring devices 300. As a method of combining the plurality of pieces of three-dimensional point cloud data, for example, one of the following methods may be employed. Point cloud combination method M1
[0026] The coordinate values of each point in the three-dimensional point cloud data obtained by each distance measuring device 300 are converted from the device coordinate system of the distance measuring device 300 to a specific coordinate system such as the reference coordinate system Σr, and the converted coordinate values are summed. At this time, if there are a plurality of points whose difference in coordinate values after conversion is less than or equal to the allowable error, the plurality of points are replaced with one representative point. The coordinate value of the representative point is a representative value (such as an average value) of the coordinate values of the plurality of points. Point cloud combination method M2
[0027] (a) Performing position adjustment by matching on the first three-dimensional point cloud data and the second three-dimensional point cloud data, replacing corresponding points with one representative point, and adding points without corresponding points without change, thereby creating first combined point cloud data. The position coordinates of the representative point are representative values (such as an average value) of the position coordinates of the two corresponding points. (b) Creating second combined point cloud data by performing the same processing as processing (a) on the first combined point cloud data and the third three-dimensional point cloud data. Thereafter, by repeating process (b), any number of three-dimensional point cloud data can be combined. For position adjustment by matching, for example, an iterative closest point algorithm (ICP (Interactive Closest Point) algorithm) can be used. In terms of processing speed, the above-mentioned combination method M1 is more preferable.
[0028] The estimation unit 230 estimates the position and orientation of the vehicle 100 using the combined point cloud data obtained by the point cloud combination unit 220 or the single three-dimensional point cloud data obtained by the single distance measuring device 300. In the present embodiment, the estimation unit 230 estimates the position and orientation of the vehicle 100 by performing template matching using the template point cloud TP stored in the memory 202. In the case where the three-dimensional point cloud data cannot be used, the estimation unit 230 may estimate the position and orientation of the vehicle 100 using the driving history of the vehicle 100 and the position information detected by the GPS receiver 140 equipped in the vehicle 100. The estimation unit 230 may estimate only one of the position and orientation of the vehicle 100. In this case, the other of the position and orientation of the vehicle 100 is determined using the driving history of the vehicle 100, etc.
[0029] Information about the position and orientation of the vehicle is also referred to as “vehicle position information.” In the present embodiment, the vehicle position information includes the position and orientation of the vehicle 100 in the reference coordinate system of the factory.
[0030] The remote control command generation unit 240 generates a control command for remote control using the estimated position and orientation of the vehicle 100, and transmits the control command to the vehicle 100. The control command is a command to cause the vehicle 100 to travel along the target route TR stored in the memory 202. The control command can be generated as a command including a driving force or a braking force and a steering angle. Alternatively, the control command can be generated as a command including at least one of the position and orientation of the vehicle 100 and the route along which the vehicle 100 will travel.
[0031] In the present embodiment, the control command includes the acceleration and the steering angle of vehicle 100 as parameters. In another embodiment, the control command may include the speed of vehicle 100 as a parameter instead of or in addition to the acceleration of vehicle 100.
[0032] The process management device 400 manages the entire manufacturing process of the vehicle 100 in the factory. For example, when one vehicle 100 starts traveling along the target route TR, individual information indicating the identification number, model, and other of the vehicle 100 is transmitted from the process management device 400 to the remote control device 200. The position of the vehicle 100 detected by the remote control device 200 is also transmitted to the process management device 400. The functions of the process management device 400 may be implemented in the same device as the remote control device 200.
[0033] The remote control device 200 is also referred to as a “server”, and the distance measuring device 300 is also referred to as an “external sensor”. In addition, the control command is also referred to as a “driving control signal”, the target route TR is also referred to as a “reference path”, and the reference coordinate system is also referred to as a “global coordinate system”.
[0034] Figure 3 1 is a flowchart showing the processing flow of the remote control in the first embodiment. The processing of the remote control device 200 is performed in a constant cycle. Alternatively, each time the three-dimensional point cloud data acquisition unit 210 newly acquires three-dimensional point cloud data from a plurality of distance measurement devices 300 performing measurement on the vehicle 100 as the control object, the remote control device 200 may execute the remote control process. Figure 3 The three-dimensional point cloud data acquisition unit 210 may perform pre-processing to remove background point cloud data indicating a stationary object from the newly acquired three-dimensional point cloud data.
[0035] In step S10, the point cloud combination unit 220 determines whether the required accuracy of the remote control performed by the remote control device 200 is equal to or higher than the accuracy reference value. For example, the required accuracy of the remote control is set in advance for each of the plurality of sections set along the target route TR. In addition, the required accuracy may be set according to the driving condition of the vehicle 100. The position of the vehicle 100 when the required accuracy is determined may be the position estimated in the last estimation process, or may be the position estimated based on the driving history of the vehicle 100. The accuracy reference value may be a preset threshold value, or the accuracy reference value may be changed according to the driving condition of the vehicle 100.
[0036] When it is determined that the required accuracy is lower than the accuracy reference value, the process proceeds to step S50, and the estimation unit 230 selects one distance measurement device 300 from the plurality of distance measurement devices 300, and performs an estimation process (second estimation process) of estimating the position and orientation of the vehicle 100 using a single piece of three-dimensional point cloud data obtained by the selected distance measurement device 300. The estimation process may be performed by performing matching between the three-dimensional point cloud data and the template point cloud TP.
[0037] On the other hand, when it is determined that the required accuracy is equal to or higher than the accuracy reference value, the process proceeds to step S20, and the point cloud combination unit 220 determines whether the allowable processing time for the estimation process of the position and orientation of the vehicle 100 is equal to or longer than the time reference value. The allowable processing time is set to a time in which no failure occurs in the remote control even when the processing time of the estimation process is long. For example, the allowable processing time is set in advance for each of a plurality of sections set along the target route TR. In addition, the allowable processing time can be set according to the driving condition of the vehicle 100. When the processing time of the estimation process can be sufficiently ensured, for example, before the remote control starts, during a temporary stop of the vehicle 100, or during low-speed driving of the vehicle 100, the allowable processing time is set to a long time. The time reference value may be a preset threshold value, or the time reference value may be changed according to the driving condition of the vehicle 100.
[0038] When it is determined that the allowable processing time is shorter than the time reference value, the process proceeds to the above-described step S50 , and the position and orientation of the vehicle 100 are estimated using a single piece of three-dimensional point cloud data.
[0039] On the other hand, when it is determined that the allowable processing time is equal to or longer than the time reference value, the process proceeds to step S30. In step S30, the point cloud combining unit 220 selects two or more distance measuring devices 300 from the plurality of distance measuring devices 300 included in the remote control system 10, and creates combined point cloud data by combining two or more pieces of three-dimensional point cloud data obtained from the two or more distance measuring devices 300. In step S40, the estimating unit 230 estimates the position and orientation of the vehicle 100 by performing matching between the combined point cloud data and the template point cloud TP (first estimation process).
[0040] In step S60, the remote control command generation unit 240 generates a control command value using the position and orientation of the vehicle 100 estimated in step S40 or step S50, and transmits the control command value to the vehicle 100. Details of step S60 will be described below.
[0041] In step S60, first, the remote control command generation unit 240 uses the vehicle position information including the position and orientation of the vehicle 100 and the target route TR to determine the target position to which the vehicle 100 will go from now on. The remote control command generation unit 240 determines the target position on the target route TR in front of the current location of the vehicle 100, and generates a control command value for driving the vehicle 100 toward the target position. In the present embodiment, the control command value includes the acceleration and steering angle of the vehicle 100 as parameters. The remote control command generation unit 240 calculates the driving speed of the vehicle 100 according to the transition of the position of the vehicle 100, and compares the calculated driving speed with the target speed. Overall, the remote control command generation unit 240 determines the acceleration to accelerate the vehicle 100 when the driving speed is lower than the target speed, and determines the acceleration to decelerate the vehicle 100 when the driving speed is higher than the target speed. In addition, the remote control command generation unit 240 determines the steering angle and acceleration so that the vehicle 100 does not deviate from the target route TR when the vehicle 100 is located on the target route TR, and determines the steering angle and acceleration so that the vehicle 100 returns to the target route TR when the vehicle 100 is not located on the target route TR, in other words, when the vehicle 100 deviates from the target route TR. In another embodiment, the control command value may include the speed of the vehicle 100 as a parameter instead of or in addition to the acceleration of the vehicle 100. The control command value generated in this way is transmitted from the remote control device 200 to the vehicle 100.
[0042] The control process of the processor 111 of the vehicle 100 includes step S70 and step S80. In step S70, the vehicle control unit 115 waits until the control command value is acquired from the remote control device 200. When the control command value is acquired, the process proceeds to step S80, and the vehicle control unit 115 controls the actuator group 120 according to the acquired control command value. In the remote control system 10 in the present embodiment, it is possible to drive the vehicle 100 by remote control and move the vehicle 100 along the target route TR without using a transport device such as a crane or a conveyor.
[0043] Figure 4 FIG. 2 is an explanatory diagram showing an example of changing the number of pieces of three-dimensional point cloud data used. Figure 4 In the example shown in the upper part of , when the required accuracy of the remote control is lower than the accuracy reference value, only a single piece of three-dimensional point cloud data obtained by one distance measuring device 300_1 is used for the estimation process. As the single piece of three-dimensional point cloud data for the estimation process, for example, the three-dimensional point cloud data in which the number of points is the largest is selected. Alternatively, it is allowed to select the three-dimensional point cloud data obtained by one distance measuring device 300 presumed to be installed at the best position for the estimation process from the driving history of the vehicle 100.
[0044] exist Figure 4 In the example in the lower middle part, when the required accuracy of the remote control is equal to or higher than the accuracy reference value, three pieces of three-dimensional point cloud data obtained by three distance measuring devices 300_1, 300_2, and 300_3 are used for estimation processing. In this case, the three pieces of three-dimensional point cloud data are combined, and combined point cloud data are created. For example, two or more pieces of three-dimensional point cloud data for estimation processing are selected in the order of the three-dimensional point cloud data with the largest number of points. Alternatively, it is allowed to select two or more three-dimensional point cloud data obtained by two or more distance measuring devices 300 that are presumed to be installed at the best position for estimation processing from the driving history of the vehicle 100. In addition, the two or more distance measuring devices 300 used for estimation processing should preferably include distance measuring devices 300 that observe the vehicle 100 from different directions, and in particular should include at least two distance measuring devices 300 that observe the vehicle 100 from the front and rear of the vehicle 100, respectively.
[0045] The number of pieces of three-dimensional point cloud data used for estimation processing should preferably be set to be larger as the required accuracy of remote control is higher. In addition, the number of pieces of three-dimensional point cloud data used for estimation processing should preferably be set to be larger as the allowable processing time of estimation processing is longer.
[0046] The estimation unit 230 in this embodiment performs estimation processing using the combined point cloud data when the required accuracy of the remote control is equal to or higher than the accuracy reference value. Thus, when the required accuracy of the remote control is high, the estimation accuracy can be improved.
[0047] In addition, when the allowable processing time of the estimation process is equal to or longer than the time reference value, the estimation unit 230 uses the combined point cloud data to perform the estimation process. Thus, when the allowable processing time of the estimation process is long, the estimation accuracy can be improved. In addition, when the allowable processing time is shorter than the time reference value, the estimation unit 230 uses a single piece of three-dimensional point cloud data to perform the estimation process. Thus, when the allowable processing time of the estimation process is short, the processing speed can be improved. One of the above steps S10 and S20 can be omitted.
[0048] Preferably, the required accuracy of the remote control and the allowable processing time of the estimation process are changed according to the running condition of the vehicle 100. The running condition in this case refers to the position and speed of the vehicle 100 on the target route TR.
[0049] As described above, in the first embodiment, combined point cloud data is created by combining two or more pieces of three-dimensional point cloud data, and at least one of the position and orientation of the vehicle 100 is estimated using the combined point cloud data. Therefore, the estimation accuracy of the position and orientation of the movable body can be improved. In addition, according to the driving condition of the vehicle 100, at least one of the position and orientation of the vehicle 100 is estimated by selecting one of the estimation process using the combined point cloud data obtained by combining two or more pieces of three-dimensional point cloud data and the estimation process using a single piece of three-dimensional point cloud data. Therefore, one of the estimation accuracy and the processing speed can be given priority according to the driving condition.
[0050] In the present embodiment, as the driving condition of the vehicle 100, the position and speed of the vehicle 100 on the target route TR are used, but the driving condition of the vehicle 100 may be specified based on factors other than the position and speed of the vehicle 100. For example, the orientation and steering angle of the vehicle 100 may be used as the driving condition.
[0051] In the above-mentioned embodiments, the vehicle 100 only needs to have a configuration in which the vehicle 100 can be moved by remote control, and for example, it can have a platform containing the configuration described below. Specifically, the vehicle 100 only needs to include at least a vehicle control unit 115 and a communication device 130 for realizing the three functions of "travel", "turn" and "stop" by remote control. That is, the vehicle 100 that can be moved by remote control does not need to be provided with a driver's seat and at least some internal components (such as an instrument panel), nor does it need to be provided with at least some external components (such as a bumper and a fender), and does not need to be provided with a body shell. In this case, other components (such as a body shell) can be attached to the vehicle 100 before the vehicle 100 is shipped from the factory, or, other components (such as a body shell) can be attached to the vehicle 100 after the vehicle 100 is shipped from the factory in a state where the vehicle 100 has not yet attached other components (such as a body shell). In addition, in the case of a platform, position determination can also be performed similarly to the vehicle 100 of these embodiments. B. Second Embodiment
[0052] Figure 5 2 is a block diagram showing the configuration of the vehicle 100 and the remote control device 200 in the second embodiment. Figure 2 The differences between the configurations shown in are mainly the following three points. (1) The functions of the three-dimensional point cloud data acquisition unit 121 , the point cloud combination unit 122 , the estimation unit 123 , and the control command generation unit 124 are added to the functions of the processor 111 of the vehicle 100 . (2) The template point cloud TP and the target route TR are stored in the memory 112 of the vehicle 100 . (3) The functions of the three-dimensional point cloud data acquisition unit 210 , the point cloud combination unit 220 , the estimation unit 230 , and the remote control command generation unit 240 are not included in the functions of the processor 201 of the remote control device 200 .
[0053] The functions of the three-dimensional point cloud data acquisition unit 121, the point cloud combination unit 122, the estimation unit 123 and the control command generation unit 124 are almost the same as those of the three-dimensional point cloud data acquisition unit 210, the point cloud combination unit 220, the estimation unit 230 and the remote control command generation unit 240, respectively, and therefore their description is omitted.
[0054] In the second embodiment, the process of creating combined point cloud data by combining two or more pieces of three-dimensional point cloud data and using the combined point cloud data to estimate at least one of the position and orientation of the vehicle 100 is performed by the vehicle 100. That is, in the second embodiment, the vehicle control device 110 of the vehicle 100 corresponds to the "control device" in the present disclosure.
[0055] Figure 6 : is a flowchart showing the processing flow of vehicle control in the second embodiment. Figure 6 Steps S110 to S160 in the first embodiment correspond to Figure 3 In step S160 , a control command value is created by the control command generation unit 124 of the vehicle 100 , and thus control of the actuator group 120 of the vehicle 100 is performed without transmitting and receiving the control command value between the remote control device 200 and the vehicle 100 .
[0056] Before the vehicle 100 starts traveling along the target route TR, the template point cloud TP and the target route TR are stored in the memory 112 of the vehicle 100. The template point cloud TP and the target route TR may be supplied from the remote control device 200 or the process management device 400, or may be written in the memory 112 of the vehicle 100 using other devices.
[0057] As described above, in the second embodiment, similarly to the first embodiment, combined point cloud data is created by combining two or more pieces of three-dimensional point cloud data, and at least one of the position and orientation of the movable body is estimated using the combined point cloud data. Therefore, the estimation accuracy of the position and orientation of the movable body can be improved. In addition, according to the driving condition of the vehicle 100, at least one of the position and orientation of the vehicle 100 is estimated by selecting one of the estimation processing using the combined point cloud data obtained by combining two or more pieces of three-dimensional point cloud data and the estimation processing using a single piece of three-dimensional point cloud data. Therefore, one of the estimation accuracy and the processing speed can be given priority according to the driving condition. C. Other Embodiments
[0058] In the embodiment described below, "server 200" refers to the remote control device 200, and "external sensor" refers to the distance measuring device 300. In addition, "travel control signal" refers to a control command, "reference path" refers to the target route TR, and "global coordinate system" refers to the reference coordinate system Σr.
[0059] (C1) In the above embodiment, the external sensor is a light detection and ranging (LiDAR). However, the external sensor does not need to be a LiDAR, and may be, for example, a camera. In the case where the external sensor is a camera, the server 200 acquires the position of the vehicle 100 by, for example, detecting the external shape of the vehicle 100 based on a captured image, calculating the coordinates of the position measurement point of the vehicle 100 in the coordinate system of the captured image (i.e., the local coordinate system), and converting the calculated coordinates into coordinates in the global coordinate system. The external shape of the vehicle 100 included in the captured image can be detected, for example, by inputting the captured image into a detection model using an artificial model. For example, the detection model is prepared inside the system 10 or outside the system 10, and is pre-stored in the memory of the server 200. Examples of the detection model include a machine learning model that has performed learning to achieve one of semantic segmentation and instance segmentation. As a machine learning model, for example, a convolutional neural network (hereinafter referred to as a CNN) that has performed learning by supervised learning using a learning data set can be used. For example, the learning data set includes a plurality of training images including the vehicle 100, and a label indicating whether each region in the training image is a region of the vehicle 100 or a region other than the vehicle 100. When learning the CNN, the parameters in the CNN are preferably updated by a back propagation method to reduce the error between the output result of the detection model and the label. In addition, for example, the server 200 can use an optical flow method to perform an estimation based on the orientation of the moving vector of the vehicle 100 calculated according to the position change of the feature point of the vehicle 100 between the frames of the captured image to obtain the orientation of the vehicle 100.
[0060] (C2) In the first embodiment described above, the processing from acquiring the vehicle position information including the position and orientation of the vehicle 100 to generating the travel control signal is performed by the server 200. However, at least a part of the processing from acquiring the vehicle position information to generating the travel control signal may be performed in the vehicle 100. For example, the following modes (1) to (3) may be adopted.
[0061] (1) The server 200 may acquire vehicle position information, may determine a target position to which the vehicle 100 will go from now on, and may generate a path from the current position of the vehicle 100 shown in the acquired vehicle position information to the target position. The server 200 may generate a path to the target position between the current position and the destination, or may generate a path to the destination. The server 200 may send the generated path to the vehicle 100. The vehicle 100 may generate a travel control signal for causing the vehicle 100 to travel on the path received from the server 200, and may control an actuator of the vehicle 100 using the generated travel control signal.
[0062] (2) The server 200 may acquire vehicle position information, and may transmit the acquired vehicle position information to the vehicle 100. The vehicle 100 may determine a target position to which the vehicle 100 will go from now on, may generate a path from the current location of the vehicle 100 shown in the received vehicle position information to the target position, may generate a travel control signal for causing the vehicle 100 to travel on the generated path, and may control an actuator of the vehicle 100 using the generated travel control signal.
[0063] (3) In the above modes (1) and (2), the vehicle 100 may be equipped with an internal sensor, and the detection results output from the internal sensor may be used for at least one of the generation of the path and the generation of the driving control signal. The internal sensor is a sensor equipped in the vehicle 100. Specifically, for example, the internal sensor may include a camera, LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyroscope sensor, etc. For example, in the above mode (1), the server 200 may obtain the detection results of the internal sensor, and may reflect the detection results of the internal sensor in the path when generating the path. In the above mode (1), the vehicle 100 may obtain the detection results of the internal sensor, and may reflect the detection results of the internal sensor in the driving control signal when generating the driving control signal. In the above mode (2), the vehicle 100 may obtain the detection results of the internal sensor, and may reflect the detection results of the internal sensor in the path when generating the path. In the above-mentioned mode (2), the vehicle 100 can obtain the detection result of the internal sensor, and can reflect the detection result of the internal sensor in the travel control signal when generating the travel control signal.
[0064] (C3) In the above embodiment, the vehicle 100 may be equipped with an internal sensor, and the detection result output from the internal sensor may be used for at least one of the generation of the route and the generation of the travel control signal. For example, the vehicle 100 may obtain the detection result of the internal sensor, and may reflect the detection result of the internal sensor in the route when generating the route. The vehicle 100 may obtain the detection result of the internal sensor, and may reflect the detection result of the internal sensor in the travel control signal when generating the travel control signal.
[0065] (C4) In the first embodiment, the server 200 automatically generates the travel control signal sent to the vehicle 100. However, the server 200 may generate the travel control signal sent to the vehicle 100 according to the operation of an external operator outside the vehicle 100. For example, the external operator may operate a manipulation device including a display device that displays a captured image output from an external sensor, a steering wheel, an accelerator pedal, and a brake pedal for remotely operating the vehicle 100, and a communication device for communicating with the server 200 through wired communication or wireless communication, and the server 200 may generate the travel control signal according to the operation of the manipulation device.
[0066] (C5) In the above embodiment, the vehicle 100 only needs to have a configuration in which the vehicle 100 can be moved by unmanned driving, and for example, it can have a platform containing the configuration described below. Specifically, the vehicle 100 only needs to include at least a control device for controlling the travel of the vehicle 100 and an actuator of the vehicle 100 for realizing the three functions of "traveling", "turning" and "stopping" by unmanned driving. In the case where the vehicle 100 obtains information for unmanned driving from the outside, the vehicle 100 only needs to further include a communication device. That is, the vehicle 100 that can be moved by unmanned driving does not need to be provided with a driver's seat and at least some internal components (such as an instrument panel), does not need to be provided with at least some external components (such as a bumper and a fender), and does not need to be provided with a body shell. In this case, other components (such as a body shell) can be attached to the vehicle 100 before the vehicle 100 is shipped from the factory, or other components (such as a body shell) can be attached to the vehicle 100 after the vehicle 100 is shipped from the factory in a state where the vehicle 100 has not yet attached other components (such as a body shell). The components can be attached from any direction of the vehicle 100, such as the upper side, the lower side, the front side, the rear side, the right side, and the left side, and can be attached from the same direction as each other, or can be attached from different directions from each other. In addition, in the case of a platform, position determination can also be performed similarly to the vehicle 100 of the first embodiment.
[0067] (C6) The vehicle 100 can be manufactured by combining a plurality of modules. A module refers to a unit composed of a plurality of components gathered according to the location or function of the vehicle 100. For example, the platform of the vehicle 100 can be manufactured by combining a front module constituting the front part of the platform, a middle module constituting the middle part of the platform, and a rear module constituting the rear part of the platform. The number of modules constituting the platform is not limited to three, and may be two or less or may be four or more. In addition, in addition to or instead of the components constituting the platform, the components constituting the portion of the vehicle 100 that is different from the platform may be modularized. In addition, each module may include any external components such as a bumper and a grille and any internal components such as a seat and a console. In addition, without being limited to the vehicle 100, any kind of movable body may be manufactured by combining a plurality of modules. For example, such a module may be manufactured by joining a plurality of components by welding, fixing, etc., or may be manufactured by molding at least some of the components constituting the module as one component in one piece by casting. The molding technique of molding a component (especially a relatively large component) in one piece is also referred to as giga casting or mega casting. For example, the front module, the middle module, and the rear module may be manufactured by integral casting.
[0068] (C7) The transport of the vehicle 100 using unmanned driving to make the vehicle 100 travel is also referred to as "autonomous driving transport". In addition, a configuration for realizing autonomous driving transport is also referred to as "vehicle remote control autonomous driving transport system". In addition, a production method for producing the vehicle 100 using autonomous driving transport is also referred to as "autonomous driving production". In autonomous driving production, for example, in a factory that manufactures the vehicle 100, at least a part of the transport of the vehicle 100 is realized by autonomous driving transport.
[0069] The present disclosure is not limited to the above-mentioned embodiments, and can be implemented in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in the modes described in the summary of the invention can be appropriately replaced or combined to solve some or all of the above-mentioned problems, or to achieve some or all of the above-mentioned effects. In addition, except for the technical features described as necessary in this specification, the technical features can be removed when appropriate.
Claims
1. A control device configured to generate a control command for controlling a movable body using measurement results of a plurality of distance measuring devices, characterized in that The control device comprises: a point cloud combining unit configured to create combined point cloud data by combining two or more pieces of three-dimensional point cloud data obtained by two or more distance measuring devices among the plurality of distance measuring devices; and An estimating unit is configured to perform a first estimation process of estimating at least one of a position and an orientation of the movable body using the combined point cloud data.
2. The control device according to claim 1, characterized in that: The estimation unit is configured to execute the first estimation process when an allowable processing time of the first estimation process is equal to or longer than a time reference value.
3. The control device according to claim 2, characterized in that: The estimation unit is configured to, when the allowable processing time is shorter than the time reference value, perform a second estimation process of estimating at least one of the position and the orientation of the movable body using a single piece of three-dimensional point cloud data.
4. The control device according to claim 1, characterized in that: The estimation unit is configured to execute the first estimation process when the required accuracy of the control of the movable body is equal to or higher than an accuracy reference value.
5. The control device according to claim 1, characterized in that: The estimating unit is configured to select and execute one of the first estimating process and a second estimating process of estimating at least one of the position and orientation of the movable body using a single piece of three-dimensional point cloud data, according to the running condition of the movable body.
