Remote control device, control device, movable object, control system, and control method
By dynamically adjusting the number of distance measurement devices, and prioritizing accuracy or speed according to vehicle driving conditions, the balance between accuracy and time in vehicle remote control is solved, and the stability and efficiency of remote control are improved.
Patent Information
- Application Number
- CN202410674276.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-12-05
- Filing Date
- 2024-05-28
- Publication Date
- 2024-12-03
AI Technical Summary
During the vehicle manufacturing process, when driving the vehicle through remote control, it is necessary to maintain a balance between accuracy and processing time in the process of estimating the position and orientation of the movable body. The prior art is difficult to effectively achieve this balance, resulting in accuracy and speed tradeoffs.
The measurement results of multiple distance measurement devices are used to generate control commands, and the number of distance measurement devices for processing objects is dynamically adjusted according to the required accuracy and driving conditions to prioritize the estimated accuracy or processing speed, and through the three-dimensional point cloud data and template point cloud The matching repeats the estimation, increasing the number of processing object distance measurement devices to improve accuracy and reliability.
The estimation accuracy and processing speed are achieved based on the situation, reducing the possibility of problems occurring during vehicle driving, and reducing the amount of vehicle movement during the estimation processing by reducing the movement speed, improving the stability and efficiency of remote control.
Smart Images

Figure CN119065358A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a remote control device, a control device, a movable body, a control system and a control method. Background Art
[0002] A technology for driving a vehicle by remote control during vehicle manufacturing is known (for example, Japanese Unexamined Patent Application Publication No. 2017-538619 (Translation of PCT Application)). Summary of the invention
[0003] When the movable body is moved by remote control, a process of estimating the position and orientation of the movable body is performed. The position and orientation of the movable body are estimated using three-dimensional point cloud data acquired using a distance measuring device such as a camera and a radar. In the estimation process, short-cycle calculation is required for the stability of the driving control of the movable body. However, since there is a trade-off relationship between the estimation accuracy of the position and orientation of the movable body and the processing time, it is desirable to maintain a balance between the estimation accuracy and the processing time depending on the situation.
[0004] The present disclosure can be implemented in the following modes.
[0005] A first aspect of the present disclosure provides a remote control device that uses measurement results of multiple distance measuring devices to generate a control command for remote control of a movable body, and sends the control command to the movable body. The remote control device includes: one or more memories, and one or more processors that communicate with the one or more memories, and the one or more processors are configured to perform processing. The processing includes: specifying one or more processing object distance measuring devices from the multiple distance measuring devices, each processing object distance measuring device is a distance measuring device used in an estimation process of estimating at least one of the position and orientation of the movable body; and performing estimation using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device. This remote control device specifies a processing target distance measuring device used for estimation and executes estimation using the processing target distance measuring device, and thus can prioritize one of estimation accuracy and processing speed according to the situation.
[0006] In the above remote control device, the process may further include increasing the number of processing target distance measuring devices when the required accuracy of the remote control is a first level compared to when the required accuracy is a second level lower than the first level. With this remote control device, when the required accuracy of the remote control is high, the processor increases the number of processing target distance measuring devices, and thus a decrease in estimation accuracy can be suppressed compared to when the required accuracy is low.
[0007] In the above remote control device, the processing may further include deciding the number of the processing target distance measuring devices according to the driving condition of the movable body. With this remote control device, the processor determines the number of processing target distance measuring devices according to the traveling condition of the movable body, and thus it is possible to prioritize one of the estimation accuracy and the processing speed according to the traveling condition of the movable body.
[0008] In the above remote control device, the processing may also include: repeatedly performing estimation by matching the above-mentioned three-dimensional point cloud data with the template point cloud, and when the consistency of the match in the estimation is below a threshold, increasing the number of processing object distance measurement devices compared to the number of processing object distance measurement devices in the last matching. With this remote control device, when the matching degree of consistency is low, the device specifying unit increases the number of processing target distance measurement devices, and thus it is possible to suppress a decrease in estimation accuracy when the matching degree of consistency is low.
[0009] In the above remote control device, the processing may also include: repeatedly performing estimation by matching the three-dimensional point cloud data with the template point cloud; and when the reliability of the match in a preset number of estimates is below a threshold, increasing the number of processing object distance measurement devices compared to the number of processing object distance measurement devices in the last match. With this remote control device, when the reliability of matching is low, the device specifying unit increases the number of processing target distance measuring devices, thereby suppressing the reduction of estimation accuracy when the reliability of matching is low. In addition, when the reliability of matching in the preset number of estimation processes is below a threshold, the device specifying unit increases the number of processing target distance measuring devices, thereby suppressing frequent changes in the number of processing target distance measuring devices and suppressing the processing of the remote control device from becoming complicated, compared with a configuration in which the number of processing target distance measuring devices is changed according to the matching result in each estimation process.
[0010] In the above remote control device, the process may further include: when the device specifying unit increases the number of processing target distance measuring devices, reducing a moving speed of the movable body. With this remote control device, when the number of processing target distance measuring devices increases, the moving speed of the movable body decreases, thereby suppressing the moving amount of the movable body during the estimation process. Therefore, the possibility of a problem occurring in the travel of the movable body can be reduced.
[0011] A second aspect of the present disclosure provides a control device that generates a control command for controlling a movable body using measurement results of a plurality of distance measuring devices. The control device includes one or more memories, and one or more processors that communicate with the one or more memories, and the one or more processors are configured to perform processing. The processing includes: specifying one or more processing object distance measuring devices from a plurality of distance measuring devices, each processing object distance measuring device being a distance measuring device used in at least one of estimating the position and orientation of the movable body; and performing estimation using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device. The control device specifies the processing target distance measuring device used in the estimation process and executes the estimation process using the processing target distance measuring device, so that one of the estimation accuracy and the processing speed can be prioritized according to the situation.
[0012] A third aspect of the present disclosure provides a movable body that can travel unmanned using measurement results of multiple distance measuring devices. The movable body includes one or more memories, and one or more processors communicating with the one or more memories, and the one or more processors are configured to perform processing. The processing includes: specifying one or more processing object distance measuring devices from multiple distance measuring devices, each processing object distance measuring device is a distance measuring device used in an estimation process of estimating at least one of the position and orientation of the movable body; and performing estimation processing using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device. The movable body specifies the processing target distance measuring device used in the estimation and performs the estimation process using the processing target distance measuring device, so that one of the estimation accuracy and the processing speed can be prioritized according to the situation. A fourth aspect of the present disclosure provides a control system that uses measurement results of multiple distance measuring devices to generate control commands for controlling a movable body. The control system includes one or more memories, and one or more processors communicating with the one or more memories, and the one or more processors are configured to perform processing. The processing specifies one or more processing object distance measuring devices from multiple distance measuring devices, each processing object distance measuring device is a distance measuring device used in estimating at least one of the position and orientation of the movable body; and uses three-dimensional point cloud data obtained by measurement by the processing object distance measuring device to perform estimation. This control system specifies a processing target distance measuring device used in the estimation process and executes the estimation process using the processing target distance measuring device, and thus can prioritize one of the estimation accuracy and the processing speed according to the situation. A fifth aspect of the present disclosure provides a control method for controlling a movable body using measurement results of multiple distance measuring devices. The control method includes: one or more processors specifying one or more processing object distance measuring devices from multiple distance measuring devices, each processing object distance measuring device is a distance measuring device used in at least one of estimating the position and orientation of the movable body; and one or more processors using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device to perform estimation. In this control method, a processing target distance measuring device used in the estimation process is designated, and the estimation process is executed using the processing target distance measuring device. Therefore, it is possible to give priority to either estimation accuracy or processing speed according to the situation. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which like symbols represent like elements, and in which: Figure 1 is an explanatory diagram showing a schematic configuration of a remote control system to which the remote control device in the first embodiment is applied; Figure 2 is a block diagram showing the configuration of a vehicle and a remote control device; Figure 3A is a flowchart showing the procedure of the travel control in the first embodiment; Figure 3B is a flowchart showing the procedure of the processing of the remote control device in the first embodiment; Figure 4 is a flowchart showing the procedure of distance measurement device designation processing in the first embodiment; Figure 5 is a flowchart showing the procedure of distance measurement device designation processing in the second embodiment; Figure 6 is a flowchart showing the procedure of distance measurement device designation processing in the third embodiment; Figure 7 is a block diagram showing the configuration of a control system in the third embodiment; and Figure 8 is a flowchart showing the procedure of the travel control in the third embodiment. DETAILED DESCRIPTION A. First Embodiment
[0014] Figure 11 is an explanatory diagram showing a schematic configuration of a remote control system 10 to which the remote control device 200 in the first embodiment is applied. The remote control system 10 includes one or more vehicles 100 as exemplary movable bodies, a remote control device 200 that generates a control command for remote control of each vehicle 100 and transmits the control command to the vehicle 100, a plurality of distance measuring devices 300 that measure three-dimensional point cloud data about the vehicle 100, and a process management device 400 that manages a manufacturing process of the vehicle 100. The remote control system 10 corresponds to the "control system" in the present disclosure. In addition, in the embodiment, the remote control device 200 corresponds to the "control device" in the present disclosure.
[0015] In an embodiment, the vehicle 100 is configured as a Battery Electric Vehicle (BEV).
[0016] In the present disclosure, a "movable body" refers to a movable object, and is, for example, a vehicle or an electric vertical take-off and landing aircraft (a so-called flying vehicle). The vehicle may be a vehicle that runs on wheels, or may be a vehicle that runs on tracks, and is, for example, a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, and the like. Vehicles include pure electric vehicles (BEVs), gasoline vehicles, hybrid electric vehicles, and fuel cell electric vehicles. In the case where the movable body is not a vehicle, the expression "vehicle" and the expression "car" in the present disclosure may be replaced with "movable body" when appropriate, and the expression "running" may be replaced with "moving" when appropriate.
[0017] The vehicle 100 is configured to be able to travel by unmanned driving. "Unmanned driving" means driving that does not rely on the driving operation of the occupants. The driving operation refers to an operation related to at least one of "running", "steering" and "stopping" of the vehicle 100. Unmanned driving is achieved by automatic or manual remote control using a device located outside the vehicle 100 or by autonomous control of the vehicle 100. Occupants who do not perform driving operations can ride in the vehicle 100 that travels by unmanned driving. Examples of occupants who do not perform driving operations include people who only sit on the seats of the vehicle 100, people who perform work different from driving operations in the vehicle 100 (examples are installation, inspection, and operation of switches). Driving that relies on the driving operation of the occupants is sometimes referred to as "manned driving".
[0018] In this specification, “remote control” includes “full remote control” in which all actions of the vehicle 100 are completely determined from outside the vehicle 100, and “partial remote control” in which some actions of the vehicle 100 are determined from outside the vehicle 100. In addition, “autonomous control” includes “full autonomous control” in which the vehicle 100 autonomously controls its actions without receiving any information from a device outside the vehicle 100, and “partial autonomous control” in which the vehicle 100 autonomously controls its actions using information received from a device outside the vehicle 100.
[0019] In the present embodiment, remote control of the vehicle 100 is performed in a factory that manufactures the vehicle 100. The reference coordinate system of the factory is a global coordinate system, and any position in the factory can be expressed as an XYZ coordinate in the global coordinate system. The factory includes a first location PL1 and a second location PL2. For example, the first location PL1 is a location where assembly of the vehicle 100 is performed, and for example, the second location PL2 is a location where inspection of the vehicle 100 is performed. The first location PL1 and the second location PL2 are connected by a travel road SR along which the vehicle 100 can travel.
[0020] A plurality of distance measuring devices 300 using the vehicle 100 as a measurement object are installed around the travel road SR. The distance measuring device 300 is an external sensor positioned outside the vehicle 100. As the external sensor, not limited to the distance measuring device 300, various sensors may be installed around the travel road SR. The external sensor includes a communication device (not shown), and may communicate with other devices such as the vehicle 100 and the remote control device 200 through wired communication or wireless communication. The remote control device 200 may use the three-dimensional point cloud data measured by each distance measuring device 300 to acquire the position and orientation of the vehicle 100 relative to the target route TR in real time. The distance measuring device 300 in this embodiment may switch between execution and stop of acquisition of the three-dimensional point cloud data by receiving a control command from the remote control device 200. As the distance measuring device 300, a camera and light detection and ranging (LiDAR) may be used. In particular, LiDAR is preferred because high-precision three-dimensional point cloud data may be obtained. Preferably, the plurality of distance measuring devices 300 are arranged so that when the vehicle 100 is present at any position on the target route TR, the vehicle 100 can always be measured by two or more distance measuring devices 300. The positions of the respective distance measuring devices 300 are fixed, and the relative relationship between the reference coordinate system Σr and the device coordinate system of each distance measuring device 300 is known in advance. A coordinate conversion matrix for mutual conversion between coordinate values in the reference coordinate system Σr and coordinate values in the device coordinate system of each distance measuring device 300 is stored in advance in the remote control device 200.
[0021] The remote control device 200 generates a control command for causing the vehicle 100 to travel along the target route TR, and transmits the control command to the vehicle 100. The vehicle 100 travels in accordance with the received control command. Therefore, in the remote control system 10, the vehicle 100 can be moved from the first location PL1 to the second location PL2 by remote control without using a conveying device such as a crane or a conveyor. In addition, the remote control device 200 in this embodiment generates a control command for giving an instruction to execute or stop acquiring three-dimensional point cloud data by the distance measuring device 300, and transmits the control command to each distance measuring device 300.
[0022] Figure 2 1 is a block diagram showing the configuration of the vehicle 100 and the remote control device 200. The vehicle 100 includes: a vehicle control device 110 for controlling components of the vehicle 100, an actuator group 120 driven under the control of the vehicle control device 110, a communication device 130 for communicating with the remote control device 200 through wireless communication, and a GPS receiver 140 for acquiring position information about the vehicle 100. In an embodiment, the actuator group 120 includes an actuator of a driving device for accelerating the vehicle 100, an actuator of a steering device for changing the moving direction of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100. The driving device includes: a battery, a travel motor driven by the power of the battery, and a driving wheel rotated by the travel motor. The actuator of the driving device includes a travel motor. The actuator group 120 may also include an actuator for swinging a windshield wiper of the vehicle 100, an actuator for opening and closing an electric window of the vehicle 100, and the like. Furthermore, the vehicle 100 may include various object detectors, such as a camera, a millimeter wave radar, and a LiDAR.
[0023] The vehicle control device 110 is composed of a computer including a processor 111, a memory 112, an input-output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input-output interface 113 are connected in a bidirectional communication manner via the internal bus 114. The input-output interface 113 is connected to the actuator group 120, the communication device 130, and the GPS receiver 140.
[0024] In the present embodiment, by executing the program PG1 pre-stored in the memory 112, the processor 111 functions as the vehicle control unit 115 and the position information acquisition unit 116. The vehicle control unit 115 controls the actuator group 120. When the driver rides the vehicle 100, the vehicle control unit 115 can make the vehicle 100 travel by controlling the actuator group 120 according to the driver's operation. In addition, regardless of whether the driver rides the vehicle 100, the vehicle control unit 115 can make the vehicle 100 travel by controlling the actuator group 120 according to the control command sent from the remote control device 200. The position information acquisition unit 116 acquires position information indicating the current location of the vehicle 100 using the GPS receiver 140. The position information acquisition unit 116 and the GPS receiver 140 may be excluded.
[0025] The remote control device 200 is constituted by a computer including a processor 201, a memory 202, an input-output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input-output interface 203 are connected in a bidirectionally communicable manner through the internal bus 204. The input-output interface 203 is connected to a communication device 205 for communicating with the vehicle 100 through wireless communication. The communication device 205 can communicate with the vehicle 100 through wireless communication, and can communicate with each distance measuring device 300 through wired communication or wireless communication.
[0026] In the embodiment, by executing the program PG2 pre-stored in the memory 202, the processor 201 functions as a three-dimensional point cloud data acquisition unit 210, a device designation unit 220, a point cloud combination unit 230, a position estimation unit 240 and a remote control command generation unit 250.
[0027] The three-dimensional point cloud data acquisition unit 210 acquires three-dimensional point cloud data measured by the distance measurement device 300. The three-dimensional point cloud data is data indicating the three-dimensional position of the point cloud detected by the distance measurement device 300.
[0028] The device specifying unit 220 specifies a distance measuring device 300 used in an estimation process of estimating the position and orientation of the vehicle 100 from among a plurality of distance measuring devices 300. Hereinafter, the distance measuring device 300 specified by the device specifying unit 220 is referred to as a "processing target distance measuring device".
[0029] In the case where the device specifying unit 220 specifies two or more distance measuring devices 300 as the processing target distance measuring devices, the point cloud combining unit 230 creates combined point cloud data by combining two or more pieces of three-dimensional point cloud data obtained by the two or more distance measuring devices 300. As a combining method for the plurality of pieces of three-dimensional point cloud data, for example, one of the following methods may be employed. Point cloud combination method M1 The coordinate values of each point in the three-dimensional point cloud data obtained by each distance measuring device 300 are converted from the device coordinate system of the distance measuring device 300 to a specific coordinate system such as the reference coordinate system Σr, and the coordinate values after the conversion are summed. In this case, when there are a plurality of points whose difference in coordinate values after conversion is equal to or less than the allowable error, the plurality of points are replaced with one representative point. The coordinate value of the representative point is a representative value such as an average value of the coordinate values of the plurality of points. Point cloud combination method M2
[0030] (a) Creating first combined point cloud data by matching the first three-dimensional point cloud data and the second three-dimensional point cloud data, performing position adjustment using a representative value such as an average value of the position coordinates of corresponding points, and adding points without corresponding points without change. (b) Creating second combined point cloud data by performing the same processing as processing (a) on the first combined point cloud data and the third three-dimensional point cloud data. By repeating this process, any number of pieces of three-dimensional point cloud data can be combined. For position adjustment by matching, for example, an iterative closest point algorithm (ICP algorithm) can be used. In terms of processing speed, the above-mentioned combining method M1 is preferred.
[0031] The position estimation unit 240 performs an estimation process of estimating the position and orientation of the vehicle 100 using the three-dimensional point cloud data obtained by the distance measurement device 300 through measurement. In an embodiment, the position estimation unit 240 estimates the position and orientation of the vehicle 100 by performing template matching using the template point cloud TP stored in the memory 202. In the case where the three-dimensional point cloud data cannot be used, the position estimation unit 240 may estimate the position and orientation of the vehicle 100 using the driving history of the vehicle 100 and the position information detected by the GPS receiver 140 equipped in the vehicle 100. The position estimation unit 240 may estimate only one of the position and orientation of the vehicle 100. In this case, the other of the position and orientation of the vehicle 100 is decided using the driving history of the vehicle 100 and the like.
[0032] The remote control command generation unit 250 generates a control command for remotely controlling the actuator group 120 of the vehicle 100 using the estimated position and orientation of the vehicle 100, and sends the control command to the vehicle 100. Thus, the vehicle 100 can be driven by remote control. The control command is a command to drive the vehicle 100 along the target route TR stored in the memory 202. In an embodiment, the control command includes the acceleration and steering angle of the vehicle 100 as parameters. In another embodiment, instead of or in addition to the acceleration of the vehicle 100, the control command may include the speed of the vehicle 100 as a parameter. The control command can be generated as a command including a driving force or a braking force and a steering angle. Alternatively, the remote control command generation unit 250 may generate a control command as a command including at least one of the position and orientation of the vehicle 100 and the route along which the vehicle 100 will travel. Hereinafter, the control command is also referred to as a "driving control signal". For example, in addition to the driving control signal, the remote control command generation unit 250 can also generate and output a control signal for controlling the actuators of various auxiliary machines and various devices such as windshield wipers, power windows, and lights included in the vehicle 100. That is, the remote control command generation unit 250 can operate various devices and various auxiliary machines through remote control.
[0033] The process management device 400 manages the entire manufacturing process of the vehicle 100 in the factory. For example, when one vehicle 100 starts traveling along the target route TR, individual information indicating the identification number, model, etc. of the vehicle 100 is transmitted from the process management device 400 to the remote control device 200. The position of the vehicle 100 detected by the remote control device 200 is also transmitted to the process management device 400. The functions of the process management device 400 may be implemented in the same device as the remote control device 200.
[0034] Figure 3A is a flowchart showing a processing procedure for driving control of the vehicle 100 in the first embodiment. In step S1, the remote control command generation unit 250 uses the detection result output from the external sensor to obtain the vehicle position information. The vehicle position information is the position information based on which the driving control signal is generated. In an embodiment, the vehicle position information includes the position and orientation of the vehicle 100 in the global coordinate system of the factory. Specifically, in step S1, the remote control command generation unit 250 uses the captured image obtained from the camera as the external sensor to obtain the vehicle position information. In addition, in step S1, the remote control command generation unit 250 can obtain the vehicle position information by using the three-dimensional point cloud data obtained by the distance measuring device 300 and showing the vehicle 100 through template matching using the three-dimensional point cloud data and the previously prepared template point cloud TP.
[0035] In detail, in step S1, the remote control command generation unit 250 acquires the position of the vehicle 100, for example, by detecting the shape of the vehicle 100 from the captured image, calculating the coordinates of the position measurement point of the vehicle 100 in the coordinate system for the captured image (i.e., the local coordinate system), and converting the calculated coordinates into coordinates in the global coordinate system. For example, the shape of the vehicle 100 included in the captured image can be detected by inputting the captured image to a detection model using artificial intelligence. For example, the detection model is prepared in the interior of the remote control system 10 or in the exterior of the remote control system 10, and is pre-stored in the memory 202 of the remote control device 200. Examples of the detection model include a machine learning model for which learning has been performed so as to achieve one of semantic segmentation and instance segmentation. As the machine learning model, for example, a convolutional neural network (hereinafter referred to as CNN) can be used for which learning has been performed by supervised learning using a learning data set. For example, the learning data set includes a plurality of training images containing the vehicle 100, and labels each indicating whether an area in the training image is an area of the vehicle 100 or an area outside the vehicle 100. When learning CNN, it is preferred to update the parameters in CNN by back propagation method to reduce the error between the output result of the detection model and the label. For example, the remote control command generation unit 250 can obtain the orientation of the vehicle 100 by estimating the orientation of the moving vector of the vehicle 100 calculated based on the position change of the feature point of the vehicle 100 in the frame of the captured image using the optical flow method.
[0036] In step S2, the remote control command generation unit 250 determines the target position that the vehicle 100 will go to from now on. In the present embodiment, the target position is represented as XYZ coordinates in a global coordinate system. The target route TR, which is the path along which the vehicle 100 will travel, is pre-stored in the memory 202 of the remote control device 200. The path is shown by a node indicating a departure point, a node indicating a passing point, a node indicating a destination, and a link connecting the nodes. The remote control command generation unit 250 uses the vehicle position information and the target route TR to determine the target position that the vehicle 100 will go to from now on. The remote control command generation unit 250 determines the target position on the target route TR in front of the current location of the vehicle 100.
[0037] In step S3, the remote control command generation unit 250 generates a driving control signal for driving the vehicle 100 to the determined target position. The remote control command generation unit 250 calculates the driving speed of the vehicle 100 according to the change of the position of the vehicle 100, and compares the calculated driving speed with the target speed. In general, the remote control command generation unit 250 determines the acceleration to accelerate the vehicle 100 when the driving speed is lower than the target speed, and determines the acceleration to decelerate the vehicle 100 when the driving speed is higher than the target speed. In addition, when the vehicle 100 is located on the target route TR, the remote control command generation unit 250 determines the steering angle and the acceleration so that the vehicle 100 does not deviate from the target route TR. When the vehicle 100 is not located on the target route TR, in other words, when the vehicle 100 deviates from the target route TR, the remote control command generation unit 250 determines the steering angle and the acceleration so that the vehicle 100 returns to the target route TR.
[0038] In step S4, the remote control command generation unit 250 transmits the generated travel control signal to the vehicle 100. The remote control command generation unit 250 repeats the acquisition of vehicle position information, determination of target position, generation of travel control signals, transmission of travel control signals, etc. at a predetermined cycle.
[0039] In step S5, the vehicle control unit 115 receives the travel control signal transmitted from the remote control device 200. In step S6, the vehicle control unit 115 controls the actuator group 120 using the received travel control signal, thereby causing the vehicle 100 to travel at the acceleration and steering angle shown in the travel control signal. The vehicle control unit 115 repeats the reception of the travel control signal and the control of the actuator group 120 at a predetermined cycle. In the remote control system 10 of the present embodiment, the vehicle 100 can be caused to travel by remote control, and the vehicle 100 can be moved without using a conveying device such as a crane or a conveyor.
[0040] Figure 3B 2 is a flowchart showing a processing procedure of the remote control apparatus 200 in the embodiment. This processing is started when the remote control apparatus 200 is activated, and is repeatedly executed during the operation of the remote control apparatus 200.
[0041] In step S100 , the device specifying unit 220 performs distance measurement device specifying processing. Figure 4 FIG. 4 is a flowchart showing the procedure of the distance measurement device designation process in the first embodiment. Figure 4 In step S110 shown, the device specifying unit 220 determines whether the required accuracy for remote control by the remote control device 200 is high. For example, the determination may be a determination as to whether the required accuracy is higher than a reference level.
[0042] The required accuracy of the remote control is set for each of the plurality of sections set along the target route TR. For example, for a straight section or a wide section on the travel road SR, the allowable amount of position deviation of the vehicle 100 is large, and therefore, the required accuracy is set to be low. On the other hand, for a curved section or a narrow section on the travel road SR, the allowable amount of position deviation of the vehicle 100 is small, and therefore, the required accuracy is set to be high. Information related to the form and width of the travel road SR may be stored in the memory 202 in advance.
[0043] In addition, the required accuracy of the remote control can be set according to the driving state of the vehicle 100. For example, when the steering angle of the vehicle 100 is above a preset threshold, since the amount of change in the orientation of the vehicle 100 within a predetermined time interval is large, the required accuracy is set to be higher than when the steering angle is less than the threshold. In addition, when obstacles such as other vehicles and people around the vehicle 100 are detected by an unillustrated object detection device, the required accuracy is set to high in order to avoid contact with the obstacles. The position of the vehicle 100 when the required accuracy is determined may be the position estimated in the last estimation process or may be the position estimated based on the driving history of the vehicle 100.
[0044] like Figure 4 As shown, when it is determined that the required accuracy is low (step S110: Yes), the device specifying unit 220 sets a reference number of distance measuring devices 300 as the processing target distance measuring devices. The "reference number" refers to the minimum number of distance measuring devices 300 required to specify the position of the vehicle 100, and in the embodiment, the reference number is preset to 1. The reference number is not limited to one, and may be set to two or more, or may be set to a different number for each section of the travel road SR. After this step is completed, the device specifying unit 220 ends the distance measuring device designation process.
[0045] On the other hand, when it is determined that the required accuracy is high (step S110: No), the device specifying unit 220 determines in step S112 whether the allowable processing time under the current driving condition of the vehicle 100 is equal to or greater than the required processing time for the estimation process, even if the number of distance measuring devices 300 is greater than the reference number as described later. The "allowable processing time" refers to a time set as a time that does not cause a malfunction in the remote control even if the processing time for the estimation process is long. In the present embodiment, the allowable processing time is set according to the driving condition of the vehicle 100. More specifically, when the processing time for the estimation process can be sufficiently ensured under the driving condition of the vehicle 100 (for example, before starting the remote control, during the temporary stop of the vehicle 100, or during the low-speed driving of the vehicle 100), the allowable processing time is set to be long. This is because, in this case, the vehicle 100 stops or moves slowly, and therefore the possibility that the movement amount of the vehicle 100 becomes large during the execution of the estimation process, in other words, during the period when the position of the vehicle 100 cannot be acquired, is low. In addition, the allowable processing time may also be set in advance for each of the plurality of sections set along the target route TR. In the embodiment, the "required processing time for the estimation process" is specified by conducting experiments in advance, etc., and is set to the processing time for the estimation process using the three-dimensional point cloud data acquired from a preset number of distance measuring devices 300 greater than the reference number. Generally, the larger the number of distance measuring devices 300 that acquire the three-dimensional point cloud data, the longer the processing time for the estimation process.
[0046] When the allowable processing time is more than the required processing time (step S112: Yes), the device specifying unit 220 executes step S122 described later. On the other hand, when the allowable processing time is less than the required processing time (step S112: No), in step S114, the remote control command generating unit 250 generates a control command for giving an instruction to decelerate the vehicle 100, and sends the control command to the vehicle 100. In an embodiment, the speed in the instruction to the vehicle 100 is pre-set to a speed lower than the speed at the time of normal transmission. Thus, since the speed of the vehicle 100 is reduced, it is possible to ensure sufficient processing time for the estimation process. After step S114, the device specifying unit 220 executes step S122 described later.
[0047] In step S122, the device specifying unit 220 sets a greater number of distance measuring devices 300 than the reference number as the processing target distance measuring devices. Therefore, three-dimensional point cloud data is acquired from a greater number of distance measuring devices 300 than the reference number, synthesized, and used, and thus, a decrease in the accuracy of the estimation process can be suppressed compared to when the three-dimensional point cloud data is acquired only from the reference number of distance measuring devices 300. Then, the distance measuring device specifying process ends.
[0048] exist Figure 3B In step S200 shown, the position estimation unit 240 acquires three-dimensional point cloud data from the processing target distance measurement device specified in the above-mentioned distance measurement device designation process. In an embodiment, the position estimation unit 240 sends a control command for giving an instruction to execute acquisition of three-dimensional point cloud data to the processing target distance measurement device through the remote control command generation unit 250, and sends a control command for giving an instruction to stop acquiring three-dimensional point cloud data to other distance measurement devices 300.
[0049] In step S300 , the position estimation unit 240 estimates the position and orientation of the vehicle 100 by performing matching between the acquired three-dimensional point cloud data and the template point cloud TP.
[0050] In step S400, the remote control command generation unit 250 generates a control command using the estimated position and orientation of the vehicle 100, and transmits the control command to the vehicle 100. After this step ends, the remote control device 200 performs step S100 again.
[0051] The remote control device 200 in the above-described first embodiment specifies a processing target distance measuring device used in the estimation process and executes the estimation process using the processing target distance measuring device, and thus can prioritize one of the estimation accuracy and the processing speed according to the situation.
[0052] Furthermore, when the required accuracy of the remote control is high, the device specification unit 220 increases the number of processing target distance measurement devices, and therefore, a decrease in estimation accuracy can be suppressed compared to when the required accuracy is high.
[0053] Furthermore, since the device specifying unit 220 determines the number of the processing target distance measuring devices according to the running condition of the vehicle 100 , it is possible to prioritize one of the estimation accuracy and the processing speed according to the running condition of the vehicle 100 .
[0054] Furthermore, when the number of processing target distance measuring devices is increased, the remote control device 200 reduces the moving speed of the vehicle 100 , thereby suppressing the moving amount of the vehicle 100 during execution of the estimation process, and thus the possibility of a problem occurring in the travel of the vehicle 100 can be reduced. B. Second Embodiment
[0055] Figure 5 FIG. 4 is a flowchart showing the procedure of the distance measurement designation process in the second embodiment. Figure 5As shown, the remote control device 200 in the second embodiment is different from the remote control device 200 in the first embodiment in that step S110A is performed instead of step S110. The configuration of the system and the device is the same as those in the first embodiment. The configuration of the remote control device 200 in the second embodiment and other procedures in the distance measuring device designation process are the same as those in the first embodiment. Therefore, the same construction and the same procedures are denoted by the same reference symbols, and detailed description thereof is omitted.
[0056] In step S110A, the device specifying unit 220 determines whether the matching consistency in the last estimation process is greater than a preset threshold value. "Matching consistency" refers to the consistency between the three-dimensional point cloud data acquired in the matching and the template point cloud TP. As the consistency, for example, the number of corresponding points found by matching all points of the template point cloud TP, or the ratio of the number of corresponding points can be used.
[0057] When the matching degree is greater than the threshold value (step S110A: Yes), the device specifying unit 220 executes the above step S120. On the other hand, when the matching degree is less than the threshold value (step S110A: No), the device specifying unit 220 executes the above step S112. In this way, when the matching degree is less than the threshold value, the device specifying unit 220 of this embodiment increases the number of processing target distance measurement devices compared to when the matching degree is greater than the threshold value.
[0058] In the remote control device 200 of the second embodiment described above, the device specification unit 220 increases the number of processing target distance measurement devices when the matching consistency is low, and thus it is possible to suppress a decrease in estimation accuracy when the matching consistency is low. C. Third Embodiment
[0059] Figure 6 FIG. 4 is a flowchart showing the procedure of the distance measurement device designation process in the third embodiment. Figure 6 As shown, the remote control device 200 in the third embodiment is different from the remote control device 200 in the second embodiment in that step S110B is executed instead of step S110A. The configuration of the remote control device 200 in the third embodiment and other procedures in the distance measuring device designation process are the same as those in the second embodiment. Therefore, the same construction and the same procedures are denoted by the same reference symbols, and detailed description thereof is omitted.
[0060] In step S110B, the device specifying unit 220 determines whether the reliability of the match in the last estimation process is above a preset threshold value. "Reliability of the match" refers to the reliability or stability of the consistency of the match in the predetermined number of matches. For example, the average value of the consistency of the last match in the predetermined number of matches can be used. The reliability of the match is not limited to the average value, and can be calculated using various statistics such as the median, standard deviation, and dispersion.
[0061] When the reliability of the match is above the threshold (step S110B: Yes), the device designation unit 220 executes the above step S120. On the other hand, when the reliability of the match is less than the threshold (step S110B: No), the device designation unit 220 executes the above step S112. In this way, when the reliability of the match is less than the threshold, the device designation unit 220 in the present embodiment increases the number of distance measuring devices to be processed compared to when the reliability of the match is above the threshold. For example, even if the consistency at the last match is less than the threshold, when a decrease in consistency unexpectedly occurs in a predetermined number of matches, the reliability is also above the threshold, and the number of distance measuring devices to be processed does not increase. The reason for the unexpected decrease in the consistency of the match is considered to be that an obstacle such as a person or another vehicle passes between the vehicle 100 and the distance measuring device 300.
[0062] In the remote control device 200 in the third embodiment described above, when the reliability of matching is low, the device specifying unit 220 increases the number of processing target distance measuring devices, so that it is possible to suppress the estimation accuracy from being reduced when the reliability of matching is low. In addition, when the matching reliability in the estimation process of a preset number of times is below the threshold, the device specifying unit 220 increases the number of processing target distance measuring devices, so that compared with the configuration in which the number of processing target distance measuring devices is changed according to the matching result in each estimation process, it is possible to suppress the number of processing target distance measuring devices from being frequently changed and suppress the processing of the remote control device 200 from becoming complicated. D. Fourth Embodiment
[0063] Figure 7 1 is an explanatory diagram showing a schematic configuration of a remote control system 10v in a fourth embodiment. This embodiment differs from the first embodiment in that the remote control system 10v does not include a remote control device 200. In addition, the vehicle 100v of this embodiment is capable of traveling by autonomous control of the vehicle 100v. The other configurations are the same as those of the first embodiment unless otherwise specified.
[0064] In the embodiment, by executing the program PG1 stored in the memory 112v, the processor 111v of the vehicle control device 110v is used as a vehicle control unit 115v, a position information acquisition unit 116, a three-dimensional point cloud data acquisition unit 191, a device designation unit 192, a point cloud combination unit 193, and a position estimation unit 194. The vehicle control unit 115v obtains the output result of the sensor, generates a driving control signal using the output result, and outputs the generated driving control signal to operate the actuator group 120. Thus, the vehicle control unit 115v is able to drive the vehicle 100v through autonomous control. In the present embodiment, in addition to the program PG1, the template point cloud TP and the target route TR are pre-stored in the memory 112v. The vehicle control device 110v in the embodiment corresponds to the "control device" in the present disclosure.
[0065] Figure 8 1 is a flowchart showing the processing procedure of the driving control of the vehicle 100v in the second embodiment. In step S11, the vehicle control unit 115v uses the detection result output from the external sensor to obtain the vehicle position information. In step S12, the vehicle control unit 115v determines the target position that the vehicle 100v will go to from now on. In step S13, the vehicle control unit 115v generates a driving control signal for causing the vehicle 100v to travel to the determined target position. In step S14, the vehicle control unit 115v uses the generated driving control signal to control the actuator group 120, thereby causing the vehicle 100v to travel according to the parameters indicated by the driving control signal. The vehicle control unit 115v repeats the acquisition of the vehicle position information, the determination of the target position, the generation of the driving control signal, and the control of the actuator at a predetermined cycle. In the remote control system 10v of this embodiment, even if the remote control device 200 does not perform remote control of the vehicle 100v, the vehicle 100v can be driven by the autonomous control of the vehicle 100v. E. Other Embodiments
[0066] (E1) In the above embodiment, the speed in the instruction to the vehicle 100 is set in advance in step S114 of the distance measurement device designation process, but the present disclosure is not limited thereto. In addition, the speed in the instruction to the vehicle 100 can be set according to the size of the difference between the allowable processing time and the required processing time. By this setting method, the deceleration amount of the vehicle 100 can be set to the minimum necessary amount, and therefore, the increase in the transmission time due to the deceleration of the vehicle 100 can be suppressed.
[0067] (E2) In the above embodiment, the position estimation unit 240 acquires three-dimensional point cloud data from the processing object distance measuring device specified in the distance measuring device designation process, but the present disclosure is not limited thereto. The position estimation unit 240 may acquire three-dimensional point cloud data from all distance measuring devices 300, and may estimate the position and orientation of the vehicle 100 using only the three-dimensional point cloud data acquired from all distance measuring devices 300 and acquired from the processing object distance measuring device specified in the distance measuring device designation process. In this mode, it is not necessary to send a control command for giving an instruction to execute or stop acquiring three-dimensional point cloud data to each distance measuring device 300 according to a change in the processing object distance measuring device, and thus it is possible to suppress the processing of the remote control device 200 from becoming complicated.
[0068] (E3) In the first embodiment above, the process from acquiring the vehicle position information to generating the travel control signal is performed by the remote control device 200. However, at least a part of the process from acquiring the vehicle position information to generating the travel control signal may be performed by the vehicle 100. For example, the following modes (1) to (3) may be adopted.
[0069] (1) The remote control device 200 can acquire vehicle position information, can determine the target position that the vehicle 100 will go to from now on, and can generate a path from the current position of the vehicle 100 shown in the acquired vehicle position information to the target position. The remote control device 200 can generate a path to the target position between the destination and the current position, or can generate a path to the destination. The remote control device 200 can send the generated path to the vehicle 100. The vehicle 100 can generate a travel control signal so that the vehicle 100 travels on the path received from the remote control device 200, and can control the actuator group 120 using the generated travel control signal.
[0070] (2) The remote control device 200 may acquire vehicle position information, and may transmit the acquired vehicle position information to the vehicle 100. The vehicle 100 may determine a target position to which the vehicle 100 will go from now on, may generate a path from the current position of the vehicle 100 shown in the received vehicle position information to the target position, may generate a travel control signal so that the vehicle 100 travels on the generated path, and may control the actuator group 120 using the generated travel control signal.
[0071] (3) In the above modes (1) and (2), an internal sensor may be equipped in the vehicle 100, and the detection result output from the internal sensor may be used for at least one of the generation of the path and the generation of the travel control signal. The internal sensor is a sensor equipped in the vehicle 100. For example, the internal sensor may include a sensor for detecting the motion state of the vehicle 100, a sensor for detecting the operating state of each component of the vehicle 100, and a sensor for detecting the environment around the vehicle 100. Specifically, for example, the internal sensor may include a camera, a LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyroscope sensor, etc. For example, in the above mode (1), the remote control device 200 may obtain the detection result of the internal sensor, and may reflect the detection result of the internal sensor in the path when generating the path. In the above mode (1), the vehicle 100 may obtain the detection result of the internal sensor, and may reflect the detection result of the internal sensor in the travel control signal when generating the travel control signal. In the above mode (2), the vehicle 100 can obtain the detection results of the internal sensors, and can reflect the detection results of the internal sensors in the path when generating the path. In the above mode (2), the vehicle 100 can obtain the detection results of the internal sensors, and can reflect the detection results of the internal sensors in the travel control signal when generating the travel control signal.
[0072] (E4) In the above fourth embodiment, an internal sensor may be provided in the vehicle 100v, and the detection result outputted from the internal sensor may be used for at least one of the generation of the path and the generation of the driving control signal. For example, the vehicle 100v may obtain the detection result of the internal sensor, and the detection result of the internal sensor may be reflected in the path when the path is generated. The vehicle 100v may obtain the detection result of the internal sensor, and the detection result of the internal sensor may be reflected in the driving control signal when the driving control signal is generated. The vehicle 100v may obtain the target arrival time and congestion information from the outside of the vehicle 100v, and the target arrival time and congestion information may be reflected in at least one of the path and the driving control signal.
[0073] (E5) In the above first embodiment, the remote control device 200 automatically generates the travel control signal sent to the vehicle 100. However, the remote control device 200 may also generate the travel control signal sent to the vehicle 100 according to the operation of an external operator outside the vehicle 100. For example, the external operator may operate a manipulation device including a display device that displays a captured image output from an external sensor, a steering wheel for remotely operating the vehicle 100, an accelerator pedal, and a brake pedal, and a communication device for communicating with the remote control device 200 through wired communication or wireless communication, and the remote control device 200 may generate the travel control signal according to the operation of the manipulation device.
[0074] (E6) In the above embodiment, the vehicle 100 only needs to have a configuration in which the vehicle 100 can be moved by unmanned driving, and, for example, can have a platform containing the configuration described below. Specifically, the vehicle 100 only needs to include at least a vehicle control device 110 and an actuator group 120 for realizing the three functions of "running", "steering" and "stopping" by unmanned driving. In the case where the vehicle 100 obtains information from the outside for unmanned driving, the vehicle 100 only needs to further include a communication device 130. That is, the vehicle 100 that can be moved by unmanned driving does not need to be provided with a driver's seat and at least some internal components such as an instrument panel, does not need to be provided with at least some external components such as a bumper and a fender, and does not need to be provided with a body shell. In this case, other components such as a body shell can be attached to the vehicle 100 before the vehicle 100 is shipped from the factory, or other components such as the body shell can be attached to the vehicle 100 after the vehicle 100 is shipped from the factory in a state where other components such as the body shell have not been attached to the vehicle 100. The components can be attached from any direction such as the upper side, the lower side, the front side, the rear side, the right side, and the left side of the vehicle 100, and can be attached from the same direction as each other, or can be attached from different directions from each other. In addition, in the case of a platform, the position can also be determined in the same manner as the vehicle 100 of the first embodiment.
[0075] (E7) The vehicle 100 can be manufactured by combining a plurality of modules. A module means a unit composed of a plurality of components collected according to the location or function of the vehicle 100. For example, the platform of the vehicle 100 can be manufactured by combining a front module configuring the front of the platform, a central module configuring the middle of the platform, and a rear module configuring the rear of the platform. The number of modules constituting the platform is not limited to three, and may be two or less, or may be four or more. In addition, in addition to or instead of the components constituting the platform, components constituting the vehicle 100 that are different from a part of the platform may be modularized. In addition, each module may include any external components such as a bumper and a grille and any internal components such as a seat and a console. In addition, without being limited to the vehicle 100, any kind of movable body may be manufactured by combining a plurality of modules. For example, such a module may be manufactured by joining a plurality of components by welding, fasteners, etc., or may be manufactured by molding at least some of the components constituting the module as one component in one piece by casting. The molding technique of molding one component (especially a relatively large component) in one piece is also referred to as giga casting or mega casting. For example, the above front module, the above center module, and the above rear module may be manufactured by integral casting.
[0076] (E8) Transporting the vehicle 100 using the travel of the unmanned vehicle 100 is also referred to as "self-driving transport". In addition, a configuration for implementing self-driving transport is also referred to as "vehicle remote control autonomous driving transport system". In addition, a production method for producing the vehicle 100 using self-driving transport is also referred to as "self-driving production". In self-driving manufacturing, at least a portion of the transport of the vehicle 100 is achieved by self-driving transport, for example, in a factory that manufactures the vehicle 100.
[0077] (E9) Some or all of the functions and processes implemented by software in the above embodiments may be implemented by hardware. In addition, some or all of the functions and processes implemented by hardware may be implemented by software. As hardware for implementing the various functions in the above embodiments, for example, various circuits such as integrated circuits and discrete circuits may be used.
[0078] The present disclosure is not limited to the above-mentioned embodiments, and can be implemented in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in the modes described in the summary of the invention can be replaced or combined as appropriate to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. In addition, except for the technical features described as necessary in this specification, the technical features can be removed as appropriate.
Claims
1. A remote control device that generates a control command for remote control of a movable body using measurement results of a plurality of distance measuring devices and sends the control command to the movable body, characterized in that: The remote control device includes: one or more memories, and one or more processors in communication with the one or more memories, wherein the one or more processors are configured to perform the following processing: specifying one or more processing target distance measuring devices from the plurality of distance measuring devices, each of the processing target distance measuring devices being a distance measuring device used in estimating at least one of the position and orientation of the movable body; and The estimation is performed using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device.
2. The remote control device according to claim 1, characterized in that: The process further includes increasing the number of the processing target distance measuring devices when the required accuracy of the remote control device is a first level compared to when the required accuracy is a second level lower than the first level.
3. The remote control device according to claim 1, characterized in that: The processing further includes determining the number of the processing target distance measuring devices according to the travel status of the movable body.
4. The remote control device according to claim 1, characterized in that: The processing also includes: performing the estimation by matching the three-dimensional point cloud data with a template point cloud; and When the matching degree in the estimation process is equal to or less than a threshold value, the number of the processing target distance measurement devices is increased compared with the number of the processing target distance measurement devices in the previous matching.
5. The remote control device according to claim 1, characterized in that: The processing also includes: Repeating the estimating by matching the three-dimensional point cloud data with a template point cloud; and When the reliability of matching in the estimation process of a preset number of times is equal to or less than a threshold value, the number of the processing target distance measurement devices is increased compared with the number of the processing target distance measurement devices in the previous matching.
6. The remote control device according to any one of claims 2 to 5, characterized in that: The processing further includes reducing a moving speed of the movable body when the number of the processing target distance measuring devices increases.
7. A control device that generates a control command for controlling a movable body using measurement results of a plurality of distance measuring devices, characterized in that: The control device includes: one or more memories, and one or more processors communicating with the one or more memories, wherein the one or more processors are configured to perform the following processing: specifying one or more processing target distance measuring devices from the plurality of distance measuring devices, each of the processing target distance measuring devices being a distance measuring device used in estimating at least one of the position and orientation of the movable body; and The estimation is performed using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device.
8. A movable body capable of traveling unmanned using measurement results of a plurality of distance measuring devices, characterized in that: The movable body includes: one or more memories, and one or more processors communicating with the one or more memories, wherein the one or more processors are configured to perform the following processing: specifying one or more processing target distance measuring devices from the plurality of distance measuring devices, each of the processing target distance measuring devices being a distance measuring device used in an estimation process of estimating at least one of the position and orientation of the movable body; and The estimation is performed using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device.
9. A control system for generating a control command for controlling a movable body using measurement results of a plurality of distance measuring devices, characterized in that The control system includes: one or more memories, and one or more processors in communication with the one or more memories, the one or more processors being configured to perform a process including: specifying one or more processing target distance measuring devices from the plurality of distance measuring devices, each of the processing target distance measuring devices being a distance measuring device used in an estimation process of estimating at least one of the position and orientation of the movable body; and The estimation is performed using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device.
10. A control method for controlling a movable body using measurement results of a plurality of distance measuring devices, characterized in that: The control method comprises: specifying, by one or more processors, one or more processing target distance measuring devices from the plurality of distance measuring devices, each of the processing target distance measuring devices being a distance measuring device used in estimating at least one of the position and orientation of the movable body; and An estimation process is performed by one or more processors using three-dimensional point cloud data obtained by measurement by the processing object distance measuring device.
