Information processing device and mobile body

Abstract

To provide a technique for supporting traveling of a mobile body when the mobile body is brought into a support requiring state due to an operation mistake of an operator in a case where the operator remotely controls the mobile body by operating an operator input device provided at a place different from the mobile body.SOLUTION: An information processing device includes: a manual command generation section for generating a manual control command which regulates a traveling behavior of a mobile body in response to an operation of an operator input device by an operator; an information acquisition section for acquiring detection information generated when it is detected that the mobile body is brought into a support requiring state requiring a traveling support in a period when the mobile body is executing control of a remote manual driving mode to travel according to the manual control command by receiving the manual control command; an automatic command generation section for generating an automatic control command to allow the mobile body to travel without using the manual control command; and a transmission section for transmitting at least the automatic control command to the mobile body when the information acquisition section acquires the detection information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an information processing device and a mobile object. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a moving object that travels automatically by remote control has been known (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]

[0004] In some cases, a moving body is remotely controlled by an operator operating an operator input device provided at a location different from the moving body. In this case, the moving body may deviate from a predetermined target route or collide with an object present around the moving body due to an operation error of the operator. Therefore, when the moving body travels by remote control via the operator input device, a traveling assistance is desired to reduce the possibility that the moving body deviates from the target route or collide with an object present around the moving body. Furthermore, even when the moving body deviates from the target route or collides with an object present around the moving body, a traveling assistance is desired to correct the traveling defect of the moving body. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first embodiment of the present disclosure, an information processing device is provided. The information processing device used for driving a moving body capable of moving by unmanned driving includes: a manual command generating unit that generates a manual control command that specifies the driving operation of the moving body in response to an operation by an operator of an operator input device provided at a location different from the moving body; an information acquiring unit that acquires detection information that is generated when it is detected that the moving body has entered an assistance-required state that requires driving assistance during a period in which the moving body receives the manual control command and controls the moving body to drive according to the manual control command; an automatic command generating unit that generates an automatic control command for driving the moving body without using the manual control command; and a transmitting unit that transmits at least the automatic control command to the moving body when the information acquiring unit acquires the detection information. According to this embodiment, when the moving body enters an assistance-required state due to an operation error by an operator during a period in which the control of the remote manual driving mode is being performed, the information processing device can acquire detection information and transmit an automatic control command to the moving body that has entered the assistance-required state. In this way, when the moving body enters an assistance-required state, the information processing device can cause the moving body to drive using an automatic control command without using a manual control command corresponding to the operation of the operator. Thereby, the information processing device can support the traveling of the moving body when the moving body is in a state requiring support. Specifically, the information processing device can reduce the possibility that the moving body deviates from the target route or collides with an object present around the moving body due to, for example, an operation error by the operator. Furthermore, the information processing device can support the traveling of the moving body to correct the traveling defect of the moving body even when, for example, the moving body deviates from the target route or collides with an object present around the moving body due to an operation error by the operator. (2) In the above embodiment, when the information acquisition unit acquires the detection information, the transmission unit may execute either one of a first process of transmitting the automatic control command to the moving body without transmitting the manual control command to the moving body, and a second process of transmitting the manual control command, the automatic control command, and incidental information for instructing the moving body to use the automatic control command in preference to the manual control command to the moving body. According to this embodiment, when the information processing device acquires the detection information, the information processing device executes the first process to transmit an automatic control command to the moving body that has entered the support-requiring state without transmitting a manual control command. Also, when the information processing device acquires the detection information, the information processing device executes the second process to transmit the manual control command, the automatic control command, and incidental information to the moving body that has entered the support-requiring state. This allows the information processing device to instruct the moving body to use the automatic control command in preference to the manual control command. (3) According to a second aspect of the present disclosure, a moving body is provided. The moving body capable of moving by unmanned driving includes a control device that controls the traveling operation of the moving body, and when the control device receives both a manual control command and an automatic control command, the control device uses the automatic control command in preference to the manual control command, the manual control command being a command that specifies the traveling operation of the moving body generated in response to an operation by an operator of an operator input device provided in a location different from the moving body, and the automatic control command being a command generated to make the moving body travel without using the manual control command. According to this aspect, when the control device of the moving body receives both a manual control command and an automatic control command, the control device of the moving body can use the automatic control command in preference to the manual control command. (4) According to a third aspect of the present disclosure, a moving body is provided. The moving body capable of moving by unmanned driving includes a control device that controls the traveling operation of the moving body, and when the control device receives a manual control command, an automatic control command, and additional information, the control device uses the automatic control command in preference to the manual control command, and the manual control command is a command that specifies the traveling operation of the moving body generated in response to an operation by an operator of an operator input device provided in a location different from the moving body, the automatic control command is a command that is generated by an information processing device provided in a location different from the moving body in order to travel the moving body without using the manual control command, and the additional information is information for the information processing device to instruct the moving body to use the automatic control command in preference to the manual control command. According to this aspect, when the control device of the moving body receives a manual control command, an automatic control command, and additional information, the control device of the moving body can use the automatic control command in preference to the manual control command. The present disclosure can be realized in various forms other than the above-mentioned information processing device and mobile body, for example, in the form of an information processing system including the information processing device and the mobile body, an information processing method, a manufacturing method for the information processing device and the mobile body, a control method for the information processing device and the mobile body, a computer program for realizing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing the configuration of an information processing system according to a first embodiment. [Diagram 2] FIG. 1 is a diagram showing the configuration of an information processing device according to a first embodiment. [Diagram 3] 4 is a flowchart showing a processing flow in the first embodiment. [Figure 4] FIG. 11 is a diagram showing an example of a method for determining whether a moving object is in a state requiring assistance. [Diagram 5] FIG. 13 is a diagram showing the configuration of an information processing system according to a second embodiment. [Figure 6]10 is a flowchart showing a processing flow in a second embodiment. [Figure 7] FIG. 13 is a diagram showing the configuration of an information processing system according to a third embodiment. [Figure 8] 13 is a flowchart showing a processing flow in a third embodiment. [Figure 9] FIG. 13 is a diagram showing the configuration of an information processing system according to a fourth embodiment. [Figure 10] 13 is a flowchart showing a processing flow in a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: FIG. 1 is a diagram showing a configuration of an information processing system 1 in a first embodiment. When a mobile object remotely controlled by an operator from a location different from the mobile object becomes in an assistance-required state in which it requires assistance with traveling, the information processing system 1 assists the traveling of the mobile object or corrects the traveling defect of the mobile object. The assistance-required state is, for example, at least one of a deviation prediction state, an already-departed state, a collision prediction state, and an already-collision state. The deviation prediction state is a state in which it is predicted that the mobile object will deviate from a predetermined target route if the traveling is continued under the current traveling conditions. The traveling conditions are conditions that define the traveling operation of the mobile object. The traveling conditions include a plurality of elements that define the traveling operation of the mobile object. The traveling conditions include, for example, the acceleration and steering angle of the mobile object as elements that define the traveling operation of the mobile object. The already-departed state is a state in which the mobile object has deviated from the target route. The collision prediction state is a state in which it is predicted that the mobile object will collide with an object (hereinafter, an obstacle) present around the mobile object if the traveling is continued under the current traveling conditions. The collision-previous state is a state in which the moving object has collided with an obstacle. The information processing system 1 includes one or more moving objects, a sensor, an input system 2, and an information processing device 5.

[0009] The term "mobile body" refers to an object that can move, such as a vehicle 10 or an electric vertical take-off and landing aircraft (a so-called flying car). In this embodiment, the mobile body is the vehicle 10. The vehicle 10 may be a vehicle that runs on wheels or a vehicle that runs on caterpillar tracks, such as a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, etc. The vehicle 10 includes an electric vehicle (BEV: Battery Electric Vehicle), a gasoline vehicle, a hybrid vehicle, and a fuel cell vehicle. In addition, when the mobile body is other than the vehicle 10, the expressions "vehicle" and "car" in this disclosure can be appropriately replaced with "mobile body", and the expression "running" can be appropriately replaced with "movement".

[0010] The vehicle 10 includes a driving device 11, a steering device 12, a braking device 13, a communication device 14, and a vehicle control device 15. The driving device 11 accelerates the vehicle 10. The steering device 12 changes the traveling direction of the vehicle 10. The braking device 13 decelerates the vehicle 10. The communication device 14 communicates with an external device using wireless communication or the like. The external device is another device such as an information processing device 5, and another vehicle 10. The communication device 14 is, for example, a wireless communication device. The vehicle control device 15 controls the traveling operation of the vehicle 10. The vehicle control device 15 includes a CPU 150, an input / output interface 159, and a storage unit 157. The input / output interface 159 is used to communicate with various devices mounted on the vehicle 10. The storage unit 157 of the vehicle control device 15 stores various information including various programs P15 that control the operation of the vehicle control device 15. The CPU 150 of the vehicle control device 15 functions as an operation control unit 153 by developing various programs P15 stored in the storage unit 157. The operation control unit 153 executes driving control of the vehicle 10 by controlling the operation of actuators that change the acceleration / deceleration and steering angle of the vehicle 10. "Driving control" refers to various controls for driving actuators that perform the three functions of the vehicle 10, "running," "turning," and "stopping," such as adjusting the acceleration, speed, and steering angle of the vehicle 10. In this embodiment, the actuators include the actuators of the drive device 11, the actuators of the steering device 12, and the actuators of the braking device 13.

[0011] The vehicle 10 can run by unmanned driving. "Unmanned driving" means driving without the driving operation of a passenger. Driving operation means at least one of the operations of "running", "turning" and "stopping" of the vehicle 10. Unmanned driving is realized by automatic / manual remote control using a device provided outside the vehicle 10, or autonomous control of the vehicle 10. The vehicle 10 running by unmanned driving may have a passenger who does not perform driving operation on board. The passenger who does not perform driving operation includes, for example, a person who simply sits in the seat of the vehicle 10, and a person who performs work other than driving operation, such as assembly, inspection, and operation of switches, while riding on the vehicle 10. Note that driving by a passenger's driving operation is sometimes called "manned driving". In this embodiment, the vehicle 10 runs in a factory by being remotely controlled by the information processing device 5. Note that the configuration of the vehicle 10 is not limited to the above.

[0012] The sensor acquires information for detecting that the vehicle 10 has entered a state requiring assistance. In this embodiment, the sensor is a camera 90 (hereinafter, external camera 90) as an external sensor provided at a location different from the vehicle 10. The external camera 90 is a camera that captures an image of a detection range RG including the vehicle 10 from outside the vehicle 10. In order to capture an image of the entire road Ro by one or more external cameras 90, the installation positions and the number of external cameras 90 to be installed are determined in consideration of the detection range RG (angle of view) of the external camera 90, etc.

[0013] The input system 2 is a system that enables an operator to remotely control the traveling operation of the vehicle 10. The input system 2 includes an operator input device 3 and a display device 4.

[0014] The operator input device 3 is an input device for an operator to remotely control the traveling operation of the vehicle 10. The operator input device 3 is provided at a location different from the vehicle 10. The operator input device 3 is communicably connected to the information processing device 5. The operator input device 3 has one or more input operation units. The operator input device 3 includes, for example, a steering wheel, an accelerator pedal, and a brake pedal as the input operation units. The steering wheel changes the traveling direction of the vehicle 10 by remote control. The accelerator pedal accelerates the vehicle 10 by remote control. The brake pedal decelerates the vehicle 10 by remote control. Note that the configuration of the operator input device 3 is not limited to the above. The operator input device 3 may include, for example, a push button switch or a lever switch as the input operation units.

[0015] The display device 4 is used for visually checking the surrounding conditions of the vehicle 10 to be controlled when the operator operates the operator input device 3. The display device 4 includes a display screen. The display device 4 is disposed at a position where the operator can visually check the display screen. For example, a captured image acquired by a camera having a detection range RG that includes at least the surrounding area of ​​the vehicle 10 to be controlled is displayed on the display screen. The captured image is either an original image as raw data acquired by the camera or a processed image obtained by processing the original image.

[0016] 2 is a diagram showing a configuration of the information processing device 5 in the first embodiment. The information processing device 5 is used for driving the vehicle 10. The information processing device 5 includes a communication unit 51, a CPU 52, and a storage unit 53. In the information processing device 5, the communication unit 51, the CPU 52, and the storage unit 53 are each connected to each other via, for example, an internal bus.

[0017] The communication unit 51 of the information processing device 5 communicably connects the information processing device 5 to devices other than the information processing device 5. The communication unit 51 of the information processing device 5 is, for example, a wireless communication device.

[0018] The storage unit 53 of the information processing device 5 stores various information including various programs P5 for controlling the operation of the information processing device 5, a detection model Md, and an ideal path Ip. Details of the detection model Md and the ideal path Ip will be described later. The storage unit 53 of the information processing device 5 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0019] The CPU 52 of the information processing device 5 deploys various programs P5 stored in the storage unit 53, thereby functioning as a manual command generating unit 521, a calculating unit 522, an automatic command generating unit 523, and a detecting unit 524. Furthermore, the CPU 52 of the information processing device 5 deploys various programs P5 stored in the storage unit 53, thereby functioning as an information acquiring unit 525, a display control unit 526, a transmitting unit 527, and a notification control unit 528.

[0020] The manual command generating unit 521 generates a manual control command that specifies the driving operation of the vehicle 10 in response to an operation by the operator of the operator input device 3. Hereinafter, a driving mode in which the vehicle control device 15 of the vehicle 10 receives a manual control command and controls an actuator according to the manual control command, thereby causing the vehicle 10 to drive according to the manual control command, is referred to as a "remote manual driving mode." In this embodiment, the manual control command is a manual control signal that includes the acceleration and steering angle of the vehicle 10 as parameters. Note that in other embodiments, the manual control signal may include the speed of the vehicle 10 as a parameter.

[0021] The calculation unit 522 calculates the position of the vehicle 10. In this embodiment, the calculation unit 522 calculates the position of the vehicle 10 using a captured image acquired by the external camera 90. The calculation unit 522 calculates the coordinates of the positioning point 10e of the vehicle 10 in the image coordinate system, for example, using the external shape of the vehicle 10 detected from the captured image. Then, the calculation unit 522 calculates the position of the vehicle 10 by converting the calculated coordinates into coordinates in the global coordinate system. In this embodiment, the position information of the vehicle 10 includes X, Y, and Z coordinates in the global coordinate system of the factory.

[0022] The external shape of the vehicle 10 included in the captured image can be detected by, for example, inputting the captured image into a detection model Md that utilizes artificial intelligence. The detection model Md can be, for example, a trained machine learning model that has been trained to realize either semantic segmentation or instance segmentation. The detection model Md can be, for example, a convolutional neural network (hereinafter, CNN) trained by supervised learning using a training dataset. The training dataset has, for example, a plurality of training images including the vehicle 10, and a correct answer label indicating whether each region in the training image is a region indicating the vehicle 10 or a region indicating something other than the vehicle 10. During training of the CNN, it is preferable that the parameters of the CNN are updated by backpropagation (error backpropagation method) so as to reduce an error between the output result by the detection model Md and the correct answer label.

[0023] The calculation unit 522 may further calculate the orientation of the vehicle 10. The orientation of the vehicle 10 may be estimated based on the orientation of a movement vector of the vehicle 10 calculated from a position change of a feature point of the vehicle 10 between frames of captured images using, for example, an optical flow method. The orientation of the vehicle 10 may be calculated using, for example, an output result of a yaw rate sensor mounted on the vehicle 10.

[0024] The automatic command generating unit 523 generates an automatic control command for driving the vehicle 10 without using a manual control command. In this embodiment, the automatic control command is generated based on the acceleration and The automatic command generating unit 523 first determines a target position to which the vehicle 10 should next head in order to generate the automatic control signal. In this embodiment, the target position is represented by X, Y, and Z coordinates in the global coordinate system of the factory. The automatic command generating unit 523 determines a target position to which the vehicle 10 should next head using the position information of the vehicle 10 and an ideal route Ip stored in advance in the storage unit 53 of the information processing device 5. The ideal route Ip is a route along which the vehicle 10 should travel. The route is represented by a node indicating a departure point, a node indicating a passing point, a node indicating a destination, and links connecting the nodes. The automatic command generating unit 523 determines a target position on the ideal route Ip ahead of the current location of the vehicle 10. The automatic command generating unit 523 generates an automatic control signal for driving the vehicle 10 toward the determined target position. The automatic command generating unit 523 calculates the traveling speed of the vehicle 10 from the transition of the position of the vehicle 10, and compares the calculated traveling speed with a predetermined target speed of the vehicle 10. When the traveling speed is lower than the target speed, the automatic command generating unit 523 determines the acceleration so that the vehicle 10 accelerates, and when the traveling speed is higher than the target speed, the automatic command generating unit 523 determines the acceleration so that the vehicle 10 decelerates. When the vehicle 10 is located on the ideal route Ip, the automatic command generating unit 523 determines the steering angle so that the vehicle 10 does not deviate from the ideal route Ip, and when the vehicle 10 is not located on the ideal route Ip, in other words, when the vehicle 10 deviates from the ideal route Ip, the automatic command generating unit 523 determines the steering angle so that the vehicle 10 returns to the ideal route Ip. In this way, the automatic command generating unit 523 generates a traveling control signal including the acceleration and steering angle of the vehicle 10 as parameters. Note that in other embodiments, the automatic control signal may include the speed of the vehicle 10 as a parameter.

[0025] The detection unit 524 detects that the vehicle 10 has entered an assistance-requiring state during the period in which control of the remote manual driving mode is being executed, and generates detection information. The detection information is information indicating the vehicle 10 that has entered an assistance-requiring state during the period in which control of the remote manual driving mode is being executed. The detection unit 524 detects that the vehicle 10 has entered an assistance-requiring state using sensor information acquired by a sensor. In this embodiment, the detection unit 524 determines whether the vehicle 10 has entered an assistance-requiring state using the position of the vehicle 10 calculated by the calculation unit 522 by analyzing the captured image output from the external camera 90. The information acquisition unit 525 acquires the detection information generated by the detection unit 524.

[0026] The display control unit 526 controls the display contents to be displayed on the display screen. In this embodiment, the display control unit 526 displays on the display screen a processed image obtained by processing an original image of the vehicle 10 to be controlled and the surrounding area captured by the external camera 90. Note that the display control unit 526 may display on the display screen an image captured by an in-vehicle camera mounted on the vehicle 10 to be controlled in addition to or instead of the image captured by the external camera 90.

[0027] The transmission unit 527 transmits a control command to the vehicle 10. Specifically, when the information acquisition unit 525 acquires detection information during the period in which control of the remote manual driving mode is being executed, the transmission unit 527 transmits at least an automatic control command to the vehicle 10 that has entered the support-required state. In this embodiment, when the information acquisition unit 525 does not acquire detection information during the period in which control of the remote manual driving mode is being executed, the transmission unit 527 transmits a manual control signal to the vehicle 10. On the other hand, when the information acquisition unit 525 acquires detection information during the period in which control of the remote manual driving mode is being executed, the transmission unit 527 executes a first process of transmitting an automatic control signal to the vehicle 10 that has entered the support-required state without transmitting a manual control signal.

[0028] The notification control unit 528 notifies the operator that the vehicle 10 is running using the automatic control signal during a period in which the vehicle 10 is running using the automatic control signal. In this embodiment, when the transmission unit 527 transmits the automatic control signal to the vehicle 10, the notification control unit 528 displays the automatic control information on the display screen. The automatic control information is information indicating the vehicle 10 running using the automatic control signal. Note that the notification control unit 528 may notify the operator that the vehicle 10 is running according to the automatic control signal by, for example, emitting a voice such as a message directed to the operator from a speaker provided in either the operator input device 3 or the display device 4.

[0029] The configuration of the information processing device 5 is not limited to the above. Each unit of the information processing device 5 may be configured separately by multiple devices, or may be realized as a function of another device such as a sensor. Each unit of the information processing device 5 may be realized by cloud computing configured by one or more computers. At least some of the functions of the information processing device 5 may be a function of the vehicle control device 15.

[0030] Fig. 3 is a flowchart showing a process flow in the first embodiment. Each step shown in Fig. 3 is repeatedly executed at predetermined time intervals during the period in which control in the remote manual operation mode is being executed, for example.

[0031] In step S101, the external camera 90 captures an image of the vehicle 10 to be controlled, thereby acquiring an original image. In step S102, the external camera 90 transmits the original image to the information processing device 5. In step S103, the display control unit 526 of the information processing device 5 generates a processed image by performing processing such as distortion correction and noise removal on the original image acquired from the external camera 90. In step S104, the display control unit 526 transmits the processed image to the display device 4. In step S105, the display device 4 displays the processed image on the display screen. In step S106, the operator input device 3 accepts an input from the operator. In step S107, the operator input device 3 transmits the operation amount of each input operation unit to the information processing device 5. In step S108, the manual command generation unit 521 of the information processing device 5 generates a manual control signal indicating an acceleration corresponding to the operation amount of the accelerator pedal and the brake pedal of the operator input device 3, and a steering angle corresponding to the operation amount of the steering wheel of the operator input device 3. In step S109, the transmission unit 527 transmits the manual control signal to the vehicle 10 to be controlled. In step S110, the operation control unit 153 of the vehicle control device 15 mounted on the vehicle 10 controls the actuator using the received manual control signal to cause the vehicle 10 to travel at the acceleration and steering angle indicated in the manual control signal. The operation control unit 153 repeats receiving the manual control signal and controlling the actuator at a predetermined cycle.

[0032] In step S111, the calculation unit 522 of the information processing device 5 calculates the position of the vehicle 10 to be controlled by using the captured image. In step S112, the automatic command generation unit 523 determines a target position to which the vehicle 10 should next head, by using the position information of the vehicle 10 and the ideal route Ip. In step S113, the automatic command generation unit 523 generates an automatic control signal for driving the vehicle 10 toward the determined target position.

[0033] In step S114, the detection unit 524 uses the position information of the vehicle 10 to determine whether or not the vehicle 10 has entered a support-requiring state. If it is determined that the vehicle 10 has entered a support-requiring state (step S114: Yes), the detection unit 524 generates detection information in step S115. If the information acquisition unit 525 acquires the detection information (step S116: Yes), in step S117, the transmission unit 527 transmits an automatic control signal to the vehicle 10 that has entered a support-requiring state.

[0034] In step S118, the operation control unit 153 of the vehicle control device 15 mounted on the vehicle 10 controls the actuator using the received automatic control signal to make the vehicle 10 run at the acceleration and steering angle indicated in the automatic control signal. The operation control unit 153 repeats receiving the control signal and controlling the actuator at a predetermined cycle.

[0035] In step S119, the notification control unit 528 of the information processing device 5 transmits a display signal to the display device 4 for displaying an automatic control display on the display screen of the display device 4. In step S120, the display device 4 displays an automatic control display on the display screen. When the assistance required state is resolved, the information processing device 5 may again drive the vehicle 10 by a manual control signal, or may continue to drive the vehicle 10 by an automatic control signal.

[0036] FIG. 4 is a diagram showing an example of a method for determining whether the vehicle 10 has entered a state requiring assistance using the position of the vehicle 10. The two imaginary determination lines LL, LR are imaginary lines that do not actually exist on the road Ro for determining whether the vehicle 10 is in at least one of a departure predicted state and an already deviated state. The two imaginary determination lines LL, LR are each generated based on the ideal route Ip. The two imaginary determination lines LL, LR are each parallel to an n-th order function (n is an integer equal to or greater than 1) that indicates the ideal route Ip. The two imaginary determination lines LL, LR are positioned symmetrically with respect to the ideal route Ip. In FIG. 4, the area Ar (hereinafter, the delimited area Ar) defined by the two imaginary determination lines LL, LR is hatched.

[0037] The detection unit 524 judges whether the vehicle 10 is in at least one of a departure predicted state and an already deviated state based on the position of the vehicle 10 relative to the imaginary judgment lines LL and LR. Specifically, the detection unit 524 judges whether the vehicle 10 is located within the demarcated area Ar by using the relative positional relationship between each part 10p to 10s of the vehicle 10 and the positioning point 10e, and the vehicle coordinate point. The vehicle coordinate point is a coordinate point indicating the position of the vehicle 10. The parts 10p to 10s of the vehicle 10 are, for example, the left rear end 10p, the right rear end 10q, the left front end 10r, and the right front end 10s of the vehicle 10. When the detection unit 524 judges that the vehicle 10 is located within the demarcated area Ar, it calculates the shortest distance D1 between the vehicle 10 and the imaginary judgment lines LL and LR. The detection unit 524 determines that the vehicle 10 is in a departure predicted state when the shortest distance D1 between the vehicle 10 and the imaginary judgment lines LL, LR is less than a predetermined first distance. On the other hand, the detection unit 524 determines that the vehicle 10 is not in a departure predicted state when the shortest distance D1 between the vehicle 10 and the imaginary judgment lines LL, LR is equal to or greater than the first distance. The detection unit 524 also determines that the vehicle 10 is in an already deviated state when at least one of the following cases is determined: when the vehicle 10 is located outside the demarcated area Ar, and when the vehicle 10 is located on the imaginary judgment lines LL, LR. When the vehicle 10 is located on the imaginary judgment lines LL, LR, the shortest distance D1 between the vehicle 10 and the imaginary judgment lines LL, LR is zero.

[0038] The detection unit 524 judges whether the vehicle 10 is in a predicted collision state or a collision-already-collision state based on the position of the vehicle 10. Specifically, the detection unit 524 judges whether the vehicle 10 is in a predicted collision state or a collision-already-collision state by using the relative positional relationship between each of the parts 10p to 10s of the vehicle 10 and the positioning point 10e, the vehicle coordinate point, and the object coordinate point. The object coordinate point is a coordinate point indicating the position of the obstacle Ob. The object coordinate point is represented by a coordinate value in the global coordinate system. The detection unit 524 calculates the shortest distance D2 between the vehicle 10 and the obstacle Ob. The detection unit 524 judges that the vehicle 10 is in a predicted collision state when the shortest distance D2 between the vehicle 10 and the obstacle Ob is less than a predetermined second distance. On the other hand, the detection unit 524 judges that the vehicle 10 is not in a predicted collision state when the shortest distance D2 between the vehicle 10 and the obstacle Ob is equal to or greater than the second distance. Furthermore, when the shortest distance D2 between the vehicle 10 and the obstacle Ob is zero, the detection unit 524 determines that the vehicle 10 is in a collision state. Note that the method of determining whether the vehicle 10 is in a support-requiring state is not limited to the above.

[0039] According to the first embodiment, the vehicle 10 can be driven in a remote manual driving mode in which the vehicle 10 is remotely controlled by the operator operating the operator input device 3 provided at a location different from the vehicle 10. At this time, when the vehicle 10 is in a state requiring support due to an operation error of the operator during the period in which control of the remote manual driving mode is being performed, the information processing device 5 can acquire the detection information and transmit an automatic control signal to the vehicle 10 in the state requiring support. In this way, when the vehicle 10 is in a state requiring support, the information processing device 5 can drive the vehicle 10 using an automatic control signal without using a manual control signal according to the operation of the operator. In this way, when the vehicle 10 is in a state requiring support, the information processing device 5 can support the driving of the vehicle 10 when the vehicle 10 is in a state requiring support. Specifically, the information processing device 5 can reduce the possibility that the vehicle 10 deviates from the ideal route Ip or collides with an object Ob present around the vehicle 10 due to an operation error of the operator. Furthermore, the information processing device 5 can assist the driving of the vehicle 10 in correcting driving problems when the vehicle 10 deviates from the ideal route Ip due to an operator's operational error, and when the vehicle 10 collides with an object Ob present around the vehicle 10.

[0040] Furthermore, according to the first embodiment, when the information processing device 5 acquires the detection information, it can execute a first process of transmitting an automatic control signal to the vehicle 10 to inform the vehicle 10 that it has entered an assistance required state, without transmitting a manual control signal. In other words, the information processing device 5 can select a control signal to be used by the vehicle 10 depending on whether the vehicle 10 has entered an assistance required state, and transmit to the vehicle 10 only the control signal to be used by the vehicle 10.

[0041] Furthermore, according to the first embodiment, the information processing device 5 can display a captured image showing the surrounding conditions of the vehicle 10 to be controlled on a display screen arranged in a position visible to an operator operating the operator input device 3. This allows the operator to operate the operator input device 3 while viewing the display screen and checking the surrounding conditions of the vehicle 10 to be controlled.

[0042] According to the first embodiment, the information processing device 5 can notify the operator that the vehicle 10 is traveling according to an automatic control signal. This allows the operator to quickly grasp the vehicle 10 traveling according to an automatic control signal.

[0043] According to the first embodiment, the information processing device 5 can repeatedly calculate the automatic control signal at predetermined time intervals during the period in which the control of the remote manual driving mode is being executed. This allows the information processing device 5 to quickly transmit the automatic control signal to the vehicle 10 when it acquires the detection information.

[0044] Moreover, according to the first embodiment, the information processing device 5 includes the detection unit 524. This enables the information processing device 5 to determine whether or not the vehicle 10 has entered a support-required state.

[0045] According to the first embodiment, the information processing system 1 can remotely control the vehicle 10 to travel within the factory. Therefore, the information processing system 1 can move the vehicle 10 within the factory without using transportation equipment such as a crane or a conveyor. The vehicle 10 may travel unmanned outside the factory.

[0046] B. Second embodiment: FIG. 5 is a diagram showing a configuration of an information processing system 1a in the second embodiment. The information processing system 1a includes one or more vehicles 10a, an external camera 90, an input system 2, and an information processing device 5a. In this embodiment, the manual control command is a manual control signal including the acceleration and steering angle of the vehicle 10a as parameters. The automatic control command is an automatic control signal including the acceleration and steering angle of the vehicle 10a as parameters. The same components as those in the first embodiment are denoted by the same reference numerals.

[0047] The CPU 52a of the information processing device 5a functions as a manual command generating unit 521, a calculation unit 522a, an automatic command generating unit 523a, an information acquiring unit 525a, and a transmission unit 527a by developing various programs P5a stored in the storage unit 53a. The information acquiring unit 525a acquires detection information generated by a detection unit provided in an external device (for example, an external camera 90) different from the information processing device 5a. When the information acquiring unit 525a acquires the detection information, the calculation unit 522a calculates the position of the vehicle 10a using the captured image. When the information acquiring unit 525a acquires the detection information, the automatic command generating unit 523a determines a target position to which the vehicle 10a should next head, using the position information of the vehicle 10a and the ideal route Ip. Then, the automatic command generating unit 523a generates an automatic control signal for driving the vehicle 10a toward the determined target position. Furthermore, the automatic command generating unit 523a generates supplementary information. The incidental information is information for instructing the vehicle 10a to use the automatic control command in preference to the manual control command. In this embodiment, when the information acquisition unit 525a acquires the detection information, the transmission unit 527a executes a second process of transmitting the manual control signal, the automatic control signal, and the incidental information to the vehicle 10a that is in the support-required state.

[0048] The vehicle 10a functions as the operation control unit 153a by deploying various programs P15a stored in the storage unit 157a. When the vehicle 10a receives a manual control command, an automatic control command, and associated information from the information processing device 5a via the communication device 14, the vehicle control device 15a uses the automatic control command in preference to the manual control command. In this embodiment, when the manual control signal, the automatic control signal, and associated information are received, the operation control unit 153a controls the actuator using the automatic control signal to make the vehicle 10a run at the acceleration and steering angle indicated in the automatic control signal.

[0049] Fig. 6 is a flowchart showing a process flow in the second embodiment. Each step shown in Fig. 6 is repeatedly executed at predetermined time intervals during the period in which control in the remote manual operation mode is being executed, for example.

[0050] In step S201, the external camera 90 captures an image of the vehicle 10a to be controlled, thereby acquiring an original image. In step S202, the external camera 90 transmits the original image to the display device 4. In step S203, the display device 4 displays the original image on the display screen. In step S204, the operator input device 3 accepts an input from the operator. In step S205, the operator input device 3 transmits the operation amount of each input operation unit to the information processing device 5a. In step S206, the manual command generating unit 521 of the information processing device 5a generates a manual control signal indicating an acceleration corresponding to the operation amount of the accelerator pedal and the brake pedal of the operator input device 3, and a steering angle corresponding to the operation amount of the steering wheel of the operator input device 3. In step S207, the transmitting unit 527 transmits the manual control signal to the vehicle 10a to be controlled. In step S208, the operation control unit 153a of the vehicle control device 15a mounted on the vehicle 10a controls the actuator using the received manual control signal to make the vehicle 10a run at the acceleration and steering angle represented by the manual control signal. The information processing device 5a repeats receiving the manual control signal and controlling the actuator at a predetermined cycle.

[0051] When the information acquisition unit 525 of the information processing device 5a acquires the detection information (step S209: Yes), in step S210, the information acquisition unit 525 transmits a request signal to the external camera 90 to acquire an original image from the external camera 90. In step S211, the external camera 90 that has received the request signal transmits the original image to the information processing device 5a. In step S212, the calculation unit 522a calculates the position of the vehicle 10a to be controlled using the captured image. In step S213, the automatic command generation unit 523a determines the target position to which the vehicle 10a should next move using the position information of the vehicle 10a and the ideal route Ip. In step S214, the automatic command generation unit 523a generates an automatic control signal for driving the vehicle 10a toward the determined target position. In step S215, the automatic command generation unit 523a generates accompanying information. In step S216, the transmission unit 527a transmits the manual control signal, the automatic control signal, and the accompanying information to the vehicle 10a to be controlled.

[0052] The operation control unit 153 of the vehicle control device 15a mounted on the vehicle 10a that has received the manual control signal, the automatic control signal, and the accompanying information controls the actuator using the automatic control signal in step S217. As a result, the operation control unit 153 causes the vehicle 10a to travel with the acceleration and steering angle represented by the automatic control signal. The vehicle control device 15a repeats receiving the control signal and controlling the actuator at a predetermined cycle.

[0053] According to the second embodiment, when the information processing device 5a acquires the detection information, it can transmit the manual control signal, the automatic control signal, and the incidental information to the vehicle 10a that is in the support-required state. In this way, the information processing device 5a can instruct the vehicle 10a to use the automatic control signal in preference to the manual control signal. As a result, when the vehicle 10a receives the manual control signal, the automatic control command signal, and the incidental information, it can use the automatic control signal in preference to the manual control command signal. In other words, the vehicle 10a can select the control signal to be used from the manual control signal and the automatic control signal by using the incidental information.

[0054] Furthermore, according to the second embodiment, when the information processing device 5a acquires the detection information, it can calculate the position of the vehicle 10a and generate an automatic control signal. In this way, during a period when it is not necessary to transmit an automatic control signal to the vehicle 10a, it is possible to generate a manual control signal without calculating the position of the vehicle 10a or generating an automatic control signal. This can reduce the processing load of the information processing device 5a.

[0055] C. Third embodiment: FIG. 7 is a diagram showing a configuration of an information processing system 1b in the third embodiment. The information processing system 1b includes one or more vehicles 10b, an external camera 90, an input system 2, and an information processing device 5b. In this embodiment, the manual control command is also a manual control signal including the acceleration and steering angle of the vehicle 10b as parameters. In contrast, in this embodiment, the automatic control command includes either an automatic control signal including the acceleration and steering angle of the vehicle 10b as parameters, or a stop signal for stopping the vehicle 10b. The same reference numerals are used for the same configurations as those in the first and second embodiments.

[0056] The CPU 52b of the information processing device 5b deploys various programs P5b stored in the memory unit 53b, and thereby functions as a manual command generating unit 521, a calculating unit 522, an automatic command generating unit 523b, an information acquiring unit 525a, and a transmitting unit 527b.

[0057] When the information acquisition unit 525a acquires the detection information, if a predetermined start condition is satisfied, the automatic command generation unit 523b generates an automatic control signal. When the information acquisition unit 525a acquires the detection information, if a predetermined start condition is not satisfied, the automatic command generation unit 523b generates a stop signal. The start condition is a condition that triggers the start of the generation of the automatic control signal. The start condition includes, for example, at least one of a time condition, an accuracy condition, and a transition condition. The time condition is a condition related to the timing of starting the generation of the automatic control signal. The case where the time condition is satisfied is, for example, a case where a predetermined time has elapsed from the time when the control of the remote manual driving mode is started. The accuracy condition is a condition related to the accuracy of the position information of the vehicle 10b used to generate the automatic control signal. The case where the accuracy condition is satisfied is, for example, a case where the accuracy of the position information of the vehicle 10b calculated by the calculation unit 522 is equal to or higher than a predetermined accuracy. The transition condition is a condition for enabling a smooth transition from remote control according to a manual control signal to control according to an automatic control signal. The case where the transition condition is satisfied is a case where the vehicle 10b is unlikely to have a driving problem when the remote control according to the manual control signal is transitioned to the remote control according to the automatic control signal. Specifically, the case where the transition condition is satisfied is, for example, a case where the vehicle 10b is expected to travel at less than a predetermined steering angle for a predetermined distance or more when the vehicle 10b travels along the ideal route Ip. In other words, the case where the transition condition is satisfied is, for example, a case where the vehicle 10b is expected to travel a substantially straight route without traveling a curved route for a predetermined distance or more.

[0058] When the information acquisition unit 525a acquires the detection information, the transmission unit 527b transmits a manual control signal and an automatic control command to the vehicle 10a that has entered the support-requiring state.

[0059] The CPU 150b of the vehicle control device 15b functions as an operation control unit 153b by developing various programs P15b stored in the storage unit 157b. When the vehicle 10b receives both a manual control command and an automatic control command from the information processing device 5b via the communication device 14, the vehicle control device 15b uses the automatic control command in preference to the manual control command. In this embodiment, when both the manual control signal and the automatic control signal are received, the operation control unit 153b controls the actuator using the automatic control signal to make the vehicle 10b run at the acceleration and steering angle indicated by the automatic control signal.

[0060] Fig. 8 is a flowchart showing a process flow in the third embodiment. Each step shown in Fig. 8 is repeatedly executed at predetermined time intervals during the period in which the control of the remote manual operation mode is being executed. Note that each step from step S301 to step S308 is the same as each step from step S201 to step S208 shown in Fig. 6.

[0061] In step S309, the calculation unit 522 calculates the position of the vehicle 10b to be controlled using the captured image. When the information acquisition unit 525 acquires the detection information (step S310: Yes), and when the start condition is satisfied (step S311: Yes), the automatic command generation unit 523b executes step S312. In step S312, the automatic command generation unit 523b uses the position information of the vehicle 10b and the ideal route Ip to determine a target position to which the vehicle 10b should next move. In step S313, the automatic command generation unit 523b generates an automatic control command including an automatic control signal for driving the vehicle 10b toward the determined target position. On the other hand, when the information acquisition unit 525 acquires the detection information (step S310: Yes), and when the start condition is not satisfied (step S311: No), the automatic command generation unit 523b executes step S314. In step S314, the automatic command generating unit 523b generates an automatic control command including a stop signal instead of the automatic control signal. In step S315, the transmitting unit 527b transmits the manual control signal and the automatic control command including either the automatic control signal or the stop signal to the vehicle 10b to be controlled.

[0062] The operation control unit 153b of the vehicle control device 15b of the vehicle 10b that has received both the manual control signal and the automatic control signal controls the actuator using the automatic control signal in step S316. As a result, the operation control unit 153b causes the vehicle 10b to run at the acceleration and steering angle indicated in the automatic control signal. The vehicle control device 15a repeats receiving the control signal and controlling the actuator at a predetermined cycle. On the other hand, when both the manual control signal and the stop signal are received, the operation control unit 153b controls the actuator using the stop signal in step S316. As a result, the operation control unit 153b causes the vehicle 10b to stop.

[0063] According to the third embodiment, when the information processing device 5b acquires the detection information, it can transmit a manual control signal and an automatic control command, which is either an automatic control signal or a stop signal, to the vehicle 10b that is in the support-required state. Then, when the vehicle control device 15b of the vehicle 10b receives both the manual control signal and the automatic control command, it can use the control signal related to the automatic control command in preference to the manual control signal. In other words, the vehicle 10b can select the control signal to be used.

[0064] Furthermore, according to the third embodiment, when the information processing device 5b acquires the detection information, if the start condition is satisfied, it can generate an automatic control signal.

[0065] According to the third embodiment, when the start condition includes a time condition, the information processing device 5b can generate a manual control signal without generating an automatic control signal during a period from when the control of the remote manual driving mode is started until a predetermined time has elapsed. This can reduce the processing load of the information processing device 5b during a period from when the control of the remote manual driving mode is started until a predetermined time has elapsed. Furthermore, when the information processing device 5b acquires detection information after a predetermined time has elapsed from when the control of the remote manual driving mode is started, the information processing device 5b can quickly transmit either an automatic control signal or a stop signal to the vehicle 10b depending on whether the start condition is satisfied.

[0066] According to the third embodiment, when the start condition includes the accuracy condition, the information processing device 5b can generate an automatic control signal using a calculation result of the position of the vehicle 10b that has an accuracy equal to or higher than a predetermined accuracy. This allows the information processing device 5b to reduce the error between the ideal route Ip and the actual travel route when the vehicle 10b is traveled according to the automatic control signal.

[0067] According to the third embodiment, when the start condition includes a transition condition, the information processing device 5b can generate and transmit an automatic control signal when the transition condition is satisfied. This can reduce the possibility that the vehicle 10b will have a driving problem when the remote control is transitioned from the remote control according to the manual control signal of the vehicle 10b to the remote control according to the automatic control signal.

[0068] D. Fourth embodiment: FIG. 9 is a diagram showing a configuration of an information processing system 1c in the fourth embodiment. The information processing system 1c includes one or more vehicles 10c equipped with a vehicle control device 15c having some functions of the information processing device 5, an external camera 90, an input system 2, and an information processing device 5c. In this embodiment, the manual control command is a manual control signal including the acceleration and steering angle of the vehicle 10c as parameters. In contrast, in this embodiment, the automatic control command is a route control command indicating the route along which the vehicle 10c should travel. The same reference numerals are used for the same configurations as those in the first to third embodiments.

[0069] The CPU 52c of the information processing device 5c deploys various programs P5c stored in the storage unit 53c, thereby functioning as a manual command generating unit 521, an automatic command generating unit 523c, an information acquiring unit 525a, and a transmitting unit 527. The automatic command generating unit 523c generates a path control command by using an ideal path Ip stored in advance in the storage unit 53c.

[0070] The CPU 150c of the vehicle control device 15c loads various programs P15c stored in the storage unit 157c, thereby functioning as a calculation unit 151, a signal generation unit 152, and an operation control unit 153c.

[0071] When the vehicle 10c receives a route control command as an automatic control command from the information processing device 5c via the communication device 14, the calculation unit 151 calculates the position of the vehicle 10c using the captured image. The signal generation unit 152 determines a target position to which the vehicle 10c should next head, using the position information of the vehicle 10c and the route control command. Then, the signal generation unit 152 generates an automatic control signal for traveling toward the determined target position. The operation control unit 153c uses the generated automatic control signal to control the actuator, thereby causing the vehicle 10c to travel at the acceleration and steering angle indicated in the automatic control signal.

[0072] Fig. 10 is a flowchart showing a process flow in the fourth embodiment. Each step shown in Fig. 10 is repeatedly executed at predetermined time intervals during the period in which the control of the remote manual operation mode is being executed. Each step from step S401 to step S408 is the same as each step from step S201 to step S208 shown in Fig. 6.

[0073] When the information acquisition unit 525a of the information processing device 5c acquires the detection information (step S409: Yes), in step S410, the automatic command generation unit 523c generates a route control command as an automatic control command. In step S411, the transmission unit 527 transmits the route control command to the vehicle 10c to be controlled. When the vehicle control device 15c of the vehicle 10c receives the route control command, in step S412, the calculation unit 151 of the vehicle control device 15c mounted on the vehicle 10c transmits a request signal to the external camera 90 to acquire an original image from the external camera 90. In step S413, the external camera 90 that has received the request signal transmits the original image to the vehicle 10c. In step S414, the calculation unit 151 of the vehicle control device 15c mounted on the vehicle 10c calculates the position of the vehicle 10c to be controlled using the captured image. In step S415, the signal generating unit 152 uses the position information of the vehicle 10c and the route control command to determine a target position to which the vehicle 10c should next head. In step S416, the signal generating unit 152 generates an automatic control signal for traveling toward the determined target position. In step S417, the operation control unit 153c uses the generated automatic control signal to control the actuator, thereby causing the vehicle 10c to travel at the acceleration and steering angle represented in the automatic control signal.

[0074] According to the fourth embodiment, when the information processing device 5c acquires the detection information, it can transmit a route control command as an automatic control command to the vehicle 10c that has entered the support-required state. In addition, the external camera 90 can transmit an original image of the vehicle 10c that has entered the support-required state to the vehicle 10c in response to a request from the vehicle 10c. In this way, the vehicle control device 15c can calculate the position of the vehicle 10c and generate an automatic control signal using the position information of the vehicle 10c and the route control command. In other words, when the vehicle 10c enters the support-required state, it can eliminate the departure prediction state, the already-departure state, the collision prediction state, and the already-collision state by automatically traveling using the automatic control signal generated by the vehicle 10c itself without using a manual control command.

[0075] Furthermore, according to the fourth embodiment, after transmitting the automatic control command, the information processing system 1c can cause the vehicle 10c to run by autonomous control without remote control by the information processing device 5c.

[0076] The automatic control command is not limited to an automatic control signal and a route control command. The automatic control command is, for example, any one of an automatic control signal, a route control command, and a destination control command. The destination control command is a control command that indicates the destination of the vehicle 10. The destination control command may include a target arrival time at the destination.

[0077] E. Other embodiments: E-1. Other embodiment 1: The calculation units 151, 522, and 522a may calculate the position and orientation of the vehicles 10, 10a to 10c using sensor information acquired by a sensor of a type different from the external camera 90. The calculation units 151, 522, and 522a may calculate the position and orientation of the vehicles 10, 10a to 10c using, for example, measurement point cloud data that three-dimensionally represents the external shape of the vehicles 10, 10a to 10c acquired by a LiDAR (Light Detection And Ranging) (hereinafter, external LiDAR) as an external sensor. In this case, the calculation units 151, 522, and 522a calculate the position and orientation of the vehicles 10, 10a to 10c by, for example, matching the measurement point cloud data with the reference point cloud data. The reference point cloud data is reference data used as a template in matching with the measurement point cloud data. The reference point cloud data is, for example, virtual three-dimensional point cloud data generated based on three-dimensional CAD data representing the external shape of the vehicle 10, 10a to 10c. For example, one of ICP (Iteractive Closest Point) and NDT (Normal Distributions Transform) is used as the matching algorithm by the calculation unit 151, 522, 522a. The calculation unit 151, 522, 522a may calculate the position and orientation of the vehicle 10, 10a to 10c using measurement point cloud data acquired by detecting the vehicle 10, 10a to 10c of the calculation target by an on-board lidar mounted on another vehicle 10, 10a to 10c different from the vehicle 10, 10a to 10c of the calculation target.

[0078] E-2. Alternative embodiment 2: The detection unit 524 may detect that the vehicle 10, 10a to 10c has entered an assistance-required state by analyzing sensor information acquired by a sensor without using the position of the vehicle 10, 10a to 10c. In this case, the sensor information may be information acquired by an external sensor or information acquired by an in-vehicle sensor. When the sensor information is a captured image acquired by the external camera 90, the detection unit 524 detects the vehicle 10, 10a to 10c and a reference object in the captured image, for example, and calculates the separation distance between the vehicle 10, 10a to 10c and the reference object. Then, the detection unit 524 determines whether the vehicle 10, 10a to 10c is in at least one of a departure predicted state and an already-departed state according to the separation distance between the vehicle 10, 10a to 10c and the reference object. The reference object is, for example, at least one of a dividing line on the roadway Ro, such as a road outer side line, a lane boundary line, a center line, and a roadside object such as a guard rail provided on both ends in the width direction of the roadway Ro. In addition, the detection unit 524 detects the vehicle 10, 10a to 10c and the obstacle Ob in the captured image, for example, and calculates the distance between the vehicle 10, 10a to 10c and the obstacle Ob. Then, the detection unit 524 judges whether the vehicle 10, 10a to 10c is in at least one of a collision prediction state and a collision already occurred state according to the distance between the vehicle 10, 10a to 10c and the obstacle Ob. In this embodiment, the detection unit 524 can detect that the vehicle 10, 10a to 10c has entered an assistance required state without using the position of the vehicle 10, 10a to 10c.

[0079] E-3. Other embodiment 3: The detection unit 524 may detect that the vehicle 10, 10a to 10c has entered an assistance-required state according to the driving conditions of the vehicle 10, 10a to 10c. In this case, the detection unit 524 acquires the driving speed, acceleration, and steering angle of the vehicle 10, 10a to 10c from the detection results of an in-vehicle sensor mounted on the vehicle 10, 10a to 10c, for example. The detection unit 524 may also acquire the driving speed, acceleration, steering angle, and the like of the vehicle 10, 10a to 10c based on the acceleration, steering angle, speed, and the like represented by the manual control command and the automatic control command, for example. In this form, the detection unit 524 can detect that the vehicle 10, 10a to 10c has entered an assistance-required state according to the driving conditions of the vehicle 10, 10a to 10c.

[0080] E-4. Other embodiment 4: The detection unit 524 may detect that the vehicle 10, 10a to 10c has entered the support-required state by using sensor information acquired by a type of sensor different from the camera and the lidar. Specifically, the detection unit 524 may detect that the vehicle 10, 10a to 10c has entered the support-required state by using, for example, sensor information acquired by an on-board radar mounted on the vehicle 10, 10a to 10c to be detected. The detection unit 524 may also detect that the vehicle 10, 10a to 10c has entered the support-required state by using, for example, a radar (Radar: Radio Detection and Ranging) (hereinafter, external radar) as an external sensor. Furthermore, the detection unit 524 may detect that the vehicle 10, 10a to 10c has entered the support-required state by using, for example, sensor information acquired by a laser scanner installed on the road Ro. In order to determine whether the vehicle 10, 10a to 10c is in at least one of a predicted collision state and a collision state, the laser scanner is provided at a position where it can be detected within a predetermined distance range from the obstacle Ob. Furthermore, the detection unit 524 may detect that the vehicle 10, 10a to 10c has entered a support-required state using sensor information acquired by an impact sensor that detects an impact. In this case, the impact sensor may be mounted on the vehicle 10, 10a to 10c to be detected, or may be attached to the obstacle Ob. In this embodiment, the detection unit 524 can detect that the vehicle 10, 10a to 10c has entered a support-required state using sensor information acquired by a type of sensor different from the camera and the lidar.

[0081] E-5. Alternative embodiment 5: The manual command generating unit 521, the detecting unit 524, and the information acquiring unit 525, 525a may be one function of the vehicle control device 15, 15a to 15c mounted on the vehicle 10, 10a to 10c. In this embodiment, the vehicle control device 15, 15a to 15c can generate a manual control signal by acquiring the amount of operation by the operator of the operator input device 3, and can travel in a remote manual driving mode. Furthermore, the vehicle control device 15, 15a to 15c can detect that the vehicle 10, 10a to 10c has entered an assistance-required state. Furthermore, the vehicle control device 15, 15a to 15c can recognize that the vehicle 10, 10a to 10c has entered an assistance-required state by acquiring the detection information. Furthermore, when the vehicle control device 15, 15a to 15c acquires the detection information, it can generate an automatic control signal and control the actuator using the automatic control signal, thereby enabling automatic travel without using a manual control command. That is, the vehicles 10, 10a to 10c can travel without receiving a control command from the information processing devices 5, 5a to 5c.

[0082] E-6. Other embodiment 6: The manual command generating unit 521 may uniformly multiply the operation amounts of the input operation units of the operator input device 3, which correspond to the elements that define the traveling operation of the vehicles 10, 10a to 10c, by a predetermined constant to generate a manual control signal. Specifically, when the manual control signal includes the acceleration and steering angle of the vehicles 10, 10a to 10c as parameters, the manual command generating unit 521 uniformly multiplies the operation amounts of the accelerator pedal, the brake pedal, and the steering wheel of the operator input device 3 by a predetermined constant. Then, the manual command generating unit 521 generates a manual control signal so that the acceleration of the vehicles 10, 10a to 10c is a value obtained by multiplying the operation amounts of the accelerator pedal and the brake pedal of the operator input device 3 by a constant. Also, the manual command generating unit 521 generates a manual control signal so that the steering angle of the vehicles 10, 10a to 10c is a value obtained by multiplying the operation amount of the steering wheel of the operator input device 3 by a constant. With this configuration, when the upper and lower limit values ​​of the output value of the operator input device 3 differ from the upper and lower limit values ​​of the output value of the actuator of the vehicle 10, 10a to 10c, a manual control signal can be generated that matches the output possible range of the actuator of the receiving vehicle 10, 10a to 10c.

[0083] E-7. Other embodiment 7: The manual command generating unit 521 may set an upper limit value for each element that defines the traveling operation of the vehicles 10, 10a to 10c, for the operation amount of each input operation unit of the operator input device 3, which corresponds to each element that defines the traveling operation of the vehicles 10, 10a to 10c. For example, when the manual control signal includes the acceleration and steering angle of the vehicles 10, 10a to 10c as parameters, the manual command generating unit 521 executes the following process. In this case, when the operation amount of the accelerator pedal of the operator input device 3 is equal to or greater than a first upper limit value, the manual command generating unit 521 generates a manual control signal so that the acceleration of the vehicles 10, 10a to 10c becomes the first upper limit value. Also, when the operation amount of the steering wheel of the operator input device 3 is equal to or greater than a second upper limit value, the manual command generating unit 521 generates a manual control signal so that the steering angle of the vehicles 10, 10a to 10c becomes the second upper limit value. In this manner, by setting an upper limit value for each element that defines the running operation of the vehicle 10, 10a to 10c, it is possible to prevent the vehicle 10, 10a to 10c from suddenly changing in the running operation, such as the vehicle 10, 10a to 10c suddenly accelerating or turning.

[0084] E-8. Other embodiment 8: In each of the first to third embodiments, the information processing devices 5, 5a, and 5b execute processes from acquiring the position information of the vehicles 10, 10a, and 10b to generating the automatic control signal. In contrast, the vehicles 10, 10a, and 10b may execute at least a part of the processes from acquiring the position information of the vehicles 10, 10a, and 10b to generating the automatic control signal. For example, the following forms (1) to (3) may be used.

[0085] (1) The information processing device 5, 5a, 5b may acquire the position information of the vehicle 10, 10a, 10b, determine the target position to which the vehicle 10, 10a, 10b should next head, and generate a route from the current location of the vehicle 10, 10a, 10b represented in the acquired position information to the target position. The information processing device 5, 5a, 5b may generate a route to the target position between the current location and the destination, or may generate a route to the destination. The information processing device 5, 5a, 5b may transmit the generated route to the vehicle 10, 10a, 10b. The vehicle 10, 10a, 10b may generate an automatic control signal so that the vehicle 10, 10a, 10b runs on the route received from the information processing device 5, 5a, 5b, and control the actuator using the generated automatic control signal.

[0086] (2) The information processing devices 5, 5a, 5b may acquire position information of the vehicles 10, 10a, 10b and transmit the acquired position information to the vehicles 10, 10a, 10b. The vehicles 10, 10a, 10b may determine a target position to which the vehicles 10, 10a, 10b should next travel, generate a route from the current position of the vehicles 10, 10a, 10b represented in the received position information to the target position, generate an automatic control signal so that the vehicles 10, 10a, 10b travel on the generated route, and control an actuator using the generated automatic control signal.

[0087] (3) In the above-mentioned (1) and (2) forms, the vehicles 10, 10a, and 10b may be equipped with an internal sensor, and a detection result output from the internal sensor may be used for at least one of generating a route and generating an automatic control signal. The internal sensor may include, for example, an on-board camera, an on-board lidar, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, and a gyro sensor. For example, in the above-mentioned (1) form, the information processing device 5, 5a, and 5b may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating a route. In the above-mentioned (1) form, the vehicles 10, 10a, and 10b may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the automatic control signal when generating an automatic control signal. In the above-mentioned (2) form, the vehicles 10, 10a, and 10b may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating a route. In the above embodiment (2), the vehicles 10, 10a, 10b may acquire the detection result of the internal sensor, and when generating an automatic control signal, may reflect the detection result of the internal sensor in the automatic control signal.

[0088] (4) In the above fourth embodiment, an internal sensor may be mounted on the vehicle 10c, and a detection result output from the internal sensor may be used for at least one of generating a route and generating an automatic control signal. For example, the vehicle 10c may obtain the detection result of the internal sensor, and when generating a route, may reflect the detection result of the internal sensor in the route. The vehicle 10c may obtain the detection result of the internal sensor, and when generating an automatic control signal, may reflect the detection result of the internal sensor in the automatic control signal.

[0089] (5) In the above fourth embodiment, the vehicle 10c acquires the position information of the vehicle 10c using the detection result of the external sensor. In contrast, the vehicle 10c may be equipped with an internal sensor, and the vehicle 10c may acquire position information using the detection result of the internal sensor, determine a target position to which the vehicle 10c should next go, generate a route from the current location of the vehicle 10c represented in the acquired position information to the target position, generate an automatic control signal for traveling the generated route, and control the actuator using the generated automatic control signal. In this case, the vehicle 10c can travel without using any detection result of the external sensor. The vehicle 10c may acquire a target arrival time or traffic congestion information from outside the vehicle 10c, and reflect the target arrival time or traffic congestion information in at least one of the route and the automatic control signal. In addition, all of the functional configurations of the information processing systems 1, 1a to 1c may be provided in the vehicles 10, 10a to 10c. That is, the processes realized by the information processing systems 1, 1a to 1c, such as obtaining the detection information and generating the control commands described in the present disclosure, may be realized by the vehicles 10, 10a to 10c alone.

[0090] E-9. Other embodiment 9: In each of the above embodiments, the vehicles 10, 10a to 10c may have a configuration capable of moving by unmanned driving, and may be in the form of a platform having the configuration described below, for example. Specifically, the vehicles 10, 10a to 10c may have at least the vehicle control devices 15, 15a to 15c, the drive device 11, the steering device 12, and the braking device 13 in order to perform the three functions of "running", "turning", and "stopping" by unmanned driving. When the vehicles 10, 10a to 10c acquire information from the outside for unmanned driving, the vehicles 10, 10a to 10c may further have the communication device 14. That is, the vehicles 10, 10a to 10c capable of moving by unmanned driving may not be equipped with at least a part of interior parts such as a driver's seat and a dashboard, may not be equipped with at least a part of exterior parts such as a bumper and a fender, and may not be equipped with a body shell. In this case, the remaining parts such as the body shell may be attached to the vehicle 10, 10a to 10c before the vehicle 10, 10a to 10c is shipped from the factory, or the remaining parts such as the body shell may be attached to the vehicle 10, 10a to 10c after the vehicle 10, 10a to 10c is shipped from the factory without the remaining parts such as the body shell being attached to the vehicle 10, 10a to 10c. Each part may be attached from any direction such as the upper side, lower side, front side, rear side, right side, or left side of the vehicle 10, 10a to 10c, and may be attached from the same direction or from different directions. Note that the position determination and the like may be performed for the form of the platform in the same manner as for the vehicles 10, 10a to 10c in each of the above embodiments.

[0091] E-10. Other embodiment 10: The vehicle 10, 10a to 10c may be manufactured by combining a plurality of modules. The module means a unit composed of a plurality of parts grouped according to the location or function of the vehicle 10, 10a to 10c. For example, the platform of the vehicle 10, 10a to 10c may be manufactured by combining a front module constituting the front part of the platform, a central module constituting the central part of the platform, and a rear module constituting the rear part of the platform. The number of modules constituting the platform is not limited to three, and may be two or less or four or more. In addition to or instead of the parts constituting the platform, parts constituting parts of the vehicle 10, 10a to 10c other than the platform may be modularized. The various modules may include any exterior parts such as a bumper or a grille, or any interior parts such as a seat or a console. In addition to the vehicle 10, 10a to 10c, any mobile body may be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of parts by welding or fasteners, or by integrally molding at least some of the parts constituting the module as a single part by casting. The molding method of integrally molding a single part, particularly a relatively large part, is also called gigacast or megacast. For example, the front module, the center module, and the rear module described above may be manufactured using gigacast.

[0092] E-11. Other embodiment 11: The transportation of the vehicles 10, 10a to 10c using the unmanned driving of the vehicles 10, 10a to 10c is also called "self-propelled transportation." The configuration for realizing the self-propelled transportation is also called "vehicle remote-controlled autonomous traveling transportation system." The production method for producing the vehicles 10, 10a to 10c using the self-propelled transportation is also called "self-propelled production." In the self-propelled production, for example, at a factory where the vehicles 10, 10a to 10c are manufactured, at least a part of the transportation of the vehicles 10, 10a to 10c is realized by the self-propelled transportation.

[0093] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0094] 1, 1a to 1c...information processing system, 2...input system, 3...operator input device, 4...display device, 5, 5a to 5c...information processing device, 10, 10a to 10c...vehicle, 10e...positioning point, 10p...left rear end, 10q...right rear end, 10r...left front end, 10s...right front end, 11...driving device, 12...steering device, 13...braking device, 14...communication device, 15, 15a to 15c...vehicle control device, 51...communication unit of information processing device, 52, 52a to 52c...CPU of information processing device, 53, 53a to 53c...storage unit of information processing device, 90...external camera, 150, 150a to 150c...CPU of vehicle control device, 151, 522, 522a...calculation unit, 152...signal generation unit, 153, 153a to 153c...operation control unit, 157, 157a to 157c...storage unit of vehicle control device, 159...input / output interface, 521...manual command generation unit, 523, 523a to 523c...automatic command generation unit, 524...detection unit, 525, 525a...information acquisition unit, 526...display control unit, 527, 527a, 527b...transmission unit, 528...notification control unit, Ar...division area, D1...shortest distance between moving body and virtual judgment line, D2...shortest distance between moving body and obstacle, Ip...ideal route, LL, LR...virtual judgment line, Md...detection model, Ob...obstacle, P5, P5a to P5c, P15, P15a to P15c...program, RG...detection range, Ro...road

Claims

1. An information processing device used for traveling of a moving body that can move by unmanned operation, a manual command generating unit configured to generate manual control commands that define a traveling operation of the moving body in response to an operation by an operator of an operator input device provided at a location different from the moving body; an information acquisition unit that acquires detection information generated when it is detected that the moving body has entered an assistance-required state requiring driving assistance during a period in which the moving body receives the manual control command and controls a remote manual driving mode in which the moving body travels in accordance with the manual control command; an automatic command generating unit that generates an automatic control command for causing the moving body to travel without using the manual control command; a transmitting unit that transmits at least the automatic control command to the moving body when the information acquiring unit acquires the detection information.

2. 2. The information processing device according to claim 1, When the information acquisition unit acquires the detection information, the transmission unit a first process of transmitting the automatic control command to the moving body without transmitting the manual control command to the moving body; an information processing device that executes one of the following processes: a first process of transmitting to the moving body the manual control command, the automatic control command, and additional information for instructing the moving body to use the automatic control command in priority over the manual control command.

3. A mobile body capable of moving by unmanned operation, A control device for controlling a traveling operation of the moving body, When the control device receives both a manual control command and an automatic control command, the control device uses the automatic control command in priority to the manual control command; the manual control command is a command that specifies the traveling operation of the moving body and is generated in response to an operation by an operator of an operator input device that is provided at a location different from the moving body, A moving body, wherein the automatic control command is a command generated to cause the moving body to travel without using the manual control command.

4. A mobile body capable of moving by unmanned operation, A control device for controlling a traveling operation of the moving body, When the control device receives a manual control command, an automatic control command, and additional information, the control device uses the automatic control command in priority to the manual control command; the manual control command is a command that specifies the traveling operation of the moving body and is generated in response to an operation by an operator of an operator input device that is provided at a location different from the moving body, the automatic control command is a command generated by an information processing device provided at a location different from the moving body in order to cause the moving body to travel without using the manual control command, A mobile body, wherein the auxiliary information is information for the information processing device to instruct the mobile body to use the automatic control command in priority over the manual control command.

Images