Control device, remote control device, display device, remote control system, and remote control method

JP7643502B2Active Publication Date: 2025-03-11TOYOTA JIDOSHA KK
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
JP2023180214
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-10-19
Publication Date
2025-03-11
Estimated Expiration
2043-03-30

AI Technical Summary

Benefits of technology

【0006】 (1)本開示の第1形態によれば、制御装置が提供される。車両の動作を制御する制御装置において、前記車両は、前記車両に乗車した運転者が、前記車両に設けられた運転者入力装置を操作することにより、前記車両の走行動作を規定する走行条件を決定して走行する有人運転モードと、前記車両の前記走行条件を前記車両の外部から前記車両が受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔無人運転モードと、を有し、前記遠隔無人運転モードは、前記車両とは異なる場所に設けられた遠隔手動制御装置に接続されたオペレータ入力装置をオペレータが操作することにより生成された前記走行条件を受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔手動運転モードと、前記車両とは異なる場所に設けられた遠隔自動制御装置によって生成された前記走行条件を受信することにより、前記運転者入力装置と前記オペレータ入力装置とのいずれも操作されることなく走行する遠隔自動運転モードと、を有し、前記制御装置は、予め定められた第1切替条件が満たされた場合に、前記車両の運転モードを前記遠隔自動運転モードから前記遠隔手動運転モードに切り替える切替実行部を備え、前記第1切替条件は、前記遠隔自動運転モードによる走行中おいて、(1a)前記車両の状態が異常状態となった第1状態と、(1b)前記車両の周囲に予め設定された周辺領域に移動物体が侵入した第2状態と、(1c)前記第1状態とは異なる第3状態であって、前記遠隔自動運転モードによる走行に支障が発生した第3状態と、の少なくともいずれかの状態となった場合を条件として含む。この形態によれば、遠隔自動運転モードによる走行中において、不具合の一例である第1状態と第2状態と第3状態との少なくともいずれかの状態となった場合に、車両の運転モードを、遠隔自動運転モードから遠隔手動運転モードに切り替えることができる。このようにすると、遠隔自動運転モードで第1場所から第2場所に車両を移動させる過程において、第1状態と第2状態と第3状態との少なくともいずれかの状態になることで、車両が停止した場合であっても、オペレータ入力装置を介したオペレータの遠隔操作により、車両を移動させることができる。これにより、車両の停止位置とオペレータの待機位置との距離が大きい場合であっても、オペレータが停止した車両に乗車することなく、車両を移動させることができる。そのため、車両の搬送が滞ることを抑制することができる。 (2)上記形態であって、さらに、前記切替実行部によって前記運転モードが前記遠隔自動運転モードから前記遠隔手動運転モードに切り替えられた場合に、前記車両に搭載されている車載センサによって取得した車載センサ情報と、前記車両とは異なる場所に設けられた検知装置によって取得した検知情報と、の少なくとも一方を前記遠隔手動制御装置に送信する情報送信部を備え、前記車載センサ情報は、前記車両の位置を示す車両位置情報と、前記周辺領域の状態を示す周辺情報と、の少なくとも一方の情報を含み、前記検知情報は、前記車両の外部から取得された情報であって、前記車両の位置を示す車両位置情報と、前記周辺領域の状態を示す周辺情報と、の少なくとも一方の情報を含んでもよい。この形態によれば、車両の運転モードが遠隔自動運転モードから遠隔手動運転モードに切り替えられた場合に、車載センサ情報と検知情報との少なくとも一方を遠隔手動制御装置に送信することができる。これにより、遠隔手動制御装置において、車両の運転モードを切り替えるか否かを判断したり、車両の走行条件を示す運転制御値を作成したりすることができる。 (3)上記形態であって、前記車載センサは、少なくとも前記周辺領域の前記状態を検知できるセンサであってもよい。この形態によれば、車載センサによって検知した車両の周辺領域の状態を遠隔手動制御装置に送信することができる。これにより、車載センサの情報を用いて、遠隔手動制御装置が、車両の運転モードを切り替えるか否かを判断したり、車両の走行条件を示す運転制御値を作成したりすることができる。 (4)上記形態であって、前記車載センサは、カメラと、ライダと、レーダと、の少なくともいずれかであってもよい。この形態によれば、カメラとライダとレーダとの少なくともいずれかによって取得された情報を遠隔手動制御装置に送信することができる。これにより、カメラとライダとレーダとの少なくともいずれかによって取得された情報を用いて、遠隔手動制御装置が、車両の運転モードを切り替えるか否かを判断したり、車両の走行条件を示す運転制御値を作成したりすることができる。 (5)上記形態であって、さらに、前記切替実行部によって前記運転モードが前記遠隔自動運転モードから前記遠隔手動運転モードに切り替えられた場合に、前記車両が前記遠隔手動運転モードにより走行していることを外部に報知する報知制御部を備えてもよい。この形態によれば、車両の運転モードが遠隔自動運転モードから遠隔手動運転モードに切り替えられた場合に、遠隔手動運転モードにより走行していることを車両の外部に報知することができる。これにより、遠隔手動運転モードにより走行している車両と、遠隔自動運転モード等の他の運転モードにより走行している車両と、を車両の外部から容易に区別することができる。 (6)上記形態であって、予め定められた第2切替条件が満たされた場合に、前記切替実行部は、前記運転モードを前記遠隔手動運転モードから前記遠隔自動運転モードに切り替え、前記第2切替条件は、前記第1状態と前記第2状態と前記第3状態とのいずれの状態にも該当しない場合を条件として含んでもよい。この形態によれば、第1状態と第2状態と第3状態とのいずれの状態にも該当しない場合に、車両の運転モードを、遠隔自動運転モードから遠隔手動運転モードに切り替えることができる。これにより、車両の移動に要するオペレータの人数や作業負荷を低減することができる。 (7)本開示の第2形態によれば、遠隔操作装置が提供される。車両の動作を遠隔操作する遠隔操作装置は、遠隔手動制御装置と、前記遠隔手動制御装置に接続されたオペレータ入力装置と、を備え、前記車両は、前記車両に乗車した運転者が、前記車両に設けられた運転者入力装置を操作することにより、前記車両の走行動作を規定する走行条件を決定して走行する有人運転モードと、前記車両の前記走行条件を前記車両の外部から前記車両が受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔無人運転モードと、を有し、前記遠隔無人運転モードは、前記車両とは異なる場所に設けられた前記オペレータ入力装置をオペレータが操作することにより生成された前記走行条件を受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔手動運転モードと、前記車両とは異なる場所に設けられた遠隔自動制御装置によって生成された前記走行条件を受信することにより、前記運転者入力装置と前記オペレータ入力装置とのいずれも操作されることなく走行する遠隔自動運転モードと、を有し、前記遠隔手動制御装置は、(2a)前記車両の状態が異常状態となったことを示す異常情報と、(2b)前記車両の周囲に予め設定された周辺領域に移動物体が侵入したことを示す侵入情報と、(2c)前記遠隔自動運転モードによる走行に支障が発生したことを示す支障情報と、の少なくともいずれかの情報を取得する切替情報取得部と、予め定められた第1切替条件が満たされた場合に、前記車両の運転モードを前記遠隔自動運転モードから前記遠隔手動運転モードに切り替える第1切替指示を前記車両に送信する切替指示部と、を備え、前記第1切替条件は、前記遠隔自動運転モードによる走行中において、(1a)前記車両の状態が異常状態となった第1状態と、(1b)前記周辺領域に前記移動物体が侵入した第2状態と、(1c)前記第1状態とは異なる第3状態であって、前記遠隔自動運転モードによる走行に支障が発生した第3状態と、の少なくともいずれかの状態となった場合を条件として含む。この形態によれば、遠隔自動運転モードによる走行中において、不具合の一例である第1状態と第2状態と第3状態との少なくともいずれかの状態となった場合に、車両の運転モードを、遠隔自動運転モードから遠隔手動運転モードに切り替えることができる。このようにすると、遠隔自動運転モードで第1場所から第2場所に車両を移動させる過程において、第1状態と第2状態と第3状態との少なくともいずれかの状態になることで、車両が停止した場合であっても、オペレータ入力装置を介したオペレータの遠隔操作により、車両を移動させることができる。これにより、車両の停止位置とオペレータの待機位置との距離が大きい場合であっても、オペレータが停止した車両に乗車することなく、車両を移動させることができる。そのため、車両の搬送が滞ることを抑制することができる。 (8)上記形態であって、前記第1切替条件は、さらに、前記車両の現在地と前記車両に乗車して前記運転者入力装置を操作することが可能な前記オペレータの待機位置との距離が予め定められた距離以上となった場合を条件として含んでもよい。この形態によれば、車両の現在地とオペレータの待機位置との距離が大きい場合であっても、オペレータ入力装置を介したオペレータの遠隔操作により、車両を移動させることができる。これにより、車両の搬送が滞ることをさらに抑制することができる。 (9)上記形態であって、前記第1切替条件は、さらに、前記遠隔手動運転モードにより走行する前記車両に乗車して、前記運転者入力装置を操作することが可能な前記オペレータの人数が予め定められた人数以下となった場合を条件として含んでもよい。この形態によれば、車両にオペレータが乗車して運転者入力装置を操作することにより、車両を移動させることが困難な場合であっても、オペレータ入力装置を介したオペレータの遠隔操作により、車両を移動させることができる。これにより、車両の搬送が滞ることをさらに抑制することができる。 (10)上記形態であって、前記第1切替条件が満たされた場合であっても、前記オペレータ入力装置を操作することが可能な前記オペレータの人数が予め定められた人数以下である場合に、前記切替指示部は、前記第1切替指示を前記車両に送信することなく、前記車両を停止させる停止指示を前記車両に送信してもよい。この形態によれば、オペレータ入力装置を介したオペレータの遠隔操作により、車両を移動させることが困難な場合に、車両を安全に停止させることができる。 (11)上記形態であって、前記遠隔手動制御装置は、さらに、前記運転モードが前記遠隔自動運転モードから前記遠隔手動運転モードに切り替えられた場合に、前記車両に搭載されている車載センサによって取得した車載センサ情報と、前記車両とは異なる場所に設けられた検知装置によって取得した検知情報と、の少なくとも一方を取得する情報取得部を備え、前記車載センサ情報は、前記車両の位置を示す車両位置情報と、前記周辺領域の状態を示す周辺情報と、の少なくとも一方の情報を含み、前記検知情報は、前記車両の外部から取得された情報であって、前記車両の位置を示す車両位置情報と、前記周辺領域の状態を示す周辺情報と、の少なくとも一方の情報を含んでもよい。この形態によれば、遠隔手動制御装置は、車両の運転モードが遠隔自動運転モードから遠隔手動運転モードに切り替えられた場合に、車載センサ情報と検知情報との少なくとも一方を取得することができる。これにより、遠隔手動制御装置は、車両の運転モードを切り替えるか否かを判断したり、車両の走行条件を示す運転制御値を作成したりすることができる。 (12)上記形態であって、前記車載センサは、少なくとも前記周辺領域の前記状態を検知できるセンサであってもよい。この形態によれば、遠隔手動制御装置は、車載センサによって検知した車両の周辺領域の状態を取得することができる。これにより、遠隔手動制御装置は、車載センサの情報を用いて、車両の運転モードを切り替えるか否かを判断したり、車両の走行条件を示す運転制御値を作成したりすることができる。 (13)上記形態であって、前記車載センサは、カメラと、ライダと、レーダと、の少なくともいずれかであってもよい。この形態によれば、遠隔手動制御装置は、カメラとライダとレーダとの少なくともいずれかによって取得された情報を取得することができる。これにより、遠隔手動制御装置は、カメラとライダとレーダとの少なくともいずれかによって取得された情報を用いて、車両の運転モードを切り替えるか否かを判断したり、車両の走行条件を示す運転制御値を作成したりすることができる。 (14)上記形態であって、さらに、前記車載センサ情報と前記検知情報との少なくとも一方を前記オペレータに伝達する表示部を有する表示装置を備え、前記遠隔手動制御装置は、さらに、前記表示部に表示する表示内容を制御する表示制御部を備えてもよい。この形態によれば、車載センサ情報と検知情報との少なくとも一方を表示部に表示することができる。これにより、車載センサ情報と検知情報との少なくとも一方を、オペレータ入力装置を操作するオペレータに伝達することができる。 (15)上記形態であって、前記表示制御部は、前記車載センサ情報と前記検知情報とのうち、前記表示部に表示させる情報を前記オペレータが指定するための情報指定画面を前記表示部に表示させてもよい。この形態によれば、車載センサ情報と検知情報とのうち、表示部に表示させる情報をオペレータが指定するための情報指定画面を表示部に表示させることができる。これにより、オペレータ入力装置を操作するオペレータは、表示部に表示させる情報を容易に指定することができる。 (16)上記形態であって、前記車載センサと前記検知装置との少なくとも一方は、カメラであり、前記検知情報は、前記カメラによって撮像された撮像情報と、前記周辺領域に侵入した前記移動物体の位置を示す侵入位置情報を含み、前記表示制御部は、前記撮像情報上に前記侵入位置情報を重畳した情報を前記表示部に表示させてもよい。この形態によれば、撮像情報上に侵入位置情報を重畳した情報を表示部に表示させることができる。これにより、オペレータは、車両の周辺領域に侵入した移動物体の存在や位置を容易に把握することができる。 (17)上記形態であって、前記遠隔手動制御装置は、さらに、前記運転モードが前記遠隔自動運転モードから前記遠隔手動運転モードに切り替えられた場合に、前記車両の周囲に設定される侵入禁止領域であって、前記移動物体の侵入を禁止する侵入禁止領域の範囲が、前記遠隔自動運転モードにおける前記侵入禁止領域の範囲よりも大きくなるように前記侵入禁止領域を設定する領域設定部を備えてもよい。この形態によれば、遠隔手動運転モードにおける侵入禁止領域の範囲を、遠隔自動運転モードにおける侵入禁止領域の範囲よりも大きく設定することができる。これにより、遠隔手動運転モードによって、より安全に車両を移動させることができる。 (18)上記形態であって、前記遠隔手動制御装置は、さらに、前記運転モードが前記遠隔自動運転モードから前記遠隔手動運転モードに切り替えられた場合に、前記車両が前記遠隔手動運転モードにより走行していることを外部に報知する報知制御部を備えてもよい。この形態によれば、車両の運転モードが遠隔自動運転モードから遠隔手動運転モードに切り替えられた場合に、遠隔手動運転モードにより走行していることを車両の外部に報知することができる。これにより、遠隔手動運転モードにより走行している車両と、遠隔自動運転モード等の他の運転モードにより走行している車両と、を車両の外部から容易に区別することができる。 (19)上記形態であって、予め定められた第2切替条件が満たされた場合に、前記運転モードを前記遠隔手動運転モードから前記遠隔自動運転モードに切り替える第2切替指示を前記車両に送信し、前記第2切替条件は、前記第1状態と前記第2状態と前記第3状態とのいずれの状態にも該当しない場合を条件として含んでもよい。この形態によれば、第1状態と第2状態と第3状態とのいずれの状態にも該当しない場合に、車両の運転モードを遠隔手動運転モードから遠隔自動運転モードに切り替える第2切替指示を車両に送信することができる。これにより、車両の運転モードを、遠隔自動運転モードから遠隔手動運転モードに切り替えることができるため、車両の移動に要するオペレータの人数や作業負荷を低減することができる。 (20)上記形態であって、前記第2切替条件は、さらに、前記車両が前記遠隔自動運転モードによる走行を再開する基準となる復帰領域に到達した場合を条件として含んでもよい。この形態によれば、車両の位置や進行方向等を検知しやすい場所に復帰領域を予め配置しておくことで、遠隔自動運転モードによる走行を再開するときに、より正確な車両位置情報を取得することができる。 (21)上記形態であって、前記車両は、前記車両を製造する工場と、前記工場において製造した前記車両を保管する保管場所と、前記工場と前記保管場所とを接続する搬送通路と、の少なくともいずれかを走行してもよい。この形態によれば、工場から搬送通路を通って保管場所まで、遠隔自動運転モードにより車両を移動させる場合に、車両の運転モードを遠隔自動運転モードから遠隔手動運転モードに切り替えることができる。 (22)本開示の第3形態によれば、表示装置が提供される。車両の動作を遠隔操作する遠隔操作装置に接続された表示装置において、前記車両は、前記車両に乗車した運転者が、前記車両に設けられた運転者入力装置を操作することにより、前記車両の走行動作を規定する走行条件を決定して走行する有人運転モードと、前記車両の前記走行条件を前記車両の外部から前記車両が受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔無人運転モードと、を有し、前記遠隔無人運転モードは、前記車両とは異なる場所に設けられたオペレータ入力装置をオペレータが操作することにより生成された前記走行条件を受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔手動運転モードと、前記車両とは異なる場所に設けられた遠隔自動制御装置によって生成された前記走行条件を受信することにより、前記運転者入力装置と前記オペレータ入力装置とのいずれも操作されることなく走行する遠隔自動運転モードと、を有し、前記表示装置は、表示部と、前記表示部に表示する表示内容を制御する表示制御部と、を備え、前記表示制御部は、前記遠隔自動運転モードによる走行中において、(1a)前記車両の状態が異常状態となった第1状態と、(1b)前記車両の周囲に予め設定された周辺領域に移動物体が侵入した第2状態と、(1c)前記第1状態とは異なる第3状態であって、前記遠隔自動運転モードによる走行に支障が発生した第3状態と、の少なくともいずれかの状態となった前記車両に搭載されたカメラである車載カメラによって取得された内部撮像情報と、インフラカメラによって取得された外部撮像情報と、の少なくとも一方を前記表示部に表示させ、前記外部撮像情報は、前記車両を外部から撮像することにより取得した情報であって、前記車両の位置を示す車両位置情報と前記周辺領域の状態を示す周辺情報との少なくとも一方の情報を含む。この形態によれば、内部撮像情報と外部撮像情報との少なくとも一方を表示部に表示することができる。これにより、内部撮像情報と外部撮像情報との少なくとも一方を、オペレータ入力装置を操作するオペレータに伝達することができる。 (23)上記形態であって、前記表示制御部は、前記内部撮像情報と前記外部撮像情報とのうち、前記表示部に表示させる情報を前記オペレータが指定するための情報指定画面を前記表示部に表示させてもよい。この形態によれば、内部撮像情報と外部撮像情報とのうち、表示部に表示させる情報をオペレータが指定するための情報指定画面を表示部に表示させることができる。これにより、オペレータ入力装置を操作するオペレータは、表示部に表示させる情報をより詳細に指定することができる。 (24)本開示の第4形態によれば、遠隔操作システムが提供される。遠隔操作システムは、制御装置と、運転者入力装置を有する車両と、遠隔手動制御装置と、オペレータ入力装置と、を有する遠隔操作装置と、遠隔自動制御装置と、を備え、前記車両は、前記車両に乗車した運転者が、前記運転者入力装置を操作することにより、前記車両の走行動作を規定する走行条件を決定して走行する有人運転モードと、前記車両の前記走行条件を前記車両の外部から前記車両が受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔無人運転モードと、を有し、前記遠隔無人運転モードは、前記車両とは異なる場所に設けられた前記遠隔手動制御装置に接続された前記オペレータ入力装置をオペレータが操作することによって生成された前記走行条件を受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔手動運転モードと、前記車両とは異なる場所に設けられた前記遠隔自動制御装置によって生成された前記走行条件を受信することにより、前記運転者入力装置と前記オペレータ入力装置とのいずれも操作されることなく走行する遠隔自動運転モードと、を有し、前記制御装置は、予め定められた第1切替条件が満たされた場合に、前記車両の運転モードを前記遠隔自動運転モードから前記遠隔手動運転モードに切り替える切替実行部を備え、前記第1切替条件は、前記遠隔自動運転モードによる走行中において、(1a)前記車両の状態が異常状態となった第1状態と、(1b)前記車両の周囲に予め設定された周辺領域に移動物体が侵入した第2状態と、(1c)前記第1状態とは異なる第3状態であって、前記遠隔自動運転モードによる走行に支障が発生した第3状態と、の少なくともいずれかの状態となった場合を条件として含む。この形態によれば、遠隔自動運転モードによる走行中において、不具合の一例である第1状態と第2状態と第3状態との少なくともいずれかの状態となった場合に、車両の運転モードを、遠隔自動運転モードから遠隔手動運転モードに切り替えることができる。このようにすると、遠隔自動運転モードで第1場所から第2場所に車両を移動させる過程において、第1状態と第2状態と第3状態との少なくともいずれかの状態になることで、車両が停止した場合であっても、オペレータ入力装置を介したオペレータの遠隔操作により、車両を移動させることができる。これにより、車両の停止位置とオペレータの待機位置との距離が大きい場合であっても、オペレータが停止した車両に乗車することなく、車両を移動させることができる。そのため、車両の搬送が滞ることを抑制することができる。 (25)本開示の第5形態によれば、遠隔操作方法が提供される。車両の遠隔操作方法において、前記車両は、前記車両に乗車した運転者が、前記車両に設けられた運転者入力装置を操作することにより、前記車両の走行動作を規定する走行条件を決定して走行する有人運転モードと、前記車両の前記走行条件を前記車両の外部から前記車両が受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔無人運転モードと、を有し、前記遠隔無人運転モードは、前記車両とは異なる場所に設けられた遠隔手動制御装置に接続されたオペレータ入力装置をオペレータが操作することにより生成された前記走行条件を受信することにより、前記運転者が前記車両に乗車することなく走行する遠隔手動運転モードと、前記車両とは異なる場所に設けられた遠隔自動制御装置によって生成された前記走行条件を受信することにより、前記運転者入力装置と前記オペレータ入力装置とのいずれも操作されることなく走行する遠隔自動運転モードと、を有し、前記遠隔操作方法は、予め定められた第1切替条件が満たされた場合に、前記車両の運転モードを前記遠隔自動運転モードから前記遠隔手動運転モードに切り替える切替工程を備え、前記第1切替条件は、前記遠隔自動運転モードによる走行中において、(1a)前記車両の状態が異常状態となった第1状態と、(1b)前記車両の周囲に予め設定された周辺領域に移動物体が侵入した第2状態と、(1c)前記第1状態とは異なる第3状態であって、前記遠隔自動運転モードによる走行に支障が発生した第3状態と、の少なくともいずれかの状態となった場合を条件として含む。この形態によれば、遠隔自動運転モードによる走行中において、不具合の一例である第1状態と第2状態と第3状態との少なくともいずれかの状態となった場合に、車両の運転モードを、遠隔自動運転モードから遠隔手動運転モードに切り替えることができる。このようにすると、遠隔自動運転モードで第1場所から第2場所に車両を移動させる過程において、第1状態と第2状態と第3状態との少なくともいずれかの状態になることで、車両が停止した場合であっても、オペレータ入力装置を介したオペレータの遠隔操作により、車両を移動させることができる。これにより、車両の停止位置とオペレータの待機位置との距離が大きい場合であっても、オペレータが停止した車両に乗車することなく、車両を移動させることができる。そのため、車両の搬送が滞ることを抑制することができる。 本開示は、上記の制御装置、遠隔操作装置、表示装置、遠隔操作システム、および、遠隔操作方法以外の種々の形態で実現することが可能である。例えば、制御装置、遠隔操作装置、表示装置、遠隔操作システムの製造方法、制御装置、遠隔操作装置、表示装置、および、遠隔操作システムの制御方法、その制御方法を実現するコンピュータプログラム、そのコンピュータプログラムを記録した一時的でない記録媒体等の形態で実現することができる。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007643502000001
    Figure 0007643502000001
  • Figure 0007643502000002
    Figure 0007643502000002
  • Figure 0007643502000003
    Figure 0007643502000003
Patent Text Reader

Abstract

To provide a technique with which it is possible to suppress vehicle transport from getting stacked up when a vehicle stops during automatic driving by remote operation.SOLUTION: A control device comprises a switching execution unit that, when a predetermined first switching condition is satisfied, switches the driving mode of a vehicle from a remote automatic driving mode to a remote manual driving mode. The first switching condition includes, as a condition, that at least one of the states below is entered during traveling by the remote automatic driving mode: (1a), a first state in which the vehicle state has become an abnormal state; (1b), a second state in which a moving body has entered a surrounding region preliminarily set in the surrounding of the vehicle; and (1c), a third state in which trouble has occurred to traveling by the remote automatic driving mode.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a control device, a remote control device, a display device, a remote control system, and a remote control method. [Background technology]

[0002] 2. Description of the Related Art Vehicles that are automatically driven by remote control are known in the art (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]

[0004] Vehicles that can be driven by remote control may be manufactured in a factory and then stored in a storage location different from the factory. In this case, a method of moving the vehicle from the factory to the storage location by remotely driving the vehicle automatically may be considered. In the process of moving the vehicle from a first location to a second location by remotely driving it automatically, some kind of malfunction may cause the vehicle to stop during remotely driven automatic driving. When the vehicle stops, if the distance between the stopping position of the vehicle and the waiting position of the worker is large, it may take time for the worker to get into the stopped vehicle and move the vehicle. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first aspect of the present disclosure, there is provided a control device for controlling the operation of a vehicle, the vehicle having a manned driving mode in which a driver aboard the vehicle operates a driver input device provided on the vehicle to determine driving conditions that define the driving operation of the vehicle, and a remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle and the vehicle runs without the driver aboard the vehicle, the remote unmanned driving mode being a remote manual driving mode in which the driver aboard the vehicle runs without the driver aboard the vehicle by receiving the driving conditions generated by an operator operating an operator input device connected to a remote manual control device provided at a location different from the vehicle, and a remote automatic control device provided at a location different from the vehicle. and a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions set forth in the remote automatic driving mode, the control device including a switching execution unit that switches the driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, the first switching condition including at least one of the following conditions occurring during traveling in the remote automatic driving mode: (1a) a first state in which the state of the vehicle is abnormal, (1b) a second state in which a moving object has entered a peripheral area set forth in advance around the vehicle, and (1c) a third state different from the first state in which an impediment has occurred to traveling in the remote automatic driving mode. According to this embodiment, the driving mode of the vehicle can be switched from the remote automatic driving mode to the remote manual driving mode when at least one of the first state, the second state, and the third state, which are examples of malfunctions, occurs during traveling in the remote automatic driving mode. In this manner, during the process of moving the vehicle from a first location to a second location in remote automatic driving mode, even if the vehicle is stopped due to being in at least one of the first state, the second state, and the third state, the vehicle can be moved by remote operation by the operator via the operator input device.This allows the vehicle to be moved without the operator having to get into the stopped vehicle even if the distance between the vehicle's stopping position and the operator's waiting position is large, thereby preventing delays in vehicle transportation. (2) The above aspect further includes an information transmission unit that transmits at least one of on-board sensor information acquired by an on-board sensor mounted on the vehicle and detection information acquired by a detection device provided at a location different from the vehicle to the remote manual control device when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode by the switching execution unit, and the on-board sensor information includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area, and the detection information may be information acquired from outside the vehicle and includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area. According to this aspect, when the driving mode of the vehicle is switched from the remote automatic driving mode to the remote manual driving mode, at least one of the on-board sensor information and the detection information can be transmitted to the remote manual control device. This allows the remote manual control device to determine whether to switch the driving mode of the vehicle and to create a driving control value indicating the driving conditions of the vehicle. (3) In the above embodiment, the on-board sensor may be a sensor capable of detecting at least the state of the surrounding area. According to this embodiment, the state of the surrounding area of ​​the vehicle detected by the on-board sensor can be transmitted to the remote manual control device. This allows the remote manual control device to use the information from the on-board sensor to determine whether or not to switch the driving mode of the vehicle and to create a driving control value indicating the driving conditions of the vehicle. (4) In the above aspect, the on-board sensor may be at least one of a camera, a lidar, and a radar. According to this aspect, information acquired by at least one of the camera, the lidar, and the radar can be transmitted to the remote manual control device. This allows the remote manual control device to use the information acquired by at least one of the camera, the lidar, and the radar to determine whether to switch the driving mode of the vehicle or to create a driving control value indicating the driving conditions of the vehicle. (5) The above aspect may further include a notification control unit that notifies an outside party that the vehicle is running in the remote manual driving mode when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode by the switching execution unit. According to this aspect, when the driving mode of the vehicle is switched from the remote automatic driving mode to the remote manual driving mode, it is possible to notify an outside party of the vehicle that the vehicle is running in the remote manual driving mode. This makes it easy to distinguish a vehicle running in the remote manual driving mode from a vehicle running in another driving mode such as the remote automatic driving mode from outside the vehicle. (6) In the above embodiment, when a predetermined second switching condition is satisfied, the switching execution unit may switch the driving mode from the remote manual driving mode to the remote automatic driving mode, and the second switching condition may include a case where none of the first state, the second state, and the third state applies as a condition. According to this embodiment, when none of the first state, the second state, and the third state applies, the driving mode of the vehicle can be switched from the remote automatic driving mode to the remote manual driving mode. This makes it possible to reduce the number of operators and the workload required to move the vehicle. (7) According to a second aspect of the present disclosure, there is provided a remote operation device. The remote operation device for remotely operating the operation of a vehicle includes a remote manual control device and an operator input device connected to the remote manual control device, and the vehicle has a manned operation mode in which a driver aboard the vehicle operates a driver input device provided on the vehicle to determine driving conditions that define the driving operation of the vehicle and drives the vehicle, and a remote unmanned operation mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle and the vehicle drives without the driver aboard the vehicle, and the remote unmanned operation mode includes a remote manual operation mode in which the driver aboard the vehicle drives the vehicle by receiving the driving conditions generated by an operator operating the operator input device provided at a location different from the vehicle, and a remote unmanned operation mode in which the vehicle drives the vehicle without the driver aboard the vehicle by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle, and a remote unmanned operation mode in which the vehicle drives the vehicle without the driver operating either the driver input device or the operator input device. and an automatic driving mode, wherein the remote manual control device is equipped with a switching information acquisition unit that acquires at least one of the following information: (2a) abnormality information indicating that the state of the vehicle has become abnormal; (2b) intrusion information indicating that a moving object has intruded into a predetermined surrounding area around the vehicle; and (2c) interference information indicating that an interference has occurred in driving in the remote automatic driving mode; and a switching instruction unit that transmits to the vehicle a first switching instruction to switch the driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, wherein the first switching condition includes, as a condition, a case in which at least one of the following states occurs during driving in the remote automatic driving mode: (1a) a first state in which the state of the vehicle has become abnormal; (1b) a second state in which the moving object has intruded into the surrounding area; and (1c) a third state different from the first state in which an interference has occurred in driving in the remote automatic driving mode. According to this embodiment, when the vehicle is traveling in the remote automatic driving mode, if at least one of the first state, the second state, or the third state, which are examples of malfunctions, occurs, the vehicle's driving mode can be switched from the remote automatic driving mode to the remote manual driving mode.In this way, even if the vehicle stops in the process of moving the vehicle from the first location to the second location in the remote automatic driving mode by entering at least one of the first state, the second state, and the third state, the vehicle can be moved by remote operation of the operator via the operator input device. As a result, even if the distance between the vehicle's stopping position and the operator's waiting position is large, the vehicle can be moved without the operator getting into the stopped vehicle. Therefore, it is possible to prevent delays in the transportation of the vehicle. (8) In the above embodiment, the first switching condition may further include a condition that the distance between the current location of the vehicle and the standby location of the operator who is in the vehicle and can operate the driver input device is equal to or greater than a predetermined distance. According to this embodiment, even if the distance between the current location of the vehicle and the standby location of the operator is large, the vehicle can be moved by remote operation of the operator via the operator input device. This can further prevent delays in transportation of the vehicle. (9) In the above embodiment, the first switching condition may further include a condition that the number of the operators who are able to operate the driver input device while riding in the vehicle traveling in the remote manual driving mode is equal to or less than a predetermined number. According to this embodiment, even if it is difficult for the operators to ride in the vehicle and operate the driver input device to move the vehicle, the vehicle can be moved by remote operation of the operator via the operator input device. This can further prevent delays in transportation of the vehicle. (10) In the above aspect, even if the first switching condition is satisfied, when the number of the operators who can operate the operator input device is equal to or less than a predetermined number, the switching instruction unit may transmit to the vehicle a stop instruction to stop the vehicle without transmitting the first switching instruction to the vehicle. According to this aspect, when it is difficult to move the vehicle by remote operation of the operator via the operator input device, the vehicle can be stopped safely. (11) In the above embodiment, the remote manual control device further includes an information acquisition unit that acquires at least one of on-board sensor information acquired by an on-board sensor mounted on the vehicle and detection information acquired by a detection device installed in a location different from the vehicle when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode, and the on-board sensor information includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area, and the detection information may be information acquired from outside the vehicle and includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area. According to this embodiment, the remote manual control device can acquire at least one of the on-board sensor information and the detection information when the driving mode of the vehicle is switched from the remote automatic driving mode to the remote manual driving mode. This allows the remote manual control device to determine whether to switch the driving mode of the vehicle and to create a driving control value indicating the driving conditions of the vehicle. (12) In the above aspect, the on-board sensor may be a sensor capable of detecting at least the state of the surrounding area. According to this aspect, the remote manual control device can acquire the state of the surrounding area of ​​the vehicle detected by the on-board sensor. As a result, the remote manual control device can use the information of the on-board sensor to determine whether or not to switch the driving mode of the vehicle and to create a driving control value indicating the driving conditions of the vehicle. (13) In the above aspect, the on-board sensor may be at least one of a camera, a lidar, and a radar. According to this aspect, the remote manual control device can acquire information acquired by at least one of the camera, the lidar, and the radar. As a result, the remote manual control device can use the information acquired by at least one of the camera, the lidar, and the radar to determine whether to switch the driving mode of the vehicle or to create a driving control value indicating the driving conditions of the vehicle. (14) In the above embodiment, the remote manual control device may further include a display device having a display unit that transmits at least one of the on-board sensor information and the detection information to the operator, and the remote manual control device may further include a display control unit that controls display content to be displayed on the display unit. According to this embodiment, at least one of the on-board sensor information and the detection information can be displayed on the display unit. This makes it possible to transmit at least one of the on-board sensor information and the detection information to an operator who operates the operator input device. (15) In the above embodiment, the display control unit may cause the display unit to display an information designation screen for the operator to designate information to be displayed on the display unit from among the on-board sensor information and the detection information. According to this embodiment, the display unit can display an information designation screen for the operator to designate information to be displayed on the display unit from among the on-board sensor information and the detection information. This allows the operator operating the operator input device to easily designate information to be displayed on the display unit. (16) In the above aspect, at least one of the on-board sensor and the detection device may be a camera, the detection information may include image information captured by the camera and intrusion position information indicating the position of the moving object that has intruded into the surrounding area, and the display control unit may cause the display unit to display information in which the intrusion position information is superimposed on the image information. According to this aspect, the display unit may display information in which the intrusion position information is superimposed on the image information. This allows the operator to easily grasp the presence and position of a moving object that has intruded into the surrounding area of ​​the vehicle. (17) In the above aspect, the remote manual control device may further include an area setting unit that sets an intrusion forbidden area around the vehicle when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode, the intrusion forbidden area being set so that the range of the intrusion forbidden area into which the moving object is forbidden is larger than the range of the intrusion forbidden area in the remote automatic driving mode. According to this aspect, the range of the intrusion forbidden area in the remote manual driving mode can be set larger than the range of the intrusion forbidden area in the remote automatic driving mode. This makes it possible to move the vehicle more safely in the remote manual driving mode. (18) In the above embodiment, the remote manual control device may further include a notification control unit that notifies the outside that the vehicle is running in the remote manual driving mode when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode. According to this embodiment, when the driving mode of the vehicle is switched from the remote automatic driving mode to the remote manual driving mode, it is possible to notify the outside of the vehicle that the vehicle is running in the remote manual driving mode. This makes it easy to distinguish from the outside of the vehicle between a vehicle running in the remote manual driving mode and a vehicle running in another driving mode such as the remote automatic driving mode. (19) In the above embodiment, when a predetermined second switching condition is satisfied, a second switching instruction to switch the driving mode from the remote manual driving mode to the remote automatic driving mode may be transmitted to the vehicle, and the second switching condition may include a case where none of the first state, the second state, and the third state is applicable. According to this embodiment, when none of the first state, the second state, and the third state is applicable, a second switching instruction to switch the driving mode of the vehicle from the remote manual driving mode to the remote automatic driving mode can be transmitted to the vehicle. This allows the driving mode of the vehicle to be switched from the remote automatic driving mode to the remote manual driving mode, thereby reducing the number of operators and the workload required to move the vehicle. (20) In the above aspect, the second switching condition may further include a condition in which the vehicle reaches a return area that serves as a criterion for resuming driving in the remote automated driving mode. According to this aspect, by locating the return area in advance in a location where the position, traveling direction, etc. of the vehicle can be easily detected, more accurate vehicle position information can be obtained when resuming driving in the remote automated driving mode. (21) In the above aspect, the vehicle may travel in at least one of a factory where the vehicle is manufactured, a storage location where the vehicle manufactured in the factory is stored, and a transport passage connecting the factory and the storage location. According to this aspect, when the vehicle is moved in a remote automatic driving mode from the factory through the transport passage to the storage location, the driving mode of the vehicle can be switched from the remote automatic driving mode to the remote manual driving mode. (22) According to a third aspect of the present disclosure, there is provided a display device. In the display device connected to a remote operation device that remotely operates the operation of a vehicle, the vehicle has a manned operation mode in which a driver aboard the vehicle operates a driver input device provided on the vehicle to determine driving conditions that define the driving operation of the vehicle, and the vehicle runs in that mode, and a remote unmanned operation mode in which the vehicle runs without the driver aboard the vehicle by receiving the driving conditions of the vehicle from outside the vehicle, and the remote unmanned operation mode includes a remote manual operation mode in which the driver runs without a driver aboard the vehicle by receiving the driving conditions generated by an operator operating an operator input device provided at a location different from the vehicle, and a remote unmanned operation mode in which the vehicle runs without the driver operating either the driver input device or the operator input device by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle. The display device has a remote automatic driving mode, and the display device includes a display unit and a display control unit that controls the display content to be displayed on the display unit, and the display control unit displays at least one of internal image information acquired by an in-vehicle camera, which is a camera mounted on the vehicle, and external image information acquired by an infrastructure camera when the vehicle is in at least one of the following states during driving in the remote automatic driving mode: (1a) a first state in which the state of the vehicle is abnormal; (1b) a second state in which a moving object has entered a peripheral area preset around the vehicle; and (1c) a third state different from the first state and in which driving in the remote automatic driving mode is hindered. The external image information is information acquired by capturing an image of the vehicle from the outside, and includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area. According to this embodiment, at least one of the internal image information and the external image information can be displayed on the display unit. This makes it possible to transmit at least one of the internal image information and the external image information to an operator operating an operator input device. (23) In the above embodiment, the display control unit may cause the display unit to display an information designation screen for the operator to designate which of the internal captured information and the external captured information is to be displayed on the display unit. According to this embodiment, the display unit can display an information designation screen for the operator to designate which of the internal captured information and the external captured information is to be displayed on the display unit. This allows the operator operating the operator input device to designate in more detail the information to be displayed on the display unit. (24) According to a fourth aspect of the present disclosure, there is provided a remote operation system. The remote operation system includes a control device, a vehicle having a driver input device, a remote operation device having a remote manual control device and an operator input device, and a remote automatic control device, and the vehicle has a manned operation mode in which a driver aboard the vehicle operates the driver input device to determine driving conditions that define the driving operation of the vehicle and drives the vehicle, and a remote unmanned operation mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle and the vehicle drives without the driver aboard the vehicle, and the remote unmanned operation mode is a remote manual operation mode in which the driver aboard the vehicle drives the vehicle without the driver aboard the vehicle by receiving the driving conditions generated by an operator operating the operator input device connected to the remote manual control device provided in a location different from the vehicle, and the vehicle and and a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by the remote automatic control device provided at a different location, the control device including a switching execution unit that switches the driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, the first switching condition including at least one of the following conditions occurring during traveling in the remote automatic driving mode: (1a) a first state in which the state of the vehicle becomes abnormal, (1b) a second state in which a moving object enters a peripheral area preset around the vehicle, and (1c) a third state different from the first state in which an obstacle occurs to traveling in the remote automatic driving mode. According to this embodiment, when traveling in the remote automatic driving mode, at least one of the first state, the second state, and the third state, which are examples of a malfunction, occurs, the driving mode of the vehicle can be switched from the remote automatic driving mode to the remote manual driving mode.In this way, even if the vehicle stops in the process of moving the vehicle from the first location to the second location in the remote automatic driving mode by entering at least one of the first state, the second state, and the third state, the vehicle can be moved by remote operation of the operator via the operator input device. As a result, even if the distance between the vehicle's stopping position and the operator's waiting position is large, the vehicle can be moved without the operator getting into the stopped vehicle. Therefore, it is possible to prevent delays in the transportation of the vehicle. (25) According to a fifth aspect of the present disclosure, there is provided a remote control method for a vehicle, the vehicle having a manned driving mode in which a driver aboard the vehicle operates a driver input device provided on the vehicle to determine driving conditions that define the driving operation of the vehicle and driving the vehicle, and a remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle and the vehicle drives without the driver aboard the vehicle, the remote unmanned driving mode being a remote manual driving mode in which the driver drives the vehicle without a driver aboard the vehicle by receiving the driving conditions generated by an operator operating an operator input device connected to a remote manual control device provided at a location different from the vehicle, and a remote unmanned driving mode in which the vehicle receives the driving conditions generated by an operator operating an operator input device connected to a remote automatic control device provided at a location different from the vehicle, and a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving condition, the remote operation method comprising a switching step of switching the driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, the first switching condition including a condition that, during traveling in the remote automatic driving mode, at least one of the following occurs: (1a) a first state in which the state of the vehicle is abnormal, (1b) a second state in which a moving object has entered a peripheral area preset around the vehicle, and (1c) a third state different from the first state in which an obstacle occurs to traveling in the remote automatic driving mode. According to this embodiment, during traveling in the remote automatic driving mode, when at least one of the first state, the second state, and the third state, which are examples of a malfunction, occurs, the driving mode of the vehicle can be switched from the remote automatic driving mode to the remote manual driving mode. In this manner, during the process of moving the vehicle from a first location to a second location in remote automatic driving mode, even if the vehicle is stopped due to being in at least one of the first state, the second state, and the third state, the vehicle can be moved by remote operation by the operator via the operator input device.This allows the vehicle to be moved without the operator having to get into the stopped vehicle even if the distance between the vehicle's stopping position and the operator's waiting position is large, thereby preventing delays in vehicle transportation. The present disclosure can be realized in various forms other than the above-mentioned control device, remote control device, display device, remote control system, and remote control method. For example, the present disclosure can be realized in the form of a control device, a remote control device, a display device, a method for manufacturing a remote control system, a control device, a remote control device, a display device, and a control method for a remote control system, a computer program for realizing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a remote control system. [Diagram 2] FIG. 1 is a diagram showing a schematic configuration of a vehicle. [Diagram 3] FIG. 2 is a block diagram showing a schematic configuration of a vehicle CPU. [Figure 4] FIG. 2 is a block diagram showing a schematic configuration of a remote control device. [Diagram 5] FIG. 4 is a block diagram showing a schematic configuration of a remote CPU. [Figure 6] FIG. 1 is a block diagram showing a schematic configuration of an infrastructure camera serving as a detection device. [Figure 7] FIG. 2 is a block diagram showing a schematic configuration of a camera CPU. [Figure 8] 4 is a flowchart showing a first operation control method. [Figure 9] 6 is a flowchart showing a second operation control method. [Figure 10] 1 is a first flowchart showing a first mode switching method. [Figure 11] 6 is a second flowchart showing the first mode switching method. [Figure 12] 11 is a third flowchart showing the first mode switching method. [Figure 13] 13 is a fourth flowchart showing the first mode switching method. [Figure 14] 5 is a fifth flowchart showing the first mode switching method. [Figure 15] 4 is a flowchart showing a first switching determination method. [Figure 16] 11 is a flowchart showing a second switching determination method. [Figure 17] 13 is a flowchart showing a third switching determination method. [Figure 18] 13 is a flowchart showing a fourth switching determination method. [Figure 19] 11 is a first flowchart showing a second mode switching method. [Figure 20] 11 is a second flowchart showing the second mode switching method. [Figure 21] 11 is a third flowchart showing the second mode switching method. [Figure 22] 10 is a fourth flowchart showing the second mode switching method. [Diagram 23] 13 is a fifth flowchart showing a second mode switching method. [Figure 24] 11 is a first flowchart showing a third mode switching method. [Diagram 25] 11 is a second flowchart showing the third mode switching method. [Figure 26] 11 is a third flowchart showing a third mode switching method. [Figure 27] 4 is a fourth flowchart showing the third mode switching method. [Figure 28] 5 is a fifth flowchart showing the third mode switching method. [Figure 29] 1 is a first flowchart showing a method for returning to automatic driving. [Diagram 30] 5 is a second flowchart showing the method for returning to automatic driving. [Diagram 31] 11 is a third flowchart showing the method for returning to automatic driving. [Diagram 32] 4 is a fourth flowchart showing the method for returning to automatic driving. [Diagram 33] 4 is a flowchart showing a first return determination method. [Diagram 34] 11 is a flowchart showing a second return determination method. [Diagram 35] FIG. 13 is a conceptual diagram showing an example of a return region. [Diagram 36] 4 is a flowchart showing a display control method. [Figure 37] 11 is a flowchart showing a notification start method. [Figure 38] 11 is a flowchart showing a method for stopping notification. [Figure 39] 11 is a flowchart showing a method for setting an intrusion prohibition area. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. Embodiment: A-1. Remote control system configuration: 1 is a diagram showing a schematic configuration of a remote operation system 1. The remote operation system 1 remotely controls the automatic driving of a vehicle 10. The remote operation system 1 includes one or more vehicles 10, a control device 2, and a remote operation device 5 having a remote manual control device 50A and a remote automatic control device 50B. In this embodiment, the control device 2 is a vehicle control device 20 (ECU) mounted on each vehicle 10.

[0009] The vehicle 10 has a manned driving mode and a remote unmanned driving mode. The manned driving mode is a driving mode in which a driver who gets on the vehicle 10 operates a driver input device provided on the vehicle 10 to determine the driving conditions of the vehicle 10 and drive the vehicle 10. The driving conditions here are conditions that specify the driving operation. The driving conditions include, for example, the driving route (course), position, driving speed, acceleration, and wheel steering angle of the vehicle 10 of the vehicle 10. The remote unmanned driving mode is a driving mode in which the vehicle 10 receives the driving conditions of the vehicle 10 from outside the vehicle 10, and the vehicle 10 drives without the driver getting on the vehicle 10. The remote unmanned driving mode has a remote manual driving mode and a remote automatic driving mode. The remote manual driving mode is a driving mode in which the vehicle 10 receives driving conditions generated by an operator operating an operator input device connected to the remote manual control device 50A, and the vehicle 10 drives without the driver getting on the vehicle 10. The remote automatic driving mode is a driving mode in which the vehicle 10 receives driving conditions generated by a remote automatic control device 50B provided at a location different from the vehicle 10, and the vehicle 10 runs without operating either the driver input device or the operator input device. When running in the remote unmanned driving mode, the vehicle 10 runs by controlling the operation of various actuators and the like mounted on the vehicle 10 according to the driving control values ​​received from the remote operation device 5.

[0010] In this embodiment, the vehicle 10 travels from the factory FC through the transport path R to a predetermined parking position (not shown) in a storage location Y (e.g., a yard). The factory FC is a location where the vehicle 10 is manufactured. The transport path R is a running path connecting the factory FC and the storage location Y. The storage location Y is a location where the vehicle 10 manufactured in the factory FC is stored. In FIG. 1, a first vehicle 101 and a second vehicle 102 are illustrated as a plurality of vehicles 10 traveling in a remote automatic driving mode from the factory FC through the transport path R to a predetermined parking position in the storage location Y. Also, FIG. 1 illustrates a case where the second vehicle 102 stops in the storage location Y due to some malfunction while traveling in the remote automatic driving mode. In the following, when it is not necessary to distinguish between the vehicles 101 and 102, they will simply be referred to as "vehicles 10". Also, when it is necessary to distinguish between the vehicle control devices 20 mounted on the vehicles 10, the vehicle control device 20 mounted on the first vehicle 101 will be referred to as the first vehicle control device 201. The vehicle control device 20 mounted on the second vehicle 102 is referred to as a second vehicle control device 202. When there is no need to distinguish between the vehicle control devices 20, they are simply referred to as "vehicle control devices 20."

[0011] In addition, the factory FC and the storage location Y are not limited to being present in one building, or at one site or address, and each location FC, Y may be present across multiple buildings, multiple sites, multiple addresses, etc. For example, the vehicle 10 may travel on a public road, not limited to a private road, to move from the factory FC to the storage location Y that is at an address different from the factory FC. In addition, the vehicle 10 may be a completed vehicle as a product, or a vehicle as a semi-finished product or work in progress. In the following, a completed vehicle as a product and a vehicle as a semi-finished product or work in progress are collectively referred to as "vehicle 10".

[0012] FIG. 2 is a diagram showing a schematic configuration of the vehicle 10. FIG. 2 shows a representative part of the configuration of the vehicle 10. In this embodiment, the vehicle 10 is an electric vehicle that runs by the driving force of a traction motor (not shown) serving as a motor generator. Although not shown in FIG. 2, the vehicle 10 includes a traction motor, a main battery that supplies power to the traction motor, a power converter having an inverter and a converter, a reduction gear, a braking device, auxiliary devices such as lamps and a car navigation device, an auxiliary battery that supplies power to the auxiliary devices, and the like. Note that the vehicle 10 may be, for example, a hybrid vehicle that runs by at least one of the driving force of an engine (not shown) serving as an internal combustion engine and the traction motor, or may be a gasoline vehicle or a diesel vehicle that runs using only an engine as a driving force source. In addition, the vehicle 10 is not limited to a private automobile such as a passenger car, but may be a commercial automobile such as a truck, a bus, or a construction vehicle.

[0013] Furthermore, the vehicle 10 includes a driver input device 24 , a steering device 25 , an external sensor group 26 and an internal sensor group 27 having on-board sensors 260 , 270 , an actuator 29 , and a vehicle control device 20 .

[0014] The driver input device 24 is an input device that is operated by a driver in the vehicle 10 to determine the driving conditions of the vehicle 10. That is, the driver input device 24 is used when the vehicle 10 is driven in a manned driving mode, and is provided inside the vehicle 10. The driver input device 24 includes a steering wheel 241, an accelerator pedal 242, and a brake pedal 243. The steering wheel 241 is an input device for steering the wheels (not shown). The accelerator pedal 242 is an input device for accelerating the vehicle 10. The brake pedal 243 is an input device for decelerating the vehicle 10. Note that the configuration of the driver input device 24 is not limited to this.

[0015] The steering device 25 rotates a steering shaft (not shown) by torque from a steering motor (not shown) according to the driving control value received from the remote operation device 5 so that the steering angle of the steering wheel 241 coincides with the target steering angle of the wheels. As a result, in the remote unmanned driving mode, the steering device 25 automatically steers the wheels without the driver's steering operation. In other words, the steering device 25 is used when traveling in the remote unmanned driving mode.

[0016] The external sensor group 26 includes a plurality of types of first on-vehicle sensors 260. The first on-vehicle sensors 260 are on-vehicle sensors that acquire peripheral information indicating the state of a peripheral area preset around the vehicle 10. In other words, the first on-vehicle sensors 260 are on-vehicle sensors that can detect at least the state of the peripheral area. Each of the first on-vehicle sensors 260 transmits the acquired data to the vehicle control device 20. In this embodiment, the external sensor group 26 includes an on-vehicle camera 261, a radar 262, and a lidar 263 as the first on-vehicle sensors 260.

[0017] The vehicle-mounted camera 261 captures an image of an imaging range RG including at least a part of the surrounding area to obtain internal imaging information. The internal imaging information may be still image data or video data. The internal imaging information may be color data or monochrome data.

[0018] The radar 262 emits search waves (radio waves) into a predetermined search range and receives the waves reflected by objects in the surrounding area, thereby detecting the distance, angle, relative speed, etc., to objects in the surrounding area.

[0019] The lidar 263 detects the distance, angle, shape, etc. of the object in the surrounding area by irradiating a laser light to a predetermined measurement range and detecting the light reflected by the object in the surrounding area. Note that the configuration of the external sensor group 26 is not limited to this.

[0020] The internal sensor group 27 includes a plurality of types of second in-vehicle sensors 270. The second in-vehicle sensors 270 are a sensor group that acquires various physical quantities necessary for controlling the operation of the vehicle 10. That is, the second in-vehicle sensors 270 are in-vehicle sensors that can acquire various physical quantities in order to acquire or calculate state quantities indicating the state of the vehicle 10. In this embodiment, the internal sensor group 27 includes a current sensor 271, a voltage sensor 272, a battery temperature sensor 273, a wheel speed sensor 274, and a steering angle sensor 275 as the second in-vehicle sensors 270. The current sensor 271 measures an actual measurement value of the output current value of the main battery. The voltage sensor 272 measures an actual measurement value of the output voltage value (potential difference) of the main battery. The battery temperature sensor 273 measures the temperature of the main battery. The wheel speed sensor 274 measures the rotation speed of each wheel (hereinafter, wheel speed). The steering angle sensor 275 measures an actual steering angle of each wheel. Each of the second on-board sensors 270 transmits the acquired data to the vehicle control device 20. Note that the configuration of the internal sensor group 27 is not limited to this.

[0021] The vehicle control device 20 includes a vehicle communication unit 21 as a communication unit in the vehicle control device 20, a vehicle memory unit 23 as a memory unit in the vehicle control device 20, and a vehicle CPU 22 as a central processing unit in the vehicle control device 20. The vehicle communication unit 21, the vehicle memory unit 23, and the vehicle CPU 22 are connected to each other via an internal bus and an interface circuit.

[0022] The vehicle communication unit 21 connects the internal devices and external devices to the vehicle control device 20 so that they can communicate with each other. The internal devices are devices mounted on the vehicle 10, such as the main battery, the in-vehicle sensors 260 and 270, and the driver input device 24, and are devices that can communicate with the vehicle control device 20. The external devices are devices provided in a location different from the vehicle 10, such as the remote control device 5. The vehicle communication unit 21 is, for example, a wireless communication device. The vehicle communication unit 21 communicates with the internal devices, for example, by CAN (Controller Area Network) communication. The CAN communication is a communication standard that allows multi-directional transmission or reception. The vehicle communication unit 21 communicates with external devices, such as the remote control device 5, connected to the network N, via access points (not shown) such as the factory FC, the transport passage R, and the storage location Y shown in FIG. 1. The vehicle communication unit 21 may communicate with the vehicle control device 20 of another vehicle 10. The communication method by the vehicle communication unit 21 is not limited to this.

[0023] 2 stores various information including various programs for controlling the operation of the vehicle 10 and abnormality determination information. The abnormality determination information is used by an abnormality detection unit described later. The vehicle storage unit 23 includes, for example, a RAM, a ROM, a hard disk drive (HDD), and the like.

[0024] 3 is a block diagram showing a schematic configuration of the vehicle CPU 22. The vehicle CPU 22 loads various programs stored in the vehicle storage unit 23, thereby functioning as a vehicle identification information acquisition unit 220, a charging rate calculation unit 221, a deterioration degree calculation unit 222, a vehicle speed calculation unit 223, a vehicle state acquisition unit 224, and an abnormality detection unit 225. Furthermore, the vehicle CPU 22 loads various programs stored in the vehicle storage unit 23, thereby functioning as a vehicle information transmission unit 226, a switching execution unit 227, an operation control unit 228, a notification control unit 229, and an operation reception unit 230.

[0025] The vehicle identification information acquisition unit 220 acquires vehicle identification information, which is identification information of the vehicle 10. The vehicle identification information is a unique vehicle ID (identifier) ​​assigned to each vehicle 10 so as not to overlap among the vehicles 10 in order to identify a plurality of vehicles 10. The vehicle identification information may include shipping destination information such as the vehicle model and grade of the vehicle 10 and the destination. The vehicle identification information acquisition unit 220 acquires the vehicle identification information by receiving read information acquired by a vehicle reading unit (not shown) installed in advance on the driving route from the factory FC to the parking position of the storage location Y, for example. The read information is information acquired by the vehicle reading unit optically reading the vehicle identification unit (not shown) attached to the vehicle 10. In this case, the vehicle identification unit is, for example, a one-dimensional code such as a barcode in which the vehicle identification information is recorded, a stack type two-dimensional code, a matrix type two-dimensional code such as a QR code (registered trademark), etc. In this case, the vehicle reading unit is a code reader such as a camera that can optically read the vehicle identification unit. Note that the method of acquiring the vehicle identification information is not limited to this. The vehicle identification information acquisition unit 220 may acquire the vehicle identification information, for example, by reading out vehicle identification information stored in advance in the vehicle storage unit 23. The vehicle identification information acquisition unit 220 may also acquire the vehicle identification information, for example, by accepting input of the vehicle identification information from a user via an input operation unit (not shown). The vehicle identification information acquisition unit 220 appropriately stores the acquired vehicle identification information in the vehicle storage unit 23.

[0026] The charging rate calculation unit 221 calculates the charging rate (SOC: State Of Charge) of the main battery. For example, the charging rate calculation unit 221 calculates the charging rate of the main battery by dividing the integrated value of the output current value acquired by the current sensor 271 by the rated capacity (full charge capacity) of the main battery. Note that the method of calculating the charging rate of the main battery is not limited to this. For example, the charging rate calculation unit 221 may calculate the charging rate of the main battery by acquiring data such as the output voltage value of the main battery acquired by the voltage sensor 272 and comparing it with the SOC database. The SOC database includes, for example, an SOC-OCV curve that indicates the relationship between the charging rate of the main battery and the open circuit voltage value (OCV), temperature characteristic data that indicates the relationship between the charging rate of the main battery and the temperature, and the like. In addition, the charging rate calculation unit 221 may calculate the charging rate of the main battery based on, for example, the driving time or driving distance of the vehicle 10, the elapsed time since the previous charging rate calculation, and the like.

[0027] The deterioration degree calculation unit 222 calculates the deterioration degree (SOH: State Of Health) of the main battery. The deterioration degree calculation unit 222 acquires, for example, the temperature of the main battery acquired by the battery temperature sensor 273, the open circuit voltage of the main battery, and the charging rate of the main battery (hereinafter, battery charging rate) calculated by the charging rate calculation unit 221. Then, the deterioration degree calculation unit 222 calculates the apparent charging rate, for example, using the temperature of the main battery and the open circuit voltage of the main battery. Then, the deterioration degree calculation unit 222 calculates the deterioration degree of the main battery by the following formula (1), for example, using the battery charging rate and the apparent charging rate. Note that the method of calculating the deterioration degree of the main battery is not limited to this. Degradation level = (100 - battery charge rate) ÷ (100 - apparent charge rate) Equation (1)

[0028] The vehicle speed calculation unit 223 calculates the running speed of the vehicle 10 using the output value of the wheel speed sensor 274, i.e., the wheel speed of the vehicle 10 acquired by the wheel speed sensor 274. The vehicle speed calculation unit 223 performs a calculation process such as averaging the wheel speeds of the wheels, and then calculates the running speed of the vehicle 10 based on the wheel speed per unit time. Note that the method of calculating the running speed of the vehicle 10 is not limited to this. Also, at least a part of the functions of the vehicle speed calculation unit 223 may be executed by the wheel speed sensor 274.

[0029] The vehicle state acquisition unit 224 acquires on-board sensor information. The on-board sensor information is information acquired by the first on-board sensor 260. In this embodiment, the on-board sensor information includes internal imaging information acquired by the on-board camera 261 and first surrounding information, which is information acquired by the first on-board sensor 260 and indicates the state of the surrounding area of ​​the vehicle 10. The first surrounding information is, for example, radar information acquired by the radar 262 and lidar information acquired by the lidar 263. Note that, in other embodiments, the on-board sensor information may include vehicle position information indicating the position of the vehicle 10. In this case, the vehicle 10 includes a position information receiving unit such as a GNSS receiver capable of receiving radio waves transmitted from a GNSS satellite as the on-board sensor 260. The vehicle state acquisition unit 224 appropriately stores the acquired on-board sensor information in the vehicle storage unit 23.

[0030] The vehicle state acquisition unit 224 further acquires one or more types of state quantities acquired or calculated from the data acquired by the second vehicle-mounted sensor 270 in association with acquisition timings corresponding to each state quantity. The state quantity is data indicating the state of the vehicle-mounted device, which is an on-vehicle device mounted on the vehicle 10 and includes internal equipment and internal members. The internal members are members mounted on the vehicle 10 and do not communicate with the vehicle control device 20. The internal members are, for example, vibration-damping members such as suspensions including dampers. The state quantity is, for example, the running speed of the vehicle 10, the actual steering angle of the wheels, the output current value, output voltage value, temperature, charging rate, and deterioration degree of the main battery, and the degree of vibration that changes depending on the vibration-damping force of the damper. In this embodiment, the vehicle state acquisition unit 224 acquires multiple types of state quantities multiple times at different timings at predetermined time intervals. Specifically, the vehicle state acquisition unit 224 acquires a first state quantity and multiple second state quantities acquired multiple times at different timings for each type of state quantity. The first state quantity is a state quantity acquired at a timing before the vehicle 10 starts traveling in the remote automatic driving mode. The second state quantity is a state quantity acquired at a timing when the vehicle 10 is traveling in the remote automatic driving mode. The vehicle state acquisition unit 224 appropriately stores the acquired multiple types of state quantities in the vehicle storage unit 23 for each acquisition timing.

[0031] The abnormality detection unit 225 detects an abnormality that has occurred in the vehicle-mounted device by using the state quantity acquired by the vehicle state acquisition unit 224 and the abnormality judgment information previously stored in the vehicle storage unit 23. That is, the abnormality detection unit 225 detects that the state of the vehicle 10 has become an abnormal state by using the state quantity and the abnormality judgment information. In other words, the abnormality detection unit 225 detects that the first state has been entered during traveling in the remote automatic driving mode. The abnormality detection unit 225 detects an abnormal state, for example, by relatively comparing the state quantities acquired for each of the multiple vehicles 10. Specifically, the abnormality detection unit 225 calculates the similarity between the same type of state quantities acquired at the same acquisition timing for each of the multiple vehicles 10. Then, the abnormality detection unit 225 clusters the same type of state quantities acquired at the same acquisition timing for each of the multiple vehicles 10 using the calculated similarity. At this time, the abnormality detection unit 225 classifies the state quantities whose similarity is less than a predetermined threshold into a normal cluster. In response to this, the anomaly detection unit 225 classifies a state quantity whose similarity is equal to or greater than a predetermined threshold as an outlier not included in the normal cluster. Then, the anomaly detection unit 225 detects that the state of the vehicle 10 has become abnormal by determining that an abnormality has occurred in the vehicle-mounted device mounted on the vehicle 10 from which the state quantity classified as an outlier was acquired. When detecting that the state of the vehicle 10 has become abnormal, the anomaly detection unit 225 creates anomaly information indicating that the state of the vehicle 10 is abnormal. The anomaly information may include, for example, an event that is the basis for the anomaly detection unit 225 determining that the state is abnormal, that is, the cause of the anomaly. Note that the method of detecting an abnormal state is not limited to this. For example, the anomaly detection unit 225 may detect an abnormal state by generating time-series data by arranging the same type of state quantities acquired for each of the multiple vehicles 10 in chronological order, and comparing the time-series data relative to each other. Also, for example, the anomaly detection unit 225 may detect an abnormal state by absolute comparison in which the state quantity is compared with a predetermined threshold.

[0032] The vehicle information transmission unit 226 transmits various information to the remote control device 5. For example, the vehicle information transmission unit 226 transmits vehicle side information including on-board sensor information and vehicle state information indicating the state of the vehicle 10 together with vehicle identification information to the remote control device 5. The vehicle state information includes, for example, a plurality of types of state quantities. Also, for example, when the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode by the switching execution unit 227, the vehicle information transmission unit 226 transmits the vehicle side information together with the vehicle identification information to the remote control device 5. Also, for example, the vehicle information transmission unit 226 transmits abnormality information generated by the abnormality detection unit 225 together with the vehicle identification information to the remote control device 5. Note that the vehicle information transmission unit 226 may transmit various information to the detection device 9. Also, when the vehicle information transmission unit 226 transmits various information, it is not essential to transmit the vehicle identification information together with the various information. The vehicle information transmission unit 226 may be configured to transmit various types of information in a manner that allows identification of which vehicle 10 the information relates to.

[0033] The switching execution unit 227 switches the driving mode of the vehicle 10 between the remote manual driving mode and the remote automatic driving mode in response to an instruction transmitted from the remote control device 5. Specifically, the switching execution unit 227 receives a first switching instruction transmitted from the remote control device 5 when a predetermined first switching condition is satisfied. The first switching instruction is an instruction to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode. The first switching condition includes, as a first basic condition, a case where at least one of a first state, a second state, and a third state occurs during traveling in the remote automatic driving mode. The first state is a state where the state of the vehicle 10 becomes an abnormal state. An example of the case where the vehicle 10 becomes the first state is a case where any of the vehicle-mounted devices mounted on the vehicle 10 breaks down. The second state is a state where a moving object has entered the surrounding area of ​​the vehicle 10. An example of the case where the vehicle 10 becomes the second state is a case where an animal has unexpectedly entered the surrounding area of ​​the vehicle 10. The third state is a state different from the first state, and is a state where a problem occurs in driving in the remote automatic driving mode. The third state occurs, for example, when a communication problem occurs between the remote automatic control device 50B and the vehicle control device 20, and the remote automatic control device 50B is unable to transmit a driving control value to the vehicle control device 20. The third state occurs, for example, when the position of the vehicle 10 cannot be identified using the imaging information due to disturbances such as weather conditions such as sunny or rainy, time periods such as day and night, and shadows caused by reflections of sunlight, moonlight, street lights, and indoor lights. When the switching execution unit 227 receives the first switching instruction, it switches the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode. That is, when the switching execution unit 227 receives the first switching instruction, it transfers the operation authority from the remote automatic control device 50B to the remote manual control device 50A. Furthermore, the switching execution unit 227 receives a second switching instruction transmitted from the remote operation device 5 when a predetermined second switching condition is satisfied. The second switching instruction is an instruction to switch the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode. The second switching condition includes, as a second basic condition, a case where none of the first state, the second state, and the third state applies.As a result, when the switching execution unit 227 receives the second switching instruction, it switches the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode. In other words, when the switching execution unit 227 receives the second switching instruction, it transfers the operation authority from the remote manual control device 50A to the remote automatic control device 50B.

[0034] The operation control unit 228 controls the operations of the steering device 25 and the actuator 29 in accordance with the operation control values ​​received from the remote control device 5.

[0035] The notification control unit 229 notifies the outside that the vehicle 10 is traveling in the remote manual driving mode when the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode by the switching execution unit 227. Furthermore, the notification control unit 229 stops notifying the outside when the driving mode of the vehicle 10 is switched from the remote manual driving mode to the remote automatic driving mode by the switching execution unit 227.

[0036] The operation reception unit 230 receives input from the driver input device 24. Furthermore, the operation reception unit 230 creates instruction values ​​for controlling the operation of the actuator 29 and the like, based on the input received from the driver input device 24.

[0037] At least some of the functions of the vehicle CPU 22 may be realized as one function of the remote control device 5 or the detection device 9. For example, the abnormality detection unit 225 may be realized as one function of the remote control device 5. In this case, the abnormality detection unit 225 detects an abnormal state, for example, by using vehicle state information transmitted from the vehicle control device 20 and abnormality judgment information pre-stored in the remote storage unit 53. In addition, for example, the control device 2 may be a detection device control device that is a control device of the detection device 9. In other words, the switching execution unit 227 may be realized as one function of the detection device 9 that is the central processing unit of the camera control device.

[0038] Fig. 4 is a block diagram showing a schematic configuration of the remote operation device 5. In this embodiment, as shown in Fig. 1 and Fig. 4, the remote manual control device 50A and the remote automatic control device 50B are integrally configured as a remote control device 50. Thus, the remote operation device 5 includes an operator input device 57, a display device 55, a speaker 56, and a remote control device 50 that integrates the remote manual control device 50A and the remote automatic control device 50B.

[0039] The operator input device 57 is an input device that an operator operates to determine the driving conditions of the vehicle 10 in the remote manual driving mode. That is, the operator input device 57 is used when driving in the remote manual driving mode, and is provided in a location different from the vehicle 10. The operator input device 57 includes a steering wheel 571, an accelerator pedal 572, and a brake pedal 573. The steering wheel 571 is an input device for steering the vehicle 10 by remote control. The accelerator pedal 572 is an input device for accelerating the vehicle 10 by remote control. The brake pedal 573 is an input device for decelerating the vehicle 10 by remote control. Note that the configuration of the operator input device 57 is not limited to this. The operator input device 57 may include, for example, a lever switch instead of the steering wheel 571. The operator input device 57 may include, for example, a push button switch or a lever switch instead of the accelerator pedal 572 or the brake pedal 573.

[0040] The display device 55 transmits various information to the operator. The display device 55 is disposed, for example, at a position where the operator operating the operator input device 57 can view it. The display device 55 includes a display unit 550 that displays various information. For example, the display unit 550 displays at least one of the on-board sensor information and the detection information and transmits it to the operator. Also, for example, the display unit 550 displays an information designation screen for the operator to designate the information to be displayed on the display unit 550 from the on-board sensor information and the detection information. The display unit 550 is, for example, a liquid crystal display. The detection information is information acquired by the detection device 9. The detection information is information acquired from the outside of the vehicle 10, and includes at least one of vehicle position information indicating the position of the vehicle 10 and second surrounding information indicating the state of the surrounding area of ​​the vehicle 10. In this embodiment, the detection device 9 is an infrastructure camera 90 including an imaging unit capable of imaging the vehicle 10 and the surrounding area from the outside of the vehicle 10, and a camera control device. Therefore, the detection information includes the external imaging information as the second peripheral information, the object information generated using the external imaging information, and the vehicle position information specified from the external imaging information. The external imaging information is information acquired by imaging the vehicle 10 from the outside by an imaging unit provided in the infrastructure camera 90 as the detection device 9. The external imaging information may be still image data or video data. The external imaging information may be color data or monochrome data. The object information includes object basic information indicating the detection result of the position and shape of an object present in the peripheral area of ​​the vehicle 10 included in the external imaging information, and intrusion position information indicating the position of a moving object that has invaded the peripheral area of ​​the vehicle 10. The moving object is an object other than a fixed object fixed in advance to a land or a runway such as a pole or a curb, and is an object that can approach the vehicle 10 by moving. The moving object is, for example, a living thing such as a human being or an animal, another vehicle 10, or an unmanned guided vehicle. The configuration and function of the display device 55 are not limited to this. The display unit 550 may display an information designation screen for allowing the operator to designate information to be displayed on the display unit 550, out of the internally captured information and the externally captured information, for example.

[0041] The speaker 56 plays back audio data acquired together with imaging data by a microphone (not shown) attached to the in-vehicle camera 261 or the infrastructure camera 90 serving as the detection device 9. In other words, the speaker 56 plays back audio data included in the internal imaging information or the external imaging information. Note that the configuration and function of the speaker 56 are not limited to this. The speaker 56 may be configured to emit audio, such as a message for an operator, in response to a command from the remote control device 5, for example.

[0042] The remote control device 50 includes a remote communication unit 51 as a communication unit in the remote control device 50, a remote storage unit 53 as a storage unit in the remote control device 50, and a remote CPU 52 as a central processing unit in the remote control device 50. The remote communication unit 51, the remote storage unit 53, and the remote CPU 52 are connected to each other via an internal bus and an interface circuit.

[0043] The remote communication unit 51 connects the vehicle control device 20 and the detection device 9 to the remote control device 50 so that they can communicate with each other. The remote communication unit 51 is, for example, a wireless communication device. However, the communication method used by the remote communication unit 51 is not limited to this.

[0044] The remote storage unit 53 stores various information including various programs for controlling the operation of the remote operation device 5 and driving route information. The driving route information is information that indicates a target driving route of the vehicle 10 that is driving in the remote automatic driving mode. The remote storage unit 53 includes, for example, a RAM, a ROM, a hard disk drive (HDD), etc.

[0045] 5 is a block diagram showing a schematic configuration of the remote CPU 52. The remote CPU 52 deploys various programs stored in the remote storage unit 53, thereby functioning as a travel information acquisition unit 521, an obstacle detection unit 522, a switching information acquisition unit 523, a switching determination unit 524, and a switching instruction unit 525. Furthermore, the remote CPU 52 deploys various programs stored in the remote storage unit 53, thereby functioning as a control value creation unit 526, a remote transmission unit 527, a display control unit 528, and an area setting unit 529.

[0046] The travel information acquisition unit 521 acquires travel information by receiving information on the travel conditions of the vehicle 10 (hereinafter, travel information) from at least one of the vehicle 10 and the detection device 9. In this embodiment, the travel information acquisition unit 521 acquires vehicle side information, detection information, and travel route information as the travel information. Note that the travel information acquisition unit 521 may acquire other information. For example, the travel information acquisition unit 521 may receive, as travel information for starting travel in the remote automatic driving mode, a preparation completion signal transmitted from the vehicle 10 when preparation for starting travel in the remote automatic driving mode is completed.

[0047] The interference detection unit 522 detects that an interference has occurred in driving in the remote automatic driving mode by using vehicle side information, detection information, and other information (for example, communication status information indicating the communication status between the vehicle 10 and the remote operation device 5). In other words, the interference detection unit 522 detects that the third state has occurred while driving in the remote automatic driving mode.

[0048] The switching information acquisition unit 523 receives at least one of abnormality information, intrusion information, and obstruction information to acquire at least one of abnormality information, intrusion information, and obstruction information. The abnormality information is information indicating that the state of the vehicle 10 has become abnormal. That is, the abnormality information is information indicating a first state. The intrusion information is information indicating that a moving object has intruded into the surrounding area of ​​the vehicle 10. That is, the intrusion information is information indicating a second state. The obstruction information is information indicating that an obstruction has occurred to traveling in the remote automatic driving mode. That is, the obstruction information is information indicating a third state. The switching information acquisition unit 523 may acquire other information.

[0049] The switching determination unit 524 determines whether or not to switch the driving mode of the vehicle 10 between the remote manual driving mode and the remote automatic driving mode. For example, when at least one of the first state, the second state, and the third state, which are examples of a malfunction, occurs during driving in the remote automatic driving mode, the switching determination unit 524 determines to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode. That is, in this case, the first switching condition does not include the first additional condition, but includes the first basic condition. Therefore, in this case, the switching determination unit 524 determines to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode when the first basic condition included in the first switching condition is satisfied. In addition, the switching determination unit 524 determines to switch the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode, for example, when none of the first state, the second state, and the third state occurs during driving in the remote manual driving mode. That is, in this case, the second switching condition includes the second basic condition without including the second additional condition. Therefore, in this case, the switching determination unit 524 determines that the driving mode of the vehicle 10 is switched from the remote manual driving mode to the remote automatic driving mode when the second basic condition included in the second switching condition is satisfied. Note that the method of determining whether to switch the driving mode of the vehicle 10 is not limited to this. The first switching condition may include a predetermined first additional condition in addition to the first basic condition. That is, the switching determination unit 524 may determine to switch from the remote automatic driving mode to the remote manual driving mode when the predetermined first additional condition is satisfied when at least one of the first state, the second state, and the third state is reached during traveling in the remote automatic driving mode. At this time, the first switching condition may include one or more first additional conditions. The second switching condition may include a predetermined second additional condition in addition to the second basic condition. The switching judgment unit 524 may, for example, judge to switch from the remote manual driving mode to the remote automatic driving mode when a state that does not correspond to any of the first state, the second state, and the third state while driving in the remote manual driving mode is reached and a predetermined second additional condition is satisfied.In this case, the second switching condition may include one or more second additional conditions.

[0050] The switching instruction unit 525 transmits to the vehicle 10 any one of a first switching instruction, a second switching instruction, or a stop instruction, depending on the result of the determination by the switching determination unit 524. The stop instruction is an instruction to interrupt and stop the traveling of the vehicle 10. Note that the switching instruction unit 525 may transmit various instructions to the vehicle 10 via the detection device 9. In this case, the switching instruction unit 525 transmits various instructions to the detection device 9. Then, the detection device 9 transmits the various instructions received from the remote operation device 5 to the vehicle 10.

[0051] The control value creation unit 526 uses the driving information acquired by the driving information acquisition unit 521 to create driving control values ​​for controlling the operation of the vehicle 10. For example, the control value creation unit 526 creates driving control values ​​for the vehicle 10 to travel along a target driving route. Also, for example, the control value creation unit 526 creates driving control values ​​for correcting the relative position of the vehicle 10 with respect to the target driving route while analyzing the external imaging information and the internal imaging information.

[0052] The remote transmission unit 527 transmits the driving control value created by the control value creation unit 526 to the vehicle 10. The remote transmission unit 527 may transmit the driving control value to the vehicle 10 via the detection device 9. In this case, the remote transmission unit 527 transmits the driving control value to the detection device 9. Then, the detection device 9 transmits the driving control value received from the remote operation device 5 to the vehicle 10.

[0053] The display control unit 528 controls the contents to be displayed on the display unit 550. Specifically, the display control unit 528 causes the display unit 550 to display, for example, at least one of the on-board sensor information acquired by the on-board sensor 260 mounted on the target vehicle 10 and the detection information acquired by the detection device 9. The target vehicle 10 is a vehicle 10 that has entered at least one of the first state, the second state, and the third state while traveling in the remote automatic driving mode. The target vehicle 10 is, for example, the second vehicle 102 shown in FIG. 1. The display control unit 528 shown in FIG. 5 causes the display unit 550 to display, for example, an information designation screen for an operator to designate information to be displayed on the display unit 550 out of the on-board sensor information and the detection information. Then, the display control unit 528 causes the display unit 550 to display the information designated by the information designation screen out of the on-board sensor information and the detection information. At this time, the display control unit 528 may display information in which the intrusion position information is superimposed on at least one of the internal imaging information included in the in-vehicle sensor information and the external imaging information included in the detection information, on the display unit 550. Note that the control of the display control unit 528 and the display contents of the display unit 550 are not limited to this.

[0054] The area setting unit 529 sets the range of the intrusion forbidden area in accordance with the driving mode of the vehicle 10. The intrusion forbidden area is an area set around the vehicle 10, and is an area into which moving objects are prohibited from entering. For example, when switching from the remote automatic driving mode to the remote manual driving mode, the area setting unit 529 sets the intrusion forbidden area so that the range of the intrusion forbidden area in the remote manual driving mode is larger than the range of the intrusion forbidden area in the remote manual driving mode.

[0055] At least some of the functions of the remote CPU 52 may be realized as one function of the vehicle control device 20 or the detection device 9. For example, the switching determination unit 524 may be realized as one function of the vehicle control device 20. In this case, the vehicle control device 20 may directly acquire detection information from the detection device 9 without going through the remote operation device 5. In addition, for example, the display control unit 528 may be realized as one function of the detection device 9.

[0056] 6 is a block diagram showing a schematic configuration of an infrastructure camera 90 serving as a detection device 9. The detection device 9 is a device that acquires information on the vehicle 10 and the surrounding area from outside the vehicle 10. In this embodiment, as described above, the detection device 9 is an infrastructure camera 90 including an imaging unit 95 and a camera control device 900.

[0057] The imaging unit 95 acquires external imaging information that overlooks the running path such as the transport passage R and the storage place Y and the vehicle 10 traveling on the running path from above. The external imaging information acquired by the imaging unit 95 is used for remote operation of the vehicle 10 by the remote operation device 5, generation of vehicle position information, and detection of moving objects that have entered the surrounding area of ​​the vehicle 10. The installation positions and the number of the detection devices 9 are determined in consideration of the imaging range RG (angle of view) of the imaging unit 95, etc., so that the imaging unit 95 can image the entire running path. Specifically, as shown in FIG. 1, the detection devices 901 and 902 are installed so that the first imaging range RG1, which is the imaging range RG of the imaging unit 95 of the adjacent first detection device 901, and the second imaging range RG2, which is the imaging range RG of the imaging unit 95 of the second detection device 902, overlap with each other. Note that the imaging unit 95 is not limited to information from above the vehicle 10, and may acquire information from the front, rear, side, etc. of the vehicle 10. Furthermore, the detection device 9 may include a sensor other than the imaging unit 95.

[0058] 6, the camera control device 900 includes a camera communication unit 91 as a communication unit of the camera control device 900, a camera storage unit 93 as a storage unit of the camera control device 900, and a camera CPU 92 as a central processing unit of the camera control device 900. The camera communication unit 91, the camera storage unit 93, and the camera CPU 92 are connected to each other via an internal bus and an interface circuit.

[0059] The camera communication unit 91 communicably connects the remote control device 50, the vehicle control device 20, the imaging unit 95, and the camera control device 900. The camera communication unit 91 is, for example, a wireless communication device. Note that the communication method by the camera communication unit 91 is not limited to this.

[0060] The camera memory unit 93 stores various information including various programs that control the operation of the detection device 9. Furthermore, the camera memory unit 93 may store, for example, a vehicle position detection model and an intrusion detection model. The vehicle position detection model is a trained machine learning model used to identify the position (e.g., coordinate values) of the vehicle 10 included in the external image capture information. The intrusion detection model is a trained machine learning model used to detect a moving object that has intruded into the surrounding area of ​​the vehicle 10. For example, a neural network algorithm is used for the vehicle position detection model and the intrusion detection model.

[0061] 7 is a block diagram showing a schematic configuration of the camera CPU 92. The camera CPU 92 deploys various programs stored in the camera storage unit 93, thereby functioning as a camera control unit 921, a camera information acquisition unit 922, a vehicle position information generation unit 923, an object information generation unit 924, and a detection information transmission unit 925. The camera control unit 921 controls the operation of the imaging unit 95.

[0062] The camera information acquisition unit 922 acquires various information. For example, the camera information acquisition unit 922 acquires external imaging information acquired by the imaging unit 95. In addition, for example, the camera information acquisition unit 922 receives and acquires abnormality information or interference information transmitted from the vehicle 10 or the remote control device 5.

[0063] The vehicle position information generating unit 923 generates vehicle position information by performing image analysis on the external imaging information acquired by the camera information acquiring unit 922 and identifying the position of the vehicle 10. For the image analysis by the vehicle position information generating unit 923, an image of the entire vehicle 10 may be used, or an image of a part of the vehicle 10, such as an alignment mark provided on the vehicle 10, may be used.

[0064] The object information generating unit 924 performs image analysis on the external image capturing information acquired by the camera information acquiring unit 922 to generate object information including basic object information and intrusion position information. For example, the object information generating unit 924 detects the position, shape, and the like of an object present in the peripheral area of ​​the vehicle 10. As a result, the object information generating unit 924 generates basic object information. Then, the object information generating unit 924 judges whether the detected object is a fixed object or a moving object. Then, when it is judged that the detected object is a moving object, the object information generating unit 924 generates intrusion information indicating that the moving object has intruded into the peripheral area of ​​the vehicle 10. Then, the object information generating unit 924 specifies the position of the object included in the external image capturing information and judged to be a moving object. As a result, the object information generating unit 924 generates intrusion position information. That is, the object information generating unit 924 detects that the second state has been entered during driving in the remote automatic driving mode.

[0065] The detection information transmission unit 925 transmits detection information including the external imaging information acquired by the camera information acquisition unit 922, the vehicle position information generated by the vehicle position information generation unit 923, and the object information generated by the object information generation unit 924 to the remote control device 5. Note that the information transmitted by the detection information transmission unit 925 is not limited to this. The detection information transmission unit 925 may transmit the detection information directly to the vehicle 10 without going through the remote control device 5. The detection information transmission unit 925 transmits, for example, various information together with device identification information. The device identification information is a unique ID (identifier) ​​assigned to each detection device 9 so as not to overlap among the detection devices 9 in order to identify multiple detection devices 9. In this embodiment, the device ID is a camera ID that identifies multiple infrastructure cameras 90.

[0066] A-2. How to control the vehicle in remote automated driving mode: 8 is a flowchart showing the first operation control step (step S1). The first operation control method (step S1) is a method for controlling the operation of the vehicle 10 in order to start traveling of the vehicle 10 in the remote automatic driving mode.

[0067] In the first operation control method (step S1), when preparations to start traveling in the remote automatic driving mode are completed (step S101: Yes), the vehicle information transmission unit 226 of the vehicle control device 20 executes the following process. In this case, the vehicle information transmission unit 226 transmits a preparation completion signal to the remote operation device 5 (step S102).

[0068] When the driving information acquisition unit 521 of the remote control device 5 receives a preparation completion signal (step S103: Yes), the driving information acquisition unit 521 acquires driving information (step S104). At this time, the driving information includes at least driving route information indicating a target driving route. After step S104, the control value creation unit 526 uses the driving information to create a driving control value (hereinafter, a reference control value) for the vehicle 10 to travel along the target driving route (step S105). After step S105, the remote transmission unit 527 transmits the reference control value created by the control value creation unit 526 to the vehicle 10 (step S106). Note that the reference control value may include only the target driving route, or may include specific driving conditions (e.g., a target steering angle) for realizing the target driving route.

[0069] When the operation control unit 228 of the vehicle control device 20 receives and acquires the reference control value (step S107: Yes), the operation control unit 228 sets the driving route of the vehicle 10 as a target driving route and starts driving in the remote automatic driving mode (step S108). In other words, the operation control unit 228 controls the steering device 25 and the actuator 29 mounted on the vehicle 10 so that the vehicle 10 drives along the target driving route.

[0070] 9 is a flowchart showing the second operation control method (step S2). The second operation control method (step S2) is a method for correcting the driving route of the vehicle 10 that is traveling in the remote automatic driving mode.

[0071] When the predetermined correction timing is reached (step S201: Yes), the camera information acquisition unit 922 of the detection device 9 acquires the external image capture information acquired by the imaging unit 95 (step S202). The correction timing is, for example, a timing when a predetermined time has elapsed since the timing when the vehicle started traveling in the remote automatic driving mode, or a timing when a predetermined time has elapsed since the previous correction timing. After step S202, the vehicle position information generation unit 923 generates vehicle position information using the external image capture information acquired by the camera information acquisition unit 922 (step S203). After step S203, the detection information transmission unit 925 transmits detection information including at least the vehicle position information and the external image capture information to the remote operation device 5 (step S204).

[0072] When the travel information acquisition unit 521 of the remote control device 5 acquires the detection information (step S205: Yes), the control value creation unit 526 executes the following process. In this case, the control value creation unit 526 uses the travel information to create a driving control value (hereinafter, a corrected control value) for correcting the relative position of the vehicle 10 with respect to the target travel route (step S206). Specifically, the control value creation unit 526 calculates the actual travel trajectory of the vehicle 10 by, for example, arranging in chronological order a plurality of pieces of external image information captured by different image capture ranges RG1 and RG2. Then, the control value creation unit 526 compares the target travel route with the actual travel trajectory of the vehicle 10 to calculate the difference of the travel trajectory with respect to the target travel route. Furthermore, the control value creation unit 526 creates a corrected control value for realizing the target travel route while analyzing the external image information so as to reduce the difference between the target travel route and the actual travel trajectory of the vehicle 10. Note that the calculation method of the relative position and travel trajectory of the vehicle 10 is not limited to this. The corrected control value may include only the corrected driving route, or may include specific driving conditions for realizing the corrected driving route. In addition, the driving information acquisition unit 521 may acquire vehicle side information including on-board sensor information and vehicle state information in addition to the detection information. After step S206, the remote transmission unit 527 transmits the corrected control value created by the control value creation unit 526 to the vehicle 10 (step S207).

[0073] When the operation control unit 228 of the vehicle control device 20 receives and acquires the corrected control value (step S208: Yes), the operation control unit 228 corrects the driving route of the vehicle 10 to the corrected driving route in accordance with the corrected control value, and continues driving in the remote automatic driving mode (step S209). In other words, the operation control unit 228 controls the operation of the steering device 25 and the actuator 29 mounted on the vehicle 10 so that the vehicle 10 drives along the corrected driving route. When step S209 is completed, the process returns to step S201, and each step from step S201 to step S209 is repeatedly executed.

[0074] A-3.How to switch operation modes: FIG. 10 is a first flowchart showing the first mode switching process (step S3). FIG. 11 is a second flowchart showing the first mode switching process (step S3). FIG. 12 is a third flowchart showing the first mode switching process (step S3). FIG. 13 is a fourth flowchart showing the first mode switching process (step S3). FIG. 14 is a fifth flowchart showing the first mode switching process (step S3). The first mode switching method (step S3) is a method for switching the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode when the first state occurs during driving in the remote automatic driving mode.

[0075] 10, when the abnormality detection unit 225 of the vehicle 10 detects that the state of the vehicle 10 has become an abnormal state (step S301: Yes), the vehicle information transmission unit 226 executes the following process. That is, when the abnormality detection unit 225 detects the first state, the vehicle information transmission unit 226 transmits vehicle side information including the on-board sensor information and the vehicle state information and the abnormality information together with the vehicle identification information to the remote operation device 5 (step S302).

[0076] 11, when the traveling information acquisition unit 521 of the remote control device 5 receives and acquires abnormality information (step S303: Yes), the remote transmission unit 527 transmits the abnormality information to the detection device 9 (step S304). Note that when the vehicle information transmission unit 226 transmits the abnormality information to the detection device 9 without going through the remote control device 5, step S304 is omitted.

[0077] 12, when the camera information acquisition unit 922 of the detection device 9 acquires abnormality information (step S305: Yes), the camera information acquisition unit 922 acquires external image capture information (step S306). After step S306, the vehicle position information generation unit 923 generates vehicle position information using the external image capture information acquired by the camera information acquisition unit 922 (step S307). After step S307, the detection information transmission unit 925 transmits detection information including at least the external image capture information and the vehicle position information to the remote control device 5 (step S308).

[0078] 13, when the switching determination unit 524 of the remote operation device 5 receives and acquires the vehicle side information and the detection information (step S309: Yes), a switching determination step is executed. In the switching determination step, the switching determination unit 524 determines whether or not to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode, or whether or not to stop the vehicle 10, by, for example, any of the switching determination methods shown in Figs. 15 to 18 described later.

[0079] 15 is a flowchart showing a first switching determination method (step S61) in the switching determination step. The first switching determination method (step S61) is a first method for determining whether or not to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode.

[0080] In the first switching determination method (step S61), first, the switching determination unit 524 determines whether or not the first basic condition is satisfied (step S611). In this embodiment, the switching determination unit 524 checks whether or not information for determining whether or not to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (in the first switching determination method, abnormality information) has been received. If the abnormality information has been received (step S611: Yes), the switching determination unit 524 determines that the first basic condition is satisfied and executes step S612. In this case, the switching determination unit 524 determines that the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode (step S612). On the other hand, if the abnormality information has not been received, the switching determination unit 524 determines that the first basic condition is not satisfied and determines to continue traveling in the remote automatic driving mode (step S613).

[0081] 16 is a flowchart showing a second switching judgment method (step S62) in the switching judgment step. The second switching judgment method (step S62) is a second method for judging whether or not the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode. In the second switching judgment method (step S62), the first switching condition includes a first additional condition in addition to the first basic condition. The first additional condition in the second switching judgment method (step S62) is a condition related to the distance between the current location of the vehicle 10 and the waiting position of an operator who is in the vehicle 10 and can operate the driver input device 24.

[0082] In the second switching determination method (step S62), first, the switching determination unit 524 determines whether or not the first basic condition is satisfied (step S621). In this embodiment, the switching determination unit 524 checks whether or not information for determining whether or not to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (abnormal information in the second switching determination method) has been received.

[0083] If the abnormality information has been received (step S621: Yes), the switching determination unit 524 determines that the first basic condition is satisfied, and executes step S622. In step S622, the switching determination unit 524 checks whether the first additional condition is satisfied. Specifically, the switching determination unit 524 checks whether the distance between the current location of the vehicle 10 and the waiting position of the operator who is in the vehicle 10 and can operate the driver input device 24 is equal to or greater than a predetermined distance threshold. At this time, the switching determination unit 524 calculates the distance between the current location of the vehicle 10 and the waiting position of the operator, for example, by using the waiting position of the operator stored in advance in the remote storage unit 53 and the vehicle position information. Note that the method of calculating the distance between the current location of the vehicle 10 and the waiting position of the operator is not limited to this. For example, the switching determination unit 524 may acquire the distance between the current location of the vehicle 10 and the waiting position of the operator by accepting an input from a user via an input operation unit (not shown) equipped with a mouse, a keyboard, or the like.

[0084] When the distance between the current location of the vehicle 10 and the standby location of the operator who is in the vehicle 10 and can operate the driver input device 24 is equal to or greater than the distance threshold (step S622: Yes), the switching determination unit 524 determines that the first additional condition is satisfied and makes the following determination. In this case, the switching determination unit 524 determines that the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode (step S623). That is, when the first additional condition is satisfied in addition to the first basic condition, the switching determination unit 524 determines that the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode. In this way, even if the distance between the current location of the vehicle 10 and the standby location of the operator is large, the vehicle 10 can be moved by remote operation of the operator via the operator input device 57.

[0085] On the other hand, when the distance between the current location of the vehicle 10 and the standby location of the operator is less than the distance threshold (step S622: No), the switching determination unit 524 determines that the first additional condition is not satisfied and makes the following determination. In this case, the switching determination unit 524 determines to stop the vehicle 10 without switching the driving mode of the vehicle 10 from the remote automatic driving mode (step S624). In other words, even if the first basic condition is satisfied, when the first additional condition is not satisfied, the switching determination unit 524 determines to stop the vehicle 10 without switching the driving mode of the vehicle 10 from the remote automatic driving mode. Even in this way, when the distance between the current location of the vehicle 10 and the standby location of the operator is less than the predetermined distance threshold, the operator waiting at the standby location can get in the target vehicle 10 and operate the driver input device 24 to quickly move the target vehicle 10.

[0086] 17 is a flowchart showing a third switching judgment method (step S63) in the switching judgment step. The second switching judgment method (step S63) is a third method for judging whether or not the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode. In the third switching judgment method (step 63), the first switching condition includes a first additional condition in addition to the first basic condition. The first additional condition in the third switching judgment method (step 63) is a condition related to the number of operators who are able to board the vehicle 10 and operate the driver input device 24 (hereinafter, the first waiting number of operators).

[0087] In the third switching determination method (step 63), first, the switching determination unit 524 determines whether or not the first basic condition is satisfied (step S631). In this embodiment, the switching determination unit 524 checks whether or not information for determining whether or not to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (abnormal information in the third switching determination method) has been received.

[0088] If the abnormality information has been received (step S631: Yes), the switching determination unit 524 determines that the first basic condition is satisfied and executes step S632. In step S632, the switching determination unit 524 checks whether the first additional condition is satisfied. Specifically, the switching determination unit 524 checks whether the first waiting number of people is less than a predetermined first number threshold. At this time, the switching determination unit 524 acquires the first waiting number of people, for example, by input from a user via an input operation unit (not shown). Then, if the first waiting number of people is less than the first number threshold (step S632: Yes), the switching determination unit 524 determines that the first additional condition is satisfied and makes the following determination. In this case, the switching determination unit 524 determines that the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode (step S633). That is, when the first additional condition is satisfied in addition to the first basic condition, the switching determination unit 524 determines to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode. In this way, even if it is difficult for an operator to get in the vehicle 10 and operate the driver input device 24 to move the vehicle 10, the vehicle 10 can be moved quickly by remotely operating the operator via the operator input device 57.

[0089] On the other hand, when the first waiting number of people is equal to or greater than the first number of people threshold (step S632: No), the switching determination unit 524 determines that the first additional condition is not satisfied and executes the following process. In this case, the switching determination unit 524 determines to stop the vehicle 10 without switching the driving mode of the vehicle 10 from the remote automatic driving mode (step S634). In other words, even if the first basic condition is satisfied, when the first additional condition is not satisfied, the switching determination unit 524 determines to stop the vehicle 10 without switching the driving mode of the vehicle 10 from the remote automatic driving mode. Even in this way, when the first waiting number of people is equal to or greater than the first number of people threshold, the waiting operator can get in the target vehicle 10 and operate the driver input device 24 to move the target vehicle 10 quickly. Note that the method of acquiring the first waiting number of people is not limited to this. The switching determination unit 524 may acquire the first waiting number of people by image processing such as detecting the number of operators included in the external image capture information.

[0090] 18 is a flowchart showing a fourth switching judgment method (step S64) in the switching judgment step. The fourth switching judgment method (step S64) is a fourth method for judging whether or not to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode. In the fourth switching judgment method (step S64), the first switching condition includes a first additional condition in addition to the first basic condition. The first additional condition in the fourth switching judgment method (step S64) is a condition related to the number of operators who can operate the operator input device 57 connected to the remote manual control device 50A (hereinafter, the second waiting number of operators).

[0091] In the fourth switching determination method (step S64), first, the switching determination unit 524 determines whether or not the first basic condition is satisfied (step S641). In this embodiment, the switching determination unit 524 checks whether or not information for determining whether or not to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (abnormal information in the fourth switching determination method) has been received.

[0092] If the abnormality information has been received (step S641: Yes), the switching determination unit 524 determines that the first basic condition is satisfied and executes step S642. In step S642, the switching determination unit 524 checks whether the first additional condition is satisfied. Specifically, the switching determination unit 524 checks whether the second waiting number of people is less than a predetermined second number of people threshold. At this time, the switching determination unit 524 acquires the second waiting number of people, for example, by input from a user via an input operation unit (not shown). Then, if the second waiting number of people is less than the second number of people threshold (step S642: Yes), the switching determination unit 524 determines that the first additional condition is satisfied and makes the following determination. In this case, the switching determination unit 524 determines to stop the vehicle 10 without switching the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (step S643). That is, when the first additional condition is satisfied in addition to the first basic condition, the switching determination unit 524 determines to stop the vehicle 10 without switching the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode. In this way, the vehicle 10 can be safely stopped when it is difficult to move the vehicle 10 by remote operation by the operator via the operator input device 57.

[0093] On the other hand, when the second waiting number of people is equal to or greater than the second number of people threshold (step S642: No), the switching determination unit 524 determines that the first additional condition is not satisfied and executes the following process. In this case, the switching determination unit 524 determines to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (step S644). When the second waiting number of people is equal to or greater than the second number of people threshold, that is, even when the first basic condition is satisfied, when the first additional condition is not satisfied, there is a sufficient number of operators who can operate the operator input device 57 waiting. Therefore, in this case, by switching the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode, the vehicle 10 can be moved quickly by remote operation by the operator via the operator input device 57. Note that the method of acquiring the second waiting number of people is not limited to this. For example, the switching determination unit 524 may acquire the second waiting number of people by image processing such as detecting the number of operators included in the external imaging information.

[0094] 13, when the switching determination unit 524 determines in the switching determination step that the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode (step S310: Yes), the switching instruction unit 525 executes the following process. In this case, the switching instruction unit 525 transmits a first switching instruction to the vehicle 10 to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (step S311). On the other hand, when the switching determination unit 524 determines in the switching determination step that the vehicle 10 is to be stopped (step S310: No), the switching instruction unit 525 transmits a stop instruction to the vehicle 10 (step S312).

[0095] As shown in FIG. 14, when the switching execution unit 227 of the vehicle 10 receives the first switching instruction (step S313: Yes), the switching execution unit 227 switches the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (step S314).

[0096] 13, after step S311, the control value creation unit 526 of the remote operation device 5 creates a driving control value (hereinafter, remote manual control value) according to the operation of the operator input device 57 (step S315). Then, the remote transmission unit 527 transmits the remote manual control value created by the control value creation unit 526 to the vehicle 10 (step S316). Note that, either step S313 and step S314 shown in FIG. 14 or step S315 and step S316 shown in FIG. 13 may be executed first, or may be executed simultaneously in parallel.

[0097] 14, when the operation control unit 228 of the vehicle 10 receives a remote manual control value (step S317: Yes), the operation control unit 228 drives the steering device 25 and the actuator 29 mounted on the vehicle 10 in accordance with the received remote manual control value (step S318). On the other hand, when the operation control unit 228 receives a stop instruction (step S319: Yes), it stops the vehicle 10 (step S320).

[0098] Note that the processing contents of the first mode switching method (step S3) are not limited to this. For example, when the abnormality detection unit 225 is realized as one function of the remote control device 5, the remote transmission unit 527 may transmit abnormality information to the vehicle 10 and the detection device 9, respectively. Then, the vehicle information transmission unit 226 of the vehicle 10 may transmit the vehicle side information together with the vehicle identification information to the remote control device 5.

[0099] FIG. 19 is a first flowchart showing the second mode switching method (step S4). FIG. 20 is a second flowchart showing the second mode switching method (step S4). FIG. 21 is a third flowchart showing the second mode switching method (step S4). FIG. 22 is a fourth flowchart showing the second mode switching method (step S4). FIG. 23 is a fifth flowchart showing the second mode switching method (step S4). The second mode switching method (step S4) is a method for switching the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode when the second state occurs during driving in the remote automatic driving mode.

[0100] As shown in FIG. 19, when the object information generation unit 924 of the detection device 9 detects the second state (step S401: Yes), the detection information transmission unit 925 transmits the intrusion information generated by the object information generation unit 924 to the remote control device 5 (step S402).

[0101] 20, when the traveling information acquisition unit 521 of the remote control device 5 receives and acquires the intrusion information (step S403: Yes), the remote transmission unit 527 transmits the intrusion information to the vehicle 10 (step S404). Note that when the detection information transmission unit 925 transmits the intrusion information to the vehicle 10 without going through the remote control device 5, step S404 is omitted.

[0102] 21, when the traveling information acquisition unit 521 of the vehicle 10 receives and acquires intrusion information (step S405: Yes), the vehicle information transmission unit 226 executes the following process. In this case, the traveling information acquisition unit 521 acquires in-vehicle sensor information and vehicle state information (step S406). After step S406, the vehicle information transmission unit 226 transmits vehicle side information including the in-vehicle sensor information and the vehicle state information together with the vehicle identification information to the remote operation device 5 (step S407).

[0103] 19, after step S402, the detection information transmission unit 925 of the detection device 9 transmits the detection information to the remote control device 5 (step S408). Note that any of steps S403 and S404 shown in FIG 20 and steps S405 to S407 shown in FIG 21 and step S408 shown in FIG 19 may be executed first, or may be executed simultaneously in parallel.

[0104] As shown in FIG. 22, when the switching determination unit 524 of the remote operation device 5 receives and acquires the vehicle side information and the detection information (step S409: Yes), a switching determination step as shown in FIG. 15 to FIG. 18 is executed. When the switching determination unit 524 determines in the switching determination step that the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode (step S410: Yes), the switching instruction unit 525 executes the following process. In this case, the switching instruction unit 525 transmits a first switching instruction to the vehicle 10 to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (step S411). On the other hand, when the switching determination unit 524 determines in the switching determination step that the vehicle 10 is to be stopped (step S410: No), the switching instruction unit 525 transmits a stop instruction to the vehicle 10 (step S412).

[0105] As shown in FIG. 23, when the switching execution unit 227 of the vehicle 10 receives the first switching instruction (step S413: Yes), the switching execution unit 227 switches the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (step S414).

[0106] As shown in Fig. 22, after step S411, the control value creation unit 526 of the remote operation device 5 creates a remote manual control value (step S415). Then, the remote transmission unit 527 transmits the remote manual control value created by the control value creation unit 526 to the vehicle 10 (step S416). Note that, either step S413 and step S414 shown in Fig. 23 or step S415 and step S416 shown in Fig. 22 may be executed first, or may be executed simultaneously in parallel.

[0107] 23, when the operation control unit 228 of the vehicle 10 receives a remote manual control value (step S417: Yes), the operation control unit 228 drives the steering device 25 and the actuator 29 mounted on the vehicle 10 in accordance with the received remote manual control value (step S418). On the other hand, when the operation control unit 228 receives a stop instruction (step S419: Yes), it stops the vehicle 10 (step S420).

[0108] Note that the processing content of the second mode switching method (step S4) is not limited to this. For example, when the object information generating unit 924 is realized as one function of the remote control device 5, the remote control device 5 may transmit the intrusion information to the vehicle 10 and the detection device 9. Then, the camera information acquiring unit 922 of the detection device 9 may transmit the detection information to the remote control device 5.

[0109] FIG. 24 is a first flowchart showing the third mode switching method (step S5). FIG. 25 is a second flowchart showing the third mode switching method (step S5). FIG. 26 is a third flowchart showing the third mode switching method (step S5). FIG. 27 is a fourth flowchart showing the third mode switching method (step S5). FIG. 28 is a fifth flowchart showing the third mode switching method (step S5). The third mode switching method (step S5) is a method for switching the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode when the third state occurs during driving in the remote manual driving mode.

[0110] As shown in FIG. 24, when the obstruction detection unit 522 of the remote control device 5 detects the third state (step S501: Yes), the remote transmission unit 527 transmits obstruction information to each of the vehicle 10 and the detection device 9 (step S502).

[0111] 25, when the camera information acquisition unit 922 of the detection device 9 receives and acquires the obstruction information (step S503: Yes), the camera information acquisition unit 922 acquires external image capture information (step S504). After step S504, the vehicle position information generation unit 923 generates vehicle position information using the external image capture information acquired by the camera information acquisition unit 922 (step S505). After step S505, the detection information transmission unit 925 transmits the detection information to the remote control device 5 (step S506).

[0112] As shown in FIG. 26, when the traveling information acquisition unit 521 of the vehicle 10 receives and acquires the obstruction information (step S507: Yes), the vehicle information transmission unit 226 executes the following process. In this case, the traveling information acquisition unit 521 acquires the in-vehicle sensor information and the vehicle state information (step S508). After step S508, the vehicle information transmission unit 226 transmits the vehicle side information including the in-vehicle sensor information and the vehicle state information together with the vehicle side identification information to the remote control device 5 (step S509). Note that, any of the steps from step S503 to step S506 shown in FIG. 25 and the steps from step S507 to step S509 shown in FIG. 26 may be executed first, or may be executed simultaneously in parallel.

[0113] As shown in FIG. 27, when the switching determination unit 524 of the remote operation device 5 receives and acquires the vehicle side information and the detection information (step S510: Yes), a switching determination step as shown in FIG. 15 to FIG. 18 is executed. In the switching determination step, when the switching determination unit 524 determines that the driving mode of the vehicle 10 is to be switched from the remote automatic driving mode to the remote manual driving mode (step S511: Yes), the switching instruction unit 525 executes the following process. In this case, the switching instruction unit 525 transmits a first switching instruction to the vehicle 10 to switch the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (step S512). On the other hand, when the switching determination unit 524 determines that the vehicle 10 is to be stopped (step S511: No) in the switching determination step, the switching instruction unit 525 transmits a stop instruction to the vehicle 10 (step S513).

[0114] As shown in FIG. 28, when the switching execution unit 227 of the vehicle 10 receives the first switching instruction (step S514: Yes), the switching execution unit 227 switches the driving mode of the vehicle 10 from the remote automatic driving mode to the remote manual driving mode (step S515).

[0115] As shown in Fig. 27, after step S512, the control value creation unit 526 of the remote operation device 5 creates a remote manual control value (step S516). Then, the remote transmission unit 527 transmits the remote manual control value created by the control value creation unit 526 to the vehicle 10 (step S517). Note that, any one of steps S514 and S515 shown in Fig. 28 and steps S516 and S517 shown in Fig. 27 may be executed first, or may be executed simultaneously in parallel.

[0116] 28, when the operation control unit 228 of the vehicle 10 receives a remote manual control value (step S518: Yes), the operation control unit 228 drives the steering device 25 and the actuator 29 mounted on the vehicle 10 in accordance with the received remote manual control value (step S519). On the other hand, when the operation control unit 228 receives a stop instruction (step S520: Yes), it stops the vehicle 10 (step S521).

[0117] The process of the third mode switching method (step S5) is not limited to this. For example, when the obstacle detection unit 522 is realized as one function of the vehicle control device 20, the vehicle 10 may transmit the obstacle information to the remote control device 5 and the detection device 9.

[0118] FIG. 29 is a first flowchart showing the method for returning to autonomous driving (step S9). FIG. 30 is a second flowchart showing the method for returning to autonomous driving (step S9). FIG. 31 is a third flowchart showing the method for returning to autonomous driving (step S9). FIG. 32 is a fourth flowchart showing the method for returning to autonomous driving (step S9). The method for returning to autonomous driving (step S9) is a method for switching the driving mode of the vehicle 10 from a remote manual driving mode to a remote autonomous driving mode.

[0119] As shown in FIG. 29, after starting traveling in the remote manual driving mode, if a predetermined transmission time has elapsed since the timing of the previous transmission of detection information (step S901: Yes), the camera information acquisition unit 922 executes the following process. In this case, the camera information acquisition unit 922 acquires external image capture information (step S902). After step S902, the vehicle position information generation unit 923 generates vehicle position information using the external image capture information acquired by the camera information acquisition unit 922 (step S903). After step S903, the object information generation unit 924 detects an object present in the surrounding area of ​​the vehicle 10 using the external image capture information acquired by the camera information acquisition unit 922 (step S904). Then, when the object information generation unit 924 determines that an object exists in the surrounding area, it generates object basic information. Furthermore, when the detected object is a moving object, the object information generation unit 924 generates entry position information. As a result, the object information generation unit 924 generates object information according to the presence or absence of an object in the surrounding area. After step S904, the detection information transmitting unit 925 transmits the detection information to the remote control device 5 (step S905).

[0120] As shown in FIG. 30, after starting traveling in the remote manual driving mode, if a predetermined transmission time has elapsed since the timing of the previous transmission of detection information (step S906: Yes), the traveling information acquisition unit 521 executes the following process. In this case, the traveling information acquisition unit 521 acquires in-vehicle sensor information and vehicle state information (step S907). After step S907, the vehicle information transmission unit 226 transmits vehicle side information including the in-vehicle sensor information and the vehicle state information to the remote operation device 5 (step S908). Note that the steps from step S901 to step S905 shown in FIG. 29 and the steps from step S906 to step S908 shown in FIG. 30 may be executed first, or may be executed simultaneously in parallel.

[0121] 31, when the switching determination unit 524 of the remote operation device 5 receives and acquires the vehicle side information and the detection information (step S909: Yes), a return determination step is executed. In the return determination step, the switching determination unit 524 determines whether or not to switch the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode by, for example, any of the return determination methods shown in Fig. 33 and Fig. 34 described later.

[0122] 33 is a flowchart showing a first return determination method (step S65) in the return determination step. The first return determination method (step S65) is a first method for determining whether or not to switch the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode.

[0123] In the first return determination method (step S65), first, the switching determination unit 524 determines whether or not the second basic condition is satisfied (step S651). In other words, in step S651, the switching determination unit 524 determines whether or not the state does not fall into any of the first state, the second state, and the third state.

[0124] If the second basic condition is satisfied, that is, if the state does not fall into any of the first state, the second state, and the third state (step S651: Yes), the switching determination unit 524 executes step S652. In this case, the switching determination unit 524 determines that the driving mode of the vehicle 10 is to be switched from the remote manual driving mode to the remote automatic driving mode (step S652). That is, in the first return determination method (step S65), if the second basic condition is satisfied, the switching determination unit 524 determines that the driving mode of the vehicle 10 is to be switched from the remote manual driving mode to the remote automatic driving mode. In this way, when the state returns to one in which driving in the remote automatic driving mode can be performed, the vehicle 10 can be moved again in the remote automatic driving mode.

[0125] On the other hand, when the second basic condition is not satisfied, that is, when at least one of the first state, the second state, and the third state is satisfied (step S651: No), the switching determination unit 524 executes step S653. In this case, the switching determination unit 524 determines to continue traveling in the remote manual driving mode (step S653). That is, in the first return determination method (step S65), when the second basic condition is not satisfied, the switching determination unit 524 determines to switch the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode. In this way, it is possible to continue traveling in the remote manual driving mode until the causal state among the first state, the second state, and the third state is eliminated. This allows the vehicle 10 to continue traveling by remote operation of the operator via the operator input device 57 while ensuring safety.

[0126] 34 is a flowchart showing a second return judgment method (step S66) in the return judgment step. The second return judgment method (step S66) is a second method for judging whether or not to switch the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode. In the second return judgment method (step S66), the second switching condition includes a second additional condition in addition to the second basic condition. The second additional condition in the second return judgment method (step S66) is a condition related to the position of the vehicle 10.

[0127] In the second return determination method (step S66), first, the switching determination unit 524 determines whether or not the second basic condition is satisfied (step S661). Then, if the second basic condition is satisfied (step S661: Yes), the switching determination unit 524 determines whether or not the vehicle 10 has reached the return area (step S662). At this time, the switching determination unit 524 determines whether or not the vehicle 10 has reached the return area by using, for example, external imaging information. Also, the switching determination unit 524 may detect that the vehicle 10 has reached the return area by, for example, receiving an input from an operator via an input operation unit (not shown).

[0128] FIG. 35 is a conceptual diagram showing an example of a return area RA. The return area RA is an area that is a reference for resuming traveling in the remote automatic driving mode. The return area RA is arranged at predetermined intervals in a place where the position and traveling direction of the target vehicle 10 are easily detected by the detection device 9. The easily detectable place here is, for example, a place where moving objects are unlikely to enter, a place where it is unlikely to be affected by disturbances such as sunlight, and a place where it is unlikely to be affected by radio interference. That is, the return area RA is installed in a place where vehicle position information can be obtained more accurately when resuming traveling in the remote automatic driving mode. Note that a support post (pole) P may be arranged around the return area RA as a landmark so that the operator who operates the operator input device 57 and the driver who operates the driver input device 24 can easily recognize the return area RA. Also, a division line WL such as a white line indicating the return area RA may be attached to the transport passage R or the storage place Y. For example, the operator operates the operator input device 57 to move the vehicle 10 from the current location of the vehicle 10 to the closest return area RA.

[0129] 34, when the vehicle 10 has reached the return area RA (step S662: Yes), the switching determination unit 524 determines that the second additional condition is satisfied and executes the following process. In this case, the switching determination unit 524 determines to switch the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode (step S663). In this way, more accurate vehicle position information can be obtained when traveling in the remote automatic driving mode is resumed.

[0130] On the other hand, if the vehicle 10 has not reached the return area RA (step S662: No), the switching determination unit 524 determines that the second additional condition is not satisfied and determines to continue traveling in the remote manual driving mode (step S664). In this way, the vehicle 10 can be moved in the remote manual driving mode from the current location to the return area RA where more accurate vehicle position information can be acquired.

[0131] 31, when the switching determination unit 524 determines in the return determination step that the driving mode of the vehicle 10 is to be switched from the remote manual driving mode to the remote automatic driving mode (step S910: Yes), the switching instruction unit 525 executes the following process. In this case, the switching instruction unit 525 transmits to the vehicle 10 a second switching instruction to switch the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode (step S911). Note that, when the switching determination unit 524 determines in the return determination step that the vehicle 10 is to continue traveling in the remote manual driving mode (step S910: No), the automatic driving return method (step S9) is terminated.

[0132] As shown in FIG. 32, when the switching execution unit 227 of the vehicle 10 receives a second switching instruction (step S912: Yes), the switching execution unit 227 switches the driving mode of the vehicle 10 from the remote manual driving mode to the remote automatic driving mode (step S913).

[0133] As shown in Fig. 31, after step S911, the control value creation unit 526 of the remote operation device 5 creates a driving control value (hereinafter, a return control value) for resuming driving in the remote automatic driving mode (step S914). Then, the remote communication unit 51 transmits the return control value created by the control value creation unit 526 to the vehicle 10 (step S915). Note that, either step S912 and step S913 shown in Fig. 32 or step S914 and step S915 shown in Fig. 31 may be executed first, or may be executed simultaneously in parallel.

[0134] As shown in FIG. 32, when the operation control unit 228 of the vehicle 10 receives the return control value (step S916: Yes), the operation control unit 228 drives the steering device 25 and the actuator 29 mounted on the vehicle 10 in accordance with the received return control value (step S917).

[0135] A-4. Display control method: Fig. 36 is a flowchart showing the display control method (step S7). The display control method (step S7) is a control method for providing various information to an operator who operates the operator input device 57 via the display unit 550. The display control method (step S7) is executed, for example, when it is determined as "Yes" in step S310 shown in Fig. 13, step S410 shown in Fig. 22, or step S511 shown in Fig. 27.

[0136] First, the display control unit 528 displays an information designation screen on the display unit 550 for designating information to be displayed on the display unit 550 out of the internal image information acquired by the in-vehicle camera 261 and the external image information acquired by the detection device 9 (step S701). When an input from the operator operating the operator input device 57 is accepted via the information designation screen (step S702: Yes), the display control unit 528 executes the following process. When the information designated via the information designation screen is the external image information (step S703: Yes), the display control unit 528 displays information in which the intrusion position information is superimposed on the external image information on the display unit 550 (step S704). On the other hand, when the information designated via the information designation screen is the internal image information (step S703: No), the display control unit 528 displays the internal image information on the display unit 550 (step S705). Note that the display control unit 528 may display information included in the in-vehicle sensor information and the detection information on the display unit 550 in addition to the above display contents. Furthermore, when the information specified via the information specification screen is internal image capture information (step S703: No), the display control unit 528 may superimpose information acquired by the radar 262 or the lidar 263 mounted on the vehicle 10 on the internal image capture information and display it on the display unit 550. Furthermore, the display unit 550 may display information in which an intrusion prohibition area is superimposed on at least one of the external image capture information and the internal image capture information.

[0137] A-5. Notification control method: Fig. 37 is a flowchart showing the notification start method (step S81). The notification start method (step S81) is a method for notifying that the vehicle 10 is traveling in the remote manual driving mode when the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode by the switching execution unit 227. Therefore, the notification start method (step S81) is executed after, for example, step S318 in Fig. 14, step S418 in Fig. 23, or step S519 in Fig. 28.

[0138] When the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode (step S811: Yes), the notification control unit 229 executes the following process. In this case, the notification control unit 229 transmits a notification start instruction to the notification device 80 (step S812). Then, when the notification start instruction is received (step S813: Yes), the notification device 80 starts notification according to the received notification start instruction (step S814). This notifies an operator or the like outside the vehicle 10 that the vehicle is traveling in the remote manual driving mode.

[0139] At this time, when the notification is made by the notification device 80 (hereinafter, the in-vehicle notification device) mounted on the vehicle 10, the notification control unit 229 transmits a notification start instruction to the vehicle 10 in step S812. Then, the vehicle control device 20 receives the notification start instruction and drives the in-vehicle notification device. The in-vehicle notification device is, for example, a pair of lamps provided at the left and right ends along the vehicle width direction at the front and rear of the vehicle 10. In this case, the vehicle control device 20 notifies the outside that the vehicle is traveling in the remote manual driving mode, for example, by turning on or blinking the lamps as the in-vehicle notification device. Note that the notification mode by the in-vehicle notification device is not limited to this. The in-vehicle notification device may be, for example, a warning device (not shown) connected to a horn (not shown) mounted on the vehicle 10 and equipped with an in-vehicle speaker that emits sound toward the outside of the vehicle 10. In this case, the vehicle control device 20 notifies the outside that the vehicle is traveling in the remote manual driving mode, for example, by reproducing an alarm sound from an in-vehicle speaker of an alarm device serving as an in-vehicle alarm device.

[0140] In addition, when the notification is made by the notification device 80 (hereinafter, the vehicle exterior notification device) provided at a location different from the vehicle 10, the notification control unit 229 transmits a notification start instruction to the vehicle exterior notification device in step S812. The vehicle exterior notification device is, for example, an outdoor monitor having a display, which is installed in the transport passage R or the storage location Y. In this case, the outdoor monitor as the vehicle exterior notification device notifies the outside that the vehicle is traveling in the remote manual driving mode, for example, by displaying the vehicle identification information and a notification display indicating that the vehicle is traveling in the remote manual driving mode on the display. Note that the notification mode by the vehicle exterior notification device is not limited to this. The vehicle exterior notification device may be, for example, a projection device (not shown) capable of projecting an image or the like onto the vehicle 10. In this case, the projection device as the vehicle exterior notification device notifies the outside that the vehicle is traveling in the remote manual driving mode, for example, by projecting light of a predetermined color tone onto the outer surface (for example, the body) of the vehicle 10 traveling in the remote manual driving mode. The projection device as an external notification device notifies the outside that the vehicle is traveling in the remote manual driving mode by projecting an image of a human figure imitating an operator inside (cabin) of the vehicle 10. The notification control unit 229 may also notify the outside of the intrusion prohibited area by also projecting an image showing the intrusion prohibited area of ​​the vehicle 10 traveling in the remote manual driving mode.

[0141] 38 is a flowchart showing the notification stopping method (step S82). The notification stopping method (step S82) is a method for stopping notification when the driving mode of the vehicle 10 is switched from the remote manual driving mode to the remote automatic driving mode by the switching execution unit 227. Therefore, the notification stopping method (step S82) is executed after, for example, step S917 in FIG.

[0142] When the driving mode of the vehicle 10 is switched from the remote manual driving mode to the remote automatic driving mode (step S821: Yes), the notification control unit 229 transmits a notification stop instruction to the notification device 80 (step S822). Then, when the notification stop instruction is received (step S823: Yes), the notification device 80 stops the notification (step S824).

[0143] A-6.How to set the exclusion zone: 39 is a flowchart showing a method for setting an intrusion-prohibited area (step S85). When the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode (step S851: Yes), the area setting unit 529 executes the following process. In this case, the area setting unit 529 sets the intrusion-prohibited area so that the range of the intrusion-prohibited area in the remote manual driving mode is larger than the range of the intrusion-prohibited area in the remote automatic driving mode (step S852). That is, the area setting unit 529 enlarges the range of the intrusion-prohibited area when traveling in the remote manual driving mode compared to when traveling in the remote automatic driving mode.

[0144] According to the above embodiment, when at least one of the first state, the second state, and the third state occurs during traveling in the remote automatic driving mode, the driving mode of the vehicle 10 can be switched from the remote automatic driving mode to the remote manual driving mode. In this way, even if at least one of the first state, the second state, and the third state occurs during the process of moving the vehicle 10 from the first location to the second location in the remote automatic driving mode and the vehicle 10 stops, the vehicle 10 can be moved by remote operation of the operator. As a result, even if the distance between the stopping position of the vehicle 10 and the waiting position of the operator is large, the vehicle 10 can be moved without the operator getting on the stopped vehicle 10, thereby reducing the possibility of delaying the transportation of the vehicle 10.

[0145] Furthermore, according to the above embodiment, when the condition that caused the switch to the remote manual driving mode is eliminated and the condition does not fall into any of the first, second, and third conditions, the remote manual driving mode can be switched to the remote automatic driving mode. In other words, the vehicle 10 that can travel in the remote automatic driving mode can be moved in the remote manual driving mode. This can reduce the number of operators and the workload.

[0146] Further, according to the above embodiment, when the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode, the vehicle control device 20 as the control device 2 can transmit the in-vehicle sensor information to the remote operation device 5. Furthermore, when the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode, the detection device 9 can transmit the detection information to the remote operation device 5. In this way, when the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode, the remote operation device 5 can acquire the in-vehicle sensor information and the detection information including the vehicle position information and the surrounding information. This allows the remote operation device 5 to determine whether to switch the driving mode of the vehicle 10 and to create a driving control value.

[0147] According to the above embodiment, the vehicle 10 includes the on-board camera 261, the radar 262, and the lidar 263 as the first on-board sensor 260 capable of detecting the state of the surrounding area of ​​the vehicle 10. This makes it possible to obtain internal imaging information, radar information, and lidar information and transmit them to the remote control device 5 or the like.

[0148] Furthermore, according to the above embodiment, at least one of the in-vehicle sensor information and the detection information can be displayed on the display unit 550. This makes it possible to transmit information necessary for operation to the operator input device 57 that operates the operator input device 57.

[0149] Furthermore, according to the above embodiment, an information designation screen for allowing an operator to designate information to be displayed on the display unit 550 out of the in-vehicle sensor information and the detection information can be displayed on the display unit 550. This allows the operator to easily designate the information to be displayed on the display unit 550.

[0150] Furthermore, according to the above embodiment, an information designation screen for allowing an operator to designate which of the internal captured image information and the external captured image information is to be displayed on the display unit 550 can be displayed on the display unit 550. This allows the operator to designate in detail the information to be displayed on the display unit 550.

[0151] Furthermore, according to the above embodiment, information in which intrusion position information is superimposed on external image capture information can be displayed on the display unit 550. In this way, the operator can easily grasp the presence and position of a moving object that has intruded into the surrounding area of ​​the vehicle 10 while operating the operator input device 57. In other words, the possibility that the operator will overlook a moving object that has intruded into the surrounding area of ​​the vehicle 10 can be reduced. This allows the vehicle 10 to be driven more safely in the remote manual driving mode.

[0152] Furthermore, according to the above embodiment, when the driving mode of the vehicle 10 is switched from the remote automatic driving mode to the remote manual driving mode, it is possible to notify the outside that the vehicle 10 is running in the remote manual driving mode. This makes it easy for the outside to distinguish between the vehicle 10 running in the remote automatic driving mode and the vehicle 10 running in the remote manual driving mode.

[0153] According to the above embodiment, when the remote automatic driving mode is switched to the remote manual driving mode, the range of the intrusion forbidden area in the remote manual driving mode can be set to be larger than the range of the intrusion forbidden area in the remote automatic driving mode. In this way, the intrusion forbidden area of ​​the vehicle 10 is notified to the outside, so that, for example, the possibility that a worker or the like will inadvertently intrude into the intrusion forbidden area can be reduced. In addition, by displaying information in which the intrusion forbidden area is superimposed on the image information on the display unit 550, the operator can quickly perform an operation such as slowing down or stopping the vehicle 10 when, for example, a moving object intrudes into the intrusion forbidden area. This allows the vehicle 10 to be moved more safely in the remote manual driving mode.

[0154] Furthermore, according to the above embodiment, vehicle position information can be acquired using external imaging information acquired by the detection device 9. In this way, automatic driving of the vehicle 10 by remote control can be realized without using the first on-board sensor 260 such as the on-board camera 261, the radar 262, or the lidar 263. Note that in order to prevent collisions between vehicles 10 being remotely operated, at least a part of the first on-board sensor 260 may be used auxiliary to acquire vehicle position information of the vehicle 10 traveling in the remote automatic driving mode.

[0155] B. Other embodiments: B-1. Alternative embodiment 1: In the above embodiment, the position of the vehicle 10 is identified by analyzing external imaging information acquired by capturing an image using the imaging unit 95 of the infrastructure camera 90 serving as the detection device 9. In contrast, in other embodiments, the position of the vehicle 10 may be identified using information acquired by a position information receiving unit (not shown) mounted on the vehicle 10. In this case, the position information receiving unit is, for example, a GNSS receiver capable of receiving radio waves transmitted from GNSS satellites.

[0156] B-2. Alternative embodiment 2: In the above embodiment, the remote manual control device 50A and the remote automatic control device 50B are integrally configured as the remote operation device 5. In contrast, in other embodiments, the remote manual control device 50A and the remote automatic control device 50B may be configured separately.

[0157] B-3. ​​Alternative embodiment 3: In the above embodiment, the driving control value created in the remote operation device 5 is transmitted from the remote operation device 5 to the vehicle 10. In contrast, in another embodiment, the driving control value created in the remote operation device 5 may be transmitted to the detection device 9, and then transmitted from the detection device 9 to the vehicle 10. In this way, the vehicle 10 can receive the driving control value from the detection device 9 that is closer to the vehicle 10. This makes it possible to reduce the influence of communication failure. Therefore, it is possible to reduce the possibility that the vehicle 10 running in the remote unmanned driving mode will stop due to a communication failure.

[0158] B-4. Alternative embodiment 4: In another embodiment, the remote operation system 1 may include a plurality of remote manual control devices 50A to remote automatic control devices 50B. In this way, even if a failure or the like occurs in the remote communication unit 51 of any of the remote control devices 50A, 50B in the remote operation system 1, it is possible to obtain driving information, etc., and transmit driving control values, etc., using the remote communication units 51 of the other remote control devices 50A, 50B. This can reduce the possibility that the vehicle 10 traveling in the remote unmanned driving mode will stop due to a communication failure.

[0159] B-5. Alternative embodiment 5: In the above embodiment, the control device 2 is the vehicle control device 20. In contrast, in another embodiment, the function of the control device 2 may be realized as one function of the detection device 9. In this case, the camera CPU 92 includes at least a detection information transmission unit 925, a switching execution unit 227, and a notification control unit 229.

[0160] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0161] 1... remote operation system, 2... control device, 5... remote operation device, 9... detection device, 10... vehicle, 20... vehicle control device, 21... vehicle communication unit, 22... vehicle CPU, 23... vehicle memory unit, 24... driver input device, 25... steering device, 26... external sensor group, 27... internal sensor group, 29... actuator, 50... remote control device, 50A... remote manual control device, 50B... remote automatic control device, 51... remote communication unit, 52... remote CPU, 53... remote memory unit, 55... display device, 56... speaker, 57... operator input device, 80... notification device, 90... infrastructure camera, 91... camera communication unit, 92... camera CPU, 93... camera storage unit, 95... imaging unit, 101... first vehicle, 102... second vehicle, 201... first vehicle control device, 202... second vehicle control device, 220... vehicle identification information acquisition unit, 221... charging rate calculation unit, 222... deterioration degree calculation unit, 223... vehicle speed calculation unit, 224... vehicle state acquisition unit, 225... abnormality detection unit, 226... vehicle information transmission unit, 227... switching execution unit, 228... operation control unit, 229... notification control unit, 230... operation reception unit, 241... steering Wheel, 242...accelerator pedal, 243...brake pedal, 260...first vehicle sensor, 261...vehicle camera, 262...radar, 263...lidar, 270...second vehicle sensor, 271...current sensor, 272...voltage sensor, 273...battery temperature sensor, 274...wheel speed sensor, 275...steering angle sensor, 521...driving information acquisition unit, 522...obstacle detection unit, 523...switching information acquisition unit, 524...switching determination unit, 525...switching instruction unit, 526...control value creation unit, 527...remote transmission unit, 528...display control unit, 529...area Setting unit, 550...display unit, 571...steering wheel, 572...accelerator pedal, 573...brake pedal, 900...camera control device, 901...first detection device, 902...second detection device, 921...camera control unit, 922...camera information acquisition unit, 923...vehicle position information generation unit, 924...object information generation unit, 925...detection information transmission unit, FC...factory, N...network, P...support, R...transport passage, RA...return area, RG...imaging range, RG1...first imaging range, RG2...second imaging range, WL...dividing line, Y...storage location

Claims

1. A control device for controlling an operation of a vehicle, The vehicle is A manned driving mode in which a driver in the vehicle operates a driver input device provided in the vehicle to determine driving conditions that specify the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating an operator input device connected to a remote manual control device provided in a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle, The control device includes a switching execution unit that switches a driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, The first switching condition includes (1b) a second state in which a moving object has entered a predetermined surrounding area around the vehicle; and and an information transmission unit that transmits, when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode by the switching execution unit, at least one of on-vehicle sensor information acquired by an on-vehicle sensor mounted on the vehicle and detection information acquired by a detection device provided at a location different from the vehicle to the remote manual control device, The on-board sensor information includes at least one of vehicle position information indicating a position of the vehicle and surrounding area information indicating a state of the surrounding area, A control device, wherein the detection information is information obtained from outside the vehicle and includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area.

2. A control device for controlling an operation of a vehicle, The vehicle is A manned driving mode in which a driver in the vehicle operates a driver input device provided in the vehicle to determine driving conditions that specify the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating an operator input device connected to a remote manual control device provided in a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle, The control device includes a switching execution unit that switches a driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, The first switching condition includes, during driving in the remote automatic driving mode, (1c) a third state in which an obstacle occurs to driving in the remote automatic driving mode, The third state is a state in which the position of the vehicle cannot be identified using external image information acquired by an infrastructure camera.

3. The control device according to claim 2, The third state is determined based on at least one of weather conditions and a time of day.

4. The control device according to claim 2, The control device, wherein the third state is identified based on a shadow occurrence state caused by light reflection.

5. The control device according to claim 1 , The control device, wherein the on-board sensor is a sensor capable of detecting at least the condition of the surrounding area.

6. The control device according to claim 5, A control device, wherein the on-board sensor is at least one of a camera, a lidar, and a radar.

7. The control device according to claim 1 or 2, further comprising: A control device comprising: a notification control unit that notifies outside the vehicle that the vehicle is traveling in the remote manual driving mode when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode by the switching execution unit.

8. The control device according to claim 1 , When a predetermined second switching condition is satisfied, the switching execution unit switches the operation mode from the remote manual operation mode to the remote automatic operation mode, The control device, wherein the second switching condition includes a case where the second state does not apply.

9. The control device according to claim 2, When a predetermined second switching condition is satisfied, the switching execution unit switches the operation mode from the remote manual operation mode to the remote automatic operation mode, The control device, wherein the second switching condition includes a case where the third state does not apply.

10. A remote control device for remotely controlling an operation of a vehicle, A remote manual control device; an operator input device connected to the remote manual control device; The vehicle is A manned driving mode in which a driver in the vehicle operates a driver input device provided in the vehicle to determine driving conditions that specify the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating the operator input device provided at a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle, The remote manual control device includes: (2b) a switching information acquisition unit that acquires intrusion information indicating that a moving object has intruded into a peripheral area that is preset around the vehicle; a switching instruction unit that transmits a first switching instruction to the vehicle to switch a driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, The first switching condition includes, during traveling in the remote autonomous driving mode, (1b) a second state in which the moving object has entered the surrounding area, an information transmission unit that transmits the first switching instruction to the vehicle by the switching instruction unit, and when the driving mode of the vehicle is switched from the remote automatic driving mode to the remote manual driving mode, transmits at least one of on-vehicle sensor information acquired by an on-vehicle sensor mounted on the vehicle and detection information acquired by a detection device provided at a location different from the vehicle to the remote manual control device, The on-board sensor information includes at least one of vehicle position information indicating a position of the vehicle and surrounding area information indicating a state of the surrounding area, A remote control device, wherein the detection information is information obtained from outside the vehicle and includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area.

11. A remote control device for remotely controlling an operation of a vehicle, A remote manual control device; an operator input device connected to the remote manual control device; The vehicle is A manned driving mode in which a driver in the vehicle operates a driver input device provided in the vehicle to determine driving conditions that specify the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating the operator input device provided at a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle, The remote manual control device includes: (2c) a switching information acquisition unit that acquires interference information indicating that an interference has occurred in driving in the remote automatic driving mode; and a switching instruction unit that transmits a first switching instruction to the vehicle to switch a driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, The first switching condition includes, during driving in the remote automatic driving mode, (1c) a third state in which a problem occurs in driving in the remote automatic driving mode, The third state is a state in which the position of the vehicle cannot be identified using external image information acquired by an infrastructure camera.

12. The remote control device according to claim 10 or 11, The first switching condition further includes a condition where a distance between a current location of the vehicle and a waiting position of the operator who is in the vehicle and can operate the driver input device becomes equal to or greater than a predetermined distance.

13. The remote control device according to claim 10 or 11, The first switching condition further includes a condition that a number of the operators who are in the vehicle and can operate the driver input device is equal to or less than a predetermined number.

14. The remote control device according to claim 10 or 11, A remote control device, wherein even when the first switching condition is satisfied, if the number of operators who can operate the operator input device is equal to or less than a predetermined number, the switching instruction unit transmits a stop instruction to the vehicle to stop the vehicle without transmitting the first switching instruction to the vehicle.

15. The remote control device according to claim 10 or 11, The remote manual control device further includes an information acquisition unit that acquires at least one of on-board sensor information acquired by an on-board sensor mounted on the vehicle and detection information acquired by a detection device provided at a location different from the vehicle when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode, The on-board sensor information includes at least one of vehicle position information indicating a position of the vehicle and surrounding area information indicating a state of a surrounding area preset around the vehicle, A remote control device, wherein the detection information is information obtained from outside the vehicle and includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area.

16. The remote control device according to claim 15, The vehicle-mounted sensor is a sensor capable of detecting at least the condition of the surrounding area.

17. 17. The remote control device according to claim 16, A remote control device, wherein the on-board sensor is at least one of a camera, a lidar, and a radar.

18. The remote control device according to claim 15, further comprising: a display device having a display unit that transmits at least one of the on-board sensor information and the detection information to the operator, The remote manual control device further includes a display control unit that controls the display content to be displayed on the display unit.

19. 20. The remote control device according to claim 18, The display control unit is a remote control device that causes an information designation screen to be displayed on the display unit, which allows the operator to designate information to be displayed on the display unit out of the in-vehicle sensor information and the detection information.

20. 20. The remote control device according to claim 18, At least one of the on-board sensor and the detection device is a camera, the detection information includes image information captured by the camera and intrusion position information indicating a position of a moving object that has intruded into the surrounding area, The display control unit causes the display unit to display information in which the intrusion position information is superimposed on the imaging information.

21. The remote control device according to claim 10 or 11, The remote manual control device further includes an area setting unit that sets an intrusion prevention area around the vehicle when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode, the area setting unit setting the intrusion prevention area so that the range of the intrusion prevention area that prohibits the intrusion of moving objects is larger than the range of the intrusion prevention area in the remote automatic driving mode.

22. The remote control device according to claim 10 or 11, The remote manual control device further includes an alarm control unit that, when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode, notifies outside the vehicle that the vehicle is traveling in the remote manual driving mode.

23. The remote control device according to claim 10, When a predetermined second switching condition is satisfied, a second switching instruction is transmitted to the vehicle to switch the driving mode from the remote manual driving mode to the remote automatic driving mode; The second switching condition includes a case where the second state does not apply.

24. The remote control device according to claim 11, When a predetermined second switching condition is satisfied, a second switching instruction is transmitted to the vehicle to switch the driving mode from the remote manual driving mode to the remote automatic driving mode; The second switching condition includes a condition where the third state does not apply.

25. The remote control device according to claim 23 or 24, A remote control device, wherein the second switching condition further includes a condition in which the vehicle reaches a return area that is a criterion for resuming driving in the remote automatic driving mode.

26. The remote control device according to claim 10 or 11, A remote control device in which the vehicle travels in at least one of a factory where the vehicle is manufactured, a storage location where the vehicle manufactured in the factory is stored, and a transport path connecting the factory and the storage location.

27. A display device connected to a remote control device that remotely controls the operation of a vehicle, The vehicle is A manned driving mode in which a driver in the vehicle operates a driver input device provided in the vehicle to determine driving conditions that specify the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating an operator input device provided in a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle, The display device includes: A display unit; A display control unit that controls the display content to be displayed on the display unit, The display control unit, during driving in the remote autonomous driving mode, (1c) displays at least one of internal imaging information acquired by an in-vehicle camera mounted on the vehicle in a third state in which driving in the remote autonomous driving mode has been impaired, and external imaging information acquired by an infrastructure camera on the display unit; The external image capture information is information acquired by capturing an image of the vehicle from outside, and includes at least one of vehicle position information indicating a position of the vehicle and surrounding information indicating a state of a surrounding area preset around the vehicle, The third state is a state in which the position of the vehicle cannot be identified using the external image capture information.

28. 28. The display device according to claim 27, The display control unit causes an information designation screen to be displayed on the display unit, the information designation screen allowing the operator to designate information to be displayed on the display unit out of the internal captured information and the external captured information.

29. 1. A remote operation system, comprising: A control device; a vehicle having a driver input device; a remote control device having a remote manual control device and an operator input device; A remote automatic control device, The vehicle is A manned driving mode in which a driver in the vehicle operates the driver input device to determine driving conditions that define the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating the operator input device connected to the remote manual control device provided in a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by the remote automatic control device provided at a location different from the vehicle, The control device includes a switching execution unit that switches a driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, The first switching condition includes, during traveling in the remote autonomous driving mode, (1b) a second state in which a moving object has entered a predetermined surrounding area around the vehicle, and an information transmission unit that transmits, when the driving mode is switched from the remote automatic driving mode to the remote manual driving mode by the switching execution unit, at least one of on-vehicle sensor information acquired by an on-vehicle sensor mounted on the vehicle and detection information acquired by a detection device provided at a location different from the vehicle to the remote manual control device, The on-board sensor information includes at least one of vehicle position information indicating a position of the vehicle and surrounding area information indicating a state of the surrounding area, A remote control system, wherein the detection information is information obtained from outside the vehicle and includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area.

30. 1. A remote operation system, comprising: A control device; a vehicle having a driver input device; a remote control device having a remote manual control device and an operator input device; A remote automatic control device, The vehicle is A manned driving mode in which a driver in the vehicle operates the driver input device to determine driving conditions that define the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating the operator input device connected to the remote manual control device provided in a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by the remote automatic control device provided at a location different from the vehicle, The control device includes a switching execution unit that switches a driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied, The first switching condition includes, during driving in the remote automatic driving mode, (1c) a third state in which a problem occurs in driving in the remote automatic driving mode, The third state is a state in which the position of the vehicle cannot be identified using external image information acquired by an infrastructure camera.

31. 1. A method for remotely operating a vehicle executed by one or more processors, comprising: The vehicle is A manned driving mode in which a driver in the vehicle operates a driver input device provided in the vehicle to determine driving conditions that specify the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating an operator input device connected to a remote manual control device provided in a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle, the one or more processors execute a switching step of switching a driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied; The first switching condition includes, during traveling in the remote autonomous driving mode, (1b) a second state in which a moving object has entered a predetermined surrounding area around the vehicle, When the driving mode is switched from the remote automatic driving mode to the remote manual driving mode by the switching step, at least one of on-vehicle sensor information acquired by an on-vehicle sensor mounted on the vehicle and detection information acquired by a detection device provided at a location different from the vehicle is transmitted to the remote manual control device, The on-board sensor information includes at least one of vehicle position information indicating a position of the vehicle and surrounding area information indicating a state of the surrounding area, A remote operation method, wherein the detection information is information obtained from outside the vehicle and includes at least one of vehicle position information indicating the position of the vehicle and surrounding information indicating the state of the surrounding area.

32. 1. A method for remotely operating a vehicle executed by one or more processors, comprising: The vehicle is A manned driving mode in which a driver in the vehicle operates a driver input device provided in the vehicle to determine driving conditions that specify the driving operation of the vehicle and drive the vehicle; A remote unmanned driving mode in which the vehicle receives the driving conditions of the vehicle from outside the vehicle, and the vehicle travels without the driver being in the vehicle; The remote unmanned operation mode includes: a remote manual driving mode in which the vehicle is driven without the driver getting in the vehicle by receiving the driving conditions generated by an operator operating an operator input device connected to a remote manual control device provided in a location different from the vehicle; a remote automatic driving mode in which the vehicle travels without operating either the driver input device or the operator input device by receiving the driving conditions generated by a remote automatic control device provided at a location different from the vehicle, the one or more processors execute a switching step of switching a driving mode of the vehicle from the remote automatic driving mode to the remote manual driving mode when a predetermined first switching condition is satisfied; The first switching condition includes, during driving in the remote automatic driving mode, (1c) a third state in which a problem occurs in driving in the remote automatic driving mode, The third state is a state in which the position of the vehicle cannot be identified using external image information acquired by an infrastructure camera, in the remote control method.

Citation Information

Patent Citations

  • Method for operating a vehicle and method for operating a manufacturing system

    JP2017538619A

  • Remote control method of unmanned driving system and remote control apparatus

    JP2018063615A

  • Management system, management method, and management program

    JP2018142265A

  • Autonomous driving device, system, and method, and remotely piloted vehicle

    JP2021528790A

  • Remote control system of travel testing vehicle

    JP2022084288A