Information processing device, information processing system, information processing method, and vehicle

Abstract

We will improve the technology for switching vehicles from autonomous driving to manual driving. [Solution] The information processing device 30 acquires evaluation information based on measurement data from a sensor 12A provided in a vehicle 10 which has an automatic driving mode and a manual driving mode that allows manual driving by an operator. When switching from the automatic driving mode to the manual driving mode, the control unit 33 sets a response level for manual driving based on the evaluation information and selects an operator to whom manual driving of the vehicle 10 is permitted based on the response level and the operator's skill level.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing system, an information processing method, and a vehicle.

Background Art

[0002] Conventionally, technologies related to automatic control of vehicle driving have been known. For example, Patent Document 1 discloses an automatic driving switching system for a vehicle that can easily determine a driver's driving skill level and determine whether it is possible to switch from automatic driving to manual driving according to the driver's driving skill level.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, there are cases where manual driving by an operator is required according to the situation encountered by an autonomous vehicle. At this time, depending on the situation, it is also assumed that it may be difficult for some operators to handle manual driving. It was necessary to assign an operator who can handle manual driving according to the situation. Therefore, there was room for improvement in the technology for switching a vehicle from automatic driving to manual driving.

[0005] In view of such circumstances, an object of the present disclosure is to improve the technology for switching a vehicle from automatic driving to manual driving.

Means for Solving the Problems

[0006] An information processing device according to one embodiment of the present disclosure acquires evaluation information based on measurement data from sensors provided in a vehicle having an automatic driving mode and a manual driving mode that allows manual driving by an operator, and includes a control unit that, when switching from the automatic driving mode to the manual driving mode, sets a response level for the manual driving based on the evaluation information, and selects an operator to whom the manual driving of the vehicle is permitted based on the response level and the operator's skill level.

[0007] An information processing system according to one embodiment of the present disclosure comprises the above-described information processing device and the vehicle which acquires the measurement data using the sensor and acquires the evaluation information based on the acquired measurement data and provides it to the information processing device.

[0008] An information processing method according to one embodiment of the present disclosure is an information processing method performed by an information processing device, comprising: acquiring evaluation information based on measurement data from sensors provided in a vehicle having an automatic driving mode and a manual driving mode that enables manual driving by an operator; setting a response level for the manual driving based on the evaluation information when switching from the automatic driving mode to the manual driving mode; and selecting an operator to whom the manual driving of the vehicle is permitted based on the response level and the operator's skill level.

[0009] A vehicle according to one embodiment of the present disclosure has an automatic driving mode and a manual driving mode that enables manual driving by an operator, and comprises a sensor and a control unit that acquires evaluation information based on measurement data from the sensor, wherein when switching from the automatic driving mode to the manual driving mode, the control unit sets a response level for the manual driving based on the evaluation information and provides the evaluation information to an information processing device that selects an operator to whom manual driving of the vehicle is permitted based on the response level and the operator's skill level. [Effects of the Invention]

[0010] According to one embodiment of this disclosure, the technology for switching a vehicle from autonomous driving to manual driving is improved. [Brief explanation of the drawing]

[0011] [Figure 1] This is a diagram showing an example of the configuration of an information processing system including an information processing device according to one embodiment of the present disclosure. [Figure 2] This is a table showing a first example of evaluation information generated by the vehicle determination device in Figure 1. [Figure 3] This is a table showing a second example of evaluation information generated by the vehicle determination device in Figure 1. [Figure 4] This is a table showing a third example of evaluation information generated by the vehicle determination device in Figure 1. [Figure 5] Figure 1 is a table illustrating an example of the operation of the information processing device. [Figure 6] This is the first flowchart illustrating an example of an information processing method performed by the information processing system shown in Figure 1. [Figure 7] This is a second flowchart illustrating an example of an information processing method performed by the information processing system shown in Figure 1. [Figure 8] This is the third flowchart illustrating an example of an information processing method performed by the information processing system shown in Figure 1. [Figure 9] This is the fourth flowchart illustrating an example of an information processing method performed by the information processing system shown in Figure 1. [Figure 10] This is the fifth flowchart illustrating an example of an information processing method performed by the information processing system shown in Figure 1. [Figure 11] This is the sixth flowchart illustrating an example of an information processing method performed by the information processing system shown in Figure 1. [Figure 12] This is the seventh flowchart illustrating an example of an information processing method performed by the information processing system shown in Figure 1. [Modes for carrying out the invention]

[0012] Hereinafter, an embodiment of the present disclosure will be described with reference to the accompanying drawings. The following description of the information processing apparatus 30 also applies to an information processing system 1 including the information processing apparatus 30 to which the present disclosure is applied, an information processing method executed by the information processing apparatus 30, a program for causing the information processing apparatus 30 as a computer to execute operations, a vehicle 10, and a determination apparatus 20.

[0013] (Overview of the Embodiment) FIG. 1 is a configuration diagram showing an example of the configuration of an information processing system 1 including an information processing apparatus 30 according to an embodiment of the present disclosure. An example of the configuration of an information processing system 1 including the information processing apparatus 30 according to an embodiment of the present disclosure will be mainly described with reference to FIG. 1. The information processing system 1 includes a vehicle 10, a determination apparatus 20 mounted on the vehicle 10, an information processing apparatus 30 and a remote manual driving apparatus 40 installed in an operation management center 3, and an in-vehicle manual driving apparatus 50 temporarily installed in the vehicle 10.

[0014] In FIG. 1, for the purpose of simple illustration, only one of each of the vehicle 10, the determination apparatus 20, the information processing apparatus 30, the remote manual driving apparatus 40, and the in-vehicle manual driving apparatus 50 is shown, but the number of each apparatus included in the information processing system 1 may be two or more. The vehicle 10, the determination apparatus 20, the information processing apparatus 30, the remote manual driving apparatus 40, and the in-vehicle manual driving apparatus 50 are communicably connected to a network 2 including, for example, the Internet or a mobile communication network. Any one of the vehicle 10, the determination apparatus 20, the information processing apparatus 30, the remote manual driving apparatus 40, and the in-vehicle manual driving apparatus 50 is communicably connected to other apparatuses via the network 2.

[0015] Vehicle 10 includes, for example, a motor vehicle having a Level 4 (Lv4) autonomous driving function. The motor vehicle may be a gasoline vehicle, a battery electric vehicle (BEV), a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), a fuel cell electric vehicle (FCEV), or the like, but is not limited thereto. Vehicle 10 is communicably connected to determination device 20, information processing device 30, remote manual driving device 40, and in-vehicle manual driving device 50 via network 2. Vehicle 10 may be communicably connected to each of determination device 20 and in-vehicle manual driving device 50 by wire.

[0016] Vehicle 10 has an autonomous driving mode and a manual driving mode that enables manual driving by an operator. In the present disclosure, the "manual driving mode" includes, for example, a remote manual driving mode in which manual driving is performed outside Vehicle 10 and an in-vehicle manual driving mode in which manual driving is performed inside Vehicle 10. The remote manual driving mode is, for example, a mode in which Vehicle 10 is manually driven using remote manual driving device 40 from operation management center 3 located at a remote location. The in-vehicle manual driving mode is, for example, a mode in which Vehicle 10 is manually driven using in-vehicle manual driving device 50 temporarily installed in the vehicle interior of Vehicle 10.

[0017] Determination device 20 includes, for example, a computer mounted on Vehicle 10. Determination device 20 is communicably connected to Vehicle 10, information processing device 30, remote manual driving device 40, and in-vehicle manual driving device 50 via network 2. Determination device 20 may be communicably connected to each of Vehicle 10 and in-vehicle manual driving device 50 by wire.

[0018] The information processing device 30 includes, for example, one or multiple server devices that can communicate with each other. The information processing device 30 is not limited to these and may include any general-purpose electronic device such as a PC (Personal Computer), tablet PC, smartphone, or wearable device such as a smartwatch, or it may include other electronic devices dedicated to the information processing system 1. The information processing device 30 is, for example, a device operated by the operation management center 3. The information processing device 30 is communicated via the network 2 to the vehicle 10, the determination device 20, the remote manual driving device 40, and the in-vehicle manual driving device 50.

[0019] The remote manual driving device 40 includes, for example, a control device installed in the operation management center 3. The remote manual driving device 40 monitors and manually drives the vehicle 10 via the network 2, through operation by an operator at the operation management center 3, which is located remotely from the vehicle 10. The remote manual driving device 40 is communicatively connected to the vehicle 10, the determination device 20, the information processing device 30, and the in-vehicle manual driving device 50 via the network 2.

[0020] The in-vehicle manual driving device 50 includes a control device used, for example, when an operator of the response team 4, who has received a request from the information processing device 30 to rush to the vehicle 10, rushes to the vehicle 10 and drives the vehicle 10 manually. The in-vehicle manual driving device 50 is temporarily installed, for example, in a vehicle 10 that does not have a mechanical operating unit for manual driving. The response team 4 is a unit that is on standby in a rescue center, which is located within the operation management center 3 or in a location away from the operation management center 3. The in-vehicle manual driving device 50 is communicatively connected to the vehicle 10, the determination device 20, the information processing device 30, and the remote manual driving device 40 via the network 2. The in-vehicle manual driving device 50 may also be communicatively connected to the vehicle 10 and the determination device 20, respectively, by wire.

[0021] The components of Information Processing System 1 will be described in detail.

[0022] (Vehicle configuration 10) The vehicle 10 includes a communication unit 11, a measurement unit 12, a storage unit 13, a control unit 14, and a determination device 20.

[0023] The communication unit 11 includes one or more communication interfaces connected to the network 2. These communication interfaces support, but are not limited to, mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation), wired LAN (Local Area Network) standards, or wireless LAN standards. For example, the communication interface may also support short-range wireless communication standards. In one embodiment, the vehicle 10 is communicably connected to the network 2 via the communication unit 11. The communication unit 11 transmits and receives various types of information via the network 2. In addition, the communication unit 11 also includes a communication interface that connects via a wired connection to the vehicle 10's CAN (Controller Area Network).

[0024] The measurement unit 12 has a sensor 12A. The sensor 12A includes, for example, a first sensor that monitors the environment around the autonomously driven vehicle 10 and a second sensor that diagnoses a failure mode, such as one mounted on the vehicle 10's self-diagnostic device. The first sensor of sensor 12A includes a camera, 3D-LiDAR (Light Detection and Ranging), millimeter-wave sensor, acceleration sensor, or GPS (Global Positioning System) sensor. The type of sensor 12A is not limited to these.

[0025] The storage unit 13 includes one or more memories. In this disclosure, “memory” includes, but is not limited to, semiconductor memory, magnetic memory, or optical memory. Each memory included in the storage unit 13 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 13 stores information necessary to realize the operation of the vehicle 10. For example, the storage unit 13 may store a system program, an application program, or embedded software. The storage unit 13 also stores information obtained through the operation of the vehicle 10. The information stored in the storage unit 13 may be updatable with information obtained from the network 2 via, for example, the communication unit 11.

[0026] The control unit 14 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. In this disclosure, "processor" refers to, but is not limited to, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. "Programmable circuit" refers to, but is not limited to, an FPGA (Field-Programmable Gate Array). "Dedicated circuit" refers to, but is not limited to, an ASIC (Application Specific Integrated Circuit). The control unit 14 controls the operation of the entire vehicle 10, including Level 4 automated driving and manual driving in conjunction with a remote manual driving device 40 or an in-vehicle manual driving device 50, which will be described later.

[0027] (Configuration of the determination device 20) The determination device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0028] The communication unit 21 includes one or more communication interfaces connected to the network 2. These communication interfaces support, but are not limited to, mobile communication standards such as LTE, 4G, or 5G, wired LAN standards, or wireless LAN standards, and may support any communication standard. For example, the communication interface may also support short-range wireless communication standards. In one embodiment, the determination device 20 is communicably connected to the network 2 via the communication unit 21. The communication unit 21 transmits and receives various information via the network 2. In addition, the communication unit 21 also includes a communication interface that is wired to the CAN of the vehicle 10.

[0029] The storage unit 22 includes one or more memories. Each memory included in the storage unit 22 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 22 stores information necessary to realize the operation of the determination device 20. For example, the storage unit 22 may store a system program, an application program, or embedded software. The storage unit 22 stores information obtained by the operation of the determination device 20. The information stored in the storage unit 22 may be updateable with information obtained from the network 2 via, for example, the communication unit 21.

[0030] The control unit 23 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 23 is communicatively connected to each component constituting the determination device 20 and controls the operation of the entire determination device 20. The control unit 23 constantly monitors whether the design requirements (hereinafter also referred to as ODD (Operational Design Domain)) that enable the vehicle 10 to operate autonomously are satisfied, based on measurement data from the sensor 12A, etc. The control unit 23 generates evaluation information, which will be described later, when the ODD is no longer satisfied. When predetermined conditions are met, the control unit 23 provides the evaluation information to the information processing device 30 operated by the operation management center 3 and requests that some action be taken regarding the ODD deviation.

[0031] (Configuration of the information processing device 30) The information processing device 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0032] The communication unit 31 includes one or more communication interfaces connected to the network 2. These communication interfaces support, but are not limited to, mobile communication standards such as LTE, 4G, or 5G, wired LAN standards, or wireless LAN standards, and may support any communication standard. For example, the communication interface may also support short-range wireless communication standards. In one embodiment, the information processing device 30 is communicably connected to the network 2 via the communication unit 31. The communication unit 31 transmits and receives various types of information via the network 2.

[0033] The storage unit 32 includes one or more memories. Each memory included in the storage unit 32 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 32 stores information necessary to realize the operation of the information processing device 30. For example, the storage unit 32 may store system programs, application programs, or embedded software. The storage unit 32 also stores information obtained through the operation of the information processing device 30. The information stored in the storage unit 32 may be updateable with information obtained from the network 2 via, for example, the communication unit 31.

[0034] The control unit 33 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 33 is communicatively connected to each component constituting the information processing device 30 and controls the operation of the entire information processing device 30.

[0035] (Configuration of the remote manual operation device 40) The remote manual operation device 40 includes a communication unit 41, a storage unit 42, an input unit 43, an output unit 44, and a control unit 45.

[0036] The communication unit 41 includes one or more communication interfaces connected to the network 2. These communication interfaces support, but are not limited to, mobile communication standards such as LTE, 4G, or 5G, wired LAN standards, or wireless LAN standards, and may support any communication standard. For example, the communication interface may also support short-range wireless communication standards. In one embodiment, the remote manual driving device 40 is communicably connected to the network 2 via the communication unit 41. The communication unit 41 transmits and receives various types of information via the network 2.

[0037] The storage unit 42 includes one or more memories. Each memory included in the storage unit 42 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 42 stores information necessary to realize the operation of the remote manual operation device 40. For example, the storage unit 42 may store a system program, an application program, or embedded software. The storage unit 42 also stores information obtained through the operation of the remote manual operation device 40. The information stored in the storage unit 42 may be updateable with information obtained from the network 2 via, for example, the communication unit 41.

[0038] The input unit 43 includes one or more input interfaces that detect user input and acquire input information based on the operator's actions. These input interfaces include physical keys, capacitive keys, pointing devices such as mice, touchscreens integrated with the display of the output unit 44, imaging modules such as cameras, microphones for receiving voice input, or biometric authentication modules such as vein authentication or fingerprint authentication. The input unit 43 is not limited to these and may include any other input interfaces. Instead of being provided in the remote manual operation device 40, the input unit 43 may be connected to the remote manual operation device 40 as an external input device. Any connection method can be used, such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or Bluetooth (Bluetooth).

[0039] The output unit 44 includes one or more output interfaces for outputting information to an operator using the remote manual operation device 40. These output interfaces include a display for visually outputting information as an image, a speaker for audibly outputting information as sound, or a vibrator for tactilely outputting information as vibration. In this disclosure, "display" includes, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescent) display. The output unit 44 may be connected to the remote manual operation device 40 as an external output device, rather than being provided within the device itself. Any connection method can be used, such as USB, HDMI®, or Bluetooth®.

[0040] The control unit 45 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 45 is communicatively connected to each component constituting the remote manual operation device 40 and controls the operation of the entire remote manual operation device 40.

[0041] (Configuration of the in-vehicle manual driving device 50) In Figure 1, for the purpose of simplifying the drawing, the configuration of the in-vehicle manual driving device 50 is omitted from the illustration, but the in-vehicle manual driving device 50 has the same configuration as the remote manual driving device 40. For example, the in-vehicle manual driving device 50 has a communication unit, a storage unit, an input unit, an output unit, and a control unit. The communication unit has the same configuration as the communication unit 41. The storage unit has the same configuration as the storage unit 42. The input unit has the same configuration as the input unit 43. The output unit has the same configuration as the output unit 44. The control unit has the same configuration as the control unit 45.

[0042] (Content of evaluation information) Figure 2 is a table showing a first example of evaluation information generated by the vehicle 10 determination device 20 in Figure 1. Referring to Figure 2, we will mainly explain the first example of the content of the evaluation information. The first evaluation information shown in Figure 2 includes, for example, urgency, danger, impact, and difficulty as evaluation items.

[0043] In this disclosure, "urgency" is an evaluation item that corresponds to, for example, the estimated time for the vehicle 10 to return to operation by autonomous driving. Urgency is evaluation information based on measurement data from the second sensor among the sensors 12A provided on the vehicle 10. For example, the urgency is "1" if the estimated return time is less than 1 minute. For example, the urgency is "2" if the estimated return time is 1 minute or more but less than 3 minutes. For example, the urgency is "3" if the estimated return time is 3 minutes or more but less than 5 minutes. For example, the urgency is "4" if the estimated return time is 5 minutes or more but less than 10 minutes. For example, the urgency is "5" if the estimated return time is 10 minutes or more, or if return is not expected.

[0044] In this disclosure, "risk level" is an evaluation item categorized into levels according to the degree of injury risk to the occupants of vehicle 10. The risk level is evaluation information based on measurement data from a camera, which is the first sensor among the sensors 12A installed in vehicle 10. For example, a risk level of "1" means no injury. For example, a risk level of "2" means there is a risk of minor injury. For example, a risk level of "3" means there is a risk of multiple minor injuries. For example, a risk level of "4" means there is a risk of serious injury. For example, a risk level of "5" means there is a risk of death.

[0045] In this disclosure, "impact level" is an evaluation item that indicates the level of social impact related to the ODD deviation of the autonomous driving of vehicle 10. The impact level is evaluation information based on measurement data from the camera, which is the first sensor among the sensors 12A installed in vehicle 10. The impact level is "1" if there is no impact. The impact level is "2" if there is traffic congestion around vehicle 10. The impact level is "3" if an accident occurs in or around vehicle 10 and an emergency vehicle dispatch request is made. "Emergency vehicles" include, for example, fire engines, ambulances, rescue vehicles, or police vehicles. The impact level is "4" if there is a situation that requires contact with local government, such as damage to public facilities. "Local government" includes, for example, the police headquarters of each prefecture or other arbitrary organizations at the prefectural level. The impact level is "5" if there is a situation that requires contact with government agencies, such as the occurrence of a natural disaster. "Government agencies" include, for example, any agency at the national level.

[0046] In this disclosure, "difficulty level" is an evaluation item categorized into levels according to the possibility of taking action, such as manual driving, when the vehicle 10 deviates from the ODD during automated driving. The difficulty level is evaluation information based on measurement data from the first sensor, the camera, or the second sensor, among the sensors 12A provided in the vehicle 10. The difficulty level is "1" if the vehicle 10 can automatically adapt to the situation. The difficulty level is "2" if it can be adapted to remote manual driving using the remote manual driving device 40. The difficulty level is "3" if it can be adapted to manual driving inside the vehicle 10 using the in-vehicle manual driving device 50. The difficulty level is "4" if it can be adapted to by introducing a replacement vehicle for the vehicle 10. The difficulty level is "5" if it is impossible to continue operation with either the vehicle 10 or a replacement vehicle.

[0047] Figure 3 is a table showing a second example of evaluation information generated by the vehicle determination device 20 of Figure 1. The second example of the content of the evaluation information will be mainly explained with reference to Figure 3. The second evaluation information shown in Figure 3 includes, for example, environmental compatibility as an evaluation item. "Environment" includes, for example, communication environment, weather environment, road environment, and vehicle environment.

[0048] In this disclosure, "communication environment" is an evaluation item that corresponds to conditions related to wireless communication, such as communication delay. The communication environment is evaluation information based on measurement data from the second sensor among the sensors 12A provided on the vehicle 10. For example, the communication environment is "1" if the latency is 500ms or more and less than 1s. For example, the communication environment is "2" if the latency is 1s or more and less than 3s. For example, the communication environment is "3" if the latency is 3s or more and less than 10s. For example, the communication environment is "4" if the latency is 10s or more and less than 15s. For example, the communication environment is "5" if the latency is 15s or more.

[0049] In this disclosure, “weather conditions” is an evaluation item that corresponds to weather conditions such as rainfall, heavy rain, snowfall, heavy snow, snow cover, hail, fog, or dense fog. Weather conditions are evaluation information based on measurement data from a camera, which is the first sensor among the sensors 12A installed in the vehicle 10. Weather conditions are, for example, “1” if the weather conditions are good. Weather conditions are, for example, “2” if there is fog or rainfall. Weather conditions are, for example, “3” if there is snowfall. Weather conditions are, for example, “4” if there is dense fog or hail. Weather conditions are, for example, “5” if there is heavy rain, heavy snow, or snow cover.

[0050] In this disclosure, “road environment” is an evaluation item that corresponds to road conditions such as road structures, infrastructure such as traffic signals, the movement of emergency vehicles, or other moving objects including people or mobility devices. The road environment is evaluation information based on measurement data from the first sensor, a camera, 3D-LiDAR, or millimeter-wave sensor, among the sensors 12A installed in the vehicle 10. For example, the road environment is “1” if the road conditions are good. The road environment is “2” if there is an obstacle such as a moving object obstructing the path by a person or mobility device. The road environment is “3” if the path is impassable due to the movement of an emergency vehicle or infrastructure such as traffic signals. The road environment is “4” if the path is impassable due to an obstacle in a road structure. The road environment is “5” if the path is impassable for other significant reasons.

[0051] In this disclosure, "vehicle environment" refers to evaluation items that correspond to vehicle conditions such as power loss, inability to drive, engine function failure, or driving / turning / stopping function failure. The vehicle environment is evaluation information based on measurement data from the second sensor among the sensors 12A provided in the vehicle 10. For example, the vehicle environment is "1" when the vehicle condition is normal. For example, the vehicle environment is "4" when an ADK (Autonomous Driving Kit) failure occurs that causes a malfunction in the autonomous driving function. For example, the vehicle environment is "5" when a power loss, inability to drive, engine function failure, or driving / turning / stopping function failure occurs.

[0052] Figure 4 is a table showing a third example of evaluation information generated by the vehicle determination device 20 of Figure 1. Referring to Figure 4, the third example of the evaluation information content will be primarily explained. The third evaluation information shown in Figure 4 includes, for example, an ODD deviation score as an evaluation item. "ODD" includes, for example, road conditions, geographical conditions, environmental conditions, and other conditions.

[0053] In this disclosure, “road conditions” include, for example, the type of road such as a closed road, public road, expressway, or other motorway, the presence or absence of lanes, the number of lanes, the presence or absence of sidewalks, lane width, or climbing performance. Road conditions are evaluation information based on measurement data from, for example, the first sensor among the sensors 12A provided on the vehicle 10, which is a camera, 3D-LiDAR, millimeter-wave sensor, or GPS. Road conditions are, for example, “1” if they conform to the design requirements. Road conditions are, for example, “5” if they deviate from the design requirements.

[0054] In this disclosure, “geographic conditions” include conditions such as whether or not the driving area is limited. Geographic conditions are evaluation information based on measurement data from the GPS, which is the first sensor among the sensors 12A provided on the vehicle 10. Geographic conditions are, for example, “1” if they conform to the design requirements. Geographic conditions are, for example, “5” if they deviate from the design requirements.

[0055] In this disclosure, “environmental conditions” include, for example, conditions such as the presence or absence of sunshine, the presence or absence of rainfall or heavy rain, the presence or absence of snowfall or snow cover, or temperature. Environmental conditions are evaluation information based on measurement data from a camera, which is the first sensor among the sensors 12A provided on the vehicle 10. Environmental conditions are, for example, “1” if they conform to the design requirements. Environmental conditions are, for example, “5” if they deviate from the design requirements.

[0056] In this disclosure, “other conditions” include, for example, conditions such as speed limits, signal information, whether or not coordination with infrastructure is required, whether or not security personnel are needed, or continuous driving time. Other conditions are, for example, evaluation information based on measurement data from the first sensor, such as a camera, 3D-LiDAR, millimeter-wave sensor, GPS, or second sensor, among the sensors 12A provided on the vehicle 10. Other conditions are, for example, “1” if they conform to the design requirements. Other conditions are, for example, “5” if they deviate from the design requirements.

[0057] (Skill level details) Figure 5 is a table illustrating an example of the operation of the information processing device 30 shown in Figure 1. Figure 5 shows evaluation items for determining the skill level of an operator performing manual driving using the remote manual driving device 40 or the in-vehicle manual driving device 50. The skill level evaluation items include, for example, knowledge, practical experience, and proficiency. In this disclosure, "skill level" means, for example, the level of proficiency in manual driving. The skill level is determined for each operator, for example by the manager of the operation management center 3, and is pre-stored in the storage unit 32 of the information processing device 30 as a database associated with each operator's identification information.

[0058] The "knowledge" evaluation item is a categorized evaluation item that is leveled according to the operator's knowledge of manual operation. If the knowledge is at the level of basic knowledge based on laws and regulations, the skill level will be "1". If the knowledge is sufficient for the safe operation of vehicle 10, the skill level will be "2". If the knowledge is at the practical level of knowledge regarding the operation of vehicle 10, the skill level will be "3". If the knowledge is at the applied level of knowledge regarding the operation of vehicle 10, the skill level will be "4". If the knowledge is at the teacher level of knowledge regarding the operation of vehicle 10, the skill level will be "5".

[0059] The evaluation item "Practical Experience" is a categorized evaluation item that is leveled according to the operator's practical experience in manual operation. If practical experience is less than one year, the skill level is "1". If practical experience is one year or more but less than three years, the skill level is "2". If practical experience is three years or more but less than five years, the skill level is "3". If practical experience is five years or more but less than ten years, the skill level is "4". If practical experience is ten years or more, the skill level is "5".

[0060] The "proficiency" evaluation item is a tiered evaluation item based on the operator's level of proficiency in manual operation. If the proficiency level is beginner, the skill level will be "1". If the proficiency level is intermediate, the skill level will be "2". If the proficiency level is advanced, the skill level will be "3". If the proficiency level is expert, the skill level will be "4". If the proficiency level is master, the skill level will be "5".

[0061] (Operation flow of Information Processing System 1) The following section primarily describes the sequence of processes performed by the information processing system 1 when an action is taken in response to a deviation in the vehicle's ODD (Operational Design Dimension).

[0062] Figure 6 is a first flowchart illustrating an example of an information processing method performed by the information processing system 1 in Figure 1. Figure 6 shows an example of a series of processes performed first in the vehicle determination device 20 when the vehicle 10 deviates from the ODD.

[0063] In step S101, the control unit 23 of the vehicle 10 determination device 20 determines whether the vehicle 10 has deviated from the automatic driving ODD. If the control unit 23 determines that the vehicle has deviated from the automatic driving ODD, it executes the process in step S102. If the control unit 23 determines that the vehicle has not deviated from the automatic driving ODD, it executes the process again from step S101.

[0064] In step S102, if the control unit 23 of the vehicle 10 determination device 20 determines in step S101 that the vehicle 10 has deviated from the ODD for autonomous driving, it starts the process of acquiring evaluation information. For example, the control unit 23 acquires evaluation information based on the measurement data of the sensor 12A of the measurement unit 12 of the vehicle 10. The control unit 23 generates the first evaluation information, second evaluation information, and third evaluation information described above, and acquires them together as evaluation information.

[0065] In step S103, the control unit 23 of the vehicle 10 determination device 20 determines whether or not the emergency stop button of the vehicle 10 has been pressed. For example, the emergency stop button may be installed inside the vehicle 10 and be pressable by the occupants of the vehicle 10, or it may be installed in the operation management center 3 and be pressable by the administrator. If the control unit 23 determines that the emergency stop button has been pressed, it executes the process in step S105. If the control unit 23 determines that the emergency stop button has not been pressed, it executes the process in step S104.

[0066] In step S104, if the control unit 23 of the vehicle 10 determination device 20 determines that the emergency stop button was not pressed in step S103, it determines whether or not Level 4 autonomous driving is possible for the vehicle 10.

[0067] For example, if all evaluation items included in the first evaluation information, second evaluation information, and third evaluation information acquired in step S102 are "1", the control unit 23 determines that the vehicle 10 is capable of Level 4 autonomous driving. Once the control unit 23 determines that Level 4 autonomous driving is possible, it repeats the process from step S102.

[0068] For example, if at least one of the evaluation items included in the first evaluation information, second evaluation information, and third evaluation information acquired in step S102 is "2" or higher, the control unit 23 determines that Level 4 autonomous driving is not possible for the vehicle 10. If the control unit 23 determines that Level 4 autonomous driving is not possible, it executes the process in step S105.

[0069] In step S105, if the control unit 23 of the vehicle 10 determination device 20 determines that the emergency stop button was pressed in step S103, or determines in step S104 that automatic driving at Lv4 is impossible, it stops and fixes the vehicle 10.

[0070] For example, consider a case where an emergency stop button is installed at the operation management center 3. The administrator of the operation management center 3 is stationed at the operation management center 3 and remotely monitors information from the vehicle 10 to decide whether to permit or deny its operation. The information processing system 1 has a mechanism that allows the administrator to quickly stop and fix the vehicle 10 if the administrator decides that the operation of the vehicle 10 is not permitted. For example, the control unit 33 of the information processing device 30 constantly sends operation permission flag information to the vehicle 10, reflecting the intentions of the administrator stationed at the operation management center 3. On the other hand, the control unit 33 of the information processing device 30 may also stop and fix the vehicle 10 by ceasing to send operation permission flag information, reflecting the intentions of the administrator.

[0071] In step S106, the control unit 23 of the vehicle 10 determination device 20 determines whether the urgency level included in the evaluation information acquired in step S102 is equal to or greater than a third predetermined value. The third predetermined value may be, for example, "2". If the control unit 23 determines that the urgency level is equal to or greater than the third predetermined value, for example "2" or greater, it executes the process in step S108. If the control unit 23 determines that the urgency level is less than the third predetermined value, for example "1", it executes the process in step S107.

[0072] In step S107, if the control unit 23 of the vehicle 10 determination device 20 determines in step S106 that the urgency level is less than the third predetermined value, it waits, for example, until the vehicle 10 returns to operation by automatic driving.

[0073] In step S108, if the control unit 23 of the vehicle 10 determination device 20 determines in step S106 that the urgency is equal to or greater than the third predetermined value, it provides the evaluation information acquired in step S102 to the information processing device 30. For example, the control unit 23 transmits the evaluation information acquired in step S102 to the information processing device 30 via the communication unit 21 and the network 2. As a result, the control unit 23 issues a request to the operation management center 3 for action in the event of vehicle 10's ODD deviation.

[0074] Figure 7 is a second flowchart illustrating an example of an information processing method performed by the information processing system 1 in Figure 1. Figure 7 shows an example of a series of processing steps in the information processing device 30 that are performed following step S108 in Figure 6 when the vehicle 10 deviates from the ODD.

[0075] In step S201, the control unit 33 of the information processing device 30 acquires evaluation information based on measurement data from the sensor 12A installed in the vehicle 10, which has an automatic driving mode and a manual driving mode that allows manual driving by an operator. The control unit 33 acquires the evaluation information acquired by the vehicle 10's determination device 20 in step S102 of Figure 6 from the vehicle 10. For example, the control unit 33 receives the evaluation information acquired in step S102 of Figure 6 from the vehicle 10 via the network 2 and the communication unit 31. As a result, the control unit 33 receives a request from the vehicle 10 for action to be taken when the vehicle 10 deviates from the ODD.

[0076] In step S202, when the control unit 33 of the information processing device 30 receives correction information from the administrator of the operation management center 3, which manages the operation of the vehicle 10, to correct the evaluation information acquired from the vehicle 10 in step S201, the control unit 33 corrects the evaluation information based on the correction information. For example, the administrator remotely checks the status of the vehicle 10's ODD deviation using a monitor or the like, and inputs a correction value for at least one evaluation item included in the evaluation information as correction information in any way at their own discretion. The control unit 33 acquires the input correction information in any way at its own discretion and corrects the value of at least one evaluation item included in the evaluation information to the corrected value based on the acquired correction information.

[0077] In step S203, the control unit 33 of the information processing device 30 determines whether the risk level included in the evaluation information is equal to or greater than a second predetermined value. The second predetermined value may be, for example, "2". If the control unit 33 determines that the risk level is equal to or greater than the second predetermined value, for example, "2" or greater, it executes the process in step S204. If the control unit 33 determines that the risk level is less than the second predetermined value, for example, "1", it executes the process in step S301 in the third flowchart of Figure 8.

[0078] The risk level used in step S203 may be, for example, the value that was modified in step S202, or it may be the value obtained in step S201 without any modification in step S202. The same applies to the following explanations of other evaluation items included in the evaluation information.

[0079] In step S204, if the control unit 33 of the information processing device 30 determines in step S203 that the level of danger is equal to or greater than a second predetermined value, it stops the operation of the vehicle 10.

[0080] In step S205, if the control unit 33 of the information processing device 30 determines in step S203 that the level of danger is equal to or greater than the second predetermined value, it requests the dispatch of an ambulance.

[0081] In step S206, the control unit 33 of the information processing device 30 requests, in addition to the request for the dispatch of an ambulance in step S205, at least one of the following: the towing of vehicle 10 by a tow truck or the like, and rescue operations by a rescue team. For example, the control unit 33 requests both the towing of vehicle 10 and rescue operations.

[0082] Figure 8 is a third flowchart illustrating an example of an information processing method performed by the information processing system 1 in Figure 1. Figure 8 shows an example of a series of processing steps in the information processing device 30 that are performed following step S206 in Figure 7 when vehicle 10 deviates from the ODD. The control unit 33 of the information processing device 30 communicates the status of the ODD deviation to at least one of the government agency and the local government, depending on the degree of impact included in the evaluation information.

[0083] In step S301, the control unit 33 of the information processing device 30 determines whether the influence level included in the evaluation information is "2" or less. If the control unit 33 determines that the influence level is "2" or less, it executes the process in step S401 in the fourth flowchart of Figure 9. If the control unit 33 determines that the influence level is "3" or more, it executes the process in step S302.

[0084] In step S302, if the control unit 33 of the information processing device 30 determines in step S301 that the influence level is "3" or higher, it then determines whether the influence level is "5". If the control unit 33 determines that the influence level is "5", it executes the process in step S303. If the control unit 33 determines that the influence level is "3" or "4", it executes the process in step S305.

[0085] In step S303, if the control unit 33 of the information processing device 30 determines in step S302 that the impact level is "5", it notifies the government agency of the vehicle 10's ODD deviation status.

[0086] In step S304, the control unit 33 of the information processing device 30, in addition to contacting government agencies in step S303, also informs local authorities of the vehicle 10's ODD deviation.

[0087] In step S305, if the control unit 33 of the information processing device 30 determined in step S302 that the influence level is "3" or "4", it then determines whether the influence level is "4". If the control unit 33 determines that the influence level is "4", it executes the process in step S304. If the control unit 33 determines that the influence level is "3", it executes the process in step S306.

[0088] In step S306, if the control unit 33 of the information processing device 30 determines in step S305 that the impact level is "3", it notifies the relevant parties of the status of the vehicle 10's ODD deviation. The relevant parties include, for example, any organization that, upon receiving notification of the vehicle 10's ODD deviation, dispatches an emergency vehicle to the scene of the vehicle 10's ODD deviation.

[0089] Figure 9 is a fourth flowchart illustrating an example of an information processing method performed by the information processing system 1 in Figure 1. Figure 9 shows an example of a series of processing steps in the information processing device 30 that are performed following step S301 in Figure 8 when the vehicle 10 deviates from the ODD.

[0090] In step S401, the control unit 33 of the information processing device 30 determines whether the vehicle 10 is capable of self-propulsion. For example, the control unit 33 determines whether the vehicle 10 is capable of self-propulsion based on three evaluation items: difficulty level included in the first evaluation information, and road environment and vehicle environment included in the second evaluation information. If the control unit 33 determines that the vehicle 10 is capable of self-propulsion, it executes the process in step S403. If the control unit 33 determines that the vehicle 10 is not capable of self-propulsion, it executes the process in step S402.

[0091] In step S402, if the control unit 33 of the information processing device 30 determines in step S401 that the vehicle 10 is unable to move under its own power, it requests at least one of the following: towing the vehicle 10 by a tow truck or the like, and introducing a replacement vehicle for the vehicle 10. For example, the control unit 33 requests both towing the vehicle 10 and introducing a replacement vehicle.

[0092] In step S403, if the control unit 33 of the information processing device 30 determines in step S401 that the vehicle 10 is capable of self-propulsion, it determines whether the wireless communication status of the vehicle 10 is good or not based on the communication environment and vehicle environment of the vehicle 10 included in the evaluation information. If the control unit 33 determines that the wireless communication status is good, it executes the process in step S405. If the control unit 33 determines that the wireless communication status is not good, it executes the process in step S404.

[0093] In step S404, if the control unit 33 of the information processing device 30 determines in step S403 that the wireless communication status is not good, it requests the operator performing manual operation to rush to the vehicle 10. For example, the control unit 33 requests the response team 4 to rush to the vehicle 10 of the operator.

[0094] In step S405, if the control unit 33 of the information processing device 30 determines in step S403 that the wireless communication status is good, it determines whether the difficulty level of manual operation included in the evaluation information is equal to or greater than a first predetermined value. The first predetermined value may be, for example, "3". If the control unit 33 determines that the difficulty level is equal to or greater than the first predetermined value, for example "3", it executes the process of step S601 in the sixth flowchart of Figure 11. If the control unit 33 determines that the difficulty level is less than the first predetermined value, for example "2", it executes the process of step S501 in the fifth flowchart of Figure 10.

[0095] If the control unit 33 of the information processing device 30 determines in step S403 that the wireless communication status of the vehicle 10 is good, it executes the corresponding level setting process and the operator selection process for the vehicle 10, which will be described later. If the difficulty level of manual driving included in the evaluation information is equal to or greater than a first predetermined value, the control unit 33 sets the in-vehicle manual driving mode as the manual driving mode, as shown in Figure 11. If the difficulty level of manual driving included in the evaluation information is less than the first predetermined value, the control unit 33 sets the remote manual driving mode as the manual driving mode, as shown in Figure 10.

[0096] Figure 10 is a fifth flowchart illustrating an example of an information processing method performed by the information processing system 1 in Figure 1. Figure 10 shows an example of a series of processing steps in the information processing device 30 when setting the response level and selecting the operator of vehicle 10 in remote manual driving mode when vehicle 10 deviates from the ODD.

[0097] In step S501, if the control unit 33 of the information processing device 30 determines that the difficulty level in step S405 in Figure 9 is relatively low ("2"), it sets the manual driving mode of the vehicle 10 to remote manual driving mode.

[0098] In step S502, the control unit 33 of the information processing device 30 sets the response level for manual driving based on evaluation information when switching from automatic driving mode to manual driving mode. For example, the control unit 33 sets the response level in remote manual driving mode using information on the communication environment, weather environment, and road environment of the vehicle 10 included in the second evaluation information. In this disclosure, the "response level" may be, for example, the value of the evaluation item with the largest value among a plurality of evaluation items used for setting, the sum of the values ​​of the plurality of evaluation items, or a statistical value of the values ​​of the plurality of evaluation items. The "statistical value" may include, for example, the mean or median.

[0099] In step S503, the control unit 33 of the information processing device 30 refers to the operator's skill level included in the database pre-stored in the storage unit 32.

[0100] In step S504, the control unit 33 of the information processing device 30 selects an operator to whom manual operation of the vehicle 10 is permitted based on the correspondence level set in step S502 and the operator's skill level referenced in step S503. For example, the control unit 33 selects an operator whose skill level is equal to or higher than the correspondence level as an operator to whom manual operation of the vehicle 10 is permitted.

[0101] In step S505, the control unit 33 of the information processing device 30 determines whether or not it has received first response information from the operator selected in step S504 indicating that manual operation is difficult. For example, the operator remotely checks the status of the vehicle 10's ODD deviation using the output unit 44 of the remote manual operation device 40 and inputs information indicating that manual operation is difficult based on their own judgment using the input unit 43 of the remote manual operation device 40. When the control unit 33 receives this input information from the remote manual operation device 40, it determines that it has received first response information. If the control unit 33 determines that it has received first response information, it executes the process in step S506. If the control unit 33 determines that it has not received first response information, it executes the process in step S508.

[0102] In step S506, if the control unit 33 of the information processing device 30 determines that it received first response information from the selected operator in step S505 indicating that manual operation is difficult, it re-selects an operator with a higher skill level than the selected operator as the operator authorized to operate the vehicle 10 manually.

[0103] In step S507, the control unit 33 of the information processing device 30 determines whether it has received second response information indicating that manual operation is difficult from the operator selected in step S506 of the remote manual operation mode. For example, the operator remotely checks the status of the vehicle 10's ODD deviation using the output unit 44 of the remote manual operation device 40 and inputs information indicating that manual operation is difficult based on their own judgment using the input unit 43 of the remote manual operation device 40. When the control unit 33 receives this input information from the remote manual operation device 40, it determines that it has received second response information.

[0104] When the control unit 33 of the information processing device 30 determines that it has received the second response information, it executes a series of processes shown in the sixth flowchart of Figure 11. When the control unit 33 receives the second response information from an operator selected in remote manual driving mode indicating that it is difficult to perform manual driving, it re-selects an operator who will perform manual driving in in-vehicle manual driving mode as an operator authorized to perform manual driving. When the control unit 33 determines that it has not received the second response information, it executes the process in step S508.

[0105] In step S508, if the control unit 33 of the information processing device 30 determines that it has not received the first response information in step S505, or that it has not received the second response information in step S507, it will perform an action. For example, the control unit 33 may display the remote manual driving mode screen on the output unit 44 of the remote manual driving device 40 so that a selected operator can manually drive the vehicle 10 in remote manual driving mode.

[0106] Figure 11 is a sixth flowchart illustrating an example of an information processing method performed by the information processing system 1 in Figure 1. Figure 11 shows an example of a series of processing steps in the information processing device 30 when the vehicle 10 deviates from its ODD and the corresponding level setting process and the operator selection process for the vehicle 10 are performed in the in-vehicle manual driving mode.

[0107] In step S601, the control unit 33 of the information processing device 30 determines that the difficulty level is relatively high, "3," in step S405 of Figure 9, or executes the process in step S404 of Figure 9, and sets the manual driving mode of the vehicle 10 to the in-vehicle manual driving mode.

[0108] In step S602, the control unit 33 of the information processing device 30 sets the response level for manual driving based on evaluation information when switching from automatic driving mode to manual driving mode. For example, the control unit 33 sets the response level in the in-vehicle manual driving mode using information on the weather and road conditions of the vehicle 10 included in the second evaluation information. Thus, the control unit 33 may set the response levels using different types of evaluation information for the in-vehicle manual driving mode and the remote manual driving mode shown in Figure 10.

[0109] In step S603, the control unit 33 of the information processing device 30 refers to the operator's skill level included in the database pre-stored in the storage unit 32.

[0110] In step S604, the control unit 33 of the information processing device 30 selects an operator to whom manual operation of the vehicle 10 is permitted based on the correspondence level set in step S602 and the operator's skill level referenced in step S603. For example, the control unit 33 selects an operator whose skill level is equal to or higher than the correspondence level as an operator to whom manual operation of the vehicle 10 is permitted.

[0111] In step S605, the control unit 33 of the information processing device 30 determines whether it has received third response information indicating that manual driving is difficult from the operator selected in step S604 of the in-vehicle manual driving mode. For example, the operator remotely checks the status of the vehicle 10's ODD deviation using the output unit of the in-vehicle manual driving device 50, or actually goes to the scene of the vehicle 10's ODD deviation to directly check the situation, and inputs information indicating that manual driving is difficult based on their own judgment using the input unit of the in-vehicle manual driving device 50. When the control unit 33 receives this input information from the in-vehicle manual driving device 50, it determines that it has received third response information. When the control unit 33 determines that it has received third response information, it executes the process in step S607. When the control unit 33 determines that it has not received third response information, it executes the process in step S606.

[0112] In step S606, if the control unit 33 of the information processing device 30 determines that it has not received the third response information in step S605, it initiates an action. For example, the control unit 33 may control the vehicle of the response team 4 so that it can move to the scene of vehicle 10 so that a selected operator can directly manually drive vehicle 10 using the in-vehicle manual driving device 50. For example, the control unit 33 may guide the vehicle to the scene of vehicle 10 by automatic driving, or it may display the route to the scene of vehicle 10 on a monitor installed inside the vehicle to support the operator in driving the vehicle to the scene.

[0113] In step S607, if the control unit 33 of the information processing device 30 determines in step S605 that it has received third response information from the operator selected in the in-vehicle manual driving mode indicating that it is difficult to perform manual driving, it requests at least one of the following: towing vehicle 10 or introducing a replacement vehicle for vehicle 10. For example, the control unit 33 requests both towing vehicle 10 and introducing a replacement vehicle.

[0114] Figure 12 is a seventh flowchart illustrating an example of an information processing method performed by the information processing system 1 of Figure 1. Figure 12 shows an example of a series of processing steps performed by a server or terminal device on the management company side that manages, for example, a tow truck for towing vehicle 10 and a replacement vehicle for vehicle 10. In this disclosure, "management company" may be, for example, the company to which the response team 4 belongs, or it may be another company unrelated to the response team 4.

[0115] In step S701, the management company's device receives a request for at least one of the following: towing vehicle 10 or providing a replacement vehicle for vehicle 10, either when the process in step S402 in Figure 9 or the process in step S607 in Figure 11 is executed. For example, the device may receive requests for both towing vehicle 10 and providing a replacement vehicle.

[0116] In step S702, the management company's equipment selects an operator to move at least one of the tow truck and / or replacement vehicle to the site of vehicle 10, in response to the request received in step S701.

[0117] In step S703, the management company's device causes the operator selected in step S702 to carry out the necessary actions. For example, the device may control a tow truck or a replacement vehicle so that the selected operator can move to the site of vehicle 10 to perform the required work. For example, the device may guide the vehicle to the site of vehicle 10 by autonomous driving, or it may display the route to the site of vehicle 10 on a monitor installed inside the vehicle to assist the operator in driving the vehicle to the site.

[0118] (effect) According to the information processing device 30 of the above embodiment, the technology for switching the vehicle 10 from automatic driving to manual driving is improved. When switching from automatic driving mode to manual driving mode, the information processing device 30 selects an operator who is authorized to manually drive the vehicle 10 based on the response level and the operator's skill level. As a result, the information processing device 30 can assign an operator who is capable of handling the vehicle 10's ODD deviation situation to manual driving. The assigned operator can appropriately handle the vehicle 10's ODD deviation situation to manual driving.

[0119] The information processing device 30 can, for example, appropriately assign operators to remotely or manually operate vehicles 10 that have deviated from the ODD, even when the operation management center 3 manages the automated operation of multiple vehicles 10. The information processing device 30 can also efficiently assign operators to each request, even when multiple requests for action occur simultaneously in a large-scale operation management center 3.

[0120] The information processing device 30 selects operators whose skill level is equal to or higher than the corresponding level as operators authorized to operate manually. This allows the information processing device 30 to assign operators who can appropriately respond to the vehicle 10's ODD deviation situation.

[0121] When the information processing device 30 receives the first response information from the selected operator, it re-selects an operator with a higher skill level than the selected operator as the operator authorized to operate manually. This allows the information processing device 30 to assign a new operator who can appropriately deal with the situation of the vehicle 10's ODD deviation.

[0122] The information processing device 30 sets the in-vehicle manual driving mode if the difficulty level is above a first predetermined value, and sets the remote manual driving mode if the difficulty level is below the first predetermined value. This allows the information processing device 30 to set the in-vehicle manual driving mode for situations that are considered to be more difficult to deal with. The information processing device 30 can also have the operator directly take manual control of the vehicle 10 at the scene of the vehicle 10's ODD deviation, allowing them to take appropriate action depending on the situation of the vehicle 10's ODD deviation.

[0123] When the information processing device 30 receives second response information from the operator selected in remote manual driving mode, it re-selects an operator to perform manual driving in in-vehicle manual driving mode. This allows the information processing device 30 to have the operator directly perform manual driving of the vehicle 10 at the scene of the ODD deviation, even in situations that are difficult to handle in remote manual driving mode, and to take appropriate action depending on the situation of the vehicle 10's ODD deviation.

[0124] When the information processing device 30 receives third response information from an operator selected in the in-vehicle manual driving mode, it requests at least one of the following: towing vehicle 10 or introducing a replacement vehicle for vehicle 10. This allows the information processing device 30 to deal with the vehicle 10's ODD deviation situation by alternative means, such as towing vehicle 10 or introducing a replacement vehicle for vehicle 10, even if it is difficult for the operator to deal with the situation by manually driving vehicle 10.

[0125] The information processing device 30 sets the response level using different types of evaluation information for the in-vehicle manual driving mode and the remote manual driving mode. This allows the information processing device 30 to set the response level appropriately according to each mode.

[0126] When the information processing device 30 receives correction information from the administrator of the operation management center 3 to modify the evaluation information acquired from the vehicle 10, it modifies the evaluation information based on the correction information. This allows the administrator of the operation management center 3 to modify the numerical values ​​of each evaluation item included in the evaluation information to more appropriate values ​​based on their own judgment, such as their extensive experience. As a result, the information processing device 30 can also improve the accuracy of the evaluation information.

[0127] The information processing device 30 requests the dispatch of an ambulance if the level of risk included in the evaluation information is equal to or greater than the second predetermined value. This allows the information processing device 30 to appropriately address situations where there is a high risk of injury to the occupants of the vehicle 10.

[0128] The information processing device 30 communicates the status of the ODD deviation to at least one of the government agencies and local authorities, depending on the degree of impact included in the evaluation information. This allows the information processing device 30 to transmit the necessary information to the appropriate target organizations according to the level of social impact related to the ODD deviation of the vehicle 10's autonomous driving.

[0129] When the information processing device 30 determines that the wireless communication status of the vehicle 10 is good, it performs the response level setting process and the operator selection process. As a result, the information processing device 30 can perform the above-mentioned response level setting process and operator selection process after confirming that a communication connection has been established between the operation management center 3 and the vehicle 10.

[0130] If the information processing device 30 determines that the wireless communication status is not good, it requests an operator to rush to the vehicle 10 to perform manual driving. This allows the information processing device 30 to have an operator directly perform manual driving of the vehicle 10 at the scene of the ODD deviation, even if a communication connection has not been established between the operation management center 3 and the vehicle 10, and to appropriately address the situation of the vehicle 10's ODD deviation.

[0131] Vehicle 10 provides evaluation information to the information processing device 30 if the urgency level included in the evaluation information is equal to or greater than the third predetermined value. This allows vehicle 10 to provide evaluation information to the information processing device 30 only when the urgency level is high. Therefore, the information processing device 30 can only accept requests for action and execute a series of processes related to the action when the urgency level is high. As a result, the processing load on the information processing device 30 can be reduced.

[0132] (modified version) While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art can make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions included in each configuration or step can be rearranged in a logically consistent manner, and multiple configurations or steps can be combined into one or divided.

[0133] For example, it is also possible to configure a general-purpose electronic device such as a smartphone or computer to function as one of the devices in the information processing system 1 according to the above-described embodiment. Specifically, a program describing the processing content that realizes each function of the information processing device 30, etc., according to the embodiment is stored in the memory of the electronic device, and the processor of the electronic device reads and executes the program. Therefore, this disclosure can also be implemented as a program that can be executed by a processor.

[0134] Alternatively, the disclosure may also be realized as a non-temporary computer-readable medium storing a program executable by one or more processors for causing an information processing device 30 or the like to perform each function according to one embodiment. It should be understood that these are also included within the scope of the disclosure.

[0135] For example, at least some of the processing operations performed in the information processing device 30 in the above-described embodiment may also be performed in at least one of the vehicle 10, the determination device 20, the remote manual driving device 40, and the in-vehicle manual driving device 50.

[0136] In addition to the processing in the above embodiment, the control unit 33 of the information processing device 30 may identify an operator who intends to operate the manual driving device of the vehicle 10 used for manual driving, and if the identified operator's skill level does not meet the corresponding level, it may determine that the operator is not authorized to operate the vehicle manually. In this disclosure, "manual driving device" may include a remote manual driving device 40 or an in-vehicle manual driving device 50.

[0137] For example, the control unit 33 may identify the operator using an imaging module or a biometric authentication module included in the input unit 43 of the remote manual driving device 40. The control unit 33 may read a skill bell associated with the identification information of the identified operator from a database stored in the storage unit 32 and compare it with the corresponding level. The same applies to the in-vehicle manual driving device 50. If the control unit 33 determines that the operator is not authorized to drive manually, it may lock the functions of the manual driving device and control the manual driving device so that manual driving of the vehicle 10 is impossible.

[0138] Through the processing described above, the information processing device 30 can easily identify an individual attempting to operate the manual driving controls of the vehicle 10 and determine whether or not that person is authorized to operate the vehicle. The information processing device 30 can also make it impossible for unauthorized persons to manually drive the vehicle 10, thereby preventing hijacking of the vehicle 10 and improving security. As a result, the information processing device 30 can also ensure the safety of the occupants of the vehicle 10.

[0139] In addition, the information processing device 30 can also detect, for example, the operator's physical condition using an imaging module or a biometric authentication module for the manual driving device. If the information processing device 30 determines that it is difficult for the operator to continue manual driving due to physical condition, it can also stop and fix the vehicle 10. As a result, the information processing device 30 can avoid situations in which the operator's driving operations are not performed properly and the occupants of the vehicle 10 are put at risk.

[0140] In the above embodiment, it was explained that the information processing device 30 selects operators whose skill level is equal to or higher than the corresponding level as operators authorized to operate manually, but this is not limited to this. The information processing device 30 may, in some cases, select operators whose skill level is below the corresponding level as operators authorized to operate manually.

[0141] In the above embodiment, it was explained that the information processing device 30 receives first response information from the selected operator indicating that manual operation is difficult, but it is not limited to this. The information processing device 30 does not have to receive first response information. When the information processing device 30 receives first response information, it was explained that it re-selects an operator with a higher skill level than the selected operator as the operator authorized to perform manual operation, but it is not limited to this. The information processing device 30 does not have to perform such a re-selection process.

[0142] In the above embodiment, it was explained that the information processing device 30 sets the in-vehicle manual driving mode when the difficulty level is equal to or greater than a first predetermined value, and sets the remote manual driving mode when the difficulty level is less than the first predetermined value, but it is not limited to this. The information processing device 30 may perform a similar setting process based on other evaluation items included in the evaluation information, instead of or in addition to the difficulty level.

[0143] In the above embodiment, it was explained that the information processing device 30 receives second response information from the operator selected in the remote manual driving mode indicating that it is difficult to perform manual driving, but it is not limited to this. The information processing device 30 does not have to receive second response information. It was explained that when the information processing device 30 receives second response information, it re-selects the operator who will perform manual driving in the in-vehicle manual driving mode as an operator authorized to perform manual driving, but it is not limited to this. The information processing device 30 does not have to perform such re-selection processing.

[0144] In the above embodiment, it was explained that the information processing device 30 receives third response information from the operator selected in the in-vehicle manual driving mode indicating that manual driving is difficult, but it is not limited to this. The information processing device 30 does not have to receive third response information. It was explained that when the information processing device 30 receives third response information, it requests at least one of towing the vehicle 10 or introducing a replacement vehicle for the vehicle 10, but it is not limited to this. The information processing device 30 does not have to perform such a request process.

[0145] In the above embodiment, the information processing device 30 was described as using different types of evaluation information to set the corresponding level for the in-vehicle manual driving mode and the remote manual driving mode, but it is not limited to this. The information processing device 30 may also use the same type of evaluation information to set the corresponding level for the in-vehicle manual driving mode and the remote manual driving mode.

[0146] In the above embodiment, the information processing device 30 was described as setting the response level in the remote manual driving mode using information on the communication environment, weather environment, and road environment of the vehicle 10 included in the evaluation information, but it is not limited to this. The information processing device 30 may set the response level in the remote manual driving mode using other evaluation items included in the evaluation information in place of, or in addition to, at least a part of the communication environment, weather environment, and road environment.

[0147] In the above embodiment, the information processing device 30 was described as setting the response level in the in-vehicle manual driving mode using information on the weather and road conditions of the vehicle 10 included in the evaluation information, but it is not limited to this. The information processing device 30 may also set the response level in the in-vehicle manual driving mode using other evaluation items included in the evaluation information in place of, or in addition to, at least one of the weather and road conditions.

[0148] In the above embodiment, it was explained that the information processing device 30 receives correction information from the administrator of the operation management center 3 to correct the evaluation information acquired from the vehicle 10, but it is not limited to this. The information processing device 30 does not have to accept correction information. It was explained that the information processing device 30 corrects the evaluation information based on the correction information, but it is not limited to this. The information processing device 30 does not have to perform the evaluation information correction process.

[0149] In the above embodiment, the information processing device 30 was described as requesting the dispatch of an ambulance if the level of danger is equal to or greater than a second predetermined value, but it is not limited to this. The information processing device 30 may perform a similar request process based on other evaluation items included in the evaluation information, instead of or in addition to the level of danger. Alternatively, the information processing device 30 may not perform the process of requesting the dispatch of an ambulance.

[0150] In the above embodiment, the information processing device 30 was described as informing at least one of the government agencies and local authorities of the status of the ODD deviation, depending on the degree of impact, but is not limited to this. The information processing device 30 may perform a similar notification process based on other evaluation items included in the evaluation information, instead of or in addition to the degree of impact. Alternatively, the information processing device 30 may not perform any process to notify the status of the ODD deviation.

[0151] In the above embodiment, it was explained that the information processing device 30 performs a response level setting process and an operator selection process when it determines that the wireless communication status of the vehicle 10 is good based on the communication environment and the vehicle environment, but it is not limited to this. The information processing device 30 may determine the wireless communication status of the vehicle 10 based on other evaluation items included in the evaluation information, instead of or in addition to at least one of the communication environment and the vehicle environment. Alternatively, the information processing device 30 may not perform a process to determine the wireless communication status of the vehicle 10 before performing the response level setting process and the operator selection process.

[0152] In the above embodiment, it was explained that if the information processing device 30 determines that the wireless communication status is not good, it requests the operator performing manual operation to rush to the vehicle 10, but it is not limited to this. The information processing device 30 does not have to perform such a request process.

[0153] In the above embodiment, it was explained that the vehicle 10 provides evaluation information to the information processing device 30 if the urgency level is equal to or greater than a third predetermined value, but this is not limited to this. The vehicle 10 may perform a similar provision process based on other evaluation items included in the evaluation information, instead of or in addition to the urgency level. Alternatively, the vehicle 10 may not perform the process of providing evaluation information to the information processing device 30. For example, the vehicle 10 may provide only the measurement data of the sensor 12A to the information processing device 30. In this case, the information processing device 30 may generate and acquire evaluation information based on the measurement data acquired from the vehicle 10.

[0154] In the above embodiment, it was explained that when the vehicle 10 determines that it has deviated from the ODD of autonomous driving, it starts the process of acquiring evaluation information, but it is not limited to this. The vehicle 10 may start the process of acquiring evaluation information based on any other process, either in lieu of or in addition to the process of determining the deviation from the ODD of autonomous driving.

[0155] In the above embodiment, the operation management center 3 was described as having one remote manual driving device 40, but is not limited thereto. If the operating area is extensive, the operation management center 3 may have multiple remote manual driving devices 40 for each area. The remote manual driving devices 40 may also be installed in a rescue center where the response team 4 is on standby.

[0156] In the above embodiment, the vehicle 10 on which the determination device 20 is mounted was described as an automobile, but it is not limited to this. The determination device 20 may be mounted on other forms of mobility, such as an autonomous motorboat or a drone.

[0157] In the above embodiment, vehicle 10 was described as including an automobile having Level 4 autonomous driving capabilities, but is not limited thereto. Vehicle 10 may include an automobile having autonomous driving capabilities at a level other than Level 4, or it may include an automobile driven by a normal driver.

[0158] In the above embodiment, it was explained that the vehicle 10 does not have a mechanical operating section for manual driving, but rather has a temporary in-vehicle manual driving device 50 installed. However, the embodiment is not limited to this. Even if the vehicle 10 is, for example, an automobile with Level 4 autonomous driving capabilities, it may still have a driver's seat with a mechanical operating section permanently installed.

[0159] Some embodiments of the present disclosure are described below. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Note 1] A control unit is provided that acquires evaluation information based on measurement data from sensors installed in a vehicle having an automatic driving mode and a manual driving mode that allows manual driving by an operator, sets a response level for the manual driving based on the evaluation information when switching from the automatic driving mode to the manual driving mode, and selects an operator to whom the manual driving of the vehicle is permitted based on the response level and the operator's skill level. Information processing device. [Note 2] The information processing device described in Appendix 1, The control unit selects the operator whose skill level is equal to or higher than the corresponding level as the operator authorized to perform the manual operation. Information processing device. [Note 3] An information processing device as described in Appendix 1 or 2, When the control unit receives the first response information from the selected operator, it re-selects an operator with a higher skill level than the selected operator as the operator authorized to perform the manual operation. Information processing device. [Note 4] An information processing device described in any one of the appendices 1 to 3, The control unit identifies the operator who intends to operate the manual driving device of the vehicle used for manual driving, and if the identified operator's skill level does not meet the corresponding level, it determines that the operator is not authorized to perform the manual driving. Information processing device. [Note 5] An information processing device described in any one of the appendices 1 to 4, The control unit sets an in-vehicle manual driving mode, in which the manual driving is performed inside the vehicle, as the manual driving mode if the difficulty level of the manual driving included in the evaluation information is equal to or greater than a first predetermined value, and sets a remote manual driving mode, in which the manual driving is performed outside the vehicle, as the manual driving mode if the difficulty level is less than the first predetermined value. Information processing device. [Note 6] The information processing device described in Appendix 5, When the control unit receives second response information from the operator selected in the remote manual driving mode, it re-selects the operator who will perform the manual driving in the in-vehicle manual driving mode as the operator authorized to perform the manual driving. Information processing device. [Note 7] An information processing device as described in Appendix 5 or 6, When the control unit receives third response information from the operator selected in the in-vehicle manual driving mode, it requests at least one of the following: towing the vehicle or introducing a replacement vehicle for the vehicle. Information processing device. [Note 8] An information processing device described in any one of the appendices 5 to 7, The control unit sets the corresponding level using different types of evaluation information for the in-vehicle manual driving mode and the remote manual driving mode. Information processing device. [Note 9] The information processing device described in Appendix 8, The control unit sets the corresponding level in the remote manual driving mode using the information on the vehicle's communication environment, weather environment, and road environment included in the evaluation information. Information processing device. [Note 10] An information processing device as described in Appendix 8 or 9, The control unit sets the corresponding level in the in-vehicle manual driving mode using the weather and road environment information of the vehicle included in the evaluation information. Information processing device. [Note 11] An information processing device described in any one of the appendices 1 to 10, When the control unit receives correction information from the administrator of the operation management center that manages the operation of the vehicle, which corrects the evaluation information acquired from the vehicle, it corrects the evaluation information based on the correction information. Information processing device. [Note 12] An information processing device described in any one of the appendices 1 to 11, The control unit requests the dispatch of an ambulance if the risk level included in the evaluation information, which is categorized into levels according to the degree of injury risk to the occupants of the vehicle, is equal to or greater than a second predetermined value. Information processing device. [Note 13] An information processing device described in any one of the appendices 1 to 12, The control unit communicates the status of the ODD deviation to at least one of the government agencies and local authorities, according to the impact level included in the evaluation information, which indicates the level of social impact related to the ODD deviation of the vehicle's autonomous driving. Information processing device. [Note 14] An information processing device described in any one of the appendices 1 to 13, When the control unit determines that the vehicle's wireless communication status is good based on the vehicle's communication environment and vehicle environment included in the evaluation information, it executes the corresponding level setting process and the operator selection process. Information processing device. [Note 15] The information processing device described in Appendix 14, If the control unit determines that the wireless communication status is not good, it requests the operator performing the manual operation to rush to the vehicle. Information processing device. [Note 16] An information processing device described in any one of the appendices 1 to 15, A vehicle that acquires measurement data using the sensor, acquires evaluation information based on the acquired measurement data and provides it to the information processing device, Equipped with, Information processing system. [Note 17] An information processing method performed by an information processing device, To acquire evaluation information based on measurement data from sensors installed in a vehicle that has both an automated driving mode and a manual driving mode that allows manual operation by an operator, When switching from the automatic driving mode to the manual driving mode, the response level for the manual driving is set based on the evaluation information, Selecting an operator to whom manual operation of the vehicle is permitted based on the aforementioned response level and the operator's skill level, including, Information processing methods. [Note 18] A vehicle having an automatic driving mode and a manual driving mode that allows manual operation by an operator, Sensors and, A control unit that acquires evaluation information based on the measurement data of the aforementioned sensor, Equipped with, The control unit, when switching from the automatic driving mode to the manual driving mode, sets a response level for the manual driving based on the evaluation information and provides the evaluation information to an information processing device that selects an operator to whom the manual driving of the vehicle is permitted based on the response level and the operator's skill level. vehicle. [Note 19] The vehicles listed in Appendix 18, The control unit provides the information processing device with the evaluation information if the urgency included in the evaluation information, which is associated with the estimated time for the vehicle to return to operation by autonomous driving, is equal to or greater than a third predetermined value. vehicle. [Note 20] Vehicles described in Appendix 18 or 19, When the control unit determines that the vehicle has deviated from the ODD for autonomous driving, it starts the process of acquiring the evaluation information. vehicle. [Explanation of symbols]

[0160] 1. Information Processing System 2 Network 3. Operations Management Center 4. Emergency response team 10 vehicles 11 Communications Department 12 Measurement section 12A sensor 13 Storage section 14 Control Unit 20 Judgment device 21 Communications Department 22 Memory section 23 Control Unit 30 Information Processing Devices 31 Communications Department 32 Storage section 33 Control Unit 40 Remote manual operation device 41 Communications Department 42 Storage section 43 Input section 44 Output section 45 Control Unit 50 Manual driving controls inside the vehicle

Claims

1. A control unit is provided that acquires evaluation information based on measurement data from sensors installed in a vehicle having an automatic driving mode and a manual driving mode that allows manual driving by an operator, sets a response level for the manual driving based on the evaluation information when switching from the automatic driving mode to the manual driving mode, and selects an operator to whom the manual driving of the vehicle is permitted based on the response level and the operator's skill level. Information processing device.

2. An information processing apparatus according to claim 1, The control unit selects the operator whose skill level is equal to or higher than the corresponding level as the operator authorized to perform the manual operation. Information processing device.

3. An information processing apparatus according to claim 1, When the control unit receives the first response information from the selected operator, it re-selects an operator with a higher skill level than the selected operator as the operator authorized to perform the manual operation. Information processing device.

4. An information processing apparatus according to any one of claims 1 to 3, The control unit identifies the operator who intends to operate the manual driving device of the vehicle used for manual driving, and if the identified operator's skill level does not meet the corresponding level, it determines that the operator is not authorized to perform the manual driving. Information processing device.

5. An information processing apparatus according to any one of claims 1 to 3, The control unit sets an in-vehicle manual driving mode, in which the manual driving is performed inside the vehicle, as the manual driving mode if the difficulty level of the manual driving included in the evaluation information is equal to or greater than a first predetermined value, and sets a remote manual driving mode, in which the manual driving is performed outside the vehicle, as the manual driving mode if the difficulty level is less than the first predetermined value. Information processing device.

6. An information processing device according to claim 5, When the control unit receives second response information from the operator selected in the remote manual driving mode, it re-selects the operator who will perform the manual driving in the in-vehicle manual driving mode as the operator authorized to perform the manual driving. Information processing device.

7. An information processing device according to claim 5, When the control unit receives third response information from the operator selected in the in-vehicle manual driving mode, it requests at least one of the following: towing the vehicle or introducing a replacement vehicle for the vehicle. Information processing device.

8. An information processing device according to claim 5, The control unit sets the corresponding level using different types of evaluation information for the in-vehicle manual driving mode and the remote manual driving mode. Information processing device.

9. An information processing apparatus according to claim 8, The control unit sets the corresponding level in the remote manual driving mode using the information on the vehicle's communication environment, weather environment, and road environment included in the evaluation information. Information processing device.

10. An information processing apparatus according to claim 8, The control unit sets the corresponding level in the in-vehicle manual driving mode using the weather and road environment information of the vehicle included in the evaluation information. Information processing device.

11. An information processing apparatus according to any one of claims 1 to 3, When the control unit receives correction information from the administrator of the operation management center that manages the operation of the vehicle, which corrects the evaluation information acquired from the vehicle, it corrects the evaluation information based on the correction information. Information processing device.

12. An information processing apparatus according to any one of claims 1 to 3, The control unit requests the dispatch of an ambulance if the risk level included in the evaluation information, which is categorized into levels according to the degree of injury risk to the occupants of the vehicle, is equal to or greater than a second predetermined value. Information processing device.

13. An information processing apparatus according to any one of claims 1 to 3, The control unit communicates the status of the ODD deviation to at least one of the government agencies and local authorities, according to the degree of impact included in the evaluation information, which indicates the level of social impact related to the ODD deviation of the vehicle's autonomous driving. Information processing device.

14. An information processing apparatus according to any one of claims 1 to 3, When the control unit determines that the vehicle's wireless communication status is good based on the vehicle's communication environment and vehicle environment included in the evaluation information, it executes the corresponding level setting process and the operator selection process. Information processing device.

15. An information processing apparatus according to claim 14, If the control unit determines that the wireless communication status is not good, it requests the operator performing the manual operation to rush to the vehicle. Information processing device.

16. An information processing device according to any one of claims 1 to 3, A vehicle that acquires measurement data using the sensor, acquires evaluation information based on the acquired measurement data and provides it to the information processing device, Equipped with, Information processing system.

17. An information processing method performed by an information processing device, To acquire evaluation information based on measurement data from sensors installed in a vehicle that has both an automated driving mode and a manual driving mode that allows manual operation by an operator, When switching from the automatic driving mode to the manual driving mode, the response level for the manual driving is set based on the evaluation information, Selecting an operator to whom manual operation of the vehicle is permitted based on the aforementioned response level and the operator's skill level, including, Information processing methods.

18. A vehicle having an automatic driving mode and a manual driving mode that allows manual operation by an operator, Sensors and, A control unit that acquires evaluation information based on the measurement data of the aforementioned sensor, Equipped with, The control unit, when switching from the automatic driving mode to the manual driving mode, sets the response level for the manual driving based on the evaluation information and provides the evaluation information to an information processing device that selects the operator to be authorized to manually drive the vehicle based on the response level and the operator's skill level. vehicle.

19. A vehicle according to claim 18, The control unit provides the information processing device with the evaluation information, and the urgency included in the evaluation information, which is associated with the estimated time for the vehicle to return to operation by autonomous driving, is equal to or greater than a third predetermined value. vehicle.

20. A vehicle according to claim 18 or 19, When the control unit determines that the vehicle has deviated from the ODD of autonomous driving, it starts the process of acquiring the evaluation information. vehicle.

Images