Control system for unmanned vehicle and control method

Abstract

To provide a technique for appropriately keeping an indoor environment while reducing a burden on a remote supporter in a driverless vehicle.SOLUTION: Provided is a control system for an unmanned vehicle which eliminates the need for a driver or a conductor to get on the vehicle. In the control system, based on information of items relating to an indoor environment of the vehicle, an index relating to maintenance of the indoor environment is calculated, and it is determined whether or not the index is equal to or more than a threshold. In a case where the index is less than the threshold, a predetermined message is reported using an indoor reporting device of the vehicle and in a case where the index is equal to or more than the threshold, a remote support request is transmitted to a remote support terminal that remotely supports the vehicle.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to systems and methods for controlling unmanned vehicles that do not require a driver or conductor on board. [Background technology]

[0002] Patent Document 1 discloses technology related to remote monitoring of an autonomous vehicle. The autonomous vehicle is equipped with an autonomous sensor including at least a camera. If an obstacle is detected by the autonomous sensor, the autonomous vehicle is automatically stopped. After the automatic stop, if a transmission signal is received from a remote monitoring center that remotely monitors the autonomous vehicle, the autonomous vehicle is allowed to resume traveling. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2019-087015 Summary of the Invention [Problem to be solved by the invention]

[0004] Driverless vehicles are known that do not require a driver or conductor on board. Consider providing a passenger transportation service using driverless vehicles. During the transportation service, situations may arise where the interior environment of the vehicle is not maintained properly, such as when passengers cause a nuisance. While a conductor can respond to such situations in vehicles with an onboard conductor, a driverless vehicle does not have a conductor on board at all times. Furthermore, a remote support person is on standby outside the vehicle in case of an incident that cannot be handled by the vehicle itself. However, if remote support is constantly required when an incident occurs that affects the maintenance of the interior environment, there is a possibility that there will not be enough remote support people.

[0005] One object of the present disclosure is to provide a technology for appropriately maintaining the interior environment in a driverless vehicle while reducing the burden on a remote assistant. [Means for solving the problem]

[0006] The present disclosure provides a control system for an unmanned vehicle that does not require a driver or conductor on board. The control system of the present disclosure includes one or more processors. The one or more processors calculate an index for maintaining the interior environment based on information on items related to the interior environment of the vehicle, determine whether the index is equal to or greater than a threshold, and if the index is less than the threshold, notify the vehicle of a predetermined message using a notification device in the vehicle. If the index is equal to or greater than the threshold, send a remote assistance request to a remote assistance terminal that provides remote assistance for the vehicle.

[0007] The present disclosure provides a control method for an unmanned vehicle that does not require a driver or conductor on board. The control method of the present disclosure includes calculating an index related to maintaining an interior environment based on information on items related to the interior environment of the vehicle, determining whether the index is equal to or greater than a threshold, notifying the vehicle owner of a predetermined message using a notification device in the vehicle if the index is less than the threshold, and transmitting a remote assistance request to a remote assistance terminal that provides remote assistance for the vehicle if the index is equal to or greater than the threshold. [Effects of the Invention]

[0008] According to the technology disclosed herein, an index relating to the maintenance of an interior environment is calculated, and if the index is below a threshold, a predetermined message is sent from a notification device to the interior of the vehicle. By sending the predetermined message, it is possible that the situation affecting the maintenance of the interior environment will be resolved. On the other hand, if the index is equal to or greater than the threshold, a remote assistance request is sent. Appropriate measures can be taken according to the index without always requesting remote assistance.

[0009] In this way, the technology of the present disclosure makes it possible to reduce the frequency of requests for remote assistance while appropriately responding to situations that affect the maintenance of the indoor environment. [Brief explanation of the drawings]

[0010] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a control system according to an embodiment of the present invention. [Figure 2] 10 is a flowchart illustrating an example of processing by the control system. [Figure 3] 10 is a table showing an example of information for calculating an index related to maintaining an indoor environment. [Figure 4] FIG. 10 is a diagram for explaining a modified example. [Figure 5] 10 is a flowchart illustrating an example of processing in a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0011] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0012] 1. Overview of the control system FIG. 1 is a diagram showing an overview of a control system 20 according to this embodiment. The control system 20 controls a vehicle 1. The vehicle 1 is a driverless vehicle capable of autonomous driving. A driverless vehicle does not require an onboard driver to drive the vehicle 1 or to monitor the operation of the vehicle 1. A driverless vehicle is also called an unmanned vehicle because it does not have a driver or conductor on board.

[0013] Vehicle 1 is used to provide a mobility service to a user. For example, vehicle 1 may be a vehicle used as public transportation, a shared vehicle used by multiple users, or an unmanned taxi.

[0014] The control system 20 may be mounted on the vehicle 1. For example, the control system 20 may be configured by one or more ECUs (Electronic Control Units). Alternatively, the control system 20 may be mounted on an external device outside the vehicle 1. For example, the control system 20 may be part of a management server that manages the vehicle 1. Alternatively, the control system 20 may be distributed between the vehicle 1 and the external device.

[0015] The control system 20 includes one or more processors 21 (hereinafter simply referred to as processors 21) and one or more storage devices 22 (hereinafter simply referred to as storage devices 22). The processors 21 execute various processes. For example, the processors 21 include a CPU (Central Processing Unit). The storage devices 22 store various programs and various pieces of information. Examples of the storage devices 22 include volatile memory, non-volatile memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0016] The vehicle 1 is equipped with a notification device 11 and a sensor 12. The notification device 11 is a device that issues notifications to passengers 3 of the vehicle 1. In the example of FIG. 1, the notification device 11 is a speaker that issues notifications to passengers 3 by voice. As another example, the notification device 11 may be a display device that issues notifications to passengers 3 visually. The sensor 12 is a sensor that detects the status inside the vehicle 1. In the example of FIG. 1, the sensor 12 is an in-vehicle camera that captures images of the interior of the vehicle 1. As another example, the sensor 12 may be an odor sensor, an illuminance sensor, a temperature sensor, or the like. There may be multiple notification devices 11 and sensors 12. The control system 20 is configured to be able to communicate with each of the notification devices 11 and sensors 12.

[0017] The control system 20 is also connected to one or more remote support terminals 40 via a communication network. A remote supporter 4 is on standby at each remote support terminal 40. When the vehicle 1 requires remote support, for example when the vehicle 1 is no longer able to make decisions about driving on its own, a remote support request is issued from the control system 20, and the assigned remote supporter 4 operates the remote support terminal 40 to provide remote support.

[0018] Now, let us consider the indoor environment of the vehicle 1. The indoor environment is an important factor for passengers 3 using the transportation service, and it is desirable that it be maintained at a certain level or above. Hereinafter, in this specification, a situation that hinders the maintenance of the indoor environment, that is, a situation that may affect the indoor environment and impair the comfort of passengers 3, is referred to as a specific situation. For example, a situation in which the interior is too bright or too dark, a situation in which an unpleasant odor is emitted by passengers 3, a situation in which the indoor temperature is too low or too high, a situation in which some passengers are engaging in nuisance behavior, etc. Examples of nuisance behavior by passengers include smoking in a non-smoking vehicle, shouting, walking around the interior more than necessary, and eating and drinking in a room where eating and drinking are prohibited.

[0019] For example, in the example in Figure 1, the setting sun is shining into the interior of car 1, causing passenger 3 to feel dazzled. If a conductor were stationed in car 1, the conductor would be able to notice and respond if a specific situation occurs. For example, if the interior becomes excessively bright as in Figure 1, the conductor who notices this can close the curtains, thereby maintaining a comfortable interior environment again. However, in the case of an unmanned vehicle, the conductor is not necessarily on board car 1.

[0020] One way to deal with a specific situation is to improve the situation by remote support, that is, by having the remote supporter 4 take on the role of a conductor. For example, if the sensor 12 (illuminance sensor) detects that the interior of the vehicle is excessively bright, a remote support request can be sent from the vehicle 1 to request a response from the remote supporter 4. Upon receiving the remote support request, the remote supporter 4 can confirm that the interior of the vehicle is too bright from the image of the sensor 12 (in-vehicle camera), and can respond to the situation by operating the remote support terminal 40 to close the curtains inside the vehicle.

[0021] However, when providing remote support to vehicles 1, one remote supporter 4 is often responsible for multiple vehicles 1, and if remote support requests are frequently sent from each of the multiple vehicles 1, there is a possibility that the number of remote supporters 4 will be insufficient. In particular, the remote supporter 4 is also responsible for remote support related to the driving of the vehicle 1. It is undesirable for the remote support required to fulfill the role of conductor to strain resources available for more necessary remote support, such as support related to driving. Therefore, when a specific situation is detected, the control system 20 first determines whether the detected specific situation is a high-level emergency. Then, different measures are selected depending on the level of emergency.

[0022] 2. Processing example Fig. 2 is a flowchart showing an example of processing by the control system 20. The processing shown in the flowchart of Fig. 2 is realized, for example, by the processor 21 executing a program stored in the storage device 22. This series of processing is repeatedly executed at predetermined control cycles.

[0023] In step S110, processor 21 determines whether a specific situation has been detected. If a specific situation has been detected (step S110; Yes), the process proceeds to step S120. If a specific situation has not been detected (step S110; No), the process for this cycle ends.

[0024] The processor 21 can detect a specific situation based on information acquired from the sensor 12. For example, the processor 21 can detect a situation in which the interior of the vehicle is excessively bright by image analysis of an image captured by an in-vehicle camera. Alternatively, such a situation may be detected by an illuminance sensor. As another example, the processor 21 may detect an odor generated in the interior of the vehicle by an odor sensor. As yet another example, the processor 21 may detect a passenger engaging in nuisance behavior by image analysis of an image captured by an in-vehicle camera.

[0025] In step S120, the processor 21 calculates an index related to maintaining the indoor environment. Once the index is calculated, the process proceeds to step S130.

[0026] Information for calculating an index related to maintaining an indoor environment is stored in advance in the storage device 22. Fig. 3 is a table showing an example of information stored in the storage device 22 as information for calculating an index related to maintaining an indoor environment.

[0027] In the example of FIG. 3, items related to maintaining the indoor environment are set, and a score is set for each item according to the type of specific situation. The scores are set in advance so that the higher the item's urgency, the higher the score, and the lower the item's urgency, the lower the score. For example, if the interior is dark, passenger 3 may feel uncomfortable, but the situation is not one that will immediately harm passenger 3, so the score is set relatively low. If a passenger is smoking in a non-smoking car, the score is set relatively high because there is a risk of harm to the health of surrounding passengers. When the control system 20 detects a specific situation, it calculates an index based on these scores. For example, the sum of the scores corresponding to the specific situations detected within a predetermined time period is calculated as the index related to maintaining the indoor environment.

[0028] In step S130, processor 21 determines whether the index calculated in step S120 is equal to or greater than a threshold. The threshold is set in advance and stored in storage device 22. If the index is less than the threshold (step S130; No), the process proceeds to step S140. On the other hand, if the index is equal to or greater than the threshold (step S130; Yes), the process proceeds to step S150.

[0029] In step S140, the processor 21 plays an automatic message using the notification device 11. The automatic message is preset depending on the type of specific situation. For example, in a situation where the room is too bright, a message such as "The sun is shining in and it is dazzling. Please be careful" may be set.

[0030] In step S150, the processor 21 transmits a remote support request to the remote support terminal 40. Upon receiving the remote support request, the remote supporter 4 operates the remote support terminal 40 to provide remote support to resolve the specific situation. Once the remote support request is transmitted, the series of processes ends.

[0031] The effect of this series of processes will be described with an example. For example, even if sunlight shines in and the interior becomes dazzling, the problem may be resolved immediately by changing the position or posture of the vehicle 1. In such a case, the indicator for maintaining the interior environment is low, so an automatic message (e.g., "Please let us know from your device if you experience any inconvenience during your ride") is played to prompt the passenger 3 to report the occurrence of the specific situation. If the situation does not resolve within a predetermined time, the specific situation will be detected multiple times within the predetermined time, and the scores will be added up, increasing the indicator. In such a case, remote assistance is requested, and the remote supporter 4 can take action such as closing the curtains, allowing the situation to be dealt with appropriately. Furthermore, if a specific situation with a high score is detected, the indicator for maintaining the interior environment will also increase. Since such a situation is considered to be a high-urgency situation, a remote assistance request will be sent even if it is detected only once.

[0032] In this way, the control system 20 calculates an index using the score, making it possible to distinguish between situations requiring remote assistance and situations that can be handled with an automatic message. When the index related to maintaining the indoor environment is low and the situation is low in urgency, an automatic message is played instead of a request for remote assistance. This reduces the frequency with which remote assistance requests are sent, preventing a shortage of remote supporters 4. Furthermore, when the index related to maintaining the indoor environment is high and the situation is high in urgency, remote assistance is requested, making it possible to appropriately handle specific situations with high urgency.

[0033] 3. Whistleblowing by passengers 3-1. Internal reporting of nuisance behavior In the above example, the control system 20 detects a specific situation by acquiring information from the sensor 12. However, the detection of a specific situation may also be performed based on an internal report from the passenger 3. The internal report from the passenger 3 is performed through a user terminal. The user terminal is a mobile communication terminal owned by the passenger 3. Each user terminal is configured to be able to communicate with the control system 20. The passenger 3 can use any communication terminal as a user terminal. For example, a mobile communication terminal such as a smartphone, a smartwatch, or a tablet may be used as a user terminal by installing an application on it.

[0034] As an example, consider a specific situation where a passenger is engaging in nuisance behavior. Fig. 4 shows a situation where multiple passengers 3a and 3b are riding in vehicle 1. Although vehicle 1 is a non-smoking vehicle, passenger 3b is smoking inside vehicle 1. Passenger 3a, noticing passenger 3b's nuisance behavior (here, smoking), reports the incident to control system 20 internally via user terminal 30a that he or she carries.

[0035] For example, passenger 3a launches an application installed on user terminal 30a and inputs a request to make an internal report. Information for identifying passenger 3b who is engaging in nuisance behavior may be input into the application along with the request for internal reporting. The information for identifying passenger 3b does not necessarily need to pinpoint passenger 3b. For example, when the vehicle 1 is divided into several areas, the area in which passenger 3b is located may be selected, or the vehicle number in which passenger 3b is located may be input. Alternatively, the control system 20 may acquire an image from an in-vehicle camera in response to a request for internal reporting from passenger 3a and display the image on the user terminal 30a through an application. The passenger 3a may then be asked to select passenger 3b from the image by pointing. Passenger 3a can designate passenger 3b on the application, for example, by circling passenger 3b. Alternatively, the control system 20 may extract potential individuals from the in-vehicle camera image and ask passenger 3a to select the passenger 3b who is engaging in nuisance behavior from the extracted individuals. As another example, the control system 20 may identify the passenger 3b who is causing trouble by image analysis of images acquired from an in-vehicle camera.

[0036] The control system 20 that receives the internal report calculates an index relating to the maintenance of the indoor environment, and depending on the magnitude of the index, plays an automatic message or requests remote assistance.

[0037] The index for maintaining the indoor environment can be calculated using a score, similar to step S120 in FIG. 2. First, the processor 21 identifies nuisance behavior that hinders the maintenance of the indoor environment based on the internal report. Next, it acquires a score corresponding to the identified nuisance behavior. Then, it adds up the scores acquired within a certain period of time and calculates it as the index for maintaining the indoor environment. If there are multiple passengers 3 who have made internal reports, the scores acquired from those internal reports are added up. In other words, if there is an internal report about a nuisance behavior with a high level of urgency or if there are internal reports from multiple passengers 3, the index for maintaining the indoor environment will be large and it will be determined that the level of urgency is high. In such cases, a remote assistance request is immediately sent to the remote assistance terminal 40.

[0038] For example, consider a case where the threshold is set to 4. If there is an internal report that there is a smoker in the car, the index is 4, which is above the threshold, so a remote assistance request is sent immediately. If there is an internal report from one person that there is someone making noise in the car, the index is 2, which is below the threshold, so an automatic message is played first. Similarly, if there is an internal report from two people that there is someone making noise in the car, the index is 4, so a remote assistance request is sent immediately.

[0039] When a specific situation is detected based on an internal report by a passenger 3, it means that the passenger 3 actually wants the situation resolved. Therefore, the specific situation can be detected in a manner that is more in line with reality. Furthermore, since passengers 3 can use their own user devices, they can make internal reports without worrying about the passenger who is causing the nuisance finding out. Furthermore, when internal reports are made by multiple passengers 3, the scores are added together and an index is calculated that indicates a situation with a higher degree of urgency. This makes it possible to select whether to play an automatic message or respond with remote support depending on the situation.

[0040] 3-2. Multiple messages When a passenger 3 makes an internal report about a nuisance, the following modification is possible: Fig. 5 is a flowchart showing an example of processing performed by the processor 21 in this modification.

[0041] In steps S210 to S230, the same processes as in steps S110 to S130 in Fig. 3 are performed. If the index is less than the threshold in step S230, the process proceeds to step S240.

[0042] In step S240, an automated message is played back, similar to step S140 in FIG. 3. However, here an "abstract message" is played back. The abstract message corresponds to two or more items among the items indicating the types of nuisance behavior. For example, a message such as "Please let us know if you see any nuisance behavior" is set as the abstract message. By playing an abstract message first as the automated message, it is possible to gently alert everyone without suddenly irritating the passenger who is engaging in nuisance behavior.

[0043] In step S250, processor 21 determines whether the nuisance has ended. Whether the nuisance has ended can be determined by image analysis of images captured by the in-car camera. As another example, a message "Has the nuisance ended?" may be sent to passenger terminal 30, and the determination may be made based on the response from passenger 3. If the nuisance has ended (step S250; Yes), the process ends. On the other hand, if the nuisance has not ended (step S250; No), the process proceeds to step S260.

[0044] In step S260, processor 21 again plays an automatic message. However, unlike step S240, a "specific message" is played here. The specific message is a message that is set depending on the type of nuisance behavior. For example, if the nuisance behavior is making noise indoors, the specific message played back is "Please do not make noise inside the car." By playing back the specific message, passengers can be clearly warned. Once the specific message has been played back, processing proceeds to step S270.

[0045] In step S270, processor 21 determines whether the passenger's nuisance behavior has ended. The determination method is the same as in step S250. If the passenger's nuisance behavior has ended (step S270; Yes), the series of processes ends. If the passenger's nuisance behavior has not ended (step S270; No), the process proceeds to step S280.

[0046] In step S280, the processor 21 transmits a remote support request to the remote support terminal 40. Upon receiving the remote support request, the remote supporter 4 deals with the nuisance behavior by making an announcement appropriate to the individual situation. At this time, the remote supporter 4 may make an announcement that clearly identifies the passenger who is causing the nuisance, such as "People wearing green clothes, please be quiet on the train."

[0047] In step S290, the processor 21 determines whether the nuisance behavior has ended. The determination may be made in the same manner as in step S250. Alternatively, the remote supporter 4 may determine whether the nuisance behavior has ended based on images from an in-car camera, and the determination may be made by acquiring the determination result entered by the remote supporter 4. If the nuisance behavior by the passenger has ended (step S290; Yes), the series of processes ends. If the nuisance behavior by the passenger has not ended (step S290; No), the process returns to step S280, and a remote support request is sent again.

[0048] In this way, in this modified example, the automatic message includes two types of messages, an abstract message and a specific message, and the abstract message is played before the specific message. At first, the passenger who is causing trouble is gently warned, and if the passenger still does not stop the trouble, a specific message corresponding to the individual situation is played. This makes it possible to warn the passenger who is causing trouble appropriately while avoiding suddenly provoking them.

[0049] Furthermore, the same applies to this modified example in that remote support is requested when the indicators related to maintaining the indoor environment are high, that is, when the level of urgency is high. While reducing the frequency with which remote support requests are sent and reducing the burden on the remote supporter 4, remote support can be requested when necessary, and an appropriate response from the remote supporter 4 can be requested.

[0050] 4.Effects As described above, the control system according to this embodiment eliminates the need to constantly rely on remote support when a situation arises that hinders the maintenance of the indoor environment. It is possible to take appropriate measures to resolve the situation while reducing the burden on the remote supporter. [Explanation of symbols]

[0051] 1 vehicle, 3 passenger, 4 remote supporter, 11 notification device, 12 sensor, 20 control system, 21 processor, 22 storage device, 30a user terminal, 40 remote support terminal

Claims

1. A control system for an unmanned vehicle that does not require a driver or conductor to be on board, One or more processors; The one or more processors: Calculating an index relating to maintenance of the indoor environment based on information on items relating to the indoor environment of the unmanned vehicle; determining whether the index is equal to or greater than a threshold; If the index is less than a threshold value, a predetermined message is notified using a notification device in the interior of the unmanned vehicle; If the index is equal to or greater than a threshold, a remote support request is transmitted to a remote support terminal that performs remote support for the unmanned vehicle. Control system.

2. 2. The control system of claim 1, The information on the items related to the indoor environment is obtained from an internal report transmitted from a mobile terminal of a passenger of the unmanned vehicle, For the items related to the indoor environment, a score is set corresponding to a nuisance behavior that hinders the maintenance of the indoor environment, The one or more processors: Identifying nuisance behavior that hinders maintenance of the indoor environment based on the internal report; The index is calculated based on the total score obtained within a certain period of time. Control system.

3. 3. The control system of claim 2, The items related to the indoor environment include a plurality of items, the predetermined message includes a specific message corresponding to each of the plurality of items and an abstract message including at least two of the plurality of items; The one or more processors: If there is an item among the plurality of items for which the index is less than a threshold value, The notification of the abstract message is prioritized over the notification of the concrete message. Control system.

4. 2. The control system of claim 1, The information on the items related to the indoor environment is acquired from a camera image obtained by a camera of the unmanned vehicle capturing an image of the interior of the unmanned vehicle, For the items related to the indoor environment, a score is set corresponding to a nuisance behavior that hinders the maintenance of the indoor environment, The one or more processors: Identifying the nuisance based on the camera image; The index is calculated based on a total value of the scores corresponding to the nuisance acts identified within a certain period of time. Control system.

5. A method for controlling an unmanned vehicle that does not require a driver or conductor to be on board, comprising: Calculating an index relating to maintenance of the indoor environment based on information on items relating to the indoor environment of the unmanned vehicle; determining whether the indicator is equal to or greater than a threshold; If the indicator is less than a threshold value, notifying a predetermined message using a notification device in the interior of the unmanned vehicle; If the index is equal to or greater than a threshold, transmitting a remote assistance request to a remote assistance terminal that performs remote assistance for the unmanned vehicle. A control method comprising:

Images