Control system and control method for unmanned vehicles

Abstract

To provide a technique for appropriately keeping an indoor environment while reducing a burden on a remote supporter in a driverless vehicle.SOLUTION: Provided is a control system for an unmanned vehicle which eliminates the need for a driver or a conductor to get on the vehicle. In the control system, based on information of items relating to an indoor environment of the vehicle, an index relating to maintenance of the indoor environment is calculated, and it is determined whether or not the index is equal to or more than a threshold. In a case where the index is less than the threshold, a predetermined message is reported using an indoor reporting device of the vehicle and in a case where the index is equal to or more than the threshold, a remote support request is transmitted to a remote support terminal that remotely supports the vehicle.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a system and method for controlling a driverless vehicle that does not require a driver or a conductor to board.

Background Art

[0002] Patent Document 1 discloses a technique related to remote monitoring of an autonomous driving vehicle. The autonomous driving vehicle includes an autonomous sensor including at least a camera. When an obstacle is detected by the autonomous sensor, the autonomous driving vehicle is automatically stopped. After the automatic stop, when a transmission signal is received from a remote monitoring center that remotely monitors the autonomous driving vehicle, the driving of the autonomous driving vehicle is restarted.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Driverless vehicles that do not require a driver or a conductor to board are known. Consider providing a passenger mobility service using a driverless vehicle. During the provision of the mobility service, a situation may occur where the interior environment of the vehicle is not properly maintained, such as nuisance behavior by passengers. If there is a conductor on the vehicle, the conductor can handle such a situation, but there is no conductor constantly on board in a driverless vehicle. Also, although a remote supporter is waiting outside the vehicle in case of a situation that cannot be handled on the vehicle side, if remote support is always requested when a situation that affects the maintenance of the interior environment occurs, there may be a shortage of remote supporters.

[0005] One object of the present disclosure is to provide a technique for appropriately maintaining the interior environment while reducing the burden on remote supporters in a driverless vehicle. [Means for solving the problem]

[0006] This disclosure provides a control system for an unmanned vehicle that does not require a driver or conductor to be on board. The control system of this disclosure comprises one or more processors. One or more processors calculate an index for maintaining the interior environment based on information on items related to the interior environment of the vehicle, determine whether the index is above a threshold, notify a predetermined message using a notification device inside the vehicle if the index is below the threshold, and send a remote support request to a remote support terminal that provides remote support for the vehicle if the index is above the threshold.

[0007] This disclosure provides a method for controlling an unmanned vehicle that does not require a driver or conductor to be on board. The control method of this disclosure includes: calculating an index for maintaining the interior environment based on information on items relating to the interior environment of the vehicle; determining whether the index is above a threshold; if the index is below the threshold, notifying a predetermined message using a notification device inside the vehicle; and if the index is above the threshold, transmitting a remote support request to a remote support terminal that provides remote support for the vehicle. [Effects of the Invention]

[0008] According to the technology disclosed herein, an indicator related to maintaining the indoor environment is calculated, and if the indicator is below a threshold, a predetermined message is sent from the notification device to the vehicle's interior. The notification of this predetermined message may resolve the situation affecting the maintenance of the indoor environment. On the other hand, if the indicator is above the threshold, a remote assistance request is sent. Appropriate action can be taken according to the indicator without always requiring a remote assistance request.

[0009] Thus, the technology of this disclosure makes it possible to reduce the frequency of requests for remote assistance while appropriately responding to situations that affect the maintenance of the indoor environment. [Brief explanation of the drawing]

[0010] [Figure 1]This is a conceptual diagram showing an example of the configuration of the control system according to this embodiment. [Figure 2] This flowchart shows an example of processing by a control system. [Figure 3] This table shows examples of information used to calculate indicators related to maintaining an indoor environment. [Figure 4] This is a diagram illustrating variations. [Figure 5] This flowchart shows an example of the process in the modified case. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described with reference to the attached drawings.

[0012] 1. Overview of the control system Figure 1 is a diagram illustrating the overview of the control system 20 according to this embodiment. The control system 20 controls vehicle 1. Vehicle 1 is a driverless vehicle capable of autonomous driving. A driverless vehicle does not require a driver to operate or monitor the operation of vehicle 1. Because there is no driver or conductor on board, a driverless vehicle is also called an unmanned vehicle.

[0013] Vehicle 1 is used to provide mobility services to users. For example, Vehicle 1 may be a vehicle used as public transport, a shared vehicle used by multiple users, or an unmanned taxi.

[0014] The control system 20 may be mounted on the vehicle 1. For example, the control system 20 may consist of one or more ECUs (Electronic Control Units). Alternatively, the control system 20 may be mounted on an external device outside the vehicle 1. For example, the control system 20 may be part of a management server that manages the vehicle 1. Alternatively, the control system 20 may be distributed between the vehicle 1 and the external device.

[0015] The control system 20 comprises one or more processors 21 (hereinafter simply referred to as processor 21) and one or more storage devices 22 (hereinafter simply referred to as storage devices 22). The processor 21 performs various processes. For example, the processor 21 includes a CPU (Central Processing Unit). The storage devices 22 store various programs and various information. Examples of storage devices 22 include volatile memory, non-volatile memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0016] Vehicle 1 is equipped with a notification device 11 and a sensor 12. The notification device 11 is a device that provides notifications to passenger 3 of vehicle 1. In the example in Figure 1, the notification device 11 is a speaker that provides notifications to passenger 3 by voice. In another example, the notification device 11 may be a display device that provides notifications to passenger 3 by visual means. The sensor 12 is a sensor that detects the conditions inside vehicle 1. In the example in Figure 1, the sensor 12 is an in-vehicle camera that takes images of the inside of vehicle 1. In another example, the sensor 12 may be an odor sensor, an illuminance sensor, a temperature sensor, etc. There may be multiple notification devices 11 and sensors 12. The control system 20 is configured to communicate with each notification device 11 and sensor 12.

[0017] Furthermore, the control system 20 is connected to one or more remote support terminals 40 via a communication network. A remote support worker 4 is stationed at each remote support terminal 40. When vehicle 1 requires remote support, for example, when vehicle 1 is unable to make decisions regarding driving on its own, a remote support request is sent from the control system 20, and the assigned remote support worker 4 operates the remote support terminal 40 to provide remote support.

[0018] Here, consider the interior environment of the vehicle 1. The interior environment is an important factor for the passengers 3 using the mobility service and is preferably maintained above a certain level. Hereinafter, in this specification, a situation that hinders the maintenance of the interior environment, that is, a situation that affects the interior environment and may impair the comfort of the passengers 3 is referred to as a specific situation. For example, a situation where the interior is too bright, too dark, an unpleasant odor is generated for the passengers 3, the temperature inside the vehicle is too low, too high, a nuisance behavior by some passengers occurs, etc. are assumed as specific situations. Examples of nuisance behavior by passengers include smoking in the non-smoking vehicle interior, making loud noises and causing disturbances, walking around the interior more than necessary, eating and drinking in the interior where eating and drinking are prohibited, etc.

[0019] For example, in the example of FIG. 1, the western sun shines into the interior of the vehicle 1, and the passenger 3 feels dazzled. If a conductor is permanently stationed in the vehicle 1, even if a specific situation occurs, the conductor can notice and respond. For example, when the interior becomes overly bright as shown in FIG. 1, the conductor who notices this can close the curtain to maintain the interior environment in a comfortable state again. However, in the case of a driverless vehicle, the conductor is not necessarily riding in the vehicle 1.

[0020] As a countermeasure when a specific situation occurs, it is conceivable to improve the situation by remote support, that is, the remote supporter 4 plays the role of a conductor. For example, when it is detected from the sensor 12 (illuminance sensor) that the interior is overly bright, etc., it is conceivable to transmit a remote support request from the vehicle 1 and request the remote supporter 4 to respond. The remote supporter 4 who receives the remote support request can confirm from the image of the sensor 12 (in-vehicle camera) that the interior is in a dazzling state and operate the remote support terminal 40 to close the curtain inside the vehicle to respond to the situation.

[0021] However, in the remote support of vehicle 1, a single remote supporter 4 often takes charge of multiple vehicles 1. If remote support requests are frequently transmitted from each of the multiple vehicles 1, there is a possibility that the number of remote supporters 4 will be insufficient. In particular, the remote supporter 4 is also responsible for remote support related to the running of vehicle 1. It is not desirable that resources for more necessary remote support such as support related to running are compressed due to remote support for fulfilling the role as a conductor. Therefore, when a specific situation is detected, the control system 20 first determines whether the detected specific situation is a situation with a high degree of urgency. Then, different coping methods are selected according to the degree of urgency.

[0022] 2. Processing Example FIG. 2 is a flowchart showing an example of the processing by the control system 20. The processing represented by the flowchart of FIG. 2 is realized, for example, when the processor 21 executes a program stored in the storage device 22. This series of processing is repeatedly executed at a predetermined control cycle.

[0023] In step S110, the processor 21 determines whether a specific situation has been detected. If a specific situation has been detected (step S110; Yes), the processing proceeds to step S120. If a specific situation has not been detected (step S110; No), the processing for this cycle ends.

[0024] The processor 21 can detect a specific situation based on the information acquired from the sensor 12. For example, the processor 21 can detect a situation where the interior is overly bright by image analysis of the image captured by the in-vehicle camera. Alternatively, such a situation may be detected by an illuminance sensor. As another example, the processor 21 may detect an odor generated in the interior by an odor sensor. As yet another example, the processor 21 may detect that there is a passenger performing a nuisance act by image analysis of the image captured by the in-vehicle camera.

[0025] In step S120, the processor 21 calculates an index related to maintaining the indoor environment. Once the index is calculated, the process proceeds to step S130.

[0026] The storage device 22 pre-stores information for calculating indicators related to maintaining the indoor environment. Figure 3 is a table showing an example of the information stored in the storage device 22 for calculating indicators related to maintaining the indoor environment.

[0027] In the example shown in Figure 3, items related to maintaining the indoor environment are set, and a score is assigned to each item according to the type of specific situation. The scores are pre-set so that items with a higher urgency are higher, and items with a lower urgency are lower. For example, if the interior is dark, passenger 3 may feel uncomfortable, but since this is not a situation that will immediately cause harm to passenger 3, the score is set relatively low. If a passenger is smoking in a non-smoking car, the score is set relatively high because this could lead to health problems for other passengers. When the control system 20 detects a specific situation, it calculates an index based on these scores. For example, the sum of the scores corresponding to the specific situations detected within a predetermined time is calculated as an index related to maintaining the indoor environment.

[0028] In step S130, the processor 21 determines whether the index calculated in step S120 is equal to or greater than a threshold. The threshold is predetermined and stored in the memory device 22. If the index is less than the threshold (step S130; No), the process proceeds to step S140. On the other hand, if the index is equal to or greater than the threshold (step S130; Yes), the process proceeds to step S150.

[0029] In step S140, the processor 21 plays an automated message using the notification device 11. The automated message is pre-set according to the type of specific situation. For example, for a situation where the room is too bright, the message "The sun is shining in and it is dazzling. Please be careful." may be set.

[0030] In step S150, the processor 21 sends a remote support request to the remote support terminal 40. Upon receiving the remote support request, the remote supporter 4 operates the remote support terminal 40 to provide remote support to resolve the specific situation. Once the remote support request is sent, the series of processes ends.

[0031] The effects of this series of processes will be explained with an example. For instance, even if sunlight streams in and makes the interior too bright, the problem may be quickly resolved by changing the position or orientation of vehicle 1. In such cases, the indicator for maintaining the interior environment is small, so an automated message (for example, "If you experience any inconvenience while riding, please inform us using your device.") is played to prompt passenger 3 to report the occurrence of the specific situation. If the situation is not resolved within a predetermined time, the specific situation is detected multiple times within that time, and the score is added up, increasing the indicator. In such cases, remote assistance is requested, and the remote assistant 4 can take action such as closing the curtains to appropriately address the situation. Furthermore, if a specific situation with a large score is detected, the indicator for maintaining the interior environment also increases. Since such a situation is considered to be a high-priority situation, a remote assistance request is sent even if it is detected only once.

[0032] Thus, according to the control system 20, by calculating an index using a score, it is possible to distinguish between situations requiring remote assistance and situations that can be handled with automated messages. If the index related to maintaining the indoor environment is small and the situation is not urgent, an automated message is played instead of requesting remote assistance. This reduces the frequency of remote assistance requests and prevents a shortage of remote support personnel 4. On the other hand, if the index related to maintaining the indoor environment is large and the situation is urgent, remote assistance is requested, so that specific urgent situations can be dealt with appropriately.

[0033] 3. Whistleblowing by passengers 3-1. Internal reporting of disruptive behavior In the example above, the control system 20 detects a specific situation by acquiring information from the sensor 12. However, the detection of a specific situation may also be based on an internal report from passenger 3. The internal report from passenger 3 is made through a user terminal. The user terminal is a mobile communication terminal owned by passenger 3. Each user terminal is configured to communicate with the control system 20. Passenger 3 can use any communication terminal as a user terminal. For example, a mobile communication terminal such as a smartphone, smartwatch, or tablet may be used as a user terminal by installing an application on it.

[0034] Here, as an example, consider a situation where a passenger is engaging in disruptive behavior. Figure 4 shows a situation where multiple passengers 3a and 3b are on board vehicle 1. Despite vehicle 1 being a non-smoking vehicle, passenger 3b is smoking inside vehicle 1. Passenger 3a, noticing passenger 3b's disruptive behavior (in this case, smoking), makes an internal report to the control system 20 via their user terminal 30a.

[0035] For example, passenger 3a launches an application installed on user terminal 30a and inputs a request to make an internal report. The application may also input information to identify passenger 3b who is engaging in disruptive behavior, along with the request for an internal report. The information to identify passenger 3b does not necessarily have to pinpoint passenger 3b; for example, it may be selected which area passenger 3b is in when vehicle 1 is divided into several areas, or the vehicle number in which passenger 3b is located may be entered. Alternatively, the control system 20 may acquire images from the in-vehicle camera in response to the internal report request from passenger 3a and display the images on user terminal 30a through the application. The control system 20 may then ask passenger 3a to pinpoint passenger 3b from the image. Passenger 3a can pinpoint passenger 3b by circling them on the application. Alternatively, the control system 20 may extract candidate individuals from the in-vehicle camera images and ask passenger 3a to select passenger 3b who is engaging in disruptive behavior from among the extracted individuals. As another example, the control system 20 may identify passenger 3b engaging in disruptive behavior by analyzing images acquired from in-vehicle cameras.

[0036] Upon receiving an internal report, the control system 20 calculates an indicator related to maintaining the indoor environment and, depending on the magnitude of the indicator, either plays an automated message or requests remote assistance.

[0037] An index related to maintaining the indoor environment can be calculated using a score, similar to step S120 in Figure 2. First, the processor 21 identifies disruptive behavior that hinders the maintenance of the indoor environment based on internal reports. Next, it obtains a score corresponding to the identified disruptive behavior. Then, it sums up the scores obtained within a certain period of time and calculates an index related to maintaining the indoor environment. If there are multiple passengers 3 who have made internal reports, the scores obtained from those internal reports are added together. In other words, if there are internal reports about disruptive behavior of high urgency, or if internal reports are made by multiple passengers 3, the index related to maintaining the indoor environment will increase, and it will be judged to be of high urgency. In such cases, a remote assistance request is immediately sent to the remote assistance terminal 40.

[0038] For example, consider a case where the threshold is set to 4. If an internal report is received that there is a smoker in the vehicle, the index becomes 4, which is above the threshold, so a remote assistance request is sent immediately. If one internal report is received that there is a noisy person in the vehicle, the index becomes 2, which is below the threshold, so an automated message is played first. Similarly, if two internal reports are received that there is a noisy person in the vehicle, the index becomes 4, so a remote assistance request is sent immediately.

[0039] When a specific situation is detected based on an internal report from Passenger 3, it means that the situation is one that Passenger 3 actually wants resolved. Therefore, the specific situation can be detected in a more realistic manner. Also, since Passenger 3 can use their own user device, they can make an internal report without worrying about the disruptive passenger finding out. Furthermore, if internal reports are received from multiple Passenger 3s, the scores are aggregated and an index is calculated to represent the most urgent situation. This allows for the selection of an automated message playback or remote assistance response based on the situation.

[0040] 3-2. Multiple messages If an internal report is made by passenger 3 regarding disruptive behavior, the following modifications are possible. Figure 5 is a flowchart showing an example of the processing performed by processor 21 in this modification.

[0041] In steps S210 to S230, the same processing as in steps S110 to S130 in Figure 3 is performed. If the index is below the threshold in step S230, the process proceeds to step S240.

[0042] Step S240 is the same as step S140 in Figure 3 in that an automated message is played. However, here an "abstract message" is played. An abstract message is a message that corresponds to two or more items that indicate the type of disruptive behavior. For example, the message "Please let us know if you witness any disruptive behavior" is set as the abstract message. By playing an abstract message as the first automated message, it is possible to gently warn everyone without immediately provoking the passenger who is engaging in disruptive behavior.

[0043] In step S250, the processor 21 determines whether the disruptive behavior has ended. This can be determined by analyzing the images captured by the in-vehicle camera. Alternatively, a message asking "Has the disruptive behavior ended?" may be sent to the passenger terminal 30, and the determination may be made based on the response from passenger 3. If the disruptive behavior has ended (step S250; Yes), the process ends. On the other hand, if the disruptive behavior has not ended (step S250; No), the process proceeds to step S260.

[0044] In step S260, the processor 21 plays an automated message again. However, unlike in step S240, a "specific message" is played here. A specific message is a message set according to the type of disruptive behavior. For example, if the disruptive behavior is making noise indoors, the message "Please do not make noise inside the vehicle" is played as the specific message. By playing a specific message, a clear warning can be given to the passengers. Once the specific message is played, the process proceeds to step S270.

[0045] In step S270, the processor 21 determines whether the passenger's disruptive behavior has ended. The determination method is the same as in step S250. If the passenger's disruptive behavior has ended (step S270; Yes), the series of processes ends. If the passenger's disruptive behavior has not ended (step S270; No), the process proceeds to step S280.

[0046] In step S280, the processor 21 sends a remote support request to the remote support terminal 40. Upon receiving the remote support request, the remote supporter 4 deals with the disruptive behavior by broadcasting an announcement tailored to the specific situation. At this time, the remote supporter 4 may make an announcement that clearly identifies the passenger engaging in disruptive behavior, such as, "Person wearing green clothing, please refrain from making noise inside the train."

[0047] In step S290, the processor 21 determines whether the disruptive behavior has ended. The determination may be made in the same way as in step S250. Alternatively, the remote supporter 4 may determine whether the disruptive behavior has ended based on the image from the in-vehicle camera and obtain the determination result entered by the remote supporter 4. If the disruptive behavior by the passenger has ended (step S290; Yes), the series of processes ends. If the disruptive behavior by the passenger has not ended (step S290; No), the process returns to step S280 and the remote support request is sent again.

[0048] Thus, in this modified version, the automated message includes two types of messages: an abstract message and a concrete message. The abstract message takes precedence over the concrete message. This allows for a gentle warning to passengers engaging in disruptive behavior, and if the passenger does not stop, it transitions to a concrete message tailored to the specific situation. This approach avoids abruptly provoking the disruptive passenger while still providing appropriate warning.

[0049] Furthermore, in this modified example, remote support is required when the indicators related to maintaining the indoor environment are high, i.e., when the situation is urgent. While reducing the frequency of remote support requests and thus reducing the burden on remote support providers 4, it is still possible to request remote support when necessary and to ask for an appropriate response from remote support providers 4.

[0050] 4. Effects As described above, with the control system according to this embodiment, even if a situation arises that hinders the maintenance of the indoor environment, it is no longer necessary to always rely on remote support. Appropriate measures can be taken to resolve the situation while reducing the burden on remote support personnel. [Explanation of symbols]

[0051] 1 Vehicle, 3 Passengers, 4 Remote support personnel, 11 Notification device, 12 Sensors, 20 Control system, 21 Processor, 22 Storage device, 30a User terminal, 40 Remote support terminal

Claims

1. A control system for an unmanned vehicle that does not require a driver or conductor to be on board, Equipped with one or more processors, The one or more processors described above are: Based on the information regarding the indoor environment of the aforementioned unmanned vehicle, an index for maintaining the indoor environment is calculated. Determine whether the aforementioned indicator is above a threshold, If the indicator is below the threshold, a predetermined message is notified using the notification device inside the unmanned vehicle. If the indicator is above a threshold, a remote support request is sent to the remote support terminal that provides remote support for the unmanned vehicle. The items relating to the indoor environment include multiple items indicating the types of nuisance behaviors that hinder the maintenance of the indoor environment, The predetermined message is set according to each of the plurality of items and includes a specific message corresponding to an individual nuisance and an abstract message that encompasses at least two of the plurality of items and does not specify a particular nuisance. The information regarding the aforementioned indoor environment is obtained from internal communications transmitted from the passengers' mobile devices in the unmanned vehicle. The items related to the indoor environment are assigned scores corresponding to nuisance behaviors that hinder the maintenance of the indoor environment. The one or more processors described above are: Based on the aforementioned internal report, the nuisance behavior that interferes with the maintenance of the indoor environment will be identified, Based on the sum of the scores obtained within a certain period of time, the index is calculated. If any of the aforementioned items has an indicator below a threshold, the notification of the abstract message will be prioritized over the notification of the specific message. Control system.

2. A control method for an unmanned vehicle that does not require a driver or conductor to be on board, Based on the information regarding the indoor environment of the aforementioned unmanned vehicle, an index for maintaining the aforementioned indoor environment will be calculated. To determine whether the aforementioned indicator is above a threshold, If the aforementioned indicator is below a threshold, a predetermined message will be notified using the notification device inside the unmanned vehicle. If the aforementioned indicator is above a threshold, a remote support request is sent to the remote support terminal that provides remote support for the unmanned vehicle. Includes, The items relating to the indoor environment include multiple items indicating the types of nuisance behaviors that hinder the maintenance of the indoor environment, The predetermined message is set according to each of the plurality of items and includes a specific message corresponding to an individual nuisance and an abstract message that encompasses at least two of the plurality of items and does not specify a particular nuisance. The information regarding the aforementioned indoor environment is obtained from internal communications transmitted from the passengers' mobile devices in the unmanned vehicle. The items related to the indoor environment are assigned scores corresponding to nuisance behaviors that hinder the maintenance of the indoor environment. The aforementioned control method further includes, Based on the aforementioned internal report, identify the nuisance behavior that interferes with the maintenance of the indoor environment, The index is calculated based on the total value of the scores obtained within a certain period of time, If any of the aforementioned items has an indicator below a threshold, the notification of the abstract message will be prioritized over the notification of the specific message. A control method including

Images