Remote operation assistance method, mobile body, and program

Abstract

A remote operation assistance method according to the present disclosure includes: assessing whether remote control performed by an operator is necessary; acquiring information indicating the situation of the operator and information pertaining to a surrounding object around a current position including a surrounding traffic participant in cases where remote control performed by the operator is necessary; assessing, on the basis of the information indicating the situation of the operator, whether the operator can respond to a remote request from the host device requesting the remote control; assessing, on the basis of the information pertaining to the surrounding object, whether the host device that is stopped at the current position is an obstacle to the surrounding traffic participant; and autonomously retreating from the current position in cases where the operator cannot respond to the remote request from the host device and where the host device that is stopped at the current position is an obstacle to the surrounding traffic participant.

Description

Remote operation support method, mobile object, and program

[0001] The present disclosure relates to a remote operation assistance method, a mobile object, and a program.

[0002] In recent years, various types of services using autonomous mobile objects have been put into practical use, and remote control systems that can remotely monitor or operate these mobile objects are being developed. In these remote control systems, an operator is assigned in response to a request for assistance (remote request) from a mobile object that requires remote control, and the assigned operator intervenes in the control of the mobile object, such as remote operation.

[0003] For example, Patent Document 1 discloses a technique for allocating an appropriate operator to a vehicle based on both the state of the vehicle being remotely monitored and the state of the operator.

[0004] Japanese Patent Application Laid-Open No. 2023-156078

[0005] However, there may be situations where it is not possible to assign an operator, or where even if an operator is assigned, the operator cannot respond immediately, for example, when the number of moving objects requiring remote control is large relative to the number of operators. On the other hand, with regard to a remote control system in which an operator remotely monitors or operates an autonomously traveling moving object, no consideration has been given to cases where it takes time for the operator to respond, and there is room for improvement in terms of realizing safe and efficient remote control.

[0006] One of the objects of the present disclosure is to realize safe and efficient remote control even when it takes time for an operator to respond.

[0007] The remote operation assistance method according to the present disclosure is a method executed on at least one of a plurality of moving bodies that is configured to be able to move autonomously or in accordance with remote operation of an operator monitoring the plurality of moving bodies and perform a predetermined task, the method determining whether a situation requires remote control by the operator, and if a situation requires remote control by the operator, acquiring information indicating the operator's situation and information on surrounding objects at the current location including surrounding traffic participants, determining based on the information indicating the operator's situation whether the operator is in a situation where he or she can respond to a remote request from the operator's own device requesting the remote control, determining based on the information on the surrounding objects whether the operator's own device stopped at its current location will be an obstacle to surrounding traffic participants, and if the operator is in a situation where he or she cannot respond to the remote request from the operator's own device and the operator's own device stopped at its current location will be an obstacle to surrounding traffic participants, autonomously evacuating from its current location.

[0008] FIG. 1 is a diagram illustrating an example of a schematic configuration of a remote control system according to an embodiment. FIG. 2 is a diagram illustrating an example of the configuration of each device included in the remote control system according to the embodiment. FIG. 3 is a diagram illustrating an example of a hardware configuration of a control device for a vehicle according to the embodiment. FIG. 4 is a flowchart illustrating an example of a flow of processes executed in the control device for a vehicle according to the embodiment. FIG. 5 is a sequence diagram illustrating an example of a flow of processes executed in the remote control system according to the embodiment. FIG. 6 is a flowchart illustrating another example of a flow of processes executed in the control device for a vehicle according to the embodiment. FIG. 7 is a diagram illustrating an example of a display screen according to a first application example. FIG. 8 is a diagram illustrating an example of a display screen according to the first application example. FIG. 9 is a diagram illustrating an example of a display screen according to the first application example. FIG. 10 is a diagram illustrating an example of a display screen according to the first application example. FIG. 11 is a diagram illustrating an example of a display screen according to the first application example. FIG. 12 is a diagram illustrating an example of a display screen according to a second application example. FIG. 13 is a diagram illustrating an example of a display screen according to a third application example. FIG. 14 is a diagram illustrating an example of a display screen according to a fourth application example. FIG. 15 is a diagram illustrating an example of a display screen according to a fifth application example. FIG. 16 is a diagram illustrating an example of a display screen according to a sixth application example. Fig. 17 is a diagram showing an example of a display screen according to the seventh application example. Fig. 18 is a diagram showing an example of a display screen according to the eighth application example. Fig. 19 is a diagram showing an example of a display screen according to the ninth application example. Fig. 20 is a diagram showing an example of a display screen according to the tenth application example. Fig. 21 is a diagram showing an example of a display screen according to the twelfth application example. Fig. 22 is a diagram showing an example of a display screen according to the thirteenth application example. Fig. 23 is a diagram showing an example of a display screen according to the fourteenth application example. Fig. 24 is a diagram showing an example of a display screen according to the fifteenth application example. Fig. 25 is a diagram showing an example of a display screen according to the sixteenth application example.

[0009] Hereinafter, with reference to the accompanying drawings, embodiments of a remote operation assistance method, a remote operation system, a remote operation assistance device, an operator terminal, a vehicle (mobile body), and a program according to the present disclosure will be described in detail.

[0010] In the description of the present disclosure, components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings may be given the same reference numerals, and descriptions thereof may be omitted as appropriate. Furthermore, even when the same or substantially the same parts are shown, the dimensions and proportions may be different depending on the drawing. Furthermore, for example, in order to ensure the visibility of the drawings, reference numerals may be given to only the main components in the description of each drawing, and reference numerals may not be given to components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings.

[0011] (Configuration example of remote control system) Fig. 1 is a diagram showing an example of a schematic configuration of a remote control system 1 according to an embodiment. As shown in Fig. 1, the remote control system 1 includes a server device 10, a plurality of operator terminals 20a, 20b to 20m (m is any integer), and a plurality of vehicles 30a, 30b to 30n (n is any integer).

[0012] 1, the server device 10 and each of the plurality of vehicles 30a, 30b to 30n are connected to each other so as to be able to communicate with each other via a network N such as the Internet. Also, the server device 10 and each of the plurality of operator terminals 20a, 20b to 20m are connected to each other so as to be able to communicate with each other via any telecommunications line such as a LAN (Local Area Network). Note that communication between the server device 10 and each of the plurality of operator terminals 20a, 20b to 20m may be realized via the network N.

[0013] In the description of the present disclosure, when the multiple operator terminals 20a, 20b to 20m are not distinguished from one another, they are simply referred to as "operator terminal 20." Similarly, when the multiple vehicles 30a, 30b to 30n are not distinguished from one another, they are simply referred to as "vehicle 30."

[0014] The number (m) of operator terminals 20 and the number (n) of vehicles 30 included in the remote control system 1 can be changed as desired depending on design conditions, etc. In addition, the server device 10 in the remote control system 1 may be realized by the cooperation of multiple server devices.

[0015] Here, the server device 10 according to the embodiment is a device (remote operation support device) that supports remote operation of the vehicle 30. The operator terminal 20 according to the embodiment is a device (terminal device) operated by an operator in a remote control room. The vehicle 30 according to the embodiment is an example of an autonomously traveling mobile body, and is used to provide various services.

[0016] As an example, the remote control system 1 according to the embodiment can be constructed by applying edge computing. In this case, for example, a vehicle 30 is used as the network edge, but other devices may also be used as the edge.

[0017] As an example, the vehicle 30 is an example of a moving object that performs various tasks, including autonomous driving, related to various services provided by the remote operation system 1, such as delivery, security, cleaning, childcare, nursing care, sales, farm work, manufacturing, loading and unloading, transportation, and construction. Hereinafter, in this disclosure, autonomous driving will be exemplified as a task of the vehicle 30.

[0018] As an example, the vehicle 30 is a mobile body configured to be able to move autonomously and execute a predetermined task. As an example, the vehicle 30 is a mobile body configured to be able to move and execute a predetermined task according to remote control by an operator who monitors the multiple vehicles 30.

[0019] The moving body is not limited to the vehicle 30, and various moving bodies configured to be movable at least in response to remote operation by an operator can be used as appropriate. The vehicle 30 may be, for example, a four-wheeled vehicle or a two-wheeled vehicle. Furthermore, the vehicle 30 may be, for example, an automatic guided vehicle (AGV), or various robots such as construction machinery, agricultural machinery, or drones. Furthermore, these moving bodies are not limited to those that transport people, but may also transport objects other than people, or may provide specific services other than transportation.

[0020] As an example, when the vehicle 30 is unable to drive autonomously, for example, when an obstacle such as a fallen object or a vehicle parked on the road is detected in its path, the vehicle 30 sends an assistance request (remote request) to the server device 10 requesting assistance through remote control, i.e., remote operation.

[0021] As an example, the server device 10 transmits an image captured by a camera mounted on the vehicle 30 to the corresponding operator terminal 20 .

[0022] As an example, when the server device 10 receives a request for assistance from the vehicle 30, the server device 10 transmits a remote operation request to the operator terminal 20, requesting remote operation by an operator.

[0023] As an example, the operator terminal 20 displays a display screen including an image captured by a camera mounted on the vehicle 30. The display screen includes at least one image related to at least one vehicle 30 that is in charge of the operator operating the operator terminal 20. The operator monitors the status of each of the at least one vehicle 30 that he or she is in charge of while looking at the display screen of the operator terminal 20.

[0024] As an example, when the operator terminal 20 receives a remote operation request (remote request) from the server device 10, the operator terminal 20 notifies the operator operating the terminal that assistance via remote operation has been requested. The operator operates the operator terminal 20 while viewing the display screen, remotely controls the vehicle 30 that has made the assistance request, and provides assistance such as moving the vehicle 30.

[0025] In this way, the remote control system 1 of this embodiment is a remote control system in which an operator remotely operates a vehicle 30, and is configured to be able to execute a remote control assistance method that assists in remote control based on images for remote control captured by a camera mounted on the vehicle 30.

[0026] 2 is a diagram showing an example of the configuration of each device (server device 10, operator terminal 20, and vehicle 30) included in the remote operation system 1 according to the embodiment. While Fig. 2 illustrates the configuration of one of at least one operator terminal 20 and at least one vehicle 30 included in the remote operation system 1, the other operator terminals 20 and vehicles 30 included in the remote operation system 1 have the same configuration.

[0027] Hereinafter, the configurations of the server device 10, the operator terminal 20, and the vehicle 30 will be described with reference to FIG.

[0028] (Example of Vehicle Configuration) First, the configuration of the vehicle 30 will be described.

[0029] As shown in Figure 2, the vehicle 30 has a sensor 301, a remote control judgment unit 302, a notification location determination unit 303, an operator status acquisition unit 304, a remote request response judgment unit 305, a surrounding information acquisition unit 306, a surrounding status judgment unit 307, a control information generation unit 308, a remote request information generation unit 309, a remote request information transmission unit 310, a control information acquisition unit 311, a control unit 312, and a drive unit 313.

[0030] The sensor 301 includes at least one sensor for acquiring information about the vehicle and its surroundings. The output (sensor information) of the sensor 301 is provided to the remote control determination unit 302 and the surrounding situation determination unit 307. The output of the sensor 301 may also be provided to the control unit 312.

[0031] As an example, the sensor 301 includes one or more cameras (image capturing devices) that capture images of the surroundings of the vehicle 30. As an example, the sensor 301 includes at least one sensor that detects at least one of the state of the vehicle 30, the speed, the acceleration, the current position, the presence or absence of surrounding objects, the number of surrounding objects, the type of surrounding objects, and the distance to the surrounding objects.

[0032] The remote control determination unit 302 determines, based on the sensor information from the sensor 301, whether or not remote control of the subject vehicle is required.

[0033] As an example, the remote control determination unit 302 acquires information about surrounding objects based on sensor information from the sensor 301. Here, the surrounding objects are objects detected around the vehicle 30, such as surrounding obstacles such as fallen objects or walls, or surrounding traffic participants such as pedestrians, bicycles, motorcycles, and automobiles. Note that surrounding traffic participants that are stopped, such as parked vehicles, may be treated as obstacles. In other words, the remote control determination unit 302 acquires information about the surrounding conditions of the vehicle 30 based on the sensor information from the sensor 301.

[0034] For example, the remote control determination unit 302 acquires information indicating the presence or absence of surrounding objects. For example, the remote control determination unit 302 acquires information indicating the type of surrounding object, such as "pedestrian," "car," "bicycle," "motorcycle," or "obstacle." For example, the remote control determination unit 302 acquires information indicating the number of surrounding objects, such as "2 pedestrians," "2 cars," or "1 obstacle." For example, the remote control determination unit 302 acquires information indicating the positions of surrounding objects, such as "pedestrian: 6 m behind right, 5 m behind left," "2 cars: 25 m in front right," or "obstacle: 4 m in front." For example, the remote control determination unit 302 acquires information indicating whether a surrounding object is present on the driving route of the host vehicle. For example, the remote control determination unit 302 acquires information indicating whether a surrounding object is approaching the host vehicle or whether there is a possibility of a collision between the host vehicle and the surrounding object.

[0035] As an example, the remote control determination unit 302 determines whether a situation has arisen in which remote control is necessary, based on the surrounding conditions of the vehicle 30 indicated by the information about the surrounding objects. For example, if the "surrounding conditions of the vehicle" are "no surrounding objects," the remote control determination unit 302 determines that remote control is "not necessary." For example, if the "surrounding conditions of the vehicle" are "one stopped vehicle has been found ahead, and it is present on the driving route," the remote control determination unit 302 determines that remote control is "necessary." For example, if the "surrounding conditions of the vehicle" are "an obstacle has been found ahead, and it is present on the driving route," the remote control determination unit 302 determines that remote control is "necessary." For example, if the "surrounding conditions of the vehicle" are "a pedestrian has been found around, and there is no sign of it approaching," the remote control determination unit 302 determines that remote control is "not necessary."

[0036] The acquisition of information about the surrounding objects based on the sensor information from the sensor 301 may be performed by the peripheral information acquisition unit 306, which will be described later. In other words, the remote control determination unit 302 may be configured to acquire information about the surrounding objects from the peripheral information acquisition unit 306, and determine whether a situation has arisen in which remote control is necessary, based on the surrounding conditions of the vehicle 30 indicated by the information about the surrounding objects.

[0037] When remote control becomes necessary, the notification location determination unit 303 determines a location to notify a remote request indicating that remote control is necessary. Specifically, when remote control becomes necessary, the notification location determination unit 303 determines whether to notify the remote request on the spot or at a location to which the vehicle has retreated so as not to disturb surrounding traffic participants. In other words, when remote control becomes necessary, the notification location determination unit 303 determines a location to notify the remote request based on whether stopping on the spot will disturb surrounding traffic participants.

[0038] As an example, if the surrounding situation determination unit 307 determines that stopping at the location would cause a hindrance to surrounding traffic participants, the notification location determination unit 303 determines to move away so as not to cause a hindrance to surrounding traffic participants and then notify the operator of a remote request. On the other hand, if the surrounding situation determination unit 307 does not determine that stopping at the location would cause a hindrance to surrounding traffic participants, the notification location determination unit 303 determines to stop at the location and notify the operator of a remote request.

[0039] In determining the location where the remote request notification is to be sent, only the criterion that takes the shortest time to obtain the determination result may be used, out of the criteria for determining whether or not the remote request will cause a disturbance to surrounding traffic participants, which will be described later.

[0040] The operator status acquisition unit 304 acquires information indicating whether the operator is available to immediately respond to a remote request. In this disclosure, "immediately" refers to within a predetermined time period, such as several seconds. This predetermined time period may be a fixed time period that is predetermined and stored in the memory of the vehicle 30, but may be dynamically changed based on, for example, the state of the vehicle 30, information about surrounding objects, the operator's proficiency, etc.

[0041] As an example, the operator status acquisition unit 304 acquires information indicating the status of the operator as information indicating whether the operator is in a situation where he or she can respond immediately. The operator status is one of "monitoring," "watching," "stopping / resuming operation," and "remotely operating," and indicates the operator's response status regarding monitoring or remotely operating at least one vehicle 30 that he or she is in charge of.

[0042] In the present disclosure, a case will be exemplified in which no distinction is made between "monitoring" and "watching," and these will be collectively referred to as "monitoring." In other words, in the present disclosure, "monitoring" refers to the state of an operator monitoring the autonomous driving status of the vehicle 30 (mobile body). Similarly, in the present disclosure, a case will be exemplified in which no distinction is made between "stop / restart operation" and "remotely controlled," and these will be collectively referred to as "remotely controlled." In other words, in the present disclosure, "remotely controlled" refers to the state of an operator remotely controlling the vehicle 30 in response to a remote request from the vehicle 30 that requires remote control because the vehicle 30 (mobile body) is in a situation where autonomous driving is difficult.

[0043] As an example, the operator status acquisition unit 304 may further acquire detailed information about the operator's status. The detailed information may include, for example, information such as the expected end time of the response. In this case, the operator status acquisition unit 304 acquires information such as "Remote operation in progress, expected end time of response 13:47" as information indicating whether the operator is available to respond immediately.

[0044] The remote request response determination unit 305 determines whether the operator can immediately respond to the remote request based on the acquired information indicating the operator's status. Note that the correspondence between the information indicating the operator's status and whether the operator can immediately respond to the remote request is determined in advance and stored in, for example, a memory of the vehicle 30.

[0045] As an example, when the operator's status is "monitoring," the remote request response determination unit 305 determines that the operator can immediately respond to the remote request issued. This is based on the fact that when the operator's status is "monitoring," no remote request has been issued, the operator is watching video of the vehicle 30 traveling, and the operator's hands are not occupied with remote operation or the like. Therefore, in this case, the operator starts remote control of the vehicle 30 in response to the remote request.

[0046] As an example, when the operator's status is "remotely operating," the remote request response determination unit 305 determines that the operator cannot immediately respond to the remote request issued. This is because when the operator's status is "remotely operating," a remote request has been issued in the past, and the operator's hands are occupied with tasks such as determining and operating the vehicle 30 to stop and resume driving, and remotely moving the vehicle 30 itself. Therefore, in this case, the remote control of the operator in response to the remote request is put on hold. During this wait, information about the surroundings of the vehicle 30 is acquired, as will be described later.

[0047] The surrounding information acquisition unit 306 acquires information about objects in the vicinity of the vehicle, i.e., surrounding objects, based on sensor information from the sensor 301. This information about the vicinity of the vehicle and the information acquired by the remote control determination unit 302 may be the same or different.

[0048] As an example, the surrounding information acquisition unit 306 acquires information indicating the presence or absence of surrounding objects. For example, the surrounding information acquisition unit 306 acquires information indicating the type of surrounding object, such as "pedestrian," "car," "bicycle," "motorcycle," or "obstacle." For example, the surrounding information acquisition unit 306 acquires information indicating the number of surrounding objects, such as "2 pedestrians," "2 cars," or "1 obstacle." For example, the surrounding information acquisition unit 306 acquires information indicating the positions of surrounding objects, such as "pedestrians: 6 m behind right, 5 m behind left," "2 cars: 25 m in front right," or "obstacle: 4 m in front." For example, the surrounding information acquisition unit 306 acquires information indicating whether surrounding objects are present in an evacuation area around the host vehicle. For example, the surrounding information acquisition unit 306 acquires information indicating whether surrounding objects are present on the driving route for the host vehicle to move to the evacuation area. For example, the surrounding information acquisition unit 306 acquires information indicating whether surrounding objects are approaching the host vehicle or whether there is a possibility of a collision between the host vehicle and the surrounding objects.

[0049] Here, the evacuation area is a space that exists around (for example, to the side of) the vehicle, and is a space where the vehicle 30 can move from its current position and stop. For example, the evacuation area is a space where the vehicle will not disturb surrounding traffic participants if it stops there. Note that the evacuation area may be a space that will cause less disturbance to surrounding traffic participants if the vehicle stops there, compared to the current position.

[0050] For example, the surrounding information acquisition unit 306 identifies an evacuation area when acquiring information indicating the presence or absence of surrounding objects. For example, map information in which areas that can be used as evacuation areas are superimposed on a map is predefined and stored in the memory of the vehicle 30. For example, the surrounding information acquisition unit 306 refers to the map information and identifies, as the evacuation area, an area that can actually be used as an evacuation area from among the usable areas based on the sensor information from the sensor 301, i.e., the sensing result.

[0051] The surrounding situation determination unit 307 determines whether it is okay to stop on the spot, that is, whether stopping on the spot will cause a hindrance to the surrounding traffic participants, based on information about the surrounding traffic participants (surrounding objects).

[0052] Here, in the present disclosure, "getting in the way of surrounding traffic participants" includes at least one of the following: obstructing the view of surrounding traffic participants, obstructing their movement, making them feel a sense of danger such as a collision if the vehicle stops on the spot, or there is a risk of any of these, or there is a risk of a collision. In other words, "getting in the way of surrounding traffic participants" means, for example, that the vehicle will have an impact on surrounding traffic participants or there is a risk of such an impact if the vehicle stops on the spot.

[0053] For example, the criterion for determining whether or not a vehicle will be a hindrance to nearby traffic participants may be whether or not there are traffic participants within a predetermined distance, such as 0 meters. If there are traffic participants within the predetermined distance, it is determined that the vehicle will be a hindrance to nearby traffic participants. The value of the predetermined distance may be predetermined and stored in a memory or the like within the vehicle 30, or may be dynamically changed based on the traveling location of the vehicle 30, etc.

[0054] For example, the criterion for determining whether or not a vehicle will be a hindrance to surrounding traffic participants may be whether or not the number of surrounding traffic participants is greater than a predetermined number. If the number of surrounding traffic participants is greater than the predetermined number, it is determined that a vehicle will be a hindrance to surrounding traffic participants.

[0055] For example, the criterion for determining whether or not a vehicle will be a hindrance to surrounding traffic participants may be whether or not there are surrounding traffic participants approaching the vehicle 30. If there are surrounding traffic participants approaching the vehicle 30, it is determined that the vehicle will be a hindrance to the surrounding traffic participants.

[0056] For example, the criterion for determining whether or not the vehicle 30 will be a hindrance to the surrounding traffic participants may be whether or not there is a possibility of a collision between the vehicle 30 and the surrounding traffic participants. For example, if a stopped surrounding traffic participant is present on the driving route of the vehicle 30, or if the driving route of the vehicle 30 intersects with the driving route of the surrounding traffic participant, it can be determined that there is a possibility of a collision between the vehicle 30 and the surrounding traffic participants. If there is a possibility of a collision between the vehicle 30 and the surrounding traffic participants, it is determined that the vehicle 30 will be a hindrance to the surrounding traffic participants.

[0057] The above-mentioned criteria are merely examples, and other criteria may be used, or a combination of multiple criteria may be used.

[0058] Based on the results determined by the surrounding situation determination unit 307, the control information generation unit 308 generates autonomous control information for evacuating the vehicle 30 from the point where remote control is required, i.e., the point where the remote request occurred, to an evacuation area, or autonomous control information for returning the vehicle 30 from the evacuation area to the point where the remote request occurred.

[0059] As an example, when it is determined that the vehicle 30 will be in the way of surrounding traffic participants, the control information generation unit 308 generates autonomous control information for autonomously evacuating the vehicle 30 to an evacuation area. The control information generation unit 308 determines the evacuation area and generates a route to the evacuation area and control information such as steering, acceleration, and deceleration for traveling along that route.

[0060] When the vehicle 30 is evacuated, the control information generating unit 308 stores the location where the remote request was made, i.e., the current location of the vehicle 30 before the evacuation, in a memory within the vehicle 30 or the like.

[0061] If there are multiple candidate evacuation areas, the control information generator 308 may select the evacuation area closest to the current position of the vehicle 30 .

[0062] In addition, when there are multiple candidate evacuation areas, the control information generation unit 308 may select an evacuation area that is easy for the operator to remotely control after evacuation. For example, the control information generation unit 308 may rank the multiple candidate evacuation areas so that the fewer the number of surrounding traffic participants, the higher the ranking, and select the top-ranked evacuation area as the evacuation area that is easy to remotely control. For example, the control information generation unit 308 may rank the multiple candidate evacuation areas so that the greater the distance from surrounding traffic participants, such as parked vehicles, the higher the ranking, and select the top-ranked evacuation area as the evacuation area that is easy to remotely control.

[0063] Note that when there are multiple candidate evacuation areas, the control information generator 308 may take the communication environment into consideration and select an evacuation area with a good communication environment after evacuation. For example, the control information generator 308 may refer to the past communication conditions for each area and rank the multiple candidate evacuation areas so that the better the communication environment, the higher the ranking, and select the top-ranked evacuation area as an evacuation area that is easy to remotely control. For example, the control information generator 308 may refer to the past communication conditions for each area and rank the multiple candidate evacuation areas so that the better the communication environment, the higher the ranking, and may not select the lower-ranked evacuation areas as evacuation areas that are easy to remotely control.

[0064] The past communication conditions for each evacuation area are acquired, for example, by referring to information indicating the correspondence between the locations of candidate evacuation areas and the past communication conditions at those locations. This information may be acquired from a source outside the vehicle 30, such as the server device 10, or may be stored in the memory of the vehicle 30. As an example, the better the past communication conditions, the better the communication environment of the evacuation area is determined to be.

[0065] The communication conditions may be, for example, a communication bandwidth, and the communication environment is judged to be better the longer the past communication bandwidth is, but other indicators related to communication speed or communication volume may also be used. Other indicators may include available communication methods.

[0066] As an example, when the vehicle 30 is evacuated, the control information generation unit 308 determines whether the vehicle 30 can return to the location where remote control became necessary or its vicinity from the perspective of "whether it is physically possible to return." Specifically, the control information generation unit 308 determines whether the vehicle 30 can return to the location where remote control became necessary or its vicinity based on information about surrounding objects in the vicinity of the current location of the vehicle 30 after evacuation and in the vicinity of the location where remote control became necessary. Furthermore, if the control information generation unit 308 determines that the vehicle 30 can return to the location where remote control became necessary or its vicinity, it generates autonomous control information for autonomously moving the vehicle 30 to the location where remote control became necessary or its vicinity. The control information generation unit 308 generates a route to the location where remote control became necessary or its vicinity and control information such as steering, acceleration, and deceleration for traveling along the route.

[0067] For example, the control information generator 308 determines that the vehicle 30 can return to the location where remote control became necessary or its vicinity when there are no surrounding objects such as nearby traffic participants around the current location (evacuation destination) of the vehicle 30 and the location where remote control became necessary, and when there are no surrounding objects such as nearby traffic participants on the driving route from the current location to the location where remote control became necessary. In this case, the vehicle 30 autonomously drives to the location where remote control became necessary or its vicinity based on the generated autonomous control information.

[0068] For example, if there are surrounding objects such as nearby traffic participants near either the current location (evacuation destination) of the vehicle 30 or the location where remote control is required, or if there are surrounding objects such as nearby traffic participants on the driving route from the current location to the location where remote control is required, the control information generator 308 determines that the vehicle 30 cannot return to the location where remote control is required or its surroundings. In this case, the remote control of an operator responding to the remote control request is put on hold at the evacuation destination. During this waiting period, information about the area around the vehicle 30 is acquired.

[0069] In addition, if it is possible to return to autonomous driving from the location where the vehicle was evacuated, such as when the surrounding object that caused the remote control to occur, such as a parked vehicle, has disappeared, the control information generation unit 308 may generate autonomous control information that returns to autonomous driving without returning to the location where the remote control occurred.

[0070] In addition, the control information generating unit 308 may determine whether or not to return the vehicle 30 to the location or its vicinity where remote control became necessary from another perspective in addition to the perspective of "whether it is physically possible to return."

[0071] For example, the control information generating unit 308 may determine whether or not the vehicle 30 will return to the location where remote control became necessary or its vicinity from the perspective of "whether the communication environment is poor." In this case, similar to the case of selecting an evacuation destination, the control information generating unit 308 may determine the communication environment of the location where remote control became necessary or its vicinity based on past communication conditions for each area, and may determine that the vehicle 30 will return to the location where remote control became necessary or its vicinity if the communication environment at the original location is not poor. The case where the communication environment is not poor refers to a case where the communication environment at the determined original location satisfies conditions such as communication bandwidth that are predetermined and stored in the memory of the vehicle 30.

[0072] For example, from the viewpoint of "service efficiency," the control information generating unit 308 may determine whether the vehicle 30 will return to the location where remote control became necessary or its vicinity. In this case, the control information generating unit 308 may determine whether the vehicle 30 will return to the location where remote control became necessary or its vicinity, depending on the type of service provided by the task of the vehicle 30.

[0073] For example, in the case of a "delivery" service, it can be said that the faster the return to autonomous driving is, the more efficient the service. Therefore, for example, if the return to autonomous driving is faster at the location where remote control became necessary than at the current location of the evacuation destination, the control information generator 308 may determine that the vehicle 30 will return to the location where remote control became necessary or its vicinity.

[0074] For example, in the case of a "monitoring" service, the more surrounding traffic participants there are, the higher the likelihood of some kind of problem occurring. Furthermore, the closer the surrounding traffic participants are, i.e., monitoring closer to a suspicious person, the more likely a problem is to be prevented from occurring and any problem can be noticed immediately, making the service more efficient. Therefore, for example, the control information generating unit 308 may determine that the vehicle 30 will return to the location where remote control is required or its vicinity when there are more surrounding traffic participants at the location where remote control is required than at the current location of the evacuation destination. For example, the control information generating unit 308 may determine that the vehicle 30 will return to the location where remote control is required or its vicinity when the location where remote control is required is closer to a monitoring target such as a suspicious person.

[0075] For example, the control information generating unit 308 may determine whether the vehicle 30 will return to the location where remote control became necessary or its vicinity from the perspective of "ease of remote control." For example, the closer surrounding objects, such as parked vehicles, are, the more difficult it is for the operator to perform remote control. Therefore, for example, the control information generating unit 308 may determine that the vehicle 30 will return to the location where remote control became necessary or its vicinity when the location where remote control became necessary is farther away from surrounding traffic participants than the current location of the evacuation destination.

[0076] For example, the control information generator 308 may determine the destination from the viewpoint of a "better location." In this case, the control information generator 308 may take into consideration the above viewpoints and, if there is a location with better conditions other than the current location of the evacuation destination and the location where remote control becomes necessary, determine to move the vehicle 30 to that location.

[0077] The remote request information generation unit 309 generates remote request information based on the determination result of the remote request response determination unit 305. This remote request information includes information for notifying the operator that an event requiring remote control has occurred, such as avoiding a roadside parking vehicle ahead that overlaps the driving route of the vehicle.

[0078] In addition, the remote request information generation unit 309 may generate information to be notified to the server device 10 regarding the vehicle status, such as "Street parking avoided, evacuated," indicating whether the vehicle 30 stopped on the spot and issued the notification or whether the notification was issued after evacuating.

[0079] In addition, the remote request information generation unit 309 may generate, for example, information regarding surrounding objects around the current location of the vehicle 30 (vehicle surrounding information), or information regarding the location where remote control becomes necessary or surrounding objects around that location (remote request location vicinity information) as information to be notified to the server device 10.

[0080] In addition, the remote request information may be generated as information to notify the server device 10 of information indicating the evacuation area to which the vehicle 30 has evacuated, such as "already evacuated to roadside parking avoidance area 2," in cases where there are multiple candidate evacuation areas.

[0081] In addition, if the vehicle 30 cannot evacuate, for example, because there are surrounding traffic participants in the evacuation area, the remote request information generation unit 309 may generate information to notify the operator of this, as information to be notified to the server device 10.

[0082] In addition, the remote request information generation unit 309 may generate information to notify the server device 10 to notify the operator that autonomous driving will be resumed at the evacuation location, or that autonomous driving will be resumed without returning to the location where remote control occurred, if autonomous driving can be resumed at the evacuation location, such as when the object that caused remote control to occur is no longer there.

[0083] The remote request information transmitting unit 310 transmits the information generated by the remote request information generating unit 309 to the server device 10 .

[0084] The control information acquisition unit 311 acquires the autonomous control information generated by the control information generation unit 308 and the remote control information transmitted from the remote control information transmission unit 202 of the operator terminal 20 .

[0085] The control unit 312 controls the operations related to various tasks of the vehicle 30, including the steering and driving such as acceleration and deceleration of the vehicle 30, based on the control information acquired by the control information acquisition unit 311.

[0086] The drive unit 313 is driven under the control of the control unit 312. The drive unit 313 is configured to include a mechanism capable of performing operations for executing various tasks of the vehicle 30, such as a mechanism that can drive the vehicle 30 in a predetermined direction for a predetermined distance at a predetermined speed.

[0087] 3 is a diagram showing an example of a hardware configuration of the control device 4 that realizes each function of the vehicle 30 according to the embodiment. The control device 4 is a device that performs overall control of the operation of the vehicle 30. The control device 4 may be realized by a computer such as an ECU (Electronic Control Unit) provided inside the vehicle 30, a DCU (Domain Control Unit) such as a CDC (Cockpit Domain Controller) that integrates multiple ECUs, or an OBU (On Board Unit).

[0088] As shown in FIG. 3, the control device 4 includes a processor 41, a ROM (Read Only Memory) 42, a RAM (Random Access Memory) 43, and a device I / F (interface) unit 44.

[0089] The processor 41 is, for example, a CPU (Central Processing Unit). The processor 41 executes, for example, a program to comprehensively control the operation of the control device 4 and realizes functions as a remote control determination unit 302, a notification location determination unit 303, an operator status acquisition unit 304, a remote request response determination unit 305, a surrounding information acquisition unit 306, a surrounding status determination unit 307, a control information generation unit 308, a remote request information generation unit 309, a remote request information transmission unit 310, a control information acquisition unit 311, and a control unit 312.

[0090] 2 illustrates only the functions necessary for explaining the main parts of this embodiment, but the functions of the vehicle 30 are not limited to these. In this embodiment, the processor 41 executes a program stored in the ROM 42 to realize the functions of the vehicle 30, including the functions of the above-mentioned parts. However, this is not limiting, and some or all of these functions may be realized by dedicated hardware circuits.

[0091] The ROM 42 is a non-volatile memory that stores various types of information including programs executed by the processor 41. The memory of the control device 4 is not limited to the ROM 42, and various types of recording media and recording devices such as a hard disk drive (HDD), a solid state drive (SSD), and flash memory can be used as appropriate. The RAM 43 is a volatile memory that has a working area for the processor 41. The device I / F unit 44 is an interface for connecting the control device 4 to other devices installed in the vehicle 30, such as a communication device (not shown), the sensor 301, and the drive unit 313.

[0092] (Configuration Example of Server Device) Next, returning to FIG. 2, the configuration of the server device 10 will be described.

[0093] As shown in FIG. 2, the server device 10 includes a remote request information acquisition unit 101 , a remote request notification unit 102 , a remote control information reception unit 103 , an operator information generation unit 104 , and an operator information transmission unit 105 .

[0094] The remote request information acquisition unit 101 acquires information transmitted from the remote request information transmission unit 310 of the vehicle 30 .

[0095] The remote request notification unit 102 transmits information for displaying camera images, notifications, etc. to the operator terminal 20 based on the information acquired by the remote request information acquisition unit 101 .

[0096] The remote control information receiving unit 103 receives information transmitted from the remote control information transmitting unit 202 of the operator terminal 20 .

[0097] The operator information generating unit 104 generates information indicating the status of the operator, based on the information received by the remote control information receiving unit 103, as to whether the operator is available to respond immediately.

[0098] The operator information transmitting unit 105 transmits the information generated by the operator information generating unit 104 to the vehicle 30 .

[0099] The server device 10 has a control device (not shown) that controls the overall operation of the server device 10. This control device is configured as a computer device, and has a hardware configuration similar to that of the control device 4 in Fig. 3. For example, the control device of the server device 10 realizes the functions of a remote request information acquisition unit 101, a remote request notification unit 102, a remote control information receiving unit 103, an operator information generation unit 104, and an operator information transmission unit 105 by having a processor execute a program loaded into RAM.

[0100] 2 illustrates only the functions necessary for explaining the main parts of this embodiment, but the functions of the server device 10 are not limited to these. In this embodiment, the processor in the control device of the server device 10 executes a program stored in the ROM to realize each function of the server device 10, including the functions of each part described above. However, this is not limiting, and some or all of these functions may be realized by dedicated hardware circuits.

[0101] The device I / F unit of the control device of the server device 10 is an interface for connecting the control device to other devices mounted on the server device 10, such as a communication device (not shown).

[0102] (Configuration Example of Operator Terminal) Next, the configuration of the operator terminal 20 will be described.

[0103] The operator terminal 20 includes a display unit 201 and a remote control information transmission unit 202 .

[0104] The display unit 201 displays, on a display device such as a liquid crystal display, a display screen including at least one image for remote operation and monitoring of each of at least one vehicle 30, which has been transmitted from the server device 10. The display unit 201 also displays, on the display device, information related to the remote request and various notifications to the operator based on the remote request information notified from the remote request notifying unit 102 of the server device 10.

[0105] The remote control information transmitting unit 202 acquires the result of an operator's operation on an input device such as a keyboard or touch panel, and transmits information indicating the operation result to the vehicle 30. For example, when a remote request is made, the operator inputs control information for the vehicle 30 using the input device. This control information is output to the vehicle 30 via, for example, the server device 10. Furthermore, this control information may be referenced by the remote control information receiving unit 103 of the server device 10 as information indicating the operator's status.

[0106] The operator terminal 20 has a control device (not shown) that comprehensively controls the operation of the operator terminal 20. This control device is configured as a computer device, and has a hardware configuration similar to that of the control device 4 in Fig. 3. For example, the control device of the operator terminal 20 realizes the functions of the display unit 201 and the remote control information transmission unit 202 by having a processor execute a program loaded into RAM.

[0107] 2 illustrates only the functions necessary for explaining the main parts of this embodiment, but the functions of the operator terminal 20 are not limited to these. In this embodiment, the processor in the control device of the operator terminal 20 executes a program stored in ROM to realize each function of the operator terminal 20, including the functions of each part described above. However, this is not limiting, and some or all of these functions may be realized by dedicated hardware circuits.

[0108] The device I / F section of the control device of the operator terminal 20 is an interface for connecting to devices installed on other operator terminals 20 of the control device, such as a communication device (not shown), a display device such as an LCD display, and an input device such as a keyboard or touch panel.

[0109] Next, an example of the operation of the remote control system 1 according to the embodiment will be described with reference to the drawings. Note that the operation procedures and processing flows described below are merely examples, and the order of steps can be changed, some steps can be deleted, and other steps can be added as desired.

[0110] FIG. 4 is a flowchart showing an example of the flow of processing executed by the control device 4 of the vehicle 30 according to the embodiment.

[0111] The control device 4 determines whether or not the situation requires remote control (S101). If it is determined that the situation does not require remote control (S101: No), the flow of FIG. 4 returns to the process of S101.

[0112] If it is determined that the situation requires remote control (S101: Yes), the control device 4 determines whether the vehicle will be an obstacle to nearby traffic participants (S102).

[0113] If it is determined that the vehicle is not interfering with the surrounding traffic participants (S102: No), the control device 4 notifies the operator of a remote request (S103) and acquires information indicating the operator's situation (S104). Thereafter, the control device 4 determines whether the operator is in a situation where he or she can immediately respond (S105).

[0114] If it is not determined that the operator is able to respond immediately (S105: No), the control device 4 waits at the remotely requested location while acquiring information about the surrounding area (information about surrounding objects) (S106), and determines whether the vehicle will be an obstacle to nearby traffic participants (S107).

[0115] If it is determined that the vehicle 30 will be an obstacle to nearby traffic participants (S107: Yes), the control device 4 causes the vehicle 30 to autonomously evacuate from the remotely requested location to the evacuation area (S108). After autonomous evacuation, the control device 4 acquires information about surrounding objects around the evacuation destination and the remotely requested location where remote control is required (S109), and determines whether the vehicle 30 can return to the remotely requested location (S110).

[0116] Note that it is not always possible to check the status of traffic participants around the original location from the evacuation destination location. If this is not possible, the surrounding conditions of the original location are unknown, which may result in the vehicle being unable to return to the remotely requested location. For this reason, movement control may be performed to return the vehicle via an evacuation destination closer to the original location than the current evacuation destination, gradually approaching the evacuation destination, so that the surrounding conditions of the original location can be checked. For example, the control device 4 may acquire information about surrounding objects around the evacuation destination and an evacuation destination closer to the original location, and determine whether the vehicle 30 can return to the evacuation destination closer to the original location. Then, the control device 4 may autonomously travel to an evacuation destination closer to the original location, and then determine whether the vehicle 30 can return to an evacuation destination even closer to the original location or to the original location (the remotely requested location).

[0117] If it is not determined that the vehicle 30 can return to the remotely requested location (S110: No), the flow of FIG. 4 proceeds to the processing of S114.

[0118] On the other hand, if it is determined that the vehicle 30 can return to the remotely requested location (S110: Yes), the control device 4 causes the vehicle 30 to move autonomously from the evacuation area to the remotely requested location (S111).

[0119] If it is determined that the vehicle is not interfering with surrounding traffic participants (S107: No), or after the vehicle has autonomously returned to the remotely requested location (S111), the flow in FIG. 4 returns to the processing of S104, and a determination is made about the operator's situation (S104 to S105).

[0120] Furthermore, if it is determined that the vehicle 30 will be in the way of nearby traffic participants when remote control becomes necessary (S102: Yes), the control device 4 autonomously drives the vehicle 30 to evacuate from the remote request location to the evacuation area (S112), similar to the process of S108. After autonomous evacuation, the control device 4 notifies the operator of the remote request (S113) and acquires information indicating the operator's status (S114), similar to the processes of S103 and S104. Thus, in the remote control system 1 according to the embodiment, if the vehicle 30 will be in the way of nearby traffic participants when remote control becomes necessary, the control device 4 evacuates the vehicle 30 before notifying the operator, and then notifies the operator after evacuation. The notification to the operator after evacuation may include details of the evacuation. Then, the control device 4 determines whether the operator is able to respond immediately (S115), similar to the process of S105.

[0121] Furthermore, if it is not determined that the operator is able to respond immediately after autonomous evacuation (S115: No), the control device 4 proceeds to processing of S109, acquires information on surrounding objects in the vicinity of the evacuation destination and at the location where remote control is required and remote control is requested (S109), and determines whether the vehicle 30 can return to the remotely requested location (S110).

[0122] On the other hand, if it is determined that the vehicle is waiting at the remotely requested location or has autonomously evacuated and the operator is able to respond immediately (S105: Yes, S115: Yes), the control device 4 operates the vehicle 30 in accordance with the remote control by the operator (S116).

[0123] FIG. 5 is a sequence diagram showing an example of the flow of processing executed in the remote control system 1 according to the embodiment.

[0124] First, assume that an event requiring remote control (e.g., avoiding roadside parking) occurs, such as when a roadside parked vehicle is parked in front of the vehicle 30. Also assume that the vehicle 30 is not interfering with surrounding traffic participants at the stage when remote control becomes necessary and has stopped at the remotely requested position.

[0125] When remote control becomes necessary, the vehicle 30 transmits remote request information and vehicle status (stopping on the spot) to the server device 10 (S201). The server device 10 also receives the remote request information and vehicle status from the vehicle 30 and, based on the received information, transmits the vehicle status and remote request to the operator terminal 20 operated by the operator monitoring the vehicle 30 (S202). The operator terminal 20 receives the vehicle status and remote request from the server device 10, notifies the operator remotely monitoring or controlling at least one vehicle 30 including the vehicle 30, and transmits information indicating the operator's status to the server device 10 (S203). The server device 10 also receives information indicating the operator's status from the operator terminal and, based on the received information, transmits operator availability information indicating whether the operator in charge can immediately respond to the vehicle 30 (S204).

[0126] Here, it is assumed that the operator is unable to respond immediately, and that the vehicle 30 remains stopped at the remotely requested position without interfering with surrounding traffic participants.

[0127] The vehicle 30 transmits information about surrounding objects (vehicle surrounding information) for its current position (remotely requested position) and the vehicle status (stopped on the spot) to the server device 10 (S205). The server device 10 also receives the vehicle surrounding information and vehicle status from the vehicle 30, and based on the received information, transmits the vehicle surrounding information and vehicle status to the operator terminal 20 operated by the operator monitoring the vehicle 30 (S206). The operator terminal 20 receives the vehicle surrounding information and vehicle status from the server device 10, notifies the operator, and transmits information indicating the operator's status to the server device 10 (S207). The server device 10 also receives information indicating the operator's status from the operator terminal, and based on the received information, transmits operator availability information to the vehicle 30 (S208).

[0128] Here, it is assumed that the operator is still unable to take immediate action, and that the vehicle 30 autonomously retreats because it is interfering with surrounding traffic participants.

[0129] The vehicle 30 transmits information about surrounding objects at the current location of the evacuation destination (vehicle surrounding information), information about surrounding objects at the remotely requested location (remote request location vicinity information), and the vehicle status (autonomous evacuation) to the server device 10 (S209). The server device 10 also receives the vehicle surrounding information, the remote request location vicinity information, and the vehicle status from the vehicle 30, and transmits the vehicle surrounding information, the remote request location vicinity information, and the vehicle status to the operator terminal 20 operated by the operator monitoring the vehicle 30 based on the received information (S210). The operator terminal 20 receives the vehicle surrounding information, the remote request location vicinity information, and the vehicle status from the server device 10, notifies the operator, and transmits information indicating the operator's status to the server device 10 (S211). The server device 10 also receives information indicating the operator's status from the operator terminal, and transmits operator availability information to the vehicle 30 based on the received information (S212).

[0130] In this way, in the remote control system 1 according to this embodiment, the operator monitoring the autonomously driving vehicle 30 assists the autonomous driving of the vehicle 30 by performing remote control, such as remote operation, to intervene in the control of the vehicle 30 in response to a remote request from the vehicle 30.

[0131] For example, a remote control system is known that assigns an appropriate operator based on the status of a vehicle and the status of a monitoring operator. However, there may be cases where an operator cannot be assigned, such as when there are too many vehicles requiring remote control compared to the number of operators, or when an assigned operator is busy remotely controlling another vehicle 30 and is therefore unable to respond immediately. For this reason, it is necessary to consider situations where an operator cannot respond immediately when a remote control request is made.

[0132] For example, a remote control system is known in which, when remote control becomes necessary, a vehicle 30 stops on the spot and sends a remote request to an operator. However, if the operator cannot respond immediately, there is a possibility that the vehicle waiting for remote control by the operator may become a nuisance to the surrounding traffic participants, such as when another vehicle (surrounding traffic participant) approaches from behind while the vehicle is stopped on the spot and waiting for remote control by the operator. For this reason, there has been a demand for technology that enables safe and efficient remote control even in situations where an operator cannot respond immediately when a remote request is issued.

[0133] In this situation, the remote control system 1 according to the present embodiment is configured to determine whether an operator can immediately respond when a remote request requiring remote control is made. Furthermore, in the case where an operator cannot immediately respond, the vehicle 30 that has made the remote request in the remote control system 1 is configured to stop on the spot if it will not interfere with surrounding traffic participants, and to autonomously retreat if it does interfere with surrounding traffic participants.

[0134] In this way, when an operator is unable to respond immediately, the control of the vehicle 30 is changed depending on whether or not it will cause a hindrance to surrounding traffic participants, so that safe and efficient remote control can be performed even when the operator is unable to respond immediately.

[0135] For example, according to the above configuration, if the operator is able to respond immediately, the operator remotely controls the vehicle 30 that has stopped at the location (the point where a situation requiring remote control occurs), thereby shortening the time it takes to return to autonomous driving and improving the efficiency of remote control.

[0136] For example, according to the above configuration, if the operator is unable to respond immediately and there is a (high) possibility that the vehicle 30 will become a nuisance, the vehicle 30 will autonomously evacuate, so that the vehicle 30 waiting for remote control by the operator will not become a nuisance to surrounding traffic participants, thereby improving the safety of remote control.

[0137] For example, according to the above configuration, even if the operator cannot respond immediately, if there is no (low) possibility of being a nuisance, the operator can remotely control the vehicle 30 that has stopped on the spot, thereby shortening the time it takes to return to autonomous driving and improving the efficiency of remote control.

[0138] Furthermore, the remote control system 1 according to this embodiment is configured to determine whether the vehicle 30 that has made the remote request can autonomously return to the remotely requested position after evacuation, based on information about surrounding objects at both the remotely requested position and its current position after evacuation. Furthermore, in the remote control system 1, the vehicle 30 that has made the remote request is configured to return from the current evacuation position to the remotely requested original position (the position before evacuation) if it can return autonomously. With this configuration, the operator remotely controls the vehicle 30 that has stopped at the remotely requested position, thereby shortening the time it takes to return to autonomous driving and improving the efficiency of remote control.

[0139] (First Modification) In the above embodiment, when a situation requires remote control, a process flow is described in which it is determined whether to stop the vehicle on the spot and notify the operator, or to autonomously withdraw and then notify the operator, depending on whether the vehicle will interfere with surrounding traffic participants. However, this is not limited to this. The operator may be notified when a situation requires remote control. Figure 6 is a flowchart showing another example of the process flow executed by the control device 4 of the vehicle 30 according to the embodiment. Here, differences from the process flow in Figure 4 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0140] If it is determined that remote control is necessary (S101: Yes), the control device 4 stops at the location, i.e., notifies the operator of a remote control request at the point where remote control is necessary (S103). Thereafter, the control device 4 acquires information indicating the operator's situation (S104) in the same manner as in the flow of Fig. 4, and determines whether the operator is able to respond immediately (S105).

[0141] Furthermore, in the same manner as in the flow of Figure 4, if the operator cannot respond immediately (S105: No) and if the vehicle is interfering with surrounding traffic participants (S107: Yes), the control device 4 autonomously retreats (S108).

[0142] Furthermore, in the same manner as in the flow of FIG. 4, if the control device 4 can return to the remotely requested position (S110: Yes), the control device 4 autonomously returns to the remotely requested position (S111).

[0143] Furthermore, in the same manner as in the flow of Figure 4, the control device 4 waits at the location (the remotely requested location or the autonomously evacuated location) until it determines that the operator is able to respond immediately at the remotely requested location or the autonomously evacuated location (S104 to S107, S109 to S110, S114 to S115).

[0144] Then, in the same manner as in the flow of FIG. 4, if it is determined that the situation allows the operator to respond immediately (S105: Yes, S115: Yes), the control device 4 executes remote control by the operator (S116).

[0145] In this way, even if the configuration is such that the operator is notified when a situation arises where remote control is necessary, the same effects as those of the above-described embodiment can be obtained. Furthermore, with this configuration, the timing of notification to the operator can be accelerated, and remote control can be started promptly when the operator is able to respond.

[0146] (Second Modification) In the above embodiment, the process flow is described in which information indicating the operator's status is acquired (S104) after notifying the operator (S103), but this is not limiting. After acquiring the operator's status (S104), it may be determined whether to stop the robot on the spot and notify the operator, or to autonomously retreat and then notify the operator, depending on the operator's status (S103).

[0147] For example, when the operator cannot immediately respond and the situation would be a hindrance to surrounding traffic participants, the control device 4 autonomously retreats and then notifies the operator. For example, when the operator can immediately respond, or when the operator cannot immediately respond but would not be a hindrance to surrounding traffic participants, the control device 4 stops at the location (the point where remote control becomes necessary) and notifies the operator.

[0148] Even with this configuration, the same effects as those of the above-described embodiment can be obtained.

[0149] Hereinafter, application examples of the remote control system 1 according to the present disclosure will be described with reference to the drawings.

[0150] Here, we will explain a use case in which four vehicles 30a to 30d are assigned to one operator who operates the operator terminal 20. In this use case, the vehicle 30 in which an event requiring remote control occurs stops on the spot and notifies the operator of the remote control request. The other vehicles 30 continue to perform tasks such as driving as usual.

[0151] The operator monitors and remotely controls the multiple vehicles 30 assigned to him / her while viewing the display screen displayed by the display device of the display unit 201 of the operator terminal 20. This display screen includes multiple images for remote operation and monitoring of each vehicle 30, which are acquired by the cameras (sensors 301) of each of the multiple vehicles 30 to be monitored.

[0152] The display screen of the operator terminal 20 may be a screen or image generated by the operator terminal 20 based on display information from the server device 10, or may display an image (display information) generated in the server device 10.

[0153] 7 to 11 are diagrams showing examples of display screens 601 to 605 according to Application Example 1. As shown in Fig. 7 to 11, each of the display screens 601 to 605 includes, for example, images 701a to 701d for remote control and monitoring of each of the vehicles 30a to 30d to be monitored.

[0154] FIG. 7 illustrates an example of a display screen 601 in which a falling object 803 (surrounding object) is detected ahead of vehicle 30c and a remote control request (avoid falling object) event occurs. As shown in FIG. 7 , a notification 711 based on the remote control request information, such as "avoid falling object," is displayed on the display screen 601. The notification on the display screen may be superimposed on the image 701, or may be displayed in an area dedicated to displaying notification information, such as by providing an image area for displaying the image 701 and a notification area for displaying notification information in the display area for each vehicle 30. Furthermore, an image 701c of vehicle 30c, in which a remote control request event has occurred, may be highlighted on the display screen 601. In the example of FIG. 7 , the image 701c is highlighted by a bold outline. In this case, the operator is "monitoring" the other vehicles 30a, 30b, and 30d and is ready to immediately respond to a remote control request from vehicle 30c. The operator understands the remote request from the vehicle 30c on the display screen 601 and begins to respond to it.

[0155] 8 illustrates an example of the display screen 602 when the operator is performing remote control in response to a remote request from vehicle 30c and a parked vehicle 805 parked on the road ahead of the other vehicle 30b. That is, FIG. 8 illustrates an example of the display screen 602 when a remote request event (avoiding roadside parking) occurs for the other vehicle 30b while the operator is performing remote control in response to the remote request from vehicle 30c. As shown in FIG. 8 , a notification 712 based on the remote request information, such as "avoiding roadside parking," is displayed on the display screen 602. In this case, the operator is "remotely operating" the other vehicle 30c and is unable to immediately respond to the newly notified remote request from vehicle 30b.

[0156] Here, since the vehicle 30b that made the remote request is in a situation where the operator cannot respond immediately, while waiting for remote control by the operator, it obtains information about surrounding objects at the remotely requested position (current position) and determines whether or not they will be an obstacle to nearby traffic participants.

[0157] 9 illustrates an example of a display screen 603 when a vehicle 30b that has issued a remote request detects an approaching vehicle 807 approaching its own vehicle from behind while waiting for remote control by an operator and determines that stopping at the current location requested by the remote request would be a hindrance to nearby traffic participants. As shown in FIG. 9 , a notification 713 based on vehicle surroundings information, such as "approaching vehicle behind," is displayed on the display screen 603. In this case, the vehicle 30b that has stopped and is waiting at the remotely requested location autonomously moves to a shelter area 801 to avoid being a hindrance to nearby traffic participants, since it has been determined that the vehicle will be a hindrance to nearby traffic participants in a situation where the operator cannot immediately respond.

[0158] 10 illustrates an example of the display screen 604 when an approaching vehicle 807, approaching the vehicle 30b from behind and issuing a remote control request, passes by the side of the vehicle 30b. That is, FIG. 10 illustrates an example of the display screen 604 when the vehicle 30b, after autonomously evacuating to the evacuation area 801, acquires information about surrounding objects for both the evacuation destination and the remote control request location, and determines that the vehicle 30b can return to the remote control request location. As shown in FIG. 10 , the display screen 604 terminates the display of the “approaching vehicle behind” notification 713 after the approaching vehicle 807 has passed, resulting in a situation where there are no surrounding traffic participants approaching the remote control request location from behind. In this case, the vehicle 30b, which is evacuating to the evacuation area 801, is still unable to respond immediately to the operator, and it is determined that the vehicle 30b can return to the remote control request location, so the vehicle 30b autonomously moves from the evacuation area 801 to the remote control request location.

[0159] Now, it is assumed that the operator's remote control in response to the remote request from the vehicle 30c has ended.

[0160] 11 illustrates a display screen 605 when the operator's remote control in response to a remote request from vehicle 30c is completed. In other words, FIG. 11 illustrates a display screen 605 when a remote request event (street parking avoidance) occurs only for vehicle 30b. In this case, the operator begins intervention in response to the remote request from vehicle 30b, which the operator has been aware of by visually checking the display screens 602 to 605.

[0161] In this way, in the operator terminal 20 of the remote operation system 1 according to the present disclosure, even if the operator is currently responding to a remote request from one vehicle 30, the operator is notified of an event that requires remote control that has occurred regarding another vehicle 30. Therefore, the operator can easily grasp the remote request that he or she must respond to after finishing the remote control he or she is currently responding to.

[0162] Furthermore, in the operator terminal 20, if the waiting vehicle 30 obstructs surrounding traffic participants in relation to a remote request event that should be handled after the current remote control is completed, the vehicle 30 autonomously retreats and a notification is made based on information about surrounding objects detected by the vehicle 30. This allows the operator to understand or consider in advance the content of the remote control for the remote request that should be handled after the current remote control is completed.

[0163] Furthermore, in the operator terminal 20, when the vehicle 30 that has been evacuated returns to the remote request location, notifications based on information about surrounding objects regarding events for which a remote control request must be made after the current remote control has been completed are hidden. This allows the operator to understand or consider in advance the content of remote control requests that will be requested at each point in time regarding events for which a remote control request must be made after the current remote control has been completed.

[0164] 12 is a diagram showing an example of a display screen 606 according to a second application example. Here, differences from the display screen 603 in FIG. 9 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0165] 12 , when the vehicle 30b waiting for the operator to attend has evacuated so as not to interfere with surrounding traffic participants, a notification 714 to that effect, such as "Vehicle 2 evacuates to evacuation area," may be displayed on the display screen 606. This notification 714 is displayed, for example, on an image 701c of the vehicle 30c currently being attended to by the operator.

[0166] According to this configuration, the operator can easily understand that vehicle 30b has evacuated to evacuation area 801. Therefore, the operator can easily understand that when the response to vehicle 30c is completed at that point, the operator can perform roadside parking avoidance by remote control, with evacuation area 801 as the starting position.

[0167] 13 is a diagram showing an example of a display screen 607 according to a third application example. Here, differences from the display screen 604 in FIG. 10 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0168] 13 , when the vehicle 30b waiting for the operator's attention has returned to the remotely requested position (original position) after being evacuated, a notification 715 to that effect, such as "Vehicle 2 moved to original position," may be displayed on the display screen 607. This notification 715 is displayed, for example, on an image 701c of the vehicle 30c currently being attended to by the operator.

[0169] With this configuration, the operator can easily understand that vehicle 30b will return to the location where the remote request was made. Therefore, the operator can easily understand that when the response to vehicle 30c is completed at that point, the operator can perform roadside parking avoidance by remote control, starting from the original location.

[0170] 14 is a diagram showing an example of a display screen 608 according to a fourth application example. Here, differences from the display screen 603 in FIG. 9 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0171] 14 illustrates the display screen 608 in the case where, in addition to the parked vehicle 805a parked on the road and the approaching vehicle 807 approaching the host vehicle, a parked vehicle 805b in the evacuation area 801 is also detected in the example of FIG. 9 . In the example of FIG. 14 , vehicle 30b is obstructing surrounding traffic participants, but cannot retreat to the evacuation area 801 due to the presence of parked vehicle 805b. In this case, as shown in FIG. 14 , a notification 716 such as "Vehicle 2 cannot autonomously move to the evacuation area" may be displayed. This notification 716 is displayed, for example, on an image 701c of vehicle 30c that the operator is currently attending to.

[0172] With this configuration, the operator receives the notification and understands that the situation that is interfering with the surrounding traffic participants cannot be resolved. Therefore, the operator can interrupt the remote operation of vehicle 30c and remotely move vehicle 30b to another evacuation location, or call out to the surrounding traffic participants (e.g., approaching vehicle 807) using a speaker mounted on vehicle 30b to inform them that evacuation is not possible. The operator may also call out to the surrounding traffic participants (e.g., parked vehicles 805a and 805b) to resolve the situation that is interfering with the surrounding traffic participants.

[0173] 15 is a diagram showing an example of a display screen 609 according to a fifth application example. Here, differences from the display screen 604 in FIG. 10 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0174] FIG. 15 illustrates the display screen 609 in the example of FIG. 10 when, after autonomous driving has moved to the evacuation area 801, in addition to a parked vehicle 805a parked on the road and an approaching vehicle 807 approaching the vehicle, a parked vehicle 805b parked on the road is detected at or near the remotely requested location. In the example of FIG. 15 , vehicle 30b cannot autonomously move to the remotely requested location (original location) due to the presence of parked vehicle 805b. In this case, as shown in FIG. 15 , a notification 717 such as "Vehicle 2 cannot autonomously move to original location" may be displayed. This notification 717 is displayed, for example, on an image 701c of vehicle 30c currently being handled by an operator.

[0175] With this configuration, the operator receives the notification and understands that the situation that is interfering with surrounding traffic participants cannot be resolved. Therefore, the operator can easily understand that the difficulty of remote control in response to the remote request from vehicle 30b has increased and that it will take time for the vehicle to return to its original position, i.e., for the remote control to end. Therefore, the operator can consider dealing with other vehicles 30 until the situation of vehicle 30b improves, such as when the parked vehicle 805b that is blocking the vehicle's movement to its original position or the parked vehicle 805a parked on the road that is the target of avoidance is removed, making it easier for the vehicle to return to its original position by remote control. The operator may also try to improve the situation of vehicle 30b by calling out to surrounding traffic participants (e.g., parked vehicles 805a, 805b).

[0176] 16 is a diagram showing an example of a display screen 610 according to a sixth application example. Here, differences from the display screen 603 in FIG. 9 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0177] 16 , a notification 718 may be displayed on the display screen 610, such as "Vehicle 2 Possibly in the Way," to indicate that the vehicle 30b waiting for the operator's attention may be in the way of a nearby traffic participant (e.g., an approaching vehicle 807). This notification 718 is displayed, for example, on an image 701c of the vehicle 30c that the operator is attending to.

[0178] With this configuration, the operator can easily understand that a remote-requested event that may cause a disturbance to nearby traffic participants has occurred by receiving the notification, and can therefore consider taking measures such as suspending the remote operation of vehicle 30c and prioritizing the remote operation of vehicle 30b, which may cause a disturbance to nearby traffic participants.

[0179] 17 is a diagram showing an example of a display screen 611 according to a seventh application example. Here, differences from the display screen 610 in FIG. 16 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0180] In the example of FIG. 16 , whether or not the vehicle 30b that issued the remote request will evacuate may be determined by the operator. For example, as shown in FIG. 17 , the display screen 611 may further display a “Evacuate” operation button 751 and a “Do not evacuate” operation button 752 in addition to a notification 718 such as “Vehicle 2 Possibly Obstructing.” The “Evacuate” operation button 751 is, for example, an operation button for instructing the vehicle 30b that is obstructing surrounding traffic participants to autonomously evacuate. The “Do not evacuate” operation button 752 is an operation button for instructing the vehicle 30b not to evacuate autonomously. These operation buttons 751 and 752 are displayed, together with the notification 718, on an image 701c of the vehicle 30c that the operator is currently handling. Note that the operation buttons 751 and 752 may display either “Evacuate” or “Do not evacuate.” The operation buttons 751 and 752 may be displayed when it is detected that there is a possibility of disturbing surrounding traffic participants and / or when it is detected that the vehicle has become a disturbance.

[0181] With this configuration, the operator can easily understand that a remotely requested event that may cause a disturbance to surrounding traffic participants has occurred by receiving the notification. Furthermore, when a remotely requested event that may cause a disturbance to surrounding traffic participants has occurred, the operator can consider whether to interrupt the remote operation of vehicle 30c that is currently being handled and have vehicle 30b evacuate, or to not evacuate vehicle 30b and call out to the surrounding traffic participants that evacuation is not possible.

[0182] 18 is a diagram showing an example of display screens 612 and 613 according to an eighth application example. Here, differences from the display screen 602 in Fig. 8 and the display screen 603 in Fig. 9 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0183] 18 illustrates a display screen 612 that appears when a new remote control request is made from vehicle 30b while the operator is remotely operating vehicle 30c that has issued the remote control request. Display screen 612 may display the current time, "11:12," and a notification 719 of the expected completion time of the remote control response. The notification 719 of the expected completion time of the remote control response is displayed, for example, on an image 701c of vehicle 30c that the operator is currently operating.

[0184] 18 illustrates a display screen 613 that appears when, after the display screen 612 is displayed, it is determined that the vehicle 30b waiting for the operator's response will be a hindrance to nearby traffic participants. The display screen 613 may display the current time "11:13" and a notification 719 of the expected time of completion of the remote control response, similar to the display screen 612. The display screen 613 may also display a notification 720 of the expected time of approach of the approaching vehicle 807 approaching from behind, as an example of the expected time when the vehicle 30b waiting after making a remote request will be a hindrance to nearby traffic participants. This notification 720 is displayed, for example, on the image 701b of the vehicle 30b waiting for the operator's response.

[0185] In this configuration, vehicle 30b obtains from server device 10 or operator terminal 20 the expected response time for the remote control currently being handled by the operator, i.e., the expected time when the operator will be able to respond immediately. Vehicle 30b also predicts the expected time when approaching vehicle 807 approaching from behind will arrive near the vehicle and the vehicle will become a hindrance to surrounding traffic participants. Based on the expected time for the operator's remote response and the expected time for the vehicle to become a hindrance to surrounding traffic participants, vehicle 30b may then determine whether to autonomously evacuate to an evacuation area or to stop on the spot (the location where the remote request was made) and wait for the operator to become available.

[0186] For example, if an approaching vehicle 807 approaching the vehicle from behind arrives near the vehicle by the expected time for the operator to respond to the remote control and the vehicle becomes an obstacle to surrounding traffic participants, vehicle 30b decides to autonomously evacuate to an evacuation area.

[0187] For example, as shown in FIG. 18, if approaching vehicle 807 does not arrive near the vehicle by the expected time for the operator to respond remotely, the operator can respond before the vehicle interferes with surrounding traffic participants, and vehicle 30b decides to wait at the remotely requested location (the same location).

[0188] The display screen 613 may display a notification 721 regarding the determination of vehicle 30b based on the expected time for the operator to respond to the remote control and the expected time for which the vehicle will interfere with surrounding traffic participants. In the example of Fig. 18, the notification 721 displays a message such as "Waiting because the expected time for remote control response is short." This notification 721 is displayed, for example, on an image 701c of vehicle 30c that the operator is currently responding to.

[0189] This configuration allows the operator to respond flexibly according to the situation.

[0190] 19 is a diagram showing an example of display screens 614 and 615 according to a ninth application example. Here, differences from the display screen 603 in Fig. 9 and the display screen 604 in Fig. 10 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0191] FIG. 19 illustrates an example of a display screen 614 in which, while an operator is remotely operating a vehicle 30c that has issued a remote request, a vehicle 30b waiting for the operator's response is determined to be interfering with surrounding traffic participants and autonomously retreats. The display screen 614 may display the current time "14:27," a notification 722 of the expected end time of the remote control response, and a notification 723 of the expected time of approach of an approaching vehicle 807 approaching from behind, i.e., the expected time of vehicle 30b interfering with surrounding traffic participants. FIG. 19 also illustrates an example of a display screen 615 in which, after the display screen 614 is displayed, the autonomously retreated vehicle 30b returns to its original position. Similar to the display screen 614, the display screen 615 may display the current time "14:29," a notification 722 of the expected end time of the remote control response, and a notification 723 of the expected time of approach of an approaching vehicle 807 approaching from behind. The notification 722 is displayed on the image 701c of the vehicle 30c currently being attended to by an operator, while the notification 723 is displayed on the image 701b of the vehicle 30b currently waiting for an operator to attend to the vehicle.

[0192] In this configuration, the vehicle 30b acquires from the server device 10 or the operator terminal 20 the expected response time for the remote control currently being handled by the operator, i.e., the expected time when the operator will be able to respond immediately. The vehicle 30b also acquires information about surrounding objects at the current location of the evacuation destination (vehicle surroundings information) and information about surrounding objects at the location where the remote control request was made (remote request location vicinity information). The vehicle 30b then determines whether to autonomously return to the location where the remote control request was made based on the remaining time until the expected time for the operator's remote response, the vehicle surroundings information, and the remote request location vicinity information.

[0193] For example, when there is one minute (less than the predetermined remaining time) remaining until the expected time for remote control response and when it is possible for the vehicle 30b to travel to the location where the remote request occurred and to stop at the location where the remote request occurred, the vehicle 30b autonomously travels to the location where the remote request occurred. Note that the value of the predetermined remaining time can be set arbitrarily and is, for example, predetermined and stored in the memory of the vehicle 30.

[0194] With this configuration, when the remaining time until an operator becomes available becomes short, vehicle 30b can autonomously return to the position where it was waiting when the remote request was issued, depending on the surrounding conditions. If vehicle 30b is in the position where it was waiting when the remote request was issued, it is possible to omit the remote control process by the operator, which involves returning vehicle 30b from the evacuation area to the location where the remote request was issued.

[0195] 20 is a diagram showing an example of a display screen 616 according to a tenth application example. Here, differences from the display screen 603 in FIG. 9 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0196] 20 illustrates a display screen 616 in which there are multiple candidates for the evacuation area 801 to which the vehicle 30b autonomously retreats when it is determined that the vehicle 30b will be in the way of nearby traffic participants. The display screen 616 displays multiple candidates for evacuation destinations (evacuation areas 801a, 801b) for the vehicle 30b that has been determined to be in the way of nearby traffic participants.

[0197] In this configuration, when there are multiple candidate evacuation areas 801, vehicle 30b selects the evacuation area 801 that is easy to remotely control, taking into consideration the possibility that vehicle 30b may not be able to return to the location where the remote request was issued after evacuation. For example, in the example of FIG. 20 , if evacuation area 801b is selected, vehicle 30b will retreat and evacuate. Under such circumstances, evacuation area 801a is an evacuation area that is difficult to remotely control because a parked vehicle 805 parked on the road nearby. Furthermore, after vehicle 30b evacuates to evacuation area 801a, if another vehicle from behind, such as approaching vehicle 807 approaching vehicle 30b from behind, parks on the road behind parked vehicle 805, vehicle 30b may not be able to return to the location where the remote request was issued. Therefore, in the example of FIG. 20 , vehicle 30b selects evacuation area 801b, which is easy to remotely control, taking into consideration the possibility that vehicle 30b may not be able to return to the location where the remote request was issued after evacuation.

[0198] As an example, in order to select an evacuation area 801 that is easy to remotely control, the vehicle 30 acquires surrounding object information for each of a plurality of candidate evacuation areas 801. This surrounding object information includes, for example, information on surrounding objects that exist within a range defined by a specified distance X, and information on approaching surrounding objects that exist within a range defined by a specified distance Y (>X). For example, the surrounding object information includes information that there are two people nearby and information that there is one approaching vehicle.

[0199] Furthermore, the vehicle 30 calculates a remote control difficulty level indicating the ease of remote control for each evacuation area 801 according to a score (influence level) set in advance according to the type of surrounding object (surrounding object type) and the number of surrounding objects. Note that information indicating the correspondence between the surrounding object type and the influence level is, for example, predetermined and stored in the memory of the vehicle 30, but may also be configured to be acquired by the vehicle 30 from an external source, such as being stored in the memory of the server device 10.

[0200] Then, based on the calculated remote control difficulty, the vehicle 30 selects an evacuation area 801 that is easy to remotely control. For example, the vehicle 30 selects the evacuation area 801 with the lowest calculated remote control difficulty.

[0201] For example, in the information indicating the correspondence between surrounding object types and influence levels, influence levels of "0.5," "1.0," "1.5," and "2.0" are pre-set for the surrounding object types of "person," "bicycle," "motorcycle," and "car," respectively. In the example of FIG. 20 , the candidate escape areas 801 are escape areas 801a and 801b. In the example of FIG. 20 , the surrounding object information for escape area 801a indicates that there is one vehicle (parked vehicle 805) in the vicinity and one approaching vehicle (approaching vehicle 807). In this case, the vehicle 30 calculates the remote control difficulty level as follows: (Remote control difficulty level for escape area 801a) = 2.0 × 1 + 2.0 × 1 = 4.0. In the example of FIG. 20 , the surrounding object information for escape area 801b indicates that there is one vehicle (approaching vehicle 807) in the vicinity and no approaching objects. In this case, the vehicle 30 calculates the remote control difficulty level as follows: (Remote control difficulty level of evacuation area 801b)=2.0×1=2.0 Therefore, in the example of FIG. 20, evacuation area 801b has the lowest remote control difficulty level of evacuation areas 801a and 801b, and therefore vehicle 30 selects evacuation area 801b.

[0202] This configuration reduces the effort required for the operator to remotely control the vehicle 30 when returning it from the evacuation area to the location where the remote request occurred, and shortens the time required for the operator to remotely control the vehicle, thereby speeding up the return to autonomous driving and improving the efficiency of remote control.

[0203] (Eleventh Application Example) Note that selection from among multiple evacuation area candidates may be based on the communication environment in addition to or instead of ease of remote control. Here, differences from the tenth application example will be mainly described with reference to Figure 20, and overlapping descriptions will be omitted as appropriate.

[0204] 20, the evacuation area 801a is assumed to be an evacuation area 801 having a poor communication environment based on past communication conditions, while the evacuation area 801b is assumed to be an evacuation area 801 having a good communication environment based on past communication conditions.

[0205] In this configuration, vehicle 30b takes into consideration the possibility that it may not be able to return to the location where the remote request occurred after evacuation, and when there are multiple candidate evacuation areas 801, vehicle 30b takes into consideration past communication conditions and selects evacuation area 801 with a good communication environment. For example, vehicle 30b will evacuate by backing up, but selects evacuation area 801b with a good communication environment based on past communication conditions.

[0206] This configuration makes it possible to avoid evacuation areas with poor communication environments, thereby stabilizing communication with the vehicle 30 and improving the safety and efficiency of remote control.

[0207] 21 is a diagram showing an example of a display screen 617 according to a twelfth application example. Here, differences from the display screen 604 in FIG. 10 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0208] 21 illustrates a display screen 617 in the example of Fig. 10 when the vehicle has moved to the evacuation area 801 by autonomous driving and there are no more parked vehicles 805 parked on the road, i.e., when the situation no longer requires remote control by the operator. When returning to autonomous driving, the operator may be notified that the vehicle has returned to autonomous driving and / or that remote control is no longer required.

[0209] If vehicle 30b retreats to evacuation area 801 and is able to return to autonomous driving directly from the retreat location, such as when the surrounding traffic participants (e.g., parked vehicle 805) that caused the remote control to occur have disappeared, vehicle 30b may not autonomously return to the location where the remote control occurred, but may instead return to autonomous driving as it was before the remote request was made.

[0210] With this configuration, when the vehicle is able to return to autonomous driving at the evacuation destination, the time required to return to the point where remote control occurred is eliminated, thereby reducing the burden on the operator related to remote control.

[0211] 22 is a diagram showing an example of a display screen 618 according to a thirteenth application example. Here, differences from the display screen 604 in FIG. 10 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0212] FIG. 22 illustrates the display screen 618 in the case where, in the example of FIG. 10, there is a location 809b where remote control is easier than returning to the location 809a where the remote request occurred.

[0213] In this configuration, the vehicle 30b determines whether there is a location that is easier to remotely control than the location 809a where the remote request was made, based on information about surrounding objects for the current location of the evacuation destination (vehicle surrounding information) and information about surrounding objects for the location 809a where the remote request was made (remote request location vicinity information). If there is a location 809b that is easier to remotely control than the location 809a where the remote request was made, the vehicle 30b autonomously moves to the location 809b where remote control is easier, instead of the location 809a where the remote request was made.

[0214] As an example, the vehicle 30 acquires surrounding object information for each of the evacuation area 801 (current location) where the evacuation destination is located and the location 809a (original location) where the remote control request was made, in order to determine and select a location 809b that is easier to remotely control than the location 809a where the remote control request was made. This surrounding object information includes, for example, information on surrounding objects that exist within a range defined by a specified distance X and information on approaching surrounding objects that exist within a range defined by a specified distance Y (>X). For example, the surrounding object information includes information that there are two people nearby and information that there is one approaching vehicle.

[0215] Furthermore, the vehicle 30 calculates a remote control difficulty level indicating the ease of remote control for each of the current location (evacuation area 801 where the vehicle 30 is to evacuate) and the original location (location 809a where the remote control request was made) according to a score (influence level) previously set according to the type of surrounding object (surrounding object type) and the number of surrounding objects. Note that information indicating the correspondence between the surrounding object type and the influence level is, for example, predetermined and stored in the memory of the vehicle 30, but may also be configured to be acquired by the vehicle 30 from an external source, such as being stored in the memory of the server device 10.

[0216] Based on the calculated remote control difficulty, the vehicle 30 determines and selects a location 809b that is easier to remotely control than the location 809a where the remote control request was made. For example, the vehicle 30 compares the remote control difficulty calculated between the current location, which is the evacuation destination, and the original location. Furthermore, for example, if the remote control difficulty of the current location, which is the evacuation destination, is lower than the remote control difficulty of the original location, the vehicle 30 notifies the operator of a remote control request from the evacuation destination. Furthermore, for example, if the remote control difficulty of the original location is equal to or lower than the remote control difficulty of the current location, the vehicle 30 determines whether the remote request difficulty of the original location is "0." If the remote request difficulty of the original location is not "0," the vehicle 30 determines whether there is a location around the original location where the remote control difficulty is lower than the remote control difficulty. Furthermore, if there is a location around the original location where the remote control difficulty is lower, the vehicle 30 autonomously moves to that location and then notifies the operator of a remote control request.

[0217] For example, in the information indicating the correspondence between surrounding object types and influence levels, influence levels of "0.5," "1.0," "1.5," and "2.0" are pre-set for the surrounding object types of "person," "bicycle," "motorcycle," and "car," respectively. In the example of FIG. 22 , the surrounding object information for the evacuation area 801, which is the current location and the evacuation destination, indicates that there are two vehicles (parked vehicle 805, approaching vehicle 807) in the vicinity and no approaching objects. In this case, the vehicle 30 calculates the remote control difficulty level as follows: (Remote control difficulty level of current location) = 2.0 × 2 = 4.0. Also, in the example of FIG. 22 , the surrounding object information for the location 809 a, which is the original location and where the remote request was made, indicates that there are two vehicles (parked vehicle 805, approaching vehicle 807) in the vicinity and no approaching objects. In this case, the vehicle 30 calculates the remote control difficulty level as follows: (Remote control difficulty level of original location) = 2.0 × 2 = 4.0.

[0218] Therefore, in the example of FIG. 22 , the relationship (remote control difficulty of current location) < (remote control difficulty of original location) does not hold, so the vehicle 30 selects the original location, location 809a, where the remote control request was made. Furthermore, since (remote control difficulty of original location = 4.0) ≈ 0, the vehicle 30 searches for a location within the surrounding radius Z where the remote control difficulty will be lower than that of the original location. Here, in the example of FIG. 22 , the surrounding object information for location 809b surrounding the original location indicates that there is one vehicle (approaching vehicle 807) in the vicinity and no approaching objects. In this case, the vehicle 30 calculates the remote control difficulty as follows: (remote control difficulty of surrounding location) = 2.0 × 1 = 2.0. Furthermore, since the relationship (remote control difficulty of surrounding location = 2.0) < (remote control difficulty of original location = 4.0) holds, the vehicle 30 selects location 809b as a location where remote control is easy.

[0219] This configuration further improves the efficiency of remote control and enables a quicker return to autonomous driving.

[0220] 23 is a diagram showing an example of a display screen 619 according to a fourteenth application example. Here, differences from the display screen 604 in FIG. 10 will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0221] 23 illustrates a display screen 619 in the example of FIG. 10 when the service provided by the remote operation system 1 is "monitoring." Here, the "monitoring" service refers to a service in which a remote operation system 1 wanders around the service area and "monitors" whether there are any dangerous objects or suspicious nearby traffic participants. When a dangerous object or suspicious nearby traffic participant is discovered, the "monitoring" service tracks the object or nearby traffic participant so as not to lose sight of it, and notifies an operator as necessary, thereby arranging for security guards, etc. In other words, FIG. 23 illustrates a display screen 619 in the case where a pedestrian 811, a nearby traffic participant to be monitored, is present near the evacuation area 801 where the vehicle 30b is to take refuge, while the "monitoring" service is being provided.

[0222] In this configuration, after retreating to the evacuation area, the vehicle 30b determines whether to return to the location where the remote request was made by further considering not only whether it is physically possible to return to the location where the remote request was made, but also whether it is better to return to the location where the remote request was made from the perspective of service efficiency. In other words, for example, after retreating to the evacuation area, the vehicle 30b determines whether to return to the location where the remote request was made in accordance with the type of service provided by the remote operation system 1.

[0223] For example, if the service is "monitoring," it is more efficient as a service if vehicle 30b is near a suspicious person, because it is less likely that a problem will occur and if a problem does occur, it will be noticed immediately. For this reason, in the example of Fig. 23, although vehicle 30b is in a situation where it can physically return, since the service is "monitoring," from the perspective of service efficiency, it determines not to return to the location where the remote request was made in order to continue monitoring in a situation close to the surrounding traffic participants (suspicious person).

[0224] According to this configuration, remote control according to the service to be provided can be realized, and the service efficiency in the remote control can be improved.

[0225] 24 is a diagram showing an example of display screens 620 and 621 according to a fifteenth application example. Here, differences from the ninth application example will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0226] In the ninth application example, the vehicle 30b autonomously returns to the location where the remote request was made when the remaining time until the expected time for the operator to respond to the remote request becomes a predetermined time, but this is not limited to this. For example, the timing for autonomously returning the vehicle 30b to the location where the remote request was made may be determined arbitrarily by the operator.

[0227] 24 , an operation button 753 such as “return vehicle 2” may be displayed on the display screens 620 and 621. The operation button 753 for “return vehicle 2” is an operation button for instructing, for example, vehicle 30b that has retreated to the retreat area 801 to autonomously return to the location where the remote request occurred.

[0228] In this configuration, when the operator operates the "return vehicle 2" operation button 753, vehicle 30b begins to move autonomously back to the location where the remote request occurred.

[0229] With this configuration, the operator can autonomously return the vehicle 30 from the evacuation position to the position where remote control becomes necessary at a desired timing. Therefore, while the operator is remotely operating one vehicle 30, the operator can return another vehicle 30 that is scheduled to be remotely operated next from the evacuation position at a desired timing, such as when the remote operation is about to end. In this way, since the vehicle 30 can be returned from the evacuation position before the remote operation begins, the operator can smoothly start remote control of the vehicle 30 and reduce the effort required.

[0230] 25 is a diagram showing an example of display screens 622 and 623 according to the sixteenth application example. Here, differences from the ninth application example or the fifteenth application example will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0231] The ninth application example illustrates a case where the vehicle autonomously returns to the location where the remote request was made when the remaining time until the expected time for the operator to respond remotely reaches a predetermined time, and the sixteenth application example illustrates a case where the vehicle autonomously returns to the location where the remote request was made at a timing desired by the operator, but this is not limiting. For example, the timing for autonomously returning to the location where the remote request was made may be determined by the vehicle 30b.

[0232] In this configuration, after the vehicle 30b has evacuated, it returns from the evacuated position to the position where remote control is required at a timing that corresponds to the operator's remote control response status (information indicating the remote control status being handled by the operator).

[0233] For example, the vehicle 30b being evacuated may acquire position information of the vehicle 30c being remotely controlled by the operator and position information of a surrounding object such as a fallen object 803 that caused the vehicle 30c to make a remote control request, and may acquire a positional relationship 813 between these as the operator's response status of the remote control. Then, the vehicle 30b determines the response status (progress) of the vehicle 30c being remotely controlled by the operator based on the positional relationship 813. Furthermore, the vehicle 30b autonomously starts moving back to the location where the remote request was made at a timing when the operator can immediately respond to the remote request from its own vehicle, such as when the operator's remote control of another vehicle 30c is about to finish, as in the situation of positional relationship 815.

[0234] For example, the vehicle 30b being evacuated may acquire video of an operator remotely controlling another vehicle 30c. This video may be acquired by a camera (not shown) installed in the operator terminal 20, or may be acquired by a camera (not shown) installed in a remote control room where the operator terminal 20 is installed. Then, based on the acquired video of the operator, the vehicle 30b may determine the response status (progress) of the operator's remote control of the other vehicle 30c.

[0235] With this configuration, the timing of the return is determined autonomously by the vehicle 30b, so that the waiting vehicle 30b can be returned autonomously at an appropriate timing without placing a burden on the operator. Therefore, the operator can start remote control of the vehicle 30b that has returned to the position where remote control is required.

[0236] (17th Application Example) In a configuration in which the operator decides whether to evacuate the vehicle 30 that has made the remote request, or whether to return the vehicle 30 to a position where remote control is required after evacuating, if the operator is concentrating on responding to the remote control of another vehicle 30, even if a notification requesting such a decision is made, the operator may not be able to respond because he or she does not have time to operate the operation button or does not notice the notification.

[0237] Therefore, in a configuration in which the operator decides whether to evacuate the vehicle 30 that has been remotely requested, or whether to return the vehicle 30 to a position where remote control is required after evacuating, the vehicle 30b may be configured to make an automatic decision if the operator's decision is not obtained within a predetermined count (which may be a predetermined time).

[0238] With this configuration, even if the operator is unable to respond to the notification of the judgment request due to responding to the remote control of another vehicle 30, it is possible to prevent a situation in which the operator has to continue waiting for a response.

[0239] (18th Application Example) In addition, an example of determining whether evacuation is necessary to separate control such as whether or not the remotely requested vehicle 30 should autonomously evacuate from its current position to the evacuation area 801 at the evacuation destination is given, where it is determined whether or not the vehicle 30 will be a hindrance to surrounding traffic participants, but this is not limited to this.

[0240] As an example, the determination of whether evacuation is necessary may be based on whether the location where the remote request occurred is within a predetermined area. This predetermined area may be, for example, an area that has been pre-registered as a dangerous area. Information indicating the predetermined area may be, for example, predetermined and stored in the memory of the vehicle 30. For example, if the location where the remote request occurred is within a dangerous area (predetermined area), the vehicle 30 may determine to evacuate to the evacuation destination. Note that the information indicating the predetermined area is not limited to areas to which evacuation is permitted, and may also include information indicating areas to which evacuation is not permitted.

[0241] As an example, the determination of whether evacuation is necessary may be based on whether there is only one candidate evacuation destination. For example, when there is only one candidate evacuation destination, the vehicle 30 may determine to evacuate to that candidate evacuation destination. This configuration can prevent situations in which the evacuation destination becomes unusable due to a change in the situation, such as a person arriving at the evacuation destination or a bicycle being parked there, making it necessary to evacuate, and making it impossible to evacuate.

[0242] As an example, the determination of whether evacuation is necessary may be based on whether the expected time for the operator to start responding exceeds a predetermined time. For example, the vehicle 30 may determine to evacuate to the evacuation destination when the expected time for the operator to start responding is longer than the predetermined time. With this configuration, even if there is a high possibility that evacuation will be necessary when a long wait is required, such as in a place with heavy traffic, the vehicle can move ahead based on the determination of whether evacuation is necessary.

[0243] If it is determined that the vehicle 30 should be evacuated to a destination based on the expected time for the operator to start responding, the vehicle 30 may wait for the best destination to become available and move to that destination when it becomes available. This best destination may be, for example, the nearest candidate destination or a candidate destination that is easy to remotely control.

[0244] The determination of whether evacuation is necessary based on the expected time when the operator will begin responding may be made when the location where the remote request occurred is a predetermined area, which may be, for example, an area registered in advance as a place with heavy traffic.

[0245] (19th Application Example) In this application example, it is assumed that each of the plurality of vehicles 30 has retreated to a retreat destination and is waiting for remote control by an operator.

[0246] As an example, when multiple vehicles 30 are waiting at an evacuation destination, remote control by the operator may be initiated in the order in which the remote requests were made, and if there is an urgent vehicle 30, the order of remote control may be changed so that the urgent vehicle 30 is given priority, and remote control by the operator may be initiated.

[0247] For example, the estimated time to start remote control by the operator changes depending on the change in the order of the vehicles 30 to be remotely controlled (remote response order). Therefore, when the estimated start time changes depending on the change in the remote response order, the server device 10 or the operator terminal 20 may transmit the changed estimated start time to the vehicle 30. Furthermore, the vehicle 30 that receives this may control the method of waiting at the evacuation destination depending on the changed time.

[0248] For example, controlling the standby method at the evacuation destination in response to a change in the estimated start time may involve extending the standby time or changing the evacuation destination. For example, if the remote response at the changed estimated start time exceeds the standby time at the evacuation destination, the vehicle 30 may change the evacuation destination. An example of a case where the standby time at the evacuation destination is set may be when the vehicle cannot be parked in a no-parking area for a long period of time.

[0249] According to this configuration, when each of the multiple vehicles 30 has retreated to a retreat location and is waiting for remote control by an operator, it is possible to appropriately handle the change in the remote response order.

[0250] (Twentieth Application Example) In this application example, a response will be described when the timing to start remote control is earlier than expected. For example, the timing of remote control may be earlier than expected when another operator assists with remote control on behalf of the vehicle, or when another vehicle that was waiting under remote control before the vehicle no longer needs remote control. An example of a situation in which another vehicle that was waiting under remote control before the vehicle no longer needs remote control is when the parked vehicle that caused the other vehicle to request assistance has disappeared, making remote control avoidance unnecessary. When the timing of remote control is earlier than expected, the remote operator is in a state where they can start remote control, but the remotely controlled vehicle 30 is waiting at an evacuation destination.

[0251] As an example, the operator terminal 20 may display a button that allows the operator to select between "starting remote control after autonomous movement" and "starting remote control from the site." Here, "starting remote control after autonomous movement" means that the operator sends a remote instruction from the operator terminal 20 to the waiting vehicle 30 to return to the original location, and then starts remote control after moving the vehicle 30 by autonomous movement. Note that this button display may display either an operation button for instructing "starting remote control after autonomous movement" or an operation button for instructing "starting remote control from the site."

[0252] As an example, when the operator terminal 20 acquires an operator operation of "starting remote control from the spot," the operator terminal 20 may further display a button that allows the operator to select between "moving to the original location, avoiding obstacles, and returning to the autonomous driving route" and "moving via a different route without passing through the original location and returning to the autonomous driving route." Note that the operator terminal 20 may present a guide indicating the route that will be taken for each option and allow the operator to select one of the options. Furthermore, after one of the options is selected, the operator terminal 20 may be configured to display a guide for the selected option when actually moving the target vehicle 30 by remote control, thereby supporting the operator in remote control.

[0253] As an example, since the vehicle 30 to be remotely controlled is currently at a shelter, the operator terminal 20 may display the original location of the vehicle 30 by superimposing it on the image or map.

[0254] As an example, the operator terminal 20 may display what the remote request originally desired to be addressed, i.e., the content of the remote request. For example, the operator terminal 20 may present what the remote request originally desired to be addressed by displaying an image of the remote request at the original location. This image may be, for example, a still image of the traveling direction, such as an image showing an obstacle in the traveling direction. Furthermore, this image may be displayed as a superimposed image on a video or map, or may be displayed separately.

[0255] This configuration makes it easier for the operator to visualize what he or she needs to do.

[0256] (21st Application Example) In this application example, a case where a save destination is determined depending on the skill of an operator will be exemplified.

[0257] As an example, when the operator who will perform remote control of the vehicle 30 is known in advance or can be predicted, the vehicle 30 may determine the evacuation destination according to the skill and experience of the operator. For example, the vehicle 30 may determine the evacuation destination under the condition that if the operator is highly skilled, it is OK to select an evacuation destination that is difficult to remotely control. For example, if the operator is less skilled, the vehicle 30 may determine the evacuation destination under the condition that the evacuation destination must be easy to remotely control.

[0258] This process may be performed not only by the vehicle 30 but also by any device in the remote operation system 1, such as the server device 10 or the operator terminal 20. The correspondence between the skills and / or experience of the operator and the conditions for determining the evacuation destination may be predetermined and stored in the memory of any device in the remote operation system 1. The determination of the conditions for determining the evacuation destination based on the skills and / or experience of the operator and the determination of the evacuation destination may be performed by different devices in the remote operation system 1.

[0259] (Twenty-Second Application Example) In this application example, a save destination is determined according to the period (remaining time) until the expected start time of remote control.

[0260] As an example, the vehicle 30 may change the evacuation destination depending on the time until remote control becomes possible. For example, the vehicle 30 may determine or change the evacuation destination if the time until the expected start time of remote control becomes possible exceeds a predetermined time stored in memory, i.e., if it takes a long time until start, under the condition that a distant evacuation destination is acceptable. For example, the vehicle 30 may determine or change the evacuation destination if the time until the expected start time of remote control becomes possible is equal to or less than a predetermined time stored in memory, i.e., if it takes only a short time until start, under the condition that a distant evacuation destination is not acceptable and a nearby evacuation destination must be found. Note that the predetermined time (threshold) for a long time and the predetermined time (threshold) for a short time may be the same or different.

[0261] (Twenty-Third Application Example) In this application example, a save destination is determined according to the degree of possibility of returning from the save destination position to the original position.

[0262] As an example, the vehicle 30 may determine the evacuation destination by determining the degree of possibility of returning to the original position from the evacuation destination position. For example, in a location with heavy traffic, it is difficult to return to the original position from the evacuation destination, so it is considered that the vehicle 30 will need to be remotely controlled from the evacuation destination. Therefore, for example, if the degree of possibility of returning to the original position from the evacuation destination position is lower than a predetermined degree that is determined in advance and stored in memory, the vehicle 30 may determine as the evacuation destination a location that is easy to remotely control.

[0263] (Twenty-fourth Application Example) In this application example, a display for people inside a vehicle when transporting people will be illustrated.

[0264] As an example, a vehicle 30 carrying passengers, such as a taxi or bus, may notify passengers of the evacuation (anxiety suppression notification). Furthermore, the vehicle 30 may notify passengers of the reason for evacuation when returning from the evacuation destination to its original location. Furthermore, the vehicle 30 may notify passengers of the estimated time until remote control begins. These notifications may be displayed on a display of the vehicle 30 that is visible to passengers inside the vehicle, or may be sent to a mobile device, such as a smartphone, carried by the passengers inside the vehicle. Instead of or in addition to a display, the notification may be made by audio or other means.

[0265] (25th Application Example) In this application example, a movement to another save destination will be illustrated.

[0266] As an example, after evacuating to a certain evacuation destination, the vehicle 30 may wait at the same location or may move to another evacuation destination. For example, the vehicle 30 may move to an evacuation destination that makes it easier to return to the original location, such as an evacuation destination close to the original location. For example, if it is difficult to return to the original location, the vehicle 30 may move to an evacuation destination that makes it easier to remotely control the vehicle 30. This is based on the fact that if it is difficult to return to the original location, there is a high possibility that remote control will be initiated from one of the evacuation destinations.

[0267] Note that multiple trips to another evacuation destination consume a large amount of battery power. Therefore, the vehicle 30 may determine whether to wait at the same evacuation destination or to move to another evacuation destination based on the battery status. For example, the vehicle 30 may determine not to move to another evacuation destination if the available battery power is less than a predetermined power amount determined and stored in memory, or if the battery voltage is less than a predetermined voltage value determined and stored in memory. For example, the vehicle 30 may determine not to move to another evacuation destination if the expected battery power consumption for moving to another evacuation destination does not satisfy a predetermined condition determined and stored in memory. Furthermore, a limit may be set on the number of trips to another evacuation destination.

[0268] 6 , the above-described embodiment illustrates a case where, in a situation where remote control is required (S101: Yes) and the vehicle 30 would be in the way of surrounding traffic participants (S102: Yes), the vehicle 30 is first moved out of the way (S112) before notifying the operator (S112). On the other hand, the remote operation system 1 according to the above-described embodiment may be configured to move the vehicle 30 out of the way (S112) before notifying the operator when remote control is required, regardless of whether the vehicle will be in the way of surrounding traffic participants, and to notify the operator after the vehicle 30 is moved out of the way (S113).

[0269] Furthermore, the remote control system 1 according to the above-described embodiment may be configured to first acquire the status of the operator (S114) and determine whether the operator is able to respond immediately to the remote control request (S115) when remote control becomes necessary, regardless of whether the vehicle itself will interfere with surrounding traffic participants. In this configuration, if the operator cannot respond immediately, the vehicle 30 may be evacuated and then notified, and if the operator can respond immediately, the operator may be notified immediately.

[0270] Note that, similar to the above-described embodiment, the notification to the operator after the saving notifies the operator that remote control is required, and may include the saved content.

[0271] The above-described embodiment may be arbitrarily combined with at least one of the above-described modified examples and application examples. Also, two or more of the above-described modified examples and application examples may be arbitrarily combined.

[0272] Note that some or all of the functions of each device in the remote control system 1 according to the above-described embodiment may be realized by other devices in the remote control system 1. For example, some of the functions of the vehicle 30 according to the above-described embodiment may be realized by at least one of the server device 10 and the operator terminal 20. Alternatively, in the remote control system 1 according to the above-described embodiment, the server device 10 and the operator terminal 20 may be configured as an integrated unit. Alternatively, in the remote control system 1 according to the above-described embodiment, any one of the multiple vehicles 30 and the server device 10 may be configured as an integrated unit.

[0273] In the remote operation system 1 according to the above-described embodiment, the plurality of vehicles 30 monitored by one operator via one operator terminal 20 includes one or more vehicles 30 that can be controlled differently depending on whether they are interfering with surrounding traffic participants, as described above. In other words, the plurality of vehicles (mobile bodies) to be monitored may be configured by mixing the vehicles 30 according to the above-described embodiment and vehicles that are not controlled differently depending on whether they are interfering with surrounding traffic participants.

[0274] In the above-described embodiment, the determination of "whether it is A or not" may be realized by determining only that it is A, by determining only that it is not A, or by determining both of these.

[0275] In the above embodiment, "any of A" means "at least one of A."

[0276] The programs executed by each device of the remote control system 1 according to the above-described embodiment may be provided by being recorded in an installable or executable file format on a computer-readable recording medium (Computer Program Product) such as a CD-ROM, FD, CD-R, or DVD.

[0277] The programs executed by the devices in the remote control system 1 according to the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The programs executed by the devices in the remote control system 1 according to the above-described embodiment may be provided or distributed via a network such as the Internet.

[0278] Furthermore, the programs executed by the devices of the remote control system 1 according to the above-described embodiment may be provided by being pre-installed in a ROM or the like.

[0279] Although the embodiments of the present disclosure have been described above, the above-described embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These novel embodiments and modifications thereof are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims.

[0280] 1 Remote operation system 10 Server device (remote operation support device) 101 Remote request information acquisition unit 102 Remote request notification unit 103 Remote control information reception unit 104 Operator information generation unit 105 Operator information transmission unit 20 Operator terminal (terminal device) 201 Display unit 202 Remote control information transmission unit 30 Vehicle (mobile body) 301 Sensor 302 Remote control determination unit 303 Notification location determination unit 304 Operator status acquisition unit 305 Remote request response determination unit 306 Surrounding information acquisition unit 307 Surrounding situation determination unit 308 Control information generation unit 309 Remote request information generation unit 310 Remote request information transmission unit 311 Control information acquisition unit 312 Control unit 313 Drive unit 4 Control device 41 Processor 42 ROM 43 RAM 44 Device I / F unit 601 to 623 Display screens 701a to 701d Images 711 to 723 Notifications 751, 752, 753 Operation buttons 801 Evacuation area 803 Falling objects (surrounding objects) 805, 805a, 805b Parked vehicles (surrounding objects) 807 Approaching vehicles (surrounding objects) 809a, 809b Location 811 Pedestrians (surrounding objects) N Network

Claims

1. A remote operation assistance method executed in at least one of a plurality of moving bodies configured to be able to move autonomously or in accordance with remote operation of an operator monitoring the plurality of moving bodies and perform a predetermined task, the method comprising: determining whether remote control by the operator is required; if the situation requires remote control by the operator, acquiring information indicating the operator's situation and information on surrounding objects at the current position including surrounding traffic participants; determining, based on the information indicating the operator's situation, whether the operator is able to respond to a remote request from the operator's own device requesting the remote control; determining, based on the information on the surrounding objects, whether the operator's own device stopped at the current position will be an obstacle to surrounding traffic participants; if the operator is unable to respond to the remote request from the operator's own device and the operator's own device stopped at the current position will be an obstacle to surrounding traffic participants, autonomously evacuating from the current position.

2. After evacuation, the remote operation assistance method according to claim 1, wherein information about the surrounding objects for the current position, which is the evacuation destination, and information about the surrounding objects for the original position from which the situation requiring remote control arose are acquired, and based on the information about the surrounding objects for each of the current position and the original position, it is determined whether the situation allows autonomous return to the original position, and if the situation allows autonomous return to the original position, autonomously moves to the original position.

3. A remote operation assistance method according to claim 1 or claim 2, wherein information about the surrounding objects is acquired when the operator is unable to respond to the remote request from the device based on information indicating the operator's status.

4. The remote operation assistance method according to claim 1 or claim 2, wherein, if the device stops at its current position based on the information about the surrounding objects and is in the way of surrounding traffic participants, it notifies the operator of the remote request after evacuating, and if the device stops at its current position and is not in the way of surrounding traffic participants, it stops at its current position and notifies the operator of the remote request.

5. The remote operation assistance method according to claim 1 or claim 2, wherein, in a situation where remote control by the operator is required, the vehicle stops at the current location and notifies the operator of the remote request, and after notifying the operator of the remote request, information indicating the operator's situation and information about surrounding objects at the current location, including surrounding traffic participants, is obtained.

6. The remote operation assistance method according to claim 1 or claim 2, wherein, in a situation where the operator is unable to respond to the remote request from the own device and the own device stopped at its current position would be a hindrance to surrounding traffic participants, the remote request is notified to the operator after the operator has evacuated; and in a situation where the operator is able to respond to the remote request from the own device or the own device stopped at its current position would not be a hindrance to surrounding traffic participants, the remote operation assistance method stops at its current position and notifies the operator of the remote request.

7. A remote operation assistance method as described in claim 1 or claim 2, wherein, when the operator is unable to respond to the remote request from the operator's own device based on information indicating the operator's status, an estimated response time for the remote control that the operator is currently responding to and an estimated arrival time for the surrounding traffic participant approaching the operator's own device to arrive near the operator's own device are obtained, and if the estimated arrival time is earlier than the estimated response time, it is determined that the operator's own device, stopped at its current location, is in a situation where it will interfere with the surrounding traffic participant.

8. The remote operation support method according to claim 2, wherein after the operator has evacuated, the expected response time for the remote control operation being handled by the operator is obtained, and when the remaining time until the expected response time is less than a predetermined time, the remote operation support method autonomously moves to the original position.

9. The remote operation support method according to claim 1 or 2, wherein, when there are a plurality of candidate save destinations, a save destination that is easiest for the operator to remotely control is selected from the plurality of candidate save destinations.

10. A remote operation support method according to claim 1 or claim 2, wherein, when there are multiple candidate evacuation destinations, a destination with a better communication environment based on past communication conditions is selected from the multiple candidate evacuation destinations.

11. The remote operation assistance method according to claim 2, wherein, after evacuation, if the situation no longer requires remote control by the operator, the robot does not autonomously return to the original position, but instead returns from the evacuation destination to the autonomous driving state it had before the situation required remote control by the operator.

12. The remote operation assistance method according to claim 2, wherein after the retreat, the robot moves to a position other than the original position based on information about the surrounding objects for each of the current position and the original position.

13. The remote operation support method according to claim 2, further comprising determining whether to move to the original position after the evacuation depending on the type of service provided by the predetermined task.

14. The remote operation support method according to claim 2, wherein, after evacuation, information indicating the status of the remote control being handled by the operator is acquired, and the operator autonomously moves to the original position at a timing based on the information indicating the status of the remote control being handled by the operator.

15. The remote operation assistance method according to claim 1, wherein autonomous movement to retreat from the current position is started at a timing according to an instruction from the operator.

16. The remote operation assistance method according to claim 2, wherein after the robot has retreated, it starts autonomously moving back to the original position at a timing according to an instruction from the operator.

17. A remote operation assistance method as described in claim 15 or claim 16, wherein if no instruction is received from the operator within a predetermined count, the timing to start the autonomous movement is automatically determined.

18. At least one mobile body among a plurality of mobile bodies, equipped with at least one processor and configured to be able to move autonomously or in accordance with remote control by an operator monitoring the plurality of mobile bodies and perform a predetermined task, wherein the at least one processor: determines whether a situation requires remote control by the operator; if a situation requires remote control by the operator, acquires information indicating the operator's situation and information on surrounding objects at the current position including surrounding traffic participants; determines, based on the information indicating the operator's situation, whether the operator is in a situation where he or she can respond to a remote request from the mobile body requesting the remote control; determines, based on the information on the surrounding objects, whether the mobile body, stopped at its current position, will be an obstacle to surrounding traffic participants; and autonomously retreats from its current position if the operator is in a situation where he or she cannot respond to the remote request from the mobile body and if the mobile body, stopped at its current position, will be an obstacle to surrounding traffic participants.

19. A program for causing a computer mounted on at least one of a plurality of mobile bodies, configured to be capable of moving autonomously or in accordance with remote control by an operator monitoring the plurality of mobile bodies and executing predetermined tasks, to: determine whether remote control by the operator is required; if the situation requires remote control by the operator, acquire information indicating the operator's situation and information on surrounding objects at the current position including surrounding traffic participants; determine, based on the information indicating the operator's situation, whether the operator is in a situation where he or she can respond to a remote request from the operator's own device requesting the remote control; determine, based on the information on the surrounding objects, whether the operator's own device stopped at the current position will be a hindrance to surrounding traffic participants; and autonomously evacuate from the current position if the operator is in a situation where he or she cannot respond to the remote request from the operator's own device and the operator's own device stopped at the current position will be a hindrance to surrounding traffic participants.

Images