Information providing device, information providing method, and information providing program

Abstract

This information providing device comprises a notification device. The notification device is configured such that, before determining the remote operator who will receive a request for remote operation from a mobile body, the notification device notifies one or more candidate operators, who are candidates for the remote operator, of presentation information indicating the details of the request.

Description

Information providing device, information providing method, and information providing program

[0001] The present disclosure relates to an information providing device, an information providing method, and an information providing program.

[0002] Japanese Patent Application Laid-Open Publication No. 2019-194809 discloses a vehicle driving system including a remote driving management device. In response to a remote driving request, the remote driving management device refers to the registration data of the remote driver to determine a driver who will remotely drive the vehicle, and requests the determined remote driver to remotely drive the vehicle.

[0003] Japanese Patent Publication No. 2019-194809

[0004] According to the technology described in Japanese Patent Application Laid-Open No. 2019-194809, after a remote driver who will perform remote driving based on a remote driving request from a vehicle is determined by a remote driving management device, the request for remote driving is notified to the remote driver. As a result, the remote driver who receives the request may feel anxious about the possibility of accepting an unintended request.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technology that enables remote operator candidates (candidate operators) to accept remote operation requests with confidence.

[0006] An information providing device according to the present disclosure includes a notification device configured to notify one or more candidate operators who are candidates for a remote operation of presentation information indicating the content of the request before a remote operator who will accept the request from a mobile object is determined.

[0007] An information providing method according to the present disclosure is a method for a computer to provide information, and includes: acquiring presentation information indicating the content of a remote operation request from a mobile object; and notifying one or more candidate operators who are remote operator candidates of the acquired presentation information via a notification device before a remote operator who will accept the request is determined.

[0008] An information providing program according to the present disclosure is executed by a computer, and causes the computer to acquire presentation information indicating the content of a remote operation request from a mobile object, and notify one or more candidate operators who are remote operator candidates of the acquired presentation information via a notification device before a remote operator who will accept the request is determined.

[0009] According to the present disclosure, before a remote operator is selected, presentation information indicating the content of the remote operation request is notified to the candidate operator, thereby allowing the candidate operator to safely decide whether or not to accept the remote operation request.

[0010] 1 is a schematic diagram showing an example of the configuration of a remote operation system to which an "information providing device" according to an embodiment is applied. FIG. 2 is a block diagram showing an example of the configuration of the remote operation device shown in FIG. 1. FIG. 3 is a block diagram showing an example of the configuration of the management device shown in FIG. 1. FIG. 4 is a diagram showing a first example of display of presentation information PST. FIG. 5 is a diagram showing a second example of display of presentation information PST. FIG. 6 is a flowchart showing an example of processing executed by the management device to provide a remote operation service.

[0011] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0012] 1. Remote Control System Fig. 1 is a schematic diagram showing an example of the configuration of a remote control system 1 to which an "information providing device" according to an embodiment is applied. The remote control system 1 is a system for remotely controlling a mobile object 10. Remote control is a concept that includes remote driving. The remote control system 1 includes the mobile object 10, a remote control device 20, a management device 30, a service provider system 40, a display device 50, and a mobile terminal 60.

[0013] The mobile object 10 is a target of remote control. The remote control device 20 is a device used by the remote operator 2 when remotely controlling the mobile object 10. In other words, the mobile object 10 is configured to be remotely controlled by the remote control device 20. The remote control device 20 and the management device 30 are installed in a remote operation management center 3 that manages the remote operation of the mobile object 10. More specifically, a plurality of remote control devices 20 are arranged in the remote operation management center 3. The mobile object 10, the remote control devices 20, and the management device 30 can communicate with each other via a communication network. Furthermore, each of the mobile object 10 and the management device 30 can communicate with a service provider system (or simply, a provider system) 40 via the communication network.

[0014] The management device 30 receives a request for remote control from the mobile object 10 via the provider system 40. In response to this request, the management device 30 assigns (determines) a remote operator 2 from among multiple candidate operators 4 to the mobile object 10 that is the target of remote control. The management device 30 manages the assignment relationship between the mobile object 10 and the remote operator 2. The management device 30 also provides information on the assignment relationship to the mobile object 10 and the remote operation device 20. Based on the information on the assignment relationship, the mobile object 10 and the remote operation device 20 establish communication. After communication is established, the mobile object 10 and the remote operation device 20 may communicate directly without going through the management device 30.

[0015] The remote operation management center 3 includes a waiting area 5 as a facility associated with the remote operation device 20. The waiting area 5 is used by one or more candidate operators 4 who are waiting to receive a request for remote operation.

[0016] Each of the candidate operators 4 is qualified to remotely operate one or more types of mobile bodies 10. The candidate operator 4 determined as the remote operator 2 moves to the location of the designated remote operation device 20 and remotely operates the designated mobile body 10 in accordance with the content of the designated remote operation service.

[0017] 1-1. Types of Mobile Objects and Types of Remote Control Services In the example of the remote control system 1 shown in FIG. 1, there are multiple types of mobile objects 10 that can be remotely controlled. Specifically, for example, the various types of mobile objects listed below can be remotely controlled. Examples of the various types of mobile objects include passenger cars, trucks, small or large MaaS (Mobility as a Service) vehicles, buses, forklifts, dollies, small boats, large cruisers, golf course carts, small personal mobility vehicles, electric wheelchairs, amusement park rides (including attractions), drones, power shovels, and bulldozers.

[0018] Furthermore, in the example of the remote operation system 1, multiple types of remote operation services are provided by the remote operation management center 3. Specifically, the various types of services listed below may correspond to the remote operation services. The various types of services include, for example, autonomous driving backup, driving / parking for private cars, driving for taxis / trucks, delivery and forwarding of shared cars, home delivery, transport of luggage, loading / unloading of factory forklifts, docking and undocking of ships, movement of personal mobility vehicles, driving for amusement park attractions, and landing of drones. In addition, autonomous driving backup refers to, for example, remotely controlling an autonomous vehicle to evacuate when it encounters a situation where autonomous driving is difficult (for example, stopping due to an obstacle or becoming stuck).

[0019] 2 is a block diagram showing an example of the configuration of the remote control device 20 shown in FIG. 1. For example, the remote control device 20 may be configured as a remote cockpit that can accommodate various types of moving bodies 10. Alternatively, for example, the remote control device 20 may be configured as a remote cockpit that has remote control members dedicated to a specific type of moving body 10.

[0020] The remote control device 20 includes a communication device 21 , an output device 22 , an input device 23 , and a control device 24 .

[0021] The communication device 21 communicates with the mobile object 10 and the management device 30. The output device 22 outputs various types of information. For example, the output device 22 includes a display 25. The display 25 displays various types of information to present the various types of information to the remote operator 2. The output device 22 also includes a speaker 26. For example, the speaker 26 outputs sounds detected by a microphone of the mobile object 10, or outputs various types of information transmitted from the management device 30 by voice.

[0022] The input device 23 accepts input from the remote operator 2. For example, the input device 23 includes remote control members that the remote operator 2 operates when remotely operating the mobile body 10. Examples of remote control members include a steering wheel, an accelerator pedal, a brake pedal, a direction indicator, a joystick, a cross key, and a switch. The remote control members may be a touch panel. The input device 23 may also include a keyboard, a mouse, a touch panel, and the like in addition to the remote control members.

[0023] The control device 24 controls the remote control device 20. The control device 24 includes one or more processors 27 (hereinafter simply referred to as processors 27) and one or more storage devices 28 (hereinafter simply referred to as storage devices 28). The processor 27 executes various processes. The storage devices 28 store various information. Examples of the processor 27 include one or a combination of a general-purpose processor, a specific-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and an integrated circuit. The processor 27 can also be referred to as circuitry or processing circuitry. Circuitry is hardware programmed to realize the described functions or hardware that executes the functions. The same applies to the processors 33, 42, 52, and 63 described below. Examples of the storage device 28 include one or a combination of a volatile memory, a non-volatile memory, a hard disk drive (HDD), and a solid-state drive (SSD). The same applies to storage devices 34, 43, 53, and 64, which will be described later.

[0024] The remote operation program PROG1 is a computer program executed by the processor 27. The functions of the control device 24 may be realized by cooperation between the processor 27, which executes the remote operation program PROG1, and the storage device 28. The remote operation program PROG1 is stored in the storage device 28. Alternatively, the remote operation program PROG1 may be recorded on a computer-readable recording medium.

[0025] The control device 24 communicates with the mobile body 10 via the communication device 21. The control device 24 receives mobile body information MOV transmitted from the mobile body 10. The mobile body 10 is equipped with various sensors including a camera. The camera photographs the surroundings of the mobile body 10 and acquires video (images) showing the situation around the mobile body 10. The mobile body information MOV is information obtained by the various sensors and includes video obtained by the camera. The control device 24 presents the mobile body information MOV, including the video, to the remote operator 2 by displaying the mobile body information MOV, including the video, on the display 25. The remote operator 2 can recognize the state of the mobile body 10 and the surrounding situation based on the mobile body information MOV displayed on the display 25.

[0026] The remote operator 2 operates a remote control member of the input device 23. The amount of operation of the remote control member is detected by a sensor installed on the remote control member. The control device 24 generates remote operation information OPE that reflects the amount of operation of the remote control member by the remote operator 2. The control device 24 then transmits the remote operation information OPE to the mobile object 10 via the communication device 21.

[0027] The storage device 28 also stores an information provision program PROG3 (see Sections 1-5 below) as a computer program executed by the processor 27. Alternatively, the information provision program PROG3 may be recorded on a computer-readable recording medium for use by the remote control device 20.

[0028] When the processor 27 executes the information provision program PROG3, the remote control device 20 equipped with the output device 22 (the display 25 and the speaker 26), which is an example of the "notification device" according to the present disclosure, functions as the "information provision device" according to the present disclosure. In this case, it can be said that the remote control device 20 is integrated with the information provision device.

[0029] 1-3. Example of Management Device Fig. 3 is a block diagram showing an example of the configuration of the management device 30 shown in Fig. 1. The management device 30 manages the remote operation system 1. For example, the management device 30 is a management server on a cloud. The management server may be configured with multiple servers that perform distributed processing.

[0030] The management device 30 includes a communication device 31 and a control device 32. The communication device 31 communicates with the mobile object 10, the remote control device 20, and the business operator system 40.

[0031] The control device 32 controls the management device 30. The control device 32 includes one or more processors 33 (hereinafter simply referred to as processors 33) and one or more storage devices 34 (hereinafter simply referred to as storage devices 34). The processors 33 execute various processes. The storage devices 34 store various types of information. The various types of information include management information MNG, the details of which will be described later as appropriate.

[0032] The management program PROG2 is a computer program executed by the processor 33. The functions of the control device 32 may be realized by cooperation between the processor 33 executing the management program PROG2 and the storage device 34. The management program PROG2 is stored in the storage device 34. Alternatively, the management program PROG2 may be recorded on a computer-readable recording medium.

[0033] The control device 32 communicates with the mobile object 10, the remote control device 20, and the operator system 40 via the communication device 31. The control device 32 receives mobile object information MOV transmitted from the mobile object 10. The control device 32 then transmits the received mobile object information MOV to the remote control device 20. The control device 32 also receives remote operation information OPE transmitted from the remote control device 20. The control device 32 then transmits the received remote operation information OPE to the mobile object 10.

[0034] The control device 32 also communicates with the provider system 40 via the communication device 31. The control device 32 receives request information REQ, which will be described later, transmitted from the provider system 40. The control device 32 determines the remote operation device 20 to be used for remote operation related to the received request information REQ, and determines the remote operator 2 to operate the determined remote operation device 20. The process executed by the control device 32 to determine the remote operator 2 includes a process of selecting one or more candidate operators 4 who are candidates for the remote operator 2. The control device 32 also generates presentation information PST, which will be described later, based on the received request information REQ. The generated presentation information PST will be described in detail in Section 3.

[0035] Furthermore, the control device 32 generates progress information PRG indicating the progress of the remote operation after receiving the request for the remote operation. Then, the control device 32 transmits the generated progress information PRG to the business operator system 40 that transmitted the request information REQ. Details of the progress information PRG will be described later in Section 2-2.

[0036] 1-4. Service Provider System FIG. 1 illustrates a plurality of service provider systems 40. Each provider system 40 is a server of a provider that provides one or more types of remote control services for one or more types of mobile objects 10. Each provider system 40 includes a communication device 41, one or more processors 42 (hereinafter simply referred to as processors 42), and one or more storage devices 43 (hereinafter simply referred to as storage devices 43). The storage device 43 stores information MI of each mobile object 10 that receives remote control services from the service provider. The information MI of each mobile object 10 includes, for example, the ID (Identification) number, type (such as type or model), and color information of the mobile object 10.

[0037] Each provider system 40 receives a request for remote operation from the mobile object 10. The request from the mobile object 10 includes, for example, information MI about the mobile object 10, location information about the mobile object 10, and information indicating the type of remote operation service requested by the mobile object 10. In response to the received request, the provider system 40 generates request information REQ. The request information REQ is information indicating the specific content of the request when requesting the remote operation management center 3 to perform remote operation based on the request from the mobile object 10. The provider system 40 transmits the generated request information REQ to the management device 30. Specific examples of the request information REQ will be described later in Section 2-1. The provider system 40 also receives progress information PRG from the management device 30.

[0038] 1-5. Display Device and Mobile Terminal One or more display devices 50 are installed in the waiting area 5. Furthermore, at least one of the one or more candidate operators 4 waiting in the waiting area 5 carries a mobile terminal 60. Each of the display device 50 and the mobile terminal 60 is capable of communicating with the management device 30 via a communication network. Each of the display device 50 and the mobile terminal 60 is configured to receive presentation information PST (described below) from the management device 30 and notify the one or more candidate operators 4 of the received presentation information PST. In this way, each of the display device 50 and the mobile terminal 60 corresponds to an example of an "information providing device" according to the present disclosure.

[0039] Specifically, the display device 50 includes a display 51, one or more processors 52 (hereinafter simply referred to as processors 52), and one or more storage devices 53 (hereinafter simply referred to as storage devices 53). The processor 52 executes various processes related to the control of the display 51, etc. The storage device 53 stores various information.

[0040] The information provision program PROG3 is a computer program executed by the processor 52. The functions of the display device 50 may be realized by cooperation between the processor 52 executing the information provision program PROG3 and the storage device 53. The information provision program PROG3 is stored in the storage device 53. Alternatively, the information provision program PROG3 may be recorded on a computer-readable recording medium.

[0041] The mobile terminal 60 is, for example, a smartphone or a tablet terminal. The mobile terminal 60 includes a display 61, a speaker 62, one or more processors 63 (hereinafter simply referred to as processors 63), and one or more storage devices 64 (hereinafter simply referred to as storage devices 64). The display 61 is, for example, a touch panel. The processor 63 executes various processes related to the control of the display 61 and speaker 62, etc. The storage device 64 stores various information.

[0042] Similar to the display device 50, the functions of the portable terminal 60 may be realized by cooperation between the processor 63 that executes the information provision program PROG3 and the storage device 64. The information provision program PROG3 is stored in the storage device 64. Alternatively, the information provision program PROG3 may be recorded on a computer-readable recording medium.

[0043] Additionally, the "notification device" according to the present disclosure is a device that notifies information by at least one of display and sound, for example, at least one of a display and a speaker. The displays 51 and 61 and the speaker 62 correspond to examples of the "notification device." That is, in the example of the display device 50, the notification of the presentation information PST is performed by displaying the presentation information PST on the display 51. Furthermore, in the example of the mobile terminal 60, the notification of the presentation information PST is performed by displaying the presentation information PST using the display 61. Alternatively, the notification of the presentation information PST may be performed by sound notification of the presentation information PST using the speaker 62, together with or instead of the display.

[0044] 2. Communication between the Management Device (Remote Operation Management Center) and the Service Provider System As described above, each service provider system 40 transmits request information REQ to the management device 30, and the management device 30 transmits progress information PRG to the provider system 40 that transmitted the request information REQ. The formats of the request information REQ and progress information PRG transmitted in this manner are standardized between the management device 30 and each provider system 40.

[0045] 2-1. Request Information REQ from Service Provider System Below are various specific examples of request information REQ indicating the content of a remote control request.

[0046] (Level of Urgency) The request information REQ may include information on the "level of urgency of remote operation" at the time of the request. The level of urgency indicates the degree to which remote operation needs to be initiated quickly. The level of urgency is specified by a level of urgency (e.g., three or five levels) previously specified by the remote operation management center 3. Examples of situations in which remote operation needs to be initiated quickly include situations in which the mobile object 10 needs to be moved quickly or situations in which the mobile object 10 needs to be delivered to a target location quickly. The following is an example of setting the level of urgency. That is, in an example in which an autonomous driving backup is requested as a remote operation service, the mobile object 10 needs to be moved quickly because the autonomous vehicle has stopped on the road, etc. For this reason, the level of urgency is set to "high." Furthermore, in an example in which delivery of a shared car is requested as a remote operation service, the shared car needs to be delivered to the customer by a specified time. For this reason, the level of urgency is set to "medium." In the case of a request to return a rental car as a remote control service, what is required is to return the rental car to the business office (waiting location) after use, and there is no need to rush. Therefore, the urgency is set to "low."

[0047] (Status of Mobile Object) The request information REQ may also include information indicating the "status of the mobile object 10" at the time of the request. This information is information necessary for the remote operator 2 when starting the requested remote operation. For example, the information indicating the status of the mobile object 10 is text information such as "parked in a garage" or "autonomous vehicle stopped in the middle of the road." Alternatively, the information indicating the status of the mobile object 10 may include, in addition to the text information, one or both of image information of the mobile object 10 at the time of the request and information about the surroundings of the mobile object 10. More specifically, the image information is, for example, an image (including video) from a camera mounted on the mobile object 10.

[0048] (Type of mobile object and type of remote control service) The request information REQ may also include information indicating the type of mobile object 10 that is the target of the requested remote control. An example of the type of mobile object 10 has already been described. The request information REQ may also include information indicating the type of the requested remote control service. An example of the type of remote control service has already been described.

[0049] (Completion Condition) The request information REQ may also include information indicating a completion condition for the requested remote control. In other words, the completion condition is a condition for ending the remote control. For example, the completion condition is "to move the moving object 10 to a goal point" or "to pick up a person or object during movement by remote control."

[0050] (Point Information) The request information REQ may also include point information, that is, information indicating the positions of various points such as the start point, goal point, and passing points in the movement of the mobile object 10 using the remote control service.

[0051] (Accompanying Information) The request information REQ may also include accompanying information. In a remote operation service that requires picking up a person or an object, the accompanying information is, for example, information about the person or object to be picked up (for example, a name and an image).

[0052] 2-2. Progress Information PRG for Service Provider System The progress information PRG is information indicating the progress of remote operation after the request is initiated. The progress information PRG may include current status information indicating the current status of the mobile object 10 after the request is initiated. For example, the current status information may include "remote operation has not yet started," "remote operator 2 is being selected," and "remote operation is in progress." This current status information may be acquired, for example, based on history information of processing executed by the processor 33. Furthermore, the current status information may include information about the remote operator 2 performing the remote operation (e.g., ID number, proficiency level, and evaluation score) in addition to "remote operation is in progress." This information about the remote operator 2 is acquired in advance and included in the management information MNG.

[0053] The progress information PRG may also include current location information of the mobile object 10 after the request has started. The current location information can be acquired from the mobile object information MOV.

[0054] Furthermore, the progress information PRG may include the following additional information. For example, in a remote control service related to the transportation of a person or an object, the additional information may include information indicating whether a person or an object has been picked up, information about the person or object that has been picked up (such as an image of the person or object), and information indicating whether the transportation has been completed. This additional information can be obtained from the moving object information MOV.

[0055] By receiving the progress information PRG from the management device 30 (remote operation management center 3), the service provider can properly grasp the progress of the remote operation.

[0056] 3. Providing Information to Candidate Operators (Presented Information PST) As described above, the management device 30 (control device 32) generates presented information PST based on the request information REQ. Then, in order to present the generated presented information PST to one or more candidate operators 4 for remote operation related to the request information REQ, the management device 30 transmits the presented information PST to an information providing device (such as the display device 50). Having acquired the presented information PST, the information providing device notifies one or more candidate operators 4 of the presented information PST by displaying the presented information PST on a notification device (such as the display 51) (see, for example, FIG. 4 or FIG. 5 described below).

[0057] In this embodiment, the notification of the presentation information PST described above is executed to one or more candidate operators 4 before the remote operator 2 who will accept the request for remote operation related to the request information REQ is determined.

[0058] 3-1. Examples of Information Providing Devices to which the Presentation Information PST is to be Transmitted In the example of the remote operation system 1 shown in FIG. 1, the "information providing device" to which the Presentation Information PST is to be transmitted is at least one of the display device 50 in the waiting area 5, the mobile terminal 60, and the remote operation device 20.

[0059] 1 , in an example where one or more candidate operators 4 are waiting at a waiting location 5, the management device 30 may transmit the presentation information PST to each of the display device 50 and the mobile terminal 60. Alternatively, the management device 30 may transmit the presentation information PST to only one of the display device 50 and the mobile terminal 60. Furthermore, the management device 30 may transmit the presentation information PST not only to one or both of the display device 50 and the mobile terminal 60, but also to the remote operation device 20 that is to be operated by the remote operator 2 to perform the remote operation related to the request information REQ. In other words, the target of notification of the presentation information PST by the "information providing device" may include the remote operator 2 who starts remote operation after a certain candidate operator 4 is determined as the remote operator 2.

[0060] On the other hand, in an example of a remote operation system having a remote operation device 20 dedicated to a certain candidate operator 4, the transmission of the presentation information PST before the remote operator 2 is determined may be executed as follows. That is, when the candidate operator 4 is waiting at a location (e.g., a remote cockpit) where the dedicated remote operation device 20 is located, the management device 30 may transmit the presentation information PST to the remote operation device 20 functioning as the "information providing device." In addition, the "location where the remote operation device 20 is located" here may include not only the remote operation management center 3 but also a location outside the remote operation management center 3, for example.

[0061] 3-2. Specific Examples of Presentation Information PST The presentation information PST may be the same as the request information REQ. Alternatively, to make the presentation information PST easier for the remote operator 2 to understand and perform remote operation, the presentation information PST may include, as described below, information processed or added based on the management information MNG of the management device 30 along with the request information REQ. The management information MNG used here includes various information previously acquired based on the mobile object information MOV, etc. The various information referred to here includes, for example, information about the surrounding area of ​​the area where remote operation is to be performed, information about the results of past remote operation, and characteristic information about the mobile object 10. The surrounding information referred to here includes, for example, information indicating points that the remote operator 2 should pay attention to when remotely operating the mobile object 10 (e.g., locations of road depressions and locations under road construction). The information about the results of past remote operation includes, for example, information collected by the management device 30 from the mobile object information MOV while the mobile object 10 was moving (e.g., while the vehicle was traveling) during previous remote operation. The characteristic information of the mobile object 10 is, for example, information indicating the operating characteristics of each type of mobile object 10.

[0062] Here, an example of the presentation information PST that shares some information with the request information REQ will be described. That is, for example, the presentation information PST is the same as the request information REQ in terms of the urgency level, the type of the mobile object 10, the type of remote control service, the state of the mobile object 10, the completion condition, and the associated information.

[0063] Furthermore, the presentation information PST may include, for example, information on required qualifications as additional information PST1 that is not included in the request information REQ. The mobile object 10 to be remotely controlled includes mobile objects (e.g., automobiles and ships) that require a special driving qualification. The information on required qualifications is information on qualifications such as a driving qualification required of the remote operator 2 for remote control (e.g., see FIG. 5).

[0064] Further, the presentation information PST may include, as another example of the additional information PST1, information on the difficulty level D of the remote operation. The difficulty level D is acquired from, for example, the management information MNG (see step S106 described later).

[0065] Furthermore, the presentation information PST may include, for example, information on a travel route as processed information PST2 obtained by processing the request information REQ. The information on the travel route may be, for example, information on the travel route itself (e.g., the vehicle's driving route), information on passing points along the way, and information on points requiring attention.

[0066] The travel route information is generated based on the location information included in the request information REQ as well as the various information described above (such as information about the surrounding area of ​​the area where the remote control is performed, performance information of past remote controls, and characteristic information of the mobile object 10). Specifically, as the travel route information, the management device 30 calculates an optimal travel route for the mobile object 10 to be remotely controlled based on the location information and the various information. A specific example of the travel route information will be described later with reference to FIG. 4.

[0067] Fig. 4 is a diagram showing a first display example of the presentation information PST. The presentation information PST shown in Fig. 4 is displayed on a "notification device" such as the display 51. In the first display example, each piece of information constituting the presentation information PST corresponding to one remote operation request is displayed in an easy-to-understand manner by being divided into individual display areas.

[0068] Specifically, text information indicating a high level of urgency is displayed in large letters. Also, text information indicating the type of remote control service (autonomous driving backup) is displayed. Furthermore, a combination of text information (MaaS vehicle), which is an example of information indicating the type of mobile object (mobility) 10, and an icon is displayed, along with the name, color, and identification number of the MaaS vehicle as accompanying information.

[0069] In addition, the first display example displays a combination of text information (stopped due to obstacle, waiting for route restoration), which is an example of information indicating the state (current state) of the MaaS vehicle at the time of the request, and image information IMG1. The image information IMG1 is an image taken by an in-vehicle camera. In the image information IMG1, obstacles to be avoided that are included in the in-vehicle camera image are marked using augmented reality (AR) technology.

[0070] Furthermore, in the first display example, a combination of text information (stopping at a specified angle at a specified goal position), which is an example of information indicating the completion condition, and image information IMG2 is displayed. In this image information IMG2, goal information (for example, a goal frame indicating the goal position) is superimposed using AR technology on the same in-vehicle camera image as that of image information IMG1. In this example, detailed goal information, including not only the designation of the goal position but also the designation of the angle of the MaaS vehicle when stopping at the goal position, is displayed as the completion condition. The goal information is then displayed in an easy-to-understand manner visually using AR superimposition.

[0071] In addition, in the first display example, two pieces of image information IMG3 and IMG4 are displayed, each of which is an example of information indicating a driving route (travel route) that avoids obstacles. The image information IMG3 is a top view, i.e., two-dimensional map information, that indicates the driving route. The image information IMG4 is three-dimensional map information in which the driving route is superimposed on the same in-vehicle camera image as the image information IMG1 using AR technology. These pieces of image information IMG3 and IMG4 visually display the obstacle avoidance route in an easy-to-understand manner. Note that the driving route information may be notified along with audio information output from the speaker 62 or the like.

[0072] Furthermore, in the first display example, numerical information (e.g., 1.0) is displayed as an example of information indicating the remote control difficulty level D. Alternatively, the difficulty level D may be displayed as text information such as "easy," "normal," or "difficult."

[0073] Furthermore, in the first display example, accompanying information is displayed using a combination of text information and image information IMG5 and IMG6. The text information is, for example, the name of the person or object to be picked up. The image information IMG5 and IMG6 is, for example, an image related to the person or object to be picked up. Note that, for example, image information indicating points to pay attention to when avoiding obstacles is basically information about the driving route, but may also be included in this accompanying information.

[0074] FIG. 5 illustrates a second display example of the presentation information PST. The presentation information PST illustrated in FIG. 5 is displayed on a "notification device" such as the display 51. In the second display example, five pieces of presentation information PST corresponding to five remote operation requests for which the remote operator 2 has not yet been determined are organized into a list and displayed in an easy-to-understand manner. Each piece of presentation information PST includes information indicating the service (type of remote operation service), mobility (type of mobile object), required qualifications, current status, and completion conditions, as well as associated information. The content of each piece of information is as shown in FIG. 5. In addition, when displaying presentation information PST based on a newly received request, the presentation information PST to be displayed may be displayed in a manner that makes it easy to distinguish from presentation information PST corresponding to other requests. For example, a frame surrounding the newly displayed presentation information PST may be displayed in a superimposed manner, or the background color of the display area of ​​the presentation information PST may be a different color from the background colors of the other display areas. Furthermore, in this list, the presentation information PST may be displayed in descending order of urgency of the remote operation, starting from the top.

[0075] 3-3. Example of Customization of Presentation Information PST Furthermore, the presentation information PST may be changed (customized) by each candidate operator 4 as follows.

[0076] For example, the presentation information PST may be generated differently depending on the proficiency of the candidate operator 4 in remote operation. Specifically, for example, for a candidate operator 4 with low experience, the management device 30 may generate the presentation information PST so that it includes a lot of accompanying information (e.g., points to note and advice). On the other hand, for a candidate operator 4 with advanced experience, the management device 30 may generate the presentation information PST so that it does not include accompanying information that is obvious to an advanced operator. Note that information about the candidate operator 4's proficiency in remote operation can be acquired, for example, based on the above-mentioned past remote operation performance information included in the management information MNG.

[0077] Also, for example, when requesting a candidate operator 4 who is specialized in a specific remote control service to perform remote control related to that specific remote control service, the management device 30 may generate presentation information PST so that it has content specialized for that specific remote control service.

[0078] 6 is a flowchart showing an example of a process executed by the management device 30 to provide a remote control service. The process of this flowchart is executed while the remote control system 1 is in operation.

[0079] Steps S100 and S102: In step S100, the management device 30 (processor 33) waits for a request for a remote control service from the service provider system 40. While there is no request for a remote control service (step S102; No), the management device 30 maintains a standby state. On the other hand, if a request for a remote control service is received during the standby state (step S102; Yes), the management device 30 continues to wait for requests for other remote control services and executes the processes from step S104 onwards in parallel.

[0080] Step S104: In step S104, the management device 30 receives the request information REQ from the business operator system 40. As already explained, the request information REQ includes information such as the type of remote control service and the urgency level.

[0081] Step S106: Next, in step S106, the management device 30 reads and updates the management information MNG stored in the storage device 34. Specifically, for example, the management information MNG includes various information included in the request information REQ (such as the type of remote control service and the urgency), the difficulty level D described above, the response priority of the request, information on the candidate operators 4 (such as past operation records, qualifications held, and availability information), and information on the remote control device 20 (such as information indicating a suitable mobile object 10 and availability information). The response priority is determined based on, for example, the type of remote control service, the urgency, and the difficulty level D.

[0082] In step S106, the management device 30 calculates (sets) the difficulty level D of the remote operation related to the current request information REQ received in step S104 using the following method. Then, the management device 30 updates the response priority of each remote operation request currently included in the management information MNG, taking into account the calculated difficulty level D.

[0083] (Method of Calculating Difficulty Level D) The difficulty level D described above can be calculated as follows, for example, based on various information included in the request information REQ transmitted from the service provider system 40. In this example, the difficulty level D is calculated by multiplying a predetermined base value (e.g., 1.0) by a coefficient Ki (i = 1 to n) corresponding to each type of information (e.g., urgency, type of remote control service, type of mobile object 10, completion conditions, travel route, and whether or not a specified time is set). Each coefficient Ki indicates the difficulty level Di for each piece of information, such as the urgency. For example, each coefficient Ki is set to a reference value (e.g., 1.0) when the difficulty level Di is standard. Furthermore, each coefficient Ki is set to a value smaller than the reference value (e.g., 0.8) when the difficulty level Di is lower than the standard level. On the other hand, each coefficient Ki is set to a value larger than the reference value (e.g., 1.5) when the difficulty level Di is higher than the standard level. The relationship between each coefficient Ki and the difficulty level Di for each piece of information, such as the urgency, is determined in advance and stored in the storage device 34. The management device 30 obtains, from the above relationship, each coefficient Ki corresponding to various information included in the current request information REQ. For example, if the urgency level included in the current request information REQ is "high," the management device 30 obtains, from the above relationship, the urgency coefficient K1 corresponding to the "high" urgency level.

[0084] Furthermore, each coefficient Ki of the difficulty level D calculated as described above may be corrected (optimized) based on, for example, the difficulty level evaluation result by the remote operator 2 who actually performed the remote operation. Specifically, each time a remote operation ends, the management device 30 acquires the difficulty level evaluation result (e.g., easy, normal, or difficult) of the remote operation from the remote operator 2 via the remote operation device 20. The management device 30 associates the acquired evaluation result with the difficulty level (set difficulty level) D set for the remote operation in step S106 and stores it in the storage device 34. This accumulates the relationship between the evaluation result and the set difficulty level D. Then, the management device 30 determines a deviation of the evaluation result from the set difficulty level D based on the accumulated information, and if the deviation is found, executes a process of statistically optimizing each coefficient Ki so as to reduce the deviation.

[0085] Furthermore, each coefficient Ki of the difficulty level D may be similarly corrected (optimized) based on, for example, at least one of the "time required for remote operation" and the "number of times a remote operation problem occurred" instead of or in addition to the evaluation result of the remote operator 2. Specifically, in an example using the time required for remote operation, the management device 30 associates the actual time required for remote operation with an estimated value for the time required for the remote operation and stores the associated time in the storage device 34 each time a remote operation is completed. This accumulates the relationship between the actual time required and the estimated value. The management device 30 then determines whether the actual time required is too long compared to the estimated value based on the accumulated information, and if the actual time required is too long, performs a process of statistically optimizing each coefficient Ki so as to shorten the time difference. Similarly, in an example using the number of times a remote operation problem occurred, the management device 30 stores the number of times a remote operation problem occurred in the storage device 34 each time a remote operation is completed. This accumulates the number of times a problem occurred. The management device 30 then determines whether the number of times the trouble has occurred is too high based on the accumulated information, and if the number of times the trouble has occurred is too high, executes a process of statistically optimizing each coefficient Ki so as to reduce the number of times the trouble has occurred.

[0086] Step S108: Next, in step S108, the management device 30 selects a remote operation device 20 to be used for the remote operation related to the current request information REQ. Specifically, the management device 30 selects the remote operation device 20 based on, for example, information included in the management information MNG (for example, the type of remote operation service and the response priority).

[0087] Steps S110 and S112: Next, in step S110, the management device 30 executes a process of checking whether the selected remote operation device 20 has a vacant space. As a result, if there is no vacant space (step S110; No), the process proceeds to step S112. In step S112, the management device 30 updates the management information MNG so that the selected remote operation device 20 is in a waiting state for a vacant space. Thereafter, the processes of steps S108 and S110 are executed repeatedly.

[0088] Step S114: On the other hand, if the selected remote operation device 20 has a vacant space (Step S110; Yes), the process proceeds to Step S114. In Step S114, the management device 30 selects a candidate operator 4 for the remote operation related to the current request information REQ. Specifically, the management device 30 selects the candidate operator 4 based on, for example, information included in the management information MNG (for example, the type of remote operation service and the response priority).

[0089] Steps S116 and S118: Next, in step S116, the management device 30 executes a process to check whether or not the selected candidate operator 4 has a vacancy. As a result, if there is no vacancy (step S116; No), the process proceeds to step S118. In step S118, the management device 30 updates the management information MNG so that the selected candidate operator 4 is in a waiting state for a vacancy. Thereafter, the processes of steps S114 and S116 are executed repeatedly.

[0090] Step S120: On the other hand, if the selected candidate operator 4 has availability (Step S116; Yes), the process proceeds to Step S120. In Step S120, the management device 30 generates the presentation information PST using the method already described.

[0091] Step S122: Next, in step S122, the management device 30 notifies the selected candidate operator 4 of the generated presentation information PST. In an example where the candidate operator 4 is waiting in the waiting area 5, the presentation information PST is notified via the display device 50. In this example, the presentation information PST may include information that identifies the candidate operator 4 to be notified. Furthermore, in a case where the candidate operator 4 is carrying a mobile terminal 60, the presentation information PST may be notified via the display device 50.

[0092] (Example of Processing on the Information Providing Device Side) More specifically, in step S122, the management device 30 transmits the presentation information PST to an "information providing device" such as the display device 50 in order to notify the candidate operator 4 of the presentation information PST. The information providing device acquires (receives) the presentation information PST. Then, the information providing device notifies the candidate operator 4 of the acquired presentation information PST by displaying it on a display or the like. As described above, according to the flowchart shown in FIG. 6 , the information providing device acquires the presentation information PST before the remote operator 2 who will perform the current remote operation is determined in step S126, which will be described later, and notifies the candidate operator 4 of the presentation information PST before the determination.

[0093] (Example of different notifications depending on the level of urgency) Additionally, in step S122, the management device 30 may transmit the presentation information PST accompanied by an instruction to display the presentation information PST in different ways depending on the level of urgency included in the presentation information PST. Alternatively, the management device 30 may transmit the presentation information PST accompanied by an instruction to notify the presentation information PST with different sounds depending on the level of urgency. Alternatively, the management device 30 may transmit the presentation information PST accompanied by both of these instructions.

[0094] Upon receiving the above-described instruction, the information providing device notifies the candidate operator 4 of the presentation information PST with at least one of a display and a sound that differs depending on the level of urgency. For example, when the presentation information PST is displayed as shown in FIG. 5 , the information providing device may display the presentation information PST in a manner that makes remote operation requests with a predetermined level of urgency or higher more easily distinguishable than other requests. Furthermore, for example, when notifying a remote operation request with a predetermined level of urgency or higher, the information providing device may notify the candidate operator 4 of the presentation information PST with a sound that attracts the candidate operator 4's attention more than when notifying other requests. This also applies to the transmission of the presentation information PST by the processing of step S126 described below.

[0095] Step S124: Next, in step S124, the management device 30 executes a process of confirming with the candidate operator 4 whether or not they are able to respond to the current remote operation request. The candidate operator 4 operates a predetermined device such as the mobile terminal 60, for example, to respond to the management device 30 as to whether or not they will accept the request.

[0096] According to the processes of steps S122 and S124, the candidate operator 4 has an opportunity to check the notified presentation information PST and decide whether or not to accept the request for remote operation before determining the remote operator 2. In addition, the candidate operator 4 can also refuse to accept the request.

[0097] If the candidate operator 4 is not available (step S124; No), the process returns to step S114. That is, another candidate operator 4 is selected. On the other hand, if the candidate operator 4 is available (step S124; Yes), the process proceeds to step S126.

[0098] Step S126: In step S126, the management device 30 executes a process of determining the currently selected candidate operator 4 as the remote operator 2 who will perform the remote operation related to this request. Then, the management device 30 transmits the presentation information PST to the remote operation device 20 that is scheduled to be used by the determined remote operator 2. The presentation information PST transmitted to the remote operation device 20 is displayed, for example, using the display 25. Alternatively, the presentation information PST may be notified by voice from the speaker 26 instead of or in addition to being displayed on the display 25.

[0099] Step S128: Next, in step S128, the management device 30 executes a process of determining whether the determined remote operator 2 is ready to start the current remote operation. For example, if the determined remote operator 2 operates a switch on the remote operation device 20 to be used this time and responds indicating that preparation is complete, the management device 30 determines that preparation is complete.

[0100] Step S130: When the remote operator 2 has completed preparation (Step S128; Yes), the management device 30 causes the remote operator 2 to start remote operation (Step S130). Specifically, the management device 30 causes the remote operator 2 to start generating remote operation information OPE for the remote operation device 20 and transmitting the generated remote operation information OPE to the mobile object 10.

[0101] Steps S132 and S134: Next, in step S132, the management device 30 determines whether the current remote operation has ended. In other words, the management device 30 determines whether the above-mentioned completion condition has been met. As a result, if the remote operation has not ended (step S132; No), the processing proceeds to step S134. In step S134, the management device 30 updates the management information MNG as necessary. For example, the management device 30 updates information indicating the state (current state) of the moving object 10 during remote operation.

[0102] On the other hand, if the remote operation has ended (step S132; Yes), the process returns to step S100.

[0103] 5. Effects As described above, according to the "information providing device" of this embodiment, before the remote operator 2 who will accept the remote operation request from the mobile object 10 is determined, the presentation information PST indicating the content of the request is notified to one or more candidate operators 4. This allows the candidate operators 4 to accept the remote operation request with peace of mind.

[0104] More specifically, by notifying the presentation information PST before the remote operator 2 is determined, the candidate operator 4 can be given an opportunity to decide whether to accept the request based on the presentation information PST. Furthermore, for example, if the information notified before the remote operator 2 is determined is simply a request notification requesting acceptance of the request, the candidate operator 4 will have little information to decide whether to accept the request. As a result, for example, the candidate operator 4 may be required to decide whether to accept the request while feeling anxious about the possibility of accepting a request that is not originally intended in order to gain an opportunity for remote operation. In contrast, the presentation information PST according to this embodiment is information that specifically indicates the content of the remote operation request. This allows the candidate operator 4 to decide whether to accept the request with confidence.

[0105] Furthermore, in the example of the remote operation system 1 shown in FIG. 1 , remote operation targets multiple types of mobile objects 10. Therefore, if proper consideration is not given to the request for remote operation to the candidate operator 4, the candidate operator 4 may feel very anxious about the possibility of accepting a request to remotely operate an unintended type of mobile object 10. Furthermore, the greater the number of types of mobile objects 10, the greater the candidate operator 4's anxiety. In this regard, the presentation information PST according to the present embodiment includes "information indicating the type of mobile object 10." This allows the candidate operator 4 to more confidently decide whether or not to accept the request, even when performing a remote operation job targeting multiple types of mobile objects 10.

[0106] Furthermore, in the example of the remote operation system 1 shown in FIG. 1 , multiple types of remote operation services are provided. Therefore, if appropriate consideration is not given to the remote operation request to the candidate operator 4, the candidate operator 4 may feel very anxious about the possibility of accepting a request for an unintended type of remote operation service. Furthermore, the more types of remote operation services there are, the greater the candidate operator 4's anxiety may become. In this regard, the presentation information PST according to this embodiment includes "information indicating the type of remote operation service." This allows the candidate operator 4 to more confidently decide whether or not to accept the request, even when performing remote operation work for multiple types of mobile objects 10.

[0107] In addition, in an example where the presentation information PST includes both information indicating the type of mobile body 10 and information indicating the type of remote control service, the ``information providing device'' can present to the candidate operator 4 in a very easy-to-understand manner what kind of remote control is required for what kind of remote control service, depending on the type of mobile body 10 and remote control service requested.

[0108] Furthermore, the presentation information PST according to this embodiment includes "information indicating the state of the moving object 10 at the time of the request," "information on the urgency of the remote operation," "information on the qualifications required of the remote operator 2," "information on the conditions for completing the remote operation," and "information on the difficulty of the remote operation." By including at least one of these various types of information in the presentation information PST, it is possible to present to the candidate operator 4 what kind of remote operation is required to be performed in an easy-to-understand manner.

[0109] In this embodiment, the presentation information PST is notified with at least one of different display and sound depending on the urgency. This allows the candidate operator 4 to decide to prioritize accepting requests with a higher urgency, for example, when exposed to multiple pieces of presentation information PST at the same time. This also increases the candidate operator 4's awareness of accepting requests with a higher urgency.

[0110] In this embodiment, the presented information PST is also notified to the remote operation device 20 that is to be operated by the remote operator 2 to perform remote operation based on the request. The presented information PST is useful not only for the candidate operator 4 to decide whether or not to accept the request, but also for performing remote operation after accepting the request. Therefore, the candidate operator 4 who has been selected as the remote operator 2 can, for example, move from the waiting area 5 and take a seat in the remote cockpit (remote operation device 20), and then quickly start remote operation while checking the presented information PST.

[0111] Furthermore, the content of the presentation information PST required by the candidate operator 4 may differ depending on the proficiency of the candidate operator 4. Therefore, in this embodiment, the presentation information PST includes different content depending on the proficiency of the candidate operator 4 in remote operation. This makes it possible to more accurately notify the presentation information PST suitable for each individual candidate operator 4.

[0112] In the present embodiment, the "information providing device" is, for example, a display device 50 arranged in the waiting area 5. This makes it possible to notify one or more candidate operators 4 waiting in the waiting area 5 of the presentation information PST before being selected as the remote operator 2. Furthermore, the "information providing device" is, for example, a portable terminal 60 carried by the waiting candidate operator 4. This makes it possible to notify the waiting candidate operator 4 of the presentation information PST before being selected as the remote operator 2.

[0113] 6. Various Other Configuration Examples of the Remote Control System The number of types of moving objects 10 to be remotely controlled in the remote control system 1 may be one.

[0114] Furthermore, the type of remote control service provided for one or more types of mobile bodies 10 may be one type.

[0115] Furthermore, the remote control system does not necessarily have to include the service provider system 40. In an example of a remote control system that does not include the service provider system 40, the request information REQ may be transmitted directly from the mobile object 10 to the management device 30.

[0116] 6, the management device 30 selects one candidate operator 4 and notifies the selected candidate operator 4 of the presentation information PST (see steps S114 and S122). If the candidate operator 4 does not accept the request for remote operation, the management device 30 selects the next candidate operator 4 (see steps S124 and S114). Alternatively, the management device 30 may notify multiple candidate operators 4 of one presentation information PST at the same time. If multiple candidate operators 4 wish to accept the request, the management device 30 may execute a process of determining one candidate operator 4 from the multiple applicants as the remote operator 2, for example, based on the management information MNG.

[0117] REFERENCE SIGNS LIST 1 Remote operation system, 2 Remote operator, 3 Remote operation management center, 4 Candidate operator, 5 Waiting location, 10 Mobile object, 20 Remote operation device (information providing device), 25, 51, 61 Display (notification device), 26, 62 Speaker (notification device), 30 Management device, 40 Service provider system, 50 Display device (information providing device), 60 Portable terminal (information providing device), MNG Management information, MOV Mobile object information, OPE Remote operation information, PRG Progress information, PST Presentation information, REQ Request information

Claims

1. An information providing device including a notification device configured to notify one or more candidate operators who are candidates for a remote operation request from a mobile body of presentation information indicating the content of the request before the remote operator who will accept the request is determined.

2. An information providing device according to claim 1, wherein there are a plurality of types of the moving object, and the presented information includes information indicating the type of the moving object.

3. An information providing device according to claim 1 or 2, wherein the remote control is provided as a plurality of types of remote control services, and the presented information includes information indicating the type of the remote control service.

4. An information providing device according to claim 1, wherein the presented information includes information indicating the state of the moving object at the time of the request.

5. An information providing device according to claim 1, wherein the presented information includes information on the urgency of the remote operation.

6. An information providing device according to claim 5, wherein said notification device notifies said presented information with at least one of a different display and a different sound depending on said urgency level.

7. An information providing device according to claim 1, wherein the presented information includes information on qualifications required of the remote operator for the remote operation.

8. An information providing device according to claim 1, wherein the presented information includes information on a condition for completing the remote operation.

9. An information providing device according to claim 1, wherein the presented information includes information on the degree of difficulty of the remote operation.

10. An information providing device according to claim 1, wherein said notification device includes at least one of a display and a speaker of a remote control device to be operated by said remote operator to perform said remote control based on said request.

11. An information providing device according to claim 1, wherein the presented information includes different contents depending on the proficiency of the one or more candidate operators in remote operation.

12. An information providing device according to claim 1, wherein said information providing device is disposed at a waiting location for said one or more candidate operators.

13. An information providing device according to claim 1, wherein the information providing device is a mobile terminal carried by the one or more candidate operators.

14. An information provision method for a computer to provide information, comprising: acquiring presentation information indicating the contents of a remote operation request from a mobile body; and notifying one or more candidate operators who are candidates for the remote operator, of the acquired presentation information via a notification device, before a remote operator who will accept the request is determined.

15. An information provision program executed by a computer, which causes the computer to execute the following: acquiring presentation information indicating the contents of a remote operation request from a mobile body; and notifying one or more candidate operators who are candidates for the remote operator, of the acquired presentation information via a notification device, before a remote operator who will accept the request is determined.

Images