Information processing method, information processing device, and program

Abstract

An information processing method is an information processing method in an information processing apparatus, in which remote operation information relating to a remote operation with respect to a first moving body is obtained, a first mobile body that is a mobile body that requires remote operation among a plurality of mobile bodies (M1 to Mm) that are each capable of autonomously moving and capable of remote operation (S104) is obtained, and mobile body information relating to one or more second mobile bodies that are mobile bodies other than the first mobile body among the plurality of mobile bodies (M1 to Mm) is obtained. The mobile body is remotely monitored by a monitor responsible for remotely monitoring the first mobile body (S101), determines whether or not to cause the monitor to remotely operate the first mobile body on the basis of the remote operation information and the mobile body information (S106), and notifies the first mobile body on the basis of the determination result of whether or not to cause the monitor to remotely operate the first mobile body (S108).

Description

Technical Field

[0001] The present disclosure relates to an information processing method, an information processing device, and a program for processing information for remotely supervising a plurality of mobile bodies. Background Art

[0002] There is a remote monitoring method in which an operator at a remote location monitors a mobile object that can move autonomously, such as an automatic delivery robot, and operates the mobile object as needed. This method is advantageous in that, when the mobile object cannot move autonomously due to some reason, such as an obstacle on the road, the mobile object can be operated remotely, so that the service can continue without rushing to the site. In addition, in such a remote monitoring method, by enabling each operator to monitor multiple mobile objects, it is expected that low-cost and efficient mobile services can be achieved.

[0003] Patent Document 1 discloses a determination device that determines a timing to change a supervisor who monitors a plurality of moving objects based on a timing at which a status of any one of the plurality of moving objects satisfies a predetermined condition.

[0004] (Prior art literature)

[0005] (Patent Document)

[0006] Patent Document 1: Japanese Patent No. 6679657 Summary of the invention

[0007] Problems to be solved by the invention

[0008] For example, a supervisor monitors multiple moving bodies and operates one moving body when there is a problem. It is conceivable that when one moving body is being operated and another moving body also needs to be operated, the latter moving body cannot be operated and is eventually left alone (hereinafter referred to as a "leave request"). In the technology described in Patent Document 1, by changing the monitoring of other moving bodies to be performed by other supervisors, it is possible to suppress leave requests.

[0009] However, when the number of moving objects being monitored by other monitors has reached the upper limit of the monitoring, it is difficult to change the monitoring of other moving objects to other monitors, and in this case, the other moving objects will be parked. In this way, the service efficiency of the moving objects will be reduced.

[0010] Therefore, the present disclosure provides an information processing method and the like that can efficiently remotely monitor a plurality of mobile objects.

[0011] Means for solving problems

[0012] An information processing method involved in one embodiment of the present invention is an information processing method in an information processing device. In the information processing method, remote operation information related to remote operation on a first mobile body is obtained, and the first mobile body is a mobile body that needs to be remotely operated among multiple mobile bodies, each of which can move autonomously and can be remotely operated. Mobile body information related to one or more second mobile bodies among the multiple mobile bodies is obtained, and the one or more second mobile bodies are mobile bodies other than the first mobile body, and are mobile bodies that are remotely monitored by a monitor who is responsible for remotely monitoring the first mobile body. Based on the remote operation information and the mobile body information, it is determined whether to allow the monitor to remotely operate the first mobile body, and notification is made based on the result of the determination of whether to allow the monitor to remotely operate the first mobile body.

[0013] An information processing device involved in one embodiment of the present invention comprises: a remote operation information obtaining unit, which obtains remote operation information related to remote operation on a first mobile body, wherein the first mobile body is a mobile body that needs to be remotely operated among multiple mobile bodies, each of which can move autonomously and can be remotely operated; a mobile body information obtaining unit, which obtains mobile body information related to one or more second mobile bodies among the multiple mobile bodies, wherein the one or more second mobile bodies are mobile bodies other than the first mobile body and are mobile bodies that are remotely monitored by a monitor who is responsible for remotely monitoring the first mobile body; a judgment unit, which judges whether to allow the monitor to remotely operate the first mobile body based on the remote operation information and the mobile body information; and a notification unit, which makes a notification based on the judgment result of whether to allow the monitor to remotely operate the first mobile body.

[0014] A program according to one aspect of the present disclosure is a program for causing a computer to execute the above-mentioned information processing method.

[0015] In addition, these general or specific aspects may be implemented by a system, method, integrated circuit, computer program, or a computer-readable CD-ROM or other recording medium, or by any combination of a system, method, integrated circuit, computer program, and recording medium. Furthermore, the recording medium may be a non-transitory recording medium.

[0016] Effects of the Invention

[0017] The information processing method and the like disclosed in the present invention can efficiently remotely monitor a plurality of mobile objects.

[0018] In addition, the better effects and advantages of one embodiment of the present disclosure can be clearly seen from the specification and the drawings. Although the advantages and / or effects involved are provided by the features described in several embodiments and the specification and the drawings, it is not necessary to provide all of them in order to obtain one or more of the same features. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 An example of the overall configuration of the remote monitoring system in the embodiment is shown.

[0020] Figure 2 This is a block diagram showing an example of the functional configuration of the server device in the embodiment.

[0021] Figure 3 An example of the data structure of the operator information in the embodiment is shown.

[0022] Figure 4 An example of the data structure of a plurality of partitions in the embodiment is shown.

[0023] Figure 5 This is a conceptual diagram showing an example of a risk map in the embodiment.

[0024] Figure 6 An example of the data structure of the remote operation information in the embodiment is shown.

[0025] Figure 7 This is a block diagram showing an example of the functional configuration of a moving object in the embodiment.

[0026] Figure 8 This is a block diagram showing an example of the functional configuration of the operator terminal device in the embodiment.

[0027] Fig. 9 An example of the monitoring / operation screen in the embodiment is shown.

[0028] Fig.10 Another example of the monitoring / operation screen in the embodiment is shown.

[0029] Fig.11 An example of a flowchart of a process executed by the server device in the embodiment is shown.

[0030] Fig.12 An example of a flowchart of a process for obtaining the required time in the embodiment is shown.

[0031] Fig.13 An example of a flowchart of a process for determining whether or not there is a risk in the embodiment is shown. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments will be described with reference to the drawings.

[0033] In addition, the embodiments described below are all general or specific examples. The values, shapes, materials, components, configuration positions and connection methods of the components, steps and the order of the steps shown in the following embodiments are all examples, and their purpose is not to limit the present disclosure. Moreover, the components of the following embodiments that are not recorded in the independent technical solutions will be described as arbitrary components.

[0034] In addition, each figure is a schematic diagram and is not a strict illustration. In addition, in each figure, the same components are given the same reference numerals.

[0035] (Implementation Method)

[0036] (Explanation on remote control of one moving object without taking over remote monitoring of other moving objects)

[0037] First, the feasibility of remotely controlling one moving object without taking over remote monitoring of other moving objects will be described.

[0038] The state in which the operator can remotely monitor a certain moving object specifically refers to the state in which the operator can confirm whether the moving object needs to be remotely operated and can start the remote operation when necessary. Regarding the placement request of other moving objects, for example, when other moving objects cannot drive autonomously and are stationary at the intersection, it will hinder traffic. On the other hand, for the operation request from other moving objects parked at the departure point or arrival point, when the operator needs to perform remote operations such as confirming the safety of the surroundings and instructing departure, the placement request is less likely to hinder traffic.

[0039] Therefore, even if a placement request is made, whether there is a risk will differ depending on the status of the other moving objects that issued the operation request. If there is no risk, there is no need to take over the remote monitoring of the other moving objects, and there is no need to perform parking control. Therefore, for example, when an operator remotely operates a certain moving object, the operator only needs to confirm that there is no risk caused by the placement request for other moving objects during remote monitoring.

[0040] In the following, a risk due to a placement request may be expressed as "risky", and a risk not due to a placement request may be expressed as "no risk". Confirmation of when there will be no risk from the current moment depends on the time required for the operator to respond to the operation request. There are also cases where this time is referred to as "required time". Regarding the time for operating a moving body, there is not only the time required for the operation itself, such as remote control using a steering wheel controller, but also the time required to decide when to perform the operation even if the operation itself is only a moment, such as using a button to issue a departure instruction. For example, the required time includes the time required for the above-mentioned decision and the time required for the remote operation itself.

[0041] For example, when an operator receives an operation request from one mobile object while remotely monitoring multiple mobile objects, the status changes of other mobile objects are predicted during the required time for remote operation of the operation request. In this way, when an operation request is received from another mobile object at any timing during the required time, it is determined whether there is a risk or not. In this way, when it is determined that there is no risk, the operator can remotely operate one mobile object without handing over the remote monitoring of other mobile objects to other operators.

[0042] Furthermore, the remote monitoring by one operator may be performed by simply confirming the presence or absence of operation requests from a plurality of moving objects, and is not limited to confirmation by video or sound, etc. Furthermore, one operator may be responsible for remote operation of a plurality of moving objects.

[0043] (Description of remote monitoring system)

[0044] Figure 1 An example of the overall configuration of the remote monitoring system 1 in the embodiment is shown.

[0045] The remote monitoring system 1 is a system for remotely monitoring a plurality of moving bodies M1 to Mm by a plurality of operators O1 to On. Figure 1 As shown, the remote monitoring system 1 includes a server device S capable of communicating with a plurality of mobile bodies M1 to Mm via a network N, a plurality of mobile bodies M1 to Mm, a plurality of operator terminal devices D1 to Dn, and a service system X. In the present embodiment, the server device S is an example of an information processing device, and each of the plurality of operators O1 to On is an example of a supervisor.

[0046] Each of the plurality of moving bodies M1 to Mm can move autonomously and can be remotely operated. In the present embodiment, each of the plurality of moving bodies M1 to Mm can be remotely operated by any one of the plurality of operators O1 to On.

[0047] For example, each of the plurality of moving bodies M1 to Mm is equipped with a camera 201, a laser sensor 202, a GNSS (Global Navigation Satellite System) 203, an ECU (Electronic Control Unit) 21, and a communication module 22 ( Figure 7 ). The camera 201 and the laser sensor 202 observe the surrounding conditions of the mobile body. The GNSS 203 obtains the position information of the mobile body. The ECU 21 controls the autonomous driving of the mobile body. The communication module 22 communicates with the server device S via LTE (Long Term Evolution) or the like. Each of the plurality of mobile bodies M1 to Mm is a vehicle capable of autonomous driving, such as an automatic delivery robot or an unmanned bus.

[0048] The allocation of vehicles to which the plurality of mobile bodies M1 to Mm should be moved depends on the service provided. For example, in the case of an automatic delivery robot for an on-demand food delivery service, a vehicle must be allocated to a loading location or a delivery location of the goods according to an order from a user. Such an allocation instruction is determined by the service system X for each service and is defined by an interface between the service system X and the server device S. The service system X sends an allocation instruction to which the plurality of mobile bodies M1 to Mm should be moved through the server device S.

[0049] The plurality of operator terminal devices D1 to Dn are devices provided corresponding to the plurality of operators O1 to On, and are devices for each of the plurality of operators O1 to On to remotely monitor all or part of the plurality of moving bodies M1 to Mm, and are also devices for remotely operating them. Each of the plurality of operator terminal devices D1 to Dn is connected to output devices such as a monitor 315 and a speaker 316, and input devices such as a steering wheel controller 311, a button 312, a keyboard 313, a mouse 314, and a microphone ( Figure 8 ).

[0050] As a method of operating remote monitoring, for example, the operator O1 may be responsible for remote monitoring and remote operation of the moving bodies M1 to M4, and the operator O2 may be responsible for remote monitoring and remote operation of the moving bodies M5 to M8, so that the remote monitoring and remote operation of the moving bodies M1 to Mm are shared by multiple operators O1 to On in advance. Alternatively, multiple operators O1 to On may remotely monitor the moving bodies M1 to Mm together, but the operation of one of the moving bodies M1 to Mm is performed by one of the multiple operators O1 to On.

[0051] In addition, the operator may be one. In this case, for example, ECU21 has an MRM (Minimum Risk Manager) function, and when it is determined that the driving control of the mobile body equipped with the ECU21 (the mobile body equipped with the ECU21 among the multiple mobile bodies M1 to Mm) does not need to be taken over by the operator, the MRM function is used to stop the mobile body at a safe position. As to which of the multiple mobile bodies M1 to Mm the multiple operators O1 to On are responsible for remote monitoring and remote operation, for example, a user interface (UI) may be provided so that the operator can freely select through the operator terminal device corresponding to the operator among the multiple operator terminal devices D1 to Dn, and the operator himself may decide. And for example, the server device S may decide the person in charge of remote monitoring and remote operation of each of the multiple mobile bodies M1 to Mm from the multiple operators O1 to On.

[0052] (Description of each device constituting the remote monitoring system)

[0053] (Server device)

[0054] Figure 2 This is a block diagram showing an example of the functional configuration of the server device S in the embodiment.

[0055] The server device S includes a communication unit 101, a mobile body information acquisition unit 102, an operator information acquisition unit 103, a risk map storage unit 104, a remote operation information storage unit 105, a first mobile body identification unit 106, a remote operation information acquisition unit 107, a judgment unit 108, an operation permission determination unit 109, an operation entrustment distribution unit 110, a mobile body information distribution unit 111, a travel control information distribution unit 112, and a vehicle allocation instruction distribution unit 113. For example, the communication unit 101 is realized by a communication module or the like. In addition, the mobile body information acquisition unit 102, the operator information acquisition unit 103, the first mobile body identification unit 106, the remote operation information acquisition unit 107, the judgment unit 108, the operation permission determination unit 109, the operation entrustment distribution unit 110, the mobile body information distribution unit 111, the travel control information distribution unit 112, and the vehicle allocation instruction distribution unit 113 are realized by a processor or the like. In addition, the risk map storage unit 104 and the remote operation information storage unit 105 are realized by a memory or the like.

[0056] The server device S is connected to a plurality of mobile bodies M1 to Mm, a plurality of operator terminal devices D1 to Dn, and a service system X via a network N.

[0057] The communication unit 101 transmits and receives information to and from a plurality of mobile bodies M1 to Mm, a plurality of operator terminal devices D1 to Dn, and a service system X via the network N.

[0058] The mobile body information acquisition unit 102 acquires mobile body information from the information received by the communication unit 101. The mobile body information is information related to the plurality of mobile bodies M1 to Mm. For example, the mobile body information is information sent from each of the plurality of mobile bodies M1 to Mm, and includes at least information indicating whether a request for remote operation of the mobile body is made, sensor data such as images and sounds obtained by the mobile body, surrounding obstacle data obtained by the mobile body for autonomous driving control, the position of the mobile body on a map, and the planned driving path of the mobile body.

[0059] The operator information acquisition unit 103 acquires operator information from the information received by the communication unit 101. The operator information is information related to a plurality of operators O1 to On. For example, the operator information includes a list of moving bodies sent from each of a plurality of operator terminal devices D1 to Dn. The moving body list is a list of moving bodies that the operator is responsible for remotely monitoring, selected by an operator corresponding to the operator terminal device among a plurality of operators O1 to On from a plurality of moving bodies M1 to Mm on a UI provided by the operator terminal device. Furthermore, for example, if there is a moving body being remotely operated among the plurality of moving bodies M1 to Mm, the operator information includes information showing the details of the moving body and the remote operation. The details of the remote operation are, for example, data showing what kind of operation signal is received by what kind of operation device such as a steering wheel controller and a button.

[0060] Figure 3 An example of the data structure of the operator information in the embodiment is shown. The operator ID is an identifier for distinguishing a plurality of operators O1 to On. The moving body ID is an identifier for distinguishing a plurality of moving bodies M1 to Mm. The moving body ID in remote monitoring indicates a moving body among the plurality of operators O1 to On. Figure 3 In the table, the operator whose operator ID is in the same column is responsible for which mobile object among the multiple mobile objects M1 to Mm is remotely monitored. The mobile object ID in the remote operation indicates which mobile object among the multiple operators O1 to On is responsible for remote monitoring. Figure 3 In the table, an operator whose operator ID is in the same column is performing remote operation on which mobile object among the multiple mobile objects M1 to Mm.

[0061] The remote operation signal is stored in different formats depending on the remote operation device. For example, when the remote operation device is a button, whether the button is pressed (press p) is stored as 0 or 1. For example, when the operation device is a steering wheel controller, the driving control of the mobile body is stored as acceleration a and steering amount s.

[0062] Return to Figure 2, the risk map storage unit 104 stores the risk map. The risk map is a map that divides a certain area on the map into multiple partitions, and shows whether each partition will become a risky partition (hereinafter referred to as a risky partition) or a risk-free partition (hereinafter referred to as a risk-free partition) due to a placement request. Here, the placement request specifically refers to a state in which each of the one or more second moving bodies is not remotely operated. In addition, the one or more second moving bodies are moving bodies that are remotely monitored by an operator who is responsible for remotely monitoring a first moving body that requires remote operation (an operator who is responsible for remote monitoring of the first moving body among multiple operators O1 to On), among the multiple moving bodies M1 to Mm, and are moving bodies other than the first moving body.

[0063] Figure 4 An example of the data structure of multiple partitions in the implementation method is shown. The partition ID is an identifier for distinguishing multiple partitions. Here, the partition is defined as a rectangular area determined by four points on the map. Here, the points on the map are stored in the form of latitude and longitude. A partition is defined by the northwestmost point (located in the northwest), the northeastmost point, the southwestmost point, and the southeastmost point. The altitude can be added to the point information on the map, and the floor can also be added if it is indoors. In addition, the partition can also be composed of a polygon with more than three points.

[0064] Figure 5 : is a conceptual diagram showing an example of a risk map in an implementation method. Here, risky partitions are represented by rectangles with diagonal lines drawn on them, and risk-free partitions are represented by rectangles with dot patterns drawn on them. Risky partitions and risk-free partitions can also be changed according to actual conditions. In the case where each of the multiple mobile bodies M1 to Mm is traveling on one side of a roadway or a sidewalk, a part of the roadway or the sidewalk can be used as a risk-free partition, and other areas of the roadway or the sidewalk can be used as risky partitions. Risky partitions and risk-free partitions can also be changed by time period and date and time. Risky partitions and risk-free partitions can also be changed according to the weather.

[0065] Back to Figure 2 The remote operation information storage unit 105 stores the remote operation information. The remote operation information is information related to the remote operation on the first moving body. For example, the remote operation information includes the required time required to remotely operate the first moving body, and the required time includes the time required to determine the timing of remotely operating the first moving body and the time required to remotely operate the first moving body. The remote operation information storage unit 105 distinguishes the remote operation on the first moving body according to at least each combination of the remotely operated device and the partition, and stores the required time for each combination.

[0066] Figure 6 An example of the data structure of the remote operation information in the embodiment is shown. The required time is distinguished according to the combination of the partition ID and the remotely operated device. The required time of each remote operation ID is stored in seconds using the value obtained by prior verification as the initial value. The value of the required time can also be updated based on statistical speculation based on actual performance. In addition, the required time to be stored can be changed based on a text such as departure after safety confirmation, calling attention, and avoiding obstacles so that the operator can understand the purpose of the remote operation. As a method for how the operator performs remote operation or needs to grasp the surroundings of the first moving body after judging the execution timing of the remote operation, the required time to be stored can be changed according to sensor data. The required time to be stored can also be changed by time period and date time. The required time to be stored can also be changed according to the weather.

[0067] The first mobile body determination unit 106 determines the first mobile body that needs remote operation based on the request information included in the mobile body information obtained by the mobile body information obtaining unit 102. For example, the first mobile body outputs the request information for requesting the remote operation of the first mobile body, the first mobile body determination unit 106 obtains the request information output by the first mobile body, and determines the first mobile body based on the request information.

[0068] The remote operation information acquisition unit 107 acquires the remote operation information. For example, the remote operation information acquisition unit 107 predicts the remotely operated device by whether the steering wheel controller is needed to avoid obstacles based on the moving body information acquired by the moving body information acquisition unit 102, determines the zone where the first moving body is located based on the position of the first moving body included in the moving body information and with reference to the risk map, and acquires the required time with reference to the remote operation information storage unit 105.

[0069] Return to Figure 2 The judgment unit 108 extracts information of an operator who is in charge of remote monitoring of the first moving body determined by the first moving body determination unit 106 from the operator information obtained by the operator information acquisition unit 103 among the multiple operators O1 to On. Furthermore, the judgment unit 108 estimates each of the zones where one or more second moving bodies may exist during the period from the current moment to the required time obtained by the remote operation information acquisition unit 107, based on the positions of each of the one or more second moving bodies included in the moving body information of the one or more second moving bodies that the operator is in charge of remotely monitoring among the multiple moving bodies M1 to Mm, and the paths of the planned travel of each of the one or more second moving bodies. Furthermore, the judgment unit 108 judges whether all the zones estimated with reference to the risk map are risky zones or non-risky zones.

[0070] The operation permission determination unit 109 determines that the first moving body can be remotely operated when the judgment unit 108 determines that all of the one or more second moving bodies are risk-free, and determines that the first moving body cannot be remotely operated when the judgment unit 108 determines that at least one of the one or more second moving bodies is risky.

[0071] When the operation permission determination unit 109 determines that the remote operation is possible, the operation delegation distribution unit 110 distributes the operation delegation information to the operator terminal device of the operator who is to perform the remote operation among the plurality of operators O1 to On (the operator terminal device corresponding to the operator among the plurality of operator terminal devices D1 to Dn) via the communication unit 101. The operation delegation distribution unit 110 is an example of a notification unit that notifies the operator based on the result of the determination of whether to allow the operator to perform the remote operation on the first moving object.

[0072] The mobile body information distribution unit 111 distributes the mobile body information of the mobile bodies that each of the multiple operators O1 to On is responsible for remotely monitoring (the mobile bodies that the operator is responsible for remotely monitoring among the multiple mobile bodies M1 to Mm), and the mobile body information of the mobile bodies that each of the multiple operators O1 to On is remotely operating (the mobile bodies that the operator is remotely operating among the multiple mobile bodies M1 to Mm) to each operator terminal device D1 to Dn.

[0073] The travel control information distribution unit 112 distributes the operation signal included in the operator information to the mobile object via the communication unit 101 .

[0074] The vehicle allocation instruction distribution unit 113 obtains the vehicle allocation instruction on where the plurality of moving bodies M1 to Mm should move, which is transmitted from the service system X from among the information received by the communication unit 101 , and distributes the vehicle allocation instruction to the moving bodies via the communication unit 101 .

[0075] In this way, the remote operation information acquisition unit 107 acquires remote operation information related to the remote operation on the first mobile body, which is a mobile body that needs to be remotely operated among the plurality of mobile bodies M1 to Mm, each of which can move autonomously and can be remotely operated. In this embodiment, the remote operation information acquisition unit 107 acquires the remote operation information stored in the remote operation information storage unit 105.

[0076] Furthermore, the moving body information obtaining unit 102 obtains moving body information related to one or more second moving bodies among the plurality of moving bodies M1 to Mm, the one or more second moving bodies being moving bodies other than the first moving body and being moving bodies that are remotely monitored by an operator in charge of remotely monitoring the first moving body (an operator in charge of remotely monitoring the first moving body among the plurality of operators O1 to On). In the present embodiment, the moving body information obtaining unit 102 obtains the moving body information from one or more second moving bodies. For example, the moving body information shows the positions of the one or more second moving bodies and the paths that the one or more second moving bodies are scheduled to travel.

[0077] Then, the determination unit 108 determines whether to cause the operator to remotely operate the first moving object based on the remote operation information and the moving object information.

[0078] The operation entrustment distribution unit 110 makes a notification based on the result of the determination of whether to allow the operator to remotely operate the first moving object. For example, the operation entrustment distribution unit 110 makes the notification through an operator terminal device corresponding to the operator in charge of remotely monitoring the first moving object among the plurality of operator terminal devices D1 to Dn.

[0079] Furthermore, in this embodiment, the first mobile body outputs request information for requesting remote operation of the first mobile body, the mobile body information obtaining unit 102 obtains the request information output by the first mobile body, and the first mobile body specifying unit 106 specifies the first mobile body based on the request information.

[0080] Furthermore, in this embodiment, the remote operation information includes a required time required to remotely operate the first mobile object, and the required time includes a time required to determine a timing for remotely operating the first mobile object and a time required to remotely operate the first mobile object.

[0081] Furthermore, in the present embodiment, the required time is predetermined by a device that remotely controls the first moving object.

[0082] Furthermore, in the present embodiment, the required time is predetermined for each of the plurality of sections 1 to 21 (for each of the plurality of sections 1 to 21 ) in the area in which the first moving body moves within the range and is divided into the plurality of sections 1 to 21 .

[0083] Alternatively, the required time may be predetermined based on at least one of the date when the first moving body is remotely operated, the time period when the first moving body is remotely operated, and the weather when the first moving body is remotely operated. For example, the required time may be determined to be slightly shorter on a date or time period with less traffic, and slightly longer on a date or time period with more traffic. Also, for example, the required time may be determined to be slightly shorter on a sunny day or other weather that makes remote operation easy, and slightly longer on a rainy day or other weather that makes remote operation difficult.

[0084] Furthermore, in this embodiment, the judgment unit 108 judges whether there is a risk when not remotely operating each of the one or more second moving bodies based on the remote operation information and the moving body information, and when it is judged that there is no risk, it is determined to allow the operator to remotely operate the first moving body.

[0085] Specifically, in this embodiment, the determination unit 108 determines that there is a risk when at least one of the one or more second moving bodies will be located in a risky partition among the multiple partitions 1 to 21 during the required time. For example, the determination unit 108 determines whether each of the one or more second moving bodies will be located in a risky partition among the multiple partitions 1 to 21 during the required time based on the respective positions of the one or more second moving bodies and the planned travel paths.

[0086] Furthermore, in the present embodiment, whether each of the plurality of partitions 1 to 21 is a risky partition is pre-set according to each of the plurality of partitions 1 to 21 in the area divided by the plurality of partitions 1 to 21 in which each of the one or more second mobile bodies moves within the range.

[0087] Alternatively, whether or not a zone is a risky zone may be predetermined based on at least one of a date when the first moving body is remotely operated, a time period when the first moving body is remotely operated, and weather when the first moving body is remotely operated. For example, for a zone among the multiple zones 1 to 21, if it is a date or time period with low traffic volume, it may be determined to be a risk-free zone, and if it is a date or time period with high traffic volume, it may be determined to be a risky zone. Also, for example, for a zone among the multiple zones 1 to 21, if it is a sunny day or other weather that is easy to remotely operate, it may be determined to be a risk-free zone, and if it is a rainy day or other weather that is difficult to remotely operate, it may be determined to be a risky zone.

[0088] Furthermore, in the present embodiment, when the operation entrustment distribution unit 110 determines that the operator is to remotely operate the first moving object, the operation entrustment distribution unit 110 issues a notification indicating that the operator is to remotely operate the first moving object. For example, the operation entrustment distribution unit 110 issues a notification indicating that the operator is to remotely operate the first moving object by distributing the operation entrustment to the operator.

[0089] Furthermore, in the present embodiment, the operation entrustment distribution unit 110 issues a notification indicating that the first moving body is to be stopped when it is determined that the operator is not to remotely operate the first moving body. For example, the operation entrustment distribution unit 110 issues a notification indicating that the first moving body is to be stopped by distributing information indicating that the first moving body is to be stopped to the operator.

[0090] (Moving body)

[0091] Figure 7 This is a block diagram showing an example of the functional configuration of the mobile object M1 in the embodiment.

[0092] The mobile body M1 includes a camera 201 and a laser sensor 202 for observing surrounding conditions, a GNSS 203 for obtaining position information, an ECU 21 for controlling autonomous driving, and a communication module 22 for communicating with a server device S via a network N.

[0093] The ECU 21 includes an acquisition unit 211 , a detection unit 212 , a self-position estimation unit 213 , a determination control unit 214 , an operation request determination unit 215 , and a moving object information output unit 216 .

[0094] The communication module 22 receives a vehicle allocation instruction indicating where the mobile body M1 should move to and driving control information such as an operator's operation and the operation of the MRM function from the server device S, and outputs the information to the ECU 21 .

[0095] The acquisition unit 211 acquires a vehicle allocation instruction and a travel control instruction from the communication module 22 .

[0096] The detection unit 212 acquires sensor data from the camera 201 , the laser sensor 202 , the sound pickup microphone, and the like for observing the surroundings of the moving object M1 , and detects surrounding obstacles.

[0097] The own position estimation unit 213 estimates the position of the moving object M1 at least using the GNSS 203 .

[0098] The judgment control unit 214 judges the planned driving path based on the vehicle allocation instructions from the communication module 22, the position of the mobile body M1 estimated by the own position estimation unit 213, and the information of the surrounding obstacles detected by the detection unit 212, and controls the acceleration, deceleration and steering of the mobile body M1 or controls the acceleration, deceleration and steering of the mobile body M1 according to the driving control instructions from the communication module 22.

[0099] The operation request determination unit 215 determines to output an operation request when the moving body M1 cannot continue autonomous driving, for example, because the moving body M1 has stopped for a certain period of time.

[0100] The moving body information output unit 216 generates moving body information including at least the presence or absence of an operation request, sensor data, surrounding obstacle data, the position of the moving body, and a planned travel route, and outputs the moving body information to the communication module 22 .

[0101] The communication module 22 receives the moving object information from the ECU 21 and transmits the information to the server device S. Alternatively, the communication module 22 may generate part or all of the moving object information.

[0102] Since each of the plurality of moving bodies M2 to Mm has the same configuration as the moving body M1 , detailed description thereof will be omitted.

[0103] (Operator Terminal Device)

[0104] Figure 8 The operator terminal device D1 includes a communication unit 301 , an input receiving unit 302 , a screen generating unit 303 , a UI control unit 304 , an operator information output unit 305 , a mobile body information obtaining unit 306 , and an operation request detecting unit 307 .

[0105] The communication unit 301 has a function of transmitting and receiving information to and from a connected server device S via the network N.

[0106] The input receiving unit 302 receives input from the operator O1 via input devices such as a steering wheel controller 311 , buttons 312 , a keyboard 313 , and a mouse 314 .

[0107] The screen generator 303 generates a login screen, a selection screen for a mobile body to be in charge of remote monitoring and a mobile body to be in charge of remote operation among the plurality of mobile bodies M1 to Mm, and a monitoring / operation screen according to the input of the operator O1 .

[0108] The UI control unit 304 controls output devices such as a monitor 315 and a speaker 316 in order to present the generated screen.

[0109] The operator information output unit 305 outputs operator information including at least the logged-in operator, the mobile object selected by the operator to be monitored / operated, and the operation signal for the mobile object to the server device S via the communication unit 301. In addition, the screen generation unit 303 uses the mobile object information obtained by the mobile object information acquisition unit 306 to generate a monitoring / operation screen.

[0110] The moving body information obtaining unit 306 obtains moving body information of a moving body that the operator O1 is responsible for remotely monitoring (a moving body that the operator O1 is responsible for remotely monitoring among the plurality of moving bodies M1 to Mm) from among the information received by the communication unit 301 .

[0111] The operation request detection unit 307 detects whether or not the information received by the communication unit 301 contains a request for remote operation to the operator O1 .

[0112] Fig. 9 An example of a monitoring / operation screen in an embodiment is shown. Specifically, Fig. 9 This is an example of a monitoring / operation screen when the operator O1 has selected remote monitoring of the moving bodies M1 to M4. The moving body information acquisition unit 306 displays the positions of the moving bodies and the planned travel routes included in the acquired moving body information of the moving bodies M1 to M4 as icons in a "map" and lines with arrows. The relative positions of the image from the first perspective and the obstacles from the bird's-eye view in the sensor data included in the moving body information of the moving body M1 are displayed as a point cloud.

[0113] The moving bodies M2 to M4 may also be displayed based on images or point clouds at the same time. When the icon of the moving body in the "map" is selected by the keyboard 313 and the mouse 314, the input receiving unit 302 may receive the input and switch the moving body for displaying the sensor data. In addition, the moving body that has issued the operation request may be displayed separately from the icons of other moving bodies by making the color of the icon of the moving body that has issued the operation request different from the color of the icons of other moving bodies.

[0114] Fig.10 Another example of the monitoring / operation screen in the embodiment is shown. Specifically, Fig.10This is another example of the monitoring / operation screen when the operator O1 selects to be in charge of remote monitoring of the moving bodies M1 to M4. For example, when the moving body M1 issues an operation request, the server device S determines whether to allow the operator O1 to remotely operate the moving body M1. When the server device S determines that the operator O1 is to remotely operate the moving body M1, the server device S issues a remote operation request to the operator O1. When the operation request detection unit 307 of the operator terminal device D1 detects the remote operation request, it compares the remote operation request with the request. Fig. 9 The operator O1 is notified of the request through a text together with the display so that the operator O1 can start the operation.

[0115] Furthermore, the input accepting unit 302 may not accept a part of the operation until the operation request detecting unit 307 detects the request for remote operation. This can prevent human errors caused by unnecessary operation intervention.

[0116] Furthermore, when multiple moving objects are displayed simultaneously through images or point clouds, if a certain moving object issues an operation request, the display is switched to display only the moving object. For example, when multiple moving objects are displayed simultaneously, only the camera image in front of each moving object is displayed. Furthermore, when a moving object is displayed, more information about the moving object can be displayed by displaying camera images in all directions of the moving object.

[0117] On the contrary, when performing remote control of a moving object, if sufficient information can be obtained by displaying multiple moving objects, the display of the multiple moving objects may not be changed, and the display contents of other moving objects can be maintained in a state where they can also be confirmed from the surrounding field of vision.

[0118] (An example of processing performed by a server device)

[0119] Fig.11 An example of a flowchart of a process executed by the server device S in the embodiment is shown. Specifically, Fig.11 This is a flowchart showing an example of the overall situation in which the server device S repeatedly executes the processing in a certain cycle.

[0120] First, the moving body information obtaining unit 102 obtains the latest moving body information transmitted from the plurality of moving bodies M1 to Mm among the information received by the communication unit 101 ( S101 ).

[0121] The operator information acquisition unit 103 acquires the latest operator information transmitted from the plurality of operator terminal devices D1 to Dn among the information received by the communication unit 101 ( S102 ).

[0122] The first moving object identification unit 106 determines whether there is a first moving object (S103). For example, the first moving object identification unit 106 determines whether there is a first moving object based on whether there is an operation request included in the moving object information obtained in S101.

[0123] Specifically, for example, the first mobile body outputs request information for requesting remote operation of the first mobile body. The mobile body information acquisition unit 102 obtains the request information output by the first mobile body. When the mobile body information acquisition unit 102 obtains the request information, the first mobile body identification unit 106 determines that there is a first mobile body. In this way, when there is a first mobile body, the first mobile body identification unit 106 identifies the first mobile body based on the request information. And, when the first mobile body identification unit 106 determines that there is no first mobile body, the processing ends.

[0124] When it is determined that the first moving object exists, the remote operation information acquisition unit 107 determines which device needs to be used for remote operation based on the moving object information acquired in S101, and estimates the required time by referring to the remote operation information storage unit 105 (S104). S104 will be described in detail later.

[0125] The determination unit 108 extracts operator information of an operator who is in charge of remote monitoring of the first moving object from among the plurality of operators O1 to On from the operator information acquired in S102 ( S105 ).

[0126] The judgment unit 108 judges whether there is a risk (S106). For example, the judgment unit 108 estimates the predicted position of one or more second moving bodies from the current moment to the required time determined in S104, based on the respective positions of the plurality of moving bodies M1 to Mm included in the moving body information obtained in S101 and the planned travel paths, for one or more second moving bodies that are being remotely monitored by the operator extracted in S105. Next, at any timing from the current moment to the required time, when there is an operation request from a certain second moving body, the judgment unit 108 refers to the risk map and judges whether the second moving body is at risk or not at the predicted position. The details of S106 will be described later.

[0127] The operation determination unit 109 determines whether the first moving body can be remotely operated (S107). For example, if the operation determination unit 109 determines in S106 that there is no risk with respect to more than one second moving body, it determines that the first moving body can be remotely operated ("Yes" in S107). In addition, if the operation determination unit 109 determines that there is a risk with respect to at least one second moving body, it determines that the first moving body cannot be remotely operated ("No" in S107), and ends the processing.

[0128] When it is determined in S107 that the first moving body can be remotely operated, the operation entrustment distribution unit 110 distributes the operation entrustment to the operator terminal device of the operator who is responsible for remotely monitoring the first moving body via the communication unit 101 (S108). In addition, when it is determined in S107 that the first moving body cannot be remotely operated, it can be considered to hand over the remote monitoring of the first moving body to other operators, hand over the remote monitoring of one or more second moving bodies to other operators, or instruct the first moving body to perform MRM control on the grounds that it cannot respond to the operation request.

[0129] First, in the handover of remote monitoring of the first moving body, for the operator who is not remotely monitoring the first moving body at this moment, by adding the time until the state of being able to remotely monitor is added to the required time, and performing the processing of S106, it is possible to determine whether other operators can remotely operate the first moving body in the same way, and then it can be handed over to the operator who is determined to be able to remotely operate. On the other hand, in the handover of remote monitoring of one or more second moving bodies, it can be handed over to the operator who is not remotely operating. When other operators are performing remote operation, the driving control information distribution unit 112 instructs one or more second moving bodies to perform MRM control via the communication unit 101 on the grounds that the operation request cannot be responded to. Alternatively, it is also possible that the judgment control unit 214 of one or more second moving bodies automatically decides to perform MRM control after a certain time has passed since the operation request.

[0130] Thus, when the operation entrustment distribution unit 110 determines that the operator is to remotely operate the first moving body ("Yes" in S107), the operation entrustment distribution unit 110 distributes the operation entrustment to the operator, thereby notifying the operator to remotely operate the first moving body. For example, when the operation entrustment distribution unit 110 determines that the operator is not to remotely operate the first moving body ("No" in S107), the operation entrustment distribution unit 110 distributes information indicating that the first moving body is to be stopped to the operator, thereby notifying the operator to stop the first moving body.

[0131] (Details of Estimation of Required Time (S104))

[0132] Fig.12 An example of a flowchart of a process for obtaining the required time in an embodiment is shown. Specifically, Fig.12 4 is a flowchart showing an example of the process of S104. For example, this process is executed after S103 in which the first moving object is determined.

[0133] First, the remote operation information acquisition unit 107 acquires the moving object ID of the first moving object and identifies the first moving object (S201). For example, the moving object ID of the first moving object is represented by m.

[0134] The remote operation information acquisition unit 107 extracts the position, the planned travel path, and the obstacle data included in the moving body information of the first moving body from the moving body information acquired in S101 (S202). Specifically, for example, the position included in the moving body information of the first moving body m is a coordinate point p showing the latitude and longitude on the map. m The direction of travel is represented by the direction θ, which is the direction rotated clockwise with the north as 0 degrees. The list of discrete coordinates on the map of the planned travel route and the destination [w1 m ,···,w N m The obstacle data is represented by the distance d(α) to the obstacle at the direction α, which is the direction rotated clockwise when the moving direction of the first moving body m is 0 degrees. The resolution of the direction of the obstacle data is represented by Δα.

[0135] The remote operation information acquisition unit 107 determines in which partition of the risk map the position of the first moving body extracted in S202 will exist (S203). Specifically, for example, regarding which partition the first moving body m will exist in, it is possible to make an inside-outside judgment by using the conventional number of intersections. For example, if the number of intersections with the edges of the polygon extending from the point is an odd number, it can be judged to be inside. The partition ID of the determined partition is represented by b m .

[0136] The remote operation information acquisition unit 107 determines whether there is an obstacle on the planned travel route based on the planned travel route and obstacle data extracted in S202 ( S204 ).

[0137] Specifically, for example, according to the position p of the first moving object m m and [w1 m ,···,w N m ], where will the first moving body move next? m to w1 m And based on the position p of the first moving body m mThe obstacle data d(α) and the direction of travel θ are converted into a conventional coordinate system to obtain the location information of the obstacle on the map. Based on the common formula for the distance between a point and a straight line, the distance information between the obstacle and the planned driving path can be obtained. The distance information is obtained by using obstacle data with a resolution Δα of 0 to 360 degrees. For example, if the distance in the obstacle data of a predetermined number or more is less than a threshold, it is determined that there is an obstacle on the planned driving path. In addition, when the first moving body reaches w1 m After that, since the next move can be made through w1 m tow s m The line segment information is obtained, so the above judgment can also be made about the subsequent path.

[0138] The remote operation information acquisition unit 107 predicts the remotely operated device (S205). For example, when there is an obstacle, the remote operation information acquisition unit 107 determines that the obstacle needs to be avoided by the operator, and predicts that the remotely operated device is a steering wheel controller. Also, when there is no obstacle, the remote operation information acquisition unit 107 predicts that the remotely operated device is a button (S205).

[0139] The remote operation information acquisition unit 107 obtains the required time by checking the risk map based on the zone in which the moving body is located determined in S203 and the remotely operated device predicted in S205 ( S206 ).

[0140] As a specific example of the above processing, the partition ID of the partition where the first moving body exists is Figure 5 The situation 2 in the figure is explained.

[0141] In S201 , the remote operation information obtaining unit 107 obtains the moving object ID of the first moving object as m.

[0142] In S202, the remote operation information obtaining unit 107 converts the position included in the moving object information of the first moving object m into p m and the travel direction θ, the planned travel path is [w1 m ,···,w N m ], the obstacle data is extracted with a resolution of Δα and a distance d(α) at ​​an orientation α.

[0143] In S203, the remote operation information obtaining unit 107 determines the zone b in which the first moving object m exists. m =2.

[0144] In S204, the remote operation information obtaining unit 107 first obtains p m to w1 mNext, based on p m , the direction of travel θ, and the obstacle data d(α) to obtain the location information of the obstacle on the map. The common formula for points and lines is used to obtain the distance information of the line segment information and the location information of the obstacle. Similarly, the distance information of the obstacle data from 0 to 360 degrees with a resolution Δα is obtained. When the distance in the obstacle data of more than a predetermined number is less than the threshold, it is determined that there is an obstacle on the planned driving path.

[0145] Since the remote operation information acquisition unit 107 determines in S204 that there is an obstacle, it predicts in S205 that the remote-operated device is a steering wheel controller.

[0146] The remote operation information acquisition unit 107 obtains the remote operation information based on the partition b determined in S203. m =2 and the remotely operated device (steering wheel controller) predicted in S205, compared Figure 6 The table determines the required time to be 15 (seconds).

[0147] (Details of determining whether there is a risk (S106))

[0148] Fig.13 An example of a flowchart of a process for determining whether there is a risk in an embodiment is shown. Specifically, Fig.13 1 is a flowchart showing an example of the process of S106. For example, this process is executed after S105 in which the operator information of the operator who is in charge of the remote monitoring of the first moving object is extracted. This process is executed for each operator.

[0149] First, the determination unit 108 obtains the required time estimated in S104 (S301). Specifically, for example, the obtained value (seconds) of the required time is expressed as Δt.

[0150] Next, the judgment unit 108 obtains the positions and the planned travel paths included in the mobile body information for one or more second mobile bodies that the operator is responsible for remotely monitoring and included in the operator information extracted in S105 (S302). Specifically, for example, the one or more second mobile bodies that the operator is responsible for remotely monitoring and included in the extracted operator information are represented as a set of mobile body IDs M = {m1, m2, . . . , m k}. The moving body whose moving body ID is m is represented as moving body m. The position included in the moving body information of moving body m is represented as the coordinate point p showing the latitude and longitude on the map. m The planned travel route and the list of discrete coordinates of the destination on the map [w1 m ,···,w Nm ] to express.

[0151] The determination unit 108 estimates a zone where one or more second moving objects in the remote monitoring may exist from the current time to the required time Δt (S303). Specifically, for example, the determination unit 108 determines the position p of the moving object m. m and [w1 m ,···,w N m ] are included in which partition. By using the conventional number of intersections, it is possible to make an inside-outside judgment. For example, if the number of intersections with the edges of the polygon extending from the point is an odd number, it can be judged to be inside. The judgment result is expressed as a list of partition IDs [b0 m ,···,b N m ].

[0152] Find the coordinates that the moving body m can reach from the current time to the concentrated time Δt, relative to the planned travel path. m Arrival w1 m The time required, if the average speed v of the moving body is used k It can be calculated by the following formula.

[0153] [Mathematical formula 1]

[0154]

[0155] Similarly, for n where 1<n≤N holds, from w n-1 m Arrival n m The required time can be calculated by the following formula.

[0156] [Mathematical formula 2]

[0157]

[0158] In addition, the average speed may be predetermined for each moving body m. In addition, another value may be determined by the route and the zone. In addition, traffic conditions or service conditions may be taken into consideration. Specifically, traffic conditions correspond to red lights or traffic jams. Service conditions correspond to loading, unloading, or refueling. For example, the average speed may be adjusted according to traffic conditions or service conditions. Alternatively, additional time may be added according to traffic conditions or service conditions.

[0159] By using the above formula, add the time required to reach the destination in order from the current position to the farthest point, and find the farthest destination w that does not exceed Δt.x m At this time, just change the duplicate partition ID from [b0 m ,···,b x m ], we can obtain the estimated result of the partition where the moving body m may exist from the current moment to the required time Δt.

[0160] The judgment unit 108 refers to all the estimated partitions Figure 5 The risk map can be used to determine whether there is a risk or no risk for each of the one or more second moving bodies (S304).

[0161] As a specific example of the above processing, the possible partitions for one or more second moving bodies from the current time to the required time Δt are Figure 5 The risk-free partition situation shown is illustrated.

[0162] In S301 , the determination unit 108 obtains the required time using Δt.

[0163] In S302, the determination unit 108 uses p m To obtain the position included in the moving body information of each moving body m included in the set M of more than one second moving body ID, [w1 m ,···,w N m ] to obtain the planned driving path.

[0164] Hereinafter, a case will be described in which, as a result of the above calculation, the determination unit 108 estimates in S303 that the sections in which one or more second moving objects are likely to exist from the current time to the concentration time Δt are [1, 2, 5, 9, 15, 17].

[0165] More specifically, when the set of mobile IDs of mobiles under remote monitoring among multiple mobiles M1 to Mm is {1, 2, 3}, the partitions of the route of the mobile M1 are [1, 2, 10, 16], and the farthest destination that takes no more than Δt is partition 2, the partition where the mobile M1 is likely to be located is [1, 2]. When the route of the mobile M2 is [5, 9, 11, 10, 12], and the farthest destination that takes no more than Δt is partition 9, the partition where the mobile M2 is likely to be located is [5, 9]. When the route of the mobile M3 is [15, 17, 18, 19], and the farthest destination that takes no more than Δt is partition 17, the partition where the mobile M3 is likely to be located is [15, 17]. The estimation result at this time is [1, 2, 5, 9, 15, 17].

[0166] In S304, the determination unit 108 refers to Figure 5 The risk map shows that the moving bodies M1 to M3 are all risk-free at [1, 2, 5, 9, 15, 17].

[0167] (About operation request in remote operation)

[0168] An example will be described in which an operation request is received from another mobile object before the remote operation of one mobile object is completed.

[0169] As described above, first, after confirming that no risky operation requests will be issued from other mobile objects under remote monitoring during the remote operation of a mobile object, it is determined that the mobile object can be remotely operated, and the operation entrustment is distributed to the operator. Therefore, the operation request that appears during the remote operation is a risk-free operation request. In this case, since there is no risk, after the current remote operation is completed, the next remote operation can be carried out by implementing Fig.11 The process can be handled by following the flowchart.

[0170] In addition, it is considered that a plurality of such operation requests are made until the remote operation is completed. In this case, the mobile body with a greater degree of influence in providing the service may be given priority. For example, in the case of food delivery service by an automatic delivery robot, the movement for loading or delivery takes precedence over the movement for returning to the base. In this case, when the service system X sends the vehicle allocation instruction to the server device S, the priority is also sent, and the server device S also considers the priority when determining the first mobile body in S103, so that the priority can be realized.

[0171] Next, in the case where there is an operation request from another mobile body during the remote operation, it is considered that the time actually required for the remote operation may be longer than the required time, the estimated partition where the other mobile body may exist may be different from the partition where the other mobile body actually exists, and the information used to determine whether there is a risk or not may be different from the actual situation. In such a case, as in the case where it is determined to be impossible in S107, the remote monitoring of the first mobile body is handed over to another operator, the remote monitoring of one or more second mobile bodies is handed over to another operator, or the first mobile body is instructed to perform MRM control on the grounds that the operation request cannot be responded to.

[0172] First, in the handover of the first moving body, for the operator who is not remotely monitoring the first moving body at this moment, by adding the time until the state of being able to remotely monitor is added to the required time, and executing the process of S106, it is possible to determine whether other operators can perform remote operation in the same way, and then the handover can be made to the operator who is determined to be able to perform remote operation. On the other hand, in the handover of one or more second moving bodies, it is possible to hand over to an operator who is not performing remote operation. When other operators are performing remote operation, the driving control information distribution unit 112 instructs the first moving body to perform MRM control via the communication unit 101 on the grounds that the operation request cannot be responded to. Alternatively, the judgment control unit 214 of the first moving body may automatically decide to perform MRM control after a certain time has passed since the operation request.

[0173] The information processing method involved in the embodiment is an information processing method in an information processing device (server device S). In the information processing method, remote operation information related to remote operation on a first mobile body is obtained, and the first mobile body is a mobile body that needs remote operation among multiple mobile bodies M1 to Mm, each of which can move autonomously and can be remotely operated (S104). In addition, mobile body information related to one or more second mobile bodies among the multiple mobile bodies M1 to Mm is obtained, and the one or more second mobile bodies are mobile bodies other than the first mobile body, and are mobile bodies that are remotely monitored by a supervisor (an operator responsible for remote monitoring of the first mobile body among multiple operators O1 to On) who is responsible for remote monitoring of the first mobile body (S101). In addition, based on the remote operation information and the mobile body information, it is determined whether to allow the supervisor to remotely operate the first mobile body (S106). In addition, notification is made based on the result of the determination of whether to allow the supervisor to remotely operate the first mobile body (S108).

[0174] According to this, since it is possible to determine whether to allow the supervisor to remotely operate the first moving object based on the remote operation information and the moving object information, the plurality of moving objects M1 to Mm can be remotely monitored efficiently.

[0175] Furthermore, in the information processing method according to the embodiment, the first mobile body outputs request information for requesting remote operation of the first mobile body, the request information output by the first mobile body is obtained (S101), and the first mobile body is identified based on the request information (S201).

[0176] According to this, since the first moving object can be easily identified, the plurality of moving objects M1 to Mm can be more efficiently remotely monitored.

[0177] Furthermore, in the information processing method involved in the embodiment, the remote operation information includes the time required to remotely operate the first mobile body, and the required time includes the time required to determine the timing of remotely operating the first mobile body and the time required to remotely operate the first mobile body.

[0178] According to this, since it is possible to determine whether to allow the supervisor to remotely operate the first moving object based on the required time, the plurality of moving objects M1 to Mm can be remotely monitored more efficiently.

[0179] Furthermore, in the information processing method according to the embodiment, the required time is predetermined by a device that remotely controls the first moving object.

[0180] According to this, whether to allow the supervisor to remotely operate the first moving object can be determined based on the time required by the device that remotely operates the first moving object, so the plurality of moving objects M1 to Mm can be remotely monitored more efficiently.

[0181] Furthermore, in the information processing method according to the embodiment, the required time is predetermined for each of the plurality of sections 1 to 21 in the area divided into the plurality of sections 1 to 21 within the range in which the first moving body moves.

[0182] According to this, since it is possible to determine whether to allow the supervisor to remotely operate the first moving object based on the required time determined for each of the plurality of sections 1 to 21, the plurality of moving objects M1 to Mm can be remotely monitored more efficiently.

[0183] Alternatively, in the information processing method according to an embodiment, the required time may be predetermined based on at least one of a date when the first mobile body is remotely operated, a time period when the first mobile body is remotely operated, and weather when the first mobile body is remotely operated.

[0184] Accordingly, since it is possible to determine whether to allow the monitor to remotely operate the first moving body based on the required time determined according to at least one of the date when the first moving body is remotely operated, the time period when the first moving body is remotely operated, and the weather when the first moving body is remotely operated, multiple moving bodies M1 to Mm can be remotely monitored more efficiently.

[0185] Furthermore, in the information processing method involved in the implementation mode, it is determined based on the remote operation information and the moving body information whether there is a risk when not remotely operating each of the one or more second moving bodies (S106). If it is determined that there is no risk, it is determined to allow the monitor to remotely operate the first moving body (S107).

[0186] According to this, since the supervisor can remotely operate the first moving object when it is determined that there is no risk, the plurality of moving objects M1 to Mm can be remotely monitored more efficiently.

[0187] Furthermore, in the information processing method involved in the implementation mode, whether each of the multiple partitions 1 to 21 is a risky partition is pre-set according to each of the multiple partitions 1 to 21 in the area divided by the multiple partitions 1 to 21 in which each of the more than one second mobile bodies moves within the range.

[0188] According to this, since it is possible to easily determine whether there is a risk, the plurality of mobile bodies M1 to Mm can be remotely monitored more efficiently.

[0189] Alternatively, in the information processing method involved in the implementation mode, whether a zone is risky is predetermined based on at least one of a date when the first mobile body is remotely operated, a time period when the first mobile body is remotely operated, and the weather when the first mobile body is remotely operated.

[0190] According to this, since it is possible to determine whether there is a risk with higher accuracy, the multiple mobile bodies M1 to Mm can be remotely monitored more efficiently.

[0191] Furthermore, in the information processing method according to the embodiment, the remote operation information includes a required time required for remotely operating the first moving body. The required time includes a time required for determining a timing for remotely operating the first moving body and a time required for remotely operating the first moving body. Furthermore, when at least one of the at least one second moving body is located in a risky partition among the plurality of partitions 1 to 21 during the required time, it is determined that there is a risk (S106).

[0192] According to this, since it is easier to determine whether there is a risk, the plurality of mobile bodies M1 to Mm can be remotely monitored more efficiently.

[0193] Furthermore, in the information processing method according to the embodiment, when it is determined that the supervisor is to remotely operate the first moving object, a notification is made to indicate that the supervisor is to remotely operate the first moving object ( S108 ).

[0194] According to this, since the supervisor can efficiently remotely operate the first moving object, the plurality of moving objects M1 to Mm can be more efficiently remotely monitored.

[0195] Furthermore, in the information processing method according to the embodiment, when it is determined that the supervisor is not allowed to remotely operate the first moving object, a notification is issued to indicate that the first moving object is to be stopped.

[0196] According to this, since the supervisor can easily recognize the stop of the first moving body, the plurality of moving bodies M1 to Mm can be more efficiently remotely supervised.

[0197] Furthermore, the information processing device according to the embodiment includes a remote operation information acquisition unit 107, a mobile body information acquisition unit 102, a judgment unit 108, and a notification unit (operation entrustment distribution unit 110). The remote operation information acquisition unit 107 acquires remote operation information related to remote operation on a first mobile body, which is a mobile body that needs to be remotely operated among a plurality of mobile bodies M1 to Mm, each of which can move autonomously and can be remotely operated. The mobile body information acquisition unit 102 acquires mobile body information related to one or more second mobile bodies among the plurality of mobile bodies M1 to Mm, which are mobile bodies other than the first mobile body and are mobile bodies that are remotely monitored by a supervisor who is responsible for remotely monitoring the first mobile body. The judgment unit 108 judges whether to allow the supervisor to remotely operate the first mobile body based on the remote operation information and the mobile body information. The notification unit (operation entrustment distribution unit 110) makes a notification based on the judgment result of whether to allow the supervisor to remotely operate the first mobile body.

[0198] According to this, the same effects as those of the above-mentioned information processing method can be achieved.

[0199] Furthermore, the program according to the embodiment is a program for causing a computer to execute the above-mentioned information processing method.

[0200] According to this, the same effects as those of the above-mentioned information processing method can be achieved.

[0201] (Other Embodiments, etc.)

[0202] Although the information processing method involved in one or more methods is described above based on the implementation methods, the present disclosure is not limited to these implementation methods. Without departing from the scope of the present disclosure, various modifications that can be thought of by those skilled in the art to the implementation methods may also be included in the scope of the present disclosure.

[0203] In addition, in the above-mentioned embodiments, each component can be formed by dedicated hardware or implemented by executing a software program suitable for each component. Each component can also be implemented by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that implements the device of the above-mentioned embodiment is a program that enables a computer to execute Figures 11 to 13 The flowchart shown includes the procedures of each step.

[0204] Furthermore, the following cases are also included in the present disclosure.

[0205] (1) Specifically, the at least one device mentioned above is a computer system composed of a microprocessor, ROM, RAM, hard disk device, display device, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk device. The microprocessor operates according to the computer program, so that the at least one device mentioned above realizes its function. Here, the computer program is a combination of multiple command codes that issue instructions to the computer in order to realize the specified function.

[0206] (2) A part or all of the components constituting at least one of the above-mentioned devices may be constituted by a system LSI (Large Scale Integration). A system LSI is a multifunctional LSI in which multiple components are integrated and manufactured on a single chip, and specifically is a computer system constituted by a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The microprocessor operates according to the computer program, so that the system LSI realizes its functions.

[0207] (3) A part or all of the components constituting at least one of the above-mentioned devices may be constituted by an IC card or a single module that can be attached to or detached from the device. The IC card or module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunctional LSI. The IC card or module realizes its functions by operating the microprocessor according to the computer program. The IC card or module may also be tamper-resistant.

[0208] (4) The present disclosure can also be utilized as the method described above. The present disclosure can be implemented by a computer program that executes the method described above on a computer, or can be implemented by a digital signal composed of the computer program.

[0209] Furthermore, the present disclosure may also be implemented as a computer-readable recording medium that records a computer program or a digital signal. Examples of computer-readable recording media include a floppy disk, a hard disk, a CD (Compact Disc)-ROM, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, and the like. Furthermore, the present disclosure may also be implemented as a computer program or a digital signal recorded on these recording media.

[0210] Furthermore, the present disclosure may transmit computer programs or digital signals via electronic communication lines, wireless or wired communication lines, networks represented by the Internet, data broadcasting, and the like.

[0211] Furthermore, by recording the computer program or digital signal on a recording medium and transmitting it, or by transmitting the program or digital signal via a network or the like, the corresponding method or the like can be executed by another independent computer system.

[0212] (Remark)

[0213] Based on the description of the above embodiments and the like, the following techniques are disclosed.

[0214] (Technique 1) An information processing method is an information processing method in an information processing device, in which remote operation information related to remote operation on a first moving body is obtained, the first moving body is a moving body that needs to be remotely operated among a plurality of moving bodies each of which can move autonomously and can be remotely operated, moving body information related to one or more second moving bodies among the plurality of moving bodies is obtained, the one or more second moving bodies are moving bodies other than the first moving body and are moving bodies that are remotely monitored by a monitor who is responsible for remotely monitoring the first moving body, based on the remote operation information and the moving body information, it is determined whether the monitor is allowed to remotely operate the first moving body, and notification is made based on the result of the determination of whether the monitor is allowed to remotely operate the first moving body.

[0215] (Technique 2) In the information processing method as described in Technique 1, the first mobile body outputs request information for requesting remote operation of the first mobile body, the request information output by the first mobile body is obtained, and the first mobile body is determined based on the request information.

[0216] (Technique 3) In the information processing method as described in Technique 1 or 2, the remote operation information includes the time required for remotely operating the first mobile body, and the required time includes the time required to determine the timing of remotely operating the first mobile body and the time required to remotely operate the first mobile body.

[0217] (Technique 4) In the information processing method according to Technique 3, the required time is predetermined by a device that remotely controls the first moving object.

[0218] (Technique 5) In the information processing method according to Technique 3 or 4, the required time is predetermined for each of the plurality of partitions in the area in which the first moving body moves and is divided into the plurality of partitions.

[0219] (Technique 6) In the information processing method as described in any one of Techniques 3 to 5, the required time is predetermined based on at least one of the date when the first mobile body is remotely operated, the time period when the first mobile body is remotely operated, and the weather when the first mobile body is remotely operated.

[0220] (Technique 7) An information processing method as described in Technique 1 or 2, which determines whether there is a risk when not remotely operating each of the one or more second mobile bodies based on the remote operation information and the mobile body information, and when it is determined that there is no such risk, it is determined to allow the monitor to remotely operate the first mobile body.

[0221] (Technology 8) As the information processing method described in Technology 7, whether each of the multiple partitions is a partition with the risk is pre-set according to each of the multiple partitions in the area divided by multiple partitions in which each of the one or more second mobile bodies moves within the range.

[0222] (Technique 9) In the information processing method as described in Technique 8, whether a zone is at risk is predetermined based on at least one of the date when the first mobile body is remotely operated, the time period when the first mobile body is remotely operated, and the weather when the first mobile body is remotely operated.

[0223] (Technique 10) In the information processing method as described in Technique 8 or 9, the remote operation information includes the required time required for remotely operating the first mobile body, and the required time includes the required time required for determining the timing of remotely operating the first mobile body and the required time required for remotely operating the first mobile body. When at least one of the one or more second mobile bodies is located in a partition having the risk among the multiple partitions during the required time, it is judged that the risk exists.

[0224] (Technique 11) In the information processing method according to any one of Techniques 1 to 10, when it is determined that the monitor is to remotely operate the first moving object, the notification indicating that the monitor is to remotely operate the first moving object is performed.

[0225] (Technique 12) In the information processing method according to any one of Techniques 1 to 11, when it is determined that the monitor is not to remotely operate the first moving object, the notification indicating that the first moving object is to be stopped is performed.

[0226] (Technology 13) An information processing device comprises: a remote operation information obtaining unit, which obtains remote operation information related to remote operation on a first mobile body, wherein the first mobile body is a mobile body that needs to be remotely operated among a plurality of mobile bodies each of which can move autonomously and can be remotely operated; a mobile body information obtaining unit, which obtains mobile body information related to one or more second mobile bodies among the plurality of mobile bodies, wherein the one or more second mobile bodies are mobile bodies other than the first mobile body and are mobile bodies that are remotely monitored by a monitor who is responsible for remotely monitoring the first mobile body; a judgment unit, which judges whether to allow the monitor to remotely operate the first mobile body based on the remote operation information and the mobile body information; and a notification unit, which makes a notification based on the judgment result of whether to allow the monitor to remotely operate the first mobile body.

[0227] (Technique 14) A program for causing a computer to execute the information processing method described in any one of Techniques 1 to 12.

[0228] Industrial Applicability

[0229] The present disclosure can be utilized in a method of processing information for remotely monitoring a mobile object, and the like.

[0230] Explanation of symbols

[0231] 1 Remote monitoring system

[0232] 21 ECU

[0233] 22 Communication Module

[0234] 101, 301 Ministry of Communications

[0235] 102, 306 Mobile body information acquisition unit

[0236] 103 Operator Information Acquisition Department

[0237] 104 Risk Map Storage Department

[0238] 105 Remote operation information storage unit

[0239] 106 First moving object determination unit

[0240] 107 Remote Operation Information Acquisition Department

[0241] 108 Judgment Department

[0242] 109 Operation permission decision section

[0243] 110 Operation Entrustment Distribution Department

[0244] 111 Mobile Information Distribution Department

[0245] 112 Driving Control Information Distribution Department

[0246] 113 Vehicle Allocation Instruction Distribution Department

[0247] 201 Camera

[0248] 202 Laser Sensor

[0249] 203 GNSS

[0250] 211 Acquisition Department

[0251] 212 Inspection Department

[0252] 213 Self-position estimation unit

[0253] 214 Judgment Control Unit

[0254] 215 Operation Request Decision Unit

[0255] 216 Mobile body information output unit

[0256] 302 Input receiving unit

[0257] 303 Screen Generation Department

[0258] 304 UI ​​Control Unit

[0259] 305 Operator information output unit

[0260] 307 Operation Entrustment Inspection Department

[0261] 311 Steering wheel controller

[0262] 312 Button

[0263] 313 Keyboard

[0264] 314 Mouse

[0265] 315 Monitor

[0266] 316 Speakers

[0267] D1, Dn Operator Terminal Devices

[0268] M1, Mm mobile body

[0269] N Network

[0270] O1, On Operator

[0271] S Server Device

[0272] X server system

Claims

1. An information processing method, which is an information processing method in an information processing device, In the information processing method, obtaining remote operation information related to remote operation on a first moving object, the first moving object being a moving object requiring remote operation among a plurality of moving objects each capable of autonomous movement and capable of remote operation, obtaining mobile body information related to one or more second mobile bodies among the plurality of mobile bodies, wherein the one or more second mobile bodies are mobile bodies other than the first mobile body and are mobile bodies that are remotely monitored by a monitor who is responsible for remotely monitoring the first mobile body, According to the remote operation information and the moving body information, it is determined whether to allow the monitor to remotely operate the first moving body, Notification is performed based on a result of determination as to whether the supervisor is to remotely operate the first moving object.

2. The information processing method according to claim 1, the first mobile body outputting request information for requesting a remote operation on the first mobile body, obtaining the request information output by the first mobile body, The first moving object is determined according to the request information.

3. The information processing method according to claim 1, The remote operation information includes a required time required to remotely operate the first moving object. The required time includes the required time required to determine the timing of remotely operating the first moving object and the required time required to remotely operate the first moving object.

4. The information processing method according to claim 3, The required time is predetermined by a device that remotely controls the first moving object.

5. The information processing method according to claim 3 or 4, The required time is predetermined for each of the plurality of sections in the area in which the first moving body moves and is divided into the plurality of sections.

6. The information processing method according to claim 3 or 4, The required time is predetermined based on at least one of a date when the first moving object is remotely operated, a time period when the first moving object is remotely operated, and weather when the first moving object is remotely operated.

7. The information processing method according to claim 1, Based on the remote operation information and the moving body information, it is determined whether there is a risk when not remotely operating each of the one or more second moving bodies. If it is determined that there is no risk, it is determined to allow the monitor to remotely operate the first moving body.

8. The information processing method according to claim 7, Whether each of the plurality of sections is the risk section is set in advance for each of the plurality of sections in the area divided into the plurality of sections and in which each of the one or more second mobile bodies moves within the range.

9. The information processing method according to claim 8, Whether or not the zone is the risk zone is predetermined based on at least one of a date when the first moving body is remotely operated, a time period when the first moving body is remotely operated, and weather when the first moving body is remotely operated.

10. The information processing method according to claim 8 or 9, The remote operation information includes a required time required to remotely operate the first moving object. The required time includes a required time required to determine a timing for remotely operating the first moving object and a required time required to remotely operate the first moving object. When at least one of the one or more second moving objects is located in a zone having the risk among the plurality of zones during the required time, it is determined that the risk exists.

11. The information processing method according to any one of claims 1 to 4, When it is determined that the supervisor is to remotely operate the first moving object, the notification indicating that the supervisor is to remotely operate the first moving object is made.

12. The information processing method according to any one of claims 1 to 4, When it is determined that the supervisor is not to remotely operate the first moving object, the notification indicating that the first moving object is to be stopped is performed.

13. An information processing device comprising: a remote operation information obtaining unit that obtains remote operation information related to remote operation on a first moving object, the first moving object being a moving object that needs to be remotely operated among a plurality of moving objects that are each capable of autonomous movement and can be remotely operated; a moving body information obtaining unit for obtaining moving body information related to one or more second moving bodies among the plurality of moving bodies, wherein the one or more second moving bodies are moving bodies other than the first moving body and are moving bodies that are remotely monitored by a monitor who is responsible for remotely monitoring the first moving body; a determination unit, which determines whether to allow the monitor to remotely operate the first moving object according to the remote operation information and the moving object information; as well as The notification unit performs notification based on a result of determination as to whether to allow the supervisor to remotely operate the first moving object.

14. A program for causing a computer to execute the information processing method according to any one of claims 1 to 4.

Images