Information processing method, information processing terminal, and information processing system

Abstract

An information processing method to be executed by a computer, the method comprising, upon receiving information about an event that requires remote operation of a mobile object, identifying a task of remote operation corresponding to the event (S17), and determining, in accordance with the identified task, a travel restriction on the mobile object under remote operation (S21), wherein the determined travel restriction is imposed when the mobile object executes travel control corresponding to the task.

Description

[Technical field]

[0001] The present disclosure relates to an information processing method, an information processing terminal, and an information processing system. [Background technology]

[0002] In recent years, remote operation systems have been studied in which a remote operator in a remote location indirectly operates an autonomously-driving vehicle as needed using wireless communication such as a wireless LAN (Local Area Network) or a mobile phone line. For example, when the vehicle reaches a situation where it cannot drive autonomously, the remote operator controls the vehicle's driving from a remote location by transmitting a control signal related to the vehicle's driving to the vehicle.

[0003] When a remote operator remotely operates a vehicle, it is desirable to drive the vehicle safely. Therefore, Patent Document 1 proposes limiting the speed of the vehicle when it is remotely operated. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-126193 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology of Patent Document 1 may reduce the efficiency of remote control by a remote operator. For example, the above technology limits the speed of a moving object in a remote control. On the other hand, the remote control required for an event that has occurred differs depending on the event that has occurred. Therefore, the speed limit may not be suitable for the remote control. As a result, there is a risk of reducing the efficiency of the remote control.

[0006] Therefore, the present disclosure provides an information processing method, an information processing terminal, and an information processing system that can suppress a decrease in efficiency of remote control while ensuring the safety of a moving object being remotely controlled. [Means for solving the problem]

[0007] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, which, when receiving information of an event requiring remote operation of a moving object, identifies a remote operation task corresponding to the event, and determines a driving restriction for the moving object when remotely operated according to the identified task, and the determined driving restriction is imposed when the moving object executes driving control according to the task.

[0008] An information processing terminal according to one aspect of the present disclosure is an information processing terminal mounted on a moving body, and includes a determination unit that determines whether an event has occurred that requires remote operation of the moving body, a transmission unit that transmits information about the event if the event has occurred, a receiving unit that receives driving restrictions for the moving body in remote operation and remote operation instructions based on the transmitted event information, and a control unit that causes the moving body to execute driving control in accordance with the remote operation instructions under the received driving restrictions.

[0009] An information processing system according to one aspect of the present disclosure includes a determination unit that determines whether an event has occurred that requires remote operation of a moving object, an identification unit that, when it is determined that the event has occurred, identifies a remote operation task corresponding to the event, a determination unit that determines driving restrictions for the moving object when remotely operated in accordance with the identified task, and a control unit that imposes the determined driving restrictions when the moving object executes driving control in accordance with the identified task. Effect of the Invention

[0010] According to an information processing method and the like according to an aspect of the present disclosure, it is possible to suppress a decrease in efficiency of remote control while ensuring the safety of a moving object being remotely controlled. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a functional configuration of a vehicle control system according to an embodiment. [Diagram 2] FIG. 2 is a sequence diagram showing the operation of the vehicle control system according to the embodiment. [Diagram 3] FIG. 3 is a flowchart showing the operation of determining whether or not switching is necessary shown in FIG. [Figure 4] FIG. 4 is a diagram illustrating various types of information acquired by the edge system according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a switching determination list according to the embodiment. [Figure 6] FIG. 6 is a flowchart showing the operation of determining the operation event shown in FIG. [Figure 7] FIG. 7 is a diagram illustrating an example of an event list according to the embodiment. [Figure 8] FIG. 8 is a diagram showing an example of the display content of the display unit according to the embodiment. [Figure 9] FIG. 9 is a diagram showing another example of the display content of the display unit according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of remote operation status history information according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a current situation of a vehicle according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of remote operation history information according to the embodiment. [Figure 13] FIG. 13 is a diagram for explaining calculation of the limit value according to the embodiment. [Figure 14] FIG. 14 is a flowchart showing the operation of applying the limit values ​​shown in FIG. [Figure 15] FIG. 15 is a first diagram for explaining the problem in the present disclosure. [Figure 16] FIG. 16 is a second diagram for explaining the problem in the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] (Background to this disclosure) Prior to describing the embodiments of the present disclosure, the background to the basis of the present disclosure will be described with reference to Figs. 15 and 16. Fig. 15 is a first diagram for describing the problem in the present disclosure. Fig. 16 is a second diagram for describing the problem in the present disclosure. In Figs. 15 and 16, it is assumed that the vehicle 100 has been traveling in an autonomous driving mode, but has entered a state in which it is no longer able to travel in autonomous driving mode (for example, has become unable to travel), and has stopped on a road 400.

[0013] FIG. 15 shows a case where the vehicle 100 travels while avoiding road construction. FIG. 16 shows a case where the vehicle pulls over (temporarily pulls over) to allow an emergency vehicle to pass. In both FIG. 15 and FIG. 16, the current position of the vehicle 100 is the remote operation start position, and the vehicle is remotely operated to a target position. Note that avoiding road construction by remote operation, and pulling over by remote operation to allow an emergency vehicle to pass are examples of operation events (tasks) during remote operation. In other words, FIG. 15 and FIG. 16 show operations and the like required for different operation events.

[0014] In Fig. 15, the remote control is performed with the vehicle speed set to "low", the steering angle set to "large", and the acceleration set to "small". This is because the remote operator drives the vehicle 100 slowly while making large steering turns in response to a driving route with sharp curves.

[0015] In Fig. 16, remote operation is performed with a vehicle speed of "medium", a steering angle of "small", and an acceleration of "medium". This is because the remote operator drives the vehicle 100 relatively fast while making small steering turns according to the driving route that has a gentle curve.

[0016] In this way, when driving by remote control, the content of the required operation often differs depending on the operation event for remote control. It is preferable that the driving restrictions during remote control are set so as to increase safety and at the same time improve driving efficiency.

[0017] However, in the technology of Patent Document 1, the driving restrictions are set uniformly regardless of the remote control operation event, which may result in a decrease in efficiency during remote control. In addition, the driving restrictions are not suitable for remote control, which may result in a decrease in safety.

[0018] Therefore, the inventors of the present application conducted extensive research into information processing methods etc. that can reduce the inhibition of remote control efficiency while ensuring the safety of the remotely controlled mobile object, and have devised the information processing methods etc. described below.

[0019] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, which, when receiving information of an event requiring remote operation of a moving object, identifies a remote operation task corresponding to the event, and determines a driving restriction for the moving object when remotely operated according to the identified task, and the determined driving restriction is imposed when the moving object executes driving control according to the task.

[0020] This allows the driving restriction to be determined for each remote control task. That is, the driving restriction suitable for the remote control task can be determined. Therefore, it is possible to suppress the imposition of unnecessary driving restrictions during remote control, and therefore it is possible to suppress the decrease in efficiency of remote control while ensuring the safety of the remotely controlled moving body. In other words, since the remote control task may differ depending on the state of the moving body, it can be said that the driving restriction according to the state of the moving body can be imposed. In addition, since the driving restriction is optimized for the task, the safety of the moving body can be improved depending on the content of the task or the situation in which the remote control is performed. For example, when pulling over to the shoulder of a highway, the risk of being rear-ended by another vehicle can be reduced by relaxing the driving restriction.

[0021] Also, for example, an execution environment of the task may be acquired, and the travel restriction may be determined in accordance with the task and the execution environment.

[0022] In this way, the driving limit is determined by further using the task execution environment, and therefore it is possible to determine a driving limit that is more suitable for the remote operation task.

[0023] Furthermore, for example, the execution environment may include at least one of attributes, capabilities, and operation history of a remote operator who executes the task.

[0024] In this way, the driving limit is determined by further using at least one of the attributes, capabilities, and operation history of the remote operator, so that the driving limit can be determined according to the remote operator who executes the remote operation task.

[0025] Also, for example, the execution environment may include at least one of attributes, performance, and specifications of the mobile object that is remotely controlled by executing the task.

[0026] In this way, the driving restriction is determined by further using at least one of the attributes, performance, and specifications of the remotely controlled moving object, and therefore it is possible to determine a driving restriction appropriate for the remotely controlled moving object.

[0027] Also, for example, the execution environment may include at least one of the weather, road conditions, and traffic environment of the location where the moving object to be remotely controlled by executing the task is located.

[0028] In this way, the driving restriction is determined by further using at least one of the weather, road surface, and traffic environment of the current location of the remotely controlled moving object, and thus it is possible to determine the driving restriction according to the environment of the current location of the remotely controlled moving object (for example, the location where the moving object is unable to travel).

[0029] Also, for example, the travel restrictions may be determined using a model generated based on the past tasks and the execution environment.

[0030] This makes it possible to determine the driving restriction without accumulating data for determining the driving restriction, such as remote operation history information that indicates the history of past remote operations.

[0031] Also, for example, the task identification may involve identifying candidate tasks, which are tasks that are estimated based on information about the event, presenting the identified candidate tasks to a remote operator, and identifying the tasks in response to input from the remote operator based on the presented candidate tasks.

[0032] This makes it possible to present candidate tasks to the remote operator. Also, by having the remote operator perform input based on the presented candidate tasks, it is possible to make the remote operator aware of the purpose of the remote operation. This is expected to reduce redundant operations by the remote operator during remote operation. Also, by presenting candidate tasks, it is possible to reduce the time or effort required for the remote operator to select a task.

[0033] Also, for example, the candidate tasks to be presented may be identified using a model generated based on information of the event at a past time and input from the remote operator based on the candidate tasks at the past time.

[0034] This makes it possible to determine candidate tasks to be presented without preparing data for determining candidate tasks to be presented, such as an event list.

[0035] Also, for example, the determined driving restriction may be imposed when the moving object executes the driving control in accordance with the remote operation instruction corresponding to the identified task.

[0036] This makes it possible to impose travel restrictions on remote operation instructions from a remote operator who executes a task, thereby suppressing a decrease in task execution efficiency.

[0037] Also, for example, the driving limitations may include limitations on at least one of speed, acceleration, braking, and steering.

[0038] This makes it possible to impose a limit on at least one of speed, acceleration, braking, and steering in response to a remote control instruction from a remote operator.

[0039] Furthermore, for example, the determined driving restrictions may be transmitted to a terminal mounted on the moving body, and driving control in accordance with the remote control instructions may be executed in the moving body under the transmitted driving restrictions.

[0040] This makes it possible to impose driving restrictions on the driving control executed in the moving body in accordance with instructions from the remote control when driving control is determined for the moving body.

[0041] Furthermore, for example, it is also possible to determine a driving control in accordance with the remote control instructions under the driving restriction, transmit the determined driving control to a terminal mounted on the moving body, and execute the transmitted driving control on the moving body.

[0042] As a result, when driving control is determined on the device side that remotely controls the moving object, the driving control to be executed by the moving object can be determined taking into account the driving restrictions.

[0043] In addition, an information processing terminal according to one aspect of the present disclosure is an information processing terminal mounted on a moving body, and includes a determination unit that determines whether an event has occurred that requires remote operation of the moving body, a transmission unit that transmits information about the event if the event has occurred, a receiving unit that receives driving restrictions for the moving body in remote operation and remote operation instructions based on the transmitted event information, and a control unit that causes the moving body to execute driving control in accordance with the remote operation instructions under the received driving restrictions.

[0044] This allows the information processing terminal to control the moving object according to the remote control instruction under the driving restriction according to the event requiring the remote control of the moving object. Therefore, the information processing terminal is prevented from imposing unnecessary driving restrictions on the remote control instruction, and therefore it is possible to prevent a decrease in the efficiency of the remote control while ensuring the safety of the moving object being remotely controlled.

[0045] In addition, an information processing system according to one aspect of the present disclosure includes a determination unit that determines whether an event requiring remote operation of a moving object has occurred, an identification unit that, when it is determined that the event has occurred, identifies a remote operation task corresponding to the event, a determination unit that determines driving restrictions for the moving object when remotely operated in accordance with the identified task, and a control unit that imposes the determined driving restrictions when the moving object executes driving control in accordance with the identified task.

[0046] This provides the same effects as the above-mentioned information processing method.

[0047] Furthermore, these comprehensive or specific aspects may be realized in a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized in any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0048] Hereinafter, a specific example of an information processing method according to one aspect of the present disclosure will be described with reference to the drawings. Each of the embodiments shown here shows one specific example of the present disclosure. Therefore, the numerical values, numerical values, components, steps, order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims will be described as optional components.

[0049] In addition, each figure is a schematic diagram and is not necessarily illustrated precisely. Therefore, for example, the scales in each figure do not necessarily match. In addition, in each figure, substantially the same configurations are given the same reference numerals, and duplicated explanations are omitted or simplified.

[0050] In addition, in this specification, terms indicating relationships between elements, such as "same" and "match," as well as numerical values ​​and numerical ranges, are not expressions that only express a strict meaning, but are expressions that include a substantially equivalent range, for example, a difference of about a few percent.

[0051] (Embodiment) Hereinafter, an information processing system according to the present embodiment will be described with reference to FIGS.

[0052] [1. Information processing system configuration] First, the functional configuration of a vehicle control system 10 according to the present embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the functional configuration of a vehicle control system 10 according to the present embodiment.

[0053] As shown in Fig. 1, the vehicle control system 10 includes a vehicle 100, a server 200, and a second remote control system 300. The vehicle 100 and the server 200, and the server 200 and the second remote control system 300 are communicatively connected to each other. The vehicle control system 10 is a system for a remote operator at a remote location to remotely monitor and remotely operate the traveling of the vehicle 100 as necessary. The vehicle control system 10 is an example of an information processing system for controlling the traveling of the vehicle 100.

[0054] The vehicle 100 is an example of a moving body in which a person rides, and is remotely monitored or remotely operated by a remote operator as necessary. The vehicle 100 is an autonomous vehicle (autonomous vehicle) capable of switching between autonomous driving and remotely operated driving. In other words, the vehicle 100 is a vehicle that can selectively switch between an autonomous driving mode in which the vehicle runs autonomously and a remotely operated mode in which the vehicle runs by remote control. Note that the autonomous vehicle is not particularly limited as long as a person can ride in the autonomous vehicle and the autonomous vehicle is capable of autonomous driving, and may be an autonomous bus, an autonomous taxi, an autonomous private car, an autonomous truck, an autonomous construction vehicle (e.g., a dump truck), or the like.

[0055] The vehicle 100 has an autonomous driving system 110, a first remote control system 120, and an edge system 130. In this specification, the term "vehicle" refers to a vehicle in which the autonomous driving system 110 and the first remote control system 120 are installed.

[0056] The autonomous driving system 110 is a control system for autonomous driving that is installed in the vehicle 100. The autonomous driving system 110 controls the traveling of the vehicle 100 when the vehicle 100 travels in an autonomous driving mode. The autonomous driving system 110 is, for example, a control system that can realize autonomous driving at an autonomous driving level equal to or higher than a predetermined level. The predetermined level is, for example, Level 3 in the level definition of the Society of Automotive Engineers (SAE), but is not limited to this.

[0057] The automatic driving system 110 acquires vehicle information of the vehicle 100 based on sensing information of various sensors (not shown) mounted on the vehicle 100. The automatic driving system 110 acquires vehicle information from the vehicle 100, for example, in an automatic driving mode. The vehicle information includes at least information related to the traveling of the vehicle 100. In the present embodiment, the vehicle information includes information related to the traveling of the vehicle 100 and information related to the surroundings of the vehicle 100. The information related to the traveling of the vehicle 100 includes, for example, at least one of the current speed, steering angle, and acceleration of the vehicle 100. In addition, the information related to the surroundings of the vehicle 100 includes, for example, at least one of information related to an obstacle (obstacle information) and an image. The information related to an obstacle is information indicating the position and speed of an obstacle around the vehicle 100. The various sensors include, but are not limited to, a speed sensor, a steering angle sensor, a LIDAR (Light Detection and Ranging), a radar (for example, a millimeter wave radar), an ultrasonic sensor, an imaging device, and the like.

[0058] The autonomous driving system 110 has a vehicle control unit 111 and a notification unit 112.

[0059] The vehicle control unit 111 controls the driving of the vehicle 100 based on a driving plan for automatic driving. The vehicle control unit 111 controls the speed, steering angle, and the like of the vehicle 100 based on the driving plan. The driving plan includes a driving route, a speed, and the like.

[0060] The notification unit 112 notifies the edge system 130 of the current state of the vehicle 100 based on the vehicle information acquired in the autonomous driving mode. The notification unit 112 may further notify the edge system 130 of obstacle information. Furthermore, when the vehicle 100 becomes unable to travel, the notification unit 112 notifies the edge system 130 of information indicating that the vehicle 100 becomes unable to travel.

[0061] The first remote control system 120 is a control system for driving the vehicle 100 by remote control, which is mounted on the vehicle 100. When the vehicle 100 runs in a remote control mode, the first remote control system 120 controls the running of the vehicle 100 based on a remote operation instruction (e.g., a control command) of the remote operation acquired from the second remote control system 300.

[0062] The first remote control system 120 acquires vehicle information of the vehicle 100 based on sensing information of various sensors (not shown) mounted on the vehicle 100. The first remote control system 120 acquires the vehicle information from the vehicle 100, for example, in a remote operation mode. The vehicle information includes at least an image. The vehicle information may also include at least one of the current speed, steering angle, acceleration, and obstacle information of the vehicle 100.

[0063] The first remote control system 120 includes a vehicle control unit 121 and a video transmission unit 122 .

[0064] Based on a remote operation instruction, the vehicle control unit 121 controls the traveling of the vehicle 100. Based on a remote operation instruction, the vehicle control unit 121 controls the speed, steering angle, etc. of the vehicle 100. The remote operation instruction includes information indicating the speed, steering angle, etc.

[0065] The video transmission unit 122 transmits the video acquired from the imaging device in the remote operation mode to the edge system 130. The video transmission unit 122 may transmit other vehicle information (e.g., current speed, steering angle, acceleration, information about obstacles, etc.) acquired from various sensors in the remote operation mode to the edge system 130.

[0066] The edge system 130 generates and transmits information to be transmitted to the server 200 based on information acquired from the automatic driving system 110 and the first remote control system 120. The edge system 130 transmits, for example, event determination information indicating an event that is a cause of the vehicle 100 being unable to travel to the server 200. The event determination information is information used in the server 200 when determining whether or not to switch to remote operation, and is generated based on at least one of sensing information from various sensors of the vehicle 100, the position of the vehicle 100 acquired by the vehicle position acquisition unit 134, and the abnormality detection result of the abnormality detection unit 135. In addition, when the edge system 130 acquires information from the second remote control system 300, it outputs the acquired information to at least one of the automatic driving system 110 and the first remote control system 120. It can also be said that the edge system 130 relays communication between the components of the vehicle 100 and the server 200. The edge system 130 is an example of an information processing terminal (terminal) mounted on the vehicle 100.

[0067] The edge system 130 includes a transmission / reception unit 131 , a switching determination unit 132 , a restriction application unit 133 , a vehicle position acquisition unit 134 , and an abnormality detection unit 135 .

[0068] The transmission / reception unit 131 performs communication between the edge system 130 and each component of the vehicle 100 (e.g., the autonomous driving system 110 and the first remote control system 120), and communication between the edge system 130 and the server 200. The transmission / reception unit 131 includes, for example, a communication module. The transmission / reception unit 131 is an example of a transmission unit and a reception unit.

[0069] The switching determination unit 132 determines whether or not to switch the driving mode of the vehicle 100 that is driving in the autonomous driving mode to the remote control mode. The switching determination unit 132 performs this determination while the vehicle 100 is driving in the autonomous driving mode, for example. The switching determination unit 132 is an example of a determination unit.

[0070] The restriction application unit 133 determines whether or not the remote operation instruction acquired from the second remote control system 300 during driving in the remote operation mode satisfies the restriction generated by the server 200, and if the instruction does not satisfy the restriction, updates the remote operation instruction so that the restriction is satisfied. The restriction application unit 133 is an example of a control unit.

[0071] The vehicle position acquisition unit 134 acquires the current position of the vehicle 100. The vehicle position acquisition unit 134 is realized by, for example, a GPS module that acquires a GPS (Global Positioning System) signal (i.e., a radio wave output from a satellite) and measures the current position of the vehicle 100 based on the acquired GPS signal to acquire the current position.

[0072] The anomaly detection unit 135 detects whether or not there is an anomaly in the autonomous driving system 110. The anomaly detection unit 135 may detect an anomaly based on the responsiveness of the autonomous driving system 110 (for example, the response of the autonomous driving system 110 is slow, etc.), or may detect an anomaly based on power consumption, etc. The anomaly detection unit 135 may detect whether or not there is an anomaly in the first remote control system 120. For example, the anomaly detection unit 135 may perform an anomaly detection for the autonomous driving system 110 while the autonomous driving system 110 is running in the autonomous driving mode, and may perform an anomaly detection for the first remote control system 120 while the autonomous driving system 110 is running in the remote operation mode. The edge system 130 having the anomaly detection unit 135 makes it possible to detect an anomaly that cannot be detected by the autonomous driving system 110, for example.

[0073] It should be noted that the edge system 130 only needs to include at least the transmission / reception unit 131, the switching determination unit 132, and the restriction application unit 133. In other words, the edge system 130 does not need to include at least one of the vehicle position acquisition unit 134 and the abnormality detection unit 135.

[0074] Each of the autonomous driving system 110, the first remote control system 120, and the edge system 130 may be realized as one of the functions of an ECU (Electronic Control Unit) as a control device. The ECU may have, for example, a processor and a memory, and may realize each of the functions of the autonomous driving system 110, the first remote control system 120, and the edge system 130 by executing a program stored in the memory.

[0075] The server 200 performs processing related to an operation event of remote control required for the current state of the vehicle 100 and processing related to generation of a restriction corresponding to the operation event, based on information acquired from the vehicle 100. The server 200 has an operation event determination unit 210, an operation event estimation unit 220, a restriction generation unit 230, and a storage unit 240. The operation event of remote control is an example of a task of remote control. The operation event of remote control may also be described as a remote control event.

[0076] The operation event determination unit 210 infers an operation event recommended for remote operation of the current vehicle 100 based on the event determination information acquired from the vehicle 100. The operation event determination unit 210 identifies an operation event recommended for remote operation of the current vehicle 100 based on, for example, the event determination information and an event list (see FIG. 7 described later) in which the event determination information and the operation events are associated with each other. Note that the number of operation events identified by the operation event determination unit 210 is not particularly limited, and may be one or more. The operation event determination unit 210 is an example of an identification unit.

[0077] The operation event determination unit 210 is not limited to inferring an operation event using an event list, and may, for example, infer an operation event corresponding to the current event determination information using a model generated based on past event determination information and a remote operator's input based on a candidate operation event at that time. The remote operator's input is an operation event selected by the remote operator from among candidate events. The model may be, for example, a machine learning model using a neural network. The machine learning model may be a trained model that is trained to input event determination information and output (identify) an operation event recommended to the remote operator. The machine learning model is trained using past event determination information as input data and the remote operator's input at that time as reference data (correct answer information).

[0078] The operation event determination unit 210 may input the current event determination information to a model and specify the output of the model as a recommended operation event. Note that the model may be any model modeled with the event determination information (parameters) and the operation event (parameters).

[0079] When there are multiple operation events determined by the operation event determination unit 210, the operation event inference unit 220 infers an operation event recommended to the remote operator from among the multiple operation events. The operation event inference unit 220 may infer the recommended operation event based on the similarity between remote operation situation history information (see FIG. 10 described later) indicating the situation around the vehicle 100 at the time of remote operation in a past remote operation event and the current situation of the vehicle 100 (see FIG. 11 described later). The operation event inference unit 220 may, for example, infer the ranking (rank of the possibility of selection) of each of the multiple operation events. Note that the similarity is calculated based on, for example, the standard Euclidean distance, but is not limited to this. In addition, the operation event inference unit 220 may perform one-hot encoding on a categorical variable such as an obstacle identification result as a preprocessing for the similarity calculation.

[0080] The restriction generating unit 230 determines a driving restriction of the vehicle 100 in remote operation according to an operation event selected by a remote operator. The restriction generating unit 230 determines the driving restriction by, for example, generating a driving restriction according to an operation event. The restriction generating unit 230 determines a driving restriction corresponding to a current operation event based on, for example, an operation event selected by a remote operator and remote operation history information (see FIG. 12 described later) indicating a history of past remote operations. The restriction generating unit 230 generates a driving restriction for each operation event. The restriction generating unit 230 is an example of a determination unit. Note that, hereinafter, a description will be given of generating a limit value as the driving restriction, but is not limited thereto.

[0081] The restriction generation unit 230 is not limited to estimating an operation event using remote operation history information, and may determine a driving restriction according to a current operation event using a model generated based on past operation events and driving restrictions imposed at that time. The model may be, for example, a machine learning model using a neural network. The machine learning model may be a trained model that is trained to input an operation event and output a driving restriction according to the operation event. The machine learning model is trained using a past operation event as input data and the driving restriction at that time as reference data (correct answer information).

[0082] The restriction generating unit 230 may input the current operation event into a model and determine the output of the model as the driving restriction corresponding to the operation event.

[0083] The restriction generating unit 230 may also generate a model based on a past operation event and the execution environment of the operation event. The restriction generating unit 230 may determine a driving restriction according to a current operation event, for example, by using a model generated based on a past operation event and the execution environment, and the driving restriction imposed at that time. The model may also be any model modeled on an operation event (parameter) and a driving restriction (parameter). The execution environment includes various information when a remote operation is performed by a remote operator, and includes at least one of information on the remote operator, information on the vehicle, and information on the environment, which will be described later.

[0084] The storage unit 240 stores various information for operation event determination processing and the like. The storage unit 240 stores at least an event list. In this embodiment, the storage unit 240 further stores history information (remote operation history information), obstacle information, and vehicle position information. The contents of the various information will be described later.

[0085] The server 200 may not have the operation event inference unit 220. That is, the server 200 is not limited to presenting the operation events based on the above-mentioned similarity. For example, the order and display mode of the multiple operation events may be set in advance.

[0086] When the switching determination unit 132 determines that it is necessary to switch from the autonomous driving mode to the remote operation mode, the second remote control system 300 receives a remote operation instruction from a remote operator and transmits the remote operation instruction to the vehicle 100 via the server 200. The second remote control system 300 has an operation signal processing unit 310, an operation signal transmitting unit 320, a display unit 330, a receiving unit 340, and a video receiving unit 350.

[0087] The operation signal processor 310 receives remote control instructions from a remote operator via an operation input device (not shown). The operation input device may be realized by, for example, a steering wheel, foot pedals (e.g., an accelerator pedal and a brake pedal), or a joystick.

[0088] The operation signal transmitting unit 320 transmits the remote operation instruction acquired by the operation signal processing unit 310 to the vehicle 100 via the server 200.

[0089] The display unit 330 is a user interface that displays various information received by the second remote control system 300. For example, when in the remote operation mode, the display unit 330 displays the video received by the video receiving unit 350. In addition, when switching from the automatic driving mode to the remote operation mode, the display unit 330 displays the operation event determined by the operation event determination unit 210. When displaying a plurality of operation events, the display unit 330 may change the display mode of the operation events according to the estimation result of the operation event estimation unit 220. The display unit 330 may display the operation event estimated by the operation event estimation unit 220 to have a high similarity so as to be more noticeable than other operation events. For example, the display unit 330 may display the operation event estimated by the operation event estimation unit 220 to have a high similarity larger than other operation events, may display it in a different color, or may display it by blinking. This can improve the efficiency when the remote operator selects an operation event. The display unit 330 is realized by a liquid crystal display or the like, but is not limited thereto.

[0090] Display unit 330 is an example of a presentation unit. Note that the presentation unit is not limited to display unit 330, and may be realized by a sound output unit such as a speaker.

[0091] The reception unit 340 is a user interface that receives a selection made by a remote operator. The reception unit 340 is realized by a push button, a keyboard, a mouse, a touch panel, or the like. The reception unit 340 may also receive a selection by voice. In this embodiment, the reception unit 340 is a touch panel, and is disposed so as to be superimposed on the display unit 330.

[0092] The video receiving unit 350 receives, via the server 200, video of the surroundings of the vehicle 100 transmitted from the video transmitting unit 122 of the vehicle while the vehicle is traveling in the remote operation mode. The video receiving unit 350 displays the received video on the display unit 330. This allows the remote operator to check the video of the surroundings of the vehicle 100 via the display unit 330 while in the remote operation mode.

[0093] As described above, the vehicle control system 10 may include, for example, a switching determination unit 132 that determines whether an event requiring remote operation of the vehicle 100 has occurred, an operation event determination unit 210 that, when it is determined that an event has occurred, identifies a remote operation event corresponding to the event, a restriction generation unit 230 that determines driving restrictions for the vehicle 100 when remotely operated in accordance with the identified operation event, and a restriction application unit 133 that imposes the determined driving restrictions when the vehicle 100 executes driving control in accordance with the identified operation event.

[0094] [2. Vehicle control system operation] Next, the operation of the vehicle control system 10 described above will be described with reference to Fig. 2 to Fig. 14. Fig. 2 is a sequence diagram showing the operation of the vehicle control system 10 according to the present embodiment. Fig. 2 shows a sequence diagram when a limit on remote operation is set in response to an operation event when remote operation is required in the vehicle 100. It is assumed that the vehicle 100 is traveling in an autonomous driving mode.

[0095] 2, while the autonomous driving system 110 is traveling in the autonomous driving mode, it transmits a status notification indicating the current status of the vehicle 100 to the edge system 130 (S11). The status notification is a notification indicating the current status of the vehicle 100, and is notified to the edge system 130 by the notification unit 112. The status notification includes, for example, the status of a sensor mounted on the vehicle 100 (such as an abnormality in the sensor) and the status of the autonomous driving system 110. The status notification may be information indicating a status, and may be a notification including information indicating normality or abnormality, or may be a notification including identification information (for example, an error code) that identifies an abnormality.

[0096] The status notification may also include information regarding the current driving status of the vehicle 100. The information regarding the driving status may include, for example, information indicating whether the vehicle 100 is driving autonomously or not, whether it is stopped or not (for example, whether the autonomous driving system 110 has output a stop instruction or not), and whether the vehicle 100 is before the start of autonomous driving, during driving, or after the end of autonomous driving.

[0097] The autonomous driving system 110 transmits a status notification to the edge system 130 sequentially or at predetermined time intervals.

[0098] The server 200 transmits an instruction notification to the edge system 130 to determine whether or not to switch from the autonomous driving mode to the remote operation mode (S12). The determination of whether or not to switch is, for example, a determination of whether or not the vehicle 100 is in a state that requires switching from the autonomous driving mode to the remote operation mode, and is also described as determining whether or not switching is necessary.

[0099] When the edge system 130 receives a state notification from the autonomous driving system 110, it determines whether or not it is necessary to switch from the autonomous driving mode to the remote operation mode (S13). In this embodiment, when the edge system 130 receives a state notification from the autonomous driving system 110 and also receives an instruction notification from the server 200, it determines whether or not it is necessary to switch.

[0100] Here, an example of the operation of determining whether or not switching is necessary by the edge system 130 will be described with reference to Fig. 3 to Fig. 5. Fig. 3 is a flow chart showing the operation (S13) of determining whether or not switching is necessary shown in Fig. 2.

[0101] 3, the switching determination unit 132 of the edge system 130 determines whether or not a remote operation switching instruction has been acquired from the server 200 (S101). The remote operation switching instruction is an instruction to switch from the automatic driving mode to the remote operation mode regardless of the result of the determination by the switching determination unit 132 as to whether or not switching is necessary, and can also be considered as an instruction to forcibly switch to the remote operation mode. The remote operation switching instruction is transmitted from the server 200 to the edge system 130, for example, when a remote operator performs an operation to switch to the remote operation mode.

[0102] When the switching determination unit 132 receives a remote operation switching instruction from the server 200 via the transmission / reception unit 131 (yes in S101), the process proceeds to step S113 and determines that remote operation switching is necessary. On the other hand, when the switching determination unit 132 does not receive a remote operation switching instruction from the server 200 (no in S101), the switching determination unit 132 executes the processes in and after step S102. That is, the switching determination unit 132 executes a process of determining whether switching is necessary.

[0103] Next, the switching determination unit 132 receives the autonomous driving system state (S102). The switching determination unit 132 may acquire the autonomous driving system state by receiving (newly receiving) a state notification from the autonomous driving system 110, or may acquire the autonomous driving system state by reading out a state notification that was received before step S101 and stored in a storage unit (not shown). The switching determination unit 132 acquires, for example, the latest state notification as the autonomous driving system state.

[0104] Next, the switching determination unit 132 receives the abnormality detection result from the abnormality detection unit 135 (S103). The switching determination unit 132 receives, for example, the latest abnormality detection result. The switching determination unit 132 may further acquire the acquisition result of the current position from the vehicle position acquisition unit 134. The current position includes, for example, at least one of latitude, longitude, and altitude. Note that, although an example in which the switching determination unit 132 acquires the acquisition result of the abnormality detection result and the acquisition result of the current position will be described below, at least one of the abnormality detection result and the acquisition result of the current position may not be acquired. For example, step S103 may not be performed.

[0105] Here, various pieces of information acquired by the switching determination unit 132 will be described with reference to Fig. 4. Fig. 4 is a diagram showing various pieces of information acquired by the edge system 130 according to the present embodiment.

[0106] As shown in FIG. 4, the switching determination unit 132 of the edge system 130 acquires the autonomous driving system state from a state notification from the autonomous driving system 110. The state notification includes, for example, whether the autonomous driving system 110 and various sensors are normal or abnormal, and an error code in the event of an abnormality. The edge system 130 also acquires an abnormality detection result from an abnormality detection unit 135 of the edge system 130. The abnormality detection result includes whether the autonomous driving system 110 and various sensors are normal or abnormal, and an error code in the event of an abnormality. The edge system 130 also acquires the vehicle driving state from a state notification from the autonomous driving system 110. The vehicle driving information indicates the current driving situation of the vehicle 100, and may be autonomous driving (vehicle speed>0), the vehicle is stopped (the autonomous driving system 110 has issued a stop command), autonomous driving has not yet started, autonomous driving has ended, etc.

[0107] 2 again, next, the switching determination unit 132 determines whether or not there is an abnormality in the autonomous driving system 110 based on the state notification (S104). For example, when an error code is acquired, the switching determination unit 132 determines that there is an abnormality in the autonomous driving system 110.

[0108] Next, if there is an abnormality in the autonomous driving system 110 (yes in S104), the switching determination unit 132 searches for the abnormality content in the switching determination list (S105). The switching determination unit 132 uses the switching determination list to determine whether the abnormality content is an abnormality that requires switching from the autonomous driving mode to the remote control mode. Figure 5 is a diagram showing an example of the switching determination list according to this embodiment.

[0109] As shown in Fig. 5, the switching determination list is a list in which event determination information is associated with whether or not switching to remote operation is necessary. The event determination information indicates the current state of the vehicle 100 identified based on the various information shown in Fig. 4. In Fig. 5, event determination information No. 1 to No. 7 are shown as examples. Each event indicated by the event determination information is an example of an occurring event.

[0110] "No. 1 Deviation from driving route" indicates, for example, deviation from the driving plan of the automatic driving by the automatic driving system 110. The switching determination unit 132 can determine whether or not there is deviation from the driving route based on the driving plan and the position acquisition result. "No. 2 Vehicle speed control error" indicates, for example, that the speed of the vehicle 100 cannot be controlled in the control of automatic driving by the automatic driving system 110. The switching determination unit 132 can determine whether or not there is a vehicle speed control error based on the state notification or the abnormality detection result.

[0111] "No. 3 Abnormal vehicle speed information" indicates, for example, that the speed of the vehicle 100 is abnormal. An abnormal speed indicates, for example, that the current speed is a speed other than that included in the driving plan, or that the current speed exceeds a specified speed (for example, a legal speed). The switching determination unit 132 can determine whether or not there is an abnormality in the vehicle speed information based on the vehicle driving state. "No. 4 Failure to acquire vehicle position" indicates that the vehicle position acquisition unit 134 has failed to acquire the current position of the vehicle 100. The switching determination unit 132 can determine whether or not acquisition of the vehicle position has failed based on the result of acquisition of the current position of the vehicle 100 by the vehicle position acquisition unit 134.

[0112] "No. 5 Camera failure" indicates, for example, a failure of the imaging device mounted on the vehicle 100. The switching determination unit 132 can determine whether or not there is a camera failure based on the status notification or the abnormality detection result. "No. 6 Long-term stop" indicates, for example, that the vehicle 100 has been stopped continuously for a predetermined period of time or more. The switching determination unit 132 can determine whether or not there is a long-term stop based on the vehicle driving state. "No. 7 Motor failure" indicates, for example, a failure of the drive motor mounted on the vehicle 100. The switching determination unit 132 can determine whether or not there is a motor failure based on the status notification or the abnormality detection result.

[0113] States No.1 to No.6 are states in which remote operation by a remote operator is possible. Therefore, the question of whether or not to switch to remote operation in states No.1 to No.6 is "necessary." Moreover, state No.7 is a state in which the vehicle 100 cannot even travel, and therefore is a state in which remote operation by a remote operator is impossible. Therefore, the question of whether or not to switch to remote operation in state No.7 is "not necessary." Although not shown, another example of a case in which it is "not necessary" is when an error code is acquired but the autonomous driving system 110 can handle it.

[0114] The switching determination list is stored in advance in a storage unit (not shown) of the edge system 130, for example.

[0115] 3 again, next, when the abnormality content exists in the switching judgment list (yes in S106), the switching judgment unit 132 judges whether or not switching to remote operation is necessary (S107). For example, when the current vehicle state deviates from the driving route, the switching judgment unit 132 judges yes in step S106 since the event judgment information includes the information. Also, for example, when the current vehicle state deviates from the driving route, the switching judgment unit 132 judges yes in step S107 since switching to remote operation is necessary.

[0116] Next, if remote control switching is not required (no in S107), the switching determination unit 132 determines that remote control switching is not required (S108). In this case, the processes from step S16 onwards shown in Fig. 2 are not performed. Note that, similarly, if it is determined that remote control switching is not required thereafter, the processes from step S16 onwards shown in Fig. 2 are not performed. Also, if remote control switching is required (yes in S107), the switching determination unit 132 proceeds to step S113.

[0117] Furthermore, if the abnormality content does not exist in the switching determination list (no in S106), the switching determination unit 132 notifies the server 200 of the abnormality (S109). By the server 200 transmitting the abnormality to the second remote control system 300, for example, it is possible to inform the remote operator that an abnormality that was not anticipated in advance has occurred. Furthermore, for example, if the current state of the vehicle is a motor failure, the switching determination unit 132 transmits information indicating the motor failure to the server 200. Then, the switching determination unit 132 determines that remote operation switching is not necessary (S110).

[0118] Furthermore, when there is no abnormality in the autonomous driving system 110 (no in S104), the switching determination unit 132 determines whether the vehicle 100 has been stopped for a long period of time (S111). The long period of time is, for example, set in advance and stored in a storage unit (not shown). The long period of time may be, for example, several minutes or several tens of minutes.

[0119] Next, if the vehicle 100 has not been stopped for a long time (no in S111), the switching determination unit 132 determines that remote operation switching is not necessary (S112). Also, if the vehicle 100 has been stopped for a long time (yes in S111), the switching determination unit 132 determines that remote operation switching is necessary (S113). In other words, the switching determination unit 132 determines that switching from the automatic driving mode to the remote operation mode is necessary. The following describes a case where the switching determination unit 132 determines that remote operation switching is necessary.

[0120] In addition, the edge system 130 is not limited to executing the processing shown in FIG. 3 in response to receipt of an instruction notification from the server 200 as a trigger, and may perform the processing from step S102 onward in FIG. 3 each time a status notification is received or at predetermined time intervals.

[0121] 2 again, when switching to remote operation is necessary, the switching determination unit 132 transmits an autonomous driving stop signal to the autonomous driving system 110 to stop the autonomous driving of the vehicle 100 (S14). Upon receiving the autonomous driving stop signal, the autonomous driving system 110 stops the vehicle 100 in a safe position and then stops driving control over the vehicle 100. Then, the autonomous driving system 110 transmits an autonomous driving stop completion signal to the edge system 130 indicating that the autonomous driving has been stopped (S15).

[0122] When the switching determination unit 132 of the edge system 130 acquires the automatic driving stop completion signal, it transmits a remote operation start signal indicating the start of remote operation and event determination information to the server 200 (S16). For example, the switching determination unit 132 transmits the event determination information after transmitting the remote operation start signal. When the switching determination unit 132 determines that switching to remote operation is necessary, it transmits the event determination information to the server 200. The event determination information transmitted here may be information indicating an event (occurring event) that is the cause of the need for remote operation in the vehicle 100, and may be an error code or the like. Note that the remote operation start signal and the event determination information may be transmitted before acquiring the automatic driving stop completion signal, or may be transmitted when it is determined that remote operation switching is necessary.

[0123] In step S16, the switching determination unit 132 may transmit the vehicle information to the server 200 in addition to the event determination information.

[0124] When the server 200 receives the remote operation start signal and the event determination information, the server 200 executes a process of identifying candidate operation events to be presented to the remote operator (S17). The operation event indicates the operation content of the vehicle 100 performed by the remote operator when remotely operating the vehicle 100, and is an example of a task. It can also be said that the task indicates the operation content of the remote operation corresponding to the current state (occurring event) of the vehicle 100 that requires remote operation. The event determination information is an example of information on an event that requires remote operation of the vehicle 100. In addition, the candidate operation event is an operation event estimated based on the event determination information, and is an example of a candidate task. In the following, the candidate operation event will also be referred to as a candidate event.

[0125] An example of the operation of identifying an operation event by the server 200 will now be described with reference to Figures 6 and 7. Figure 6 is a flow chart showing the operation (S17) of identifying an operation event shown in Figure 2.

[0126] 6, the operation event determination unit 210 of the server 200 determines whether or not a remote operation start signal has been received from the edge system 130 (S201). In the example of FIG 2, since the remote operation start signal is transmitted from the edge system 130, when the operation event determination unit 210 receives the remote operation start signal, it determines that the remote operation start signal has been received.

[0127] Next, the operation event determination unit 210 receives the remote operation start signal (yes in S201) and receives the event determination information from the edge system 130 (S202). The operation event determination unit 210 receives, for example, the event determination information transmitted after the remote operation start signal from the edge system 130.

[0128] Next, the operation event determination unit 210 searches the event list for the abnormality content indicated by the event determination information (S203). The operation event determination unit 210 determines whether or not the abnormality content indicated by the event determination information matches the abnormality content in the event list.

[0129] Here, the event list will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of an event list according to the present embodiment. The event list shown in Fig. 7 is stored in advance in storage unit 240, for example.

[0130] As shown in Fig. 7, the event list is a list in which event determination information and operation events of remote operation are associated with each other. The event determination information in the event list is composed of, for example, items for which switching to remote operation is "necessary" among the event determination information in the switching determination list shown in Fig. 5. Fig. 7 includes items No. 1 to No. 6 among No. 1 to No. 7 shown in Fig. 5. Also, the operation event indicates the operation content of remote operation performed by the remote operator when the vehicle 100 is switched to the remote operation mode.

[0131] For example, "No. 1 deviation from the driving route" is associated with "No. 1 moving onto the original route" and "No. 2 moving to the destination." This indicates that when the current state of the vehicle 100, that is, the factor for which remote operation is determined to be necessary, is "No. 1 deviation from the driving route," the remote operator is recommended to perform remote operation for "No. 1 moving onto the original driving route" and remote operation for "No. 2 moving to the destination."

[0132] An operation event is also associated with each of the items No. 2 to No. 6 of the event determination information. The number of operation events associated with one item is not particularly limited as long as it is one or more. Furthermore, when a remote operator who executes the operation event (performs remote operation of the vehicle 100) is determined in advance, the operation event corresponding to an item may have content corresponding to the remote operator. In other words, the operation event corresponding to an item may be set for each remote operator.

[0133] Furthermore, when identification information (for example, an error code) for identifying an abnormality is received from the edge system 130, the event list may be a list in which the identification information is associated with an operation event.

[0134] 6 again, next, when the abnormal content exists in the event list (yes in S204), the operation event determination unit 210 sets the corresponding operation event as the determination result (S205). That is, the operation event determination unit 210 specifies the corresponding operation event as a candidate event. It can also be said that the operation event determination unit 210 specifies the corresponding operation event as a candidate event by performing the determination in step S204.

[0135] Furthermore, if the abnormality does not exist in the event list (no in S204), the operation event determination unit 210 determines that the corresponding operation event does not exist as a determination result (no determination result) (S206). No determination result indicates that the abnormality is an abnormality that has not been anticipated in advance, and that information on an operation event for the abnormality does not exist. Therefore, the operation event determination unit 210 does not recommend an operation event for remote operation to the remote operator.

[0136] 2 again, the server 200 then transmits operation event information indicating the determination result of step S204 to the second remote control system 300 (S18). The operation event information includes the identified candidate event. Note that between steps S17 and S18, a remote operator who remotely operates the vehicle 100 may be assigned.

[0137] Next, when the operation signal processor 310 of the second remote control system 300 receives the operation event information from the server 200, it presents the operation event included in the received operation event information and accepts a selection of the operation event from the remote operator (S19). The operation event is specified, for example, by presenting candidate events to the remote operator and specifying the operation event according to an input from the remote operator based on the presented candidate events.

[0138] In this embodiment, the operation signal processing unit 310 displays the operation event information via the display unit 330. Fig. 8 is a diagram showing an example of the display content of the display unit 330 according to this embodiment. Fig. 9 is a diagram showing another example of the display content of the display unit 330 according to this embodiment. Note that Figs. 8 and 9 show an example of displaying the operation event information in the case of the event determination information "No. 6 Long-time stop".

[0139] As shown in Fig. 8, the operation signal processing unit 310 displays on the display unit 330 operation events corresponding to the event determination information "No. 6 Long-term stop", such as "Avoid obstacles on the driving route", "Pull over to the side of the road", and "Wait as is", as well as the standard display "Not applicable", and receives the remote operator's determination result as to which event (operation event) the current remote operation corresponds to via the reception unit 340. In the case of Fig. 8, the operation signal processing unit 310 receives the remote operator's determination result by touching inside the frame displaying the operation event determined by the remote operator (touching the reception unit 340). "Avoid obstacles on the driving route", "Pull over to the side of the road", and "Wait as is" are specific examples of candidate events.

[0140] When the operation signal processing unit 310 acquires vehicle information including a current image captured by an imaging device of the vehicle 100 together with the event determination information, the operation signal processing unit 310 may display the image on the display unit 330 when the remote operator is to determine the operation event of the current remote operation. The image and the selection image shown in FIG. 8 may be displayed side by side.

[0141] In addition, when the remote operator selects "not applicable", the operation signal processing unit 310 may further cause the display unit 330 to display as shown in FIG. 9. The operation event shown in FIG. 9 is an operation event other than the operation event corresponding to "No. 6 Long-time stop". For example, the operation event may be an arbitrary operation event other than the operation event corresponding to "No. 6 Long-time stop" that was previously performed during "No. 6 Long-time stop", or may be a preset operation event. In this way, by displaying the operation events set in the event list, the remote operator is more likely to be able to select an operation event according to the current state of the vehicle 100. If there is no applicable operation event among the operation events shown in FIG. 9, the remote operator can select "Other". "Other" is, for example, an option indicating that a uniform restriction is applied during remote operation regardless of the abnormality content, or that no restriction is applied. Applying a uniform restriction indicates, for example, applying a minimum necessary restriction for driving safety.

[0142] When the operation signal processing unit 310 receives operation event information indicating that there is no corresponding operation event (when the operation signal processing unit 310 receives the result of S206), the operation signal processing unit 310 may, for example, display operation events including at least "Others" on the display unit 330. The operation signal processing unit 310 may, for example, display "Others" and some or all of the preset operation events on the display unit 330.

[0143] Here, the display mode of the operation event on the display unit 330 will be described with reference to Figs. 10 and 11. Fig. 10 is a diagram showing an example of remote operation status history information according to the present embodiment. Fig. 11 is a diagram showing an example of the current status of the vehicle 100 according to the present embodiment. The remote operation status history information shown in Fig. 10 is remote operation status history information for "No. 6 Long-term stop", and is generated by the operation event inference unit 220 based on the history information, obstacle information, and vehicle position information stored in the storage unit 240. The operation event inference unit 220 generates the remote operation status history information based on, for example, the remote operation history information (history information) and the obstacle information and vehicle position information corresponding to the history information. Note that Fig. 10 illustrates only a part of the remote operation history information (such as "remote operation history No." and "corresponding remote operation event No.").

[0144] As shown in FIG. 10, the remote operation status history information includes a "remote operation history No.", a "corresponding remote operation event No.", a "remote operation start position", an "obstacle center (host vehicle coordinate system x, y)", an "obstacle identification result", and an "obstacle dimension". In FIG. 10, the centers of obstacles 1 and 2 are illustrated as the centers of the obstacles, and the identification results of obstacles 1 and 2 are illustrated as the obstacle identification results. Note that the number of obstacles and items related to the obstacles are not limited to these. The remote operation status history information may further include the obstacle dimensions, moving speed, and the like as information related to the obstacles. In addition, in the host vehicle coordinate system, x is positive in the front of the vehicle 100, and in the host vehicle coordinate system, y is positive in the right side of the vehicle 100.

[0145] "Remote operation history No." indicates the identification number of the remote operation history. "Corresponding remote operation event No." is an identification number corresponding to a preset operation event. FIG. 10 shows an example in which "Corresponding remote operation event No." includes an identification number and an operation event. "Remote operation start position" indicates the position where remote operation started. "Center of obstacle" indicates the position of the obstacle, for example, the position in the host vehicle coordinate system (position relative to vehicle 100). "Obstacle identification result" indicates the result of recognizing an obstacle by image recognition or the like. "Obstacle dimensions" indicates, for example, the three-dimensional dimensions (length, width, height) of the obstacle.

[0146] For example, remote operation history No. 1001 indicates that when vehicle 100 is at position (E135.323, N34.413) and there is a person 5 m in front of and 12 m to the right of vehicle 100, remote operation was performed for an operation event to move to destination No. 2.

[0147] As shown in FIG. 11, the current situation of the vehicle 100 (current position, obstacle information) includes the "remote operation start position," the "center of the obstacle (host vehicle coordinate system x, y)," the "obstacle identification result," and the "obstacle dimensions." The "remote operation start position" indicates the current position of the vehicle 100. The "obstacle center" indicates the position of the obstacle relative to the current position of the vehicle 100. The "obstacle identification result" indicates the result of the obstacle being recognized by image recognition or the like.

[0148] The operation event inference unit 220 calculates the similarity between the current situation of the vehicle 100 and each piece of remote operation situation history information. In the case of Fig. 10 and Fig. 11, the remote operation history No. 1001 is calculated to have the highest similarity. Then, the operation event determination unit 210 may further transmit the operation event information including the calculation result of the similarity to the second remote control system 300 in association with the operation event.

[0149] When the received operation event information includes a similarity, the operation signal processor 310 of the second remote control system 300 changes the display mode of the operation event according to the similarity. For example, the operation signal processor 310 may display the operation events on the display unit 330 in descending order of similarity. This allows a past operation event that matches or is similar to the current situation of the vehicle 100 to be preferentially displayed, improving the efficiency of the remote operator's judgment of the operation event.

[0150] Referring back to FIG. 2, the operation signal processing unit 310 of the second remote control system 300 transmits selection information indicating the operation event accepted from the remote operator to the server 200 via the operation signal transmitting unit 320 (S20).

[0151] Next, the server 200 generates a limit value corresponding to the operation event selected by the remote operator based on the selection information received from the second remote control system 300 (S21). The limit generation unit 230 of the server 200 generates a limit value based on the selection information and the remote operation history information. That is, the limit generation unit 230 generates a limit value using a history of past remote operations. The limit generation unit 230 generates a limit value by, for example, statistically processing the remote operation history information. It can also be said that the limit generation unit 230 determines a driving limit of the vehicle 100 in remote operation according to the specified operation event (task). The driving limit includes at least one of the limits of speed (vehicle speed), acceleration, braking, or steering. It can also be said that the limit value limits at least one of the limits of speed (vehicle speed), acceleration, braking, or steering. It can also be said that the limit generation unit 230 generates an operation limit of the operation performed by the remote operator according to the specified operation event (task). It can also be said that the limit generation unit 230 determines a limit value by generating a limit value as described above.

[0152] Here, the remote operation history information will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of the remote operation history information according to the present embodiment.

[0153] 12, the remote operation history information includes a “remote operation history No.”, a “vehicle type”, a “remote operator skill”, a “corresponding remote operation event No.”, a “maximum vehicle speed at that time”, a “maximum steering angle at that time”, and a “maximum steering angle at that time”. The remote operation history information may further include information such as the acceleration at that time.

[0154] "Vehicle type" is the type of automobile, and may be, for example, the purpose, model, year, etc. "Remote operator skill" indicates the driving skill of the remote operator in remote operation. "Maximum vehicle speed at that time" is the maximum speed while the remote operator is remotely operating the vehicle. "Maximum steering angle at that time" is the maximum steering angle while the remote operator is remotely operating the vehicle, and is calculated for both the right and left directions. Note that the speed and steering angle are, for example, actual measured values ​​in the past (at that time).

[0155] For example, remote operation history No. 1001 shows an example in which a remote operator using a skill moves a vehicle of type A from the remote operation start position to destination No. 2 at a maximum speed of 6 km / h, a maximum steering angle (left) of 25 degrees, and a maximum steering angle (right) of 21 degrees.

[0156] The restriction generating unit 230 performs statistics for each corresponding remote control event number, and calculates the average value and standard deviation of each of the maximum vehicle speed, maximum steering angle (left), and maximum steering angle (right). Fig. 13 is a diagram for explaining the calculation of the restriction value according to this embodiment. Specifically, Fig. 13 shows the average value and standard deviation of the maximum vehicle speed for each corresponding remote control event number.

[0157] As shown in Fig. 13, in "No. 1 Move to the original route", the average value of the maximum vehicle speed during operation (the maximum vehicle speed at that time) is 6km / h, and the standard deviation of the maximum vehicle speed during operation is 1.3km / h. This is the result of extracting the maximum vehicle speed only for "No. 1 Move to the original route" from the remote operation history information shown in Fig. 12, and calculating the average value and standard deviation based on the extracted maximum vehicle speed.

[0158] The limit generator 230 calculates the limit value based on the average value and the standard deviation. The upper and lower limit values ​​of the limit value are calculated by the following formulas 1 and 2, where the average value is x, the standard deviation is σ, and the parameter is n.

[0159] Upper limit = x + nσ (Equation 1) Lower limit = x-nσ (Equation 2)

[0160] In this way, by using the standard deviation, the upper and lower limits can be set so as to include the most likely value estimated based on the past history. The parameter n is set in advance in consideration of the safety of driving. The limit value is not limited to setting both the upper limit value and the lower limit value, and at least one of them may be set in consideration of the safety of driving. In the case of the maximum vehicle speed, only the upper limit value of the upper and lower limit values ​​may be set. In addition, data is extracted for each of the maximum steering angle (left) and the maximum steering angle (right) in the same manner as the maximum vehicle speed, and the limit value is calculated based on the statistical value of the extracted data. For example, the limit values ​​of the maximum steering angle (left) and the maximum steering angle (right) are also calculated using Equation 1 and Equation 2.

[0161] In the above, the restriction generating unit 230 calculates the average value, etc. based on the corresponding remote operation event number, but the restriction generating unit 230 may further calculate the average value, etc. based on at least one of information on the remote operator, information on the vehicle, information on the environment, etc. In this case, the restriction generating unit 230 acquires the at least one piece of information on the current vehicle 100.

[0162] The information on the remote operator is an example of an execution environment of the operation event (task), and includes, for example, at least one of the attributes, abilities, and operation history of the remote operator who executes the operation event. The attributes include whether the remote operator is a full-time or multi-time operator of the vehicle 100, age, or handedness. The abilities include driving skills, etc. The operation history includes driving history of remote operation, etc. The restriction generation unit 230 obtains the information on the remote operator, for example, from the second remote control system 300 or a server that manages information on remote operators.

[0163] The information about the vehicle is an example of an execution environment of the operation event, and includes, for example, at least one of the attributes, capabilities, and specifications of the vehicle 100 that is remotely operated by executing the operation event. The attributes include, for example, the vehicle model, year, steering position, drive system (2WD, 4WD), etc. The performance includes acceleration performance, deceleration performance, turning performance, etc. The specifications include the weight, wheelbase, tire diameter, etc. of the vehicle 100. The restriction generation unit 230 obtains the information about the vehicle, for example, from the vehicle 100 or a server that manages a catalog of the vehicle 100, etc.

[0164] The environmental information is an example of an execution environment of an operation event, and includes, for example, at least one of the weather, road surface (road surface condition), and traffic environment of the location where the vehicle 100 remotely operated by executing the operation event is located. The traffic environment includes the width of the road, the degree of congestion of the road, etc. The restriction generation unit 230 obtains the environmental information, for example, from the image analysis result of the image captured by the imaging device of the vehicle 100, or from a server that manages weather and traffic information.

[0165] At least one of these pieces of information may be included in the remote operation history information.

[0166] The limit generating unit 230 may determine a limit value according to the operation event and the execution environment. For example, the limit generating unit 230 may extract a maximum vehicle speed when at least one of the remote operation event number and information on the remote operator, information on the vehicle, and information on the environment matches from the remote operation history information, and calculate an average value from the extracted maximum speed. This makes it possible to generate a limit value that is more appropriate for the current situation of the vehicle 100.

[0167] 2 again, the server 200 transmits the generated limit value and a remote operation start signal to the edge system 130 mounted on the vehicle 100 (S22). Upon receiving the limit value and the remote operation start signal, the edge system 130 stores the limit value and outputs the remote operation start signal to the first remote control system 120 (S23). The limit value received by the edge system 130 is a limit value based on the event determination information transmitted in step S16, and is also a limit value for the vehicle 100 in remote operation.

[0168] When the first remote control system 120 receives the remote operation start signal, it transitions the vehicle 100 to an operable state and outputs a preparation complete signal to the edge system 130 (S24). The edge system 130 transmits the preparation complete signal received from the first remote control system 120 to the server 200 (S25). Then, the server 200 transmits the preparation complete signal received from the edge system 130 to the second remote control system 300 (S26).

[0169] When the second remote control system 300 receives the preparation completion signal from the server 200, the second remote control system 300 may notify the remote operator by displaying the reception of the preparation completion signal on the display unit 330. Then, the second remote control system 300 starts remote operation of the vehicle 100. That is, the operation signal processing unit 310 accepts a remote operation instruction from the remote operator (S27). The operation signal processing unit 310 accepts, for example, an operation on an operation input device. Then, the operation signal processing unit 310 transmits the accepted remote operation instruction to the server 200 (S28). For example, the operation signal processing unit 310 transmits the accepted remote operation instruction to the server 200 via the operation signal transmission unit 320. Note that, for example, a limit value is not applied to the remote operation instruction transmitted to the server 200.

[0170] Next, the server 200 transmits the remote operation instruction received from the second remote control system 300 to the edge system 130 (S29). The server 200 transmits a remote operation instruction corresponding to the operation event to the edge system 130. Note that, for example, the server 200 does not apply a limit value to the remote operation instruction. In other words, the server 200 transfers the remote operation instruction received from the second remote control system 300 to the edge system 130.

[0171] When the limit application unit 133 of the edge system 130 receives a remote operation instruction, it applies a limit value to the remote operation instruction (S30). The limit application unit 133 causes the vehicle 100 to execute driving control in accordance with the remote operation instruction under the received limit value. In this way, the limit value is imposed when the vehicle 100 executes driving control in accordance with a remote operation instruction from a remote operator. In other words, the limit value is imposed when a remote operation instruction from a remote operator is received.

[0172] The limit application unit 133 determines whether the remote operation instruction satisfies the limit value, applies the limit value if the remote operation instruction does not satisfy the limit value, and updates the remote operation instruction so that the limit value is satisfied. In this manner, the determined limit value is imposed when the vehicle 100 executes driving control in response to an operation event. In other words, the determined limit value is imposed when driving control is performed by remote operation of the remote operator.

[0173] Here, the operation of applying the limit value by the limit application unit 133 of the edge system 130 will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the operation of applying the limit value shown in Fig. 2. Note that, although the speed included in the remote operation instruction will be described below, the limit application unit 133 executes the determination shown in Fig. 14 for each of the speed and the steering angle (left and right).

[0174] 14, when the limit application unit 133 receives a remote operation instruction (S301), it determines whether or not the remote operation instruction is higher than the upper limit of the limit value (S302). For example, the limit application unit 133 determines whether or not the speed included in the remote operation instruction is higher than the upper limit of the maximum vehicle speed.

[0175] Next, if the remote operation instruction is higher than the upper limit (yes in S302), the limit application unit 133 sets the remote operation instruction after the application of the limit to the upper limit (S303). For example, the limit application unit 133 updates the speed included in the remote operation instruction to the upper limit of the maximum vehicle speed. Then, the operation of applying the limit value ends, and the process proceeds to step S31.

[0176] If the remote operation instruction is lower than the upper limit (no in S302), the limit application unit 133 further determines whether the remote operation instruction is lower than the lower limit (S304). For example, the limit application unit 133 determines whether the speed included in the remote operation instruction is lower than the lower limit of the maximum vehicle speed.

[0177] Next, if the remote operation instruction is lower than the lower limit (yes in S304), the limit application unit 133 sets the remote operation instruction after the application of the limit to the lower limit (S305). For example, the limit application unit 133 updates the speed included in the remote operation instruction to the lower limit of the maximum vehicle speed. Then, the operation of applying the limit is terminated, and the process proceeds to step S31.

[0178] Also, if the remote operation instruction is higher than the lower limit (no in S304), that is, if the remote operation instruction is within the upper and lower limits, the limit application unit 133 sets the remote operation instruction after the limit application to the remote operation instruction (S306). The limit application unit 133 does not update the received remote operation instruction. Then, the operation of applying the limit value is terminated, and the process proceeds to step S31.

[0179] Referring back to FIG. 2, next, the restriction application unit 133 transmits the remote operation instruction after the restriction application to the first remote control system 120 (S31).

[0180] Next, the first remote control system 120 controls the traveling of the vehicle 100 based on the remote operation instruction after the restriction is applied (S32).

[0181] As a result, even if the remote operation instruction of the remote operator is outside the limit value corresponding to the operation event, the edge system 130 applies the limit value to the remote operation instruction, so that the vehicle 100 can travel while satisfying the limit value. In addition, since the limit value is generated for each operation event, it is possible to suppress the addition of unnecessary driving restrictions compared to the case where the driving restrictions are set uniformly. Therefore, it is possible to suppress a decrease in the efficiency of remote operation by the remote operator. In addition, since the server 200 can impose a driving restriction specialized for each operation event, it is expected to have an effect of further improving safety during remote operation. In addition, since the operation event is presented to the remote operator, the remote operator can be aware of the purpose of the remote operation. Therefore, it is expected to reduce redundant operations by the remote operator.

[0182] As described above, the edge system 130 is, for example, a terminal mounted on the vehicle 100, and is configured to determine whether an event (occurring event) requiring remote operation of the vehicle 100 has occurred (S13), transmit the determined event determination information (S16), receive a limit value of the vehicle 100 in remote operation, receive a remote operation instruction, and cause the vehicle 100 to execute driving control according to the remote operation instruction under the received limit value. The edge system 130 is, for example, a terminal mounted on the vehicle 100, and includes a switching determination unit 132 that determines whether an event (occurring event) requiring remote operation of the vehicle 100 has occurred, a transmission / reception unit 131 that transmits the determined event determination information, a transmission / reception unit 131 that receives the limit value of the vehicle 100 in remote operation and the remote operation instruction, and a limit application unit 133 that causes the vehicle 100 to execute driving control according to the remote operation instruction under the received limit value.

[0183] The process of step S13 may be performed by the server 200. That is, the server 200 may have the function of the switching determination unit 132. In this case, however, it is necessary to transmit information required for the switching determination process from the vehicle 100 to the server 200 by wireless communication. Therefore, it takes more time to obtain the determination result by the time required for the transmission time than when the edge system 130 makes the determination. Also, if there is a delay in the communication between the vehicle 100 and the server 200, it takes even more time to obtain the determination result. Therefore, from the viewpoint of performing the switching determination in more real time, it is preferable that the edge system 130 (that is, the vehicle 100) has the switching determination unit 132.

[0184] The process of step S30 may be performed by the server 200. That is, the server 200 may have the function of the limit application unit 133. In this case, the limit value does not need to be transmitted to the edge system 130. This makes it possible to reduce the amount of processing in the edge system 130.

[0185] In the above embodiment, the restriction generating unit 230 generates a restriction value as an example of generating a restriction, but the present invention is not limited to this. Instead of generating a restriction value, or in addition to generating a restriction value, the restriction generating unit 230 may set an item (such as speed, acceleration, or steering angle) for which a restriction value is to be set. That is, an item for which a restriction value is to be set may be set for each operation event. The remote operation history information includes a list of items for which restrictions are imposed, so that the restriction generating unit 230 can set an item for which a restriction value is to be set. Setting an item is an example of determining a driving restriction.

[0186] The history of the current remote operation is also added to the remote operation history information shown in Fig. 12. That is, the remote operation history information is updated. As a result, the amount of accumulated remote operation history information increases, and it is expected that the determined limit value will become more accurate.

[0187] (Other embodiments) Although the present disclosure has been described based on the embodiments, the present disclosure is not limited to the above-mentioned embodiments. As long as it does not deviate from the gist of the present disclosure, various modifications conceived by a person skilled in the art to the present embodiments and forms constructed by combining components in different embodiments may also be included within the scope of one or more aspects of the present disclosure.

[0188] For example, in the above embodiment, the moving body is a vehicle, but is not limited thereto. The moving body is not particularly limited as long as it can switch between automatic driving and remote control driving, and may be an autonomous robot, an unmanned flying object (e.g., a drone), etc.

[0189] In the above embodiment, the vehicle selectively switches between the automatic driving mode and the remote control mode, but the driving modes of the vehicle are not limited to this. The vehicle may further have a manual driving mode in which the vehicle is driven manually by the driver, and may be selectively switched between three driving modes. When the vehicle is driven in the manual driving mode, the automatic driving mode and the remote control mode are turned off. When the vehicle has the manual driving mode, the person riding in the moving body includes the driver.

[0190] In the above embodiment, the server has been described as having the remote operator select an operation event, but the server is not limited to having the remote operator select an operation event. The server may determine an operation event for remote operation. For example, when there is only one operation event corresponding to the event determination information (occurring event) in the event list, the server may determine the operation event as the operation event to be executed. In this case, the server may transmit information indicating the determined operation event to the remote operation system. The information indicating the determined operation event is included in the operation event information.

[0191] In addition, a part of the processing performed by the edge system in the above embodiment may be performed by a server. For example, at least one of the processing of the switching determination unit and the processing of the restriction application unit may be executed by a server. The server may have the function of the switching determination unit and the function of the restriction application unit.

[0192] In the above embodiment, the restriction generating unit calculates the restriction value based on the average value and standard deviation of the maximum vehicle speed, etc. However, the present invention is not limited to this as long as it uses statistical values ​​of the past history of the maximum vehicle speed, etc. The restriction generating unit may calculate the restriction value by, for example, calculating a predetermined coefficient to the average value.

[0193] In addition, although the communication between the vehicle and the server, and between the server and the second remote control system in the above-described embodiment is performed, for example, by wireless communication, it may be performed by wired communication. For example, the communication between the server and the remote control system may be wired communication.

[0194] In the above embodiment, the machine learning model is a machine learning model using a neural network, but other machine learning models may be used. For example, the machine learning model may be a machine learning model using Random Forest, Genetic Programming, or the like, or may be other models.

[0195] In the above embodiment, an example has been described in which a driving restriction is transmitted to a mobile body, and the mobile body executes driving control in accordance with a remote control instruction under the received driving restriction. However, the server may determine driving control in accordance with a remote control instruction under the driving restriction, transmit the determined driving control to the mobile body, and execute the transmitted driving control in the mobile body.

[0196] For example, when a high-speed and highly reliable communication method such as 5G (Generation) communication is used for communication between a mobile body and a server, a mode is assumed in which even driving control is determined on the server side and the determined driving control is executed on the mobile body. In this case, the driving control is determined on the server side under driving restrictions, and thus driving control with driving restrictions is executed on the mobile body.

[0197] In this way, even when the driving control is determined on the side of the device that remotely controls the moving object, the driving control to be executed by the moving object can be determined taking into account the driving restrictions.

[0198] The order of the processes described in the above embodiment is merely an example. The order of the processes may be changed, the processes may be executed in parallel, or some of the processes may not be executed.

[0199] Moreover, each of the components described in the above embodiments may be realized as software, or may be realized as an LSI, which is typically an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip to include some or all of them. Here, LSI is used, but it may be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration. Furthermore, the method of integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. After LSI manufacture, a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connection or setting of the circuit cells inside the LSI may be used. Furthermore, if an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology or a different derived technology, it is natural that the integration of the components may be performed using that technology.

[0200] In addition, the division of functional blocks in the block diagram is one example, and multiple functional blocks may be realized as one functional block, one functional block may be divided into multiple blocks, some functions may be transferred to other functional blocks, and the functions of multiple functional blocks having similar functions may be processed in parallel or in a time-sharing manner by a single piece of hardware or software.

[0201] In addition, the server may be realized as a single device or may be realized by multiple devices. When the server is realized by multiple devices, the components of the server may be distributed among the multiple devices in any manner. In addition, the communication method between the multiple devices is not particularly limited.

[0202] Furthermore, the technology of the present disclosure may be the above-mentioned program, or a non-transitory computer-readable recording medium on which the above-mentioned program is recorded. Needless to say, the above-mentioned program can be distributed via a transmission medium such as the Internet. For example, the above-mentioned program and a digital signal consisting of the above-mentioned program may be transmitted via a telecommunication line, a wireless or wired communication line, a network such as the Internet, data broadcasting, or the like. The above-mentioned program and a digital signal consisting of the above-mentioned program may be executed by another independent computer system by being recorded on a recording medium and transferred, or by being transferred via a network, or the like.

[0203] In each embodiment, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. [Industrial Applicability]

[0204] The present disclosure is widely applicable to systems that operate mobile objects that can be remotely controlled. [Explanation of symbols]

[0205] 10 Vehicle control system (information processing system) 100 vehicles 110 Autonomous Driving System 111, 121 Vehicle control unit 112 Notification Department 120 First Remote Control System 122 Video transmission unit 130 Edge system (information processing terminal) 131 Transmitter / receiver (transmitter, receiver) 132 Switching judgment unit (judgment unit) 133 Restriction application unit (control unit) 134 Vehicle position acquisition unit 135 Anomaly detection unit 200 Servers 210 Operation event determination unit (identification unit) 220 Operation Event Prediction Unit 230 Restriction generation unit (decision unit) 240 Storage section 300 Second Remote Control System 310 Operation signal processing section 320 Operation signal transmission unit 330 Display section 340 Reception 350 Video Receiving Unit 400 road

Claims

1. 1. A computer-implemented information processing method, comprising: When receiving information about an event that requires remote operation of a moving object that is configured to be switchable between an automatic driving mode and a remote operation mode in which a remote operator remotely operates the moving object, identifying a remote operation task to be performed by the remote operator in response to the event; determining a travel restriction of the moving object in remote operation according to the identified task; The determined travel restriction is travel control according to the task, and is imposed when the moving body executes travel control according to a remote operation instruction from the remote operator in the remote operation mode. Information processing methods.

2. Obtaining an execution environment for the task; The travel restriction is determined according to the task and the execution environment. The information processing method according to claim 1 .

3. The execution environment includes at least one of attributes, capabilities, and operation history of a remote operator who executes the task. The information processing method according to claim 2 .

4. The execution environment includes at least one of attributes, performance, and specifications of the mobile object that is remotely controlled by the execution of the task.

4. The information processing method according to claim 2 or 3.

5. The execution environment includes at least one of the weather, road conditions, and traffic conditions of the location where the mobile object to be remotely operated by the execution of the task is located. The information processing method according to any one of claims 2 to 4.

6. The travel restriction is determined using a model generated based on the past tasks and the execution environment. The information processing method according to any one of claims 2 to 5.

7. The task identification includes: identifying candidate tasks, the tasks being inferred based on information of the event; presenting the identified candidate tasks to a remote operator; Identifying the task in response to input from the remote operator based on the presented candidate tasks. The information processing method according to any one of claims 1 to 6.

8. The candidate tasks to present are identified using a model generated based on information about the event at a past time point and input from the teleoperator based on the candidate tasks at the past time point. The information processing method according to claim 7.

9. The determined travel restriction is imposed when the moving object executes the travel control in accordance with the remote operation instruction in accordance with the specified task. The information processing method according to any one of claims 1 to 8.

10. The driving limitations include limitations on at least one of speed, acceleration, braking, and steering. The information processing method according to any one of claims 1 to 9.

11. Furthermore, the determined travel restriction is transmitted to a terminal mounted on the moving object; Under the transmitted travel restriction, travel control is executed in the moving object in accordance with the remote control instruction. The information processing method according to any one of claims 1 to 10.

12. moreover, determining a driving control according to the remote operation instruction under the driving restriction; Transmitting the determined driving control to a terminal mounted on the moving object; The transmitted driving control is executed in the moving body. The information processing method according to any one of claims 1 to 11.

13. An information processing terminal mounted on a moving body configured to be switchable between an automatic driving mode and a remote operation mode in which a remote operator remotely operates the moving body, a determination unit that determines whether an event that requires remote control of the moving object has occurred; a transmission unit that transmits information about the event when the event occurs; a receiving unit that receives a travel restriction of the moving object in a remote operation performed by the remote operator and an instruction for the remote operation based on the transmitted event information; a control unit that controls the moving body in the remote operation mode to perform a driving control in accordance with the remote operation instruction from the remote operator under the received driving restriction in accordance with the remote operation instruction. Information processing terminal.

14. A determination unit that determines whether or not an event that requires remote operation of a moving object has occurred, the moving object being configured to be switchable between an automatic driving mode and a remote operation mode in which a remote operator remotely operates the moving object; an identification unit that, when it is determined that the event has occurred, identifies a remote operation task to be executed by the remote operator in response to the event; A determination unit that determines a travel restriction of the moving object in remote control in accordance with the identified task; a control unit that imposes the determined travel restriction when the moving body executes travel control in accordance with the specified task in response to a remote operation instruction from the remote operator in the remote operation mode. Information processing system.

Images