Information processing method, information processing program, and information processing device

Abstract

An information reception unit 15A of this information processing device 10 receives mobile body information that includes the position of a mobile body 22. A determination unit 15B determines that the mobile body 22 is located at a specific position when a specific determination condition is satisfied, the condition being that the position of the mobile body 22 is within a specific region including the specific position of the mobile body 22.

Description

Information Processing Method, Information Processing Program, and Information Processing Apparatus

[0001] Embodiments of the present invention relate to an information processing method, an information processing program, and an information processing apparatus.

[0002] In recent years, automation such as conveyance by an autonomous mobile body such as an autonomous driving vehicle has been carried out. In addition, a technology for remotely controlling a mobile body has been disclosed. In the technology for remotely controlling movement, a technology for causing a mobile body to travel toward a specific position such as a target position and causing the mobile body that has arrived at the specific position to execute various operations has been disclosed. In addition, use such as notifying an operator that the mobile body is located at a specific position such as a departure position or a target position, or allowing a local user to execute an operation using the mobile body located at the specific position has also been made.

[0003] Here, in the real space, it may be difficult for the mobile body to be located at a specific position due to the presence of obstacles or the like. However, in the prior art, when the position of the mobile body coincides with a specific position in the real space, it is determined that the mobile body is located at the specific position, and depending on the situation of the real space, it may be difficult to execute suitable remote control support.

[0004] Japanese Patent No. 6566995

[0005] The present invention has been made in view of the above, and an object thereof is to provide an information processing method, an information processing program, and an information processing apparatus capable of performing suitable remote control support.

[0006] The information processing method according to the embodiment is an information processing method executed by an information processing apparatus, and includes a receiving step of receiving mobile body information including the position of the mobile body, and a determination step of determining that the mobile body is located at the specific position when a specific determination condition that the position of the mobile body is within a specific area including the specific position of the mobile body is satisfied.

[0007] Figure 1 is an explanatory diagram of an example of an information processing system of the embodiment. Figure 2 is a functional block diagram of an example of an information processing system of the embodiment. Figure 3A is a schematic diagram of an example of the data structure of mobile information. Figure 3B is a schematic diagram of an example of the data structure of the mobile state master DB. Figure 3C is a schematic diagram of an example of the data in the mobile information management DB. Figure 3D is a schematic diagram of an example of the data structure of the destination location management DB. Figure 3E is a schematic diagram of an example of the data structure of the travel route management DB. Figure 3F is a schematic diagram of an example of the data structure of the task management master DB. Figure 3G is a schematic diagram of an example of the data structure of the task transition management DB. Figure 4A is an explanatory diagram of an example of arrival determination. Figure 4B is an explanatory diagram of an example of arrival determination. Figure 5A is a schematic diagram of an example of a display screen. Figure 5B is a schematic diagram of an example of a display screen. Figure 5C is a schematic diagram of an example of a display screen. Figure 5D is a schematic diagram of an example of a display screen. Figure 5E is a schematic diagram of an example of a display screen. Figure 5F is a schematic diagram of an example of a display screen. Figure 6 is a sequence diagram showing an example of the flow of information processing performed by the information processing system 1. Figure 7A is a flowchart showing an example of the flow of information processing performed by the information processing device. Figure 7B is a flowchart showing an example of the flow of information processing performed by the information processing device. Figure 8A is a schematic diagram of an example of a display screen. Figure 8B is a schematic diagram of an example of a display screen. Figure 8C is a schematic diagram of an example of a display screen. Figure 8D is a schematic diagram of an example of a display screen. Figure 8E is a schematic diagram of an example of a display screen. Figure 8F is a schematic diagram of an example of a display screen. Figure 8G is a schematic diagram of an example of a display screen. Figure 8H is a schematic diagram of an example of a display screen. Figure 8I is a schematic diagram of an example of a display screen. Figure 8J is a schematic diagram of an example of a display screen. Figure 8K is a schematic diagram of an example of a display screen. Figure 8L is a schematic diagram of an example of a display screen. Figure 8M is a schematic diagram of an example of a display screen. Figure 8N is a schematic diagram of an example of a display screen. Figure 8O is a schematic diagram of an example of a display screen. Figure 8P is a schematic diagram of an example of a display screen. Figure 8Q is a schematic diagram of an example of a display screen. Figure 8R is a schematic diagram of an example of a display screen. Figure 8S is a schematic diagram of an example of a display screen. Figure 9A is a schematic diagram of an example of a display screen. Figure 9B is a schematic diagram of an example of a display screen. Figure 9C is a schematic diagram of an example of a display screen.Figure 9D is a schematic diagram of an example of a display screen. Figure 9E is a schematic diagram of an example of a display screen. Figure 9F is a schematic diagram of an example of a display screen. Figure 9G is a schematic diagram of an example of a display screen. Figure 9H is a schematic diagram of an example of a display screen. Figure 9I is a schematic diagram of an example of a display screen. Figure 10 is a hardware configuration diagram.

[0008] The embodiments of the information processing method, information processing program, and information processing device will be described in detail below with reference to the attached drawings.

[0009] Figure 1 is an explanatory diagram of an example of the information processing system 1 of this embodiment.

[0010] The information processing system 1 comprises an information processing device 10, one or more mobile systems 20, one or more operator terminals 30, and one or more worker terminals 40. The information processing device 10, the mobile systems 20, the operator terminals 30, and the worker terminals 40 are connected to each other via a network NW or the like.

[0011] The network NW that connects the information processing device 10 and the mobile system 20 in a communicative manner, the network NW that connects the information processing device 10 and the operator terminal 30 in a communicative manner, and the network NW that connects the information processing device 10 and the worker terminal 40 in a communicative manner may be the same network or different networks.

[0012] The information processing device 10 is an information processing device included in the information processing system 1 that performs tasks such as managing the mobile unit 22, the operator terminal 30, and the worker terminal 40. The information processing device 10 is a dedicated or general-purpose computer.

[0013] The mobile system 20 includes a mobile edge terminal 21 and a mobile body 22. The mobile system 20 includes one mobile edge terminal 21 and one mobile body 22. The mobile edge terminal 21 and the mobile body 22 are communicated with each other.

[0014] The mobile unit 22 is subject to remote control by operator X1 and is capable of autonomous movement.

[0015] Remote control refers to the operations related to remote control of the mobile object 22 by operator X1. Remote control includes at least one of remote operation of the mobile object 22 and remote monitoring of the mobile object 22.

[0016] The mobile device 22 is, for example, an autonomous vehicle, a robot equipped with mobility functions that can travel autonomously, or an automated guided vehicle (AGV). Autonomous vehicles are sometimes referred to as self-driving vehicles. In this embodiment, a configuration in which the mobile device 22 is an autonomously driven vehicle will be described as an example.

[0017] The mobile body edge terminal 21 is an information processing device that performs communication control and other functions to communicate information about the mobile body 22 with the information processing device 10. In this embodiment, the mobile body edge terminal 21 and the mobile body 22 are configured as separate components and are mounted on the mobile body 22, and this configuration is described as one example. However, the mobile body edge terminal 21 and the mobile body 22 may be configured as a single unit.

[0018] The operator terminal 30 is an information processing device for remotely controlling the mobile unit 22. The operator terminal 30 is operated by operator X1, who is responsible for remotely controlling the mobile unit 22. Operator X1 is an example of user X.

[0019] The worker terminal 40 is an information processing device operated by worker X2, who performs work using the mobile device 22 or work related to the mobile device 22 in real space. The worker terminal 40 is, for example, a portable terminal carried by worker X2. Worker X2 is an example of user X.

[0020] Figure 2 is a functional block diagram of an example of the information processing system 1 of this embodiment.

[0021] The mobile system 20 comprises a mobile edge terminal 21 and a mobile body 22. The mobile edge terminal 21 and the mobile body 22 are communicated with each other. As described above, the mobile system 20 includes one mobile edge terminal 21 and one mobile body 22. In this embodiment, the description will assume that the mobile system 20 includes one pair of one mobile edge terminal 21 and one mobile body 22.

[0022] The mobile unit 22 comprises a sensor 22A, a drive unit 22B, and a vehicle control unit 22C. The sensor 22A, the drive unit 22B, and the vehicle control unit 22C are communicated with each other via a bus or the like.

[0023] Sensor 22A is a sensor that detects the state of the moving object 22. Sensor 22A is, for example, an internal sensor and an external sensor.

[0024] The internal environment sensors are sensors that obtain observational information about the internal environment of the mobile body 22. The observational information includes information such as the position, velocity, acceleration, steering angle, steering direction, accelerator pedal depression angle, and vibration of the mobile body 22. Specifically, the internal environment sensors include position sensors (GNSS (Global Navigation Satellite System), GPS (Global Positioning System))). The internal environment sensors also include, for example, an inertial measurement unit (IMU), acceleration sensors, velocity sensors, gyro sensors and other velocity sensors, rotary encoders, steering angle sensors, steering direction sensors, accelerator pedal depression angle detection sensors, vibration sensors, etc. Furthermore, the internal environment sensors also include sensors for detecting mode information indicating whether the mobile body 22 is autonomously driving or not, and sensors for detecting whether the mobile body 22 is stopped or not.

[0025] The external sensor is a sensor that obtains distance information. The distance information indicates the distance from the moving object 22 to objects in the vicinity of the moving object 22.

[0026] External sensors include, for example, distance sensors. Distance sensors include, for example, millimeter-wave radar, laser sensors, and distance image sensors. Laser sensors include, for example, two-dimensional LiDAR (Laser imaging Detection and Ranging) sensors and three-dimensional LiDAR sensors that are installed parallel to the horizontal plane.

[0027] Sensor 22A sequentially detects the state of the moving object 22 in chronological order and sequentially outputs the detection results to the control unit 28.

[0028] The drive unit 22B is a dedicated control device for devices mounted on the mobile body 22 that perform the driving of the mobile body 22. The drive unit 22B includes, for example, an ECU (Electronic Control Unit) that controls the engine, a motor driver that controls the motor, etc. The drive unit 22B controls the driving of the devices through the control of the vehicle control unit 22C.

[0029] The vehicle control unit 22C controls various parts of the mobile body 22. For example, the vehicle control unit 22C controls the drive unit 22B to enable the mobile body 22 to move autonomously. Based on the detection results of the sensor 22A, the vehicle control unit 22C determines the situation around the mobile body 22 and controls the accelerator amount, brake amount, steering angle, etc., to control the drive unit 22B so that the mobile body 22 moves autonomously.

[0030] Furthermore, the vehicle control unit 22C performs remote operation based on remote operation instruction information received from the operator terminal 30 via the mobile body edge terminal 21 and the information processing device 10. Remote operation means driving in response to remote operation by the operator terminal 30. The vehicle control unit 22C may also receive remote operation instruction information from the operator terminal 30 via the information processing device 10 without going through the mobile body edge terminal 21. During remote operation, the vehicle control unit 22C controls the accelerator amount, brake amount, steering angle, etc., based on the remote operation instruction information received from the operator terminal 30, and controls the drive unit 22B so that the mobile body 22 drives in response to operation instructions from the operator terminal 30.

[0031] The mobile edge terminal 21 includes a communication unit 23, an input unit 24, an output unit 25, an imaging unit 26, a storage unit 27, and a control unit 28. The communication unit 23, input unit 24, output unit 25, imaging unit 26, and storage unit 27 and the control unit 28 are communicated to each other via a bus or the like.

[0032] The communication unit 23 communicates with the information processing device 10 via a network NW or the like. The communication unit 23 also communicates with the mobile device 22 via the network NW or directly. The input unit 24 accepts various operations from the user. The output unit 25 outputs various types of information. The output unit 25 includes a display for showing various types of information, a speaker for outputting various types of sounds, etc.

[0033] The imaging unit 26 is a camera that photographs the area around the moving body 22 and obtains captured video data. The captured video data will be referred to as captured video. For example, the moving body 22 may be provided with multiple imaging units 26 (for example, imaging units 26A to 26D) that have different imaging directions from each other.

[0034] The storage unit 27 stores various types of data. The storage unit 27 may be, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, a hard disk, or an optical disc. The storage unit 27 may also be a storage device located outside the mobile edge terminal 21. Furthermore, the storage unit 27 may be a storage medium on which programs and various types of information are downloaded and stored or temporarily stored via a LAN (Local Area Network) or the Internet.

[0035] The control unit 28 is a computer that performs information processing on the mobile edge terminal 21.

[0036] The control unit 28 comprises a communication control unit 28A, a movement control unit 28B, and a mobile information transmission unit 28C. The communication control unit 28A, the movement control unit 28B, and the mobile information transmission unit 28C are implemented by one or more processors. For example, each of the above units may be implemented by having a processor such as a CPU (Central Processing Unit) execute a program, i.e., by software.

[0037] Each of the above components may be implemented by a dedicated IC or other processor, i.e., hardware. Each of the above components may also be implemented using a combination of software and hardware. When multiple processors are used, each processor may implement one of the components, or two or more of the components.

[0038] The communication control unit 28A controls communication with the information processing device 10.

[0039] The movement control unit 28B controls the drive unit 22B via the vehicle control unit 22C to make the moving body 22 travel autonomously. Based on the detection results of the sensor 22A and the like, the movement control unit 28B determines the situation around the moving body 22, performs control such as the accelerator amount, the brake amount, and the steering angle, and controls the drive unit 22B so that the moving body 22 travels autonomously, thereby making the moving body 22 travel autonomously.

[0040] Further, the movement control unit 28B executes remote operation driving based on the remote operation information received from the operator terminal 30 via the information processing device 10. Remote operation driving means driving according to the remote operation by the operator terminal 30. During remote operation driving, the movement control unit 28B performs control such as the accelerator amount, the brake amount, and the steering angle based on the remote operation information received from the operator terminal 30, and controls the drive unit 22B so that the moving body 22 travels according to the operation instruction by the operator terminal 30.

[0041] The moving body information transmission unit 28C transmits the moving body information to the information processing device 10 via the communication control unit 28A and the communication unit 23.

[0042] The moving body information is information regarding the traveling environment of the moving body 22. The moving body information includes the moving body state and the position of the moving body 22. The moving body information may be configured to include the moving body state, the position of the moving body 22, the detection result of the sensor 22A, the captured image captured by the imaging unit 26, and the analysis result of the traveling environment.

[0043] FIG. 3A is a schematic diagram of an example of the data configuration of the moving body information 27A.

[0044] The moving body information 27A includes a moving body ID which is the identification information of the moving body 22, a moving body state, the position of the moving body 22, a detection result, a captured image, and an analysis result.

[0045] The moving body state is information representing the state of the moving body 22. The moving body information transmission unit 28C analyzes the state of the moving body 22 using the detection result, the captured image, the position of the moving body 22, the analysis result, and the like.

[0046] Fig. 3B is a schematic diagram of the mobile body state master DB 27B. The mobile body state master DB 27B is a database that prescribes in advance the types of mobile body states that the mobile body 22 can take. The data format of the mobile body state master DB 27B is not limited to a database.

[0047] The mobile body states include "stopped", "autonomously driving", "under remote operation", "under manual operation", and "offline". "Stopped" includes "normally stopped", "emergency stopped", "temporarily stopped due to an obstacle", "temporarily stopped during remote operation", and "temporarily stopped during manual operation".

[0048] The mobile body state "stopped" indicates that the mobile body 22 is stopped without moving. The mobile body state "autonomously driving" indicates that the mobile body 22 is autonomously driving. The mobile body state "under remote operation" indicates that the mobile body 22 is being remotely operated by the operator X1. The mobile body state "under manual operation" indicates that the mobile body 22 is being directly manually operated by the worker X2 or the like. The mobile body state "offline" indicates that the communication between the mobile body 22 and the information processing device 10 is blocked.

[0049] The mobile body state "normally stopped" indicates that the mobile body 22 is normally stopped. Normally stopped means a normal stop other than an emergency stop, a temporary stop due to an obstacle, a temporary stop during remote operation, and a temporary stop during manual operation. It means a stop due to arrival at a preset target position or near the target position. In the case of the mobile body state "normally stopped", the mobile body 22 does not automatically start running.

[0050] The "Emergency Stop" mobile state indicates that the mobile body 22 has stopped with a deceleration acceleration exceeding a predetermined acceleration due to the occurrence of a predetermined emergency event. The "Temporarily Stopped Due to Obstacle" mobile state indicates that the mobile body 22 has temporarily stopped due to the detection of an obstacle. In the case of the "Temporarily Stopped Due to Obstacle" mobile state, the mobile body 22 will automatically resume travel once the obstacle is no longer detected. The "Temporarily Stopped During Remote Control" mobile state indicates that the mobile body 22 has temporarily stopped due to remote control by operator X1. In the case of the "Temporarily Stopped During Remote Control" mobile state, the mobile body 22 will resume travel due to remote control by operator X1. The "Temporarily Stopped During Manual Operation" mobile state indicates that the mobile body 22 has stopped while user X, such as worker X2, is manually operating the mobile body 22. In the case of the "Temporarily Stopped During Manual Operation" mobile state, the mobile body 22 will resume travel due to manual control by user X.

[0051] The mobile object information transmission unit 28C can use the detection results, captured video, the position of the mobile object 22, and analysis results to determine which of the mobile object states registered in the mobile object state master DB 27B corresponds to the state of the mobile object 22.

[0052] The mobile object information transmission unit 28C derives the position of the mobile object 22 by acquiring the position of the mobile object 22 in real space, which is included in the detection result of the sensor 22A. The position of the mobile object 22 is expressed in terms of latitude and longitude.

[0053] The mobile information transmission unit 28C derives analysis results by analyzing the captured video obtained by the imaging unit 26 using known image processing methods. The mobile information transmission unit 28C also derives analysis results of the driving environment of the mobile unit 22 by analyzing at least one of the captured video, the detection results of the sensor 22A, and the map data using known analysis methods.

[0054] The analysis results include information about the environment surrounding the mobile body 22. For example, the analysis results include information representing obstacles present around the mobile body 22, the mobile body 22's travel area, the condition of objects around the mobile body 22, etc. Information representing obstacles present around the mobile body 22 includes information representing the type of obstacle, the size of the obstacle, the relative position and distance of the obstacle to the mobile body 22, the relative speed, etc. Information representing the mobile body 22's travel area includes, for example, information representing the road conditions while the mobile body 22 is traveling. Road conditions include the type of travel route, information about traffic signs such as pedestrian crossings and traffic lights present around the travel route, the level of road congestion, etc. Information regarding objects around the mobile body 22 includes information representing the number of objects present around the mobile body 22, the type of object, whether or not the object is in contact with the mobile body 22, the relative position and distance of the object to the mobile body 22, etc. An object is, for example, a person. The type of object includes, for example, the person's gender and age. Furthermore, the type of object may include behavioral information that describes a person's actions, such as a parent holding a child, taking a photograph, or viewing an advertisement. The person's age can be estimated, for example, from the captured video using a known method.

[0055] The mobile body information transmission unit 28C transmits mobile body information 27A to the information processing device 10, which includes the mobile body ID of the mobile body 22 on which the mobile body information transmission unit 28C is mounted, the derived mobile body state, the position of the mobile body 22, the detection result, the captured video, and the analysis result. The mobile body information transmission unit 28C repeatedly performs the derivation and transmission of the mobile body information 27A to the information processing device 10 at predetermined timings. The predetermined timing is, for example, a few seconds, but is not limited thereto. The predetermined timing can be changed as appropriate by user X's operation instructions for the input unit 24, etc.

[0056] Returning to Figure 2, we continue the explanation.

[0057] Next, we will explain the worker terminal 40.

[0058] The worker terminal 40 includes a communication unit 41, an input unit 42, an output unit 43, a storage unit 44, and a control unit 45. The communication unit 41, input unit 42, output unit 43, and storage unit 44 and the control unit 45 are communicated to each other via a bus or the like.

[0059] The communication unit 41 communicates with the information processing device 10 via a network NW or the like. The input unit 42 receives various operations from the operator X2. The output unit 43 outputs various types of information. The output unit 43 may be a display for showing various types of information, a speaker for outputting various types of sounds, etc. The input unit 42 and the output unit 43 may be a touch panel that integrates the input unit 42 and the output unit 43.

[0060] The storage unit 44 stores various types of data. The storage unit 44 may be, for example, a semiconductor memory element such as RAM or flash memory, a hard disk, or an optical disc. The storage unit 44 may also be a storage device located outside the worker terminal 40. Alternatively, the storage unit 44 may be a storage medium that stores or temporarily stores programs and various types of information downloaded via a LAN or the Internet.

[0061] The control unit 45 is a computer that performs information processing on the worker terminal 40. The control unit 45 may be implemented by having a processor such as a CPU execute a program, i.e., by software.

[0062] Next, the operator terminal 30 will be described.

[0063] The operator terminal 30 comprises a communication unit 31, an input unit 32, an output unit 33, a storage unit 34, and a control unit 35. The communication unit 31, input unit 32, output unit 33, storage unit 34, and control unit 35 are communicated with each other via a bus or the like.

[0064] The communication unit 31 communicates with the information processing device 10 via a network NW or the like. The input unit 32 receives various operations from the operator X1. The output unit 33 outputs various types of information. The output unit 33 includes a display for showing various types of information, a speaker for outputting various types of sounds, etc.

[0065] The storage unit 34 stores various types of data. The storage unit 34 may be, for example, a semiconductor memory element such as RAM or flash memory, a hard disk, or an optical disc. The storage unit 34 may also be a storage device located outside the operator terminal 30. Alternatively, the storage unit 34 may be a storage medium that stores or temporarily stores programs and various types of information downloaded via a LAN or the Internet.

[0066] The control unit 35 is a computer that performs information processing at the operator terminal 30.

[0067] The control unit 35 comprises a communication control unit 35A, an output control unit 35B, and a remote control control unit 35C. The communication control unit 35A, the output control unit 35B, and the remote control control unit 35C are implemented by one or more processors. For example, each of the above parts may be implemented by having a processor such as a CPU execute a program, i.e., by software.

[0068] Each of the above components may be implemented by a dedicated IC or other processor, i.e., hardware. Each of the above components may also be implemented using a combination of software and hardware. When multiple processors are used, each processor may implement one of the components, or two or more of the components.

[0069] Alternatively, at least one of the above-mentioned components included in the control unit 35 may be mounted on an external information processing device that is connected to the operator terminal 30 via a network NW or the like.

[0070] The communication control unit 35A controls communication with the information processing device 10.

[0071] The output control unit 35B outputs a display screen corresponding to the display screen information received from the information processing device 10 via the communication control unit 35A and the communication unit 31 to the output unit 33.

[0072] The display screen information refers to the information used to display the display screen that operator X1 displays when remotely controlling the mobile unit 22 on the output unit 33 of the operator terminal 30.

[0073] The display screen is used to show various information to operator X1 and to accept input or settings from operator X1. Details of the display screen and its information will be described later.

[0074] The output control unit 35B displays a display screen on the output unit 33's display corresponding to the display screen information received from the information processing device 10. If the display screen information includes video footage of the surroundings of the mobile body 22 captured by the mobile system 20, the output control unit 35B displays the video footage on the output unit 33. The video footage of the mobile body 22 is video footage of the surroundings of the mobile body 22 captured by the imaging unit 26 included in the mobile edge terminal 21 which is communicably connected to the mobile body 22, and which images the surroundings of the mobile body 22. By viewing the display screen including the video footage displayed on the output unit 33, the operator X1 can monitor and remotely control the mobile body 22.

[0075] The remote control unit 35C remotely controls the mobile body 22 in response to the operation of the input unit 32 by operator X1. For example, the remote control unit 35C transmits remote control instruction information corresponding to the operation instruction for the mobile body 22 input by operator X1 in response to the operation of the input unit 32 to the information processing device 10, along with the mobile body ID (identification) of the mobile body 22. The mobile body ID is the identification information of the mobile body 22. The information processing device 10 transmits the remote control instruction information to the mobile body 22 identified by the mobile body ID. The vehicle control unit 22C of the mobile body 22 moves the mobile body 22 in response to the remote control operation by operator X1 by controlling the drive unit 22B based on the remote control instruction information. Therefore, the mobile body 22 is remotely controlled by operator X1 who operates the operator terminal 30.

[0076] Next, the information processing device 10 will be described.

[0077] The information processing device 10 comprises a communication unit 11, an input unit 12, an output unit 13, a storage unit 14, and a control unit 15. The communication unit 11, input unit 12, output unit 13, and storage unit 14 and the control unit 15 are communicated with each other via a bus or the like.

[0078] The communication unit 11 communicates with the mobile system 20, the operator terminal 30, and the worker terminal 40 via a network NW or the like. The input unit 12 accepts various operations from the user. The output unit 13 outputs various types of information. The output unit 13 includes a display for showing various types of information, a speaker for outputting various types of audio, etc.

[0079] The storage unit 14 stores various types of data. The storage unit 14 may be, for example, a semiconductor memory element such as RAM or flash memory, a hard disk, or an optical disk. The storage unit 14 may also be a storage device located outside the information processing device 10. Alternatively, the storage unit 14 may be a storage medium on which programs and various types of information are downloaded and stored or temporarily stored via a LAN or the Internet. Furthermore, at least a portion of the information stored in the storage unit 14 may be stored in a storage device such as an external server device that is communicatively connected to the information processing device 10.

[0080] In this embodiment, the storage unit 14 stores the mobile body status master DB 27B, mobile body information management DB 14B, destination location management DB 14C, travel route management DB 14D, task management master DB 14E, and task transition management DB 14F, etc.

[0081] Figure 3C is a schematic diagram of an example of data from the mobile information management DB14B.

[0082] The Mobile Object Information Management DB 14B is a database for managing the information 27A of each mobile object 22 included in the Information Processing System 1. The data structure of the Mobile Object Information Management DB 14B is not limited to a database.

[0083] The control unit 15, described later, sequentially stores the received mobile body information 27A in the mobile body information management DB 14B in association with the reception time each time it receives mobile body information 27A from the mobile body 22. Therefore, the mobile body information management DB 14B is updated sequentially.

[0084] Figure 3D is a schematic diagram of an example of the data structure of the target location management DB14C.

[0085] The Target Location Management DB 14C is a database for managing a list of target locations for each of the mobile objects 22 included in the Information Processing System 1. The Target Location Management DB 14C is a database that associates target location IDs, target locations, and target area information. The data structure of the Target Location Management DB 14C is not limited to a database.

[0086] The target location ID is the identification information for the target location.

[0087] The target location is just one example of a specific location.

[0088] A specific location is a point in real space represented by latitude and longitude. In this embodiment, we will describe an example in which the specific location is the target location.

[0089] The target location is the positional information of the point that is the destination of the mobile body 22's journey. The target location is represented by latitude and longitude in real space. That is, the target location is a single point represented by latitude and longitude.

[0090] The target domain is an example of a specific domain.

[0091] A specific region is a region that includes a specific location. In other words, a specific location is a point within a specific region that occupies a part of that specific region. In this embodiment, a configuration in which the specific region is the target region will be described as an example.

[0092] The target region is the region that includes the target position of the moving object 22. The target region is a wider area than the target position. In other words, the target position is a point within the target region that occupies a part of the target region.

[0093] The target area is, for example, a circular area with a predetermined radius centered at the target location, or an area surrounding an object placed at the target location. Figure 3D shows an example where the target area is a circular area centered at the corresponding target location, and the target area information is represented by the target area radius, which is the radius of the circle centered at the target location.

[0094] The target location management DB 14C is pre-configured and stored in the storage unit 14. The target location management DB 14C may be updated as needed by operator X1 or user X who manages the information processing system 1.

[0095] Figure 3E is a schematic diagram showing an example of the data structure of the travel route management DB14D.

[0096] The Travel Route Management DB14D is a database for managing the travel routes of the mobile unit 22. The Travel Route Management DB14D is a database that associates travel route IDs, which are identification information for travel routes, with travel route information. The data format of the Travel Route Management DB14D is not limited to a database.

[0097] The travel path information represents the travel path of the mobile body 22. Figure 3E shows an example of how the travel path information is represented by the latitude and longitude of the target position of the mobile body 22 and the points that the mobile body 22 passes through, and arrows connecting these target positions and points in the direction of travel. Furthermore, the travel path information is assumed to include a flag indicating that the latitude and longitude representing the target position of the mobile body 22 is the target position.

[0098] The travel route management DB 14D is pre-configured and stored in the storage unit 14. The travel route management DB 14D may be updated as needed by operator X1 or user X who manages the information processing system 1.

[0099] Figure 3F is a schematic diagram of an example of the data structure of the task management master DB 14E. The task management master DB 14E registers the execution order of tasks performed by the mobile body 22 and the execution status of those tasks in association. In this embodiment, one example is described of a series of tasks, consisting of one or more tasks performed by the mobile body 22, that constitute a delivery task related to the delivery of objects by the mobile body 22. For this reason, the execution status of multiple tasks included in the object delivery task by the mobile body 22 is pre-registered in the task management master DB 14E in the order of execution.

[0100] Furthermore, a task may include a set of subtasks. For example, a delivery task may be configured to include subtasks that define even more detailed actions such as "travel on road A," "travel across pedestrian crossing B," and "travel on road C." Alternatively, a delivery task may be configured to include subtasks that define even more detailed actions such as "travel in corridor A," "move to the 3rd floor using elevator B," and "travel in corridor C."

[0101] Furthermore, the series of tasks performed by the mobile unit 22 are not limited to delivery tasks.

[0102] Figure 3G is a schematic diagram of an example of the data structure of the Task Progression Management DB 14F. The Task Progression Management DB 14F is a database for managing the progress of tasks being executed by each mobile unit 22 for each delivery task. The Task Progression Management DB 14F is a database that associates mobile unit ID, delivery ID, destination location ID, task execution status, and travel route ID. The data format of the Task Progression Management DB 14F is not limited to a database.

[0103] The delivery ID is identification information for a delivery task currently running or being executed on mobile 22, identified by the corresponding mobile ID. The destination location ID in the task transition management DB 14F is the destination location ID for the next destination location of the corresponding mobile 22. The task execution status in the task transition management DB 14F represents the current task execution status of mobile 22, identified by the corresponding mobile ID. The travel route ID in the task transition management DB 14F is identification information for the travel route of mobile 22, identified by the corresponding mobile ID.

[0104] The control unit 15, described later, updates the task execution status of the mobile unit 22 in order from the highest to the lowest execution order registered in the task management master DB 14E, in response to changes in the status of the mobile unit 22. For this reason, the task transition management DB 14F is updated by the control unit 15, described later.

[0105] Returning to Figure 2, we continue the explanation.

[0106] The control unit 15 is a computer that performs information processing in the information processing device 10.

[0107] The control unit 15 comprises an information receiving unit 15A, a determination unit 15B, and an output control unit 15C. The information receiving unit 15A, the determination unit 15B, and the output control unit 15C are implemented by one or more processors. For example, each of the above units may be implemented by having a processor such as a CPU execute a program, i.e., by software.

[0108] Each of the above components may be implemented by a dedicated IC or other processor, i.e., hardware. Each of the above components may also be implemented using a combination of software and hardware. When multiple processors are used, each processor may implement one of the components, or two or more of the components. Furthermore, at least one of the above components included in the information processing device 10 may be mounted on an external information processing device that is connected to the information processing device 10 via a network NW or the like in a way that allows communication.

[0109] The information receiving unit 15A receives information from the mobile unit 22, the operator terminal 30, and the worker terminal 40 via the communication unit 11.

[0110] In detail, when the information receiving unit 15A receives remote control information from the operator terminal 30, it transmits the remote control information to the mobile body 22 identified by the mobile body ID included in the remote control information. The information receiving unit 15A also receives mobile body information 27A from the mobile body 22.

[0111] The determination unit 15B determines that the moving body 22 is located at a specific position if the specific determination conditions are met. The specific determination conditions are the conditions for determining that the position of the moving body 22 is a specific position. In detail, the specific determination conditions indicate that the position of the moving body 22 is within a specific area that includes the specific position of the moving body 22.

[0112] As described above, in this embodiment, we will explain as an example the case in which the specific position is the target position and the specific region is the target region. In this embodiment, the specific determination condition will be referred to as the arrival determination condition. That is, the arrival determination condition is an example of a specific determination condition. The arrival determination condition indicates that the position of the moving body 22 is within the target region which includes the target position of the moving body 22.

[0113] In this embodiment, the determination unit 15B determines that the moving body 22 is located at the target position if the arrival determination conditions are met. That is, in this case, the determination unit 15B determines that the moving body 22 has arrived at the target position.

[0114] The determination unit 15B identifies the position of the mobile body 22, which is represented by the latest mobile body information 27A received from the mobile body 22. The determination unit 15B also reads the target area information corresponding to the target position ID associated with the mobile body ID of the mobile body 22 in the task transition management DB 14F from the target position management DB 14C. The determination unit 15B then determines whether the position P of the identified mobile body 22 is within the target area represented by the read target area information, thereby determining whether the position of the mobile body 22 is within the target area including the target position of the mobile body 22.

[0115] Figures 4A and 4B are explanatory diagrams illustrating an example of arrival determination by the determination unit 15B. Figures 4A and 4B schematically show the target region Q in real space RS.

[0116] Figure 4A shows an example where the target area Q is a circular area with a radius (target area radius) centered at the target position C. The determination unit 15B determines that the moving body 22 has not yet arrived at the target position C if its position P is outside the target area Q (position P1). The determination unit 15B also determines that the moving body 22 has arrived at the target position C if its position P is inside the target area Q (position P2). Positions P1 and P2 are examples of positions P of the moving body 22.

[0117] Figure 4B shows an example where the target area Q is the area surrounding an object D located at the target position C. In this case, the target position C is a position on the object D. The object D is an object that exists in real space RS. Figure 4B shows an example where the object D is an aircraft parked at an airport. It also shows an example where the target area Q is a polygonal shape connecting the endpoints of the outer shape of the aircraft, which is the object D. A target area Q of this shape and extent of an aircraft is sometimes called an aircraft safety zone. An aircraft safety zone means an area where elements surrounding the aircraft, which is a movable object D, must exist with due consideration for their own safety.

[0118] The determination unit 15B determines that the moving body 22 has not yet arrived at the target position C if its position P is outside the target area Q at position P3. The determination unit 15B also determines that the moving body 22 has arrived at the target position C if its position P is inside the target area Q at position P4. Positions P3 and P4 are examples of positions P of the moving body 22.

[0119] In other words, even if the position P of the moving body 22 does not match the target position C, the determination unit 15B determines that the moving body 22 has arrived at the target position C if the position P of the moving body 22 is within the target area Q.

[0120] The arrival determination condition may be that the position of the mobile body 22 is within the target area Q which includes the target position C of the mobile body 22, and the current mobile body state of the mobile body 22 is "stopped". Alternatively, the arrival determination condition may be that the position of the mobile body 22 is within the target area Q which includes the target position C of the mobile body 22, and the current mobile body state of the mobile body 22 is "normally stopped". The determination unit 15B determines whether the mobile body state of the mobile body 22 is "stopped" or "normally stopped" by determining whether the mobile body state included in the latest mobile body information 27A received from the mobile body 22 is "stopped" or "normally stopped".

[0121] Furthermore, if the arrival determination conditions are met and the determination unit 15B receives an instruction to temporarily disable the arrival determination from the operator terminal 30 via the display screen, it may determine that the mobile body 22 has not yet arrived at the target position C. In other words, even if the position P of the mobile body 22 is within the target area Q, the determination unit 15B may determine that the mobile body 22 has not yet arrived at the target position C if it receives an instruction to temporarily disable the arrival determination from the operator terminal 30 due to an operation instruction from the operator X1.

[0122] Furthermore, if the determination unit 15B receives a forced arrival determination instruction from the operator terminal 30 via the display screen, it may determine that the mobile body 22 has arrived at the target location C. In other words, even if the position P of the mobile body 22 is outside the target area Q, if the determination unit 15B receives a forced arrival determination instruction from the operator terminal 30 in response to an operation instruction from the operator X1, it may determine that the mobile body 22 has arrived at the target location C.

[0123] Furthermore, if the determination unit 15B receives another arrival determination instruction from the operator terminal 30 via the display screen after determining that the arrival determination conditions have been met, it may re-determine whether the mobile body 22 has arrived at the target position C.

[0124] Returning to Figure 2, we continue the explanation.

[0125] The output control unit 15C outputs a display screen showing the positional relationship between the target area Q and the mobile body 22 to the operator terminal 30 of the operator X1 who remotely controls the mobile body 22.

[0126] In detail, the output control unit 15C outputs display screen information, including mobile body information 27A received from the mobile body 22, to the operator terminal 30 used by the operator X1. As described above, the mobile body information 27A includes the position P of the mobile body 22, detection results, captured video, analysis results, etc. The display screen information also includes information such as the target position C of the mobile body 22, the target area Q, the travel route R, and map data representing the travel environment of the real space RS on which the mobile body 22 travels.

[0127] In detail, the output control unit 15C generates display screen information that includes mobile object information 27A, the target location ID, task execution status, and travel route ID corresponding to the mobile object ID included in the mobile object information 27A in the task transition management DB 14F, the target location and target area information corresponding to the target location ID in the target location management DB 14C, the travel route information corresponding to the travel route ID in the travel route management DB 14D, and map data. The output control unit 15C then outputs the generated display screen information to the operator terminal 30.

[0128] The output control unit 35B of the operator terminal 30, upon receiving the display screen information, outputs the display screen represented by the display screen information to the output unit 33. Therefore, the output control unit 15C of the information processing device 10 outputs the display screen represented by the display screen information to the output unit 33 of the operator terminal 30.

[0129] Operator X1 can perform remote control, such as remote operation and remote monitoring, of the mobile unit 22 by operating the input unit 32 while referring to the display screen shown on the output unit 33 of the operator terminal 30.

[0130] Furthermore, when the determination unit 15B determines that the mobile body 22 has arrived at the target position C, the output control unit 15C outputs arrival information indicating that the mobile body 22 has arrived at the target position C to at least one of the operator terminal 30 and the worker terminal 40. The arrival information is an example of information indicating that the mobile body 22 is located at a specific position.

[0131] Then, the output control unit 15C updates the task execution status in the task management master DB 14E to the task execution status of the next task in the execution order of the current task execution status, corresponding to the mobile ID of the mobile body 22 that has been determined to have arrived at the target position C, in the task transition management DB 14F.

[0132] Furthermore, if the position P of the mobile body 22 is within the target area Q, but the mobile body state of the mobile body 22 is in a state indicating movement, such as autonomous driving, the output control unit 15C may output warning information to at least one of the operator terminal 30 and the worker terminal 40.

[0133] Figure 5A is a schematic diagram of an example of the display screen 50A. The display screen 50A is an example of the display screen 50 that is displayed on the output unit 33 of the operator terminal 30.

[0134] On the display screen 50A, images representing the target location C, the position P of the mobile body 22, the target area Q, and the travel route R are superimposed on the map image M of the map data. In other words, the display screen 50A displays information representing the positional relationship between the position P of the mobile body 22 and the target location C and the target area Q. The display screen 50A also displays the left image V3, front image V1, right image V4, and rear image V2 captured by the camera unit 26 of the mobile body 22. The left image V3 is the image V captured from the left side of the mobile body 22. The front image V1 is the target location C in front of the mobile body 22, and the right image V4 is the image V captured from the right side of the mobile body 22. The rear image V2 is the image V captured from behind the mobile body 22.

[0135] Furthermore, the display screen 50A may include the detection result S contained in the moving object information 27A. The output control unit 35B displays, for example, an image representing the positional relationship between the moving object 22 and various objects present around the moving object 22, as represented by the detection result S.

[0136] The display screen 50A also includes identification information 50A1 of the mobile body 22, remaining distance information 50A2 representing the remaining distance from the mobile body 22's position P to the target position C, the task execution status 50A3 of the mobile body 22, the mobile body status 50A4 of the mobile body 22, and sensor-related information 50A5.

[0137] Sensor-related information is information representing the detection results by sensor 22A. Sensor-related information includes, for example, information representing the battery level if the mobile body 22 is an electric vehicle, the communication speed of the mobile body system 20, whether or not an operating terminal for directly operating the mobile body 22 is connected to the mobile body 22, and the attitude of the mobile body 22. The operating terminal for directly operating the mobile body 22 may be called a remote control. The attitude of the mobile body 22 is represented, for example, by roll, pitch, yaw, etc.

[0138] The output control unit 15C of the information processing device 10 should output display screen information containing this information to the operator terminal 30. The output control unit 35B of the operator terminal 30 should display the display screen 50A represented by the display screen information containing this information to the output unit 33.

[0139] Figures 5B to 5D are enlarged schematic diagrams of an example of a part of the display screen 50A shown in Figure 5A.

[0140] Figure 5B is an example of a display screen 50A that is displayed on the output unit 33 of the operator terminal 30 when the position P of the mobile body 22 is outside the target area Q.

[0141] The display screen 50A includes identification information S1 of the mobile body 22, the task currently being executed by the mobile body 22 S2, the mobile body status S3 of the mobile body 22, information S4 representing the positional relationship between the position P of the mobile body 22 and the target area Q, and remaining distance information H representing the remaining distance from the position of the mobile body 22 to the target position C. The output control unit 15C of the information processing device 10 should output the display image information containing this information to the operator terminal 30.

[0142] Furthermore, as shown in Figure 5B, the display screen 50A superimposes images representing the position P of the mobile body 22, the travel path R of the mobile body 22, the target position C of the mobile body 22, and the target area Q of the mobile body 22 onto the map image M. The output control unit 35B places and displays the image representing the position P of the mobile body 22 at the position P of the mobile body 22 on the map image M. The output control unit 35B also places and displays images representing the target position C and the target area Q of the mobile body 22 at their respective corresponding positions on the map image M. As a result, the display screen 50A shown in Figure 5B indicates that the position P of the mobile body 22 is located outside the target area Q.

[0143] Figure 5C is an example of a display screen 50A that is displayed on the output unit 33 of the operator terminal 30 when the position P of the mobile body 22 is inside the target area Q.

[0144] As shown in Figure 5C, the display screen 50A superimposes images representing the position P of the mobile object 22, the travel path R of the mobile object 22, the target position C of the mobile object 22, and the target area Q of the mobile object 22 onto the map image M. The output control unit 35B places and displays the image representing the position P of the mobile object 22 at the position P of the mobile object 22 on the map image M. The output control unit 35B also places and displays images representing the target position C and the target area Q, respectively, at positions on the map image M corresponding to the target position C and the target area Q of the mobile object 22. Therefore, the display screen 50B shown in Figure 5C indicates that the position P of the mobile object 22 is located inside the target area Q.

[0145] Preferably, the output control unit 15C of the information processing device 10 outputs a display screen 50 of the first display form to the operator terminal 30 when the position P of the mobile body 22 is within the target area Q, and outputs a display screen 50 of the second display form to the operator terminal 30 when the position P of the mobile body 22 is outside the target area Q. In other words, the output control unit 15C should output to the operator terminal 30 display screen information that includes display instructions to display the cases where the position P of the mobile body 22 is within the target area Q and the position P of the mobile body 22 is outside the target area Q in different display forms. Alternatively, the output control unit 35B of the operator terminal 30 may output to the output unit 33 a display screen with a different display form from the cases where the position P of the mobile body 22 is within the target area Q and the position of the mobile body 22 is outside the target area Q, which is included in the display screen information received from the information processing device 10.

[0146] For example, consider the case where the position P of the moving body 22 is located inside the target area Q (see Figure 5C). The output control unit 35B displays the display frame F1, which was not displayed when the position P of the moving body 22 was located outside the target area Q (see Figure 5B). Note that the output control unit 35B can display the case where the position P of the moving body 22 is inside the target area Q and the case where the position P of the moving body 22 is outside the target area Q in different display modes, and the method of changing the display mode is not limited to whether or not the display frame F1 is displayed.

[0147] Figure 5D is an example of a display screen 50A that is displayed on the output unit 33 of the operator terminal 30 when the position P of the mobile body 22 is located inside the target area Q, the information processing device 10 determines that it has arrived at the target position C, and the operator terminal 30 receives arrival information.

[0148] Let's assume a scenario where the communication control unit 35A of the operator terminal 30 receives arrival information from the information processing device 10. In this case, the arrival information functions as a display instruction to be displayed in a different display format. Upon receiving the arrival information, the output control unit 35B of the operator terminal 30 outputs the display screen 50 of the third display format to the output unit 33.

[0149] As shown in Figure 5D, the display screen 50A superimposes images representing the position P of the mobile body 22, the travel path R of the mobile body 22, the target position C of the mobile body 22, and the target area Q of the mobile body 22 onto the map image M. The output control unit 35B places and displays the image representing the position P of the mobile body 22 at the position P of the mobile body 22 on the map image M. The output control unit 35B also places and displays images representing the target position C and the target area Q at the corresponding positions of the target position C and target area Q of the mobile body 22 on the map image M. Therefore, in the state shown in Figure 5C, it is indicated that the position P of the mobile body 22 is located inside the target area Q.

[0150] Furthermore, when the output control unit 15C receives arrival information, it displays the display area of ​​the mobile body 22, identified by the mobile body ID included in the received arrival information, in a different display format than before the arrival determination. For example, when the output control unit 35B receives arrival information indicating that the mobile body 22 has arrived at the destination location C, it displays a display frame F2 in a different display format than before it was determined that the mobile body 22 had arrived at the destination location C. Display frame F2 is a frame-shaped image with a different display format than display frame F1. Note that the output control unit 35B only needs to display the mobile body 22 in different display formats before and after it is determined that the mobile body 22 has arrived at the destination location C and arrival information is received, and the method of changing the display format is not limited to whether or not display frame F2 is displayed.

[0151] Figure 5E is an enlarged schematic diagram of a part of the display screen 50A shown in Figure 5A.

[0152] On the display screen 50A, the vehicle width guide G and the driving path R are superimposed on the captured video V. In addition, a detection frame F4 indicating object detection is superimposed on objects detected by the sensor 22A that appear in the captured video V. Furthermore, an image representing the target area Q may be displayed on the captured video V.

[0153] The vehicle width guide G is an image that guides the position of the mobile body 22 in the vehicle width direction when the mobile body 22 travels along the travel path R.

[0154] The output control unit 35B can superimpose the following onto the captured video V included in the display screen information received from the information processing device 10: the travel path R represented by the travel path information included in the display screen information, the vehicle width guide G, the target area Q represented by the target area information included in the display screen information, and the detection frame F4 representing the detected objects around the moving body 22, which are represented by the analysis results.

[0155] Figure 5F is a schematic diagram of an example of display screen 50B. Display screen 50B is an example of display screen 50.

[0156] The display screen 50B may include a judgment disable button image B1, a forced arrival judgment button image B2, and a re-arrival judgment button image B3.

[0157] The judgment disable button image B1 is an instruction area for temporarily disabling the arrival judgment of the mobile body 22 by the judgment unit 15B of the information processing device 10. When operator X1 operates the display area of ​​the judgment disable button image B1, the control unit 35 of the operator terminal 30 receives the instruction to temporarily disable the arrival judgment. Upon receiving the instruction to temporarily disable the arrival judgment, the control unit 35 transmits the instruction information to the information processing device 10. Specifically, the control unit 35 transmits the instruction information to the information processing device 10, which includes the mobile body ID of the mobile body 22 corresponding to the display area of ​​the judgment disable button image B1.

[0158] When the determination unit 15B of the information processing device 10 receives an instruction to temporarily disable arrival determination from the operator terminal 30, it should temporarily disable the arrival determination of the mobile body 22 identified by the mobile body ID included in the instruction to temporarily disable arrival determination. Temporarily disabling means that for the mobile body 22, the arrival determination of the mobile body 22 to the target location C will not be performed for a predetermined time.

[0159] Operator X1 may wish to manually adjust the timing of the arrival determination. For example, this may occur in situations where an obstacle in the target area Q may move, and it is expected that the moving object 22 will be able to reach the target position C after a predetermined time has elapsed. In such cases, by operating the display area of ​​the determination disable button image B1 included in the display screen 50, the determination unit 15B of the information processing device 10 can temporarily disable the arrival determination of the moving object 22.

[0160] The forced arrival determination button image B2 is an instruction area that forces the determination unit 15B of the information processing device 10 to determine that the mobile object 22 has arrived at the target position C. When operator X1 operates the display area of ​​the forced arrival determination button image B2, the control unit 35 of the operator terminal 30 receives the forced arrival determination instruction. Upon receiving the forced arrival determination instruction, the control unit 35 transmits the forced arrival determination instruction information to the information processing device 10. Specifically, the control unit 35 transmits the forced arrival determination instruction information to the information processing device 10, which includes the mobile object ID of the mobile object 22 corresponding to the display area of ​​the forced arrival determination button image B2.

[0161] When the determination unit 15B of the information processing device 10 receives forced arrival determination instruction information from the operator terminal 30, it determines that the mobile body 22 identified by the mobile body ID included in the forced arrival determination instruction information has arrived at the destination location C, even if the arrival determination conditions are not met.

[0162] In this case, it is preferable that the output control unit 15C of the information processing device 10 outputs arrival information indicating that the mobile body 22 has arrived at the target position C, and the position P of the mobile body 22 at the time it was determined that it had arrived, to at least one of the operator terminal 30 and the worker terminal 40. This is to notify the operator X1 and the worker X2 of the position P of the mobile body 22.

[0163] The re-arrival determination button image B3 is an instruction area that causes the determination unit 15B of the information processing device 10 to re-determine whether or not the mobile object 22 has arrived at the target position C. When operator X1 operates the display area of ​​the re-arrival determination button image B3, the control unit 35 of the operator terminal 30 receives the re-arrival determination instruction. Upon receiving the re-arrival determination instruction, the control unit 35 transmits the re-arrival determination instruction information to the information processing device 10. Specifically, the control unit 35 transmits the re-arrival determination instruction information to the information processing device 10, which includes the mobile object ID of the mobile object 22 corresponding to the display area of ​​the re-arrival determination button image B3.

[0164] When the determination unit 15B of the information processing device 10 receives arrival determination instruction information again from the operator terminal 30, it updates the task execution state of the mobile body 22 identified by the mobile body ID included in the arrival determination instruction information to the task execution state of the previous task in the execution order, and then determines again whether or not the arrival determination condition is met.

[0165] The output control unit 15C may display the arrival determination button image B3 on the display screen 50 again only when the task execution status of the mobile unit 22 is in a predetermined execution state. For example, it is preferable for the output control unit 15C to display the arrival determination button image B3 on the display screen 50 again when the task execution status of the mobile unit 22 is in any of the following states: "Arrived at loading location", "Loading in progress", "Arrived at delivery destination", or "Unloading in progress", so that the operator X1 can operate the system.

[0166] Next, an example of the information processing flow performed by the information processing system 1 of this embodiment will be described.

[0167] Figure 6 is a sequence diagram showing an example of the information processing flow performed by the information processing system 1 of this embodiment.

[0168] The control unit 15 of the information processing device 10 starts updating the task execution status of each of the mobile bodies 22 included in the information processing system 1 (step S100).

[0169] The control unit 15 updates the task execution status in the task transition management DB 14F, which represents the task currently being executed by the mobile body 22, sequentially from the highest to the lowest execution order registered in the task management master DB 14E, in response to changes in the status of the mobile body 22. In the example shown in Figure 3F, the explanation assumes that among the multiple task execution statuses, "arrival at loading location," "arrival at delivery destination," and "arrival at return location" each mean arrival at the target location C.

[0170] In detail, the control unit 15 stores in the task transition management DB 14F, for each mobile body 22, the mobile body ID of the mobile body 22, the delivery ID for the mobile body 22 identified by the mobile body ID, and the travel route ID representing the travel route of the mobile body 22, in association with each other. Also, in the initial state, the control unit 15 registers in the task transition management DB 14F the destination location ID, which corresponds to the latitude and longitude of the destination location C at the uppermost point in the direction of travel, included in the travel route information represented by the travel route ID, in association with the mobile body ID. Also, in the initial state, the control unit 15 registers "Preparing" as the task execution status of the mobile body 22, which is the highest execution order "1" registered in the task management master DB 14E.

[0171] Then, when the movement of the mobile body 22 at position P begins, the control unit 15 updates the task execution status corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F from "Preparing" in execution order "1" to "Moving to loading position" in execution order "2". Then, when the control unit 15 determines that the mobile body 22 has arrived at the destination position C, it updates the task execution status corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F from "Moving to loading position" in execution order "2" to "Arrived at loading position" in execution order "3". Furthermore, the control unit 15 updates the destination position ID corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F to the destination position ID corresponding to the latitude and longitude of the next destination position C in the direction of travel, which is included in the travel route information represented by the travel route ID.

[0172] Furthermore, when a predetermined time has elapsed, the control unit 15 updates the task execution status corresponding to the mobile ID of the mobile body 22 in the task transition management DB 14F from execution order "3" "Arrived at loading position" to execution order "4" "Loading in progress". When the control unit 15 receives a loading operation signal from the worker terminal 40 indicating the completion of the loading operation to the mobile body 22, it updates the task execution status corresponding to the mobile ID of the mobile body 22 in the task transition management DB 14F from execution order "4" "Loading in progress" to execution order "5" "Waiting for delivery to start".

[0173] When the control unit 15 detects a movement of the position P of the mobile body 22, it updates the task execution status corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F from "Waiting for delivery to start" (execution order "5") to "Delivery in progress" (execution order "6").

[0174] Then, when the control unit 15 determines that the mobile body 22 has arrived at the destination location C, it updates the task execution status corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F from "in transit" (execution order "6") to "arrived at destination" (execution order "7"). In addition, the control unit 15 updates the destination location ID corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F to the destination location ID corresponding to the latitude and longitude of the next destination location in the direction of travel, which is included in the travel route information represented by the travel route ID.

[0175] Furthermore, when a predetermined time has elapsed, the control unit 15 updates the task execution status corresponding to the mobile ID of the mobile unit 22 in the task transition management DB 14F from execution order "7" "Arrived at delivery destination" to execution order "8" "Unloading". When the control unit 15 receives an unloading operation signal from the worker terminal 40 indicating the completion of the unloading operation to the mobile unit 22, it updates the task execution status corresponding to the mobile ID of the mobile unit 22 in the task transition management DB 14F from execution order "8" "Unloading" to execution order "9" "Waiting for return to start".

[0176] When the control unit 15 detects a movement of the position P of the mobile body 22, it updates the task execution status corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F from "Waiting for return to start" (execution order "9") to "Returning" (execution order "10").

[0177] Then, when the control unit 15 determines that the mobile body 22 has arrived at the target location C, it updates the task execution status corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F from "Returning" (execution order "10") to "Arrived at return location" (execution order "11"). Then, after a predetermined time has elapsed, the control unit 15 updates the task execution status corresponding to the mobile body ID of the mobile body 22 in the task transition management DB 14F from "Arrived at return location" (execution order "11") to "Delivery completed" (execution order "12").

[0178] In step S100, the control unit 15 starts updating the execution status of these tasks. This task execution status update process by the control unit 15 continues throughout the processing from step S100 onward.

[0179] The mobile information transmission unit 28C of the mobile system 20 starts transmitting the mobile information 27A of the mobile body 22 to the information processing device 10 (step S104).

[0180] The output control unit 15C of the information processing device 10 starts transmitting the display image information, including the received mobile information 27A, to the operator terminal 30 each time it receives mobile information 27A (step S106). The output control unit 35B of the operator terminal 30 outputs the display screen 50 represented by the received display image information to the output unit 33 (step S108).

[0181] The remote control unit 35C transmits remote control information corresponding to the operation instructions for the mobile body 22 input by operator X1 in response to the operation of the input unit 32 to the information processing device 10, along with the mobile body ID of the mobile body 22 (step S110). The information processing device 10 transmits the remote control information to the mobile body 22 identified by the mobile body ID (step S112). The mobile body 22 moves in response to the remote control by operator X1 by controlling the drive unit 22B based on the remote control information. Thus, the mobile body 22 is remotely controlled by operator X1 operating the operator terminal 30.

[0182] During the above processing, the determination unit 15B of the information processing device 10 determines whether the arrival determination condition is met, which is that the position P of the moving body 22 is within the target area Q (step S114). As described above, the arrival determination condition may also be that the position of the moving body 22 is within the target area Q which includes the target position C of the moving body 22, and the current state of the moving body 22 is "stopped". Alternatively, the arrival determination condition may also be that the position of the moving body 22 is within the target area Q which includes the target position C of the moving body 22, and the current state of the moving body 22 is "normally stopped". Here, we will explain assuming that the determination unit 15B has determined that the arrival determination condition is met.

[0183] The output control unit 15C of the information processing device 10 outputs arrival information to the operator terminal 30 and the worker terminal 40 (steps S116 and S118). Upon receiving the arrival information, the operator terminal 30 displays the display screen 50 in a different format than before receiving the arrival information (step S120). Then, this sequence ends.

[0184] Next, we will explain an example of the information processing flow performed by the information processing device 10.

[0185] Figure 7A is a flowchart showing an example of the information processing flow performed by the information processing device 10.

[0186] The information processing device 10 executes the processes in steps S200 to S226 for each delivery task of the mobile body 22 managed by the information processing system 1. It is assumed that while the processes in steps S200 to S226 are being executed, the information processing device 10 is also performing the task execution status update process described above for each of the mobile bodies 22.

[0187] The information receiving unit 15A receives mobile information 27A from the mobile system 20 (step S200).

[0188] The output control unit 15C outputs the display image information, including the mobile information 27A received in step S200, to the operator terminal 30 (step S202). The output control unit 35B of the operator terminal 30 outputs the display screen 50 represented by the received display image information to the output unit 33.

[0189] The information receiving unit 15A determines whether or not it has received a forced arrival determination instruction from the operator terminal 30 (step S204). The information receiving unit 15A performs the determination in step S204 by determining whether or not it has received forced arrival determination instruction information from the operator terminal 30. If the determination in step S204 is positive (step S204: Yes), the process proceeds to step S214. If the determination in step S204 is negative (step S204: No), the process proceeds to step S206.

[0190] In step S206, the information receiving unit 15A determines whether or not it has received an instruction to temporarily disable arrival determination from the operator terminal 30 (step S206). The information receiving unit 15A performs the determination in step S206 by determining whether or not it has received information about the instruction to temporarily disable arrival determination from the operator terminal 30. If the determination in step S206 is positive (step S206: Yes), the process proceeds to step S208.

[0191] In step S208, the control unit 15 disables the arrival determination of the mobile body 22, identified by the mobile body ID included in the arrival determination temporary invalidation instruction information received in step S206, to the target position C for a predetermined period of time (step S208). Then, the process returns to step S200.

[0192] If a negative result is determined in step S206 (step S206: No), proceed to step S210.

[0193] In step S210, the determination unit 15B determines whether the position P of the mobile body 22 is within the target area Q which includes the target position C of the mobile body 22 (step S210). The determination unit 15B identifies the position of the mobile body 22 represented by the latest mobile body information 27A received from the mobile body 22 in step S200 or step S224, which will be described later. The determination unit 15B also reads the target area information corresponding to the target position ID associated with the mobile body ID of the mobile body 22 in the task transition management DB 14F from the target position management DB 14C. Then, the determination unit 15B determines whether the position of the mobile body 22 is within the target area Q which includes the target position C of the mobile body 22 by determining whether the identified position P of the mobile body 22 is within the target area Q which includes the target position C of the mobile body 22.

[0194] If a negative determination is made in step S210 (step S210: No), the process proceeds to step S228. In step S228, the determination unit 15B determines whether the current mobile state of the mobile body 22 is "stopped" (step S228). The determination unit 15B may also determine whether the current mobile state of the mobile body 22 is "normally stopped". If a negative determination is made in step S228 (step S228: No), the process returns to step S200. If an affirmative determination is made in step S228 (step S228: Yes), the process proceeds to step S230. In step S230, the output control unit 15C outputs a request for restart operation information to the operator terminal 30, indicating a request for the mobile body 22 to restart its movement (step S230). Upon receiving the request for restart operation information, the operator terminal 30 displays alert information indicating a request for the mobile body 22 to restart its movement. The output control unit 15C then repeatedly makes a negative judgment until it determines that the operator X1 has performed a restart operation. Once it determines that the restart operation has been performed, it returns to step S200.

[0195] If the result in step S210 is positive (step S210: Yes), proceed to step S212.

[0196] In step S212, the determination unit 15B determines whether the current state of the mobile body 22 is "stopped" (step S212). The determination unit 15B may also determine whether the current state of the mobile body 22 is "normally stopped". If the determination in step S212 is negative (step S212: No), the process returns to step S200. If the determination in step S212 is positive (step S212: Yes), the process proceeds to step S214.

[0197] In step S214, the determination unit 15B determines that the moving body 22 is located at the target position C. That is, in this embodiment, the determination unit 15B determines that the moving body 22 has arrived at the target position C (step S214). The output control unit 15C then outputs arrival information indicating that the moving body 22 has arrived at the target position C to the operator terminal 30 and the worker terminal 40 (step S216).

[0198] Next, the output control unit 15C updates the task execution status in the task transition management DB 14F, which corresponds to the mobile ID of the mobile body 22 that has been determined to have arrived at the destination location C, to the task execution status in the task management master DB 14E that is in the next execution order of the current task execution status (step S218). As a result of the process in step S218, the task execution status of the mobile body 22 is updated to "arrived at loading location", "arrived at delivery destination", or "arrived at return location", depending on the current task execution status before the update, indicating arrival at the destination location C.

[0199] Next, the determination unit 15B determines whether or not it has received another arrival determination instruction from the operator terminal 30 (step S220). The determination unit 15B performs the determination in step S220 by determining whether or not it has received another arrival determination instruction information from the operator terminal 30. If the determination in step S220 is positive (step S220: Yes), the process proceeds to step S222.

[0200] In step S222, the control unit 15 updates the task execution state corresponding to the mobile ID of the mobile body 22 to be processed in the task transition management DB 14F to the task execution state of the execution order immediately preceding the current task execution state in the task management master DB 14E (step S222). As a result of the processing in step S222, the task execution state of the mobile body 22 is updated from "arrived at loading location", "arrived at delivery destination", or "arrived at return location", which represent arrival at the destination location C, to the task execution state corresponding to the execution order immediately preceding. Then, when the information receiving unit 15A acquires the mobile body information 27A from the mobile body 22 (step S224), the process returns to step S210.

[0201] On the other hand, if a negative determination is made in step S220 (step S220: No), the process proceeds to step S226. In step S226, the control unit 15 determines whether or not to terminate remote control of the mobile body 22 (step S226). For example, the control unit 15 executes the process in step S226 by determining whether or not it has received remote control termination instruction information from the operator terminal 30, which includes the mobile body ID of the mobile body 22 and represents a remote control termination instruction. If a negative determination is made in step S226 (step S226: No), the process returns to step S200. If an affirmative determination is made in step S226 (step S226: Yes), the routine terminates.

[0202] As described above, in this embodiment, the information processing device 10 receives mobile body information 27A including the position P of the mobile body 22 from the information receiving unit 15A. The determination unit 15B determines that the mobile body 22 is located at the target position C (specific position) if the arrival determination condition (specific determination condition) is met, which is that the position P of the mobile body 22 is within the target area Q (specific area) that includes the target position C (specific position) of the mobile body 22.

[0203] Thus, the information processing device 10 of this embodiment determines that the mobile body 22 is located at the target position C (specific position) if it determines that the position P of the mobile body 22 is within a predetermined range area Q (specific area) that includes the target position C (specific position), rather than determining that the position P of the mobile body 22 coincides with the target position C (specific position).

[0204] Therefore, in the information processing device 10 of this embodiment, even if it is difficult for the mobile body 22 to reach a position that coincides with the target position C due to the presence of obstacles around the target position C, it is possible to determine that the mobile body 22 is located at the target position C if the mobile body 22 is located within the target area Q. Thus, situations where it is difficult for the mobile body 22 to reach a position that coincides with the target position C, making it impossible to perform tasks that are scheduled to be carried out by remote control at the target position C, are suppressed, and suitable remote control support can be performed.

[0205] Therefore, the information processing device 10 of this embodiment can provide suitable remote control support.

[0206] Furthermore, in the information processing device 10 of this embodiment, situations where it is difficult for the mobile body 22 to reach a position that coincides with the target position C are suppressed, which would make it impossible to perform the work that is scheduled to be performed at the target position C. Therefore, the flexibility of the operation of various services provided by the mobile body 22 can be improved.

[0207] Furthermore, in the information processing device 10 of this embodiment, the information receiving unit 15A receives mobile body information 27A including the position P of the mobile body 22 and the mobile body state of the mobile body. The determination unit 15B determines that the mobile body 22 has arrived at the target position C if the arrival determination conditions are met, namely that the position P of the mobile body 22 is within the target area Q and the mobile body state indicates that it is stopped.

[0208] Therefore, in the information processing device 10 of this embodiment, when the mobile body 22 stops within the target area Q, it can be determined that the mobile body 22 has arrived at the target position C, making it possible to determine arrival at the target position C with high accuracy while suppressing interference with remote control.

[0209] Furthermore, in the information processing device 10 of this embodiment, the determination unit 15B determines that the mobile body 22 has not yet arrived at the target position C if the arrival determination conditions are met and an instruction to temporarily disable the arrival determination is received from the operator terminal 30 via the display screen 50.

[0210] For example, an obstacle in the target area Q may move after a predetermined time, and the remote control operator X1 may predict that after the predetermined time has elapsed, it will be possible to position the moving object 22 at or closer to the target position C. In such a case, when the operator X1 operates the judgment disable button image B1 included in the display screen 50, the information processing device 10 receives an instruction to temporarily disable the arrival judgment, and the judgment unit 15B can determine that the moving object 22 has not yet arrived at the target position C. Therefore, the information processing device 10 of this embodiment allows the operator X1 to adjust the arrival judgment manually, enabling more suitable remote control support.

[0211] In the information processing device 10 of this embodiment, when the determination unit 15B receives a forced arrival determination instruction from the operator terminal 30 via the display screen 50, it determines that the mobile body 22 has arrived at the target location C.

[0212] For example, there may be situations where it is difficult for the moving object 22 to enter the target area Q due to obstacles or other factors present in the target area Q. In such cases, when the operator X1 operates the forced arrival determination button image B2 included in the display screen 50, the information processing device 10 receives a forced arrival determination instruction, and the determination unit 15B can determine that the moving object 22 has reached the target position C. Therefore, the information processing device 10 of this embodiment allows for manual adjustment of the arrival determination by the operator X1, enabling more suitable remote control support.

[0213] Furthermore, the determination unit 15B of the information processing device 10 in this embodiment, when the arrival determination condition that the position P of the mobile body 22 is within the target area is met, and when it receives another arrival determination instruction from the operator terminal 30 via the display screen 50, re-determines whether the mobile body 22 has arrived at the target position C.

[0214] For example, even if the position P of the moving object 22 does not match the target position C, if the position P of the moving object 22 is within the target area Q, it may be possible for the position P of the moving object 22 to move to a position that matches or is closer to the target position C after the determination has been made. In such a case, the information processing device 10 receives a command to make a new arrival determination by the operator X1 operating the re-arrival determination button image B3 included in the display screen 50, allowing the determination unit 15B to re-determine whether the moving object 22 has reached the target position C. Therefore, the information processing device 10 of this embodiment allows for manual adjustment of the arrival determination by the operator X1, enabling more suitable remote control support.

[0215] Furthermore, the output control unit 15C of the information processing device 10 in this embodiment outputs a display screen 50 showing the positional relationship between the target area Q and the mobile body 22 to the operator terminal 30 of the operator X1 who remotely controls the mobile body 22.

[0216] Therefore, the information processing device 10 of this embodiment can provide the operator X2 with a recognizable positional relationship between the target area Q and the mobile body 22, enabling suitable remote control support.

[0217] Furthermore, the output control unit 15C of the information processing device 10 in this embodiment outputs the display screen 50 of the first display mode to the operator terminal 30 when the position P of the mobile body 22 is within the target area Q, and outputs the display screen 50 of the second display mode to the operator terminal 30 when the position P of the mobile body 22 is outside the target area Q.

[0218] Therefore, the information processing device 10 of this embodiment can easily provide the operator X1 with the ability to recognize whether the position P of the moving object 22 is inside or outside the target area Q by changing the display mode of the display screen 50.

[0219] Furthermore, the output control unit 15C of the information processing device 10 in this embodiment outputs a display screen 50 to the operator terminal 30, in which an image representing the position P of the mobile body 22 and an image representing the target area Q are superimposed on at least one of a map image M representing the driving environment of the mobile body 22 and a captured video V of the surroundings of the mobile body 22.

[0220] Therefore, the information processing device 10 of this embodiment can provide more suitable remote control support.

[0221] Furthermore, the output control unit 15C of the information processing device 10 in this embodiment outputs a display screen 50 to the operator terminal 30, in which an image representing the travel route R of the mobile body 22 is superimposed on at least one of a map image M representing the travel environment of the mobile body 22 and a captured video V of the surroundings of the mobile body 22.

[0222] Therefore, the information processing device 10 of this embodiment can provide more suitable remote control support.

[0223] Furthermore, the output control unit 15C of the information processing device 10 in this embodiment outputs a display screen 50 to the operator terminal 30 that includes remaining distance information representing the remaining distance from the position P of the moving body 22 to the target position C.

[0224] Therefore, the information processing device 10 of this embodiment can provide more suitable remote control support.

[0225] Furthermore, when the output control unit 15C of the information processing device 10 in this embodiment determines that the mobile body 22 has arrived at the target position C, it outputs arrival information indicating that the mobile body 22 has arrived at the target position C to at least one of the operator terminal 30 and the worker terminal 40 of the worker X2 who is performing work using the mobile body 22 in real space RS.

[0226] Therefore, in this embodiment, the information processing device 10 can notify the operator X1 of the operator terminal 30 and the worker X2 of the worker terminal 40 that the mobile body 22 has arrived at the target position C.

[0227] (Modification 1) In the above embodiment, a configuration in which the specific position is the target position and the specific region is the target region was described as an example. However, the specific position may be the starting position of the moving body 22, and the specific region may be the starting region including the starting position.

[0228] The definitions of the starting position and starting area are the same as in the above embodiment, except that the position is the starting position instead of the target position of the moving body 22, and the area is the starting area including the starting position instead of the target position.

[0229] In other words, the starting position is a single point represented by latitude and longitude in real space. The starting position is the positional information of the starting point of the moving object 22. The starting region is the region that includes the starting position of the moving object 22. The starting region is a region that is wider than the starting position. The starting position is a point within the starting region that occupies a part of the starting region.

[0230] The starting area, like the target area, can be, for example, a circular area with a predetermined radius centered at the starting position, or an area surrounding an object placed at the starting position.

[0231] In this case, the destination location management DB14C only needs to have the departure location ID, departure location (latitude and longitude), and departure area information (for example, information representing the radius from the departure location) pre-registered in association with each other (see Figure 3D). The departure location ID is the identification information of the departure location. In addition to the destination location ID, destination location, and destination area information, the destination location management DB14C may also have the departure location ID, departure location, and departure area information pre-registered in association with each other.

[0232] Furthermore, in the task transition management DB14F, the departure position ID should be registered either in place of the destination position ID or together with the destination position ID (see Figure 3G).

[0233] The determination unit 15B, similar to the above embodiment, determines that the moving body 22 is located at a specific position if the specific determination conditions are met. As described in the above embodiment, the specific determination conditions indicate that the position of the moving body 22 is within a specific region that includes the specific position of the moving body 22.

[0234] In this modified example, we will explain the case where a specific position is the starting position and a specific region is the starting region. Therefore, in this modified example, the specific determination condition is that the position of the moving body 22 is within the starting region which includes the starting position of the moving body 22.

[0235] In this modified example, the determination unit 15B determines that the mobile body 22 is located at the departure position if specific determination conditions are met, and outputs a request to start driving information to the operator terminal 30 of the operator X1 who remotely controls the mobile body 22.

[0236] In this modified example, the specific determination condition may be the same as the arrival determination condition described in the above embodiment, indicating that the position of the mobile body 22 is within the departure area including the departure position of the mobile body 22 and that the current mobile body state of the mobile body 22 is "stopped". Alternatively, the specific determination condition may be the same as the position of the mobile body 22, indicating that the position of the mobile body 22 is within the departure area including the departure position of the mobile body 22 and that the current mobile body state of the mobile body 22 is "normally stopped". The determination unit 15B determines whether the mobile body state of the mobile body 22 is "stopped" or "normally stopped" by determining whether the mobile body state included in the latest mobile body information 27A received from the mobile body 22 is "stopped" or "normally stopped".

[0237] Then, when the output control unit 15C determines that the mobile body 22 is in the starting position, it should output a request to start driving information to the operator terminal 30, indicating a request for the mobile body 22 to start driving. For this reason, the display screen 50 on the operator terminal 30 will display alert information indicating a request for the mobile body 22 to start driving.

[0238] Furthermore, the output control unit 15C may output a request to start driving information to the operator terminal 30 when it is determined that the mobile body 22 is located at the departure position and the task execution status of the mobile body 22 is "waiting for delivery to start" or "waiting for return to start".

[0239] Furthermore, the determination unit 15B may determine that the mobile body 22 is at the departure position if the position of the mobile body 22 is within a specific area including a specific position of the mobile body 22, the mobile body state is "stopped" or "normally stopped", and the task execution state of the mobile body 22 is "waiting for delivery to start" or "waiting for return to start".

[0240] Next, an example of the information processing flow performed by the information processing device 10 of this modified example will be described.

[0241] Figure 7B is a flowchart showing an example of the information processing flow performed by the information processing device 10.

[0242] The information processing device 10 executes the processes in steps S300 to S316 for each delivery task of the mobile body 22 managed by the information processing system 1. It is assumed that while the information processing device 10 is executing the processes in steps S300 to S316, it is also executing the task execution status update process described above for each of the mobile bodies 22.

[0243] The information receiving unit 15A receives mobile information 27A from the mobile system 20 (step S300).

[0244] The output control unit 15C outputs the display image information, including the mobile information 27A received in step S300, to the operator terminal 30 (step S302). The output control unit 35B of the operator terminal 30 outputs the display screen 50 represented by the received display image information to the output unit 33.

[0245] Next, the determination unit 15B determines whether the position P of the mobile body 22 is within the departure area including the departure position of the mobile body 22 (step S304). The determination unit 15B identifies the position of the mobile body 22 represented by the latest mobile body information 27A received from the mobile body 22 in step S300. The determination unit 15B also reads the departure area information corresponding to the departure position ID associated with the mobile body ID of the mobile body 22 in the task transition management DB 14F from the target position management DB 14C. Then, the determination unit 15B determines whether the position of the mobile body 22 is within the departure area including the departure position of the mobile body 22 by determining whether the identified position of the mobile body 22 is within the departure area represented by the read departure area information.

[0246] If the result in step S304 is negative (step S304: No), return to step S300. If the result in step S304 is positive (step S304: Yes), proceed to step S306.

[0247] In step S306, the determination unit 15B determines whether the current state of the mobile body 22 is "stopped" (step S306). The determination unit 15B may also determine whether the current state of the mobile body 22 is "normally stopped". If the determination in step S306 is negative (step S306: No), the process returns to step S300. If the determination in step S306 is positive (step S306: Yes), the process proceeds to step S308.

[0248] In step S308, the determination unit 15B determines that the moving body 22 is located at the starting position (step S308).

[0249] Next, the determination unit 15B determines whether the task execution status of the mobile body 22 is "waiting for delivery to begin" or "waiting for return to begin" (step S310). As described above, the determination in step S310 may be performed at a timing between step S308 and step S306. If the determination in step S310 is negative (step S310: No), the process returns to step S300. If the determination in step S310 is positive (step S310: Yes), the process proceeds to step S312.

[0250] In step S312, the output control unit 15C outputs a request for a start operation to the operator terminal 30, which represents a request for the mobile body 22 to start moving (step S312).

[0251] Next, the output control unit 15C determines whether or not the operator X1 has performed the start operation (step S314). If the determination in step S314 is negative (step S314: No), the process returns to step S312. If the determination in step S314 is positive (step S314: Yes), the process proceeds to step S316.

[0252] Next, the output control unit 15C updates the task execution status in the task management DB 14F, which corresponds to the mobile ID of the mobile body 22 that has been determined to have arrived at the departure position, to the task execution status in the task management master DB 14E that is in the next execution order after the current task execution status (step S316).

[0253] Next, the control unit 15 determines whether or not to terminate remote control of the mobile body 22 (step S318). The determination in step S318 can be performed in the same manner as in step S226. If the determination in step S318 is negative (step S318: No), the process returns to step S300. If the determination in step S318 is positive (step S318: Yes), this routine is terminated.

[0254] As described above, in this modified information processing device 10, the control unit 15 determines that the mobile body 22 is at the departure position if the position of the mobile body 22 is within the departure area including the departure position of the mobile body 22, and outputs a request to start driving information to the operator terminal 30 of the operator X1 who remotely controls the mobile body 22.

[0255] Therefore, in this modified information processing device 10, even if the mobile body 22 is located at a position shifted from the starting position, if the mobile body 22 is located within the starting area including the starting position, it is determined that the mobile body 22 is located at the starting position and information requesting the mobile body 22 to start moving can be output to the operator terminal 30. Therefore, in this modified information processing device 10, even if it is difficult for the mobile body 22 to be located at a position that coincides with the starting position for some reason, information requesting the mobile body 22 to start moving can be output to the operator terminal 30.

[0256] Therefore, the information processing device 10 in this modified example can provide suitable remote control support, similar to the embodiment described above.

[0257] (Modification 2) In this modification, the display screen 50 output to the output unit 33 of the operator terminal 30 will be described in detail. In this modification, the display screen 50 output to the output unit 33 of the operator terminal 30 will be referred to as the display screen 60. The display screen 60 is an example of the display screen 50.

[0258] Furthermore, in this modified example, we will explain a configuration in which the operator X1 remotely controls the mobile unit 22 using a single UI (User Interface).

[0259] As described in the above embodiment, the output control unit 35B of the operator terminal 30 outputs the display screen represented by the display screen information received from the information processing device 10 to the output unit 33.

[0260] Figure 8A is a schematic diagram of an example of the display screen 60A. The display screen 60A is an example of the display screen 60 and display screen 50 displayed on the output unit 33 of the operator terminal 30.

[0261] When operator X1 activates the operator terminal 30 and starts a program for remotely controlling the mobile unit 22, for example, a display screen 60A is displayed on the output unit 33.

[0262] The display screen 60A includes an input field 60A1 for the operator X1's login ID (Identification), a password input field 60A2, and a login button 60A3. Operator X1 enters their login ID, such as their email address, into the input field 60A1, enters their password into the password input field 60A2, and then operates the login button 60A3. The control unit 15 of the information processing device 10, upon confirming that the entered login ID and password pair matches a pre-configured pair, permits the execution of the program for remote control and login, and outputs the display screen information of the next display screen 60B (see Figure 8B) to the operator terminal 30.

[0263] Figure 8B is a schematic diagram of an example of display screen 60B. Display screen 60A is an example of display screen 60 and display screen 50 displayed on the output unit 33 of the operator terminal 30.

[0264] When login to operator X1 is permitted, display screen 60B is shown on the output unit 33.

[0265] The display screen 60B includes a list of remote control locations 60B1 by operator X1 and a button image 60B2 for receiving instructions to transition to the next display screen 60. Operator X1 selects one or more locations to be remotely controlled from the list of remote control locations 60B1 and operates the button image 60B2. The control unit 15 of the information processing device 10 outputs the display screen information of the display screen 60C (see Figure 8C) relating to the mobile object 22 located at the remote control location received via the display screen 60B to the operator terminal 30.

[0266] Figure 8C is a schematic diagram of an example of the display screen 60C. The display screen 60C is an example of the display screen 60 and display screen 50 displayed on the output unit 33 of the operator terminal 30.

[0267] When the button image 60B2 on the display screen 60B (Figure 8B) is operated, the output control unit 35B outputs the display screen 60C to the output unit 33. The display screen 60C is a display screen 60 in which information about the mobile object 22 located at the base that is the target of remote control, received via the display screen 60B, is superimposed on the map image M.

[0268] On the display screen 60C, images representing the respective positions P, target positions C, target areas Q, and travel routes R of one or more mobile bodies 22 that are the target of remote control by operator X1 are superimposed on the map image M. For example, the image representing the position P of a mobile body 22 is represented by an image such as an arrow indicating the direction of travel of the mobile body 22.

[0269] Furthermore, the display screen 60C includes alert information 60C2 representing alarms, warnings, or requests for operation to operator X1 that have occurred in each of the multiple mobile units 22. The alert information represents the request for action from operator X1 by the mobile unit 22. Examples of alert information include warning alerts, alarm alerts that represent a more urgent request for action than a warning alert, and so on. When an alert event occurs that notifies a mobile unit 22 of a warning alert or an emergency alert, the output control unit 15C of the information processing device 10 should output the display screen information, which includes the mobile unit ID of the mobile unit 22 where the alert event occurred and the alert information representing the alert event of the mobile unit 22, to the operator terminal 30.

[0270] Furthermore, the display screen 60C includes a list 60C1 of multiple mobile objects 22 that are subject to remote control by operator X1. The display screen 60C also includes a button image 60C3 for receiving instructions to transition to the next display screen 60. When operator X1 operates the button image 60C3 and receives instructions to transition to the next display screen 60, the output control unit 15C of the information processing device 10 outputs display screen information for displaying the video monitoring screen 60D (see Figure 8D) to the operator terminal 30.

[0271] Figure 8D is a schematic diagram of an example of the display screen 60D. The display screen 60D is an example of the display screen 60 and display screen 50 displayed on the output unit 33 of the operator terminal 30.

[0272] When the button image 60C3 on the display screen 60C (Figure 8C) is operated, the output control unit 35B outputs the display screen 60D to the output unit 33. The display screen 60D is a display screen 60 that includes the captured video V of the surroundings of the mobile object 22 that is the target of remote control.

[0273] The display screen 60D shows a list 60D1 of multiple mobile objects 22 that are subject to remote control by operator X1. Alert information 60D2 is displayed in the display column of the mobile object 22 in the list 60D1 that is experiencing an alert event. In addition, if an alert event occurs in a mobile object 22 that is not included in the display screen 60D, the output control unit 35B displays an alert 60D3 indicating that there is a mobile object 22 experiencing an alert event that is outside the screen.

[0274] Furthermore, the display screen 60D includes, for each of the remotely controlled mobile bodies 22, a display area 60D4 showing the captured video V of the area around the mobile body 22, the mobile body ID and communication status details of the mobile body 22, a display area 60D5 showing the direction and path R of the mobile body 22, the target position C and target area Q, the travel status of the mobile body 22 60D6, the communication delay time 60D7, the mobile body status 60D8, alert information 60D9, and a display area 60D10 for remote control screen transition buttons and mute buttons. By operating the display area 60D6 of the travel status of the mobile body 22, operator X1 can issue travel instructions to the mobile body 22. Therefore, operator X1 can remotely control the mobile body 22 by performing operations such as touching a part of the display screen 60 without taking their eyes off the captured video V.

[0275] It is preferable that the output control unit 35B positions each captured video V of the mobile body 22 in the central part of the display screen 60D, excluding the edges. Furthermore, it is preferable that the output control unit 35B positions the display field 60D4, which shows detailed information of the mobile body 22, such as the mobile body ID and communication status details of the mobile body 22, at the upper edge of the display screen 60D. It is also preferable that the output control unit 35B positions important information, such as alert information 60D9, at the bottom of the display area for each mobile body 22.

[0276] The output control unit 35B may change the display color of the communication delay time 60D7 according to the value of the communication delay time. Furthermore, the arrangement of the display area of ​​the mobile body 22 displayed on the display screen 60D may be changed in response to operation instructions from the operator X1 to the input unit 32. During this change operation, the control unit 35 may prohibit screen transitions on the display screen 60 and remote control of the mobile body 22.

[0277] Figure 8E is a schematic diagram of an example of the display screen 60E. The display screen 60E is an example of the display screen 60 and display screen 50 displayed on the output unit 33 of the operator terminal 30.

[0278] When the display area 60D10 of the remote control screen transition button on the display screen 60D (Figure 8D) is operated, the output control unit 35B outputs the display screen 60E for remote control of the mobile body 22, which is displayed in the display area where the display area 60D10 of the remote control screen transition button is located, to the output unit 33. The display screen 60E is the display screen 60 that is displayed when the operator X1 performs remote control.

[0279] The display screen 60E shows the captured video V of the surrounding area of ​​the mobile object 22, which is the target of remote control selected by operator X1. The travel route R and vehicle width guide G are superimposed on the captured video V.

[0280] The display screen 60E also includes a display field 60E1 showing the mobile ID and communication status details of the mobile unit 22, an emergency stop button 60E2 for instructing an emergency stop of the mobile unit 22, and a display field 60E3 showing the direction of travel and travel path R of the mobile unit 22, as well as the target position C and target area Q.

[0281] Furthermore, the display screen 60E shows the status of the moving object 60E4, an image 60E5 representing the amount the brake pedal is pressed on the moving object 22, an image 60E6 representing the steering angle of the moving object 22, alert information 60E7, the vehicle speed of the moving object 22 60E8, the amount the accelerator pedal is pressed on the moving object 22 60E9, a call button 60E10, and the controller connection status 60E11.

[0282] Figures 8F and 8G are enlarged schematic diagrams of images representing the accelerator pedal depression amount 60E9 of the mobile body 22 shown in Figure 8E. Figure 8F shows an example where the accelerator pedal depression amount and the amount converted from vehicle speed to accelerator pedal depression amount are displayed. Figure 8G shows an example where the accelerator pedal depression amount is displayed as is. The output control unit 35B displays images (60E9a, 60E9b) representing the accelerator amount input to the mobile body 22, in a display format corresponding to the magnitude of the accelerator pedal depression amount of the mobile body 22.

[0283] Figure 8H is an enlarged schematic diagram of image 60E5, which represents the amount of brake pedal depression of the mobile body 22 shown in Figure 8E. The output control unit 35B displays image 60E5, which represents the amount of depression expressed by the brake pedal depression detection sensor of the mobile body 22.

[0284] Figures 8I and 8J are enlarged schematic diagrams of image 60E6, which represents the steering angle of the mobile body 22 shown in Figure 8E. The output control unit 35B displays images (60E6a, 60E6b) representing the steering angle of the mobile body 22 by, for example, representing an icon image of a handle that represents the steering angle of the mobile body 22, with the opening portion representing a part of the handle rotated in a direction and angle corresponding to the steering direction and steering angle. Figure 8J shows an image representing the state of the handle in Figure 8I, rotated to the right.

[0285] Returning to Figure 8E, let's continue the explanation. Operator X1 remotely controls the mobile body 22 by operating the input unit 32 while viewing the display screen 60E. More specifically, the remote control unit 35C of the operator terminal 30 transmits remote control information corresponding to the operation instructions for the mobile body 22 input by operator X1 in response to the operation of the input unit 32, along with the mobile body ID of the mobile body 22, to the information processing device 10. The information processing device 10 transmits the remote control information to the mobile body 22 identified by the mobile body ID. The mobile body 22 moves in response to the remote control by operator X1 by controlling the drive unit 22B based on the remote control information. Thus, the mobile body 22 is remotely controlled by operator X1 operating the operator terminal 30.

[0286] Furthermore, while the operator X1 is remotely controlling the mobile body 22, the output control unit 35B may output a display screen 60 with the captured video V arranged according to the type of mobile body 22 to the output unit 33. The output control unit 15C of the information processing device 10 may output display image information, which further includes the type of mobile body 22, to the operator terminal 30.

[0287] Figure 8K is a schematic diagram of an example of the display screen 60K when the mobile body 22 is a small robot. The display screen 60K is an example of the display screen 60 and the display screen 50.

[0288] If the mobile body 22 is a small robot, the output control unit 35B, for example, places the map image M and the rear view image V2 at the top of the display screen 60K, the front view image V1 in the center of the display screen 60K, and the left view image V3 and the right view image V4 to the left and right of the front view image V1, respectively. Note that the display screen 60K may not include the map image M.

[0289] Figure 8L is a schematic diagram of an example of the display screen 60L when the mobile body 22 is a towing truck. The display screen 60L is an example of the display screen 60 and the display screen 50.

[0290] If the mobile body 22 is a towing truck, the output control unit 35B may, for example, place the map image M, the forward overhead view image V5, and the rear view image V2 at the top of the display screen 60L, place the forward view image V1 in the center of the display screen 60K, and place the left view image V3 and the right view image V4 to the left and right of the forward view image V1, respectively. The forward overhead view image V5 is a captured image V taken from above the position where the camera unit 26 is installed on the mobile body 22, showing the front of the mobile body 22. The forward overhead view image V5 can be generated by the information processing device 10 or the operator terminal 30 using a known method with the captured image V. The display screen 60L may not include the map image M. The output control unit 35B may also display the forward overhead view image V5 and the rear overhead view image in a switchable manner. The rear overhead view image is a captured image V taken from above the position where the camera unit 26 is installed on the mobile body 22, showing the rear of the mobile body 22. The rear overhead view image can be generated by the information processing device 10 or the operator terminal 30 using a known method with the captured video V.

[0291] Figure 8M is a schematic diagram of an example of the display screen 60M when the mobile body 22 is a large towing truck. The display screen 60M is an example of the display screen 60 and the display screen 50.

[0292] If the mobile unit 22 is a large towing truck, the output control unit 35B places, for example, the map image M and the rear view image V2 at the top of the display screen 60M. The output control unit 35B also places the front view image V1 in the center of the display screen 60K, and arranges the left view image V3 and the left rear view image V3b vertically to the left of the front view image V1. The output control unit 35B also arranges the right view image V4 and the right rear view image V4b vertically to the right of the front view image V1. The display screen 60M may not include the map image M. The output control unit 35B may also arrange the left view image V3 and the left rear view image V3b vertically in the opposite configuration to that shown in Figure 8M. Similarly, the output control unit 35B may also arrange the right view image V4 and the right rear view image V4b vertically in the opposite configuration to that shown in Figure 8M.

[0293] Figure 8N is a schematic diagram of another example of display screen 60N when the mobile body 22 is a large towing truck. Display screen 60N is an example of display screen 60 and display screen 50.

[0294] If the mobile unit 22 is a large towing truck, the output control unit 35B may, for example, place the map image M, the forward overhead view image V5, and the rear view image V2 at the top of the display screen 60N. The output control unit 35B may also place the forward view image V1 in the center of the display screen 60K, and arrange the left view image V3 and the left rear view image V3b vertically to the left of the forward view image V1. The output control unit 35B may also arrange the right view image V4 and the right rear view image V4b vertically to the right of the forward view image V1. The display screen 60N may not include the map image M. The output control unit 35B may also arrange the left view image V3 and the left rear view image V3b vertically in the opposite configuration to that shown in Figure 8N. Similarly, the output control unit 35B may also arrange the right view image V4 and the right rear view image V4b vertically in the opposite configuration to that shown in Figure 8N.

[0295] Figure 8O is a schematic diagram of another example of display screen 60O when the mobile body 22 is a small robot. Display screen 60O is an example of display screen 60 and display screen 50.

[0296] If the mobile body 22 is a small robot, the output control unit 35B may, for example, place the map image M and the rear view image V2 at the top of the display screen 60O. The output control unit 35B may also place the front view image V1 in the center of the display screen 60K. The output control unit 35B may also switch between displaying the rear overhead view image V6 and the front overhead view image V5.

[0297] Figure 8P is a schematic diagram of another example of display screen 60P when the mobile body 22 is a large towing truck. Display screen 60P is an example of display screen 60 and display screen 50.

[0298] If the mobile unit 22 is a large towing truck, the output control unit 35B places, for example, a map image M, a forward-view image V5, a rear view image V2, and an in-vehicle image V7 at the top of the display screen 60P. The in-vehicle image V7 is a captured image V taken by the camera unit 26 that photographs the interior of the mobile unit 22. For example, the in-vehicle image V7 is a captured image V of the driver's seat of the mobile unit 22.

[0299] Furthermore, the output control unit 35B positions the front image V1 in the center of the display screen 60K, and arranges the left image V3 and left rear image V3b vertically to the left of the front image V1. The output control unit 35B also arranges the right image V4 and right rear image V4b vertically to the right of the front image V1. Note that the display screen 60P may not include the map image M. Also, the vertical arrangement of the left image V3 and left rear image V3b in the output control unit 35B may be reversed from the configuration shown in Figure 8P. Similarly, the vertical arrangement of the right image V4 and right rear image V4b in the output control unit 35B may be reversed from the configuration shown in Figure 8P.

[0300] Figure 8Q is a schematic diagram of another example of display screen 60Q when the mobile body 22 is a large towing truck. Display screen 60Q is an example of display screen 60 and display screen 50.

[0301] If the mobile unit 22 is a large towing truck, the output control unit 35B arranges, for example, the map image M, the front overhead view image V5, the rear view image V2, and the in-vehicle image V7 at the top of the display screen 60Q. The output control unit 35B also arranges the front view image V1 in the center of the display screen 60K and the left view image V3 to the left of the front view image V1. The output control unit 35B also arranges the right view image V4 to the right of the front view image V1. The display screen 60P may not include the map image M. The output control unit 35B may also change the left-right display positions of the front overhead view image V5, the rear overhead view image V6, and the in-vehicle image V7.

[0302] Figure 8R is a schematic diagram of an example of the display screen 60R when the mobile body 22 is a specific vehicle type. The display screen 60R is an example of the display screen 60 and the display screen 50.

[0303] If the mobile body 22 is of a specific vehicle type, or regardless of the vehicle type of the mobile body 22, the output control unit 35B may display the display screen 60R. The output control unit 35B, for example, places the left image V3, the rear image V2, and the right image V4 at the top of the display screen 60R. The output control unit 35B also maximizes the front image V1 and places it in the center of the display screen 60R.

[0304] Figure 8S is a schematic diagram of an example of display screen 60S. Display screen 60S is an example of display screen 60 and display screen 50.

[0305] The output control unit 35B may display a display screen 60S on the left side of the display screen 60S, and on the right side of the display screen 60S, it may display a display area on one of the display screens 60 (display screens 60K to 60R) shown in Figures 8K to 8R, excluding the map image M.

[0306] (Modification 3) In this modification, the display screen 50 output to the output unit 33 of the operator terminal 30 when the mobile unit 22 is remotely controlled by operator X1 will be described in detail. In this modification, the display screen 50 output to the output unit 33 of the operator terminal 30 will be referred to as the display screen 70. The display screen 70 is an example of the display screen 50.

[0307] As described in the above embodiment, the output control unit 35B of the operator terminal 30 outputs the display screen 70, which is represented by the display screen information received from the information processing device 10, to the output unit 33.

[0308] Figure 9A is a schematic diagram of an example of the display screen 70A. The display screen 70A is an example of the display screen 70 and display screen 50 displayed on the output unit 33 of the operator terminal 30.

[0309] Figure 9A is an example of the display screen 70A shown on the output unit 33 when the mobile unit 22 is remotely controlled by operator X1. Display screen 70A is an example of display screen 50.

[0310] The display screen 70A shows captured video footage V (forward video V1, rear video V2, left video V3, right video V4) of the surrounding area of ​​the mobile object 22 that is the target of remote control. The driving path R and vehicle width guide G are superimposed on the captured video footage V. In addition, a detection frame F4 indicating object detection is superimposed on the captured video footage V in the area of ​​an object detected by the sensor 22A that is captured in the captured video footage V.

[0311] The top of the display screen 70A shows information 70A1 representing the mobile unit ID, name of the mobile unit 22, and the travel area of ​​the mobile unit 22. The top of the display screen 70A also shows the mobile unit status 70A2, vehicle status 70A3, and alert information 70A4 of the mobile unit 22.

[0312] On the right side of the display screen 70A, an image representing the positional relationship between the moving body 22 and various objects present around the moving body 22, as shown by the detection result S, and sensor-related information 70A6 are displayed.

[0313] On the map image M of the map data, images representing the target location C, the location P of the mobile body 22, the target area Q, and the travel route R are superimposed and displayed, similar to the embodiment described above. In addition, the display screen 70A displays remaining distance information 70A5, which represents the remaining distance from the location P of the mobile body 22 to the target location C.

[0314] On either the left or right side of the display area of ​​the forward image V1, on the side corresponding to the direction of lane change indicated by the turn signal of the moving body 22, an image 70A7 is displayed indicating that the turn signal for lane change in that direction is flashing on the moving body 22.

[0315] Figure 9B is an example of the display screen 70B that is displayed on the output unit 33 when the mobile unit 22 is remotely controlled by operator X1. Display screen 70B is an example of display screen 50.

[0316] Display screen 70B is the same as display screen 70B except that the display format of the travel route R is different. The image representing the travel route R may be linear as shown in Figure 9A, or it may be a strip with a width corresponding to the width of the mobile body 22 as shown in Figure 9B.

[0317] Figure 9C is a schematic diagram of an example of a display screen 70C that is displayed on the output unit 33 when the mobile unit 22 is remotely controlled by operator X1. Display screen 70C is an example of display screen 70 and display screen 50.

[0318] The display screen 70C includes a display area for each of the mobile units 22 that are remotely controlled by operator X1. In the display area for each mobile unit 22, the mobile unit ID 70C1 of the mobile unit 22 is displayed in a display color corresponding to the color of the mobile unit 22. The display screen 70C includes an instruction button 70C2 for instructing the mobile unit 22 to output sound. When operator X1 operates the instruction button 70C2, the sound represented by the sound information associated with the instruction button 70C2 is output from the speaker of the mobile unit 22.

[0319] The display screen 70C also includes a pause button 70C3, a destination setting button 70C4, and an error message display area 70C5.

[0320] The pause button 70C3 is an image area operated by operator X1 when pausing the movement of the mobile body 22. The destination setting button 70C4 is an image area operated by operator X1 when setting the destination position C of the mobile body 22. When the display area of ​​the destination setting button 70C4 is operated by operator X1, the output control unit 35B displays the display screen 70D shown in Figure 9D on the output unit 33.

[0321] Figure 9D is a schematic diagram of an example of the display screen 70D. The display screen 70D is an example of the display screens 70 and 50. The display screen 70D displays, for example, a list of destination locations C registered in the destination location management DB 14C. Operator X1 can set the next destination location C of the mobile body 22 by selecting the desired destination location C. Operator X1 can also cancel the currently set destination location C by operating the destination setting cancellation button included in the display screen 70D.

[0322] Returning to Figure 9C, let's continue the explanation. The error message display area 70C5 displays the error details if an error occurs due to an operation of any of the display areas included in the display screen 70C by operator X1.

[0323] The display screen 70C also displays the operator X1's name 70C12, the mobile unit 22's driving status 70C6, the safety stop release button 70C9, and the mobile unit 22's vehicle status 70C14. The mobile unit 22's driving status 70C6 is information that indicates, for example, the location where the mobile unit 22 is moving, whether or not the mobile unit 22 is moving, etc. The mobile unit 22's vehicle status 70C14 is information that indicates the movement state. If the mobile unit 22's movement state is an error, the error code and error details are displayed in the mobile unit 22's vehicle status 70C14 display field. The safety stop release button 70C9 is an image area operated by operator X1 when instructing to safely stop the mobile unit 22 or to release the safe stop state of the mobile unit 22.

[0324] The display screen 70C also includes a vehicle selection button 70C7, a main unit remote control button 70C8, an error clear button 70C10, and a malfunction stop release button 70C13. The vehicle selection button 70C7 is a button image area operated by operator X1 when remote control of the mobile unit 22 is initiated. The main unit remote control button 70C8 is a button image area operated by operator X1 when the mobile unit 22 is made operable by the remote control provided on the mobile unit 22. The error clear button 70C10 is a button image area operated by operator X1 when an error that has occurred in the mobile unit 22 is cleared. The malfunction stop release button 70C13 is a button image operated by operator X1 when the mobile unit 22 is released from a malfunction-induced stop state, and becomes operable when the mobile unit 22 is stopped due to a malfunction.

[0325] Figure 9E is a schematic diagram of an example of the display screen 70E. The display screen 70E is an example of the display screen 70 and display screen 50 displayed on the output unit 33 of the operator terminal 30.

[0326] The display screen 70E displays the captured video V (forward video V1, rear video V2, left video V3, right video V4) for each of the multiple mobile bodies 22 that are the target of remote control. In addition, the display area for each mobile body 22 is provided with display fields 70E1 for the travel area and mobile body ID, 70E2 for the target position C, 70E3 for the mobile body status, 70E6 for alert information, and 70E7 for the name of operator X1, surrounding the display area of ​​the captured video V. Furthermore, each display area of ​​the captured video V for each mobile body 22, namely forward video V1, rear video V2, left video V3, and right video V4, is associated with a processing target so that various processes are executed when it is operated.

[0327] For example, when operator X1 operates the display panel TP3 of the left-side video V3, the operator terminal 30 outputs remote control information, including a first standardized voice output instruction to the mobile body 22 responsible for that display panel, to the mobile body 22 via the information processing device 10. In this case, the first standardized voice is output from the speaker of the mobile body 22. Also, for example, when operator X1 operates the display panel TP4 of the right-side video V4, the operator terminal 30 outputs remote control information, including a second standardized voice output instruction from the mobile body 22 responsible for that display panel, to the mobile body 22 via the information processing device 10. In this case, the second standardized voice is output from the speaker of the mobile body 22. Also, for example, when operator X1 operates the display panel TP1 of the front-facing video V1, the operator terminal 30 outputs remote control information, including a pause or resumption of movement, to the mobile body 22 responsible for that display panel, to the information processing device 10. In this case, the remote control information is transmitted from the information processing device 10 to the mobile body 22, causing the mobile body 22 to be remotely controlled to temporarily stop or resume driving.

[0328] Furthermore, the display area of ​​each mobile unit 22 may include a display area 70E9 that shows the status of the turn signal of the mobile unit 22. In addition, the captured video V included in the display area of ​​each mobile unit 22 may have a guideline 70E5 superimposed on it that shows an approximate distance from the mobile unit 22.

[0329] Figure 9F is a schematic diagram of an example of the display screen 70F. The display screen 70F is an example of the display screen 70 and display screen 50 displayed on the output unit 33 of the operator terminal 30.

[0330] The display screen 70F displays the captured video V (front video V1, rear video V2, left video V3, right video V4) for each of the multiple mobile bodies 22 that are the target of remote control. In addition, the display area of ​​each mobile body 22 is provided with display fields 70F1 around the display area of ​​the captured video V for the mobile body 22's travel area, mobile body ID, mobile body status, and communication delay of the mobile body 22. In addition, the display area of ​​each mobile body 22 is provided with an emergency stop button 70F2 for instructing the mobile body 22 to make an emergency stop, and a start / end button 70F5 for instructing the start or end of remote control of the mobile body 22, both around the display area of ​​the captured video V. Operator X1 can make the mobile body 22 make an emergency stop by operating the emergency stop button 70F2. Operator X1 can also start or end the remote control of the mobile body 22 (instructing the mobile body 22 to travel) by operating the start / end button 70F5. In this specification, "remote control" refers to both giving driving instructions to the mobile unit 22 and remote control using the steering wheel, accelerator, brakes, etc.

[0331] Furthermore, in the display area of ​​each mobile body 22 on the display screen 70F, an alert information display area 70F4 and a remote control screen transition instruction button 70F3 are displayed around the display area of ​​the captured video V. When operator X1 operates the remote control screen transition instruction button 70F3 of the desired mobile body 22, the output control unit 35B should display the display screen 70 of the display screen 70A or display screen 70B (see Figures 9A and 9B) to be displayed when the mobile body 22 is remotely controlled on the output unit 33.

[0332] Furthermore, the display screen 70F displays a list of information 70F6 for each of the multiple mobile bodies 22 that are remotely controlled by operator X1. The output control unit 35B may group the information for each mobile body 22 according to the travel area in which the mobile body 22 travels, and then display the list of information 70F6 for the mobile bodies 22 on the display screen 70F.

[0333] Figure 9G is a schematic diagram of an example of display screen 70G. Display screen 70G is an example of display screen 70 and display screen 50.

[0334] Display screen 70G is a display screen 70 in which information regarding multiple mobile objects 22 that are remotely controlled by operator X1 is superimposed on a map image M.

[0335] On the display screen 70G, images representing the respective positions P, target position C, target area Q, and travel path R of one or more mobile bodies 22 that are the target of remote control by operator X1 are superimposed on the map image M. For example, the image representing the position P of a mobile body 22 is represented by an image such as an arrow indicating the direction of travel of the mobile body 22. The travel path R is represented by a line image with an arrow indicating the direction of travel of the mobile body 22.

[0336] Furthermore, the display screen 70G includes alert information 70G1 representing alarms or warnings occurring in each of the multiple mobile units 22.

[0337] Furthermore, the display screen 70G includes the current time 70G2, a logout button 70G3 operated when logging out, and a display switching button 70G4 for receiving instructions to switch to other display screens 70. The display screen 70G may also include the location of the mobile unit 22 and the number of currently occurring alerts.

[0338] Figures 9H and 9I are schematic diagrams of examples of display screens 70H and 70I. Display screens 70H and 70I are examples of display screens 70 and 50.

[0339] Display screen 70H is an input screen for operator X1 to input memo information for the mobile units 22. Operator X2 can input information noticed during remote control, error details, and error causes for each mobile unit 22 via display screen 70H by operating the input unit 32. The output control unit 35B may also display a display screen 70I for selecting error details. Operator X1 can input memo information for each mobile unit 22 by selecting the desired error details from the list of error details displayed on display screen 70I.

[0340] The control unit 35 of the operator terminal 30, upon receiving memo information input for the mobile unit 22, outputs to the information processing device 10 the received memo information, the mobile unit ID of the mobile unit 22 corresponding to the memo information, and the event (alert) that occurred on the mobile unit 22 (e.g., safety confirmation request / vehicle approach / remote control switching / emergency stop, etc.). The information processing device 10 only needs to store the memo information and the event for each mobile unit 22, associating them with the received mobile unit ID of the mobile unit 22.

[0341] Next, an example of the hardware configuration of the information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 in the above embodiment and modified examples will be described.

[0342] Figure 10 is a hardware configuration diagram of an example of an information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 of the above embodiment and modified examples.

[0343] The information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 of the above embodiment and modified examples are interconnected via a bus 88, with a CPU (Central Processing Unit) 80, ROM (Read Only Memory) 82, RAM (Random Access Memory) 84, and I / F 86, etc., and have a hardware configuration that utilizes a normal computer.

[0344] The CPU 80 is a computing device that controls the information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 of the above embodiment and modified versions. The ROM 82 stores programs and the like that realize information processing by the CPU 80. The RAM 84 stores data necessary for various processes performed by the CPU 80. The I / F 86 is an interface connected to the storage unit, input unit, output unit, sensor, and communication unit, etc., for sending and receiving data.

[0345] In the above embodiment and modified information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40, the CPU 80 reads a program from ROM 82 onto RAM 84 and executes it, thereby realizing each of the above-mentioned functional units on the computer.

[0346] Furthermore, the programs for executing the above-mentioned processes performed by the information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 in the above-described embodiment and modified version may be stored in an HDD (hard disk drive). Alternatively, the programs for executing the above-mentioned processes performed by the information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 in the above-described embodiment and modified version may be pre-installed and provided in ROM 82.

[0347] Furthermore, the programs for executing the above-described processes performed by the information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 of the above-described embodiments and modifications may be provided as computer program products by being stored in an installable or executable file format on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD). Alternatively, the programs for executing the above-described information processing performed by the information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 of the above-described embodiments and modifications may be stored on a computer connected to a network such as the Internet and provided by allowing downloads via the network. Furthermore, the programs for executing the above-described information processing performed by the information processing device 10, mobile unit 22, operator terminal 30, and worker terminal 40 of the above-described embodiments and modifications may be provided or distributed via a network such as the Internet.

[0348] Although embodiments have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0349] Furthermore, this technology can also be configured as follows: (1) An information processing method performed by an information processing device, comprising: a receiving step of receiving mobile information including the position of a mobile object; and a determination step of determining that the mobile object is located at the specified position if the position of the mobile object satisfies a specific determination condition that the mobile object is within a specific area including a specific position of the mobile object. (2) The information processing method according to (1), wherein the receiving step receives the mobile information including the position of the mobile object and the mobile state of the mobile object, and the determination step determines that the mobile object is located at the specific position if the position of the mobile object is within the specific area and the mobile state indicates that it is stopped, satisfying the specific determination condition. (3) The information processing method according to (1) or (2), comprising an output control step of outputting a display screen showing the positional relationship between the specific area and the mobile object to the operator terminal of an operator remotely controlling the mobile object. (4) The information processing method according to (3), wherein the output control step outputs the display screen in the first display mode to the operator terminal when the position of the moving body is within the specific area, and outputs the display screen in the second display mode to the operator terminal when the position of the moving body is outside the specific area. (5) The information processing method according to (3) or (4), wherein the output control step outputs the display screen to the operator terminal, wherein an image representing the position of the moving body and an image representing the specific area are superimposed on at least one of a map image representing the driving environment of the moving body and a captured video of the surrounding area of ​​the moving body. (6) The information processing method according to any one of (3) to (5), wherein the output control step outputs the display screen to the operator terminal, wherein an image representing the driving route of the moving body is superimposed on at least one of a map image representing the driving environment of the moving body and a captured video of the surrounding area of ​​the moving body. (7) The information processing method according to any one of (3) to (6), wherein the output control step outputs the display screen, which includes remaining distance information representing the remaining distance from the position of the moving body to the specific position, to the operator terminal.(8) The information processing method according to any one of (3) to (7), wherein the output control step determines that the moving body is located at the specific position, and outputs information indicating that the moving body is located at the specific position to at least one of the operator terminal and the operator terminal of the worker performing work using the moving body in real space. (9) The information processing method according to any one of (3) to (8), wherein the determination step determines whether the moving body is located at the specific position if the specific determination condition is met and a specific determination instruction is received again from the operator terminal via the display screen. (10) The information processing method according to any one of (1) to (9), wherein the specific position is the target position of the moving body, the specific area is the target area including the target position, the specific determination condition is an arrival determination condition indicating that the position of the moving body is within the target area including the target position of the moving body, and the determination step determines that if the position of the moving body satisfies the arrival determination condition, the moving body is located at the target position and has arrived at the target position. (11) The information processing method according to any one of (1) to (10), wherein the specific position is the departure position of the moving body, the specific area is the departure area including the departure position, the specific determination condition indicates that the position of the moving body is within the departure area including the departure position of the moving body, and the determination step determines that if the position of the moving body satisfies the specific determination condition, the moving body is located at the departure position and outputs a request to start driving information to the operator terminal of the operator remotely controlling the moving body. (12) The information processing method according to any one of (1) to (11), wherein the specific region is a circular region with a predetermined radius centered at the specific position in real space. (13) The information processing method according to any one of (1) to (12), wherein the specific region is a region surrounding an object placed at the specific position.(14) An information processing program for a computer to execute, comprising: a receiving step of receiving mobile information including the position of a mobile object; and a determination step of determining that the mobile object is located at the specific position if a specific determination condition is met that the position of the mobile object is within a specific area including a specific position of the mobile object. (15) An information processing device comprising: a communication control unit that receives display screen information including mobile information including the position of a mobile object, and information indicating that a specific determination condition is met that the position of the mobile object is within a specific area including a specific position of the mobile object, and that it has been determined that the mobile object is located at the specific position; and an output control unit that outputs a display screen represented by the display screen information to an output unit.

[0350] 10 Information processing device 15B Determination unit 15C Output control unit 30 Operator terminal 35B Output control unit

Claims

1. An information processing method performed by an information processing device, comprising: a receiving step of receiving information about a moving object, including the position of the moving object; and a determination step of determining that the moving object is located at the specified position if the position of the moving object satisfies a specific determination condition that the position of the moving object is within a specific area including a specific position of the moving object.

2. The information processing method according to claim 1, wherein the receiving step receives the mobile body information including the position of the mobile body and the mobile body state of the mobile body, and the determination step determines that the mobile body is located at the specific position if the specific determination conditions are met, namely that the position of the mobile body is within the specific area and the mobile body state is stopped.

3. The information processing method according to claim 1, further comprising an output control step of outputting a display screen showing the positional relationship between the specific area and the mobile body to the operator terminal of an operator remotely controlling the mobile body.

4. The information processing method according to claim 3, wherein the output control step involves outputting the display screen in the first display mode to the operator terminal when the position of the moving body is within the specified area, and outputting the display screen in the second display mode to the operator terminal when the position of the moving body is outside the specified area.

5. The information processing method according to claim 3, wherein the output control step outputs to the operator terminal a display screen in which an image representing the position of the moving body and an image representing the specific area are superimposed on at least one of a map image representing the driving environment of the moving body and a video image of the surroundings of the moving body.

6. The information processing method according to claim 3, wherein the output control step outputs to the operator terminal a display screen in which an image representing the travel route of the mobile body is superimposed on at least one of a map image representing the travel environment of the mobile body and a video image of the surroundings of the mobile body.

7. The information processing method according to claim 3, wherein the output control step outputs the display screen, which includes remaining distance information representing the remaining distance from the position of the moving body to the specific position, to the operator terminal.

8. The information processing method according to claim 3, wherein, when the output control step determines that the moving body is located at the specific position, it outputs information indicating that the moving body is located at the specific position to at least one of the operator terminal and the worker terminal of the worker performing work using the moving body in real space.

9. The information processing method according to claim 3, wherein the determination step re-determines whether the moving body is located at the specified position if the specified determination condition is met and a specified determination instruction is received again from the operator terminal via the display screen.

10. The information processing method according to claim 1, wherein the specified position is the target position of the moving body, the specified area is a target area including the target position, the specified determination condition is an arrival determination condition indicating that the position of the moving body is within the target area including the target position of the moving body, and the determination step determines that, if the position of the moving body satisfies the arrival determination condition, the moving body is located at the target position and has arrived at the target position.

11. The information processing method according to claim 1, wherein the specified position is the departure position of the mobile body, the specified area is a departure area including the departure position, the specified determination condition indicates that the position of the mobile body is within the departure area including the departure position of the mobile body, and the determination step determines that the mobile body is located at the departure position if the position of the mobile body satisfies the specified determination condition, and outputs a request to start driving information to the operator terminal of the operator remotely controlling the mobile body.

12. The information processing method according to claim 1, wherein the specific region is a circular region with a predetermined radius centered at the specific position in real space.

13. The information processing method according to claim 1, wherein the specific area is the area surrounding an object placed at the specific location.

14. An information processing program for execution by a computer, comprising: a receiving step of receiving information about a moving object, including the position of the moving object; and a determination step of determining that the moving object is located at the specified location if the position of the moving object satisfies a specific determination condition that the position of the moving object is within a specific area including a specific location of the moving object.

15. Information processing apparatus comprising: a communication control unit that receives display screen information including mobile body information including the position of a mobile body, and information indicating that a specific determination condition is met in which the position of the mobile body is within a specific area including a specific position of the mobile body, and that it has been determined that the mobile body is located at the specific position; and an output control unit that outputs a display screen represented by the display screen information to an output unit.

Images