Control device, information processing method, information processing program, and information processing system

Abstract

To further improve the operability of systems that operate multiple mobile units. [Solution] A control device comprising: a map creation control unit that controls the creation of environmental map data representing the environment around a mobile body; a map conversion unit that converts the environmental map data created by the mobile body into a predetermined format; a registration unit that registers the environmental map data converted into the predetermined format; and an output unit that outputs the registered environmental map data to another mobile body different from the mobile body.

Description

[[Technical Field]]

[0001] The present invention relates to a control device, an information processing method, an information processing program, and an information processing system. [[Background Art]]

[0002] In recent years, with advances in robot-related technology, development of systems for operating robots that perform autonomous work has been progressing. Such systems are expected to serve as a means to solve labor shortages accompanying population decline.

[0003] For example, by utilizing technologies such as Simultaneous Localization and Mapping (SLAM) for robots, Artificial Intelligence (AI), and map data inside and between buildings, technologies and systems that enable robots to autonomously travel and move between locations inside and between buildings have been developed.

[0004] On the other hand, at present, events occur that are difficult for robots to handle or judge autonomously during work. In such cases, the robot sends a notification requesting manual operation to a remote operator, and responds by operating under the manual control of the remote operator. This allows events that are difficult for the robot to handle or judge to be resolved by manual operation from a remote operator.

[0005] For example, Patent Document 1 below discloses a technology in which when a communication robot determines that it is difficult to respond only by autonomous control, an operator call request is transmitted to a central control device, the central control device selects one or more operators that satisfy the conditions requested by the interlocutor from the interlocutor's voice, and the selected one or more operators remotely operate the robot using an operation terminal.

[0006] Furthermore, Patent Document 2, listed below, discloses a technology for converting the coordinates of an environmental map internally constructed by an autonomous driving device using SLAM into coordinates in a plane rectangular coordinate system used by the user for position management of the autonomous driving device. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2009-90420 [Patent Document 2] Japanese Patent Publication No. 2020-042366 [Overview of the project] [Problems that the invention aims to solve]

[0008] In an operational system that manages multiple autonomous robots of various types, it is envisioned that the environmental map data created by each robot will be registered and managed centrally within the operational system.

[0009] In such cases, the operational system can simplify the management of environmental map data by managing the environmental map data created by each robot in a unified format.

[0010] However, since the format of the environmental map data created by each robot (e.g., map name, map size, or data format) differs for each robot, the technologies described in Patent Documents 1 and 2 require a user (e.g., a remote operator) to manually convert the format of the environmental map data from multiple robots when registering the environmental map data created by multiple robots into an operating system. This increases the burden of format conversion work for the environmental map data and reduces the operability of the system.

[0011] Therefore, the present invention has been made in view of the above problems, and the object of the present invention is to provide a novel and improved control device, information processing method, information processing program, and information processing system that can further improve the operability of a system for operating multiple robots (hereinafter referred to as mobile units). [Means for solving the problem]

[0012] To solve the above problems, according to one aspect of the present invention, a control device is provided, comprising: a map creation control unit that controls the creation of environmental map data representing the environment around a mobile body; a map conversion unit that converts the environmental map data created by the mobile body into a predetermined format; a registration unit that registers the environmental map data converted into the predetermined format; and an output unit that outputs the registered environmental map data to another mobile body different from the mobile body.

[0013] The map conversion unit may convert at least one of the settings of the environmental map data into settings that conform to the predetermined format.

[0014] The output unit may be characterized by outputting the environmental map data to the other mobile device based on a request from the other mobile device to transmit the environmental map data.

[0015] The system may further include a system map conversion unit that converts the environmental map data output to the other mobile device into a format compatible with the other mobile device.

[0016] The system map conversion unit may be characterized by calculating a scaling conversion rate using the environmental map data output to the other mobile device and the environmental map data created by the other mobile device, and converting the environmental map data output to the other mobile device into a format compatible with the other mobile device by scaling it up or down.

[0017] The other mobile device may be characterized by using the environmental map data converted into a format corresponding to the other mobile device.

[0018] The system may further include a state control unit for controlling the driving state of the moving body, characterized in that, in each of the driving states, the processing set for that driving state is consistently executed.

[0019] The system may further include a remote control unit for remotely controlling the operating state of the mobile body.

[0020] The mobile remote control unit may be characterized by operating multiple mobile units.

[0021] Furthermore, in order to solve the above problems, according to another aspect of the present invention, a computer-based information processing method is provided, which includes the steps of: controlling the creation of environmental map data representing the environment around a mobile body; converting the environmental map data created by the mobile body into a predetermined format; registering the environmental map data converted into the predetermined format; and outputting the registered environmental map data to another mobile body different from the mobile body.

[0022] Furthermore, in order to solve the above problems, according to another aspect of the present invention, an information processing program is provided that causes a computer to function as: a map creation control unit that controls the creation of environmental map data representing the environment around a mobile body; a map conversion unit that converts the environmental map data created by the mobile body into a predetermined format; a registration unit that registers the environmental map data converted into the predetermined format; and an output unit that outputs the registered environmental map data to another mobile body different from the mobile body.

[0023] Furthermore, in order to solve the above problem, according to another aspect of the present invention, there is provided an information processing system including a moving body and a control device, the information processing system comprising: a map creation control unit that controls creation of environment map data representing an environment surrounding the moving body; a map conversion unit that converts said environment map data in a format created by said moving body into a predetermined format; a registration unit that registers said environment map data converted into said predetermined format; and an output unit that outputs the registered said environment map data to another moving body different from said moving body. Effects of the Invention

[0024] As explained above, according to the present invention, environment map data created by each moving body can be automatically converted into a unified format, and the converted environment map data can be registered in an operation system. This eliminates the need for a user to manually perform manual conversion work of environment map data, thereby reducing the work burden on the user when registering environment map data in the operation system. In addition, since environment map data can be efficiently shared and utilized among a plurality of moving bodies, it is possible to further improve the operability of the system. Brief Description of the Drawings

[0025] [Figure 1] It is a block diagram showing the functional configuration of a control device according to an embodiment of the present invention. [Figure 2] It is a block diagram showing the detailed functional configuration of a moving body control unit and a moving body storage unit according to the same embodiment. [Figure 3] It is a block diagram showing the detailed functional configuration of a moving body operation system unit according to the same embodiment. [Figure 4] It is a flowchart explaining the operation of the moving body according to the same embodiment. [Figure 5] It is a flowchart showing the flow of interrupt processing according to the same embodiment. [Figure 6] It is a flowchart showing the flow of temporary stop processing according to the same embodiment. [Figure 7]This flowchart illustrates the flow of the abnormal shutdown process according to the same embodiment. [Figure 8] A flowchart illustrating the flow of the map creation process according to the same embodiment. [Figure 9] This is an explanatory diagram showing an example of the conversion of environmental map data by the map conversion unit according to the present embodiment. [Figure 10] This flowchart shows the operation flow of the mobile device operation system unit when receiving environmental map data from a mobile device according to the same embodiment. [Figure 11] This flowchart shows the flow of manual driving in the manual driving process and map creation process according to the embodiment of the same principle. [Figure 12] A flowchart illustrating the flow of autonomous driving processing according to the same embodiment is shown. [Figure 13] This is a flowchart illustrating the operation flow of the autonomous driving control unit according to the same embodiment. [Figure 14] This is a flowchart illustrating the flow of the remote driving process according to the same embodiment. [Figure 15] This is a flowchart illustrating the operation flow of the remote driving control unit according to the same embodiment. [Figure 16] This is a block diagram showing the functional configuration of the control device according to the first modified example. [Figure 17] This is a block diagram showing the detailed functional configuration of the mobile operation system unit according to the first modified example. [Figure 18] This is a flowchart showing the flow of the map creation process related to the first modified example. [Figure 19] This flowchart shows the operation flow of the mobile device operation system unit when receiving environmental map data related to the first modified example from a mobile device. [Figure 20] This is a block diagram showing the functional configuration of the control device according to the second modified example. [Figure 21] This is a block diagram showing the detailed functional configuration of the mobile operation system unit according to the second modified example. [Figure 22] This is a flowchart illustrating the operation of the moving body in the second modified example. [Figure 23] This flowchart shows the operation flow of the mobile device operation system when receiving environmental map data from a mobile device, relating to the second modified example. [Figure 24] This is a flowchart illustrating the flow of map reception processing related to the second modified example. [Figure 25] This is an explanatory diagram showing an example of the conversion of environmental map data by the system map conversion unit according to the second modified example. [Figure 26] This is a block diagram showing the functional configuration of the control device according to the third modified example. [Figure 27] This is a flowchart illustrating the operation of the moving body in the third modified example. [Figure 28] This is a flowchart illustrating the flow of the map reception conversion process related to the third modified example. [Figure 29] This is a block diagram showing an example of the hardware configuration of an information processing device that embodies the mobile operation system unit according to the same embodiment. [Modes for carrying out the invention]

[0026] Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings. In this specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0027] <1. Example Configuration> Referring to Figure 1, the overall configuration of the control device 1 according to one embodiment of the present invention will be described. Figure 1 is a block diagram showing the functional configuration of the control device 1 according to this embodiment.

[0028] As shown in Figure 1, the control device 1 includes, for example, N mobile units 100-1, 100-2, ... 100-N (collectively referred to as mobile unit 100), M remote control terminals 200-1, 200-2, ... 200-M (collectively referred to as remote control terminal 200), and a mobile unit operation system unit 300. M and N are any integers of 1 or more that are independent of each other. However, in order to efficiently operate the N mobile units 100-1, 100-2, ... 100-N with the M remote control terminals 200-1, 200-2, ... 200-M, M may be an integer smaller than N.

[0029] N mobile units 100-1, 100-2, ... 100-N are connected to M remote control terminals 200-1, 200-2, ... 200-M via a network 400, enabling them to send and receive data to and from each other. Each of the M remote control terminals 200-1, 200-2, ... 200-M is operated by M operators OP-1, OP-2, ... OP-M, and can remotely control each of the mobile units 100-1, 100-2, ... 100-N.

[0030] The mobile unit 100 is a robot capable of autonomous movement and can also be moved by remote control from the remote control terminal 200. The mobile unit 100 can, for example, perform autonomous movement, connect to the mobile unit operation system unit 300, notify the remote control terminal 200, move by remote control from the remote control terminal 200, and transmit data in response to requests from the remote control terminal 200.

[0031] Here, "robot" may also refer to a machine or device defined as a so-called "artificial human," a "device that performs actions or tasks similar to those of a human," or a "device that performs actions or tasks through computer control based on instructions." Furthermore, "robot" may also refer to a machine or device defined, for example, as "a machine used in industry that has an automatic control manipulation function or a movement function and can perform various tasks by program" (JIS B 0134-1998), "a robot that serves humans" (JIS B 0187:2005), or "an intelligent machine system having three elemental technologies: a sensor, an intelligent / control system, and a drive system" (Ministry of Economy, Trade and Industry Robot Policy Study Group).

[0032] In the control device 1 according to this embodiment, the mobile unit 100 is used for various purposes or operations. The mobile unit 100 may be used, for example, for patrolling, monitoring, or security at a facility under security; for inspection of public facilities or equipment; for equipment inspection, material transport, or manufactured parts transport at a manufacturing site; for building parts transport or construction status confirmation at a construction site; for passenger transport, luggage transport, tourist guidance, or luggage inspection in the passenger transport sector; or for cargo inspection, cargo vehicle inspection, or cargo transport in the logistics sector.

[0033] The remote control terminal 200 is a terminal device used by the operator OP when remotely controlling the mobile unit 100. The remote control terminal 200 may include, for example, a personal computer (PC), tablet, microcomputer, or smartphone, and peripheral devices connected to the terminal. The remote control terminal 200 is capable of outputting data regarding the status of the mobile unit 100, remotely issuing movement instructions to the mobile unit 100 based on the operator OP's input operations, outputting notifications from the mobile unit 100, or monitoring the status of the operator OP. The remote control terminal 200 can enable the operator OP to understand the environment surrounding the mobile unit 100 by presenting the operator OP with an image that includes the mobile unit 100 and the environment surrounding the mobile unit 100. This allows the operator OP to remotely control the mobile unit 100 while confirming the environment surrounding the mobile unit 100.

[0034] The mobile unit operation system 300 is the device on which the main system that manages the entire control device 1 operates. The mobile unit operation system 300 manages information on the mobile unit 100, the remote control terminal 200, and the operator OP, manages the operating status of the mobile unit 100, manages the connection between the mobile unit 100 and the remote control terminal 200, and manages the operation information of the remote control terminal 200 by the operator OP. Furthermore, the installation location of the mobile unit operation system 300 is not particularly limited. For example, the mobile unit operation system 300 may be installed in the same location as the remote control terminal 200, or it may be installed in a different location from the remote control terminal 200.

[0035] Network 400 is a communication network that interconnects the mobile unit 100, the remote control terminal 200, and the mobile unit operation system unit 300. Network 400 enables the transmission and reception of data between the mobile unit 100, the remote control terminal 200, and the mobile unit operation system unit 300. Network 400 may be a communication network that can adapt to various communication methods, such as the Internet, satellite communication network, mobile communication network, LAN (Local Area Network), or WAN (Wide Area Network).

[0036] The communication method between the mobile unit 100 and the network 400, and the communication method between the remote control terminal 200 and the network 400, may be the same or they may be different. It is preferable that the mobile unit 100, which connects to the network 400 at any location, connects to the network 400 using wireless LAN communication. On the other hand, it is preferable that the remote control terminal 200, which connects to the network 400 at its installation location, connects to the network 400 using wired LAN communication. Various known communication methods can be used for communication between the mobile unit 100, the remote control terminal 200, the mobile unit operation system unit 300, and the network 400.

[0037] Furthermore, with reference to Figure 1, the detailed configurations of the mobile unit 100 and the remote control terminal 200 will be described.

[0038] (Mobile unit 100) As shown in Figure 1, the mobile unit 100 comprises a mobile unit control unit 101, a camera unit 102, a sensor unit 103, a display unit 104, a speaker 105, a microphone 106, a mobile unit 107, a mobile unit LAN hub unit 108, a mobile unit communication unit 109, an AI unit 110, a map conversion unit 114, a mobile unit operation unit 111, a mobile unit storage unit 112, and a battery unit 113.

[0039] The mobile unit control unit 101 processes various data acquired by the mobile unit 100 and controls the overall operation of the mobile unit 100. The mobile unit control unit 101 consists of, for example, software including a program that controls the overall operation of the mobile unit 100, and hardware on which the software is installed. Examples of hardware on which the software is installed include a CPU (Central Processing Unit), RAM (Read Only Memory), and ROM (Read Only Memory). The mobile unit control unit 101 may also be composed of a DSP (Digital Signal Processor), a microprocessor, or an IC (Integrated Circuit) instead of a CPU. It is also possible to create a program that enables the above-mentioned hardware such as the CPU, ROM, and RAM to perform functions equivalent to those of the mobile unit control unit 101. Furthermore, a computer-readable recording medium on which such a program is stored can also be provided.

[0040] The camera unit 102 includes at least one imaging device capable of acquiring images or videos of the environment surrounding the mobile body 100. The camera unit 102 may also include a 360-degree camera capable of acquiring environmental images capturing the entire surroundings of the mobile body 100. Furthermore, the camera unit 102 may include, for example, multiple web cameras or fisheye cameras installed in each direction of the mobile body 100. In such a case, the remote control terminal 200 can generate an environmental image that provides an aerial view of the environment surrounding the mobile body 100 by seamlessly combining the images or videos from the multiple cameras transmitted from the mobile body 100. In addition, the camera unit 102 may include, for example, so-called network cameras installed on the mobile body 100 to capture the front, back, left, and right sides of the mobile body 100. However, the camera unit 102 is not limited to the above. The camera unit 102 may include, for example, other cameras.

[0041] The sensor unit 103 includes at least one type and one or more sensors capable of sensing parameters of the environment surrounding the mobile body 100. The sensor unit 103 may include, for example, a Time of Flight (ToF) sensor, LiDAR (Light Detection and Ranging), or millimeter-wave radar, which are capable of mapping the surrounding environment and estimating the self-position of the mobile body 100. The sensor unit 103 may also include additional sensors capable of sensing, for example, temperature, humidity, illuminance, atmospheric pressure, vibration, position information, proximity of objects, or changes in the tilt of the mobile body 100, a thermal image measurement sensor (so-called thermographic camera), or sensors capable of sensing the generation of smoke, chemical substances, or static electricity. Furthermore, the sensor unit 103 may acquire sensing information about the environment surrounding the mobile body 100 from a sensor device provided outside the mobile body 100 via the mobile body communication unit 109. In such a case, the sensor unit 103 may acquire sensing information from the external sensor device by, for example, 920MHz band multi-hop wireless communication. However, the sensor unit 103 is not limited to the above. The sensor unit 103 may include other sensors. In the above, "sensing" includes not only detecting various parameters of the environment surrounding the mobile body 100, but also further measuring, identifying, and analyzing the detected parameters.

[0042] In this embodiment, the mobile body 100 is described as comprising both a camera unit 102 and a sensor unit 103, but this embodiment is not limited to such examples. The mobile body 100 may, for example, comprise only a camera unit 102, or only a sensor unit 103. If the mobile body 100 comprises only a camera unit 102, the camera unit 102 includes at least two imaging devices. On the other hand, if the mobile body 100 comprises only a sensor unit 103, the sensor unit 103 includes two or more sensors.

[0043] The display unit 104 outputs various information as images based on the control of the mobile unit control unit 101. The display unit 104 may include, for example, a liquid crystal display (LCD) device, an OLED (organic light-emitting diode) display device (organic EL display device), a touch panel display device, or a lamp.

[0044] The speaker 105 outputs sound around the mobile body 100 based on the control of the mobile body control unit 101. The speaker 105 may output, for example, sound effects or synthesized voice corresponding to the movement of the mobile body 100, or the voice of the operator OP, around the mobile body 100. The microphone 106 picks up sounds around the mobile body 100. The microphone 106 may pick up, for example, the voices of people present around the mobile body 100, or ambient sounds around the mobile body 100.

[0045] The moving unit 107 is a moving mechanism that can move the mobile body 100 to any position based on the control of the mobile body control unit 101. The moving unit 107 may be, for example, a two-dimensional moving mechanism of various types such as wheeled, legged, crawler, or air cushion type, a moving mechanism capable of three-dimensional movement, a moving mechanism that can move through the air such as a rotor blade, or a moving mechanism that can move on or underwater such as a screw.

[0046] The mobile LAN hub unit 108 is a network device that transmits and receives data and packets. The mobile LAN hub unit 108 can transmit and receive data and packets between the mobile control unit 101, the AI ​​unit 110, the map conversion unit 114, and the mobile communication unit 109. The mobile LAN hub unit 108 may include, for example, network devices such as a switching hub, a bridging hub, or a router.

[0047] The mobile communication unit 109 is a communication interface for connecting to the network 400. The mobile communication unit 109 may be, for example, a communication interface that can be connected wirelessly or via a wired connection to a base station that can connect to the network 400, or it may be a communication interface that can be connected to the network 400 via a network such as a mobile phone network or a wireless LAN.

[0048] The AI ​​unit 110 performs judgments and decisions based on images or videos of the environment surrounding the mobile body 100 captured by the camera unit 102 and sensing information obtained by the sensor unit 103 from various parameters of the environment surrounding the mobile body 100. The AI ​​unit 110 may, for example, use a machine learning algorithm to perform judgments and decisions based on the environment image and sensing information. Furthermore, the AI ​​unit 110 can recognize the environment surrounding the mobile body 100 based on the environment image and sensing information, and perform movement and actions according to the recognized environment.

[0049] The mobile unit operation unit 111 is an input device that receives input for operations on the mobile unit 100. Input operations that specify the direction of travel, turning direction, and travel speed of the mobile unit 100 are input to the mobile unit operation unit 111. The mobile unit operation unit 111 may include, for example, an input device such as a keyboard, mouse, touch panel, trackpad, directional pad, joystick, controller, or control lever.

[0050] The mobile body storage unit 112 is a recording device that stores various information and images based on the control of the mobile body control unit 101. The mobile body storage unit 112 may store, for example, basic information and map information of the location where the mobile body 100 autonomously travels, environmental map data and travel route of the mobile body 100 created by the mobile body control unit 101, sensing results from the sensor unit 103, log information from the mobile body control unit 101, information regarding the shape of the mobile body 100 (for example, the shape, total length, total width, or height of the mobile body 100), or information regarding the movement characteristics of the mobile body 100 (for example, the movement speed, movement range, or movable area of ​​the mobile body 100).

[0051] The battery unit 113 supplies power to each part of the mobile unit 100. The battery unit 113 may be composed of a secondary battery, such as a lithium-ion secondary battery. In Figure 1, the battery unit 113 is connected to the mobile unit control unit 101, the mobile unit communication unit 109, the AI ​​unit 110, the map conversion unit 114, and the mobile unit 107, but it may be connected to other components of the mobile unit 100 as needed.

[0052] When the map conversion unit 114 transmits the environmental map data created by the mobile control unit 101 of the mobile unit 100 to the mobile unit operation system unit 300, it converts the format of the environmental map data, which is the format of the mobile unit 100, to a predetermined unified format of the mobile unit operation system unit 300.

[0053] (Remote control terminal 200) The remote control terminal 200 comprises a terminal control unit 201, a camera unit 202, a sensor unit 203, a display unit 204, a speaker 205, a microphone 206, a terminal operation unit 207, a terminal LAN hub unit 208, a terminal communication unit 209, a terminal memory unit 210, and a power supply unit 211.

[0054] The terminal control unit 201 processes various data acquired by the remote control terminal 200 and controls the overall operation of the remote control terminal 200. The terminal control unit 201 consists of, for example, software including a program that controls the overall operation of the remote control terminal 200, and hardware on which the software is installed. Examples of hardware on which the software is installed include personal computers, tablets, microcomputers, or smartphones.

[0055] The camera unit 202 includes an imaging device capable of acquiring images or video of the area surrounding the remote control terminal 200. The camera unit 202 may include, for example, one or more webcams, fisheye cameras, or 360-degree cameras that image the operator OP or the area around the operator OP. Furthermore, the camera unit 202 may include, for example, a so-called network camera positioned to capture images of the operator OP or the area around the operator OP. However, the camera unit 202 is not limited to the above and may include other cameras.

[0056] The sensor unit 203 includes at least one type and one or more sensors capable of sensing the environment surrounding the remote control terminal 200 or the parameters of the operator OP. The sensor unit 203 may also include sensors capable of sensing vital signs such as the operator OP's body temperature, heart rate, blood pressure, or respiratory rate. Furthermore, the sensor unit 203 may acquire sensing information about the environment surrounding the remote control terminal 200 or about the operator OP from a sensor device provided outside the remote control terminal 200 via the terminal communication unit 209. However, the sensor unit 203 is not limited to the above. The sensor unit 203 may include other sensors.

[0057] The display unit 204 outputs various information as images based on the control of the terminal control unit 201. The display unit 204 may include, for example, a liquid crystal display device, an OLED display device (organic EL display device), or a touch panel display device.

[0058] The display unit 204 may display images or videos generated using environmental images captured by the mobile body 100, along with various information about the mobile body 100, by the operating system of the mobile body operation system unit 300. The operator OP can understand the environment around the mobile body 100 by viewing the images displayed on the display unit 204. If the camera unit 102 of the mobile body 100 includes a 360-degree camera, the display unit 204 can display an image capturing the entire surroundings of the mobile body 100. If the camera unit 102 of the mobile body 100 includes multiple fisheye cameras provided in each direction of the mobile body 100, the display unit 204 can display an omnidirectional environmental image (an image of the environment around the mobile body 100 viewed from above) by seamlessly combining images captured by the multiple fisheye cameras.

[0059] Speaker 205 outputs sound around the remote control terminal 200 based on the control of the terminal control unit 201. Speaker 205 may output, for example, voice related to the control of the terminal control unit 201, or sound picked up by the microphone 106 of the mobile unit 100 when the mobile unit 100 is being remotely controlled (for example, the voices of people speaking around the mobile unit 100, or ambient sounds of the environment in which the mobile unit 100 is operating). Microphone 206 picks up sounds around the remote control terminal 200. Microphone 206 may pick up, for example, the voice of the operator OP operating the remote control terminal 200.

[0060] The terminal control unit 207 is an input device that receives operations from the operator OP. The terminal control unit 207 receives input operations from the operator OP that specify the direction of travel, turning direction, and travel speed of the mobile body 100. The terminal control unit 207 may include, for example, input devices such as a keyboard, mouse, touch panel, trackpad, directional pad, joystick, controller, or control lever.

[0061] The terminal LAN hub unit 208 is a network device that sends and receives data and packets. The terminal LAN hub unit 208 can send and receive data and packets between the terminal control unit 201 and the terminal communication unit 209. The terminal LAN hub unit 208 may include, for example, network devices such as a switching hub, a bridging hub, or a router.

[0062] The terminal communication unit 209 is a communication interface for connecting to the network 400. The terminal communication unit 209 may be, for example, a communication interface that can connect wirelessly or via a wired connection to a base station that can connect to the network 400, or it may be a communication interface that can connect to the network 400 via a network such as a mobile phone network or a wireless LAN.

[0063] The power supply unit 211 supplies power to each part of the remote control terminal 200 from an external power source or an internal power source. The power supply unit 211 may, for example, transform the power input from the external power source before supplying it to each part of the remote control terminal 200.

[0064] Next, the detailed configuration of the mobile control unit 101 and the mobile storage unit 112 will be described with reference to Figure 2. Figure 2 is a block diagram showing the detailed functional configuration of the mobile control unit 101 and the mobile storage unit 112.

[0065] As shown in Figure 2, the mobile unit control unit 101 includes a SLAM unit 501, a driving route setting unit 502, a driving state control unit 503, a manual driving control unit 504, an autonomous driving control unit 505, a driving control unit 506, a remote control receiving unit 507, and a remote driving control unit 508. The mobile unit storage unit 112 includes an environmental map data holding unit 601, a driving route holding unit 602, a driving state holding unit 603, and a driving route recording holding unit 604.

[0066] The SLAM unit 501 creates environmental map data around the mobile object 100 and estimates the mobile object 100's own position. When creating the map, the SLAM unit 501 simultaneously estimates the mobile object 100's own position and creates environmental map data around the mobile object 100 using images of the surrounding environment captured by the camera unit 102 or distance measurement information of the surrounding environment sensed by the sensor unit 103. The environmental map data created by the SLAM unit 501 is stored in the environmental map data holding unit 601 of the mobile object storage unit 112. Furthermore, the SLAM unit 501 can autonomously drive the mobile object 100 by performing self-position estimation using images captured by the camera unit 102, sensor values ​​from the sensor unit 103, environmental map data created by the SLAM unit 501, and the estimated self-position. However, the SLAM unit 501 is not limited to the above, and may perform SLAM using either the camera unit 102 or the sensor unit 103, or it may perform SLAM using both the camera unit 102 and the sensor unit 103.

[0067] The driving route setting unit 502 sets the route that the mobile unit 100 will travel when autonomous driving is performed, using the environmental map data created by the SLAM unit 501. The driving route setting unit 502 may set the driving route by specifying multiple coordinates in the environmental map data created by the SLAM unit 501, or by specifying a line connecting the coordinates in the environmental map data.

[0068] The driving state control unit 503 controls the driving state of the mobile unit 100 to one of the following: completed state, paused state, abnormal stop state, map creation state, autonomous driving state, and remote driving state. The driving state control unit 503 may control the driving state of the mobile unit 100 based, for example, on input operations from the mobile unit operation unit 111, remote operations from the remote operation terminal 200, or determinations made by the mobile unit 100. In each driving state, the processing set for that driving state is consistently executed. That is, until the processing set for that driving state is completed, no interrupt processing, etc., is accepted, and the driving state is maintained.

[0069] The manual driving control unit 504 controls the mobile body 100 to move when the mobile body 100 is in a manual driving state, using the mobile body operation unit 111. The autonomous driving control unit 505 controls the mobile body 100 to move autonomously based on environmental map data and the driving route when the mobile body 100 is in an autonomous driving state.

[0070] The driving control unit 506 controls the movement of the mobile body 100 by controlling the drive of the moving unit 107. When the mobile body 100 is in a manual driving state, the operation of the driving control unit 506 is controlled by the manual driving control unit 504. When the mobile body 100 is in an autonomous driving state, the operation of the driving control unit 506 is controlled by the autonomous driving control unit 505. When the mobile body 100 is in a remote driving state, the operation of the driving control unit 506 is controlled by the remote driving control unit 508.

[0071] The remote control receiving unit 507 receives control information for the mobile unit 100 from the remote control terminal 200 when the driving state control unit 503 is in a remote driving state, and outputs the received control information for the mobile unit 100 to the remote driving control unit 508. The remote control receiving unit 507 also outputs information to the driving state control unit 503 that instructs the start or end of remote driving, which it has received from the remote control terminal 200.

[0072] When the mobile body 100 is in a remote driving state, the remote driving control unit 508 remotely controls the mobile body 100 to move based on the control information received by the remote control receiving unit 507.

[0073] The environmental map data storage unit 601 stores and stores environmental map data created by the SLAM unit 501. The driving route storage unit 602 stores and stores information on the driving route of the mobile unit 100 set by the driving route setting unit 502. The driving state storage unit 603 stores and stores information on the driving state of the mobile unit 100. The driving route record storage unit 604 stores and stores information on the route actually autonomously driven by the mobile unit 100.

[0074] Next, with reference to Figure 3, the detailed configuration of the mobile operation system unit 300 will be described. Figure 3 is a block diagram showing the detailed functional configuration of the mobile operation system unit 300.

[0075] As shown in Figure 3, the mobile operation system unit 300 includes a system communication unit 301, a mobile remote control unit 302, a registration unit 303, and a registration data storage unit 304.

[0076] The system communication unit 301 is a communication interface for connecting to the network 400. The system communication unit 301 may be, for example, a communication interface that can be connected wirelessly or via a wired connection to a base station that can connect to the network 400, or it may be a communication interface that can connect to the network 400 via a network such as a mobile phone network or a wireless LAN.

[0077] The mobile unit remote control unit 302 remotely controls the state of the mobile unit 100. Specifically, based on input operations from the remote control terminal 200, the mobile unit remote control unit 302 remotely controls the state of the mobile unit 100 to terminated state, paused state, abnormal stop state, map creation state, autonomous driving state, and remote driving state. The mobile unit remote control unit 302 corresponds to one embodiment of the map creation control unit in the present invention.

[0078] The registration unit 303 registers the environmental map data, which has been converted to a predetermined format by the map conversion unit 114, into the registration data storage unit 304. Since the environmental map data received from each of the multiple mobile units 100 has been converted to a unified predetermined format, the registration unit 303 registers the environmental map data received in a unified predetermined format into the registration data storage unit 304 without performing any conversion of the environmental map data.

[0079] The registration data storage unit 304 is a recording device that stores environmental map data received from the mobile body 100 based on the control of the registration unit 303. The registration data storage unit 304 may store environmental map data that has been created for each of the multiple mobile bodies 100 and then converted into a unified predetermined format. Alternatively, the registration data storage unit 304 may store each of the environmental map data created for each of the multiple mobile bodies 100 overlaid with a pre-prepared map of the area that the mobile body 100 can travel.

[0080] <2. Example of operation> Next, an example of the operation of the control device 1 according to this embodiment will be described with reference to Figures 4 to 15.

[0081] In the control device 1, the mobile unit operation system unit 300 is started first. After the mobile unit operation system unit 300 is started, the mobile unit 100 and the remote control terminal 200 are started. When the mobile unit 100 is started, the mobile unit control unit 101 transmits log information from the mobile unit control unit 101, information regarding the shape of the mobile unit 100, and information regarding the movement characteristics of the mobile unit 100 to the mobile unit operation system unit 300, and stores the transmitted information in the mobile unit storage unit 112. The information transmitted to the mobile unit operation system unit 300 is used, for example, to display the status of the mobile unit 100 on the remote control terminal 200 when the mobile unit 100 is remotely controlled by the remote control terminal 200.

[0082] Figure 4 is a flowchart illustrating the operation of the mobile body 100 according to this embodiment. As shown in Figure 4, after the mobile body 100 is started, the driving state of the mobile body 100 is changed to a paused state (S101), and an interrupt process to change the driving state of the mobile body 100 is executed (S102).

[0083] Figure 5 is a flowchart showing the flow of interrupt processing. As shown in Figure 5, when the interrupt processing in step S102 is initiated, the driving state held in the driving state holding unit 403 is changed based on an input operation from the mobile body operation unit 111, a remote operation from the remote operation terminal 200, or a determination by the mobile body 100 (S201). After the interrupt processing is completed, the driving state of the mobile body 100 is determined (S103).

[0084] If the mobile unit 100 is in the terminated state (S103 / terminated state), the mobile unit 100's driving control is terminated (S104). On the other hand, if the mobile unit 100 is in the paused state (S103 / paused state), the mobile unit 100's paused process is executed (S105). If the mobile unit 100 is in the abnormal stop state (S103 / abnormal stop state), the mobile unit 100's abnormal stop process is executed (S106). If the mobile unit 100 is in the map creation state (S103 / map creation state), the mobile unit 100's map creation process is executed (S107). If the mobile unit 100 is in the autonomous driving state (S103 / autonomous driving state), the mobile unit 100's autonomous driving process is executed (S108). If the mobile unit 100 is in a remote control state (S103 / remote control state), the remote control process for the mobile unit 100 is executed (S109).

[0085] Figure 6 is a flowchart showing the flow of the pause process. As shown in Figure 6, when the pause process in step S105 is initiated, the travel control unit 506 controls the mobile body 100 to temporarily suspend its movement (S202). Subsequently, it is determined whether the travel state of the mobile body 100 has changed from the paused state (S203). If the travel state has changed from the paused state (S203 / YES), the pause process ends. On the other hand, if the travel state has not changed from the paused state (S203 / NO), the determination is repeated until the travel state changes from the paused state to another state.

[0086] Figure 7 is a flowchart showing the flow of the abnormal stop process. As shown in Figure 7, when the abnormal stop process in step S106 is initiated, the travel control unit 506 controls the mobile body 100 to abnormally stop its movement (S204). Subsequently, it is determined whether the travel state of the mobile body 100 has changed from the abnormal stop state (S205). If the travel state has changed from the abnormal stop state (S205 / YES), the abnormal stop process ends. On the other hand, if the travel state has not changed from the abnormal stop state (S205 / NO), the determination is repeated until the travel state changes from the abnormal stop state to another state.

[0087] Figure 8 is a flowchart showing the flow of the map creation process. When an operation is performed from the mobile unit operation unit 111 of the mobile unit 100 or the remote control terminal 200 to change the driving state of the mobile unit 100 to the map creation state, an instruction to create environmental map data is sent to the mobile unit 100. Upon receiving the instruction to create environmental map data, the driving state of the mobile unit 100 becomes the map creation state, and the map creation process in step S107 begins.

[0088] As shown in Figure 8, when the map creation process in step S107 is started, the operation of the manual driving control unit 504 is started first (S301), and the creation of environmental map data is started at the same time. After that, it is determined whether or not the creation of environmental map data has been completed (S302). If the creation of environmental map data has not been completed (S302 / NO), the determination is repeated until the creation of environmental map data is completed. On the other hand, if the creation of environmental map data has been completed (S302 / YES), it is determined whether or not a request to transmit environmental map data has been received from the mobile operation system unit 300 (S303).

[0089] If a request to transmit environmental map data has not been received (S303 / NO), the determination is repeated until a request to transmit environmental map data is received. On the other hand, if a request to transmit environmental map data is received (S303 / YES), the environmental map data is converted by the map conversion unit 114 (S304), and the converted environmental map data is transmitted to the mobile operation system unit 300 (S305).

[0090] The map conversion unit 114 can apply various conversions, for example. For example, the map conversion unit 114 may convert the data name of the environmental map data into a predetermined format, convert settings including the origin, resolution, size, or compression format of the map data (e.g., PNG format) of the environmental map data into a predetermined format, or convert at least one of the settings of the origin, resolution, or size of the environmental map data into a predetermined format.

[0091] Furthermore, the map conversion unit 114 may convert the data name of the environmental map data created by the mobile unit 100 into a format that includes the device name of the mobile unit 100, the name of the mobile unit 100 (for example, a nickname or identification name), the identification number of the mobile unit 100, the date the environmental map data was created, or the name of the location where the environmental map data was created, so that the created mobile unit 100 can be identified. This allows the mobile unit operation system unit 300 to handle environmental map data created by multiple mobile units 100 separately without confusion.

[0092] Figure 9 is an explanatory diagram showing an example of environmental map data conversion by the map conversion unit 114. In Figure 9, the format of the map name held within the environmental map data is changed (for example, the map name "map" is converted to "Map", and then further converted to the map name "Map001" which includes the identification number "001").

[0093] Next, it is determined whether the driving state of the mobile unit 100 has changed from the map creation state (S306). If the driving state has not changed from the map creation state (S306 / NO), the determination is repeated until the driving state changes from the map creation state to another state. If the driving state has changed from the map creation state (S306 / YES), the operation of the manual driving control unit 504 ends (S307).

[0094] Figure 10 is a flowchart showing the operation flow of the mobile operation system unit 300 when receiving environmental map data from the mobile unit 100. As shown in Figure 10, first, a transmission request is sent from the remote control terminal 200 to the mobile operation system unit 300 requesting that it transmit environmental map data, and is received by the mobile operation system unit 300 (S351). As a result, the mobile operation system unit 300 starts processing the reception of environmental map data (S352) and sends a transmission request for environmental map data to the mobile unit 100.

[0095] Next, the mobile operation system unit 300 determines whether or not it has received environmental map data from the mobile unit 100 (S353). If it has not received environmental map data from the mobile unit 100 (S353 / NO), the mobile operation system unit 300 repeats this determination until it receives the environmental map data. On the other hand, if it has received environmental map data from the mobile unit 100 (S353 / YES), the mobile operation system unit 300 registers the received environmental map data in the registration data storage unit 304 (S354). After that, the mobile operation system unit 300 terminates the environmental map data reception process (S355).

[0096] Figure 11 is a flowchart showing the operation flow of the manual driving control unit 504. As shown in Figure 11, first it is determined whether or not the driving state of the mobile body 100 is in the map creation state (S401).

[0097] If the mobile unit 100 is not in a map creation state (S401 / NO), it is determined that the mobile unit 100 is in a manual driving state, and the manual driving process is started. First, manual driving is started based on the input operation from the mobile unit operation unit 111 (S402). After that, it is determined whether or not the manual driving has finished (S403). If the manual driving has not finished (S403 / NO), the determination is repeated until the manual driving is finished. On the other hand, if the manual driving has finished (S403 / YES), the driving state is changed to a paused state (S404).

[0098] On the other hand, if the driving state is in the map creation state (S401 / YES), the SLAM unit 501 starts SLAM as part of the map creation process (S405). After the SLAM unit 501 starts operating, manual driving is started based on the input operation from the mobile operation unit 111 (S406). Subsequently, it is determined when manual driving has ended (S407), and if manual driving has ended (S407 / YES), the created environmental map data is saved in the environmental map data holding unit 601 (S408), and SLAM by the SLAM unit 501 is stopped (S409). After that, the driving state is changed to the paused state (S404). On the other hand, if manual driving has not ended (S407 / NO), the determination is repeated until manual driving is completed.

[0099] Once environmental map data is created, the mobile unit operation unit 111 or remote control terminal 200 sets a travel route indicating which points in the environmental map data the mobile unit 100 will travel through. The set travel route is stored in the travel route holding unit 602.

[0100] Figure 12 is a flowchart showing the flow of the autonomous driving process. As shown in Figure 12, when the autonomous driving process in step S108 is started, first the autonomous driving control unit 505 starts operating (S501), and the autonomous driving of the mobile body 100 begins. After that, it is determined whether the driving state of the mobile body 100 has changed from the autonomous driving state (S502). If the driving state has changed from the autonomous driving state (S502 / YES), the operation of the autonomous driving control unit 505 ends (S503). On the other hand, if the driving state has not changed from the autonomous driving state (S502 / NO), the determination is repeated until the driving state changes from the autonomous driving state to another state.

[0101] Figure 13 is a flowchart showing the operation flow of the autonomous driving control unit 505. As shown in Figure 13, when autonomous driving is started in step S501, the autonomous driving control unit 505 determines whether environmental map data is stored in the environmental map data storage unit 601 and whether a driving route is stored in the driving route storage unit 602 (S511). If neither environmental map data nor a driving route is stored (S511 / NO), the autonomous driving control unit 505 changes the driving state to an abnormal stop state (S512) and terminates autonomous driving control.

[0102] On the other hand, if environmental map data and a driving route are saved (S511 / YES), the autonomous driving control unit 505 uses the saved environmental map data and driving route to start autonomous driving (S513) and causes the mobile unit 100 to autonomously drive along the driving route (S514). At this time, the mobile unit 100 records the autonomously driven route in the driving route recording and holding unit 604 (S515).

[0103] Subsequently, the autonomous driving control unit 505 determines whether autonomous driving has ended or not (S516). If autonomous driving has not ended (S516 / NO), the determination is repeated until autonomous driving is completed. On the other hand, if autonomous driving has ended (S516 / YES), the recording of the autonomous driving route to the driving route record holding unit 604 is stopped (S517), and the driving status is changed to a paused state (S518).

[0104] Figure 14 is a flowchart showing the flow of the remote driving process. As shown in Figure 14, when the remote driving process in step S109 is started, first the remote driving control unit 508 starts operating (S601), and the remote driving of the mobile body 100 begins. After that, it is determined whether the driving state of the mobile body 100 has changed from the remote driving state (S602). If the driving state has changed from the remote driving state (S602 / YES), the operation of the remote driving control unit 508 ends (S603). On the other hand, if the driving state has not changed from the remote driving state (S602 / NO), the determination is repeated until the driving state changes from the remote driving state to another state.

[0105] Figure 15 is a flowchart showing the operation flow of the remote driving control unit 508. As shown in Figure 15, when remote driving is initiated, the remote driving control unit 508 starts remote driving (S611) and starts receiving remote driving control information from the remote operation terminal 200 (S612).

[0106] Subsequently, the remote driving control unit 508 determines whether or not the remote driving has ended (S613). If the remote driving has not ended (S613 / NO), the determination is repeated until the remote driving is completed. On the other hand, if the remote driving has ended (S613 / YES), the reception of remote driving control information from the remote operation terminal 200 is stopped (S614), and the driving state is changed to a paused state (S615).

[0107] As described above, the control device 1 according to this embodiment can automatically convert the format of the environmental map data created by each mobile unit 100 into a unified format, and transmit the converted environmental map data to the operation system for registration. As a result, the control device 1 according to this embodiment eliminates the need for the operator OP to manually convert the environmental map data when registering it to the operation system, thereby reducing the workload of the operator OP when registering it to the operation system. Furthermore, by handling the environmental map data in a unified format, the environmental map data can be efficiently shared and utilized among multiple mobile units, thereby further improving the overall operability of the system.

[0108] <3. Variant> (First variation) The configuration of the control device 1A according to the first modified example will be described. The first modified example is a modification in which environmental map data is converted on the mobile operation system unit 300A side. Figure 16 is a block diagram showing the functional configuration of the control device 1A according to the first modified example. Figure 17 is a block diagram showing the detailed functional configuration of the mobile operation system unit 300A according to the first modified example.

[0109] The control device 1A according to the first modification differs from the control device 1 shown in Figure 1 in that the mobile unit 100A does not have a map conversion unit 114, and the mobile unit operation system unit 300A further includes an environmental map data conversion unit 305.

[0110] As shown in Figure 16, the control device 1A includes a mobile unit 100A, a remote control terminal 200, and a mobile unit operation system unit 300A.

[0111] The mobile unit 100A comprises a mobile unit control unit 101, a camera unit 102, a sensor unit 103, a display unit 104, a speaker 105, a microphone 106, a mobile unit 107, a mobile unit LAN hub unit 108, a mobile unit communication unit 109, an AI unit 110, a mobile unit operation unit 111, a mobile unit storage unit 112, and a battery unit 113. The mobile unit operation system unit 300A comprises a system communication unit 301, a mobile unit remote control unit 302, a registration unit 303, a registration data storage unit 304, and an environmental map data conversion unit 305.

[0112] When the environmental map data conversion unit 305 receives environmental map data from the mobile unit 100A, it converts the received environmental map data into a format compatible with the operating system. The environmental map data converted into a format compatible with the operating system is registered and stored in the registered data storage unit 304 by the registration unit 303.

[0113] As explained with reference to Figures 1 to 3, the environmental map data stored in the registration data storage unit 304 of the mobile operation system unit 300A is converted into a unified predetermined format. Therefore, the mobile operation system unit 300A can use the received environmental map data for autonomous driving of the mobile unit 100A by converting it into a format compatible with the operation system.

[0114] Next, an example of the operation of the control device 1A according to the first modified example will be described. In the control device 1A, first the mobile unit operation system unit 300A is started, and then the mobile unit 100A and the remote control terminal 200 are started. When an operation is performed from the mobile unit operation unit 111 of the mobile unit 100A or from the remote control terminal 200 to change the driving state of the mobile unit 100A to the map creation state, the driving state of the mobile unit 100A becomes the map creation state, and the map creation process in step S107 is started.

[0115] Figure 18 is a flowchart showing the flow of the map creation process for the mobile unit 100A in the first modified example. As shown in Figure 18, when the map creation process in step S107 is started, the operation of the manual driving control unit 504 is started first (S301), and at the same time, the creation of environmental map data is started. After that, it is determined whether or not the creation of environmental map data has been completed (S302). If the creation of environmental map data has been completed (S302 / YES), it is determined whether or not a request to transmit environmental map data has been received from the mobile unit operation system unit 300A (S303).

[0116] If a request to transmit environmental map data has not been received (S303 / NO), the determination is repeated until a request to transmit environmental map data is received. On the other hand, if a request to transmit environmental map data is received (S303 / YES), the created environmental map data is transmitted to the mobile operation system unit 300A (S308). Next, it is determined whether the driving state of the mobile unit 100A has changed from the map creation state (S306). If the driving state has not changed from the map creation state (S306 / NO), the determination is repeated until the driving state changes from the map creation state to another state. If the driving state has changed from the map creation state (S306 / YES), the operation of the manual driving control unit 504 ends (S307).

[0117] Figure 19 is a flowchart showing the operation flow of the mobile operation system unit 300A when receiving environmental map data from the mobile unit 100. As shown in Figure 19, first, a transmission request is sent from the remote control terminal 200 to the mobile operation system unit 300A requesting that it transmit environmental map data, and this request is received by the mobile operation system unit 300A (S351). As a result, the mobile operation system unit 300A starts processing the reception of environmental map data (S352) and sends a transmission request for environmental map data to the mobile unit 100A.

[0118] The mobile operation system unit 300A determines whether or not it has received environmental map data from the mobile unit 100A (S353). If it has not received environmental map data from the mobile unit 100A (S353 / NO), the mobile operation system unit 300A repeats this determination until it receives the environmental map data. On the other hand, if it has received environmental map data from the mobile unit 100A (S353 / YES), the mobile operation system unit 300A converts the received environmental map data (S356) and registers the converted environmental map data in the registration data storage unit 304 (S357). After that, the mobile operation system unit 300 terminates the environmental map data reception process (S355).

[0119] As described above, the control device 1A according to the first modified example can automatically convert the environmental map data created by each mobile unit 100A into a unified format on the operating system side and register the converted environmental map data. As a result, even if the mobile unit 100 does not have a map conversion unit that converts environmental map data into a predetermined format, the control device 1A according to the first modified example eliminates the need for the operator OP to manually convert the environmental map data when registering the environmental map data in the operating system, thereby reducing the workload of the operator OP when registering in the operating system. Furthermore, by handling environmental map data in a unified format, the control device 1A according to the first modified example can efficiently share and utilize environmental map data among multiple mobile units 100, thereby further improving the operability of the system.

[0120] (Second variation) Next, the configuration of the control device 1B according to the second modified example will be described. The second modified example is a case where environmental map data created with another mobile unit 100B-2 of the same model is registered in the mobile unit operation system unit 300B. Figure 20 is a block diagram showing the functional configuration of the control device 1B according to the second modified example. Figure 21 is a block diagram showing the detailed functional configuration of the mobile unit operation system unit 300B according to the second modified example.

[0121] The control device 1B according to the second modification differs from the control device 1 shown in Figure 1 in that the mobile unit 100B-1 further comprises a system map conversion unit 115, and the mobile unit operation system unit 300B further comprises an environmental map data output unit 306.

[0122] As shown in Figure 20, the control device 1A includes a mobile unit 100B-1, a mobile unit 100B-2, a remote control terminal 200, and a mobile unit operation system unit 300B. The mobile unit 100B-1 includes a mobile unit control unit 101, a camera unit 102, a sensor unit 103, a display unit 104, a speaker 105, a microphone 106, a mobile unit 107, a mobile unit LAN hub unit 108, a mobile unit communication unit 109, an AI unit 110, a map conversion unit 114, a mobile unit operation unit 111, a mobile unit storage unit 112, a battery unit 113, and a system map conversion unit 115. The mobile unit operation system unit 300B includes a system communication unit 301, a mobile unit remote control unit 302, a registration unit 303, a registration data storage unit 304, and an environmental map data output unit 306.

[0123] The environmental map data output unit 306 is located in the mobile operation system unit 300B and outputs the environmental map data stored in the registration data storage unit 304 to the mobile unit 100B-1. Specifically, the environmental map data output unit 306 outputs the environmental map data stored in the registration data storage unit 304 to the mobile unit 100B-1 based on a request from the mobile unit 100B-1 to transmit environmental map data.

[0124] The system map conversion unit 115 is installed in the mobile unit 100B-1 and, upon receiving environmental map data from the mobile unit operation system unit 300B, converts the received environmental map data into a format compatible with the mobile unit 100B-1. The environmental map data converted into a format compatible with the mobile unit 100B-1 is output to the mobile unit control unit 101 and used for autonomous driving of the mobile unit 100B-1, etc.

[0125] As explained with reference to Figures 1 to 3, the environmental map data stored in the registration data storage unit 304 of the mobile operation system unit 300B is converted into a unified predetermined format. Therefore, the mobile unit 100B-1 can use the received environmental map data for its own autonomous driving, etc., by converting the received environmental map data into a format compatible with the mobile unit 100B-1.

[0126] Next, an example of the operation of the control device 1B according to the second modification will be described. Mobile unit 100B-1 is a mobile unit that travels in the same environment as the environmental map data created by mobile unit 100B-2. Mobile unit 100B-1 is in a state where the creation of the environmental map data is not yet complete, while mobile unit 100B-2 is in a state where the creation of the environmental map data and the registration of the created environmental map data to the operating system are complete.

[0127] Figure 22 is a flowchart illustrating the operation of the mobile unit 100B-1 in the second modified example. As shown in Figure 22, after the mobile unit 100B-1 is started, its driving state is changed to a paused state (S101), and an interrupt process is executed to change the driving state of the mobile unit 100B-1 (S102). After that, the driving state of the mobile unit 100B-1 is determined (S103).

[0128] In the second modification, in addition to the termination state, paused state, abnormal stop state, map creation state, autonomous driving state, and remote driving state described with reference to Figure 4, there is also a map reception state as a driving state of the mobile unit 100B-1. When the driving state of the mobile unit 100B-1 is the map reception state, the map reception process of the mobile unit 100B-1 is executed (S110).

[0129] Figures 23 and 24 are flowcharts showing the flow of the map reception process. Figure 23 is a flowchart showing the flow of the environmental map data output unit 306 of the mobile unit operation system unit 300B when the mobile unit 100B-1 enters the map reception process. Figure 24 is a flowchart showing the flow of the map reception process of the mobile unit 100B-1.

[0130] If the mobile unit 100B-1 is instructed to receive environmental map data from the mobile unit operation system unit 300B based on an input operation from the mobile unit operation unit 111 or the remote operation terminal 200, the driving state of the mobile unit 100B-1 is changed to the map reception state and the map reception process is started.

[0131] First, as shown in Figure 24, the mobile unit 100B-1 starts receiving environmental map data (S701), and a request to transmit environmental map data is sent to the mobile unit operation system 300B (S702).

[0132] On the other hand, as shown in Figure 23, the mobile operation system unit 300B determines whether or not there is a request from the mobile unit 100B-1 to transmit environmental map data (S358). If there is no request from the mobile unit 100B-1 to transmit environmental map data (S358 / NO), the mobile operation system unit 300B repeatedly makes this determination until there is a request from the mobile unit 100B-1 to transmit environmental map data. If there is a request from the mobile unit 100B-1 to transmit environmental map data (S358 / YES), the mobile operation system unit 300B transmits the registered environmental map data (S359), and after the transmission is complete, it terminates the transmission of environmental map data (S360).

[0133] Then, as shown in Figure 24, it is determined whether or not the mobile unit 100B-1 has received environmental map data from the mobile unit operation system unit 300B (S703). If environmental map data has not been received from the mobile unit operation system unit 300B (S703 / NO), the mobile unit 100B-1 repeats the determination until it receives the environmental map data. On the other hand, if environmental map data has been received from the mobile unit operation system unit 300B (S703 / YES), the received environmental map data (environmental data registered in the mobile unit operation system unit 300B) is converted to a format compatible with the mobile unit 100B-1 by the system map conversion unit 115 (S704). The converted environmental map data is stored in the environmental map data holding unit 601 of the mobile unit storage unit 112 (S705).

[0134] The system map conversion unit 115 may, for example, convert the data name of the environmental map data to a format compatible with the mobile unit 100B-1, or convert settings including the origin, resolution, or size of the environmental map data to a format compatible with the mobile unit 100B-1. Figure 25 is an explanatory diagram showing an example of environmental map data conversion by the system map conversion unit 115. In Figure 25, the map name "Map002" of the environmental map data is converted to the map name "map". As a result, the mobile unit 100B-1 can perform autonomous driving based on the environmental map data received from the mobile unit operation system unit 300B, even in areas where it has not created environmental map data itself.

[0135] Next, it is determined whether the driving state of the mobile unit 100B-1 has changed from the map reception state (S706). If the driving state has not changed from the map reception state (S706 / NO), the determination is repeated until the driving state changes from the map reception state to another state. If the driving state has changed from the map reception state (S706 / YES), the map reception process ends (S707).

[0136] As explained above, the control device 1B according to the second modification can transmit environmental map data created by another mobile unit 100B-2 and registered in the mobile unit operation system unit 300B to another mobile unit 100B-1 using the same model. As a result, the control device 1B according to the second modification can hold the environmental map data necessary for autonomous driving, etc., without requiring the mobile unit 100B-1 to perform map creation processing, thereby reducing the workload of on-site workers or operators. Furthermore, the control device 1B according to the second modification can reduce the number of environmental map data managed by the operation system, thereby further improving the operability of the system.

[0137] (Third variation) Furthermore, the configuration of the control device 1C according to the third modified example will be described.

[0138] In the third modification, environmental map data created by different models of mobile units (i.e., environmental map data created by mobile unit 100C-1 and environmental map data created by mobile unit 100C-2) have different scales, etc. Therefore, mobile unit 100C-1 can use the received environmental map data for its own autonomous driving by converting it to the same scale as the environmental map data it created. The control device 1C according to the third modification has the same configuration as the control device 1B according to the second modification shown in Figures 20 and 21. However, although mobile unit 100C-2 has the same internal equipment configuration as mobile unit 100C-1, it is a different model of mobile unit from mobile unit 100C-1.

[0139] In the third modified example, the system map conversion unit 115A receives environmental map data from the mobile operation system unit 300B and converts the received environmental map data into a format compatible with the mobile unit 100C-1 by enlarging or reducing the received environmental map data. The enlarged or reduced environmental map data is output to the mobile unit control unit 101 and used for autonomous driving of the mobile unit 100C-1, etc.

[0140] Next, an example of the operation of the control device 1C according to the third modification will be described. Mobile unit 100C-1 is a mobile unit that travels in the same environment as the environmental map data created by mobile unit 100C-2. Mobile unit 100C-1 is in a state where the creation of the environmental map data is not yet complete, while mobile unit 100C-2 is in a state where the creation of the environmental map data and the registration of the created environmental map data to the operating system are complete.

[0141] Figure 27 is a flowchart illustrating the operation of the mobile unit 100C-1 in the third modified example. As shown in Figure 27, after the mobile unit 100C-1 is started, its running state is changed to a paused state (S101), and an interrupt process is executed to change the running state of the mobile unit 100C-1 (S102). Subsequently, the running state of the mobile unit 100C-1 is determined (S103).

[0142] In the third modification, in addition to the termination state, paused state, abnormal stop state, map creation state, autonomous driving state, and remote driving state described with reference to Figure 4, there is also a map reception conversion state as a driving state of the mobile unit 100C-1. When the driving state of the mobile unit 100C-1 is the map reception conversion state, the map reception conversion process of the mobile unit 100C-1 is executed (S111).

[0143] Figure 28 is a flowchart showing the flow of the map reception conversion process. When the map reception conversion process in step S111 is started, the manual driving control unit 504 starts operating first (S751), and at the same time, the creation of environmental map data begins. After that, it is determined whether or not the creation of some of the environmental map data has been completed (S752). If the creation of environmental map data has not been completed (S752 / NO), the determination is repeated until the creation of environmental map data is completed. On the other hand, if the creation of environmental map data has been completed (S752 / YES), the reception of environmental map data begins (S753), and a request to transmit environmental map data is sent to the mobile operation system unit 300B.

[0144] The range of the environmental map data created in step S751 is sufficient to allow for the identification of the correspondence between the created environmental map data and the environmental map data received from the mobile operation system unit 300B. Specifically, the range of the created environmental map data is sufficient to allow for the establishment of three or more corresponding points between the created environmental map data and the environmental map data received from the mobile operation system unit 300B.

[0145] Subsequently, it is determined whether or not a request for transmission of environmental map data has been received from the mobile operation system unit 300B (S754). If environmental map data has not been received from the mobile operation system unit 300B (S754 / NO), the mobile unit 100C-1 repeatedly makes this determination until it receives the environmental map data. On the other hand, if environmental map data has been received from the mobile operation system unit 300B (S754 / YES), the environmental map data received from the mobile operation system unit 300B is enlarged or reduced (S755). Specifically, the system map conversion unit 115A enlarges or reduces the environmental map data received from the mobile operation system unit 300B so that the scale is the same as some of the environmental map data that has been created. As a result, the environmental map data received from the mobile operation system unit 300B is converted into a format compatible with the mobile unit 100C-1. The converted environmental map data is stored in the environmental map data holding unit 601 of the mobile storage unit 112 (S756).

[0146] Various methods can be used for scaling the environmental map data by the system map conversion unit 115A. For example, the system map conversion unit 115A may compare the environmental map data created by the mobile unit 100C-1 with the environmental map data received from the mobile unit operation system unit 300B and calculate a scaling conversion rate based on their corresponding points. Specifically, the system map conversion unit 115A first compares the environmental map data created by the mobile unit 100C-1 with the received environmental map data (environmental map data registered in the mobile unit operation system unit 300B) and calculates a scaling conversion rate based on their corresponding points. Next, the system map conversion unit 115A converts the received environmental map data into a format compatible with the mobile unit 100C-1 by scaling the environmental map data based on the calculated conversion rate.

[0147] Next, it is determined whether the driving state of the mobile unit 100C-1 has changed from the map reception conversion state (S757). If the driving state has not changed from the map reception conversion state (S757 / NO), the determination is repeated until the driving state changes from the map reception conversion state to another state. If the driving state has changed from the map reception conversion state (S757 / YES), the map reception conversion process ends (S758).

[0148] As described above, the control device 1C according to the third modification can transmit environmental map data created on another mobile device of a different model and registered in the mobile device operation system unit 300B to the mobile device 100C-1 and perform scaling or reduction. This allows the control device 1C according to the third modification to use environmental map data created on another mobile device 100C-2 of a different model in the mobile device 100C-1. Therefore, the control device 1C according to the third modification can hold environmental map data necessary for autonomous driving, etc., without requiring the mobile device 100C-1 to perform map creation processing, thereby further reducing the workload of on-site workers or operators. In addition, the control device 1C according to the third modification can reduce the number of environmental map data managed by the operation system, thereby further improving the operability of the system.

[0149] <4. Hardware Configuration Example> Embodiments of the present invention have been described above. The various information processing operations performed by the mobile operation system units 300, 300A, and 300B described above are realized through the cooperation of software and the hardware of the information processing device 900 described below.

[0150] Figure 29 is a block diagram showing an example of the hardware configuration of an information processing device 900 that embodies the mobile operation system units 300, 300A, and 300B according to this embodiment.

[0151] As shown in Figure 29, the information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, a drive 911, and a communication device 912. A removable storage medium 913 is mounted in the drive 911.

[0152] The CPU 901 functions as both an arithmetic processing unit and a control unit, controlling the overall operation of the information processing unit 900 according to various programs. The CPU 901 may be a microprocessor. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901, and parameters that change as appropriate during program execution. These CPU 901, ROM 902, and RAM 903 are interconnected by a host bus 904, which consists of a CPU bus and the like. The functions of the mobile remote control unit 302, registration unit 303, environmental map data conversion unit 305, and environmental map data output unit 306 described above are realized through the cooperation of the CPU 901, ROM 902, and RAM 903.

[0153] The host bus 904 is connected to an external bus 906, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 905. Note that the host bus 904, bridge 905, and external bus 906 do not necessarily have to be separate; these functions may be implemented on a single bus.

[0154] The input device 908 consists of an input means for the user to input information, such as a mouse, keyboard, touch panel, buttons, switches, or microphone, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901. The user operating the information processing device 900 can input various types of data to the information processing device 900 or instruct it to perform processing operations by operating the input device 908.

[0155] The output device 909 includes, for example, a CRT (Cathode Ray Tube) display device, a Liquid Crystal Display (LCD) device, an OLED (Organic Light Emitting Diode) device, a display device such as a lamp, or an audio output device such as a speaker.

[0156] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deletion device for deleting data recorded on the storage medium. The storage device 910 can, for example, implement the functions of the registered data storage unit 304.

[0157] Drive 911 is a reader / writer for storage media and is externally connected to the information processing unit 900. Drive 911 reads information recorded on removable storage media 913, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs it to RAM 903. Drive 911 can also write information to the removable storage media 913.

[0158] The communication device 912 is a communication interface composed of communication devices and the like for performing communication. The communication device 912 may be a wireless LAN (Local Area Network) compatible communication device or a wired communication device for wired communication. The communication device 912 can, for example, implement the functions of the system communication unit 301.

[0159] The hardware configuration of the information processing device 900 is not limited to the configuration shown in Figure 29. For example, the information processing device 900 does not need to include an input device 908 or an output device 909. Also, some or all of the configuration shown in Figure 29 may be implemented using one or more integrated circuits (ICs).

[0160] Although preferred embodiments of the present invention have been described in detail above with reference to the attached drawings, the present invention is not limited to these examples. It is clear to any person with ordinary skill in the art to which the present invention belongs that various modifications or alterations can be conceived within the scope of the technical idea described in the claims, and these are also understood to fall within the technical scope of the present invention.

[0161] For example, each component provided in the mobile operation system units 300, 300A, and 300B may also be provided in the mobile units 100, 100A, 100B-1, 100C-1, or the remote control terminal 200. The map conversion unit 114 and the system map conversion units 115 and 115A of the mobile unit 100 may also be provided in the mobile operation system units 300, 300A, 300B, or the remote control terminal 200.

[0162] Furthermore, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0163] Furthermore, the series of processes performed by the information processing device described herein may be implemented using software, hardware, or a combination of software and hardware. The programs constituting the software are pre-stored in a storage medium (non-transitory media) provided inside or outside each device. Each program is then loaded into RAM 903, for example, when executed by a computer, and executed by a processor such as CPU 901. The storage medium is, for example, a magnetic disk, optical disk, magneto-optical disk, flash memory, etc. The computer programs may also be distributed without using a storage medium, for example, via a network. [Explanation of symbols]

[0164] 1,1A,1B,1C Control Unit 100, 100A, 100B-1, 100C-1 Mobile Unit 101 Mobile Unit Control Unit 102 Camera Department 103 Sensor section 104 Display section 105 speakers 106 Mike 107 Mobile Unit 108 Mobile LAN Hub Unit 109 Mobile Communications Department 110 AI Department 111 Mobile object operation section 112 Mobile Storage Unit 113 Battery section 114 Map Conversion Unit 115,115A System Map Conversion Unit 200 remote control terminals 201 Terminal Control Unit 202 Camera Department 203 Sensor Unit 204 Display section 205 Speakers 206 Mike 207 Terminal Operation Section 208 Terminal LAN Hub Section 209 Terminal Communications Department 210 Terminal Storage Unit 211 Power supply section 300, 300A, 300B Mobile Operation System Department 301 Systems Communications Department 302 Mobile Remote Control Unit 303 Registration Department 304 Registered Data Storage Unit 305 Environmental Map Data Conversion Unit 306 Environmental Map Data Output Unit 400 Networks 501 SLAM Department 502 Route setting unit 503 Driving State Control Unit 504 Manual Driving Control Unit 505 Autonomous Driving Control Unit 506 Driving Control Unit 507 Remote control receiver 508 Remote Driving Control Unit 601 Environmental map data storage unit 602 Travel path holding unit 603 Driving state holding unit 604 Driving route recording and retention unit 900 Information Processing Equipment 901 CPU 902 ROM 903 RAM 904 Host Bus 905 Bridge 906 External bus 907 Interface 908 Input devices 909 Output devices 910 Storage Devices 911 Drive 912 Communication equipment 913 Removable recording media OP Operator

Claims

1. A map creation control unit that controls the creation of environmental map data representing the environment around a moving object, A map conversion unit that converts the environmental map data created by the mobile device into a predetermined format, A registration unit that registers the environmental map data converted to the predetermined format, An output unit that outputs the registered environmental map data to a mobile device other than the mobile device, A control device equipped with the following features.

2. The control device according to claim 1, wherein the map conversion unit converts at least one of the settings of the environmental map data into a setting conforming to the predetermined format.

3. The control device according to claim 1 or 2, characterized in that the output unit outputs the environmental map data to the other mobile body based on a request from the other mobile body to transmit the environmental map data.

4. The control device according to claim 1 or 2, further comprising a system map conversion unit that converts the environmental map data output to the other mobile device into a format corresponding to the other mobile device.

5. The control device according to claim 4, characterized in that the system map conversion unit calculates a conversion rate for scaling up or down using the environmental map data output to the other mobile device and the environmental map data created by the other mobile device, and converts the environmental map data output to the other mobile device into a format corresponding to the other mobile device by scaling up or down.

6. The control device according to claim 4, characterized in that the other mobile body uses the environmental map data converted to a format corresponding to the other mobile body.

7. The system further includes a state control unit for controlling the movement state of the moving body, The control device according to claim 1 or 2, characterized in that, in each of the aforementioned driving states, the processing set for the driving state is consistently executed.

8. The control device according to claim 1, further comprising a remote control unit for remotely controlling the operating state of the mobile body.

9. The control device according to claim 8, characterized in that the mobile remote control unit operates a plurality of the mobile units.

10. A step to control the creation of environmental map data representing the environment around a moving object, The steps include converting the environmental map data created by the mobile device into a predetermined format, The steps include registering the environmental map data that has been converted to the predetermined format, The steps include outputting the registered environmental map data to another mobile device different from the mobile device, A computer-based information processing method, including [the specified term].

11. Computers, A map creation control unit that controls the creation of environmental map data representing the environment around a moving object, A map conversion unit that converts the environmental map data created by the mobile device into a predetermined format, A registration unit that registers the environmental map data converted to the predetermined format, An output unit that outputs the registered environmental map data to a mobile device other than the mobile device, An information processing program designed to function as such.

12. An information processing system including a mobile device and a control device, A map creation control unit controls the creation of environmental map data representing the environment around the moving object, A map conversion unit that converts the environmental map data, which is in a format created by the aforementioned mobile device, into a predetermined format, A registration unit that registers the environmental map data converted to the predetermined format, An output unit that outputs the registered environmental map data to a mobile device other than the mobile device, Information processing systems, including those mentioned above.

Images