Administrative system for regulated region, administrative system for mobile body, administrative procedure for regulated region and non-transitory storage medium
The regulated region management system allows managers to intuitively mark access-regulated regions on maps, addressing inefficiencies in sensor-based updates by enabling real-time, cost-effective, and human-centric robot navigation.
Patent Information
- Application Number
- DE102023101415
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-03-17
- Filing Date
- 2023-01-20
- Publication Date
- 2024-11-07
- Estimated Expiration
- 2043-01-20
AI Technical Summary
Existing systems struggle to intuitively manage access-regulated regions for autonomously movable robots, particularly in dynamic environments where sensor-based updates are cumbersome, costly, or fail to reflect human convenience, leading to inefficient navigation and potential collisions.
A regulated region management system that allows managers to manually mark access-regulated regions on a map using hand-drawn inputs, enabling real-time updates and flexible regulation based on human will, with processors determining and sending region information to robots and operator terminals.
Enables intuitive and cost-effective management of access-regulated regions, reducing the need for sensor updates, minimizing collisions, and facilitating coexistence with humans by reflecting human intent in robot navigation.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The present invention relates to a regulated region management system, a mobile body management system, a regulated region management method, and a non-transitory storage medium. More specifically, the present invention relates to a regulated region management system, a mobile body management system, a regulated region management method, and a non-transitory storage medium for managing a regulated region for a mobile body capable of autonomous movement.
[0002] A system is known that updates an access-controlled region into which an autonomously movable robot cannot enter, based on information sensed by the robot, etc. It is cumbersome for a robot to update the access-controlled region according to environmental changes in a complex environment including humans and objects whose positions vary. Therefore, in recent years, a technique has been proposed in which a server updates map information as information about an access-controlled region, and the server occasionally provides the map information to a robot (see, for example, JP 2021-077053 A).
[0003] Furthermore, DE 10 2020 111 325 A1 discloses a method for machine learning comprising: a first learning step, which is carried out in a phase before a neural network is installed in a mobile robot, and in which a stationary first obstacle is placed in a set space, and the first obstacle is placed at different positions using simulation, so that the neural network repeatedly learns a path from a starting point to the destination that avoids the first obstacle; and a second learning step, which is carried out in a phase after the neural network has been installed in the mobile robot, and in which, when the mobile robot detects a second obstacle that operates in the environment of the mobile robot in a space in which the mobile robot moves, the neural network repeatedly learns a path to a destination that avoids the second obstacle each time the mobile robot detects the second obstacle.
[0004] DE 11 2017 002 154 B4 discloses a mobile robot comprising: a travel drive unit configured to move a main body; an image capture unit configured to capture images of an environment; and a control unit configured to detect a current position, wherein the control unit is further configured to: divide a travel area into a plurality of small areas based on a predetermined criterion; extract at least one detection feature from an image captured at an unknown current position; generate a detection descriptor corresponding to the at least one detection feature;and calculate each feature distribution of a small area and the recognition descriptor based on a predetermined estimation rule to select a small area including the current position; wherein the control unit is further configured to: extract features from images captured at a plurality of positions; generate descriptors corresponding to the respective features;and generate the feature distribution of a small region for each of the small regions based on a predetermined learning rule, wherein the predetermined learning rule comprises a predetermined classification rule for classifying the descriptors into a plurality of groups and a predetermined labeling rule for converting the descriptors included in the same group into feature descriptors, and wherein the feature distribution of a small region is generated for each of the small regions based on the type of the feature descriptors and a number of feature descriptors per type, wherein the predetermined estimation rule comprises: a predetermined conversion rule for generating a recognition feature distribution comparable to the feature distribution of a small region based on the at least one recognition descriptor;a predetermined comparison rule for comparing the feature distribution of a small area with the at least one recognition feature distribution in order to calculate a similarity between the two;
[0005] DE 102013 009 258 A1 discloses a nuclear plant manipulator system, comprising a manipulator with an image capture device and a remote-controlled drive, a remote control device, and an evaluation device for evaluating image data from the image capture device. The evaluation device is designed such that, when the manipulator moves along a movement path, known objects located along the movement path are automatically identified based on the image data. If such an object is passed, an object counter is incremented or decremented depending on the direction, with the current value of the object counter being provided to the remote control device. Also disclosed is a method for controlling a manipulator of a corresponding nuclear plant manipulator system.
[0006] EP 4 151 135 A1 discloses a cleaning device having at least one electrical interface for interacting with an electrical counter-interface of at least one base station at least for charging an energy storage device of the cleaning device, wherein the cleaning device is at least designed and configured to move autonomously in at least one processing environment, to perform cleaning tasks and to move into a service position at a base station for servicing purposes.A cleaning device that enables the determination of a service function scope of a base station is realized in that the cleaning device is further designed and configured to determine, at least in the service position, at least one measured value of a voltage present at the electrical interface and / or at least one measured value of a voltage curve present at the electrical interface as an identification means, and to determine at least one service function scope of the base station using the identification means or at least to provide the identification means at at least one communication interface for use in determining a service function scope by a data processing device. A cleaning system, a method, and a computer program product are also disclosed.
[0007] US 10 444 964 B2 discloses systems and methods comprising a gateway located at a site and forming at least one network at the site comprising a plurality of site devices. A sensor user interface is coupled to the gateway and presented to a user via a remote device. The sensor user interface includes at least one display element. The at least one display element includes a floor plan display representing at least one floor of the site. The floor plan display visually and separately displays the location and current status of each of the plurality of site devices.
[0008] In addition, US 2018 / 0 263 451 A1 discloses a method for operating a soil tillage implement that moves independently within an environment, wherein a control and evaluation device monitors the operating status of the soil tillage implement, detects a fault condition of the soil tillage implement, and, if a fault condition occurs, initiates the output of an information signal. In order to be able to locate the soil tillage implement particularly easily and conveniently during a fault condition, it is proposed that an optical signal output device of the soil tillage implement emit an optical information signal that generates a light projection representing a direction indication, wherein the direction indication points from a projection location within the environment toward the current location of the soil tillage implement.
[0009] There is a need to enable a manager to intuitively mark an access-controlled region for a robot on a map. However, JP 2021-077053 A does not mention a method that allows a manager to mark an access-controlled region for a robot.
[0010] The above object is achieved by the subject matter of the independent claims. Advantageous developments of the invention are the subject matter of the dependent claims.
[0011] The present disclosure provides a regulated region management system, a mobile body management system, a regulated region management method, and a non-transitory storage medium that allow a manager to designate an access-regulated region for a mobile body that is autonomously movable through an intuitive operation.
[0012] A first illustrative aspect of the present disclosure provides a regulated region management system comprising one or more processors. The one or more processors are configured to: acquire a hand-drawn input image input to an environment map; and determine a regulated region based on the environment map and the hand-drawn input image. The regulated region is a region to which access of a mobile body is regulated. The one or more processors are configured to: transmit regulated region information indicating the regulated region to the mobile body when the mobile body is not remotely operated using an operation terminal; and transmit the regulated region information to the operation terminal when the mobile body is remotely operated using the operation terminal.This allows a manager to enter a regulated region through intuitive operation. Furthermore, it is possible to immediately inform a subject creating a movement plan about a regulated region that reflects human will. This allows for real-time intervention based on human will. It also eliminates the effort of re-creating an environment map in real time.
[0013] In the regulated region management system according to the first aspect of the present disclosure, the one or more processors may be configured to send the regulated region information to the mobile body.
[0014] In the regulated region management system according to the first aspect of the present disclosure, the one or more processors are configured to: extract coordinate information and color information from the hand-drawn input image; determine the regulated region based on the surrounding map and the coordinate information; determine a regulation type based on the color information; and generate information indicating the regulated region and information indicating the regulation type as regulated region information. This allows the manager to input a regulated region in more detail.
[0015] In the regulated region management system according to the first aspect of the present disclosure, when identification information about the mobile body is acquired along with the hand-drawn input image and the mobile body is not remotely operated using the operation terminal, the one or more processors may be configured to transmit the regulated region information to the mobile body with the identification information. When the identification information about the mobile body is acquired along with the hand-drawn input image and the mobile body is remotely operated with the identification information, the one or more processors may be configured to transmit the regulated region information to the operation terminal used to remotely operate the mobile body with the identification information.Consequently, it is possible to apply regulations flexibly by designing the regulated region differently for the types of mobile bodies or for the mobile bodies.
[0016] A second illustrative aspect of the present disclosure provides a mobile body management system comprising: the regulated region management system according to the first aspect of the present disclosure; and the mobile body.
[0017] In the mobile body management system according to the second aspect of the present disclosure, the mobile body may be configured to generate a movement plan for autonomous movement of the mobile body based on the surrounding map and the regulated region information in response to receiving the regulated region information when the mobile body moves autonomously. This allows the mobile body to move autonomously while avoiding the regulated region.
[0018] In the mobile body management system according to the second aspect of the present disclosure, the mobile body may include a movable portion that behaves differently from a movement of the mobile body in a traveling direction. The mobile body may be configured to determine a movement range of the mobile body when the mobile body moves autonomously based on the regulated region information and the movement plan, and to create an operation plan for the movable portion of the mobile body based on the movement range. Autonomous movement of the mobile robot may be appropriately supported by reflecting the regulated region information in the operation plan for the entire mobile body during autonomous movement.
[0019] The mobile body management system according to the second aspect of the present disclosure may further include the operation terminal associated with the mobile body. The operation terminal may be configured to display the regulated region in response to receiving the regulated region information and, upon receiving an operation instruction from an operator, transmit the operation instruction to the corresponding mobile body. This allows the mobile body to be remotely operated to bypass the regulated region.
[0020] In the mobile body management system according to the second aspect of the present disclosure, the mobile body can be configured to create a movement plan to avoid the regulated region or stop moving when the mobile robot moves according to the operation instruction received from the corresponding operation terminal, in a case where a prediction is made that the mobile robot will enter the regulated region. Consequently, it is possible to reduce the burden on the operator and facilitate coexistence between humans and the mobile body.
[0021] A third explanatory aspect of the present disclosure provides a regulated region management method, which includes: acquiring a hand-drawn input image input to a surrounding map; and determining a regulated region based on the surrounding map and the hand-drawn input image. The regulated region is a region to which access of a mobile body is regulated. The regulated region management method includes: transmitting regulated region information indicating the regulated region to the mobile body when the mobile body is not remotely operated using an operation terminal; and transmitting the regulated region information to the operation terminal when the mobile body is remotely operated using the operation terminal. This allows a manager to input a regulated region through an intuitive operation.Furthermore, it is possible to immediately inform a subject creating a movement plan about a regulated region that reflects human will. This allows for real-time intervention based on human will. It also makes it possible to avoid the effort of re-creating an environment map in real time.
[0022] A fourth illustrative aspect of the present disclosure provides a non-transitory storage medium that stores instructions executable by one or more processors and that cause the one or more processors to perform functions. The functions include: obtaining a hand-drawn input image input to an environment map; and determining a regulated region based on the environment map and the hand-drawn input image. The regulated region is a region to which access of a mobile body is regulated.The functions include: sending information about a regulated region indicating the regulated region to the mobile body when the mobile body is not remotely operated using an operation terminal; and sending the information about a regulated region to the operation terminal when the mobile body is remotely operated using the operation terminal. This allows a manager to input a regulated region through an intuitive operation. It is also possible to immediately inform a subject creating a movement plan of a regulated region reflecting human will. This allows real-time intervention based on human will. It is also possible to save the trouble of re-drawing an environment map in real time.
[0023] With the present invention, it is possible to provide a regulated region management system, a mobile body management system, a regulated region management method, and a non-transitory storage medium that allow a manager to designate an access-regulated region for a mobile body that is autonomously movable by an intuitive operation.
[0024] Features and advantages as well as a technical and economic significance of exemplary embodiments of the invention are described below with reference to the accompanying drawings, in which like reference numerals designate like elements; in this figure: Fig. 1 is a block diagram showing the configuration of a mobile body management system according to a first embodiment; Fig. 2 is a perspective view showing an example of the appearance of a robot according to the first embodiment; Fig. 3 is a block diagram showing the functional configuration of the robot according to the first embodiment; Fig. 4 is a view of an environment of the robot according to the first embodiment; Fig. 5 is a view of an environment of the robot according to the first embodiment; Fig. 6 is a block diagram showing the functional configuration of a management terminal according to the first embodiment; Fig. 7 shows an example of a hand-drawn input image according to the first embodiment; Fig. 8 is a block diagram showing the functional configuration of a server according to the first embodiment; Fig. 9 is a flowchart illustrating the flow of a regulated region management process performed by the server according to the first embodiment; Fig. 10 is a block diagram showing the functional configuration of an operation terminal according to the first embodiment; Fig. 11 shows an example of a display on the operating terminal according to the first embodiment; Fig. 12 is a sequence diagram illustrating the flow of a process performed by the mobile body management system according to the first embodiment; Fig. 13 is a sequence diagram illustrating the flow of a process performed by the mobile body management system according to the first embodiment; Fig. 14 shows an example of a hand-drawn input image according to a second embodiment; Fig. 15 shows an example of a hand-drawn input image according to the second embodiment; Fig. 16 shows an autonomous control of a remotely operated robot according to a third embodiment; and Fig. 17 is a flowchart showing the flow of a movement process performed when the robot according to the third embodiment moves.
[0025] A problem of the present embodiment will be described in detail.
[0026] A robot creates a movement plan by determining whether the robot can move through a region in the robot's environment based on an environment map and information about other access-controlled regions.
[0027] The environment map is maintained by the robot and includes information about static, access-controlled regions. Static, access-controlled regions are regions containing static obstacles, such as walls, pillars, and other stationary obstacles. Based on the environment map, the robot determines that regions designated as static, access-controlled regions are impassable and avoids such regions while moving. The environment map is generated in advance based on the results of scanning by sensors attached to the robot.
[0028] The other access-controlled regions include dynamic access-controlled regions and static access-controlled regions that are not yet reflected in the environment map. Dynamic access-controlled regions are regions containing dynamic obstacles. Such access-controlled regions are detected by the sensors attached to the robot. When access-controlled regions are detected, the robot determines that such regions are impassable and avoids such regions while moving.
[0029] However, it is necessary to redraw an environment map every time the layout changes or a new object is placed, which is tedious. Furthermore, the environment map may occasionally not be updated in a timely manner to reflect environmental changes.
[0030] There is a limitation to the robot's ability to detect all obstacles using its sensors. For example, it is difficult to detect thin, linear objects, small steps, small holes, water surfaces, liquids spilled on the floor, etc. using the sensors. If a system includes multiple robots, it is necessary to attach a large number of sensors to all robots to detect all obstacles, which is costly.
[0031] Even obstacles that can be detected using the sensors can only be detected by the onboard sensors from close proximity. For example, if a path is blocked by an obstacle located around a corner, the robot must approach the blockage to confirm that the path is impassable, then re-create a movement plan, and return through the path the robot once passed through. Therefore, robot movement requires time.
[0032] There are some places where there are no obstacles and the robot can physically pass through, but where it's not practical for the robot to pass through due to human convenience. Examples of places where it's not practical for the robot to pass through include places on human-carrying roads, places where people are working, and narrow paths. However, there has been no means to move the robot while avoiding such places.
[0033] As described above, updating access-controlled regions in a complex environment where the positions of people and objects vary presents challenges. These include: updating is laborious and may occasionally not be timely for environmental changes; using expensive sensors represents a significant cost trade-off; or failing to reflect locations that are not convenient for human convenience.
[0034] To address at least one of these problems, it is desirable for a manager to manually update access-controlled regions that are not reflected in the environment map. For example, it is desirable for a manager to manually update an access-controlled region that forms a bottleneck while the robot is essentially moving autonomously. There is a need to allow the manager to intuitively mark access-controlled regions on a map in order to manually update the access-controlled regions.
[0035] The following embodiments allow the manager to intuitively mark access-controlled regions.
[0036] Embodiments of the present invention will be described in detail below with reference to the drawings. In the drawings, like or corresponding elements are designated by like reference numerals, and redundant description is omitted for clarity.
[0037] First, a first embodiment of the present invention will be described. Fig. 1 is a block diagram illustrating the configuration of a mobile body management system 1 according to the first embodiment. The mobile body management system 1 is a computer system that manages a mobile body.
[0038] The mobile body management system 1 comprises one or more robots 10, a management terminal 20, a server 30 and an operation terminal 40, which are configured to be able to communicate with each other via a network N.
[0039] Network N is a wired or wireless network. Network N can be a local area network (LAN), a wide area network (WAN), the Internet, and / or other lines, or a combination thereof.
[0040] The robot 10 is an example of a mobile robot capable of autonomous movement. The robot 10 maintains an environment map and regulated region information. The environment map is information on a map of an environment of the robot 10. The environment map includes information about the positions of static obstacles. The regulated region information is information about regulated regions to which access for a mobile body is regulated. The regulated regions indicated in the regulated region information are not reflected in the environment map. The regulated regions indicated in the regulated region information may include static regulated regions and dynamic regulated regions. In the first embodiment, the regulated region information indicates information about the positions of the regulated regions.The information about a regulated region is received by the server 30.
[0041] The robot 10, which moves autonomously, creates a movement plan based on the environment map and the information about a regulated region and moves autonomously according to the movement plan.
[0042] In the first embodiment, the robot 10 can be remotely operated by an operator of the operation terminal 40. The remotely operated robot 10 moves according to an operation instruction received from the operation terminal 40.
[0043] The management terminal 20 is a terminal device that manages regulated regions. The management terminal 20 is, for example, a tablet terminal. However, the management terminal 20 is not limited to this and may be a smartphone or a personal computer. The management terminal 20 receives an input of a hand-drawn input image for the surrounding map from a manager who is a user of the management terminal 20 and sends the received hand-drawn input image to the server 30. The hand-drawn input image is information about an image drawn on the surrounding map by the manager who inputs it using an input device such as a touch panel or a mouse.
[0044] The server 30 is a server computer that is an example of a controlled region management system. The server 30 creates controlled region information in response to receiving a hand-drawn input image input to the environment map. The server 30 sends the controlled region information to the one or more robots 10. For the robot 10 that is remotely operated, the server 30 sends the controlled region information to the robot 10 and additionally to the operator terminal 40 used to operate the robot 10.
[0045] The operation terminal 40 is a terminal device that outputs an operation instruction for remotely operating the robot 10. The operation terminal 40 is associated with each robot 10 being remotely operated. Based on the surrounding area map and the information about a controlled region displayed on the operation terminal 40, the operator inputs an operation instruction. The operation terminal 40, having received the input operation instruction, transmits the operation instruction to the robot 10.
[0046] In this way, the robot 10 moves autonomously based on the environment map and the information about a regulated region or it is operated remotely.
[0047] Fig. Fig. 2 is a perspective view showing an example of the appearance of the robot 10 according to the first embodiment. In Fig. 2 shows the appearance of the robot 10, which includes a tool having a gripping function, as an example of the robot 10. The robot 10 is roughly divided into a carriage section 110 and a main body section 120. The carriage section 110 is a movable section that contributes to movement of the robot 10. The carriage section 110 includes two drive wheels 111 and a roller 112, which are supported in a cylindrical casing and contact a traveling surface. The two drive wheels 111 are arranged such that the directions of their respective rotation axes coincide with each other. The drive wheels 111 are independently driven by an electric motor (not shown) to rotate. The roller 112 is an idler wheel.The roller 112 is configured such that a rotating shaft extending vertically from the carriage section 110 supports a wheel located away from the rotating shaft. The roller 112 follows the direction of movement of the carriage section 110.
[0048] The carriage section 110 includes a laser scanner 133 on the peripheral edge area of the upper surface. The laser scanner 133 scans a certain range within a horizontal plane at each step angle and outputs whether an obstacle is present in each direction. If an obstacle is present, the laser scanner 133 outputs the distance to the obstacle.
[0049] The main body portion 120 includes a movable portion that behaves differently than the movement of the robot 10 in the traveling direction. Specifically, the main body portion 120 includes a body portion 121 fixed to the upper surface of the carriage portion 110, a head portion 122 placed on the upper surface of the body portion 121, an arm 123 supported on the side surface of the body portion 121, a hand 124 installed at the distal end portion of the arm 123, and a handheld camera 135. The arm 123 and the hand 124 are driven by an electric motor (not shown). The hand 124 grasps an object to be grasped. The body portion 121 is rotatable about a vertical axis with respect to the carriage portion 110 using a driving force from an electric motor (not shown).
[0050] The head section 122 mainly includes stereo cameras 131 and a display unit 141. The stereo camera 131 includes two camera units with the same viewing angle. The two camera units are arranged facing away from each other. The stereo cameras 131 output image signals obtained by the respective camera units.
[0051] The display unit 141 is, for example, a liquid crystal panel and displays the face of a set character using animation, or displays information about the robot 10 using a text or a symbol.
[0052] The head portion 122 is rotatable about a vertical axis relative to the body portion 121 using a driving force of an electric motor (not shown). Therefore, the stereo cameras 131 can capture images in a desired direction. The display unit 141 can present the display content in a desired direction.
[0053] Fig. 3 is a block diagram illustrating the functional configuration of the robot 10 according to the first embodiment. The robot 10 includes a control unit 150, a chassis drive unit 145, an upper body drive unit 146, the display unit 141, the stereo cameras 131, the laser scanner 133, a memory 180, the handheld camera 135, and a communication unit 190. The upper body drive unit 146, the display unit 141, the stereo cameras 131, the laser scanner 133, and the handheld camera 135 may be omitted.
[0054] The control unit 150 is at least one processor, such as a central processing unit (CPU). The control unit 150 is stored in a control unit formed, for example, in the body portion 121. The control unit 150 performs control of the entire robot 10 and various computing processes by executing a control program read from the memory 180.
[0055] When the communication unit 190 receives regulated region information R from the server 30, the control unit 150 stores the regulated region information R in the memory 180. Then, the control unit 150 creates a movement plan P based on an environment map M and the regulated region information R stored in the memory 180. The control unit 150 can store the created movement plan P in the memory 180. When the laser scanner 133, explained later, detects an obstacle, the control unit 150 can again create a movement plan P to avoid the obstacle. At this time, the control unit 150 can add the position of the obstacle to the regulated region information R.
[0056] In an autonomous movement mode, the control unit 150 executes control to rotate the drive wheels by sending a drive signal to the carriage drive unit 145 according to the last movement plan P stored in the memory 180. The control unit 150 detects the direction and movement speed of the carriage section 110 by receiving a feedback signal from an encoder, etc., from the carriage drive unit 145.
[0057] The chassis drive unit 145 includes the drive wheels 111 and a drive circuit and an electric motor for driving the drive wheels 111.
[0058] The upper body drive unit 146 includes the arm 123, the hand 124, the body portion 121, the head portion 122, and a drive circuit and an electric motor for driving such components. The control unit 150 achieves an extension operation, a grasping operation, and gestures by sending a drive signal to the upper body drive unit 146. The control unit 150 detects the position and movement speed of the arm 123 and the hand 124, and the direction and rotation speed of the body portion 121 and the head portion 122 by receiving a feedback signal from an encoder, etc., from the upper body drive unit 146.
[0059] The display unit 141 receives and displays an image signal generated by the control unit 150.
[0060] The stereo cameras 131 capture an image of the environment in which the robot 10 is located and deliver an image signal to the control unit 150 according to a request from the control unit 150. The control unit 150 performs image processing using the image signal or converts the image signal into a captured image according to a format determined in advance. The laser scanner 133 detects whether an obstacle is present in the movement direction according to a request from the control unit 150 and delivers a detection signal indicating the detection result to the control unit 150.
[0061] The handheld camera 135 is, for example, a distance image sensor. The handheld camera 135 is used to determine the distance, shape, direction, etc. of the object to be detected. The handheld camera 135 includes imaging elements that perform photoelectric conversion on an optical image captured from an object space. The imaging elements are arranged two-dimensionally in the handheld camera 135. The handheld camera 135 outputs the distance to the subject for each imaging element to the control unit 150. Specifically, the handheld camera 135 includes an irradiation unit that irradiates an object space with pattern light. The handheld camera 135 receives reflected light using the imaging elements and outputs the distance to the subject captured by the imaging elements based on distortion or the size of the pattern in the image.The control unit 150 captures a scene in a wider environment using the stereo cameras 131 and captures a scene near the object to be captured using the handheld camera 135.
[0062] The memory 180 is a non-volatile storage medium and can be, for example, a solid-state drive. In addition to a control program for controlling the robot 10, the memory 180 stores various parameter values, functions, lookup tables, etc., used for control and calculation. In particular, the memory 180 stores the environment map M, the information R about a controlled region, and the motion plan P.
[0063] The communication unit 190 is an interface for communication with the network N and may be, for example, a wireless LAN unit. The communication unit 190 receives the regulated region information R sent from the server 30 and delivers the regulated region information R to the control unit 150. The communication unit 190 can send the image captured by the stereo cameras 131 to the server 30 according to control by the control unit 150.
[0064] Fig. 4 and Fig. 5 are views of an environment of the robot 10 according to the first embodiment. Regions in the dark colors in the Fig. 4 and Fig. 5 are regions where an obstacle is located and are normally regulated regions. The robot 10 is traveling in a space surrounded by obstacles, near a corner of a room. The shortest path for the robot 10 to travel to the center of the room (in Fig. 4 (top right) includes a motion path 600 that passes through a space between an obstacle 501 and an obstacle 502. Therefore, the robot 10 normally includes the motion path 600 in the motion plan and moves along the motion path 600 according to the motion plan.
[0065] A case is considered in which, in addition to the normal regulated regions, the space between the obstacle 501 and the obstacle 502 is converted into a regulated space, such as a case in which a cable is placed in the space between the obstacle 501 and the obstacle 502. In this case, the robot 10 receives information R about a regulated region which, in addition to the normal regulated regions, includes an additional regulated region 503 positioned between the obstacle 501 and the obstacle 502, as shown in Fig. 5, from the server 30. Therefore, based on the new regulated region information R, the robot 10 recreates a motion plan to bypass the additional regulated region 503 in addition to the normal regulated regions. The recreated motion plan includes a motion path 601 that allows the regulated regions to be bypassed and a destination to be reached via the shortest path.
[0066] This allows the robot 10 to move over a regulated region based on the last information R.
[0067] Fig. Figure 6 is a block diagram illustrating the functional configuration of the management terminal 20 according to the first embodiment. The management terminal 20 includes a memory 200, a communication unit 210, an input unit 220, a display unit 230, and a control unit 240.
[0068] The memory 200 is a non-volatile storage medium and can be, for example, a solid-state drive. In addition to a control program for controlling the management terminal, the memory 200 stores 20 different parameter values, functions, lookup tables, etc., which are used for control and calculation.
[0069] The communication unit 210 is an interface for communication with the network N. The communication unit 210 receives an environment map sent from the server 30 and delivers the environment map to the control unit 240. In cooperation with the control unit 240, the communication unit 210 sends a hand-drawn input image associated with the environment map to the server 30.
[0070] The input unit 220 includes a touch panel arranged to be superimposed on the display unit 230, push buttons formed on the peripheral edge portion of the display unit 230, and so on. The input unit 220 receives a hand-drawn input image input by the manager by touching the touch panel to designate a regulated region, and supplies the hand-drawn input image to the control unit 240.
[0071] The display unit 230 may be, for example, a liquid crystal panel. The display unit 230 displays, for example, an environment map sent from the server 30. The display unit 230 also displays the input hand-drawn input image superimposed on the environment map.
[0072] The control unit 240 is a processor such as a CPU, and performs control of the entire management terminal 20 and various calculation processes by executing a control program read from the memory 200.
[0073] Fig. 7 illustrates an example of the hand-drawn input image according to the first embodiment. The display unit 230 of the management terminal 20 displays an environment map within a predetermined range with reference to the current position of the robot 10. The manager inputs a hand-drawn input image 700 into the displayed environment map.
[0074] A method for inputting a hand-drawn input image includes directly inputting a hand-drawn input image by touching a relevant portion on the touch panel using a user's finger, a stylus, etc. However, the method for inputting a hand-drawn input image is not limited to this. For example, a hand-drawn input image may be input by selecting a predetermined figure using a mouse, etc., and designating the position and size of the figure. A hand-drawn input image may be input as a two-dimensional line or figure, or may be input as a three-dimensional object.
[0075] Fig. Figure 8 is a block diagram illustrating the functional configuration of the server 30 according to the first embodiment. The server 30 includes a memory 310, a communication unit 320, and a control unit 330.
[0076] Memory 310 is a non-volatile storage medium and can be, for example, a solid-state drive. In addition to a control program for controlling server 30, memory 310 stores various parameter values, functions, lookup tables, etc., used for control and calculation. In particular, memory 310 stores an environment map M and conversion information T.
[0077] The environment map M stored in the memory 310 is map information which is the same as the environment map M maintained by the robot 10.
[0078] The conversion information T is information for converting a hand-drawn input image into attribute information, such as the position, shape, size, etc. of a regulated region. The conversion information T may include, for example, a template for template matching and various parameters. The conversion information T may also include a trained model that receives an input of a hand-drawn input image and outputs the attribute information. The trained model may include a neural network.
[0079] The communication unit 320 is an interface for communication with the network N. The communication unit 320 receives a hand-drawn input image associated with the environment map sent from the management terminal 20 and delivers the hand-drawn input image to the control unit 330. In addition, the communication unit 320 sends the regulated region information R to the robot 10 in cooperation with the control unit 330.
[0080] The control unit 330 is a processor such as a CPU and performs control of the entire server 30 and various computing processes by executing a control program read from the memory 310. Referring to Fig. 9 describes a specific process flow.
[0081] Fig. 9 is a flowchart illustrating the flow of a regulated region management process performed by the server 30 according to the first embodiment.
[0082] First, when the communication unit 320 of the server 30 receives a hand-drawn input image associated with the surrounding map M from the management terminal 20 (step S10), the control unit 330 determines the position and range of a regulated region based on the surrounding map M and the hand-drawn input image (step S11). For example, the control unit 330 may read the conversion information T stored in the memory 310 and determine the position of a reference point of a regulated region on the surrounding map and the shape and size of the regulated region from the hand-drawn input image using the conversion information T. The reference point may be, for example, the center of gravity.
[0083] Next, the control unit 330 generates regulated region information R indicating the detected position and range of the regulated region (step S12).
[0084] In addition, the control unit 330 transmits the regulated region information R to all robots 10 via the communication unit 320. For a remotely operated robot 10, the control unit 330 also transmits the regulated region information R to an operation terminal that remotely operates the robot 10. Specifically, the control unit 330 repeatedly performs the processes specified in steps S13 to S16 for each of the robots 10.
[0085] First, in step S13, the control unit 330 determines whether the robot 10 is being remotely operated. If the robot 10 is being remotely operated (Yes in step S13), the control unit 330 transmits the regulated region information R to the robot 10 and the operation terminal 40 that operates the robot 10 (step S14).
[0086] On the other hand, if the robot 10 is not remotely operated and the robot 10 is moving autonomously (No in step S13 and Yes in step S15), the control unit 330 transmits the regulated region information R to the robot 10 (step S16). At this time, if there is an operation terminal 40 corresponding to the robot 10, the control unit 330 may or may not transmit the regulated region information R to the operation terminal 40.
[0087] When the robot 10 is not remotely operated and does not move autonomously, such as when the robot 10 is not started (No in step S13 and No in step S15), the control unit 330 may or may not send the regulated region information R to the robot 10.
[0088] Information about whether the robot 10 is being remotely operated, whether the robot 10 is moving autonomously, whether the robot 10 has not started, etc., can be sent in advance from each robot 10 to the server 30.
[0089] Fig. Figure 10 is a block diagram illustrating the functional configuration of the operation terminal 40 according to the first embodiment. The operation terminal 40 includes a memory 400, a communication unit 410, an operation unit 420, a display unit 430, and a control unit 440.
[0090] The memory 400 is a non-volatile storage medium and can be, for example, a solid-state drive. In addition to a control program for controlling the operator terminal, the memory 400 stores 40 different parameter values, functions, lookup tables, etc., which are used for control and calculation.
[0091] The communication unit 410 is an interface for communicating with the network N. The communication unit 410 sends an operation instruction, which is an input command, to the robot 10 in cooperation with the control unit 440. The communication unit 410 receives information indicating an environment of the corresponding robot 10 from the server 30 and delivers the information to the control unit 440. The information indicating an environment of the corresponding robot 10 may be the environment map M or may consist of images of an environmental scene captured by the stereo cameras 131 of the corresponding robot 10. The communication unit 410 delivers the regulated region information R received from the server 30 to the control unit 440.
[0092] The operating unit 420 is an operating unit that receives an input of an operating instruction from an operator. A method for inputting an operating instruction may be, for example, a command input.
[0093] The display unit 430 is, for example, a liquid crystal panel and displays various information. The display unit 430 displays, for example, information indicating an environment of the robot 10.
[0094] The control unit 440 is a processor such as a CPU and performs control of the entire operation terminal 40 and various calculation processes by executing a control program read from the memory 400.
[0095] Fig. 11 illustrates an example of a display on the operation terminal 40 according to the first embodiment. The display unit 430 of the operation terminal 40 displays, for example, an image captured by the stereo cameras 131 of the corresponding robot 10. The display unit 430 displays an object OB indicating a regulated region as superimposed at a position determined based on the regulated region information R on the captured image, controlled by the control unit 440. This allows the operator to intuitively grasp the position and range of the regulated region, along with the surroundings of the robot 10 within the field of view of the robot 10. Thus, the operator can easily prepare an appropriate movement plan for the immediate future.
[0096] When the environment map M is received as information indicating the environment map, the display unit 430 can display an object indicating a regulated region as superimposed on a predetermined region of the environment map M, which is determined with reference to the corresponding robot 10 controlled by the control unit 440. This allows the operator to objectively grasp the position and range of the regulated region along with the surroundings of the robot 10 within a wide field of view. Thus, the operator can easily create an appropriate overall movement plan.
[0097] When a hand-drawn input image is input as a three-dimensional object, the operation terminal 40 can display the object OB indicating a regulated region in three dimensions. The same applies to a two-dimensional object. However, the operation terminal 40 can convert the display mode of the object OB from three-dimensional to two-dimensional, or from two-dimensional to three-dimensional, according to the display mode of information indicating the environment of the corresponding robot 10.
[0098] Fig. 12 and Fig. 13 is a sequence diagram illustrating the flow of a process performed by the mobile body management system 1 according to the first embodiment. Although the number of robots 10 is shown as one for simplicity in the drawing, this is not limited.
[0099] The robot 10 may send images of an environmental scene captured by the stereo cameras 131 and information indicating whether the robot 10 is being remotely operated or moving autonomously to the server 30 at predetermined time intervals while the robot 10 is moving. The server 30, which receives the received images, may acquire information about the position of the robot 10 based on the captured images. The means for acquiring information about the position of the robot 10 is not limited to this, and the robot 10 may use information from a global positioning system (GPS). When the robot 10 is remotely operated, the server 30 may send images captured by the robot 10 to the operation terminal 40 at predetermined time intervals and cause the display unit 430 to display the captured images.
[0100] Fig. 12 shows a flow for a case in which the robot 10 moves autonomously.
[0101] When there is a request from the management terminal 20 to obtain an environment map (step S100), the server 30 sends the environment map M stored in the memory 310 to the management terminal 20 (step S101).
[0102] The management terminal 20 displays the received environment map M on the display unit 230 (step S102). Then, the management terminal 20 receives an input of a hand-drawn input image from the manager (step S103). The management terminal 20 sends the hand-drawn input image to the server 30 in association with the environment map M (step S104).
[0103] The server 30 determines a regulated region using the conversion information T based on the hand-drawn input image and the environment map M (step S105). Then, the server 30 creates regulated region information R. Then, the server 30 determines the operating state of the robot 10 (step S106). That is, the server 30 determines whether the robot 10 is being remotely operated or moving autonomously. Fig. 12, the robot 10 moves autonomously, and therefore the server 30 sends the regulated region information R to the robot 10 (step S107) and does not send the regulated region information R to the operator terminal 40.
[0104] The robot 10, having received the regulated region information R, changes a movement plan based on the regulated region information R and the environment map M maintained by the robot 10 itself (step S108). Then, the robot 10 moves autonomously according to the changed movement plan (step S109).
[0105] Fig. 13 shows a flow for a case where the robot 10 is operated remotely.
[0106] The mobile body management system 1 executes processes similar to those in steps S100 to S105. Then, the server 30 determines the operating state of the robot 10 (step S120). Fig. 13, the robot 10 is operated remotely, and therefore the server 30 sends the regulated region information R to both the robot 10 and the operation terminal 40 (steps S121 to S122). The server 30 may only send the regulated region information R to the operation terminal 40 and may not send the regulated region information R to the robot 10 because the robot 10 is operated remotely.
[0107] The operation terminal 40, which receives the regulated region information R, displays an object indicating a regulated region as superimposed on an image captured by the robot 10 (step S123). The operation terminal 40 receives an input of an operation instruction from the manager who acquired the regulated region (step S124) and sends the operation instruction to the robot 10 (step S125).
[0108] The robot 10, which has received the operation instruction, moves according to the operation instruction (step S126).
[0109] With the first embodiment, the manager can thus designate a regulated region through an intuitive operation, i.e., a hand drawing. The regulated region designated by the manager may be a region with a static obstacle that has been changed or added, or may be a region that a robot 10 is not desired to enter due to human will. The server 30 can immediately notify a subject, which creates a movement plan of the thus designated regulated region. This allows the movement range of the robot 10 to reflect a human will in real time. This also eliminates the trouble of preparing an environment map in real time every time the layout is partially changed or a new object is placed.
[0110] The manager can designate an obstacle that is not easily detectable by a sensor as a regulated region. Therefore, it is not necessary to additionally attach a dedicated sensor to the robot 10. The cost of introducing a robot 10 can therefore be reduced. The cost reduction effect becomes particularly noticeable as the number of robots 10 increases.
[0111] The movement time of the robot 10 can be shortened if the manager designates a complicated passage, such as a passage with a dead end due to an obstacle located behind a corner, as a large regulated region in advance.
[0112] In this way, the mobile body management system 1 can facilitate coexistence between humans and robots 10.
[0113] The first embodiment can be modified as follows.
[0114] While the server 30 distributes the regulated region information R to all robots 10 simultaneously, the server 30 may send the regulated region information R, for example, to some of the robots 10 that have been designated. In this case, the management terminal 20 receives, from the manager, along with a hand-drawn input image, an input of identification information for robots 10 to be regulated. The identification information for robots 10 may be information for identifying the intended use or type of robots 10. Alternatively, the identification information for robots 10 may be the model number of robots 10 or the serial number of robots 10. When identification information for robots 10 is obtained along with a hand-drawn input image, the server 30 sends the regulated region information R to the robots 10 with the identification information.When the relevant robots 10 are remotely operated, the server 30 also transmits the regulated region information R to the operating terminals 40 that remotely operate the robots 10. At this time, the server 30 may not transmit the regulated region information R to the other robots 10 and the operating terminals 40 that remotely operate the other robots 10.
[0115] In this way, it is possible to apply regulations flexibly by designing the regulated region differently for the types of robots 10 or for the robots 10.
[0116] While the regulated region information R influences the movement plan for the carriage section 110, an operation plan for the main body section 120, for example, may additionally be influenced. The operation plan determines the direction and extent of future operation of the section and determines the direction and extent of future extension and retraction of the section if the section is extendable and retractable. For example, the robot 10 may create an operation plan for the arm 123 to avoid the regulated region. Specifically, the robot 10 determines a movement range of the robot 10 based on the regulated region information R and the movement plan, and creates an operation plan for the arm 123 of the robot 10 based on the determined movement range.
[0117] In this way, autonomous movement of the robot 10 can be appropriately supported by reflecting the information R about a regulated region in the operation plan for the entire robot 10.
[0118] Next, a second embodiment of the present invention will be described. In the second embodiment, the manager can indicate not only the position and range of a regulated region, but also additional information associated with the regulated region, using a hand-drawn input image.
[0119] For example, the manager may be able to indicate the type of regulation by entering handwritten characters using a finger or a stylus. The type of regulation may be, for example, a height restriction, a width restriction, a speed restriction, or a no-entry restriction. In this case, the information R about a regulated region includes information indicating the position and range of a regulated region and information indicating the type of regulation.
[0120] Fig. 14 illustrates an example of the hand-drawn input image according to the second embodiment. The display unit 230 displays hand-drawn input images 700, 710. The hand-drawn input image 700 indicates the position and range of a regulated region from a plan view. The hand-drawn input image 710 indicates characters indicating the type of regulation. In the present example, the hand-drawn input image 710 indicates the characters "REST," which may indicate that the type of regulation is a speed limit.
[0121] The server 30, receiving such hand-drawn input images, extracts an image region (hand-drawn input image 700) indicating a figure and an image region (hand-drawn input image 710) indicating characters from the hand-drawn input images. Next, the server 30 determines the position and range of a regulated region based on the environment map M and the image region indicating a figure. The server 30 also determines the type of regulation from the image region indicating characters using a known technique such as optical character recognition (OCR). Then, the server 30 creates information indicating a regulated region and information indicating the type of regulation as regulated region information R.
[0122] The manager can then indicate the type of regulation by coloring a hand-drawn input image. In this case, the type of regulation was determined in advance based on the color. For example, green can indicate a speed limit, and blue can indicate a height limit.
[0123] The server 30, having received a hand-drawn input image including a colored figure, extracts coordinate information and color information from the hand-drawn input image. The server 30 then determines the position and range of a regulated region based on the surrounding map M and the coordinate information. The server 30 also determines the type of regulation based on the color information. The server 30 creates information indicating a regulated region and information indicating the type of regulation as regulated region information R.
[0124] The manager may express the type of regulation using a figure different from the figure indicating the position and range of a regulated region in a plan view.
[0125] Fig. 15 illustrates an example of the hand-drawn input image according to the second embodiment.
[0126] For example, if an obstacle is located at a predetermined height or more from the ground, or if the ceiling is low, the type of regulation indicates a height restriction. At this time, the manager inputs two types of figures, that is, for example, a figure (hand-drawn input image 700) indicating the position and range of an obstacle from a plan view, and a figure (hand-drawn input image 720) indicating the presence of an obstacle at a position at a predetermined height or more on the environment map M. The manager can input a specific value of the height at which the obstacle is located using hand-drawn characters associated with the hand-drawn input image 720, as indicated by the hand-drawn input image 740.
[0127] The server 30, having received such hand-drawn input images, extracts image regions (hand-drawn input images 700, 720) indicating a figure and an image region (hand-drawn input image 740) indicating characters from the hand-drawn input images.
[0128] The server 30 extracts an image region indicating the position and range of an obstacle in a plan view (also referred to as a "first figure") and an image region indicating a figure (also referred to as a "second figure") indicating the presence of an obstacle at a location at a predetermined height or higher from the image regions indicating a figure. Then, the server 30 determines the position and range of a regulated region from a plan view based on the first figure and determines the presence of the regulated region at a location at a predetermined height or higher based on the second figure. For the image region indicating characters, the server 30 determines the specific height using the OCR, etc. explained above.
[0129] With the second embodiment, the manager can intuitively mark a regulated region in more detail.
[0130] When the regulated region information R including information indicating the type of regulation is received, the robot 10 moving autonomously can create a movement plan as follows.
[0131] First, the robot 10 creates a movement path based on the environment map M, the position and range of a regulated region, and the type of regulation. Then, the robot 10 determines the movement mode in the regulated region based on the type of regulation.
[0132] For example, if the type of regulation is entry prohibition, the robot 10 creates a movement path to avoid the regulated region. If the type of regulation is speed restriction, for example, a movement path that includes the regulated region is created. However, the robot 10 sets the speed in the regulated region within a restricted speed range. If the type of regulation is, for example, width restriction or height restriction, it is determined by considering the dimensions of the robot 10. The robot 10 can create a movement path including the regulated region if the robot 10 can enter the regulated region, and can create a movement path to avoid the regulated region if the robot 10 cannot enter the regulated region.When the robot 10 cannot enter the regulated region, but the robot 10 can enter the regulated region with an extendable / retractable movable portion, such as the arm 123, retracted vertically or laterally, the robot 10 creates a movement path including the regulated region and creates an actuation plan for the movable portion to cause the movable portion to retract when the robot 10 passes through the regulated region.
[0133] Next, a third embodiment of the present invention will be described. In the third embodiment, the remotely operated robot 10 autonomously controls itself to avoid a regulated region, contrary to an operation instruction from the operation terminal 40, when it is predicted that the robot 10 will enter the regulated region if the robot 10 is moved according to the operation instruction.
[0134] Fig. 16 illustrates autonomous control of the remotely operated robot 10 according to a third embodiment. A motion path 620 indicates a motion path for the robot 10 according to an operation instruction. A water surface 504, designated as a regulated region, is located on the motion path 620. Therefore, if the robot 10 continues to follow the operation instruction, it is predicted that the robot 10 will be detected by the water surface 504. Therefore, the robot 10 autonomously changes the motion path to a motion path 621, which causes the robot 10 to travel in a direction designated by the operation instruction, avoiding the water surface 504.
[0135] Fig.17 is a flowchart illustrating the flow of a movement process performed when the robot 10 according to the third embodiment moves. First, the robot 10 receives the regulated region information R from the server 30 (step S20). If the robot 10 is not remotely operated but moves autonomously (No in step S21), a movement plan is created based on the environment map M and the regulated region information R (step S22). Then, the robot 10 moves autonomously based on the movement plan (step S23).
[0136] On the other hand, if the robot 10 is operated remotely (Yes in step S21), an operation instruction is received from the operation terminal 40 (step S24). Next, the robot 10 predicts a movement path based on the operation instruction (step S25). The robot 10 determines whether the predicted movement path passes through a regulated region (step S26). If the predicted movement path does not pass through a regulated region (No in step S26), the robot 10 moves based on the operation instruction (step S27). On the other hand, if the predicted movement path passes through a regulated region (Yes in step S26), the robot 10 determines whether the regulated region can be avoided (step S28). Specifically, the robot 10 determines whether it is possible to create a movement plan that avoids the regulated region while traveling in a direction specified by the operation instruction.If the regulated region can be avoided (Yes in step S28), the robot 10 moves in a direction specified by the operation instruction while avoiding the regulated region based on the newly generated motion plan (step S29). Conversely, if the regulated region cannot be avoided (No in step S28), the movement of the robot 10 is forcibly stopped (step S30).
[0137] With the third embodiment, the robot 10 thus autonomously controls itself in an emergency while being remotely operated. Thus, the mobile body management system 1 can reduce the burden on the operator and facilitate coexistence between humans and the robot 10.
[0138] Next, application examples of the mobile body management system 1 according to the first to fourth embodiments will be described.
[0139] The mobile body management system 1 can be used to manage robots on construction sites. For example, if work is expected on the ceiling or at a height, the manager designates a region below the work location as a regulated region for the robot 10 moving on the ground. It is difficult for the robot 10 moving on the ground to recognize a work location at height. However, with the mobile body management system 1, the manager can designate a regulated region, thus preventing the robot 10 from moving to a dangerous location.
[0140] The mobile body management system 1 can be used to manage robots in commercial facilities. For example, when preparing a venue for a special event, the manager can easily inform the robot 10 of the layout of walls and objects at the venue using the management terminal 20. This allows the robot 10 to detect traffic patterns at the venue.
[0141] The mobile body management system 1 can be used to manage robots for transportation in a warehouse. For example, if an object is temporarily placed on the floor of a passageway, narrowing the passageway, the manager uses the management terminal 20 to designate a regulated region to block the passageway. At this time, the manager can designate the type of regulation as either no-entry or no-passage. This allows the robot 10 to move around the narrowed passageway.
[0142] The mobile body management system 1 can be used to manage outdoor robots. For example, if there is a body of water in an outdoor area, the manager sets a controlled region surrounding the body of water using the management terminal 20. This allows the robot 10 to move around the body of water, preventing a malfunction due to immersion in water.
[0143] The present invention is not limited to the above embodiments and can be modified as appropriate without departing from the spirit and scope of the present invention. While the server 30 transmits the regulated region information R to the robot 10 and the operator terminal 40 each time the regulated region is updated in the above-described embodiments, the server 30 may only transmit updated information.
[0144] While the operation terminal 40 sends an operation instruction to the robot 10 via the network N, the operation terminal 40 may send an operation instruction directly to the robot 10 through wireless near field communication, etc., or may send an operation instruction to the robot 10 via the server 30.
[0145] While the regulated region information R is created by the server 30, the regulated region information R may be created by the management terminal 20. In this case, the management terminal 20 may not send the regulated region information R to the robot 10 via the server 30.
[0146] When determining a period for which regulation is enforced in a regulated region, the management terminal 20 may receive an input of the validity period along with a hand-drawn input image, and may send the information regarding the validity period along with the hand-drawn input image to the server 30. In this case, the server 30 may manage the validity period for each regulated region. Then, when the validity period of the regulated region expires, the server 30 may update the regulated region information R to delete the relevant regulated region. The server 30 may provide the updated regulated region information R to the robots 10 and the operator terminal 40 according to the update of the regulated region information R.
[0147] Although the robot 10 was mentioned as an example of a mobile body, this is not limiting, and the mobile body may be a vehicle, an unmanned aerial vehicle (drone), etc. The mobile body can essentially be moved by remote operation. In this case, the autonomous movement function can be omitted.
Claims
[1] A regulated region management system comprising one or more processors (330) configured to: obtain a hand-drawn input image that is input into an environment map; determine a regulated region based on the environment map and the hand-drawn input image, wherein the regulated region is a region to which access of a mobile body (10) is regulated; transmitting regulated region information indicating the regulated region to the mobile body (10) when the mobile body (10) is not remotely operated using an operation terminal (40); to send the information about a regulated region to the operating terminal (40) when the mobile body (10) is remotely operated using the operating terminal (40), Extract coordinate information and color information from the hand-drawn input image; to determine the regulated region based on the surrounding map and coordinate information; to determine a type of regulation based on the color information; and To generate information indicating the regulated region and information indicating the type of regulation as information about a regulated region. [2] The regulated region management system according to claim 1, wherein the one or more processors (330) are configured to send the regulated region information to the mobile body (10). [3] The regulated region management system of claim 1 or 2, wherein the one or more processors (330) are configured to: when identification information about the mobile body (10) is acquired together with the hand-drawn input image and the mobile body (10) is not remotely operated using the operation terminal (40), transmitting the regulated region information to the mobile body (10) with the identification information; and when the identification information about the mobile body (10) is acquired together with the hand-drawn input image and the mobile body (10) is remotely operated with the identification information, sending the information about a regulated region to the operation terminal (40) used to remotely operate the mobile body (10) with the identification information. [4] Mobile body management system comprising: the regulated region management system according to any one of claims 1 to 3; and the mobile body (10). [5] The mobile body management system according to claim 4, wherein the mobile body (10) is configured to generate a movement plan for autonomous movement of the mobile body (10) based on the surrounding map and the regulated region information in response to receiving the regulated region information when the mobile body (10) moves autonomously. [6] A mobile body management system according to claim 5, wherein: the mobile body (10) comprises a movable portion which behaves differently than a movement of the mobile body (10) in a direction of travel; and the mobile body (10) is configured when the mobile body (10) moves autonomously, to determine a movement range of the mobile body (10) based on the information about a regulated region and the movement plan, and to create an actuation plan for the movable section of the mobile body (10) based on the range of movement. [7] A mobile body management system according to any one of claims 4 to 6, further comprising the operating terminal (40) associated with the mobile body (10) and configured to indicate the regulated region in response to the information being received via a regulated region, and when an operating instruction is received from an operator, to send the operating instruction to the corresponding mobile body (10). [8] The mobile body management system according to claim 7, wherein the mobile body (10) is configured to, in a case where a prediction is made that the mobile body (10) will enter the regulated region when the mobile body (10) moves according to the operation instruction received from the corresponding operation terminal (40), make a movement plan to avoid the regulated region or stop moving. [9] Administrative procedures for regulated region comprising: Obtaining a hand-drawn input image that is input into an environment map; Determining a regulated region based on the environment map and the hand-drawn input image, wherein the regulated region is a region to which access of a mobile body (10) is regulated; Sending regulated region information indicating the regulated region to the mobile body (10) when the mobile body (10) is not remotely operated using an operation terminal (40); Sending the information about a regulated region to the operating terminal (40) when the mobile body (10) is remotely operated using the operating terminal (40), Extracting coordinate information and color information from the hand-drawn input image; Determine the regulated region based on the surrounding map and coordinate information; Determining a regulation type based on the color information; and Generating information indicating the regulated region and information indicating the type of regulation as regulated region information.
Citation Information
Patent Citations
Nuclear plant manipulator system
DE102013009258A1
Methods for machine learning and mobile robots
DE102020111325A1
Mobile robot and control methods for a mobile robot
DE112017002154B4
Cleaning device and cleaning system
EP4151135A1
Map information providing system for autonomous moving device and autonomous moving device
JP2021077053A
