Robot stopping position setting device and mobile robot system

Abstract

Robot stop position setting device (2) which sets a stop position (P) of a robot (5) comprising the following: a card storage unit (22a) that stores a first card (M1) and a second card (M2); a stop position storage unit (22b) that stores a robot stop position (P1) on the first card (M1), wherein the robot stop position (P) is generated by manual operation of the robot (5) by a user; a map area selection unit (21a) that outputs a first area (R1) containing a stopping reference item selected by the user from the first map (M1), a reference position output unit (22c) that detects a second area (R2) having the same shape as the first area (R1) from the second map (M2) and outputs a position of the second area (R2) as the reference position; and a stopping position calculation unit (22d) that calculates the robot stopping position (P2) on the second map (M2) using the robot stopping position (P1) on the first map (M1), a position of the first area (R1) and the reference position on the second map (M2).

Description

Field of the invention

[0001] The present invention relates to a robot stop position setting device and a mobile robot system. Description of the related field

[0002] In recent years, a mobile robot system that travels around a work site without using a guide such as a magnetic tape has been used to save labor and automate various on-site tasks. Such a robot system generally stores information about a traveling environment in a non-volatile memory device to estimate the position of a traveling robot based on sensor information and to define the robot's stopping position and travel path.

[0003] In such a robot system, if there is a large discrepancy between map information stored in the system and an actual driving environment, the robot may behave inappropriately, so various countermeasures have been taken in the related field.

[0004] As a first example, when an environmental shape changes due to changes in the arrangement of an object in a traveling environment, the robot cannot correctly estimate its own position and may not be able to travel according to a preset stopping position or travel path. On the other hand, a method has been proposed in which a robot travels according to a stopping position and travel path set before an environmental change by updating map information according to a changed environmental shape to enable it to continue self-position estimation, or by changing the stopping position and travel path stored in a mobile robot system.

[0005] As such a method, JP2013-114500A discloses "a service control system for controlling a service provided by a real-world interface system, including: a first observation unit that observes a surrounding physical environment, a work unit that performs a physical operation or information processing based on an observation result of the first observation unit, definition data that defines the operation of the work unit, and an operation instruction unit that controls the operation of the work unit according to the definition data, a second observation unit that observes a different observation target than that of the first observation unit or observes the same observation target by other means, and an update unit that updates the definition data based on a difference between the work unit,which operates on the basis of the observation result of the first observation unit, and the work unit which operates on the basis of an observation result of the second observation unit, updated, contains.,

[0006] As a second example, as a result of a change in the arrangement of an object in a travel environment and an obstacle located on a preset travel path of a robot, the robot may collide with the obstacle or may fail to reach a designated stopping position. To address this, there is technology that updates map information and then replans the stopping position and travel path so that no interference with the obstacle occurs.

[0007] As such a technique, JP2020-087248A discloses a configuration in which “a map generation unit that generates an external world map based on information about the external world and an action plan unit that generates an action plan for a robot device based on the external world map are provided, and the map generation unit generates the external world map based on a map specification generated by the action plan unit.

[0008] In this way, in the related field described above, to ensure that a robot travels as preset even when the traveling environment changes or changes a setting so as not to interfere with an obstacle, map information, a stopping position, and a path are automatically updated. However, automatically updating the data stored in the mobile robot system may result in unintended consequences for a user, so there is also a configuration in which user operation is used as an auxiliary tool.For example, there is a configuration using map information in which a user sets "an update-permitting area and an update-disallowing area" (see JP2014-203145A), and a configuration including an "acceptance / rejection instruction input unit for inputting an acceptance or rejection instruction of the updated section by an operator" (see JP2009-169845A). Furthermore, DE 11 2019 007 750 T5 describes an environment map generation device that generates two partial environment maps in different environments and combines them into a complete map. The probabilities for objects are adjusted depending on whether they occur in both environments. This enables a more accurate estimation of the vehicle's own position even when the environment changes. The generated map is stored and used for the vehicle's autonomous navigation.During the journey, the map is continuously updated using sensor data to account for changes in the real environment. EP 3 048 417 B1 describes a system for visualizing spatial information that is used in complex structures such as nuclear power plants. It uses 3D laser scanners to collect data and create three-dimensional models. The system according to EP 3 048 417 B1 identifies and visualizes free, measured, and unmeasured spaces to plan the placement and movement of objects and robots. US 10 747 229 B2 describes an environment layout robot that creates an environmental map of a target space in which a mobile robot navigates autonomously. The robot uses a distance sensor to divide the space into cells and assigns each cell a score that indicates the probability of an object being present.The robot can modify the environment by adding or removing objects to improve the accuracy of map creation. This environment map helps the mobile robot determine its position precisely and reliably reach its destination. Furthermore, JP 2021-140 583 A describes an object image display system that visualizes the arrangement of objects in a given space. A mobile robot creates an initial drawing of the space through autonomous movement and measurement. This is compared with a second drawing created before the objects were placed to identify and separate the objects. The system displays overlaid images of the objects and the space, allowing objects to be moved and new objects to be added to plan and review the spatial layout.JP 2014-071847 A describes a self-driving electronic device that moves based on route information received over a communications network. It includes a control system that instructs a communications terminal to move the self-driving device. The invention aims to easily and accurately control the movement of the device according to the user's instructions, without the need for ceiling markers or expensive GPS signals. Finally, JP 2006-113858 A describes a system for monitoring and controlling self-driving vehicles within a defined area. It uses sensors and cameras to track the position and movement of the vehicles in real time. The system can detect obstacles and adjust the vehicles' routes accordingly to avoid collisions.It also allows remote control and monitoring of vehicles through a central control system that provides users with an intuitive interface to manage and optimize vehicle movements.

[0009] Thus, according to the related field, even if the driving environment changes in the same location, it is possible to estimate one's own position and drive and stop as instructed without being affected by an environmental change, and it is also possible to change a path and a stopping position so as not to collide with an obstacle.

[0010] However, the related field has the following two problems.

[0011] First, for example, when a mobile robot having a manipulator stops in front of a workbench and operates on a work item placed on the workbench, the robot must stop at a position where it can process the work item on the workbench. When a workbench position is changed in a location, the robot's stop position must also be appropriately changed according to the workbench. However, in the related field, only avoiding interference between a surrounding obstacle and the robot is considered when updating the stop position. Therefore, a relative positional relationship between the workbench and the robot's stop position is changed, and the robot cannot perform the work.In other words, a first problem of the related field is that when the robot stops based on a specific object to perform a task, it is not possible to appropriately change the stopping position accordingly when a position of the object is changed in the same location.

[0012] Furthermore, for example, when a robot system including a process in which a mobile robot carrying a load stops in front of a robot arm picking up the load is relocated from one site having a development environment to another site having a work environment, it is not possible to apply a stop position and path information set in the development environment unchanged to the work environment because the layout of the equipment including the robot arm is different in the environment before and after the relocation. Although it is necessary to appropriately change the stop position and path according to the layout of the equipment in the work environment, no method for effectively implementing such a change is disclosed.In other words, a second problem of the related field is that when the robot stops based on a specific object to perform a task, the stop position and the path information set at the location before the change cannot be appropriately changed according to the location after the change and cannot be reused when the location where the mobile robot system is used is changed.

[0013] Due to the two problems described above, when the position of an object (hereinafter referred to as a stopping reference object) serving as a reference for a stopping position is changed, or when the entire system including the stopping reference object is relocated to another location, it is necessary to reconfigure the stopping position according to the stopping reference object in the same way as the setting work performed in advance before the change. When setting the stopping position of a robot for the stopping reference object, a user will perform the following task. First, an actual robot is manually operated and moved to a vicinity of the stopping reference object. Then, a positional relationship between the robot and the stopping reference object is measured using a tape measure or other measuring units.If the robot does not stop at a suitable position, the robot is manually operated again, and fine-tuning of the robot's position is performed. Then, after repeating the measurement and fine-tuning of the robot's stop position until the robot is at a suitable position, a current position on a map is calculated by the robot's self-position estimation function, and these coordinates are set as a new stop position. The above work requires time to repeat the measurement and fine-tuning of the stop position. In addition, an adjusting worker must work while crouching close to the robot when measuring the position with a tape measure or ruler, which imposes a physical burden on the adjusting worker.

[0014] Given these challenges, the mobile robot system's usage scenario has expanded from factories with fixed site layouts to medical sites and distribution warehouses, where layout changes are frequent. There is a growing need to quickly and easily commission a robot system at a site of use. In the future, reducing the time required to set a stop position and reducing labor will become increasingly important. Summary of the invention

[0015] The invention has been made in view of such problems, and an object of the invention is to provide a robot stop position adjustment device that can be performed in a short time without the need for actual machine operation or measurement work to set a stop position after an environmental change by utilizing a result of the stop position adjustment performed before the environmental change.

[0016] This object is achieved by the robot stop position adjustment device according to claim 1 and by a mobile robot system according to claim 9. Particular embodiments are described in the dependent claims.

[0017] With the robot stop position setting device of the invention, since the work of setting the robot stop position based on the stop reference object as a reference is performed only once, even if the position of the stop reference object is changed or the mobile robot system is relocated to another location and the environment map is updated, there is no need to operate an actual machine to reset the stop position. As a result, it is possible to reduce the time required to set the robot stop position. Brief description of the drawings Fig. 1 is a view illustrating a configuration example of a mobile robot system of a first example; Fig. 2 is a view illustrating a configuration example of a robot of the first example; Fig. 3 is a view illustrating information exchanged between devices of the mobile robot system of the first example; Fig. 4 is a flowchart for setting a robot stop position before an environment change in the first example; Fig. 5 is a flowchart for setting the robot stop position after the environment change in the first example; Fig. 6 is a view illustrating an example of a specific setting method for the robot stop position of the first example; Fig. 7 is an example of an operation screen of a stop position setting device of the first example; Fig. 8 is a flowchart for setting a robot stop position after an environment change in a second example; Fig. 9 is an example of an operation screen of a stop position setting device of the second example; Fig. 10 is a flowchart for setting a robot stop position after an environment change in a third example; Fig. 11 is an example of an operation screen of a stop position setting device of the third example; Fig. 12 is a flowchart for setting a robot stop position after an environment change in a fourth example; Fig. 13 is an example of an operation screen for a stop position setting device of the fourth example; Fig. 14 is a configuration example of the stop position setting device of the fourth example; Fig. 15 is a configuration example of a reference position output unit of the fourth example; and Fig. 16 is an example of an operation screen for a stop position setting device of a fifth example. Description of the embodiments

[0018] Examples of the invention will be described below with reference to the drawings. The examples are merely illustrations for explaining the invention and do not limit the invention, and are appropriately omitted and simplified for clarity of explanation. The invention can also be implemented in various other examples and in examples in which some or all of the examples are combined. Unless otherwise specified, each component may be singular or plural. Furthermore, in describing the examples described later, the description will focus on differences from the previously described examples, and the description of overlapping portions will be omitted where appropriate.

[0019] First, a mobile robot system 1 of a first example will be described with reference to Fig. 1 to 7 described.

[0020] Fig. 1 illustrates a configuration example of a mobile robot system according to the first example. As shown here, the mobile robot system 1 of this example includes a setting device 2, a controller 3, a control device 4, and a robot 5. The mobile robot system 1 controls a movement of the robot 5 so that it stops at a predetermined robot stop position P based on a stop reference object 0b. Although not illustrated, the setting device 2, the controller 3, the control device 4, and the robot 5 are wirelessly connected or the like so that they can communicate with each other.

[0021] The stopping reference object Ob is, for example, a workbench on which an object to be processed by the robot 5 is placed. The stopping reference object Ob may be a device to be operated by the robot 5. The stopping reference object Ob may be a conveyor on which a load conveyed by the robot 5 is placed. The stopping reference object Ob may be a movable object such as a carriage, a device with rollers, or a movable conveyor, but it must be placed at a predetermined position in the environment when generating an environment map, which will be described below.

[0022] A user confirms a state of the mobile robot system 1 using the setting device 2. Here, the state includes, for example, a remaining battery level of the robot 5, an error message generated by an element constituting the mobile robot system 1, an operation mode of the mobile robot system, current settings, and the like.

[0023] Furthermore, the user performs various settings via the setting device 2 so that the mobile robot system 1 operates in accordance with the user's intention. Here, the settings include, for example, a stopping position of the robot 5, a travel path, map information to be used, and the contents of a task that the robot 5 performs at each stopping position.

[0024] The setting device 2 is, for example, a personal computer and has a user interface 21 and an arithmetic device 22. Although Fig. 1 illustrates the user interface 21 composed of a display, a keyboard, a mouse, and the like. However, a user interface 21 with a touch panel type display may also be used, allowing the user to adjust the mobile robot system with intuitive touch operations. The arithmetic device 22 includes an arithmetic unit such as a central processing unit (CPU), a random access memory (RAM) unit such as a semiconductor memory, a storage unit such as a solid state drive (SSD) or a hard disk drive (HDD), a communication unit, and the like.The arithmetic device 22 may use a processing device such as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD) such as a field-programmable gate array (FPGA) instead of the CPU, or may be implemented as a component of the control device 4. Furthermore, the setting device 2 may be configured by a mobile terminal such as a smartphone or a tablet. In this case, the effect of allowing the user to set the mobile robot system at any position is achieved.

[0025] The controller 3 is, for example, a device that includes either a joystick or a button, or both. As an example of an operation method, for example, the robot 5 travels in a direction in which the joystick is tilted, and a travel speed of the robot 5 is adjusted according to a degree of tilt of the joystick. The setting device 2 may include some or all of the functions of the controller 3. For example, there may be a configuration in which the robot 5 travels in a direction in which a user drags a touch panel, which is the user interface 21 of the setting device 2. Conversely, the controller 3 may include some or all of the functions of the setting device 2, and the buttons of the controller 3 can be used to switch a setting item and input a setting value.

[0026] The control device 4 is a device that controls the robot 5 while communicating with the setting device 2 and the control device 3. Specifically, the control device 4 is a computer that includes similar functional units (calculation unit, RAM unit, storage unit, and communication unit) as the arithmetic device 22 described above. Although Fig. 1 illustrates the control device 4 mounted on the robot 5, the control device 4 may be installed outside the robot 5.

[0027] Robot 5 points, as in Fig. 2, a sensor 51 for measuring a surrounding running environment, a movement mechanism 52 for moving and a manipulator 53.

[0028] The sensor 51 is, for example, a distance sensor that can measure distances between the sensor 51 and surrounding objects in multiple directions. The surrounding objects include the stopping reference object Ob. This sensor 51 is an optical sensor, a LiDAR, or the like. For example, if an optical sensor is used as the sensor 51, it can detect the environment around the robot 5 by irradiating an obstacle with a laser beam, measuring the distance to the obstacle, and measuring the intensity of a reflected light beam.

[0029] The movement mechanism 52 is formed, for example, from one or more drive wheels. The movement mechanism 52 may be formed from two drive wheels, and the left and right wheels may be driven differently. The movement mechanism 52 may be configured using non-directional wheels, such as Mecanum wheels and omnidirectional wheels.

[0030] The manipulator 53 can perform tasks involving grasping and handling (operations such as gripping, picking and placing, pushing and pulling) at various positions. The robot 5 with the manipulator 53 makes the entire system less costly than if it had fixed manipulators at each work position.

[0031] Fig. 3 illustrates an example of information exchange between devices constituting the mobile robot system 1. First, the setting device 2 outputs a setting I1 input by a user to the control device 4. Then, the control device 3 outputs an operation amount I4 of the control device 3 by the user to the control device 4. Further, the robot 5 outputs a measurement result I5 of the sensor 51 and a state I9 of the robot 5 to the control device 4. The control device 4 generates a setting I8 using some or all of the setting I1, the operation amount I4, the measurement result I5, and the state I9 and outputs it to the robot 5.

[0032] When the robot 5 travels, the control device 4 controls the movement mechanism 52 by outputting a control amount I6 determined using some or all of the setting I1, the operation amount I4, the measurement result I5, and the state I9. When the robot 5 performs manipulation, the control device 4 also outputs a control amount I7 determined using some or all of the setting I1, the operation amount I4, the measurement result I5, and the state I9 to the manipulator 53 for control.

[0033] The control device 4 uses a portion of the measurement result I5 or the entire measurement result I5 to generate a measurement result I3, and uses a portion of the state I9 or the entire state I9 to generate a state I2. The setting device 3 displays a portion or all of the state I2 and the measurement result I3 to the user. The measurement result I3 contains a current position of the robot 5 on a map M.

[0034] The setting I1 that the setting device 2 outputs to the control device 4 includes data D1 relating to an operation mode of the mobile robot system 1, data D2 indicating an operation method of the moving mechanism 52 of the robot 5, and data D3 indicating the operation method of the manipulator 53 of the robot 5.

[0035] The operation mode data D1 has, for example, a table-type data structure, and character string data indicating a current control method of the robot 5 is stored in the "control mode" element. When the robot 5 is manually operated, a mode designation "MANUAL" is stored in the "control mode" element, and the data is transmitted to the control device 4. Subsequently, the control device 4 performs a mode transition such that the operation amount I4 is received from the controller 3 and determines the control amount I6 for the robot 5 according to the operation amount I4.

[0036] When the robot 5 operates under autopilot, a mode designation "AUTO" is stored in the "drive mode" item, and the data is transmitted to the control device 4. Subsequently, the control device 4 performs a mode transition such that the operation amount I4 is not received from the controller 3, and automatically drives the robot 5 based on the preset data D2 indicating the operation method of the moving mechanism 52. Furthermore, when the robot 5 reaches the destination set in the data D2, the control amount I7 is determined based on the preset data D3 indicating the operation method of the manipulator 53 of the robot 5, and the manipulator 53 is automatically controlled.

[0037] The data D2 indicating the operation method of the moving mechanism 52 includes, for example, the map M of the traveling environment of the robot 5, the robot stop position P defined on the map M, and a path L representing a trajectory curve that the robot 5 traverses when moving from its current position to the robot stop position P, as a series of positions defined on the map M.

[0038] The map M, for example, is a grid map represented by two-dimensional matrix data, and each element of the matrix data stores a probability of the presence of an obstacle at that position as a decimal number between 0 and 1. The map M is generated by surveying the traveling environment with the sensor 51 or the like mounted on the robot 5, and the grid point where an obstacle is observed is set as an obstacle area with a high probability of the presence of an obstacle. In addition, the grid point where no obstacle is observed is set to have a low probability of the presence of an obstacle. A value of each element of the matrix data constituting the grid map can be determined by the number of points included in the point group of the obstacles observed by the sensor 51.Alternatively, when the sensor 51 outputs a reflection intensity, the value of each element may be determined according to the magnitude of the reflection intensity.

[0039] The current position in the measurement result I3 described above, the robot stop position P, and the positions in the series or the like of the positions constituting the path L are, for example, data represented by (x, y, θ), which is a group of coordinates (x, y) [m] in a coordinate system defined in the map M, and an angle θ [degree] formed between the front face of the robot 5 and an X-axis of the map M.

[0040] Next, an operation procedure when the user sets a robot stop position P1 before the environment change will be described with reference to a flow chart of Fig. 4 described.

[0041] First, in step S11, the user manually operates the robot 5 using the controller 3 to generate a map M1. The map M1 is a map sequentially generated using an output of the sensor 51 as the robot 5 moves through a location before the environmental change. Therefore, it is desirable for the user to generate the map M1 of the entire location by manually operating the robot 5 so that the robot 5 moves through the entire location. However, if a movement range of the robot 5 is restricted, the map M1 may be generated only in a work range of the robot 5. An actual creator of the map M1 may be the controller 4 or the arithmetic device 22.

[0042] Then, in step S12, the user stores the generated map M1 in a storage unit of the setting device 2.

[0043] In step S13, the user sets the stored map M1 as the map for setting the stop position.

[0044] Subsequently, in step S14, the user manually operates the robot 5 using the controller 3 to move the robot 5 to the actual position that the user wants to set as the stop position. The user then operates the setting device 2 and registers the current position on the map M1 as the robot stop position P1.

[0045] Next, using the schedule in Fig. 5 an operation procedure when the user sets a robot stop position P2 after the environment change, using the robot stop position P1 stored in the map M1 in Fig. 4 is registered when the arrangement of the object in the environment in the same location is changed or when the mobile robot system 1 is relocated to another location.

[0046] First, in step S21, the user generates a map M2 by manually operating the robot 5 using the controller 3. The map M2 is a map generated using the output of the sensor 51 sequentially as the robot 5 moves in the location after the environmental change (e.g., after a displacement of the stopping reference object Ob). Therefore, the user manually operates the robot 5 at least so that the robot 5 moves around the location where the environmental change occurred.

[0047] Then, in step S22, the user stores the generated map M2 in the storage unit of the setting device 2.

[0048] In step S23, the user operates the setting device 2 such that the card M1 and the card M2 are read.

[0049] In step S24, the user operates the setting device 2 such that the robot stop position P1 set in the map M1 is read.

[0050] In step S25, the user operates the setting device 2 such that an area R1 containing the stopping reference object Ob is selected from the map M1. The details of this process are described in Fig. 7 described.

[0051] In step S26, the setting device 2 automatically calculates the robot stop position P2 in the map M2. The details of a calculation method for the robot stop position P2 are described below.

[0052] Finally, in step S27, the user operates the setting device 2 such that the robot stop position P2 is stored in the map M2 as a new stop position to replace the robot stop position P1 in the setting device 2.

[0053] Since the robot 5 is made of Fig. 5 does not need to be operated after step S22 for creating a map, a new robot stop position can be easily set in a short time.

[0054] Fig. 6 illustrates an example of a specific setting method for the robot stop position P2 by the setting device 2. Fig. 6 corresponds to the processing from step S23 to step S27 in Fig. 5.

[0055] As illustrated here, the setting device 2 of this example includes, as a configuration for setting the robot stop position P, a map area selection unit 21a, a map storage unit 22a, a stop position storage unit 22b, a reference position output unit 22c, and a stop position calculation unit 22d. An actual instance of the map area selection unit 21a is the user interface 21, and an actual instance of the map storage unit 22a, the stop position storage unit 22b, the reference position output unit 22c, and the stop position calculation unit 22d is the arithmetic device 22. The functions of the units are described below in this order.

[0056] The map storage unit 22a is a functional unit that stores the map information (the maps M1 and M2) generated in steps S11 and S21 and provides the map information to the map area selection unit 21a and the reference position output unit 22c.

[0057] The stop position storage unit 22b is a functional unit that stores the robot stop position P1 on the map M1 registered in step S15 and provides information about the robot stop position P1 to the map area selection unit 21a and the stop position calculation unit 22d.

[0058] The map area selection unit 21a is a functional unit that specifies the area R1 containing the stopping reference object Ob in the detection area selected by the user from the map M1 and outputs it to the reference position output unit 22c and the stopping position calculation unit 22d. Furthermore, the area referred to here is an area defined by, for example, data obtained by cutting out a portion of the map data and data including X and Y coordinates of a starting point of the area of ​​the cut-out map data, a width W of the area, a height H of the area, and a rotation angle θ of the area.

[0059] The map area selection unit 21a displays the map M1 and the stop position P1 on the screen, and then designates an area containing, for example, the stop reference item Ob by dragging the mouse. The method for selecting the area containing the stop reference item Ob may be an operation of filling the screen or an operation of clicking a portion of the area containing the stop reference item Ob. When the area R1 is selected by clicking, some kind of image processing such as segmentation is performed on the map information, and items having connection relationships with the clicked position are automatically extracted.

[0060] The reference position output unit 22c is a functional unit that detects an area R2 having the same shape as the area R1 from the map M2 and outputs the position of the area R2 as a reference position to the stop position calculation unit 22d.

[0061] The stop position calculation unit 22d is a functional unit that calculates the robot stop position P2 on the map M2 using a positional relationship between the robot stop position P1 and the area R1 on the map M1 and the position of the area R2 on the map M2 such that the positional relationship of the robot stop position P2 with respect to the area R2 on the map M2 is equal to that of the robot stop position P1 with respect to the area R1 of the map M1.

[0062] Fig. 7 is an example of an operation screen displayed on a display of the user interface 21 when the user selects the area R1 containing the stopping reference object Ob from the map M1 in step S25. In this way, by simultaneously displaying the map M1 and the map M2 side by side on the display, the user can easily grasp a difference before and after the environmental change, making it possible to easily specify the stopping reference object Ob in each map.

[0063] With the robot stop position setting device of this example described above, since the work of setting the robot stop position based on the stop reference object as a reference is performed only once, even if the position of the stop reference object is changed or the mobile robot system is relocated to another location and the environment map is updated, there is no need to operate the actual machine to set the stop position again. As a result, it is possible to reduce the time required to set the robot stop position.

[0064] Fig. 8 is a flowchart for setting the robot stop position P2 after an environmental change in a second example. In this example, step S25a is added between steps S25 and S26 of the first example. In this step S25a, the user designates the detection area for the stop reference object Ob in the map M2, as in step S25.

[0065] Fig. Figure 9 is an example of an operation screen in step S25a of the second example. The detection area is determined, for example, by dragging the mouse or filling the area. Subsequently, steps S26 and S27 are performed in the same manner as in the first example.

[0066] The second example includes a detection area selection unit 21b in addition to the configuration of the first example for processing step S25a. The actual instance of the detection area selection unit 21b is the user interface 21. The reference position output unit 22c of the second example receives the detection area from the detection area selection unit as an input and detects the area R2 containing the stopping reference object Ob from the detection area. By thus allowing the user to also restrict the detection area for the card M2, erroneous detection of the stopping reference object Ob in the card M2 can be reduced.

[0067] Fig. 10 is a flowchart for setting the robot stop position P2 after an environmental change in a third example. In this example, step S26 of the first example is replaced by steps S26a and S26b. In step S26a, multiple candidates for the robot stop position P2 of the map M2 are automatically calculated, so that the user selects a suitable robot stop position P2 from the generated candidates in step S26b.

[0068] Fig. 11 is an example of an operation screen in step S26b of the third example. In this example, four candidates for the robot stop position P2 are displayed on the map M2, so that the user can specify the most suitable position from among the candidates as the robot stop position P2. As a result, even if the stop reference object Ob has a point-symmetric shape and the stop position is not clearly determined, the user can set a desired stop position.

[0069] Therefore, in a stop position setting device of the third example, the reference position output unit 22c is realized by outputting a plurality of reference positions for the detected area R2, the stop position calculation unit 22d is realized by outputting a plurality of candidates for the robot stop position P2 in the map M2 using the plurality of reference positions, a stop position candidate selection unit 21c is realized by receiving a result selected by the user from the plurality of stop position candidates, and the stop position calculation unit 22d is realized by outputting the user's selection result received by the stop position candidate selection unit 21c as the robot stop position P2 on the map M2.

[0070] Fig. 12 is a flowchart for setting the robot stop position P2 after an environmental change in a fourth example. In this example, step S25 of the first example is omitted. That is, the robot stop position P2 is automatically derived in step S26 without any user intervention, thereby reducing the number of user operations and enabling the stop position to be set in a shorter period of time.

[0071] Fig. Fig. 13 is an example of an operation screen in step S26 of the fourth example. As described above, in this example, the process corresponding to step S25 of the first example is omitted, but an operation screen as shown in Fig. 13, for the purpose of providing information to the user.

[0072] In Fig. 13, portions with the same shape difference between maps M1 and M2 are detected as the area R1 and the area R2 and expressed in a highlighting manner on the map. As a method of highlighting expression, for example, image processing or magnification processing is performed on the area. The method of highlighting expression may be, for example, marking with a different color or drawing a figure such as a bounding box of the area or an arrow indicating the area.

[0073] Fig. 14 illustrates an example of a method of deriving the stop position in the setting device 2 of the fourth example. A stop position setting device in this fourth example is realized by including a map storage unit 22a for storing map information of the map M1 and the map M2, a stop position storage unit 22b for storing the robot stop position P1 on the map M1, a reference position output unit 22c that receives inputs from the map M1 and the map M2, detects the area R1 in the map M1 and the area R2 in the map M2, and outputs the positions of the area R1 and the area R2 as first and second reference positions, respectively, and a stop position calculation unit 22d that calculates the robot stop position P2 on the map M2 using the robot stop position P1, the first reference position, and the second reference position.

[0074] Fig. 15 is a diagram illustrating a configuration example of the reference position output unit 22c. As illustrated here, in the reference position output unit 22c, the map M1 and the map M2 are input to a difference calculator 22c1, the difference between the matrices representing the maps is calculated, and a difference map M3 is derived. The difference map M3 includes positive pixels and negative pixels, so that a matching shape detector 22c2 can be realized that derives figures having common shapes into a group of figures with positive pixels and a group of figures with negative pixels using appropriate segmentation and matching algorithms. Thus, the area R1 and the area R2 can be derived from the maps M1 and M2.

[0075] Therefore, the automatically derived area R1 and the automatically derived area R2 can be used to continue processing similar to that of the first example.

[0076] Fig. 16 is an example of an operation screen for a stop position setting device of a fifth example. In addition to each of the first example to the fourth example, the fifth example has a function of confirming whether interference with the environment occurs when the robot 5 stops at the stop position P2.

[0077] To implement an interference confirmation function, the fifth example includes an interference confirmation unit 22e that receives as inputs the stop position P2 of the robot 5 in the map M2 output by the stop position calculation unit 22d and the map M2, and determines whether interference will occur between the robot 5 and the surrounding environment. When interference is confirmed by the interference confirmation unit 22e, the color of the interference location is changed on the operation screen of a stop position setting device, or a mark indicating the interference is displayed (see Fig. 16).

[0078] If the stop position is set after the robot 5 has actually moved to the stop position, as in the related field, safety is ensured by not causing interference with the environment even if the robot 5 is stopped at that position. However, in the first to fourth examples, safety may not be ensured. The fifth example solves this problem.

[0079] A sixth example is a further improvement of the stop position setting device of the fifth example and has a function of suggesting a position of the stop reference object Ob at which no interference occurs when interference between the robot 5 and the environment is confirmed.

[0080] To realize the above-described function, in this example, an alternative position suggestion unit 22f is provided, which shows an alternative displacement position of the stopping reference object Ob in the map M2 where no interference occurs when the interference confirmation unit 22 detects interference between the robot 5 and the surroundings. This alternative position suggestion unit 22f can be realized by generating translation and rotation transformation matrices, repeatedly attempting to simultaneously transform and move the area R2 and the stopping position P2 in the map M2, and stopping the search and outputting the position when the position of the stopping reference object Ob where no interference occurs is found.When the transformation matrix is ​​generated in this experiment, a position as close as possible to the original stop position P2 can be suggested by searching while gradually changing a translational displacement and a rotation amount from small values ​​to larger values.

[0081] The sixth example has an effect of enabling efficient commissioning of the system by showing the user specific alternatives when the mobile robot system 1 cannot perform the task in its current state.

[0082] In the first to sixth examples, the stopping reference object Ob is detected on map M2 after the environmental change, and the stopping position P2 is derived based on the detection result. However, in general, the grid resolution of maps M1 and M2 is not necessarily high, so the detection result of area R2 on map M2 contains a certain amount of error. A seventh example provides a solution to this problem.

[0083] A stopping position setting device of the seventh example is a further improvement of the stopping position setting device of any of the first example to the sixth example, and includes a stopping reference object detection unit 22g that detects a relative position of the stopping reference object Ob to the robot 5 in real space, a stopping reference object detection result storage unit 22h that stores a detection result of the stopping reference object detection unit 22g, and a stopping position correction unit 22i that corrects the stopping position of the robot 5 on the map M2.The stopping position correction unit 22i corrects the stopping position P2 of the robot 5 on the map M2 based on a first detection result in which the stopping reference object detection unit 22g detects a relative position of the stopping reference object Ob to the stopping position P1 of the robot 5 in the map M1 in the real space corresponding to the map M1, and a second detection result in which the stopping reference object detection unit 22g detects a relative position of the stopping reference object Ob to the stopping position P2 of the robot 5 in the map M2 in the real space corresponding to the map M2.

[0084] The correction amount of the stop position P2 can be calculated as follows. Assume that the first detection result and the second detection result are represented by three-dimensional 4 × 4 transformation matrices T1 and T2, and Tc is a coordinate transformation from a base joint of the robot to the camera. When the coordinate transformation from the stop position P2 before correction to the stop position P2' after correction is T, the coordinate transformation from the stop position P2 to the camera is expressed as Tc × T2 or T × Tc × T1, and the following equation applies: T × Tc × T1 = Tc × T2. Therefore, the three-dimensional transformation matrix T for correcting the stop position P2 is calculated by the following equation: T = Tc × T2 × T1 -1 × Tc -1 . T1 -1 and Tc -1 give the inverse matrices of T1 and Tc in that order.

[0085] An eighth example is a further improvement of the stop position setting device of each of the first example to the seventh example, and includes a path calculation unit 22j that receives the map M2, the stop position P2, and a movement start position of the robot 5 on the map M2 as inputs, and derives a path for the robot 5 to move from the movement start position of the robot 5 on the map M2 to the stop position.

[0086] The path calculation unit 22j uses a search method such as the Dijkstra method to derive a path L that does not interfere with obstacles on the map M2.

[0087] According to the eighth example, the user can derive the path of the robot 5 without manually editing the path, and can easily adjust the operation of the robot 5 in a short time.

[0088] A ninth example is a further improvement of the mobile robot system 1 of any of the first to eighth examples. By calculating a difference between a measurement result I5 of the environment by the sensor 51 of the robot 5 and the map M1, and determining that there has been a major environmental change when the calculated difference is greater than or equal to a preset threshold, the user is prompted to generate the map M2 and output the stop position P2 of the robot 5 on the map M2.

[0089] As a method for calculating the difference, for example, it is possible to generate a difference map M4 in an area measurable by the sensor 51, calculate the difference between each grid point observed in the difference matrix M4 and the grid point in the map M1, and sum them. Alternatively, if the sensor 51 is a sensor that emits multiple laser beams, the difference may be the number of obstacles that do not have corresponding obstacles in the map M1 among the obstacles detected by the laser beams.

[0090] In the mobile robot system 1 of the ninth example, there is a large deviation between the map M1 set by the user and the real space, which has the effect of preventing the robot from malfunctioning during driving.

[0091] In the first example to the ninth example, the reference position output unit 22c detects from the map M2 the shape of the same area as the area R1 of the map M1. However, since the maps M1 and M2 are maps obtained by observing the surroundings with the sensor 51 while the robot 5 is traveling, an occlusion problem arises when a portion of the stopping reference object Ob is hidden behind an occluding object, such that the stopping reference object Ob in the map M1 and the stopping reference object Ob in the map M2 appear to have partially different shapes. In this case, when a shape partially different from the area R1 exists in the map M2, it is necessary to determine whether this is a stopping reference object Ob that appears different due to occlusion, or an object with a shape different from the stopping reference object Ob.A tenth example provides a solution to such a problem.

[0092] In the tenth example, each grid point of the maps M1 and M2 stored in the map storage unit 22a is provided not only with the probability of the presence of an obstacle, but also with information about whether the presence or absence of an obstacle has been confirmed. For example, grid points for which obstacles have been confirmed are labeled "KNOWN," and grid points that have not yet been confirmed are labeled "UNKNOWN." This label can be assigned based on the measurement result of the sensor 51 when creating a map.In particular, there is a method such that all grid points are set to "UNKNOWN" at the beginning of map generation, and subsequently, when an obstacle is detected at a certain position, the grid point where the obstacle exists and the grid point located between the robot and the obstacle are updated to "KNOWN".

[0093] In the tenth example, the reference position output unit 22c uses pattern matching as a method for detecting the area R1 from the map M2, but the method for calculating the degree of correspondence between the area R1 and the map M2 is improved. First, images R1' and M2' are generated with the same resolution as the number of grid points of the area R1 and the map M2. Next, the pixel values ​​of the images R1' and M2' corresponding to the grid points with "UNKNOWN" are set to 0 (zero). Then, with respect to the grid points of the area R1 and the map M2, grid points with a probability of obstacle presence greater than or equal to a threshold are considered to have obstacles, and the pixel values ​​of the corresponding images R1' and M2' are set to positive values.Furthermore, regarding the grid points of the area R1 and the map M2, grid points with a probability of obstacle presence below the threshold are considered to have no obstacle, and the pixel values ​​of the corresponding images R1' and M2' are set to negative values. M2'(i, j) is set as the pixel value at coordinates (i, j) of the image M2', and a function R1'(k, l) is set as the pixel value at coordinates (k, l) on the image R1'. Then, the matching degree P(i, j) at position (i, j) on M2' is calculated, for example, by the following equation. P(i,j)=∑k,lW(i,j,k,l)R1'(k,l)M2'(i+k,j+l)

[0094] W(i, j, k, l) is a function that calculates a weight based on the pixel values ​​of R1' and M2', and a value of W(i, j, k, l) is greater than or equal to zero.

[0095] In this way, by setting different pixel values ​​for an area that is not observed due to occlusion and an area that is confirmed to be free of obstacles based on an observation result, and calculating the matching degree, the matching degree does not decrease in the area where the shape appears different due to occlusion, but decreases in the area where an object with a shape different from that of the stopping reference object Ob is observed. As a result, the insensitivity of pattern matching to occlusion is improved.

Claims

[1] A robot stop position setting device (2) that sets a stop position (P) of a robot (5), comprising: a card storage unit (22a) storing a first card (M1) and a second card (M2); a stop position storage unit (22b) that stores a robot stop position (P1) on the first card (M1), the robot stop position (P) being generated by manual operation of the robot (5) by a user; a map area selection unit (21a) that outputs a first area (R1) containing a stopping reference item selected by the user from the first map (M1), a reference position output unit (22c) that detects a second area (R2) having the same shape as the first area (R1) from the second map (M2) and outputs a position of the second area (R2) as the reference position; and a stop position calculation unit (22d) that calculates the robot stop position (P2) on the second map (M2) using the robot stop position (P1) on the first map (M1), a position of the first area (R1) and the reference position on the second map (M2). [2] The robot stop position setting device (2) according to claim 1, wherein the reference position output unit (22c) detects the second area (R2) from a detection area designated by the user on the second map (M2). [3] A robot stop position setting device (2) according to claim 1 or 2, further comprising: a stopping position candidate selection unit (21c), wherein the reference position output unit (22c) detects a plurality of the second areas and outputs a plurality of the reference positions, the stop position calculation unit (22d) uses the plurality of reference positions to output a plurality of candidates for the robot stop position (P2) on the second map (M2), the stopping position candidate selection unit (21c) receives a result selected by the user from the plurality of candidates, and the stop position calculation unit (22d) outputs the result received by the stop position candidate selection unit (21c) as the robot stop position (P2) on the second map (M2). [4] Robot stop position setting device (2) according to claim 1, wherein the card storage unit (22a) stores a first card (M1) and a second card (M2), the stop position storage unit (22b) stores a robot stop position (P1) on the first card (M1), the reference position output unit (22c) outputs a position of a stopping reference object detected from the first map (M1) as a first reference position and outputs a position of a stopping reference object detected from the second map (M2) as a second reference position, and the stop position calculation unit (22d) calculates the robot stop position (P2) on the second map (M2) using the robot stop position (P1) on the first map (M1), the first reference position and the second reference position on the second map (M2). [5] Robot stop position setting device (2) according to one of claims 1 to 4, further comprising: an influence confirmation unit (22e), wherein the interference confirmation unit (22e) determines whether interference occurs between a robot (5) and the environment based on the robot stop position (P2) on the second map (M2) output by the stop position calculation unit (22d) and the second map (M2). [6] Robot stop position setting device (2) according to claim 5, further comprising: an alternative position suggestion unit (22f), wherein when the interference confirmation unit (22e) detects an interference between the robot (5) and the environment, the alternative position suggestion unit (22f) shows an alternative displacement position of the stopping reference object on the second map (M2). [7] Robot stop position setting device (2) according to one of claims 1 to 6, further comprising: a stop reference object recognition unit (22g), a stop reference object recognition result storage unit (22h) and a stop position correction unit (22i), wherein the stopping reference object detection unit (22g) detects a relative position of the stopping reference object to the robot (5) in real space, the stopping reference object recognition result storage unit (22h) stores a recognition result of the stopping reference object recognition unit (22g), and the stop position correction unit (22i) corrects the robot stop position (P2) on the second map (M2) based on a result of detection by the stop reference object detection unit (22g) of a relative position of the stop reference object to the robot stop position (P1) on the first map (M1) in a first real space corresponding to the first map (M1), a result of detection by the stop reference object detection unit (22g) of a relative position of the stop reference object to the robot stop position (P2) on the second map (M2) in a second real space corresponding to the second map (M2), and a stop position (P) of the robot (5) on the second map (M2). [8] Robot stop position setting device (2) according to one of claims 1 to 7, further comprising: a path calculation unit (22j), wherein the path calculation unit (22j) receives the second map (M2), the robot stop position (P2) on the second map (M2) and a movement start position of the robot (5) on the second map (M2) as inputs and derives a path along which the robot (5) moves from the movement start position of the robot (5) on the second map (M2) to a stop position. [9] A mobile robot system (1) comprising the robot stop position setting device (2) according to any one of claims 1 to 8 and a robot (5) having a sensor that measures the environment in real space, wherein the robot stop position setting device (2) calculates a difference between a measurement result of the environment by the sensor and the first map (M1) and requests a user to generate the second map (M2) and output the robot stop position on the second map (M2) when the difference is greater than or equal to a preset threshold. [10] Robot stop position setting device (2) according to claim 1, wherein the first map (M1) and the second map (M2) are grid maps represented by two-dimensional matrix data, and a probability for the presence of an obstacle in each grid point of the first map (M1) and the second map (M2) is set. [11] Robot stop position setting device (2) according to claim 1, wherein the first map (M1) and the second map (M2) are grid maps represented by two-dimensional matrix data, and each grid point of the first map (M1) and the second map (M2) is provided with information about whether the presence or absence of an obstacle is confirmed.

Images