Information processing apparatus, information processing system, information processing method, and computer program

Abstract

To provide an information processing apparatus capable of causing robots to consistently execute labor in a specified environment.SOLUTION: Methods of an information processing apparatus comprise: task acquisition means for acquiring a request for provision of labor in a specified environment; sensor information acquisition means for acquiring sensor information obtained by measuring inside the environment using one or more sensors; robot candidate acquisition means for acquiring information on a candidate group of robots that provide the labor; and robot selection means for selecting a first robot group consisting of one or more robots capable of providing the labor from the candidate group based on the request and the sensor information.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing system, an information processing method, a computer program, and the like.

Background Art

[0002] In recent years, various types of robots for providing labor have been provided and developed. For example, Patent Document 1 discloses a technique for selecting and proposing a robot to be provided from a plurality of robots in response to a request for providing labor received from a user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the method described in Patent Document 1, there are cases where the robot does not operate as expected at the site desired by the user.

[0005] Therefore, one object of the present invention is to provide an information processing apparatus capable of stably executing labor by a robot in a specified environment.

Means for Solving the Problems

[0006] An information processing apparatus according to one aspect of the present invention a task acquisition means for acquiring a request for providing labor in a specified environment, a sensor information acquisition means for acquiring sensor information measured by one or more sensors within the environment, a robot candidate acquisition means for acquiring information on a group of robot candidates for providing the labor, Robot selection means for selecting, based on the foregoing requirements and the sensor information, a first group of robots consisting of one or more robots capable of providing the labor from the candidate group It is characterized by comprising.

Effect of the Invention

[0007] According to the present invention, it is possible to realize an information processing apparatus capable of stably executing labor by a robot in a designated environment.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each figure, the same members or elements are denoted by the same reference numerals, and overlapping descriptions are omitted or simplified.

[0010] <Embodiment 1> Hereinafter, the information processing apparatus, information processing method, and computer program according to Embodiment 1 of the present invention will be described in detail with reference to the drawings.

[0011] In addition, in the present embodiment, an example of providing labor for cleaning the floor surface will be taken for explanation. Further, in the present embodiment, a robot suitable for providing labor is selected based on a labor provision request from the user and sensor data of the labor execution environment acquired by the terminal owned by the user.

[0012] In the present embodiment, the state of the floor surface of the labor execution environment is analyzed based on the sensor data, and the conditions of the robots that can operate on the site are acquired, so that a robot suitable for operation in the environment can be selected.

[0013] FIG. 1 is a diagram showing an example of the hardware configuration of the information processing apparatus according to Embodiment 1. The information processing apparatus 101 has the functions of a general PC device, and is composed of a CPU 111, a ROM 112, a RAM 113, a storage unit 114 such as an HDD or an SSD, a communication unit 115, a display control unit 116, and a system bus 117.

[0014] The display control unit 116 functions as display control means, and performs control for displaying an image and various information on a display unit such as an LCD (not shown). Note that the display unit as the display means may be provided in the information processing apparatus, or may be separate from the information processing apparatus.

[0015] Using the RAM 113 as a work memory, the CPU 111 executes an operating system (OS) and various computer programs stored in the ROM 112, the storage unit 114, etc., and controls each unit via the system bus 117. The computer programs executed by the CPU 111 as a computer include computer programs for executing the processes described later.

[0016] FIG. 2 is a diagram for explaining an example of a usage scene of the information processing system 220 according to Embodiment 1. The information processing system 220 according to Embodiment 1 includes at least a user terminal 202 and an information processing apparatus 101.

[0017] The end user 201 requests the robot providing service 210 publicly available on the network to provide labor for cleaning the floor surface of the labor execution environment 200 through the user terminal 202. That is, the user terminal 202 can send a request for providing labor in a specified environment.

[0018] The user terminal 202 is a portable computer such as a smartphone or a tablet PC equipped with a camera and connected to a network, for example. The end user 201 further uses the camera of the user terminal 202 to acquire sensor data (for example, a captured image) of the labor execution environment 200 and transmits it to the robot providing service 210.

[0019] An example of a UI screen for the user to transmit sensor data and the like will be described later with reference to FIGS. 11 and 12.

[0020] In the robot providing service 210, the information processing apparatus 101 receives a labor provision request and sensor data from the end user 201, and selects a robot suitable for providing labor from a plurality of task execution robots 211 it owns.

[0021] The robot providing service 210 proposes to the end user 201 the selected robot and dispatch conditions such as the consideration for labor. When the end user 201 accepts the proposal, a contract is established between the end user 201 and the robot providing service, and the operator 212 ships the corresponding robot to the labor execution environment 200.

[0022] FIG. 3 is a functional block diagram showing a functional configuration example of the information processing apparatus according to Embodiment 1. Note that the processing of each part shown below is implemented as software by reading a computer program from the ROM 112 or the like onto the RAM 113 and then executing the program by the CPU 111.

[0023] However, part or all of them may be realized by hardware. As the hardware, a dedicated circuit (ASIC), a processor (reconfigurable processor, DSP), or the like can be used.

[0024] Also, each functional block shown in FIG. 3 does not have to be built in the same housing, and may be configured by separate devices connected to each other via signal paths. Note that the above description regarding FIG. 3 also applies to FIG. 8 in the same manner.

[0025] The task acquisition unit 301 acquires a labor provision request transmitted by the end user 201 via the user terminal. The sensor information acquisition unit 302 acquires sensor data obtained by measuring the labor execution environment 200. The robot candidate acquisition unit 303 acquires a list of robot candidates for providing labor. The robot selection unit 304 selects one or more robots for providing labor from the list of robot candidates.

[0026] The candidate list of robots used in this embodiment corresponds to the task execution robot 211 owned by the robot providing service and is stored in the storage unit 114. The list consists of a robot specification table and a usage schedule table. The specification table describes the functions provided by each robot (such as the floor cleaning method), the material conditions of the floor that can be cleaned, the step and inclination conditions of the floor on which it can operate, the provided cost (unit price), and so on.

[0027] FIG. 4 is a diagram showing an example of the content of the candidate list of robots according to Embodiment 1, and shows an example of the robot specification table. In the specification table of FIG. 4, the robot ID, function 1 (such as suction or water blowing in the case of cleaning), floor material conditions, floor step conditions, floor inclination conditions, provided cost, etc. are tabulated. Note that in the usage schedule table (not shown), there are described the dispatch schedule and maintenance schedule of each robot, etc., and the periods during which new labor cannot be provided.

[0028] FIG. 5 is a flowchart showing a processing example of the information processing method using the information processing apparatus according to Embodiment 1. Note that the operations of each step of the flowchart in FIG. 5 are sequentially performed by the CPU 111 as a computer in the information processing apparatus executing a computer program stored in the memory.

[0029] The series of processes shown in FIG. 5 is started when the information processing apparatus 101 receives the labor provision request transmitted by the end user 201 and the image group of the labor execution environment 200.

[0030] In step S501, the CPU 111 receives a task by the task acquisition unit 301. That is, it acquires the labor provision request transmitted in advance by the end user 201. The labor provision request includes the content of the labor to be provided (here, the cleaning of the floor), the delivery address of the robot at the time of labor provision, and information on the provision period.

[0031] Here, step S501 functions as a task acquisition step of acquiring a request for providing labor in a specified environment by the task acquisition unit 301 as task acquisition means.

[0032] In step S502, the CPU 111 acquires, by the sensor information acquisition unit 302, an image group of the labor execution environment 200 that the end user 201 has transmitted in advance. Here, step S502 functions as a sensor information acquisition step of acquiring sensor information measured by one or more sensors within the environment by the sensor information acquisition unit 302 as sensor information acquisition means.

[0033] In step S503, the CPU 111 acquires, by the robot candidate acquisition unit 303, a list of candidates for robots that provide labor from the storage unit 114. Here, step S503 functions as a robot candidate acquisition step of acquiring information on a group of candidates for robots that provide labor from the robot candidate acquisition unit 303 as robot candidate acquisition means.

[0034] In step S504, the CPU 111 selects, by the robot selection unit 304, one or more robots that provide labor from the list of robot candidates acquired in step S503. Here, step S504 functions as a robot selection step of selecting, from the robot selection unit 304 as robot selection means, a first robot group consisting of one or more robots capable of providing labor from the candidate group based on the request and the sensor information.

[0035] Figure 6 is a flowchart showing details of a processing example of step S504. Note that, by the CPU 111 as a computer within the information processing apparatus executing a computer program stored in the memory, the operations of each step of the flowchart in Figure 6 are sequentially performed.

[0036] In step S601, the CPU 111 analyzes the image group acquired in step S502 through the robot selection unit 304, and acquires the state of the labor execution environment 200 related to the operation of the robot.

[0037] Specifically, the robot selection unit 304 uses a pre-trained machine learning model to estimate the material of the floor surface, the magnitude of the floor surface step, and the angle of the floor surface inclination of the labor execution environment 200 from the image. When the labor execution environment 200 consists of a plurality of regions and is estimated to have different materials, steps, and inclinations for each region, the estimation is performed for each region.

[0038] Furthermore, in step S602, the CPU 111 extracts, through the robot selection unit 304, robots that can provide labor according to the requirements acquired in step S501 from the list acquired in step S503.

[0039] Specifically, first, the usage schedule of the robot is referred to, and robots that can provide new labor at the specified date and time and the delivery address are extracted. Furthermore, from the extracted robots, robots whose floor surface material of the labor execution environment 200 estimated in step S601 matches the floor surface material conditions that can be cleaned in the specification table are extracted.

[0040] Furthermore, among them, the steps and inclinations of the labor execution environment 200 are compared with the step and inclination conditions of the floor surface that can operate in the specification table, and robots whose step and inclination magnitudes of the labor execution environment 200 are within the step and inclination conditions of the floor surface that can operate are extracted. When the labor execution environment 200 consists of a plurality of regions and is estimated to have different materials, steps, and inclinations for each region, robots that can provide labor for each region are extracted.

[0041] In step S603, the CPU 111 selects, through the robot selection unit 304, one or more combinations of robots suitable for providing the required labor from the robots extracted in step S602.

[0042] Specifically, for each area of the labor execution environment 200, one or more robots capable of providing labor are selected, and a combination is selected such that the total number of selected robots is minimized. That is, robots that can be selected in multiple areas are preferentially selected. When there are multiple combinations that satisfy the above conditions, any one of the combinations is selected.

[0043] That is, in this embodiment, the robot selection unit 304 uses the labor provision efficiency of the candidate group of robots as a selection criterion.

[0044] As described above, by implementing the information processing apparatus, information processing method, and computer program described in this embodiment, it is possible to select a robot suitable for providing labor in the on-site environment desired by the user. Therefore, it is possible to stably operate the selected robot as expected in the labor execution environment.

[0045] (Modification Example of Embodiment 1) In Embodiment 1, an example was described in which the end user 201 uses the camera of the user terminal 202 to acquire an image of the labor execution environment 200 and uses a pre-trained machine learning model to estimate the material, steps, and inclination of the floor surface of the labor execution environment 200 from the image.

[0046] However, the types of sensor data to be acquired and the method of acquiring the state of the labor execution environment 200 are not limited to this. In this embodiment, any sensor data capable of acquiring the state of the labor execution environment 200 and determining the operability of the robot can be used.

[0047] For example, instead of an image, video or measurement data from a distance sensor may be used to estimate the material, steps, inclination, etc. of the floor surface. Also, without using a machine learning model, the steps and shape of the floor surface may be estimated using a known three-dimensional reconstruction technique from a plurality of images.

[0048] Alternatively, the operable temperature range of each robot may be described in the robot's specification sheet, and the operability of the robot may be determined based on the temperature of the labor implementation environment 200 acquired by the temperature sensor.

[0049] Alternatively, when a robot requires a communication environment for providing labor, the state of the communication environment measured by the user terminal 202 may be transmitted as sensor data and used as a basis for determining the operability of the robot. By these methods, the robot can be selected based on various measurable conditions of the labor implementation environment 200.

[0050] In Embodiment 1, in step S603, a method of selecting a combination such that the total number of selected robots is minimized was described. However, the selection method is not limited to this as long as the selected combination of robots is suitable for providing labor.

[0051] For example, a combination that minimizes the total provision cost (amount) of the robots may be selected. Thereby, a selection of robots suitable for cost reduction on the end-user 201 side can be made. Thus, the robot selection unit 304 may use the workload or cost of the labor as a criterion for selection.

[0052] In Embodiment 1, the labor of cleaning the floor surface and the cleaning robot were taken as an example for explanation, but the type of labor and the robot providing it are not limited to this. This embodiment is applicable to various types of labor that can be provided by robots, such as security, luggage transportation, and part gripping operations by a robot arm. When applied to mobile security robots or transportation robots, the robot can be selected based on conditions such as the step and slope of the floor surface where each robot can operate.

[0053] Alternatively, when applied to a fixed security robot that monitors the surroundings with a camera or a robot that grasps parts recognized by a camera with a robotic arm, the illuminance of the environment estimated from the image may be used as a criterion for determining operability. By these methods, it is possible to select an appropriate robot for various tasks other than floor cleaning.

[0054] In addition, in Embodiment 1, a method of selecting a robot based on the sensor data acquired in Step S502 was described. However, there may be cases where information for selecting a robot is insufficient, such as when there is no image of the floor surface and the state of the floor surface cannot be determined in Step S601.

[0055] In such a case, the information processing apparatus 101 may interrupt the robot selection process in Step S601 and notify the end user 201 or the user terminal 202 that the information is insufficient. That is, when the sensor information is insufficient as a criterion for selection, the robot selection unit 304 may notify the device equipped with the sensor or a predetermined user that additional sensor information is required.

[0056] The notification transmitted by the information processing apparatus 101 may include the type of the missing sensor data and the required information such as the position and orientation within the labor execution environment 200 where the sensor data should be acquired. That is, the above notification may include information regarding the content of the sensor information that needs to be added.

[0057] As a result, by the end user 201 acquiring additional sensor data and transmitting it to the robot provision service 210, the robot selection process can be carried out again based on sufficient information.

[0058] Further, in Embodiment 1, the method of selecting one or more combinations of robots suitable for providing the requested labor in the process of Step S602 was described. However, depending on the state of the labor execution environment 200, there may be cases where an appropriate combination of robots cannot be selected, such as when there are no robots capable of providing labor in a certain area within the environment.

[0059] In such a case, the information processing apparatus 101 notifies the end user 201 or the user terminal 202 that no robots capable of providing labor are available for the corresponding area, and may select a combination of robots based on the remaining areas. As a result, the end user 201 can receive the provision of part of the requested labor.

[0060] Further, the reason (such as labor execution environment conditions or cost) for determining that each robot cannot provide labor in the corresponding area may be included in the content of the notification. In this case, the end user 201 can improve the state of the labor execution environment 200 based on the notification and transmit the sensor data obtained by measuring the labor execution environment 200 again.

[0061] Alternatively, the conditions regarding cost can be changed. As a result, the end user 201 can receive the provision of the requested labor with the improved labor execution environment 200 and cost. In this way, when there is no combination of robots that can be selected as the first robot group in the candidate group, the robot selection unit 304 may notify a predetermined user of the conditions for enabling any combination of robots in the candidate group to be able to provide labor.

[0062] <Embodiment 2> In Embodiment 1, the method of selecting a robot based on the measurement results of sensors installed on the user terminal was described. In Embodiment 2, a different environment measurement device from the user terminal is used to select a robot suitable for providing labor from the measurement results, and generate data that the selected robot will use when providing labor.

[0063] Since the physical configuration of the information processing apparatus 701 according to the present embodiment is equivalent to that of the information processing apparatus 101 described in the first embodiment, the description thereof will be omitted.

[0064] FIG. 7 is a diagram for explaining a usage scenario of the information processing apparatus according to the second embodiment. The end user 201 requests the robot providing service 210 published on the network to provide labor for cleaning the floor surface of the labor execution environment 200 through the user terminal 202.

[0065] The robot providing service 210 sends the environment measurement device 702 to the labor execution environment 200 in order to obtain more detailed information about the labor execution environment 200. When the user activates the environment measurement device 702, the environment measurement device 702 moves autonomously within the labor execution environment 200, acquires sensor data of the labor execution environment 200 using the mounted sensors, and transmits it to the robot providing service.

[0066] In the robot providing service 210, the information processing apparatus 701 receives the labor provision request transmitted from the end user 201 and the sensor data transmitted from the environment measurement device 702, and selects a robot suitable for providing labor from the task execution robots 211 it owns.

[0067] The robot providing service 210 proposes to the end user 201 the selected robot and dispatch conditions such as the consideration for the labor. When the end user 201 accepts the proposal, a contract is established with the robot providing service, and the operator 212 ships a robot suitable for providing labor to the labor execution environment 200.

[0068] The environment measurement device 702 appearing in the present embodiment is a moving body equipped with, for example, a 2D-LiDAR (Light Detection And Ranging) and a stereo camera as sensors. Further, based on the sensor data, map data within the environment can be generated by, for example, a known SLAM (Simultaneous Localization And Mapping) technology, and it can move autonomously within the environment.

[0069] In addition, the environmental measurement device 702 is connected to a network and can automatically transmit the measured sensor data to the robot providing service 210. The generated map data describes information on feature points in the environment observed by the 2D-LiDAR and the stereo camera, respectively.

[0070] Hereinafter, the configuration of the information processing apparatus according to the present embodiment will be described. FIG. 8 is a functional block diagram showing a functional configuration example of the information processing apparatus according to Embodiment 2. Note that the processing of each part in FIG. 8 is implemented as software by reading a computer program from the ROM 112 or the like onto the RAM 113 and then executing the program by the CPU 111.

[0071] Since the basic functions of each of the task acquisition unit 301, the sensor information acquisition unit 302, the robot candidate acquisition unit 303, and the robot selection unit 304 are the same as those in Embodiment 1, the description thereof will be omitted.

[0072] The data generation unit 801 functions as data generation means and generates data used by the task execution robot 211 in the labor execution environment 200 based on the sensor data obtained by measuring the labor execution environment 200. That is, the data generation unit 801 generates work data used by any one of the robots in the first robot group for providing labor in the environment based on the sensor information.

[0073] The list of robot candidates used in this embodiment is composed of a robot specification table and a usage schedule table, similar to Embodiment 1. In addition to the content described in Embodiment 1, the robot specification table used in this embodiment describes the types of sensors used by each robot for self-position measurement during cleaning.

[0074] The types of sensors are, for example, 2D-LiDAR, a stereo camera, or both 2D-LiDAR and a stereo camera, etc. That is, any robot in the first robot group is assumed to be equipped with a sensor corresponding to at least one of the one or more sensors included in the environmental measurement device 702.

[0075] Next, FIG. 9 is a flowchart showing a processing example of the information processing method according to Embodiment 2. Note that the operations of each step in the flowchart of FIG. 9 are sequentially performed by the CPU 111 as a computer in the information processing device executing a computer program stored in the memory.

[0076] A series of processes in FIG. 9 is started when the information processing device 101 receives a request for providing labor transmitted by the end user 201 and sensor data of the labor execution environment 200 measured by the environmental measurement device 702.

[0077] The details of the processes in steps S501 and S503 and the outline of the process in step S504 are the same as those in Embodiment 1, so the description thereof is omitted. The details of the process in step S504 will be described later with reference to FIG. 10.

[0078] In step S901, the CPU 111 acquires, by the sensor information acquisition unit 302, sensor data of the labor execution environment 200 measured by the environmental measurement device 702. Here, the sensor data is measurement data of 2D-LiDAR and a stereo camera, and map data of the labor execution environment 200 generated by the environmental measurement device 702 during measurement.

[0079] In step S902, the CPU 111 generates, by the data generation unit 801, map data to be used by the task execution robot 211 in the labor execution environment 200 and transmits it to the task execution robot 211. Here, the map data to be transmitted is used by the task execution robot 211 for position measurement in the environment.

[0080] In this way, the work data generated by the data generation unit 801 includes map data for any robot in the first robot group to measure its own position and / or orientation within the environment.

[0081] Specifically, from the map data of the labor execution environment 200 obtained in step S901, data corresponding to the sensors for self-position measurement mounted on the selected task execution robot 211 is extracted and saved as new map data. When there are multiple selected robots, for each robot having a sensor corresponding to the obtained map data, generation and transmission of respective map data are performed.

[0082] The map data obtained in step S901 is map data generated using a 2D-LiDAR and a stereo camera, and includes feature data in the environment measured by both sensors.

[0083] For example, when the selected task execution robot 211 is a robot equipped with only a stereo camera, new map data is generated by extracting only the features measured by the stereo camera from the obtained map data.

[0084] Since the original map data is generated using both a 2D-LiDAR and a stereo camera, the new map data has higher accuracy than the map data generated by the stereo camera alone.

[0085] FIG. 10 is a flowchart showing a detailed example of the process of step S504 according to Embodiment 2. Note that, by the CPU 111 as a computer in the information processing apparatus executing a computer program stored in the memory, the operations of each step of the flowchart in FIG. 10 are sequentially performed.

[0086] In step S1001, the CPU 111 analyzes the measurement data of the stereo camera obtained in step S901 by the robot selection unit 304, and acquires the state related to the operation of the robot in the labor execution environment.

[0087] Specifically, the robot selection unit 304 uses a pre-trained machine learning model to estimate the material, steps, and inclination of the floor surface of the labor execution environment 200 from the images of the stereo camera. When the labor execution environment 200 consists of multiple areas and each area is estimated to have different materials, steps, and inclinations, the estimation is performed for each area.

[0088] In step S1002, the CPU 111 extracts, via the robot selection unit 304, robots that can provide labor according to the requirements obtained in step S501 from the list obtained in step S503.

[0089] Specifically, first, refer to the robot usage schedule to extract robots that can provide new labor at the specified date and time and the delivery address. Further, from among them, extract robots whose floor material of the labor execution environment 200 estimated in step S1001 matches the floor material conditions that can be cleaned in the specification table.

[0090] Furthermore, compare the steps and inclination of the labor execution environment 200 with the step and inclination conditions of the floor surface where the robot can operate in the specification table, and extract robots whose step and inclination magnitudes of the labor execution environment 200 are within the step and inclination conditions of the floor surface where the robot can operate. When the labor execution environment 200 consists of multiple areas and each area is estimated to have different materials, steps, and inclinations, extract robots that can provide labor for each area.

[0091] In step S1003, the CPU 111 acquires, via the robot selection unit 304, the map data of the labor execution environment 200. Here, the map data acquired by the sensor information acquisition unit 302 in step S901 is used as it is.

[0092] In step S1004, the CPU 111, through the robot selection unit 304, estimates the self-position measurement accuracy of each robot extracted in step S1002 within the labor execution environment 200 based on the map data obtained in step S1003. When the labor execution environment 200 consists of a plurality of areas, the position measurement accuracy between each robot and each area is estimated.

[0093] Specifically, for the corresponding area on the map data, the quantity of available feature points of the self-position measurement sensor mounted on the target robot is obtained. When the quantity of available feature points exceeds a predetermined threshold, it is determined that the robot has the self-position measurement accuracy enabling the provision of labor in the corresponding area.

[0094] In step S1005, the CPU 111, through the robot selection unit 304, selects one or more combinations of robots suitable for providing the required labor from among the robots determined in step S1004 to have the self-position measurement accuracy enabling the provision of labor.

[0095] Specifically, for each area of the labor execution environment 200, one or more robots capable of providing labor are selected, and a combination is selected such that the total number of selected robots is minimized. That is, robots selectable in multiple areas are preferentially selected. When there are multiple combinations satisfying the above conditions, any one of the combinations is selected.

[0096] As described above, in this embodiment, the robot selection unit 304 uses the accuracy of measuring the self-position and / or posture of the candidate group of robots in the environment as a selection criterion.

[0097] As described above, by implementing the information processing apparatus, information processing method, and computer program according to Embodiment 2, it is possible to select a robot suitable for providing labor while considering the self-position measurement accuracy of the robot in the environment of the site desired by the user.

[0098] Furthermore, the data used by the selected robot in the labor execution environment can be generated from the pre-measured sensor data. Therefore, it becomes possible to stably operate the selected robot as expected in the labor execution environment.

[0099] (Modification of Embodiment 2) In Embodiment 2, in step S901, an example of acquiring the measurement data of the 2D-LiDAR and the stereo camera and the map data as sensor data has been described. However, the types of sensor data to be acquired are not limited to this.

[0100] Similar to Embodiment 1, in this embodiment, any sensor data capable of determining whether the robot can operate can be used. For example, 3D-LiDAR may be used. Or, when the environment measurement device 702 is equipped with a camera for photographing the floor surface as described in Embodiment 1, the photographed image may be acquired.

[0101] Or the measured value of the temperature sensor or the state of the communication environment may be acquired. The information processing device 701 can use these various sensor data to select the robot and generate task execution data.

[0102] Furthermore, in Embodiment 2, a method of measuring the labor execution environment 200 using the environment measurement device 702 which is a self-mobile movable body equipped with sensors has been described. However, the form of the sensor device is not limited to this, and other forms may be used as long as the shape of the environment can be measured.

[0103] For example, the sensor device may be a movable body that a user moves by hand, a device that can be carried by hand, or a sensor device that travels by remote control. Or, when the user terminal 202 or the task execution robot 211 is equipped with a sensor capable of measuring the shape of the environment, it may be used. That is, the measurement information of the environment measurement device 702 and the sensor information of the user terminal 202 may be combined and used.

[0104] In Embodiment 2, an example was described in which, in step S1003, the map data acquired by the sensor information acquisition unit 302 in step S901 was used. However, the method for acquiring the map data used for robot selection is not limited to this.

[0105] For example, in step S901, sensor data or the like may be acquired from an external server, and map data may be generated in the information processing apparatus 701 using a known SLAM technique or the like from the sensor data. Thereby, even when the sensor device does not have a map creation function, robots can be appropriately selected.

[0106] Also, in Embodiment 2, based on the map data acquired in step S1003, the accuracy of self-position measurement of each robot was estimated and used as a criterion for robot selection. However, the criterion for robot selection when using map data is not limited to this.

[0107] For example, from the arrangement of the feature points of the map data, the arrangement of obstacles in the labor execution environment 200 and the spatial extent (for example, the width and height of the passage) of the movement path of the robot are estimated, and based on this, it may be determined whether each robot can move or pass through each area.

[0108] Specifically, characteristics such as the size of each robot and the turning performance during movement are described in the robot specification table, and by comparing these characteristics with the arrangement of obstacles in the environment and the spatial extent of the movement path, it can be determined whether the robot can move in the corresponding area. Thereby, the risk that the selected robot does not operate as expected due to physical obstacles in the labor execution environment can be avoided.

[0109] In addition, in Embodiment 2, in step S1004, a method for determining the position measurement accuracy based on the amount of feature points available for self-position measurement by the sensor was described. However, the method for determining the position measurement accuracy is not limited to this. For example, the spatial bias and distance distribution of the feature points observable from a certain point, the individual characteristics of the feature points, etc. may be statistically processed to estimate the position measurement accuracy.

[0110] Also, in Embodiment 2, in step S1005, a method of selecting a combination in which labor can be provided for each area of the labor execution environment 200 and the number of robots selected as a whole is minimized was described. However, the method of selecting the combination of robots is not limited to this.

[0111] For example, the request for labor provision received in step S501 may include a time constraint such as completing the execution (cleaning) of a single task within a predetermined time (for example, within 30 minutes). In such a case, based on the execution efficiency of the robots' tasks and the size of the labor execution environment 200, a combination of robots that can meet the required time constraint may be selected.

[0112] That is, the request from the user may include information on the time limit for labor provision, and the robot selection unit 304 may select a first group of robots that can execute the labor within the time limit based on the workload.

[0113] Specifically, in the robot specification table, the throughput (the area that can be cleaned per unit time) of each robot is described, and the working time for the combination of robots can be estimated by estimating the size of each area from the map data.

[0114] In such a case, the working time can be shortened by allocating a plurality of robots to each area, and the required time constraint can also be satisfied. Also, instead of estimating the size of the labor execution environment 200 from the map data, the size of the labor execution environment 200 may be directly estimated from the sensor data. By these methods, the selection of robots considering the time required for labor execution can be performed.

[0115] Furthermore, depending on the state of the labor execution environment 200, the throughput of each robot may change. In such a case, a table listing the throughput for each condition may be prepared in the specification table, and the combination of robots may be selected based on the throughput selected based on the sensor data.

[0116] For example, when the accuracy of the self-position measurement of the robot is not sufficient in a certain area, although labor can be provided, work at a lower speed may be required. In such a case, the self-position measurement accuracy can be gradually determined based on a plurality of predetermined thresholds from the amount of available feature points in step S1004, and the throughput corresponding to each stage can be obtained from the specification table.

[0117] Also, in Embodiment 2, the method of generating the map data used by the task execution robot 211 in the labor execution environment 200 was described in step S902. However, the data generated by the data generation unit 801 is not limited to this. The data to be generated may be any data as long as it improves the operation performance of the task execution robot 211 in the labor execution environment 200.

[0118] For example, the optimal exposure setting parameters of the stereo camera may be calculated based on the sensor data and transmitted to the task execution robot 211. Thereby, the operation stability of the task execution robot 211 in the labor execution environment 200 can be improved. That is, the work data generated by the data generation unit 801 may include the parameters of the sensors mounted on the first robot group.

[0119] In Embodiment 2, the method of performing the robot selection process with the environment measurement device 702, which is a mobile body equipped with sensors, and the independent information processing device 701 was described. However, the implementation form of the information processing device is not limited to this. For example, sensors may be mounted on the information processing device 701 to measure the inside of the labor execution environment 200. Or a series of processes may be performed by the information processing device mounted inside the environment measurement device 702.

[0120] In addition, in Embodiment 2, a method of selecting a robot based on the sensor data acquired in step S901 was described. However, there may be cases where information for selecting a robot is insufficient, such as when the acquired map data does not cover the work execution environment 200 and the self-position measurement accuracy of the robot cannot be estimated throughout the entire environment.

[0121] In such a case, the information processing apparatus 101 may interrupt the robot selection process in step S1004 and notify the end user 201, the user terminal 202, or the environment measurement apparatus 702 that information is insufficient.

[0122] The notification transmitted by the information processing apparatus 101 may include information such as the type of sensor data that is lacking and the position and orientation within the work execution environment 200 where the sensor data should be acquired. As a result, by having the environment measurement apparatus 702 acquire additional sensor data and transmit it to the robot provision service 210, the robot selection process can be carried out again based on sufficient information.

[0123] In addition, in Embodiment 2, the accuracy of self-position measurement is estimated from the measurement results of a sensor device capable of measuring the shape of the environment, and based on this, a robot suitable for providing labor is selected, and data used by the selected robot during the provision of labor is generated.

[0124] It is not essential to implement these methods simultaneously, and only one of these methods may be implemented. For example, in Embodiment 2, the process of step S902 may be omitted, and a map data may be separately created after the selected robot arrives at the work execution environment 200.

[0125] In addition, as in Embodiment 1, the state of the floor surface may be estimated and a robot may be selected based on the user terminal, and the brightness of the work execution environment 200 may be estimated from the captured image of the user terminal to calculate the optimal exposure setting of the stereo camera.

[0126] FIG. 11 is a diagram showing an example of a UI screen for a user of the information processing apparatus according to Embodiment 1 or 2 to transmit sensor data, and shows an example of a UI screen for the end user 201 to transmit sensor data of the labor execution environment 200 to the robot providing service.

[0127] In the area 1101 at the upper part of the screen, the content of the request for providing labor registered by the end user 201 is displayed. The end user 201 presses the button 1102 and selects an image group in the terminal through a separately displayed image selection screen. The selected image group is displayed as a thumbnail group 1103. When the end user 201 presses the button 1104, the selected image group is transmitted to the robot providing service.

[0128] FIG. 12 is a diagram showing an example of a UI screen for a user of the information processing apparatus according to Embodiment 1 or 2 to confirm the selected robot, and shows an example of a UI screen for the end user 201 to confirm a proposal from the robot providing service 210. In the area 1201 at the upper part of the screen, the content of the request for providing labor registered by the end user 201 is displayed.

[0129] In the area 1202 in the middle of the screen, information on the robot scheduled to provide labor proposed by the robot providing service is displayed. That is, information on the first robot group selected by the robot selection unit 304 as the robot selection means by the display control unit 116 is displayed on the display unit as the display means. Also, in the area 1203 at the lower left of the screen, the total fee and related information are displayed.

[0130] When the end user 201 presses the button 1204, a contract for dispatching the robot displayed in 1202 for the request displayed in 1201 is established. The button 1205 is a return button for the end user 201 to leave this screen without establishing a contract.

[0131] Although the example of the cleaning robot has been described in the above embodiments, it may be an autonomous mobile body such as an AGV (Automated Guided Vehicle) or an AMR (Autonomous Mobile Robot) instead of a robot. Further, it may be a moving body such as a vehicle or a drone for transporting people, luggage, etc.

[0132] As described above, the present invention has been described in detail based on its preferred embodiments. However, the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible based on the spirit of the present invention, and they are not excluded from the scope of the present invention. Note that the present invention includes the following combinations.

[0133] (Configuration 1) A task acquisition means for acquiring a request for providing labor in a specified environment, a sensor information acquisition means for acquiring sensor information measured by one or more sensors within the environment, a robot candidate acquisition means for acquiring information on a group of robot candidates for providing the labor, and a robot selection means for selecting, based on the request and the sensor information, a first group of robots capable of providing the labor from the group of candidates. An information processing apparatus characterized by comprising:

[0134] (Configuration 2) The information processing apparatus according to Configuration 1, further comprising a display control means for causing a display means to display information on the first group of robots selected by the robot selection means.

[0135] (Configuration 3) The robot selection means uses, as a criterion for the selection, the labor providing efficiency of the robots in the group of candidates. The information processing apparatus according to Configuration 1 or 2.

[0136] (Configuration 4) The robot selection means uses, as a criterion for the selection, the accuracy of measuring the self-position and / or posture of the robots in the group of candidates in the environment. The information processing apparatus according to any one of Configurations 1 to 3.

[0137] (Configuration 5) The robot selection means uses the workload or cost of the labor as a criterion for selection, and is the information processing apparatus according to any one of Configurations 1 to 4.

[0138] (Configuration 6) The request includes information on the time limit for the provision of the labor, and the robot selection means selects the first group of robots capable of executing the labor within the time limit based on the workload, and is the information processing apparatus according to any one of Configurations 1 to 5.

[0139] (Configuration 7) The information processing apparatus according to any one of Configurations 1 to 6, further comprising data generation means for generating work data used by any one of the robots in the first group of robots for providing the labor in the environment based on the sensor information.

[0140] (Configuration 8) The work data includes map data for any one of the robots in the first group of robots to measure its own position and / or posture in the environment, and is the information processing apparatus according to Configuration 7.

[0141] (Configuration 9) The work data includes parameters of sensors mounted on the first group of robots, and is the information processing apparatus according to Configuration 7 or 8.

[0142] (Configuration 10) Any one of the robots in the first group of robots is equipped with a sensor corresponding to at least one of the one or more sensors, and is the information processing apparatus according to any one of Configurations 1 to 9.

[0143] (Configuration 11) When the sensor information is insufficient as a criterion for selection, the robot selection means notifies the apparatus equipped with the sensor or a predetermined user that additional sensor information is required, and is the information processing apparatus according to any one of Configurations 1 to 10.

[0144] The information processing apparatus according to configuration 11, wherein the notification includes information regarding the content of the sensor information that requires addition.

[0145] The information processing apparatus according to any one of configurations 1 to 12, wherein when there is no combination of robots in the candidate group that can be selected as the first robot group, the robot selection means notifies a predetermined user of the conditions for enabling any combination of robots in the candidate group to provide the labor.

[0146] An information processing system comprising: a user terminal that transmits a request for providing labor in a specified environment; a task acquisition means that acquires the request, which is installed by the user terminal; a sensor information acquisition means that acquires sensor information measured by one or more sensors within the environment; a robot candidate acquisition means that acquires information on a candidate group of robots that provide the labor; and a robot selection means that selects a first robot group consisting of one or more robots capable of providing the labor from the candidate group based on the request and the sensor information.

[0147] An information processing method comprising: a task acquisition step of acquiring a request for providing labor in a specified environment; a sensor information acquisition step of acquiring sensor information measured by one or more sensors within the environment; a robot candidate acquisition step of acquiring information on a candidate group of robots that provide the labor; and a robot selection step of selecting a first robot group consisting of one or more robots capable of providing the labor from the candidate group based on the request and the sensor information.

[0148] A computer program for controlling each means of the information processing apparatus according to any one of configurations 1 to 13 or the information processing system according to configuration 14 by a computer.

[0149] Furthermore, in order to implement part or all of the control in the above-described embodiment, a computer program that implements the functions of the above-described embodiment may be supplied to an information processing apparatus or the like via a network or various storage media. Then, a computer (or a CPU, MPU, etc.) in the information processing apparatus or the like may read and execute the program. In that case, the program and the storage medium storing the program will constitute the present invention.

Explanation of Signs

[0150] 101: Information processing apparatus 301: Task acquisition unit 302: Sensor information acquisition unit 303: Robot candidate acquisition unit 304: Robot selection unit

Claims

1. task acquisition means for acquiring a request for provision of labor in a specified environment; sensor information acquisition means for acquiring sensor information measured by one or more sensors within the environment; robot candidate acquisition means for acquiring information on a group of robot candidates for providing the labor; robot selection means for selecting, based on the request and the sensor information, a first group of robots capable of providing the labor from among the group of candidates; An information processing apparatus comprising the above.

2. The information processing apparatus according to claim 1, further comprising display control means for causing a display means to display information regarding the first group of robots selected by the robot selection means.

3. The information processing apparatus according to claim 1, wherein the robot selection means uses the labor provision efficiency of the robots in the group of candidates as a criterion for the selection.

4. The information processing apparatus according to claim 1, wherein the robot selection means uses the accuracy of measurement of the self-position and / or posture of the robots in the group of candidates in the environment as a criterion for the selection.

5. The information processing apparatus according to claim 1, wherein the robot selection means uses the workload or cost of the labor as a criterion for the selection.

6. The request includes information on the time limit for provision of the labor, and the robot selection means selects the first group of robots capable of executing the labor within the time limit based on the workload. The information processing apparatus according to claim 1.

7. The information processing apparatus according to claim 1, comprising data generation means for generating work data used by any one of the robots in the first group of robots for providing the labor within the environment based on the sensor information.

8. The information processing apparatus according to claim 7, wherein the work data includes map data for any one of the robots in the first group of robots to measure its own position and / or posture within the environment.

9. The information processing apparatus according to claim 7, wherein the work data includes parameters of sensors mounted on the first group of robots.

10. Any one of the robots in the first group of robots is mounted with a sensor corresponding to at least one of the one or more sensors. The information processing apparatus according to claim 1.

11. The robot selection means is characterized in that, when the sensor information is insufficient as a basis for the selection, it notifies the device equipped with the sensor or a predetermined user that additional sensor information is required. The information processing apparatus according to claim 1.

12. The information processing apparatus according to claim 11, wherein the notification includes information regarding the content of the sensor information for which addition is required.

13. The robot selection means is characterized in that, when there is no combination of robots in the candidate group that can be selected as the first robot group, it notifies a predetermined user of the conditions for enabling any combination of robots in the candidate group to provide the labor. The information processing apparatus according to claim 1.

14. A user terminal that transmits a request for providing labor in a specified environment, task acquisition means for acquiring the request, which is installed by the user terminal, sensor information acquisition means for acquiring sensor information measured by one or more sensors within the environment, robot candidate acquisition means for acquiring information on a candidate group of robots that provide the labor, robot selection means for selecting, based on the request and the sensor information, a first robot group consisting of one or more robots capable of providing the labor from the candidate group, An information processing system characterized by comprising the above.

15. A task acquisition step of acquiring a request for providing labor in a specified environment, a sensor information acquisition step of acquiring sensor information measured by one or more sensors within the environment, a robot candidate acquisition step of acquiring information on a candidate group of robots that provide the labor, a robot selection step of selecting, based on the request and the sensor information, a first robot group consisting of one or more robots capable of providing the labor from the candidate group, An information processing method characterized by comprising the above.

16. A computer program for controlling each means of the information processing apparatus according to any one of claims 1 to 13 or the information processing system according to claim 14 by a computer.

Images