Information processing method, program, and information processing system

Abstract

Provided is an information processing method executed by a computer, wherein the information processing method involves: acquiring user information including the location of a user (S1); acquiring robot information including the locations of each of one or more robots (S2); identifying one or more candidate robots that are candidates for the user to pay attention to on the basis of the acquired user information and robot information (S3); and outputting notification information including at least one of information urging attention to the identified one or more candidate robots, information indicating the location of each of the one or more candidate robots, or information indicating the movement of each of the one or more candidate robots (S4).

Description

Information processing method, program, and information processing system

[0001] The present disclosure relates to an information processing method, a program, and an information processing system for presenting information about a robot to a user.

[0002] Patent Document 1 discloses a method for controlling an autonomously moving mobile robot.

[0003] Japanese Patent Application Laid-Open No. 2019-133450

[0004] The present disclosure aims to provide an information processing method and the like that allows a user to easily grasp information about a robot that may be an object of the user's attention.

[0005] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, which acquires user information including a user's position, acquires robot information including the positions of each of one or more robots, identifies one or more candidate robots that the user should pay attention to based on the acquired user information and the acquired robot information, and outputs notification information including at least one of information calling attention to the identified one or more candidate robots, information indicating the position of each of the one or more candidate robots, and information indicating the movement of each of the one or more candidate robots.

[0006] Furthermore, a program according to one aspect of the present disclosure causes a computer to execute the information processing method.

[0007] Also, an information processing system according to an aspect of the present disclosure includes a first acquisition unit, a second acquisition unit, an identification unit, and an output unit. The first acquisition unit acquires user information including a user's position. The second acquisition unit acquires robot information including the positions of one or more robots. The identification unit identifies one or more candidate robots that the user should pay attention to based on the user information acquired by the first acquisition unit and the robot information acquired by the second acquisition unit. The output unit outputs notification information including at least one of information calling attention to the one or more candidate robots identified by the identification unit, information indicating the positions of each of the one or more candidate robots, and information indicating the movement of each of the one or more candidate robots.

[0008] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0009] The present disclosure has the advantage of making it easier for the user to grasp information about a robot that may be an object of the user's attention.

[0010] FIG. 1 is a block diagram showing an overall configuration including an information processing system according to an embodiment. FIG. 2 is a schematic diagram showing an example of a third robot. FIG. 3 is a schematic diagram showing an example of a process for identifying one or more candidate robots. FIG. 4 is a schematic diagram showing another example of a process for identifying one or more candidate robots. FIG. 5 is a schematic diagram showing an example of a process for determining one or more zones. FIG. 6 is a schematic diagram showing a first example of notification information presented by a presentation device. FIG. 7 is a schematic diagram showing a second example of notification information presented by a presentation device. FIG. 8 is a schematic diagram showing a third example of notification information presented by a presentation device. FIG. 9 is a flowchart showing an example of a basic operation of an information processing system according to an embodiment.

[0011] In recent years, in response to factors such as a trend toward a decrease in the number of workers engaged in work at work sites (for example, carpenters engaged in construction work at construction sites), the introduction of various robots to perform work in place of workers at work sites is progressing. At work sites where such robots are introduced, workers and robots will coexist, and smooth positioning of each is required to prevent the workers and robots from interfering with each other.

[0012] Here, if there are other workers around, the worker can easily understand whether the other workers will interfere with the worker by looking at the facial expressions or movements of the other workers. In this case, even if the other workers are in a situation where they are likely to interfere with the worker, if the worker issues a warning to the other workers, the situation of interference with the worker can be easily avoided. In other words, if there are other workers around the worker, the worker does not feel particularly uneasy about the other workers.

[0013] On the other hand, when there is a robot nearby, it is difficult for a worker to know whether the robot will interfere with the worker even when looking at the robot. In addition, even if the robot is likely to interfere with the worker, the worker cannot warn the robot and must take measures such as moving away from the robot. In other words, when there is a robot nearby, the worker is likely to feel uneasy about the robot.

[0014] One possible method to make it easier for a worker to pay attention to each robot is to display information about all robots around the worker (e.g., the direction of movement of each robot) on the display of smart glasses worn by the worker. However, this method has the problem that, because information about all robots around the worker is displayed on the display, the worker is likely to be overwhelmed with information, which makes it difficult for the worker to grasp information about robots that may require attention.

[0015] In view of the above, the present disclosure aims to provide an information processing method etc. that allows a user (worker) to easily grasp information about a robot that may be an object of attention for the user.

[0016] More specifically, an information processing method according to a first aspect of the present disclosure is an information processing method executed by a computer, which acquires user information including the user's position, acquires robot information including the positions of each of one or more robots, identifies one or more candidate robots that the user should pay attention to based on the acquired user information and the acquired robot information, and outputs notification information including at least one of information calling attention to the identified one or more candidate robots, information indicating the position of each of the one or more candidate robots, and information indicating the movement of each of the one or more candidate robots.

[0017] This has the advantage that the user can be presented with only information about one or more candidate robots, rather than information about all robots around the user, making it easier for the user to grasp information about robots (one or more candidate robots) that may be of interest to the user.

[0018] Also, for example, in an information processing method according to a second aspect of the present disclosure, in the first aspect, the information indicating the movement of each of the one or more candidate robots includes at least one of the movement direction and movement path of each of the one or more candidate robots.

[0019] This has the advantage that it is possible to present the user with information indicating how a robot that may be an object of the user's attention moves, making it easier for the user to pay attention to the robot.

[0020] Also, for example, in an information processing method according to a third aspect of the present disclosure, in the first or second aspect, the process of identifying one or more candidate robots involves determining one or more zones based on the user's position, and identifying robots present within the determined one or more zones as candidate robots.

[0021] This has the advantage that only robots that exist within one or more zones are identified as candidate robots, making it easier for the user to grasp information about robots that may be of interest to the user.

[0022] Furthermore, for example, in an information processing method according to the fourth aspect of the present disclosure, in any one of the first to third aspects, in the process of acquiring robot information, behavioral information indicating the behavior of each of the one or more robots is further acquired, and in the process of identifying one or more candidate robots, one or more candidate robots are identified further based on the acquired behavioral information.

[0023] This has the advantage that one or more candidate robots can be identified more quickly than when one or more candidate robots are identified based solely on user information, making it easier for the user to quickly grasp information about robots that may be of interest to the user.

[0024] Also, for example, in the information processing method according to the fifth aspect of the present disclosure, in the third aspect, in the process of acquiring user information, posture information indicating the user's posture is further acquired, and in the process of identifying one or more candidate robots, one or more zones are determined further based on the acquired posture information.

[0025] This has the advantage that one or more zones can be determined more quickly than when one or more zones are determined based solely on user information, making it easier for the user to quickly grasp information about robots that may require the user's attention.

[0026] Also, for example, in the information processing method according to the sixth aspect of the present disclosure, in the third or fourth aspect, in the process of acquiring user information, orientation information indicating at least one of the user's facial orientation and gaze direction is further acquired, and in the process of identifying one or more candidate robots, one or more zones are determined further based on the acquired orientation information.

[0027] This has the advantage that it is possible to identify as candidate robots robots that would not be captured if one or more zones were determined based solely on user information, making it easier to accurately identify robots that may be of interest to the user.

[0028] Furthermore, for example, in an information processing method according to a seventh aspect of the present disclosure, in any one of the first to sixth aspects, the one or more robots include an autonomous flying robot. The process of acquiring robot information further acquires height information indicating the height of the autonomous flying robot, and the process of identifying one or more candidate robots further identifies the one or more candidate robots based on the acquired height information.

[0029] This has the advantage that candidate robots can be identified taking into account the height of the autonomous flying robot (second robot), making it easier to accurately identify robots that may require the user's attention.

[0030] Furthermore, for example, in an information processing method according to an eighth aspect of the present disclosure, in any one of the first to seventh aspects, the one or more robots include a robot having a manipulator. The process of acquiring the robot information further acquires movable range information indicating a movable range of the manipulator, and the process of identifying one or more candidate robots further identifies the one or more candidate robots based on the acquired movable range information.

[0031] This has the advantage that candidate robots can be identified taking into account the range of motion of the manipulator of the robot having the manipulator (third robot), making it easier to accurately identify robots that may require the user's attention.

[0032] Also, for example, in an information processing method according to a ninth aspect of the present disclosure, in any one of the first to eighth aspects, the process of outputting notification information outputs the notification information to smart glasses worn by the user.

[0033] This has the advantage that notification information can be superimposed on the user's field of view on the smart glasses display, allowing the user to grasp information about robots that may be of interest to the user while looking at the real space.

[0034] Also, for example, a program according to a tenth aspect of the present disclosure causes a computer to execute the information processing method of any one of the first to ninth aspects.

[0035] This has the advantage that it is possible to achieve the same effect as the above-mentioned information processing method.

[0036] Also, for example, an information processing system according to an eleventh aspect of the present disclosure includes a first acquisition unit, a second acquisition unit, an identification unit, and an output unit. The first acquisition unit acquires user information including a user's position. The second acquisition unit acquires robot information including the positions of one or more robots. The identification unit identifies one or more candidate robots that are candidates for the user to pay attention to, based on the user information acquired by the first acquisition unit and the robot information acquired by the second acquisition unit. The output unit outputs notification information including at least one of information calling for attention to the one or more candidate robots identified by the identification unit, information indicating the positions of each of the one or more candidate robots, and information indicating the movement of each of the one or more candidate robots.

[0037] This has the advantage that it is possible to achieve the same effect as the above-mentioned information processing method.

[0038] In addition, for example, an information processing system according to a twelfth aspect of the present disclosure is the eleventh aspect, further including a presentation device. The presentation device includes a communication unit that receives notification information output by the output unit, and a presentation unit that presents the notification information received by the communication unit to a user.

[0039] This has the advantage that it is possible to achieve the same effect as the above-mentioned information processing method.

[0040] Furthermore, these comprehensive or specific aspects may be realized in a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized in any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0041] Hereinafter, embodiments will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, or step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components. Note that each figure is a schematic diagram and is not necessarily an exact illustration. Furthermore, in each figure, substantially identical components are assigned the same reference numerals, and duplicated descriptions may be omitted or simplified.

[0042] (Embodiment) [1. Configuration] First, an overall configuration including an information processing system according to an embodiment will be described. FIG. 1 is a block diagram showing an overall configuration including an information processing system according to an embodiment. As shown in FIG. 1, an information processing system 100 according to an embodiment includes a server 1 and a presentation device 2. In the embodiment, the presentation device 2 is smart glasses 20 worn by a user 40 (see FIG. 3 ), who is one of one or more workers 4 present at a work site. Note that the information processing system 100 only needs to include the server 1, and may not necessarily include the smart glasses 20 (presentation device 2).

[0043] In the embodiment, the work site is a space where one or more workers 4 (here, multiple workers 4), including a user 40, and one or more robots 3 (here, multiple robots 3) each perform work, and where one or more workers 4 and one or more robots 3 coexist (see Figure 3).

[0044] The robot 3 is a robot that autonomously performs work at a work site that is equivalent to work performed by a worker 4. The robot 3 has a wireless communication interface that performs wireless communication with the server 1 via a network N1 such as the Internet or a wireless LAN (Local Area Network).

[0045] The robot 3 may include, for example, an autonomous flying robot (see the second robot 32 shown in FIG. 6 , etc., which will be described later) that performs indoor inspection work at a work site, such as a drone. The robot 3 may also include, for example, an autonomous mobile robot (see the first robot 31 shown in FIG. 6 , etc., which will be described later) that performs work such as taking photos at a work site, remotely monitoring the work site, or automatically patrolling the work site. The autonomous mobile robot may include a walking robot that walks on multiple legs, a mobile robot that travels on multiple wheels, etc.

[0046] Furthermore, as shown in Fig. 2, the robot 3 may include a third robot 33 having a manipulator that performs work equivalent to work performed manually by a worker 4 at a work site, for example. Fig. 2 is a schematic diagram showing an example of the third robot 33. In the example shown in Fig. 2, the third robot 33 is a mobile robot that travels on multiple wheels. In the example shown in Fig. 2, the third robot 33 has, as manipulators, a movable robot arm 331 and a robot hand 332 provided at the tip of the robot arm 331.

[0047] The third robot 33 may be movable in a manner other than by using multiple wheels, or may be fixed to the work site. The third robot 33 may have multiple robot arms 331. The third robot 33 may not have the robot hand 332.

[0048] The smart glasses 20 are eyeglass-type wearable terminals, so-called Augmented Reality (AR) glasses. In the embodiment, the smart glasses 20 are worn on both eyes of the user 40, but may be worn on one eye of the user 40. As shown in FIG. 1 , the smart glasses 20 (presentation device 2) include a communication unit 21, a processing unit 22, and a presentation unit 23.

[0049] The communication unit 21 is a wireless communication interface for communicating with the server 1. The communication unit 21 transmits various types of information to the server 1 and receives various types of information from the server 1 by performing wireless communication with the server 1 via the network N1. In the embodiment, the communication unit 21 receives notification information (described later) output by an output unit 124 (described later) of the server 1.

[0050] The processing unit 22 includes, for example, a processor and a memory, and performs various functions of the smart glasses 20 by the processor executing a program stored in the memory.

[0051] The presentation unit 23 presents the notification information received by the communication unit 21 to the user 40. In the embodiment, the presentation unit 23 is a transmissive display attached to the smart glasses 20, which transmits light from the surroundings. Therefore, the user 40 can see the real space through the presentation unit 23. In addition, a notification object 5 (see FIG. 6 ) indicating the notification information is displayed superimposed on the field of view of the user 40 on the presentation unit 23. Therefore, the user 40 can see the real space as well as the notification object 5, such as an image or text, that does not exist in the real space.

[0052] The server 1 is installed at a work site or a location remote from the work site, and is configured to be able to communicate with the smart glasses 20 and one or more robots 3 via a network N1. In the example shown in FIG. 1 , the server 1 is configured to be able to communicate with one pair of smart glasses 20, but if there are multiple users 40, the server 1 may be able to communicate with each of multiple pairs of smart glasses 20 worn by the multiple users 40. As shown in FIG. 1 , the server 1 includes a communication unit 11, a processing unit 12, and a storage unit 13.

[0053] The communication unit 11 is a wireless communication interface for communicating with the presentation device 2 (smart glasses 20) and each of the one or more robots 3. The communication unit 11 transmits various types of information to the presentation device 2 and receives various types of information from the presentation device 2 by performing wireless communication with the presentation device 2 via the network N1. The communication unit 11 also transmits various types of information to each of the one or more robots 3 and receives various types of information from each of the one or more robots 3 by performing wireless communication with each of the one or more robots 3 via the network N1.

[0054] The processing unit 12 includes, for example, a processor and a memory, and performs various functions of the server 1 by executing a program stored in the memory with the processor. In the embodiment, the processing unit 12 includes a first acquisition unit 121, a second acquisition unit 122, an identification unit 123, and an output unit 124.

[0055] The first acquisition unit 121 acquires user information including the position of the user 40. The first acquisition unit 121 is an entity that executes a process of acquiring user information. In the embodiment, the first acquisition unit 121 acquires the user information by receiving information indicating the position of the smart glasses 20 (in other words, the position of the user 40) from the smart glasses 20 via the communication unit 11. The information indicating the position of the smart glasses 20 can be acquired by, for example, measuring the position using a positioning system such as a GPS (Global Positioning System) provided in the smart glasses 20.

[0056] In addition, the first acquisition unit 121 may acquire one or more detection results transmitted from each of one or more human detection sensors installed at a work site via the communication unit 11, and calculate the position of the user 40 based on the acquired one or more detection results.

[0057] Furthermore, the first acquisition unit 121 (i.e., in the process of acquiring user information) may further acquire posture information indicating the posture of the user 40. Here, the posture of the user 40 may include, for example, a posture of the user 40 standing, a posture of the user 40 sitting, a posture of the user 40 about to stand up, or a posture of the user 40 performing work such as carrying an object.

[0058] For example, the first acquisition unit 121 may acquire an image of the user 40, and use an appropriate image analysis algorithm or a machine-learned model on the acquired image to detect multiple feature points such as the body, arms, legs, and joints of the user 40, and estimate the posture of the user 40 based on the detected multiple feature points. The image of the user 40 can be acquired, for example, by capturing an image of the user 40 with a camera installed at a work site and receiving the image transmitted from the camera via the communication unit 11.

[0059] Furthermore, the first acquisition unit 121 (i.e., in the process of acquiring user information) may further acquire orientation information indicating at least one of the facial orientation and gaze direction of the user 40. For example, the first acquisition unit 121 may acquire an image of the user 40, and use an appropriate image analysis algorithm or a machine-learned model on the acquired image to detect multiple facial feature points, such as the eyes, nose, and mouth, in the facial region of the user 40, and estimate the facial orientation of the user 40 based on the detected multiple facial detection points. Furthermore, for example, the first acquisition unit 121 may estimate the gaze direction of the user 40 based on a combination of the estimated facial orientation and the detected orientation of the pupils of the eyes.

[0060] The second acquisition unit 122 acquires robot information including the position of each of the one or more robots 3. The second acquisition unit 122 is an entity that executes the process of acquiring the robot information. In the embodiment, the second acquisition unit 122 acquires the robot information by receiving information indicating the position of the robot 3 from each of the one or more robots 3 via the communication unit 11. The information indicating the position of the robot 3 can be acquired, for example, by measuring the position using a positioning system such as GPS that the robot 3 is equipped with.

[0061] In addition, the second acquisition unit 122 may acquire one or more detection results transmitted from each of one or more object detection sensors installed at the work site via the communication unit 11, and calculate the position of each of one or more robots 3 based on the acquired one or more detection results.

[0062] Furthermore, the second acquisition unit 122 (i.e., in the process of acquiring robot information) may further acquire behavioral information indicating the behavior of each of the one or more robots 3. The behavioral information may include, for example, the work content to be performed by each of the one or more robots 3, or the planned movement path of each of the one or more robots 3. The behavioral information may be acquired from each of the one or more robots 3 by communicating with each of the one or more robots 3 via the communication unit 11, or may be acquired from a management system that manages the one or more robots 3 by communicating with the management system via the communication unit 11.

[0063] Furthermore, the second acquisition unit 122 (i.e., in the process of acquiring robot information) may further acquire height information indicating the height of the autonomous flying robot if an autonomous flying robot (second robot 32) is included in one or more robots 3. For example, the second acquisition unit 122 can acquire the height information by communicating with the second robot 32 via the communication unit 11 and receiving information indicating the position of the second robot 32 determined by a positioning system provided in the second robot 32.

[0064] Furthermore, the second acquisition unit 122 (i.e., in the process of acquiring robot information) may further acquire movable range information indicating the movable range of the manipulator when one or more robots 3 includes a robot having a manipulator (third robot 33). For example, the second acquisition unit 122 can acquire the movable range information by communicating with the third robot 33 via the communication unit 11 and receiving the movable range information stored in the third robot 33.

[0065] The identification unit 123 identifies one or more candidate robots 30 (see FIG. 3 etc.) that are candidates for the user 40 to pay attention to, based on the user information acquired by the first acquisition unit 121 and the robot information acquired by the second acquisition unit 122. The identification unit 123 is the entity that executes the process of identifying one or more candidate robots 30. Note that the candidate robots 30 are robots 3 that the user 40 is recommended to pay attention to. Therefore, the user 40 does not need to pay attention to the candidate robots 30.

[0066] In the embodiment, the identification unit 123 (i.e., in the process of identifying one or more candidate robots 30) determines one or more zones A1 based on the position of the user 40, and identifies robots 3 present in the determined one or more zones A1 as candidate robots 30. For example, the one or more zones A1 are concentric regions centered on the position of the user 40 in a planar view. An example of the process of identifying one or more candidate robots 30 by the identification unit 123 will be described below.

[0067] FIG. 3 is a schematic diagram showing an example of a process for identifying one or more candidate robots 30. FIG. 3 illustrates a situation in which multiple robots 3 and multiple workers 4, including a user 40, coexist at a work site. A similar situation is also illustrated in FIG. 4, which will be described later. In the example shown in FIG. 3, the user 40 is stationary and working while facing the south wall. In this case, the identification unit 123 determines not to set one or more zones A1 based on the user information that the user 40 is stationary. In other words, in this case, the identification unit 123 does not identify any robot 3 as a candidate robot 30.

[0068] Fig. 4 is a schematic diagram showing another example of the process of identifying one or more candidate robots 30. In the example shown in Fig. 4, the user 40 is moving in a northeasterly direction. The arrow in Fig. 4 indicates the direction in which the user 40 is moving. In this case, the identification unit 123 determines one or more zones A1 (here, a first zone A11 and a second zone A12) centered on the position of the user 40, based on the user information that the user 40 is moving.

[0069] The first zone A11 is a circular area of ​​a first radius centered on the position of the user 40 in a planar view. The second zone A12 is a circular area of ​​a second radius (>first radius) centered on the position of the user 40 in a planar view, excluding the first zone A11. In the example shown in Fig. 4, the robot 3 present near the user 40 is located within the second zone A12. Therefore, the identification unit 123 identifies the robot 3 as a candidate robot 30.

[0070] The following describes an example of processing for determining one or more zones A1 by the identification unit 123. Fig. 5 is a schematic diagram illustrating an example of processing for determining one or more zones A1 (here, a first zone A11 and a second zone A12).

[0071] 5A shows one or more zones A1 determined based on the position of the user 40. In the example shown in FIG. 5A, the first zone A11 and the second zone A12 are concentric regions centered on the position of the user 40 in a planar view, similar to the example shown in FIG. 4. In this way, the identification unit 123 may determine one or more zones A1 based only on the position of the user 40.

[0072] 5B shows one or more zones A1 determined based on the direction of movement of the user 40. In FIG. 5B, the solid arrow indicates the direction of movement of the user 40. In the example shown in FIG. 5B, the first zone A11 and the second zone A12 are both elliptical regions whose major axes are elongated in the direction of movement of the user 40. In this way, the identification unit 123 may determine one or more zones A1 based not only on the position of the user 40 but also on the direction of movement of the user 40.

[0073] 5C shows one or more zones A1 determined based on the facial orientation (or gaze direction) of the user 40. In FIG. 5C, the solid arrow indicates the direction in which the user 40 moves, and the dashed arrow indicates the facial orientation (or gaze direction) of the user 40. In the example shown in FIG. 5C, the first zone A11 and the second zone A12 are both elliptical regions whose major axes are elongated in the facial orientation (or gaze direction) of the user 40. In this way, the identification unit 123 may determine one or more zones A1 based not only on the position of the user 40 but also on the facial orientation (or gaze direction) of the user 40.

[0074] In other words, the identification unit 123 (i.e., in the process of identifying one or more candidate robots 30) may determine one or more zones A1 further based on the acquired orientation information. For example, if one or more robots 3 are present in the facial orientation (or gaze direction) of the user 40 indicated by the orientation information, the identification unit 123 can determine one or more zones A1 even if the user information indicates that the user 40 is stationary. This aspect has the advantage that it is possible to identify, as candidate robots 30, robots 3 that cannot be captured when determining one or more zones A1 based only on user information, making it easier to accurately identify robots 3 that may be the target of the user 40's attention.

[0075] Here, when the first acquisition unit 121 acquires posture information, the identification unit 123 (i.e., in the process of identifying one or more candidate robots 30) may determine one or more zones A1 further based on the acquired posture information. For example, when the posture information indicates that the posture of the user 40 is taking a step, the identification unit 123 can determine one or more zones A1 from a point in time before the point in time when it is determined that the user 40 has started to move based on the user information. This aspect has the advantage that, compared to determining one or more zones A1 based only on the user information, one or more zones A1 can be determined earlier, making it easier for the user 40 to quickly grasp information about a robot 3 that may be an object of attention for the user 40.

[0076] Furthermore, when the second acquisition unit 122 acquires behavioral information, the identification unit 123 (i.e., in the process of identifying one or more candidate robots 30) may identify one or more candidate robots 30 further based on the acquired behavioral information. For example, when the movement path of the robot 3 indicated by the behavioral information is a path that passes through one or more zones A1, the identification unit 123 may identify the robot 3 as a candidate robot 30 when the robot 3 is not within one or more zones A1. Furthermore, when the work content of the robot 3 indicated by the behavioral information is work content related to the user 40, the identification unit 123 may identify the robot 3 as a candidate robot 30 when the robot 3 is not within one or more zones A1. This aspect has the advantage that one or more candidate robots 30 can be identified more quickly than when one or more candidate robots 30 are identified based only on user information, making it easier for the user 40 to quickly grasp information about robots 3 that may be of interest to the user 40.

[0077] Furthermore, if the second acquisition unit 122 acquires height information, the identification unit 123 (i.e., in the process of identifying one or more candidate robots 30) may identify one or more candidate robots 30 based on the acquired height information. For example, when an autonomous flying robot (second robot 32) is present in one or more zones A1, the identification unit 123 compares the height of the second robot 32 indicated by the height information with a predetermined height. If the height of the second robot 32 is below the predetermined height, the identification unit 123 identifies the second robot 32 as a candidate robot 30, and if the height of the second robot 32 is above the predetermined height, the identification unit 123 does not identify the second robot 32 as a candidate robot 30. This aspect has the advantage that the candidate robots 30 can be identified taking the height of the autonomous flying robot (second robot 32) into consideration, making it easier to accurately identify robots 3 that may require the user 40's attention.

[0078] Furthermore, when the second acquisition unit 122 has acquired the movable range information, the identification unit 123 (i.e., in the process of identifying one or more candidate robots 30) may identify one or more candidate robots 30 further based on the acquired movable range information. For example, even if the position of a robot having a manipulator indicated in the robot information (third robot 33) is outside one or more zones A1, if the movable range of the manipulator of the third robot 33 indicated in the movable range information overlaps one or more zones A1, the identification unit 123 identifies the third robot 33 as a candidate robot 30. This aspect has the advantage that the candidate robots 30 can be identified taking into account the movable range of the manipulator of the robot having a manipulator (third robot 33), making it easier to accurately identify robots 3 that may require the user 40's attention.

[0079] The output unit 124 outputs notification information. The output unit 124 is an entity that executes a process of outputting the notification information. Here, the notification information includes at least one of information calling attention to one or more candidate robots 30 identified by the identification unit 123, information indicating the position of each of the one or more candidate robots 30, and information indicating the movement of each of the one or more candidate robots 30. In the embodiment, the output unit 124 outputs the notification information by transmitting the notification information to the presentation device 2 (smart glasses 20) via the communication unit 11. An example of a process of outputting notification information by the output unit 124 will be described below.

[0080] Fig. 6 is a schematic diagram showing a first example of notification information presented on the presentation device 2 (smart glasses 20). Fig. 6 shows the field of view of the user 40 on the presentation unit 23 (the display of the smart glasses 20). Figs. 7 and 8, which will be described later, also show the field of view of the user 40 on the presentation unit 23 in a similar manner.

[0081] 6, the first example is a situation in which two workers 4, one autonomous mobile robot (first robot 31), and two autonomous flying robots (second robots 32) are present in the field of view of the user 40 on the presentation unit 23. In the first example, the first robot 31 and the second robot 32 that is in the foreground of the two second robots 32 are each identified as a candidate robot 30. The second and third examples described below are similar situations.

[0082] In the first example, as shown in Figure 6, multiple notification objects 5 (here, two first notification objects 51 and two second notification objects 52) are displayed overlapping each other as notification information in the field of view of the user 40 on the presentation unit 23.

[0083] The two first notification objects 51 are both exclamation marks, and are information that indicates to the user 40 the position of the candidate robot 30 and also information that calls the user 40 to pay attention to the candidate robot 30. The two first notification objects 51 are displayed near the first robot 31 and near the second robot 32 on the front side, respectively.

[0084] The two second notification objects 52 are both character strings indicating the speed of the candidate robot 30, and are information indicating the position of the candidate robot 30 to the user 40 and information indicating the movement of the candidate robot 30 to the user 40. The two second notification objects 52 are displayed near the first robot 31 and near the second robot 32 in front, respectively. Note that the speed of the candidate robot 30 can be estimated based on, for example, the amount of change in the position of the candidate robot 30 indicated by the robot information. The speed of the candidate robot 30 can also be obtained by, for example, receiving the detection result of a speed sensor provided in the candidate robot 30 via the communication unit 11.

[0085] In the first example, by looking at the two first notification objects 51, the user 40 is more likely to pay attention to the first robot 31 and the second robot 32 in the foreground. In addition, by looking at the two second notification objects 52, the user 40 can grasp the movement speeds of the first robot 31 and the second robot 32 in the foreground.

[0086] 7 is a schematic diagram illustrating a second example of notification information presented on the presentation device 2 (smart glasses 20). In the second example, as shown in FIG. 7, a plurality of notification objects 5 (two third notification objects 53 in this example) are displayed overlapping each other as notification information in the field of view of the user 40 on the presentation unit 23.

[0087] The two third notification objects 53 are both arrows indicating the direction in which the candidate robot 30 will move, and are information indicating the position of the candidate robot 30 to the user 40, as well as information indicating the movement of the candidate robot 30 to the user 40. Furthermore, the two third notification objects 53 are also information indicating the movement direction (or movement path) of the candidate robot 30. The two third notification objects 53 are displayed near the first robot 31 and near the second robot 32 on the front side, respectively.

[0088] In the second example, by looking at the two third notification objects 53, the user 40 is more likely to pay attention to the first robot 31 and the second robot 32 in the foreground. Furthermore, by looking at the two third notification objects 53, the user 40 can understand how the first robot 31 and the second robot 32 in the foreground are moving. In other words, in the second example, it is possible to present to the user 40 information indicating how a robot 3 that may be an object of attention for the user 40 (one or more candidate robots 30) is moving, which has the advantage of making it even easier for the user 40 to pay attention to that robot 3.

[0089] 8 is a schematic diagram illustrating a third example of notification information presented by the presentation device 2 (smart glasses 20). In the third example, as shown in FIG. 8, a plurality of notification objects 5 (two fourth notification objects 54 in this example) are displayed overlapping each other as notification information in the field of view of the user 40 on the presentation unit 23.

[0090] The two fourth notification objects 54 are both circles surrounding the candidate robot 30, and are information that indicates the position of the candidate robot 30 to the user 40 and also information that calls the user 40's attention to the candidate robot 30. The two fourth notification objects 54 are displayed superimposed on the first robot 31 and the second robot 32 in the foreground, respectively.

[0091] In the third example, the first robot 31 is located in the first zone A11, and the second robot 32 on the near side is located in the second zone A12. Therefore, in the third example, the fourth notification object 54 overlapping the first robot 31 is displayed more emphasized than the fourth notification object 54 overlapping the second robot 32 on the near side.

[0092] In the third example, by looking at the two fourth notification objects 54, the user 40 is more likely to pay attention to the first robot 31 and the second robot 32 in the foreground. Furthermore, by looking at the fourth notification object 54 that is displayed in an emphasized manner out of the two fourth notification objects 54, the user 40 can understand that, of the first robot 31 and the second robot 32 in the foreground, the user 40 should pay particular attention to the first robot 31.

[0093] The storage unit 13 is a dedicated or general-purpose memory for storing information in the information processing system 100. The storage unit 13 may be an electric circuit. The storage unit 13 may also be a magnetic disk, an optical disk, or the like, and may be expressed as a storage or a recording medium. The storage unit 13 may also be a non-volatile memory or a volatile memory. The storage unit 13 may also be included in the processing unit 12. For example, the storage unit 13 may store information for the processing unit 12 to perform information processing, or may store a program for the processing unit 12 to perform information processing.

[0094] [2. Operation] The operation of the information processing system 100 according to the embodiment, that is, the information processing method, will be described below. Fig. 9 is a flowchart showing an example of the basic operation of the information processing system 100 according to the embodiment. The following description will be made on the assumption that steps S1 to S4 shown in Fig. 9 are repeatedly executed for each processing unit time.

[0095] First, when the operation of the information processing system 100 starts, the first acquisition unit 121 acquires user information (S1). The second acquisition unit 122 acquires robot information (S2). Note that the order of executing steps S1 and S2 may be reversed. Furthermore, steps S1 and S2 may be executed in parallel.

[0096] Next, the identification unit 123 identifies one or more candidate robots 30 based on the user information acquired by the first acquisition unit 121 and the robot information acquired by the second acquisition unit 122 (S3). Note that in step S3, the identification unit 123 may identify one or more candidate robots 30 based further on information different from the user information and robot information, such as the above-mentioned behavior information, posture information, orientation information, height information, or range of motion information.

[0097] Then, the output unit 124 outputs the notification information (S4). In step S4, the output unit 124 outputs the notification information by transmitting the notification information to the smart glasses 20 (presentation device 2) via the communication unit 11. When the presentation unit 23 of the smart glasses 20 receives the notification information via the communication unit 21, it presents the received notification information to the user 40. This allows the user 40 to grasp information about the robot 3 (one or more candidate robots 30) that may be an object of attention for the user 40.

[0098] [3. Advantages] Advantages of the information processing system 100 (information processing method) according to the embodiment will be described below. For example, if information about all robots 3 present within the field of view of the user 40 is displayed on the display of the smart glasses 20 worn by the user 40, the user 40 may be overwhelmed with information, which may make it difficult for the user 40 to identify the robots 3 that require attention.

[0099] In contrast to this, the information processing system 100 according to the embodiment can present to the user 40 only information relating to one or more candidate robots 30, rather than information relating to all robots 3 present around the user 40. Therefore, the information processing system 100 according to the embodiment has the advantage that it becomes easier for the user 40 to grasp information about robots 3 (one or more candidate robots 30) that may be targets of attention for the user 40.

[0100] Other Embodiments Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0101] In the above embodiment, the presentation device 2 is smart glasses 20, but is not limited thereto. For example, the presentation device 2 may be a goggle-type head-mounted display or a headset such as an MR (Mixed Reality) headset. Furthermore, the presentation unit 23 of the presentation device 2 is not limited to a transparent display and may be a non-transparent display. In this case, an image of real space captured by a camera provided in the presentation device 2 may be displayed on the presentation unit 23. Furthermore, the presentation device 2 is not limited to a head-mounted display and may be a portable terminal such as a smartphone or a tablet terminal. In this case, the user 40 may hold the presentation device 2 in front of him or her to display an image in front of him or her on the presentation unit 23.

[0102] Furthermore, for example, the presentation unit 23 of the presentation device 2 may present the notification information to the user 40 by outputting sound from a speaker provided in the presentation device 2. In this case, the presentation unit 23 may output, for example, a warning sound or a message such as "Robot 3 is approaching" by sound so that the user 40 can perceive it as coming from the candidate robot 30.

[0103] Furthermore, for example, the presentation device 2 may be a device that outputs only sound, such as earphones or headphones worn by the user 40. In this case, the presentation unit 23 of the presentation device 2 may output a warning sound or a message such as "Robot 3 is approaching" in a voice similar to that described above so that the user 40 perceives it as coming from the candidate robot 30.

[0104] Furthermore, for example, the presentation device 2 may be a relatively large display installed at a work site. In this case, the processing unit 12 of the information processing system 100 may determine whether the face direction (or the gaze direction) of the user 40 is facing the display, for example, based on the acquired user information and orientation information. Then, the output unit 124 may display notification information on the display, for example, when it is determined that the face direction (or the gaze direction) of the user 40 is facing the display.

[0105] Furthermore, for example, the information processing system described in the above embodiments may be realized as a single device (e.g., a server) that has all of the components, or may be realized by allocating each function to multiple devices and having these multiple devices work together.

[0106] In the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit. The order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0107] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0108] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0109] Furthermore, the general or specific aspects of the present disclosure may be realized as an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, etc. Furthermore, the general or specific aspects of the present disclosure may be realized as any combination of an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0110] For example, the present disclosure may be realized as an information processing method executed by a computer, or as a program for causing a computer to execute the information processing method. The present disclosure may also be realized as a computer-readable non-transitory recording medium on which such a program is recorded.

[0111] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the intent of this disclosure.

[0112] The present disclosure is useful in presenting information about a robot to a user.

[0113] REFERENCE SIGNS LIST 1 Server 11 Communication unit 12 Processing unit 121 First acquisition unit 122 Second acquisition unit 123 Identification unit 124 Output unit 13 Memory unit 2 Presentation device 20 Smart glasses 21 Communication unit 22 Processing unit 23 Presentation unit 3 Robot 30 Candidate robot 31 First robot 32 Second robot 33 Third robot 331 Robot arm 332 Robot hand 4 Worker 40 User 5 Notification object 51 First notification object 52 Second notification object 53 Third notification object 54 Fourth notification object 100 Information processing system A1 Zone A11 First zone A12 Second zone N1 Network

Claims

1. An information processing method executed by a computer, comprising: acquiring user information including a user's position; acquiring robot information including the positions of each of one or more robots; identifying one or more candidate robots to which the user may pay attention based on the acquired user information and the acquired robot information; and outputting notification information including at least one of information calling attention to the identified one or more candidate robots, information indicating the position of each of the one or more candidate robots, and information indicating the movement of each of the one or more candidate robots.

2. The information processing method according to claim 1, wherein the information indicating the movement of each of the one or more candidate robots includes at least one of the movement direction and movement path of each of the one or more candidate robots.

3. An information processing method as described in claim 1 or 2, wherein the process of identifying the one or more candidate robots involves determining one or more zones based on the user's position, and identifying robots that exist within the determined one or more zones as candidate robots.

4. An information processing method as described in claim 3, wherein the process of acquiring the robot information further acquires behavioral information indicating the behavior of each of the one or more robots, and the process of identifying the one or more candidate robots further identifies the one or more candidate robots based on the acquired behavioral information.

5. The information processing method of claim 3, wherein the process of acquiring user information further acquires posture information indicating the posture of the user, and the process of identifying the one or more candidate robots further determines the one or more zones based on the acquired posture information.

6. The information processing method of claim 3, wherein the process of acquiring user information further acquires orientation information indicating at least one of the user's facial orientation and gaze direction, and the process of identifying the one or more candidate robots further determines the one or more zones based on the acquired orientation information.

7. An information processing method as described in claim 1 or 2, wherein the one or more robots include an autonomous flying robot, and the process of acquiring the robot information further acquires height information indicating a height of the autonomous flying robot, and the process of identifying the one or more candidate robots further identifies the one or more candidate robots based on the acquired height information.

8. An information processing method according to claim 1 or 2, wherein the one or more robots include a robot having a manipulator, and the process of acquiring the robot information further acquires movable range information indicating a movable range of the manipulator, and the process of identifying the one or more candidate robots further identifies the one or more candidate robots based on the acquired movable range information.

9. The information processing method according to claim 1 or 2, wherein the process of outputting the notification information includes outputting the notification information to smart glasses worn by the user.

10. A program for causing a computer to execute the information processing method according to claim 1 or 2.

11. An information processing system comprising: a first acquisition unit that acquires user information including a user's position; a second acquisition unit that acquires robot information including the positions of each of one or more robots; an identification unit that identifies one or more candidate robots that are candidates for the user to pay attention to based on the user information acquired by the first acquisition unit and the robot information acquired by the second acquisition unit; and an output unit that outputs notification information including at least one of information calling attention to the one or more candidate robots identified by the identification unit, information indicating the position of each of the one or more candidate robots, and information indicating the movement of each of the one or more candidate robots.

12. The information processing system according to claim 11, further comprising a presentation device having a communication unit that receives the notification information output by the output unit, and a presentation unit that presents the notification information received by the communication unit to the user.

Images