Information processing method, information processing device, and program

The information processing method and device address the challenges of robot path planning by mapping and visualizing configuration spaces to determine suitable routes for photographing subjects, ensuring smooth and obstacle-free captures of multiple subjects, enhancing the efficiency and accuracy of robot photography operations.

WO2024232247A1PCT designated stage expired Publication Date: 2024-11-14SONY GROUP CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/015670
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-08
Filing Date
2024-04-22
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing robot path planning technologies face difficulties in efficiently planning routes for photographing subjects while avoiding obstacles and smoothly switching between multiple subjects, especially in three-dimensional spaces, making it challenging for operators to determine suitable routes that meet conditions such as no obstacles between the robot and subject, capturing multiple subjects simultaneously, and smooth subject switching.

Method used

An information processing method and device that maps the configuration space of a robot, calculating and visualizing photographable areas based on subject positions, allowing for the determination of suitable travel routes that satisfy the conditions by displaying a three-dimensional configuration space image, including photographable regions and candidate routes, enabling operators to easily plan routes that capture subjects smoothly and avoid obstacles.

Benefits of technology

The method enables easier and more accurate route planning for robots, allowing them to photograph subjects without obstacles and smoothly switch between multiple subjects, improving the efficiency and effectiveness of the photography process by providing a clear visualization of photographable areas and candidate routes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024015670_14112024_PF_FP_ABST
    Figure JP2024015670_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The present technology relates to an information processing method, an information processing device, and a program whereby a route can be more easily planned. The information processing device comprises an information processing unit that calculates, on the basis of the position of one or each of a plurality of subjects in a task space, a photographable region in which a robot can photograph the subject in a configuration space of the robot that photographs the subject, and displays an image of the configuration space in which the photographable region is displayed. The present technology can be applied to information processing systems.
Need to check novelty before this filing date? Find Prior Art

Description

Information processing method, information processing device, and program

[0001] The present technology relates to an information processing method, an information processing device, and a program, and in particular to an information processing method, an information processing device, and a program that enable easier route planning.

[0002] Many techniques for robot path planning have been proposed in the past, including a technique for quickly determining a path for even an articulated manipulator with many degrees of freedom by generating a subgoal directed graph based on the results of interference detection of geometric models of the robot and the working environment and an evaluation of the distance between subgoals (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2000-20117

[0004] However, when planning a travel path for a robot to photograph a specific subject, it is not easy to plan an appropriate path, as there are various conditions to consider, such as ensuring that obstacles are not captured in the image when photographing, and switching subjects as smoothly as possible.

[0005] The present technology has been developed in light of these circumstances, and aims to make it easier to plan routes.

[0006] An information processing method or program according to one aspect of the present technology includes a step of calculating, based on the positions of one or more subjects in a task space, a captureable area in a configuration space of a robot that captures the subjects, in which the robot can capture the subjects, and displaying an image of the configuration space in which the captureable area is displayed.

[0007] An information processing device according to one aspect of the present technology includes an information processing unit that calculates, based on the positions of one or more subjects in a task space, a captureable area in a configuration space of a robot that captures the subjects, in which the robot can capture the subjects, and displays an image of the configuration space in which the captureable area is displayed.

[0008] In one aspect of the present technology, a captureable area in a configuration space of a robot capturing an image of one or more subjects in a task space is calculated based on the position of the subject or subjects, and an image of the configuration space showing the captureable area is displayed.

[0009] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system. FIG. 2 is a diagram illustrating a task space. FIG. 3 is a diagram illustrating mapping to a configuration space. FIG. 4 is a diagram illustrating an example of a display screen. FIG. 5 is a flowchart illustrating a map generation process and an advance preparation process. FIG. 6 is a flowchart illustrating a travel preparation process and a travel process. FIG. 7 is a flowchart illustrating a travel preparation process and a travel process. FIG. 8 is a diagram illustrating an example of the configuration of a computer.

[0010] Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

[0011] First Embodiment Configuration Example of Information Processing System The present technology is applicable to an information processing system that plans a travel path for a robot having an imaging function when photographing one or more subjects. In the information processing system of the present technology, by mapping an area in which the subject can be photographed onto a configuration space, it becomes possible to more easily plan the path.

[0012] For example, one example of filming using a robot is filming a performance by multiple dancers.

[0013] When taking a photo, there are times when you want to take a picture of just one person as a subject, and times when you want to take a picture of several people. It is also desirable to be able to switch subjects as smoothly and as precisely as possible.

[0014] As a specific example, it is conceivable that the subject may be switched from dancer A to dancer B, or that the state in which dancer A is shown may be switched to a state in which dancer A and dancer B are shown.

[0015] When such photography is performed by a robot, it is necessary to determine whether the following conditions a to c are satisfied when planning the robot's travel route.

[0016] Condition a: No obstacles between the subject and the robot. Condition b: Capture multiple people at once. Condition c: Switch subjects as smoothly as possible.

[0017] For example, when an operator determines the travel rails, that is, the travel path, of the robot, it is difficult for the operator to determine by himself whether the above conditions a to c are satisfied.

[0018] Furthermore, even when a robot autonomously plans and drives a travel route, the robot needs to autonomously determine (decide) a travel route that satisfies the above-mentioned conditions a to c, but making such a determination is difficult.

[0019] Therefore, this technology makes it possible to easily determine a driving route that satisfies the above conditions a to c by mapping information about subjects and obstacles in the configuration space. In other words, it makes it easier to plan a route.

[0020] For example, in the configuration space of a robot (mobile robot) taking pictures, the position and posture of the robot can be treated as points. Note that the following description will be given of an example in which the configuration space of the robot is three-dimensional.

[0021] With this technology, the area in which the subject can be photographed, as well as subjects and obstacles present in the photographing environment, are mapped onto the robot's configuration space.

[0022] This allows the information required for path planning to be visualized in the configuration space, making it possible to present the information required for path planning in an easy-to-understand manner to, for example, the operator (user) of a robot that will be performing the path planning.

[0023] Specifically, for example, by mapping areas where subjects can be photographed, it becomes easier to determine which subjects can be photographed at which locations, and locations to avoid and locations to pass through along the route become clear in three-dimensional space.

[0024] Therefore, by planning the robot's travel path in three dimensions in a configuration space in which areas in which the subject can be photographed are mapped, it is easy to obtain a path that does not capture obstacles and captures the subject smoothly as intended.

[0025] FIG. 1 is a diagram showing an example of the configuration of an embodiment of an information processing system to which the present technology is applied.

[0026] The information processing system 11 shown in FIG. 1 includes an information processing device 21, an environmental sensor 22, and a robot 23.

[0027] In this information processing system 11, an information processing device 21 operated by an operator displays an image of the configuration space of the robot 23 onto which information about subjects and obstacles is mapped, and the travel path of the robot 23 is planned as appropriate.

[0028] In addition, a robot 23 having a photographing function is placed in a space where a subject to be photographed is located (hereinafter referred to as a task space).

[0029] The robot 23 travels within the task space along the travel route obtained by the plan, while acquiring the output of the environmental sensor 22 placed at a position within the task space or at a position near the task space as appropriate, and photographs the subject.

[0030] The information processing device 21 is, for example, a personal computer or a tablet terminal, and executes a program to realize an application related to the travel path planning of the robot 23 .

[0031] The information processing device 21 includes an information input unit 31 , an information acquisition unit 32 , an information processing unit 33 , an information output unit 34 , and an information presentation unit 35 .

[0032] The information input unit 31 is composed of input devices such as a mouse, keyboard, touch panel, etc., and supplies a signal according to an operation by an operator to the information acquisition unit 32 .

[0033] The information acquisition unit 32 has a communication unit that communicates with external devices, and receives signals from the information input unit 31 in response to the operator's operation, or receives information sent from the robot 23, thereby acquiring specified information and supplying it to the information processing unit 33.

[0034] The information processing unit 33 controls the overall operation of the information processing device 21 .

[0035] For example, the information processing unit 33 performs a predetermined information processing based on the information supplied from the information acquisition unit 32, and supplies the resulting image to the information presentation unit 35 for display, or supplies the information obtained by the information processing to the information output unit 34 for transmission to the robot 23.

[0036] The information output unit 34 is, for example, a communication unit that communicates with an external device, and transmits various pieces of information supplied from the information processing unit 33 to the robot 23. The information presentation unit 35 is, for example, a display, and displays images supplied from the information processing unit 33.

[0037] The environmental sensor 22 may be, for example, an RGB camera, an RGB-D camera, or a UWB (Ultra Wide Band) sensor, and detects the position of the subject in the task space and transmits subject position information indicating the detection results to the robot 23.

[0038] For ease of understanding, only one environmental sensor 22 is shown here, but it is also possible to provide a plurality of environmental sensors 22 .

[0039] The robot 23 is a mobile robot that photographs a target subject while traveling in a task space.

[0040] The robot 23 has a wireless communication unit 51 , a sensor 52 , a control unit 53 , a movement mechanism 54 , a camera control mechanism 55 , and a camera 56 .

[0041] The wireless communication unit 51 performs wireless communication with external devices such as the information processing device 21 and the environmental sensor 22. For example, the wireless communication unit 51 receives information transmitted from the information processing device 21 or the environmental sensor 22 and supplies the information to the control unit 53, or transmits information supplied from the control unit 53 to the information processing device 21.

[0042] The sensor 52 is composed of various sensors such as a 2D or 3D LiDAR (Light Detection and Ranging), an RGB-D camera, etc. The sensor 52 detects the positions of subjects and obstacles in the task space and supplies the detection results to the control unit 53.

[0043] The control unit 53 is composed of a computer or the like that controls the overall operation of the robot 23. The control unit 53 has an information acquisition unit 61, an information processing unit 62, and an information output unit 63.

[0044] The information acquisition unit 61 acquires various information by receiving information from the wireless communication unit 51 and acquiring the detection results of subjects and obstacles output from the sensor 52, and supplies the information to the information processing unit 62.

[0045] The information processing unit 62 performs information processing based on the information supplied from the information acquisition unit 61, and appropriately supplies various pieces of information obtained as a result to the information output unit 63. In addition, for example, the control unit 53 operates the movement mechanism 54 and the camera control mechanism 55 according to the results of the information processing in the information processing unit 62.

[0046] The information output unit 63 supplies the information supplied from the information processing unit 62 to the wireless communication unit 51 and controls the transmission of the supplied information to the information processing device 21 .

[0047] The moving mechanism 54 operates under the control of the control unit 53 to move the robot 23 by rotating wheels or the like provided on the robot 23 .

[0048] The camera control mechanism 55 operates under the control of the control unit 53 to control the shooting direction of the camera 56 and the shooting of the subject by the camera 56. The camera 56 functions as a shooting unit that shoots the task space as a shooting target. That is, the camera 56 shoots the subject in the task space under the control of the camera control mechanism 55, and supplies the resulting images (moving images) to the control unit 53 or to a recording unit (not shown) for recording.

[0049] When photographing a subject using the information processing system 11, two possible methods are possible: an operator installs a running rail in the task space and the robot 23 runs along the running rail (hereinafter also referred to as the rail method), and the robot 23 runs autonomously (hereinafter also referred to as the autonomous running method).

[0050] For example, in the rail system, the area in which a subject can be photographed, the subject, obstacles, etc. are mapped onto a configuration space and presented to the operator in the information processing device 21. Hereinafter, an image of the configuration space onto which the area in which a subject can be photographed, etc. is mapped will also be referred to as a configuration space image.

[0051] The configuration space image visualizes the area in which the subject can be photographed, the position of the subject, the positions of obstacles, etc. In other words, it visualizes which subject can be photographed at which position and posture of the robot 23.

[0052] Furthermore, when presenting the configuration space image, the information processing device 21 may determine candidate driving routes for the robot 23 and display the candidates (hereinafter also referred to as candidate routes) on the configuration space image. In other words, candidate routes may be mapped and visualized in the configuration space to suggest driving routes to the operator.

[0053] By looking at the presented configuration space image, the operator can visually grasp the position where the subject can be photographed without any obstacles being captured in the image, and can easily determine (decide) the travel route that will allow the subject to be switched smoothly as intended. After the operator determines the travel route for the robot 23 while checking the configuration space image, he or she installs travel rails in the task space to allow the robot 23 to travel along that travel route.

[0054] Here, a specific example of the configuration space image will be described.

[0055] For example, as shown in Figure 2, suppose that the target subjects A, B, and C, obstacles OB11, OB12, and OB13, robot 23, and traveling rail RA11 are arranged in the task space.

[0056] Here, the horizontal direction in the drawing is the x direction, and the depth direction in the drawing is the y direction, and the position of the robot 23, etc. is expressed by coordinates (x, y) in the xy coordinate space. Also, for example, the traveling rail RA11 is assumed to be temporary.

[0057] Furthermore, the horizontal direction of the camera 56 of the robot 23 in the task space, that is, the photographing direction of the camera 56, is represented by an angle θ.

[0058] The configuration space of the robot 23 is a space whose axes correspond to the degrees of freedom of the robot 23. In particular, in this example, the configuration space is a three-dimensional space made up of an x ​​coordinate (x axis) indicating the position of the robot 23, a y coordinate (y axis) indicating the position of the robot 23, and an angle θ (θ axis) indicating the shooting direction of the robot 23.

[0059] A position (point) in the configuration space represents a position and posture of the robot 23 in the task space. The posture here refers to the angle θ that indicates the shooting direction of the camera 56 on the robot 23.

[0060] If the above-mentioned subjects A to C, obstacles OB11 to OB13, and traveling rail RA11 are mapped onto such a configuration space, it will look like that shown in Figure 3. Note that in Figure 3, parts corresponding to those in Figure 2 are given the same reference numerals, and their explanation will be omitted as appropriate.

[0061] In the configuration space shown in FIG. 3, the horizontal direction, the depth direction, and the vertical direction in the drawing correspond to the x-axis direction, the y-axis direction, and the θ-axis direction, respectively.

[0062] In this example, a self-position mark MK11 indicating the position and posture of the robot 23, photographable areas R11 to R13, and candidate routes RT11 to RT15 are also mapped (superimposed) in the configuration space.

[0063] The self-position mark MK11 is information indicating a position (point) in the configuration space corresponding to the position (x, y) and posture (angle θ) of the robot 23 in the task space at a predetermined timing, such as the current time. In other words, the self-position mark MK11 is displayed at a position in the configuration space determined by the position (x, y) and angle θ of the robot 23 at a predetermined timing.

[0064] For example, the self-position mark MK11 may be any type of mark, such as an icon representing the robot 23 or an image of the robot 23 itself. The operator can grasp the position and posture (photography direction) of the robot 23 by checking the display position of the self-position mark MK11.

[0065] The photographable area R11 is a three-dimensional area (space) that indicates the range of positions and orientations of the robot 23 that can photograph the subject A. In other words, the photographable area R11 is an area in the configuration space where the robot 23 can photograph the subject A.

[0066] Therefore, if the position (point) in the configuration space corresponding to the position and posture of the robot 23 in the task space is within the imageable area R11, the robot 23 can photograph the subject A.

[0067] Similarly, the photographable area R12 is an area indicating the range of positions and postures of the robot 23 capable of photographing subject B, and the photographable area R13 is an area indicating the range of positions and postures of the robot 23 capable of photographing subject C.

[0068] Therefore, for example, if the position and posture of robot 23 correspond to a position (point) within the area where photographable areas R11 to R13 overlap, robot 23 can capture an image that includes subjects A to C as subjects.

[0069] Candidate routes RT11 to RT15 indicate candidate travel routes of the robot 23 in the configuration space when the robot 23 photographs one or more subjects while traveling in the task space, calculated by the information processing device 21. That is, candidate routes RT11 to RT15 indicate candidate travel routes proposed to the operator by the information processing device 21, more specifically, candidate travel routes and candidate postures of the robot 23 at each position on the travel route. Each candidate route is calculated based on, for example, the calculation results of the imageable area, etc.

[0070] The operator can easily understand which subjects will be photographed on each candidate route by visually checking which photographable areas each candidate route passes through.

[0071] For example, when the robot 23 takes pictures while moving along the candidate route RT11, the state where subjects A and B are photographed changes over time to the state where only subject A is photographed. In this case, it can be seen that it is possible to take pictures without a state where none of subjects A to C are absent from the image. In other words, it can be seen that a smooth switching of subjects can be achieved.

[0072] In candidate route RT12, the state where subjects A and B are photographed changes to the state where subjects A, B, and C are photographed, and then the state where subjects B and C are photographed again. In this example, smooth switching of subjects can also be achieved.

[0073] In candidate route RT13, the state changes from one in which subject C is being photographed to one in which subjects B and C are being photographed, and then the state changes again to one in which subject B is being photographed. In this example as well, smooth switching of subjects can be achieved.

[0074] On the other hand, in the candidate route RT14, the state changes from one in which subject A is photographed to one in which none of subjects A to C are photographed, and then the state changes again to one in which subject A is photographed.

[0075] Further, in the candidate route RT15, ​​the state changes from one in which subject B is photographed to one in which none of subjects A to C are photographed, then changes to one in which subject C is photographed, and then changes again to one in which none of subjects A to C are photographed.

[0076] These candidate routes RT14 and RT15 do not allow for smooth switching of subjects, since there are periods when the target subject is not captured, i.e., periods when the image does not include any of the target subjects A to C. Therefore, such candidate routes RT14 and RT15 require appropriate corrections, etc.

[0077] For example, the information presentation unit 35 displays an image of the configuration space in which the subject, obstacles, running rails, self-position mark, photographable area, and candidate routes are mapped (displayed) as a configuration space image, as shown in FIG. 3 .

[0078] In more detail, for example, the configuration space image displays images of models representing the subjects or obstacles, rather than the mapped subjects or obstacles themselves.

[0079] Furthermore, for example, in the configuration space image, the operator can edit (modify) the travel route indicated by the traveling rail RA11 by performing operations on the displayed traveling rail RA11 (configuration space image). In other words, editing (modification) of the traveling rail RA11 is possible. Similarly, editing (modification) of the candidate routes RT11 to RT15 is possible by performing operations on the configuration space image.

[0080] By generating and presenting the above-described configuration space image, the operator can instantly visually grasp the position and posture at which the subject can be photographed, the placement positions of the subject and obstacles, and the travel path of the travel rails. Therefore, the operator can more easily plan the path of the robot 23 by referring to the configuration space image.

[0081] Generally, when creating a traveling rail, that is, when planning a traveling path, the positions of the subject and the robot 23 are visualized on a two-dimensional plane. That is, the path is planned by referring to the positions of the subject and the robot 23 arranged on the two-dimensional plane.

[0082] However, it is difficult to determine the position and posture of the robot 23 that will allow the desired subject to be photographed using only information on the positional relationship between the subject and the robot 23 on a two-dimensional plane.

[0083] In particular, the operator must determine the posture of the robot 23 that will allow it to photograph the desired subject on each occasion, but it is difficult to determine the posture based only on the positional relationship on a two-dimensional plane. Furthermore, when there are many subjects (people) and obstacles, or when determining a travel route other than just one photographing point, it becomes even more difficult to make the determination.

[0084] In contrast, the information processing system 11, for example, by presenting an image of the configuration space shown in Figure 3, makes it possible to plan the travel path of the robot 23 more easily than when planning a path based only on positional relationships on a two-dimensional plane.

[0085] In particular, in the example of Figure 3, the photographable areas R11 to R13 for each subject are three-dimensional areas that take into account not only the x- and y-direction positions of the robot 23 that can photograph the subject, but also the posture (photographing direction) of the robot 23 that can photograph the subject, i.e., the angle θ.

[0086] Therefore, by looking at the photographable areas R11 to R13, the operator can instantly visually grasp the position and posture of the robot 23 that allows it to photograph a desired subject.

[0087] Similar to the traveling rail method described above, in which the operator creates the traveling rails, the autonomous traveling method in which the robot 23 autonomously plans the route also makes it easier to plan the route by using a configuration space image.

[0088] In the autonomous driving method, the robot 23 autonomously makes decisions (route planning) and photographs a subject while autonomously driving in accordance with those decisions.

[0089] In this case, to plan the travel path of the robot 23, information on the configuration space in which the photographable area, etc. is mapped, i.e., the above-mentioned configuration space image, as well as action plan information supplied by the operator, etc., is used.

[0090] The behavioral policy information is information that indicates the policy for photographing and moving (driving), such as which subject you want to photograph and how you should drive (move), i.e., the behavioral policy of the robot 23 regarding photographing and moving when photographing.

[0091] For example, the action plan for photography can be specified by the action plan information as follows: take as many people as possible together, always keep taking pictures of subject A, make sure subject C does not get in the shot (does not appear in the picture), take pictures of subject A in the first half and subject B in the second half.

[0092] Furthermore, for example, as an action policy for movement (running), it is possible to specify by the action policy information, for example, to keep going around the stage (task space) or to go back and forth to the front of the stage.

[0093] When the above-described action plan information is given, the robot 23 autonomously determines a travel route along which it will move (run) and take photographs in accordance with the action plan indicated by the action plan information.

[0094] In either of the rail system and the autonomous driving system described above, it is conceivable to cause the information presentation unit 35 of the information processing device 21 to display a screen including a configuration space image.

[0095] This allows the operator to check the positions of subjects and obstacles, plan routes, and determine and check rail installation positions while looking at the display screen.

[0096] Fig. 4 shows an example of a display screen displayed on the information presenting unit 35. In Fig. 4, the same reference numerals are used to designate parts corresponding to those in Fig. 2 or 3, and the description thereof will be omitted where appropriate.

[0097] On the display screen shown in FIG. 4, a task space image P11, which is an image of the task space shown in FIG. 2, and a configuration space image P12, which is an image of the configuration space shown in FIG. 3, are displayed side by side.

[0098] The task space image P11 is an image of a task space, which is a general three-dimensional space, that is, a space of an xyz coordinate system, in which an object, an obstacle, a traveling rail, and the robot 23 are arranged. In other words, the task space image P11 is an image that visually shows the respective positions of the object, the obstacle, the traveling rail, and the robot 23.

[0099] The configuration space image P12 is an image of the configuration space, which is a space whose axes are the degrees of freedom of the robot 23.

[0100] The configuration space image P12 displays the subjects, obstacles, travel rails, and the self-position mark of the robot 23, as well as the photographable area of ​​each subject and multiple candidate routes.

[0101] The operator can edit (modify) the travel rails and the candidate routes by, for example, operating the information input unit 31 to perform operations on the task space image P11 and the configuration space image P12 on the display screen, particularly operations on the travel rails and the candidate routes. In other words, by editing (modifying) the travel rails and the candidate routes, the operator can change the travel route of the robot 23 indicated by those travel rails and candidate routes.

[0102] On the display screen, a photographed image P13 is displayed on the right side of the configuration space image P12.

[0103] The captured image P13 is an image obtained by capturing the robot 23 in the position and posture indicated by the self-position mark, and is generated by a simulation based on the position of the subject or the like and the position and posture of the robot 23. In other words, the captured image P13 is a simulation image corresponding to an image that would be captured by the robot 23.

[0104] In addition, in the portion indicated by the arrow Q11 below the photographed image P13, candidate routes proposed by the information processing device 21 and check boxes for selecting the candidate routes are displayed side by side. That is, a list of candidate routes is displayed. Note that multiple candidate routes may be displayed, or only one may be displayed.

[0105] In this example, candidate routes RT11 to RT13 are presented as candidate travel routes. A check mark is displayed in check box CB11, which indicates that the candidate route RT11 corresponding to the check box CB11 is selected. Therefore, an image obtained when the robot 23 travels (moves) along candidate route RT11 is displayed as the captured image P13.

[0106] Furthermore, in the portion indicated by the arrow Q12 on the display screen, a timeline indicating the period in which one or more subjects are photographed, which is determined by the information processing section 33 based on the photographable area and the like, is displayed.

[0107] In particular, the display bar BR11 on the timeline indicates the period (time) during which subject A is photographed, and the display bar BR12 indicates the period (time) during which subject B is photographed.

[0108] In this example, candidate route RT11 is selected by check box CB11, and therefore display bars are displayed for subjects A and B that will be captured as subjects (become subjects) when the robot 23 takes a photograph along candidate route RT11.

[0109] The operator can also move the cursor CS11 on the timeline as needed. On the display screen, the task space image P11, the configuration space image P12, and the captured image P13 change according to the position of the cursor CS11, i.e., the time indicated by the cursor CS11.

[0110] Specifically, in the task space, the position of the subject and the position and posture of the robot 23 may change over time. The position of an obstacle may also change over time. Furthermore, if the positions of the subject and obstacles change, the imageable area also changes over time.

[0111] Therefore, in the task space image P11 and the configuration space image P12, the positions of the subject and obstacles, the position and posture of the robot 23 (position of the self-position mark), and the photographable area change depending on the position of the cursor CS11. The photographed image P13 also changes depending on the changes in the task space image P11 and the configuration space image P12.

[0112] The operator can easily grasp how the subject, obstacles, the position and posture of the robot 23, the photographable area, and the photographed image P13 change over time. In particular, by looking at the photographed image P13, the operator can visually grasp what kind of image will actually be photographed.

[0113] <Description of Map Generation Process and Advance Preparation Process> Next, a description will be given of the operation of the information processing system 11. Note that the processes described below are merely examples, and each process may be performed by any device constituting the information processing system 11.

[0114] First, the operation of the rail system will be described.

[0115] In the rail system, the process shown in Fig. 5 is performed before the installation of the traveling rails, and the photographable area is visualized. Hereinafter, the map generation process by the robot 23 and the advance preparation process by the information processing device 21 will be described with reference to the flowchart in Fig. 5.

[0116] The robot 23 starts the map generation process when it is placed in the task space. In this case, it is assumed that at least an obstacle exists in the task space, and the target subject may or may not be placed in the task space.

[0117] In step S11, the information acquisition unit 61 of the robot 23 collects information for generating map information indicating the task space.

[0118] For example, the operator causes the robot 23 to travel in the task space by appropriately operating the information processing device 21. While the robot 23 is traveling in the task space, the information acquisition unit 61 uses the sensor 52, the wireless communication unit 51, and the camera 56 to collect information required for generating map information and supplies the information to the information processing unit 62.

[0119] For example, the information acquisition unit 61 collects information necessary for generating map information, such as the detection results of the positions of obstacles and subjects obtained by the sensor 52, images taken by the camera 56, and the detection results of the positions of subjects received from the environmentally installed sensor 22 by the wireless communication unit 51.

[0120] In step S12, the information processing unit 62 generates map information of the task space based on the information collected in step S11 and supplied from the information acquisition unit 61.

[0121] This allows for map information of the task space, which indicates the shape of the task space and the size and shape of obstacles at their locations in the task space. If a subject is located in the task space, information on the subject's position can also be obtained from the map information.

[0122] The information processing unit 62 supplies the generated map information to the wireless communication unit 51 via the information output unit 63 and instructs it to transmit to the information processing device 21 .

[0123] In step S13 , the wireless communication unit 51 transmits the map information supplied from the information output unit 63 to the information processing device 21 .

[0124] Once the map information is transmitted, the map generation process is completed. Note that even after the map generation process is completed, the robot 23 may continue to travel in the task space as appropriate and transmit information indicating the position of the subject in the task space to the information processing device 21. In this case, the information indicating the position of the subject is obtained (acquired) by, for example, the environmental sensor 22 or the sensor 52.

[0125] When the process of step S13 is performed and the map information is transmitted, the information processing device 21 starts a preparation process.

[0126] In step S 31 , the information acquisition unit 32 receives the map information transmitted from the robot 23 and supplies it to the information processing unit 33 .

[0127] In step S32, the information processing unit 33 reads the obstacle positions in the task space based on the map information supplied from the information acquisition unit 32.

[0128] For example, since the map information includes information about the position of an obstacle in the task space, the information processing unit 33 extracts (reads) information about the position of the obstacle from the map information and sets it as obstacle position information.

[0129] In step S33, the information processing section 33 sets the subject position in the task space.

[0130] For example, if the map information already contains information about the position of the subject, the information processing unit 33 extracts (reads) the information about the position of the subject from the map information and sets it as subject position information.

[0131] In addition, if the map information does not include information regarding the position of the subject, the information processing unit 33 generates subject position information indicating the position of each of the one or more subjects based on information regarding the position of each of the subjects in the task space acquired by the information acquisition unit 32.

[0132] In this case, for example, when the operator operates the information input unit 31 to input the position of each subject, the information acquisition unit 32 acquires, as information about the position of the subject, a signal output from the information input unit 31 in response to the operation. Also, when information about the position of the subject is transmitted from the robot 23, for example, the information acquisition unit 32 receives the information to acquire information about the subject.

[0133] The information processing unit 33 sets the subject position by arranging the subject at the position in the task space indicated by the subject position information obtained in this manner.

[0134] In step S34, the information processing unit 33 reflects the obstacle position and the subject position in the task space in the configuration space.

[0135] That is, the information processing unit 33 maps (places) the obstacles and the subject in the task space onto the configuration space based on the obstacle position information and the subject position information obtained in steps S32 and S33.

[0136] If the task space and the configuration space use different coordinate systems for positions, then mapping is performed after coordinate conversion. In the following, the explanation will continue assuming that the task space and the configuration space use the same coordinate system for positions.

[0137] In step S35, the information processing unit 33 calculates the photographable area of ​​the subject in the configuration space based on the mapping results of the subject and obstacles into the configuration space, in other words, the positions of each subject and each obstacle in the task space.

[0138] That is, in step S35, a three-dimensional area consisting of a position (x, y) in the task space where the robot 23 can photograph the subject and a position (point) in the configuration space corresponding to the posture (angle θ) of the robot 23 at that position is calculated for each subject.

[0139] For example, a position and posture that allows a subject to be photographed is a position and posture that allows an image to be obtained when photographing with camera 56, which includes the desired subject and does not include any obstacles.

[0140] The position and orientation at which the subject can be photographed may be any position and orientation that satisfies predetermined conditions regarding the subject and obstacles, such as not obscuring part or all of the subject in the photographed image.

[0141] By the process of step S35, for example, the imageable areas R11 to R13 shown in FIG. 3 are calculated.

[0142] In step S36, the information processing unit 33 visualizes the imageable area of ​​each subject obtained in step S35.

[0143] Specifically, for example, the information processing unit 33 generates display data (image data) for a display screen that displays an image of the configuration space onto which each subject, obstacle, and the photographable area of ​​each subject are mapped, and supplies the display data to the information presentation unit 35 to display the display screen. This display data, particularly the configuration space image and task space image, is generated based on, for example, the subject positions and obstacle positions in the task space, the calculation results of the photographable area, map information, etc.

[0144] As a result, a screen similar to the display screen shown in Fig. 4 is displayed on the information presentation unit 35. However, on the display screen displayed in step S36, the traveling rails are not displayed on the task space image, and the traveling rails and candidate routes are not displayed on the configuration space image.

[0145] That is, the task space image displays the subject and obstacles, but does not display the traveling rails or the robot 23. Note that the traveling rails and the robot 23 may be displayed on the task space image at temporary positions such as predetermined positions.

[0146] Similarly, the configuration space image displays the subject and obstacles, i.e., the position of the subject and the position of the obstacle, and the photographable area for each subject, but does not display the self-position mark, traveling rails, or candidate routes. Note that the self-position mark and traveling rails may be displayed in temporary positions, such as predetermined positions, on the configuration space image.

[0147] When the configuration space image is displayed and the imageable area is visualized, the advance preparation process is completed.

[0148] In this way, the robot 23 generates map information of the task space and transmits it to the information processing device 21. The information processing device 21 receives the map information from the robot 23, calculates the imageable area, visualizes the imageable area, and presents it to the operator.

[0149] In this way, the operator can visually and instantly grasp which subject can be photographed in what position and posture by looking at the configuration space image, which makes it easier for the operator to plan the travel path of the robot 23.

[0150] <Explanation of Travel Preparation Processing and Travel Processing> After the map generation processing and advance preparation processing shown in Fig. 5 are performed and a display screen including a configuration space image is displayed on the information presentation unit 35, the processing shown in Fig. 6 is then performed. Hereinafter, the travel preparation processing by the information processing device 21 and the travel processing by the robot 23 will be described with reference to the flowchart in Fig. 6.

[0151] When the travel preparation process is started, in step S61, the information processing unit 33 sets the travel rails based on a signal supplied from the information input unit 31 via the information acquisition unit 32 in response to an operation by the operator.

[0152] For example, when the operator causes the information presentation unit 35 to display a display screen including a task space image and a configuration space image, he or she operates the information input unit 31 to operate the display screen and place a temporary traveling rail on the configuration space image. At this time, the display screen may be similar to the display screen shown in FIG. 4, for example, and a self-position mark may also be displayed on the configuration space image on the display screen as appropriate. At this point, the traveling rail is not displayed on the task space image.

[0153] When the operator displays the temporary running rail on the configuration space image, he / she can check the temporary running rail and further operate the information input unit 31 to appropriately change the installation position (placement position) and length of the temporary running rail, i.e., the route of the running rail.

[0154] The information processing unit 33 controls the information presentation unit 35 in response to a signal supplied from the information input unit 31 via the information acquisition unit 32, and sets the running rails by changing the position of the running rails on the configuration space image, etc.

[0155] After the travel rails have been set, in step S62 the information processing unit 33 reflects the settings made in step S61 in the task space.

[0156] That is, the information processing unit 33 maps (places) the traveling rails set in the configuration space in step S61 onto the task space.

[0157] As a result, the traveling rails are displayed on the task space image on the display screen displayed by the information presentation unit 35 at the same positions as on the configuration space image.

[0158] In step S63, the information processing unit 33 proposes a driving route.

[0159] For example, the information processing unit 33 calculates candidate routes based on the positions of subjects and obstacles, the photographable area for each subject, and the positions and routes of the traveling rails. The calculation of candidate routes may use action plan information input by the operator as appropriate.

[0160] After calculating the candidate route, the information processing unit 33 controls the information presentation unit 35 in accordance with the calculation result to update the display on the display screen. That is, the information processing unit 33 suggests a travel route for the robot 23 to the operator by displaying a list of candidate routes on the display screen or by displaying one or more candidate routes on the configuration space image.

[0161] The operator then checks the layout positions of the traveling rails while looking at the display screen, and modifies the layout positions of the traveling rails by operating the information input unit 31. In this case, the operator can select a presented candidate route, modify the selected candidate route as appropriate, and use the modified candidate route as the final traveling route of the robot 23 indicated by the traveling rails.

[0162] In step S64, the information processing unit 33 corrects the traveling rails in response to an input operation by the operator, based on a signal supplied from the information input unit 31 via the information acquisition unit 32. In other words, the final traveling path of the robot 23 is determined in response to an input operation by the operator.

[0163] In step S65, the information processing unit 33 generates travel path information indicating the travel path of the robot 23 along the travel rail based on the finally determined placement position of the travel rail, and supplies the information to the information output unit 34.

[0164] In more detail, the travel route information includes not only information indicating the travel route of the robot 23, but also speed information and posture information (angle θ) for each section of the travel route, such as the speed and posture at which the robot will travel along the travel route.

[0165] In step S66, the information output unit 34 transmits the travel route information supplied from the information processing unit 33 to the robot 23. In addition, actual travel rails are appropriately installed in the task space.

[0166] Then, the robot 23 starts running processing.

[0167] That is, in step S81, the wireless communication unit 51 of the robot 23 receives the travel route information transmitted from the information processing device 21 and supplies the travel route information to the information processing unit 62 via the information acquisition unit 61.

[0168] In step S82, the information processing unit 62 controls the traveling of the robot 23 based on the traveling route information supplied from the information acquisition unit 61.

[0169] Specifically, for example, the information processing unit 62 controls the moving mechanism 54 while identifying the position and posture of the robot 23 based on the driving route information, the map information generated in step S12 of Figure 5, and, as appropriate, information supplied from the information acquisition unit 61, and causes the robot 23 to move.

[0170] As a result, the robot 23 travels along the travel route indicated by the travel route information at the speed and in the direction (angle θ) indicated by the travel route information. At this time, the information processing unit 62 (control unit 53) also controls the camera control mechanism 55 as appropriate based on the travel route information to cause the camera 56 to take a photograph.

[0171] The information supplied from the information acquisition unit 61 during the travel control of the robot 23 includes, for example, the detection result of the subject position by the environmental sensor 22, the detection result of the positions of the subject and obstacles by the sensor 52, and the image (video) captured by the camera 56. The information processing unit 62 estimates (specifies) the position and posture of the robot 23 based on the detection result of the positions of the subject and obstacles, the image of the task space captured by the camera 56, map information, travel route information, etc.

[0172] In addition, the information processing unit 62 generates self-position information indicating the position and posture of the robot 23 at each time while the robot 23 is running, and supplies this to the wireless communication unit 51 via the information output unit 63.

[0173] In step S83, the wireless communication unit 51 transmits the self-position information supplied from the information output unit 63 to the information processing device 21, and the traveling process ends. Note that the transmission of the self-position information is successively performed until the traveling of the robot 23 ends.

[0174] Furthermore, when the self-location information is transmitted, the information processing device 21 performs the process of step S67.

[0175] That is, in step S67, the information acquisition unit 32 receives the self-position information transmitted from the robot 23 in the process of step S83 and supplies it to the information processing unit 33.

[0176] In step S68, the information processing unit 33 reflects the self-position information supplied from the information acquisition unit 32 in the task space image and configuration space image displayed on the information presentation unit 35.

[0177] For example, when the robot 23 starts running, the information processing unit 33 causes the information presenting unit 35 to display a screen similar to the display screen shown in FIG.

[0178] In this case, when self-position information at each time is received from the robot 23, the information processing unit 33 generates a task space image in which the position and posture of the robot 23 are the position and posture indicated by the received self-position information, and displays it on the display screen.

[0179] Similarly, the information processing unit 33 generates a configuration space image and displays it on the display screen so that the self-position mark in the configuration space is displayed at a position corresponding to the position and attitude indicated by the received self-position information.

[0180] As a result, the received self-position information of the robot 23 is reflected in the task space image and the configuration space image.

[0181] The information processing unit 33 also generates a simulated captured image based on a configuration space image reflecting the self-position information, i.e., information on the configuration space in which subjects, obstacles, etc. are mapped, and travel route information, and displays the image on the display screen. This allows the operator to easily understand the images captured by the robot 23. Note that while the robot 23 is traveling, an image actually captured by the robot 23 may be displayed instead of the captured image.

[0182] The information processing unit 33 continues to perform the process of step S68 until the robot 23 has finished traveling, and when the robot 23 has finished traveling, the traveling preparation process ends.

[0183] In this way, the information processing device 21 displays a display screen including a task space image and a configuration space image, sets travel rails and generates travel route information in response to the operator's operation, and reflects the self-position information of the robot 23 while traveling on the display screen. Furthermore, when the robot 23 receives travel route information from the information processing device 21, it travels based on the travel route information and transmits the self-position information to the information processing device 21.

[0184] In this way, by presenting the configuration space image to the operator, the operator can more easily plan the travel path of the robot 23.

[0185] Incidentally, even in the autonomous driving method in which the robot 23 autonomously plans a route, the information processing system 11 performs the map generation process and advance preparation process described with reference to FIG.

[0186] In the autonomous driving method, once the advance preparation process is performed, the information processing system 11 then performs the driving preparation process and driving process shown in FIG.

[0187] Hereinafter, the travel preparation process by the information processing device 21 and the travel process by the robot 23 will be described with reference to the flowchart of FIG.

[0188] When the travel preparation process is started in the information processing device 21, in step S111, the information processing unit 33 sets a course of action based on a signal supplied from the information input unit 31 via the information acquisition unit 32 in response to the operator's operation.

[0189] For example, the information presentation unit 35 displays a display screen including a task space image and a configuration space image. In this case, the task space image and the configuration space image do not display the travel rails. Furthermore, the configuration space image does not display any candidate routes.

[0190] The operator operates the information input unit 31 while viewing the display screen as appropriate to input the action plan of the robot 23 during photography, such as which subject the operator wants to photograph, how the robot should move, etc. The information processing unit 33 then generates action plan information indicating the action plan input by the operator in response to a signal supplied from the information input unit 31 via the information acquisition unit 32.

[0191] Furthermore, the information processing unit 33 generates configuration space information indicating the positions of subjects and obstacles in the configuration space and the imageable area based on the configuration space image, etc. The configuration space information is information about the configuration space, and at least includes information indicating the imageable area.

[0192] The information processing unit 33 supplies the action course information and configuration space information to the information output unit 34 and instructs it to be transmitted to the robot 23 .

[0193] In step S112, the information output unit 34 transmits the action course information and configuration space information supplied from the information processing unit 33 to the robot 23.

[0194] When the action course information and configuration space information are transmitted, the robot 23 starts running processing and performs the processing of step S131.

[0195] In step S131 , the wireless communication unit 51 receives the action policy information and configuration space information transmitted from the information processing device 21 , and supplies the action policy information and configuration space information to the information processing unit 62 via the information acquisition unit 61 .

[0196] In step S132 , the information processing unit 62 controls the running of the robot 23 based on the course of action information and configuration space information supplied from the information acquisition unit 61 .

[0197] Specifically, for example, the information processing unit 62 controls the moving mechanism 54 based on the behavioral policy information and configuration space information, the map information generated in step S12 of Figure 5, and, as appropriate, information supplied from the information acquisition unit 61, and causes the robot 23 to move autonomously.

[0198] At this time, the information that the information processing unit 62 receives from the information acquisition unit 61 may be, for example, the detection result of the subject position by the environmentally installed sensor 22, the detection result of the position of the subject or obstacle by the sensor 52, or an image (video) obtained by photographing with the camera 56.

[0199] The information processing unit 62 identifies (estimates) the position and posture of the robot 23 based on the action policy information, configuration space information, map information, etc., and autonomously determines a travel route in accordance with the action policy indicated by the action policy information.

[0200] The information processing unit 62 controls the movement mechanism 54 to make the robot 23 travel along the determined travel route, and also controls the camera control mechanism 55 as appropriate to take pictures with the camera 56. As a result, the robot 23 travels and takes pictures while autonomously determining its own travel route.

[0201] In addition, the information processing unit 62 generates self-position information indicating the position and posture of the robot 23 at each time while it is running, as in step S82 of Figure 6, and supplies the self-position information to the wireless communication unit 51 via the information output unit 63.

[0202] In step S133, the wireless communication unit 51 transmits the self-position information supplied from the information output unit 63 to the information processing device 21, and the traveling process ends. Note that the transmission of the self-position information is successively performed until the traveling of the robot 23 ends.

[0203] After the process of step S133 is performed, the information processing device 21 then performs the processes of steps S113 and S114, and the travel preparation process ends. Note that the processes of steps S113 and S114 are similar to the processes of steps S67 and S68 in Figure 6, and therefore a description thereof will be omitted.

[0204] In this manner, the information processing device 21 transmits the action course information and the configuration space information to the robot 23. The robot 23 also autonomously determines a travel route based on the action course information and the configuration space information, and travels along the route.

[0205] In this way, by supplying the action course information and configuration space information to the robot 23, the robot 23 can autonomously determine an appropriate travel route more easily. In other words, it is possible to more easily plan a travel route.

[0206] <Example of Computer Configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, that can execute various functions by installing various programs.

[0207] FIG. 8 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0208] In the computer, a CPU (Central Processing Unit) 501 , a ROM (Read Only Memory) 502 , and a RAM (Random Access Memory) 503 are interconnected by a bus 504 .

[0209] An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0210] The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, etc. The output unit 507 includes a display, a speaker, etc. The recording unit 508 includes a hard disk, a non-volatile memory, etc. The communication unit 509 includes a network interface, etc. The drive 510 drives a removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0211] In a computer configured as described above, the CPU 501 loads a program recorded in the recording unit 508, for example, into the RAM 503 via the input / output interface 505 and the bus 504, and executes the program, thereby performing the above-described series of processes.

[0212] The program executed by the computer (CPU 501) can be provided by being recorded on a removable recording medium 511 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0213] In a computer, a program can be installed in the recording unit 508 via the input / output interface 505 by inserting a removable recording medium 511 into the drive 510. The program can also be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. Alternatively, the program can be installed in the ROM 502 or the recording unit 508 in advance.

[0214] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0215] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0216] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0217] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0218] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0219] Furthermore, the present technology can also be configured as follows.

[0220] (1) An information processing method including the steps of: an information processing system calculating, based on the positions of one or more objects in a task space, a captureable area in a configuration space of a robot that captures the objects, where the robot can capture the objects; and displaying an image of the configuration space in which the captureable area is displayed. (2) The information processing method described in (1), in which the captureable area is calculated for each of the objects. (3) The information processing method described in (1) or (2), in which an image of the configuration space in which the position of the object and the captureable area are displayed is displayed. (4) The information processing method described in any one of (1) to (3), in which an image of the configuration space in which the position of an obstacle in the task space and the captureable area are displayed is displayed. (5) The information processing method according to any one of (1) to (4), further comprising: calculating, based on the imageable area, candidate paths for the robot in the configuration space when the robot photographs one or more of the subjects while traveling in the task space; and displaying an image of the configuration space in which the candidate paths and the imageable area are displayed. (6) The information processing method according to (5), further comprising: displaying a list of one or more of the candidates. (7) The information processing method according to (5) or (6), in which the candidates can be modified by operating the image. (8) The information processing method according to any one of (1) to (7), in which an image of the configuration space in which the imageable area and information indicating a position in the configuration space corresponding to the position and posture of the robot at a predetermined timing are displayed. (9) The information processing method according to any one of (1) to (8), in which an image of the configuration space in which the traveling rails of the robot and the imageable area are displayed is displayed. (10) The information processing method according to (9), wherein the traveling rail can be corrected by operating the image. (11) The information processing method according to any one of (1) to (10), wherein an image of the task space in which the subject is placed is further displayed.(12) The information processing method according to any one of (1) to (11), further generating, by simulation, images to be captured by the robot based on a position and posture of the robot, and further displaying the images generated by the simulation. (13) The information processing method according to any one of (1) to (12), further displaying, based on the captureable area, a timeline indicating a section in which one or more of the subjects will be captured. (14) The information processing method according to any one of (1) to (13), further determining a travel path for the robot in accordance with an operation by an operator, and transmitting travel path information indicating the determined travel path to the robot. (15) The information processing method according to any one of (1) to (13), further generating, in accordance with an operation by an operator, action policy information indicating a course of action for the robot regarding image capture and travel, and transmitting the action policy information and information regarding the configuration space including information indicating the captureable area to the robot. (16) An information processing device comprising an information processing unit that calculates, based on the positions of one or more subjects in a task space, a photographable area in a configuration space of a robot that photographs the subject where the robot can photograph the subject, and displays an image of the configuration space in which the photographable area is displayed. (17) A program that causes a computer to execute processing including the steps of: calculating, based on the positions of one or more subjects in a task space, a photographable area in a configuration space of a robot that photographs the subject where the robot can photograph the subject, and displaying an image of the configuration space in which the photographable area is displayed.

[0221] REFERENCE SIGNS LIST 11 Information processing system, 21 Information processing device, 22 Environmental sensor, 23 Robot, 31 Information input unit, 32 Information acquisition unit, 33 Information processing unit, 34 Information output unit, 35 Information presentation unit, 51 Wireless communication unit, 52 Sensor, 53 Control unit, 54 Movement mechanism, 55 Camera control mechanism, 56 Camera, 61 Information acquisition unit, 62 Information processing unit, 63 Information output unit

Claims

1. An information processing method including the steps of: an information processing system calculating, based on the positions of one or more objects in a task space, a captureable area in a configuration space of a robot that captures the objects, in which the robot can capture the objects; and displaying an image of the configuration space in which the captureable area is displayed.

2. The information processing method according to claim 1, further comprising the step of: calculating the photographable area for each of the subjects.

3. The information processing method according to claim 1, further comprising the step of displaying an image of the configuration space in which the position of the subject and the imageable area are displayed.

4. The information processing method according to claim 1, further comprising the step of displaying an image of the configuration space in which the positions of obstacles in the task space and the imageable area are displayed.

5. The information processing method according to claim 1, further comprising: calculating, based on the photographable area, potential paths for the robot in the configuration space when the robot photographs one or more of the subjects while traveling in the task space; and displaying an image of the configuration space showing the potential paths and the photographable area.

6. The information processing method according to claim 5, further comprising the step of displaying a list of one or more of the candidates.

7. The information processing method according to claim 5, wherein the candidate can be modified by performing an operation on the image.

8. The information processing method according to claim 1, further comprising displaying information indicating a position in the configuration space corresponding to the position and posture of the robot at a specified timing, and an image of the configuration space showing the photographable area.

9. The information processing method according to claim 1, further comprising the step of displaying an image of the configuration space in which the travel rail of the robot and the photographable area are displayed.

10. The information processing method according to claim 9, wherein the traveling rail can be modified by manipulating the image.

11. The information processing method according to claim 1, further comprising the step of displaying an image of the task space in which the subject is located.

12. The information processing method according to claim 1, further comprising generating an image captured by the robot through a simulation based on the position and posture of the robot, and further displaying the image generated through the simulation.

13. The information processing method according to claim 1, further comprising displaying a timeline indicating a period during which one or more of said subjects are photographed based on said photographable area.

14. The information processing method according to claim 1, further comprising the steps of: determining a travel route for the robot in response to an operation by an operator; and transmitting travel route information indicating the determined travel route to the robot.

15. The information processing method according to claim 1, further comprising the steps of: generating action policy information indicating the robot's action policy regarding photographing and traveling in response to an operation by an operator; and transmitting the action policy information and information regarding the configuration space, including information indicating the photographable area, to the robot.

16. An information processing device comprising an information processing unit that calculates a captureable area in a configuration space of a robot that captures one or more subjects based on the positions of the subjects in a task space, and displays an image of the configuration space in which the captureable area is displayed.

17. A program for causing a computer to execute a process including the steps of: calculating a captureable area in a configuration space of a robot that captures one or more subjects based on the positions of the subjects in a task space; and displaying an image of the configuration space in which the captureable area is displayed.

Citation Information

Patent Citations

  • Device and method for controlling photographing

    JP2007214803A

  • Controller, control method and computer program

    JP2016082441A

  • Method for photographing an unmanned aerial robot and a device for supporting the same in an unmanned aerial vehicle system

    US20200036886A1

  • Mobile device, image capturing system, and mobile device control method

    WO2022054655A1