Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, an information processing method, and a program which make it easy to set a highly reliable entry-prohibited region. An information processing device according to one aspect of the present technology sets a parameter for each region to be used for motion planning of a moving body, on the basis of the cognitive status of a moving person, estimated from human flow data. The present technology can be applied to information processing devices which control autonomously moving robots.

Description

[Technical field]

[0001] In particular, the present technology relates to an information processing device, an information processing method, and a program that enable easy setting of a highly reliable no-entry area. [Background technology]

[0002] In a space where an autonomous mobile robot is introduced, no-entry areas are set up for the following reasons: - We want to be able to plan efficient and safe trajectories taking into account known fixed obstacles such as walls. - Based on user requirements, the system administrator sets areas where the robot should not enter. To prevent malfunctions, areas near obstacles that are difficult for sensors to detect are designated as no-entry areas. - As for spatial rules, there are areas where you do not want robots to enter.

[0003] Setting appropriate no-entry areas individually is a tedious and complicated task, so various techniques have been proposed for automatically setting no-entry areas.

[0004] For example, Patent Document 1 discloses a technology that measures the positions of people for a certain period of time and sets areas where people cannot enter as no-entry areas. In this technology, after setting all areas as no-entry areas, a process is performed to lower the potential of areas where people have been present and remove them from the no-entry areas. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2017-90796 A Summary of the Invention [Problem to be solved by the invention]

[0006] In the case of the technology described in Patent Document 1, a large amount of walking data is required to set a reliable no-entry area. In addition, when people are distributed sparsely, only the area near the center of the space tends to be treated as a passable area. Furthermore, an area where a person has mistakenly entered may be set as a passable area.

[0007] The present technology has been made in consideration of such circumstances, and makes it possible to easily set no-entry areas with high reliability. [Means for solving the problem]

[0008] According to one aspect of the present technology, there is provided an information processing device for performing a motion plan for a moving object moving in a space after a person moves based on people flow data including a trajectory of each of the people moving in the space. , which indicates that the area where at least one person has not entered is a no-entry area. A parameter setting unit is provided for setting parameters for each region.

[0009] In one aspect of the present technology, a method for planning the behavior of a moving object that moves through a space after a person moves based on people flow data including the trajectories of the respective movements of a plurality of people moving through the space is provided. , which indicates that the area where at least one person has not entered is a no-entry area. The parameters for each region are set. [Brief description of the drawings]

[0010] [Figure 1] 1A and 1B are diagrams illustrating a usage state of a mobile robot according to an embodiment of the present technology. [Diagram 2] FIG. 1 is a diagram illustrating the relationship between a velocity vector and a person's cognitive state. [Diagram 3] FIG. 13 is a diagram illustrating the relationship between an acceleration vector and a person's cognitive state. [Figure 4] FIG. 1 is a diagram illustrating a relationship between a jerk vector and a person's cognitive state. [Diagram 5] FIG. 2 is a diagram showing an overall flow of processing performed by a control device and a mobile robot. [Figure 6] FIG. 2 is a block diagram showing an example of the hardware configuration of a control device. [Figure 7]2 is a block diagram showing an example of a functional configuration of a control device; [Figure 8] FIG. 11 is a diagram illustrating an example of people flow data. [Figure 9] FIG. 13 is a diagram showing a process flow of the no-entry area setting phase. [Figure 10] FIG. 13 is a diagram illustrating an example of initialization. [Figure 11] FIG. 13 is a diagram illustrating an example of updating a potential based on a velocity vector. [Figure 12] FIG. 13 is a diagram illustrating an example of updating a potential based on an acceleration vector. [Figure 13] FIG. 13 is a diagram illustrating an example of updating a potential based on a jerk vector. [Figure 14] FIG. 1 is a block diagram showing an example of the configuration of a mobile robot. [Figure 15] FIG. 11 is a diagram illustrating an example of people flow data. [Figure 16] FIG. 1 is a plan view showing an example of a space. [Figure 17] FIG. 13 is a diagram showing an example of bone shape estimation in a three-dimensional space. [Figure 18] FIG. 13 is a block diagram showing another configuration example of the information processing system. [Figure 19] FIG. 13 is a block diagram showing another configuration example of the information processing system. [Figure 20] FIG. 13 is a block diagram showing another configuration example of the information processing system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present technology will be described in the following order. 1. Overview of No-Entry Area Settings 2. Example of each device configuration 3. First embodiment: Basic form 4. Second embodiment: Response to cases where potential updates are incorrect 5. Third embodiment: Setting weight parameters according to the environment 6. Fourth embodiment: Automatic adjustment of weighting parameters using additional information 7. Fifth embodiment: Three-dimensional correspondence by considering human posture 8.Other

[0012] <Overview of setting no-entry areas> FIG. 1 is a diagram showing a usage state of a mobile robot according to an embodiment of the present technology.

[0013] A mobile robot according to an embodiment of the present technology is used, for example, indoors. The space in which the mobile robot is introduced is a space in which people can walk through.

[0014] In the example of Fig. 1, mobile robots 1-1 to 1-3 are shown. When there is no need to distinguish between the mobile robots, they will be collectively referred to as mobile robot 1. The mobile robot 1 has a cubic housing, and the bottom surface of the mobile robot 1 is provided with components such as tires used for the movement of the mobile robot 1.

[0015] The mobile robot 1 is an autonomous robot that has the ability to search for people in a room based on images captured by a camera, and approach people it detects through the search to serve them or communicate with them.

[0016] No-entry areas are set in the space in which the mobile robot 1 moves. For example, areas around obstacles are set as no-entry areas. Areas other than the no-entry areas are passable areas.

[0017] The no-entry areas are set, for example, by a control device, which is a higher-level device that manages the movement of the mobile robot 1. In the mobile robot 1, an action plan is made for moving while avoiding the no-entry areas set by the control device, and the robot moves according to the planned trajectory (route).

[0018] The behavior plan of the mobile robot 1 is performed, for example, by the potential method. The potential method is a method in which an attractive potential is set at the destination and a repulsive potential is set in an area where an obstacle exists, and the robot is moved along the negative gradient of the potential function.

[0019] The control device sets the no-entry areas based on the estimation result of the cognitive state of the person by estimating the cognitive state of the person for each area of ​​the space based on the behavior of the person. The cognitive state of the person is estimated based on people flow data measured in advance, for example by tracking each of many people moving in the space. The behavior of the person used to estimate the cognitive state of the person is the movement of the person.

[0020] As described later, the people flow data includes information such as the trajectory, speed, acceleration, and jerk of people's movement. The people flow data is generated, for example, by analyzing images taken by a camera installed near the ceiling of the space in which the mobile robot 1 moves. The jerk is the rate of change in acceleration over time.

[0021] The control device calculates the velocity vector, acceleration vector, and jerk vector of each person based on the people flow data when setting the no-entry area. The control device estimates the cognitive state of the person based on the calculated information, and sets the no-entry area based on the estimation result.

[0022] A person's cognitive state is expressed as follows using three pieces of information: a velocity vector, an acceleration vector, and a jerk vector. The relationship between a person's cognitive state and each of the velocity vector, acceleration vector, and jerk vector is defined as a policy used to set the potential.

[0023] FIG. 2 is a diagram showing the relationship between the velocity vector and a person's cognitive state.

[0024] A person who recognizes an area that may be a no-entry area rarely moves in the normal direction to the no-entry area. In other words, a person who recognizes a no-entry area moves in a direction different from the normal direction of the no-entry area, as shown by the arrow in Figure 2. Using the velocity vector, it can be estimated that the person's cognitive state is in a state where they recognize a no-entry area that is in a direction different from the direction of the velocity vector.

[0025] FIG. 3 is a diagram showing the relationship between the acceleration vector and a person's cognitive state.

[0026] A person will decelerate when approaching a recognized no-entry area. That is, a person moving toward a no-entry area as shown in the upper part of Fig. 3 will decelerate when he recognizes it as shown in the lower part of Fig. 3. From the acceleration vector when decelerating, it can be estimated that the person's cognitive state is in a state where he recognizes a no-entry area in his / her traveling direction.

[0027] FIG. 4 is a diagram showing the relationship between the jerk vector and a person's cognitive state.

[0028] If a person recognizes a no-entry area after approaching it, he or she will come to an abrupt stop, as shown in the lower part of Figure 4. If the jerk vector in the direction opposite to the direction of travel (the direction of stopping) is large, it is estimated that the person's cognitive state is one in which the person has recognized a no-entry area in the direction of travel just before.

[0029] The control device sets the no-entry area based on three pieces of information that are thought to represent a person's cognitive state: velocity vector, acceleration vector, and jerk vector.

[0030] FIG. 5 is a diagram showing the overall flow of the processing performed by the control device and the mobile robot 1, including the phase of setting the no-entry area.

[0031] The overall processing performed by the control device and the mobile robot 1 includes processing in each of the following phases: a people flow measurement phase, a no-entry area setting phase, and a traveling phase.

[0032] The people flow measurement phase is a phase in which people are tracked and their trajectories, speeds, and accelerations, which constitute people flow data, are recorded.

[0033] The no-entry area setting phase is a phase in which a no-entry area map is generated based on people flow data, assuming the above-mentioned human recognition. The no-entry area map contains information on the potential of each area in the space. Areas with high potential become no-entry areas.

[0034] The driving phase is a phase in which an action plan is made to avoid the no-entry areas based on the no-entry area map, and the vehicle drives (moves) along the planned trajectory.

[0035] For example, the people flow measurement phase and the no-entry area setting phase are phases that the control device is in charge of, and the mobile robot 1 is in charge of the traveling phase.

[0036] The processing of the people flow measurement phase may be performed by a device different from the control device, or may be performed by the mobile robot 1. The processing of all phases, including the people flow measurement phase, the no-entry area setting phase, and the traveling phase, may be performed by the mobile robot 1.

[0037] The processing of one phase may be shared among multiple devices. In this way, the devices that perform the processing of each phase can be changed as appropriate. An example of sharing the processing of each phase will be described later.

[0038] In this way, by using the estimation results of a person's cognitive state, it is possible to easily set no-entry areas with high reliability.

[0039] For example, if no-entry areas were to be set by focusing only on the positions of people represented by walking data, there is a possibility that only areas near the center of the space, where people often walk, would be set as passable areas, but it is possible to prevent this from happening.

[0040] It is also possible to prevent an area that a person has entered by mistake from being set as a passable area.

[0041] Furthermore, because the estimation results of a person's cognitive state are used, it is possible to set areas containing objects that cannot be detected by sensors, etc. as no-entry areas.

[0042] It is possible to set no-entry areas according to rules determined by people based on social backgrounds, etc. Even if the mobile robot 1 can travel there may be places that people would not approach. By using the estimation result of the person's cognitive state, it is possible to set such places that people should not approach as no-entry areas for the mobile robot 1.

[0043] Even if the shape of the target obstacle does not change, such as when an object on a shelf changes or the state of an obstacle changes, there are cases where you want to change whether the area is a no-entry area or a passable area depending on the change in state. It is possible to change such settings based on the person's cognitive state.

[0044] <Example of each device configuration> FIG. 6 is a block diagram showing an example of the hardware configuration of the control device 11 that manages the movement of the mobile robot 1. As shown in FIG.

[0045] 6, the control device 11 is configured by a computer. The control device 11 and each mobile robot 1 are connected via a network such as a wireless LAN or the Internet.

[0046] A central processing unit (CPU) 21, a read only memory (ROM) 22, and a random access memory (RAM) 23 are interconnected by a bus 24.

[0047] An input / output interface 25 is further connected to the bus 24. An input unit 26 including a keyboard, a mouse, etc., and an output unit 27 including a display, a speaker, etc. are connected to the input / output interface 25. In addition, a storage unit 28 including a hard disk or non-volatile memory, a communication unit 29 including a network interface, etc., and a drive 30 that drives a removable medium 31 are connected to the input / output interface 25.

[0048] In a computer configured as described above, the CPU 21 performs processing including setting the above-mentioned no-entry areas by, for example, loading a program stored in the storage unit 28 into the RAM 23 via the input / output interface 25 and the bus 24 and executing the program.

[0049] 7 is a block diagram showing an example of a functional configuration of the control device 11. At least a part of the functional units shown in FIG. 7 is realized by the CPU 21 in FIG.

[0050] In the control device 11, a people flow measurement unit 51, a people flow data storage unit 52, an entry-prohibited area setting unit 53, and an entry-prohibited area map storage unit 54 are realized.

[0051] The people flow measurement unit 51 acquires images captured by a camera installed in the space in which the mobile robot 1 is introduced. The images captured by the camera show people moving within the space. The people flow measurement unit 51 analyzes the acquired images and generates people flow data by tracking people moving within the space. The people flow data may be generated based on sensor data detected by sensors other than cameras, such as a human presence sensor or a ToF (Time of Flight) sensor.

[0052] FIG. 8 is a diagram showing an example of people flow data.

[0053] People flow data is generated for each person moving in a space. As shown in Fig. 8, the people flow data includes an ID, a trajectory, a speed component, an acceleration component, and a jerk component.

[0054] The ID is the identification information of a person moving within a space.

[0055] A trajectory represents the path of a person's movement. For example, the trajectory is represented by time-series data of positions in a space.

[0056] Velocity components represent the speed of a person moving through a space. Using the X-axis and Y-axis directions set in the space as references, the velocity components represent the speed in each direction (x, y).

[0057] Acceleration components represent the acceleration of a person moving within a space. Using the X-axis and Y-axis directions set within the space as references, the acceleration of each directional component (x, y) is represented by the acceleration components.

[0058] The jerk component represents the jerk of a person moving within a space. Using the X-axis and Y-axis directions set within the space as references, the jerk of each direction (x, y) is represented by the jerk component.

[0059] Other information used to set the no-entry area may be generated by the people flow measurement unit 51 and included in the people flow data. Acceleration and jerk may not be included in the people flow data, and may be calculated based on the speed included in the people flow data.

[0060] The people flow measurement unit 51 outputs people flow data including such information to the people flow data storage unit 52 (FIG. 7) for storage. A people flow data DB is constructed in the people flow data storage unit 52. The process of constructing the people flow data DB is performed as a process in the people flow measurement phase.

[0061] The no-entry area setting unit 53 estimates the cognitive state of people as described above based on the people flow data stored in the people flow data storage unit 52. The no-entry area setting unit 53 sets no-entry areas based on the estimation results of the cognitive state of people, and generates a no-entry area map.

[0062] FIG. 9 is a diagram showing the flow of processing in the forbidden area setting phase performed by the forbidden area setting unit 53. As shown in FIG.

[0063] The processing in the no-entry area setting phase includes the processes of initialization, updating of the potential based on the velocity vector, updating of the potential based on the acceleration vector, and updating of the potential based on the jerk vector.

[0064] FIG. 10 is a diagram illustrating an example of the initialization.

[0065] At the time of initialization, the no-entry area setting unit 53 sets the potential of all areas in the space, areas #1 to #9, to the maximum. In the state after initialization, all areas #1 to #9 are set as no-entry areas. When the mobile robot 1 plans its behavior, the behavior is planned so that the robot moves through areas with low potential.

[0066] In FIG. 10, the entire space is made up of nine areas, areas #1 to #9, but the shape of the no-entry area map and the number of areas that make up the no-entry area map will differ depending on the shape and size of the space. The color intensity of each area represents potential. Areas with high color intensity have high potential. The same is true for FIG. 11 and subsequent figures.

[0067] FIG. 11 is a diagram illustrating an example of updating the potential based on the velocity vector.

[0068] As described with reference to Fig. 2, a person who recognizes an area that may be a no-entry area rarely moves in the normal direction to the no-entry area. For this reason, the potential based on the velocity vector is updated so that the potential of the area in the component directions (X-axis direction, Y-axis direction) of the velocity vector is reduced in proportion to the vector length.

[0069] For example, assume that there is people flow data representing the movement of a person from area #8 to area #5, and then from area #5 to area #1, as shown in the lower part of Fig. 11. The person whose movement state is represented by this people flow data may, for example, suddenly stop when moving from area #8 to area #5, change direction, and move from area #5 to area #1. The velocity vector in area #5 measured by such a movement is represented by arrow A1.

[0070] In this case, the potentials of regions #2 and #4 in the component direction of the velocity vector represented by arrow A1 are updated so as to decrease in proportion to the vector lengths in the Y-axis and X-axis directions, respectively. In the upper part of Fig. 11, the densities of regions #2 and #4 enclosed in bold white frames are lower than those shown in Fig. 10, which indicates that the potentials of the respective regions have been decreased and updated.

[0071] A weighting parameter α, which represents the contribution rate of velocity, is used in updating the potential based on the velocity vector. The degree of updating the potential of region #2 and region #4 is calculated based on the vector length and the weighting parameter α.

[0072] The update of the potential based on the acceleration vector and the update of the potential based on the jerk vector also use a weighting parameter β representing the contribution rate of the acceleration and a weighting parameter γ representing the contribution rate of the jerk, respectively. Predetermined values ​​are set as the weighting parameters α, β, and γ.

[0073] The potentials of regions #1, #5, and #8 are updated to the minimum potentials in proportion to the vector length of the velocity vector, for example. In the upper part of Fig. 11, regions #1, #5, and #8 are shown in white, which indicates that the minimum values ​​are set as the potentials of those regions.

[0074] The update based on the velocity vector is followed by an update of the potential based on the acceleration vector.

[0075] FIG. 12 is a diagram illustrating an example of updating the potential based on the acceleration vector.

[0076] As described with reference to Fig. 3, a person decelerates when approaching a recognized no-entry area. For this reason, the potential based on the acceleration vector is updated so as to reduce the potential of the area in the component direction of the acceleration vector in proportion to the vector length.

[0077] The acceleration vector in area #5 measured by the above-mentioned movement is represented, for example, by arrow A2 in the lower part of FIG.

[0078] In this case, the potential of region #4 in the component direction of the acceleration vector represented by the arrow A2 is updated so as to decrease, for example, in proportion to the vector length. The degree of update of the potential of region #4 is calculated based on the vector length and weight parameter β. In the upper part of Fig. 12, the density of region #4 surrounded by a bold white frame is lower than the density shown in Fig. 11, which indicates that the potential of region #4 has been decreased and updated.

[0079] Following the update based on the acceleration vector, the potential is updated based on the jerk vector.

[0080] FIG. 13 is a diagram showing an example of updating the potential based on the jerk vector.

[0081] As described with reference to Fig. 4, when a person approaches a no-entry area and then recognizes the no-entry area, the person stops suddenly. For this reason, the potential based on the jerk vector is updated so as to increase the potential of the area in the moving direction (the opposite direction to the direction of the jerk vector).

[0082] The direction of the jerk vector in area #5 measured by the above-mentioned movement is represented, for example, by arrow A3 in the lower part of FIG.

[0083] In this case, the potential of region #2 in the traveling direction immediately before the sudden stop is updated so as to increase, for example, according to the vector length. The degree of updating the potential of region #2 is calculated based on the weight parameter γ. In the upper part of Fig. 13, the density of region #2 enclosed in a thick white frame is higher than the density shown in Fig. 12, which indicates that the potential of region #2 has been increased and updated.

[0084] The potential of each area in the no-entry area map is updated as described above based on the speed vector, acceleration vector, and jerk vector represented by the people flow data. The no-entry area setting unit 53 in Fig. 7 functions as a parameter setting unit that sets the potential, which is a parameter of each area used in the action plan of the mobile robot 1, based on the cognitive state of a moving person estimated from the people flow data.

[0085] The forbidden area map generated by the forbidden area setting unit 53 by setting the potential of each area is output to and stored in the forbidden area map storage unit 54. As shown in the balloon in Fig. 7, the forbidden area map is a grid map in which a potential gradient reflecting information about obstacles is set in each grid (area) that constitutes the space.

[0086] The no-entry area map stored in the no-entry area map storage unit 54 is transmitted to each mobile robot 1 and used for action planning.

[0087] Mobile Robot 1 Configuration FIG. 14 is a block diagram showing an example of the configuration of the mobile robot 1. As shown in FIG.

[0088] The mobile robot 1 is composed of a control unit 101, a moving unit 102, a camera 103, a sensor 104, a communication unit 105, and a power supply unit 106.

[0089] The control unit 101 is configured by a computer. The control unit 101 executes a predetermined program by a CPU and controls the overall operation of the mobile robot 1.

[0090] The moving unit 102 rotates the tires by driving motors and gears under the control of the control unit 101, and realizes the movement of the mobile robot 1 while adjusting the speed and direction.

[0091] The camera 103 is composed of an RGB camera that captures an RGB image, an IR camera that captures an IR image, etc. The image captured by the camera 103 is output to the control unit 101.

[0092] The sensor 104 is configured with various sensors such as an acceleration sensor, a gyro sensor, a human sensor, an encoder that detects the amount of rotation of a tire provided on the moving unit 102, and a LiDAR. Information indicating the sensing result by the sensor 104 is output to the control unit 101.

[0093] At least one of the camera 103 and the sensor 104 may be provided outside the mobile robot 1. In this case, images captured by the camera provided outside the mobile robot 1 and information indicating the sensing results by the sensor are transmitted to the mobile robot 1 via wireless communication.

[0094] The communication unit 105 performs wireless communication with the control device 11. For example, the communication unit 105 receives a no-entry area map transmitted from the control device 11, and outputs it to the control unit 101.

[0095] The power supply unit 106 includes a battery and supplies power to each component of the mobile robot 1.

[0096] In the control unit 101, a predetermined program is executed to realize a behavior planning unit 111 and a movement control unit 112. The behavior planning unit 111 and the movement control unit 112 perform the process of the traveling phase in FIG.

[0097] The behavior planning unit 111 plans various behaviors of the mobile robot 1. For example, the behavior planning unit 111 performs a behavior plan so as to move while avoiding the no-entry areas based on the no-entry area map generated by the control device 11. Information on the trajectory obtained by the behavior plan is supplied to the movement control unit 112.

[0098] In the action plan, the potential of each area may be used as is, or the potential after binarization using a threshold may be used. For example, an area with a potential set equal to or greater than a threshold is treated as a no-entry area.

[0099] The movement control unit 112 drives the moving unit 102 to move the mobile robot 1 according to a trajectory obtained by the action plan by the action planning unit 111 .

[0100] As described above, since the potential is set based on the result of estimating the person's cognitive state, the control device 11 can easily set a no-entry area with high reliability.

[0101] Furthermore, the mobile robot 1 can move while more reliably avoiding the no-entry areas based on the highly reliable no-entry areas.

[0102] <Embodiment 1: Basic Form> In the people flow measurement phase, the movement of people in the space is tracked. The movement of people may be tracked using a sensing system including a camera installed in the space, or may be tracked using an internal camera mounted on the mobile robot 1.

[0103] In the no-entry area setting phase, the potential is updated based on the method described with reference to FIGS. 9 to 13, and a no-entry area map is generated.

[0104] In the driving phase, the vehicle is planned to move using the no-entry area map. The way in which the no-entry area map is used is changed as appropriate depending on the algorithm of the move. For example, the vehicle is planned to move based on the potential gradient, or to move using a method such as RRT (Rapidly Exploring Random Tree) or A* for a black-and-white image binarized based on a threshold value.

[0105] <Second embodiment: Measures for cases where potential updates are incorrect> If potential updates are performed with a focus on people's cognitive states, when a person recognizes the presence of another person and avoids them, the potential updates will be performed as if the person had avoided the obstacle. In order to prevent such behavior in response to interference from other people from being reflected in potential updates, people flow data may be invalidated.

[0106] FIG. 15 is a diagram illustrating an example of people flow data.

[0107] 15 shows the trajectory of person H1 who has moved from area #12 to area #15, and the trajectory of person H2 who has moved from area #18 to area #15. Person H1 moves from area #15 to area #16 so as to avoid person H2 who is in front. Similarly, person H2 moves from area #15 to area #14 so as to avoid person H1 who is in front. It is assumed that there are no obstacles in the space formed by areas #11 to #19.

[0108] If potential updates are based on a person's cognitive state, such avoidance behavior will be reflected in the potential updates even if there is no obstacle present.

[0109] In the no-entry area setting unit 53 of the control device 11, when multiple people exist within a certain range, the potential is not updated based on the people flow data of the multiple people. That is, the people flow data is treated as invalid and is not used to update the potential. In other words, when the number of people existing within a certain range is less than a predetermined number, the people flow data is used to estimate the cognitive state of people, and the potential is updated based on the estimated cognitive state of people.

[0110] This makes it possible to update potentials by utilizing people flow data measured in situations where there is a certain amount of people mixed together.

[0111] <Third embodiment: Setting weight parameters according to the environment> FIG. 16 is a plan view showing an example of the space.

[0112] The space shown in the upper part of Fig. 16 is a corridor-like space in which people move in a uniform direction. There are walls on both sides of the corridor. In the example of Fig. 16, a person H1 is moving downward.

[0113] If potentials are updated based on people flow data measured in such a space, people will try to avoid oncoming objects as much as possible, so movement in the short direction (Y-axis direction) is thought to have a greater correlation with the existence of prohibited areas than movement in the long direction (X-axis direction). In other words, as shown in the lower part of Figure 16, certain areas in a passage are more likely to be set as prohibited areas.

[0114] In this case, weights (biases) are set for the elements used in updating the potential in accordance with the characteristics of the space. In the no-entry area setting unit 53 of the control device 11, the potential is updated taking the weights into consideration.

[0115] For example, instead of setting the same weighting parameters α, β, and γ for movement in the X-axis direction and Y-axis direction, different weighting parameters α, β, and γ are set for movement in the X-axis direction and Y-axis direction in accordance with the spatial shape.

[0116] In the example of FIG. 16, for example, if we look at the weighting parameter α that represents the contribution rate of the velocity vector, the weighting parameter α according to the movement in the X-axis direction is x Therefore, the weighting parameter α according to the movement in the Y-axis direction is y is set to be larger. This allows the potential update based on the velocity vector to have a characteristic that the potential update is more sensitive in response to the movement in the Y-axis direction.

[0117] In this manner, in the no-entry area setting unit 53, weighting parameters are set according to the characteristics of the space through which people move, such as the shape, and the like, and the potential is updated.

[0118] <Fourth embodiment: Automatic adjustment of weighting parameters using additional information> The weighting parameters α, β, and γ used in updating the potential may be adjusted based on additional information obtained by sensing.

[0119] As described above, the weighting parameter α is a parameter used to update the potential based on the velocity vector, the weighting parameter β is a parameter used to update the potential based on the acceleration vector, and the weighting parameter γ is a parameter used to update the potential based on the jerk vector.

[0120] For example, the weighting parameters are adjusted according to the age of the moving person. In this case, the additional information used to adjust the weighting parameters is the estimated age. In the people flow measurement phase, the age of the moving person is estimated by analyzing images captured by a camera, etc.

[0121] If the person moving is elderly, due to the decline in eyesight / cognitive function / physical function, etc., the person's recognition tends to depend on the state judgment of the surroundings. In this case, the weight parameter β is set to a small value, and adjustments are made so that movements based on prior judgments are less likely to be reflected in the potential update. In addition, the weight parameters α, β, and γ are all set to small values, and adjustments are made so that all movements are less likely to be reflected in the potential update.

[0122] On the other hand, if the person moving is young, their eyesight, cognitive function, and physical function are all high, so the reliability of their situation judgment is high. In this case, the weighting parameters α, β, and γ are all set to large values, and adjustments are made so that all movements are more likely to be reflected in the potential update.

[0123] In addition, the weighting parameters are adjusted according to the line of sight of the moving person. In this case, the additional information used to adjust the weighting parameters is the face direction. Here, it is assumed that the line of sight direction and the face direction match. In the people flow measurement phase, the face direction of the moving person is detected by analyzing the images captured by the camera, for example.

[0124] Normally, people recognize a no-entry area by vision and then start to take evasive action. If the direction of the velocity vector and the line of sight do not match, it is highly likely that the person is taking evasive action while following the no-entry area that was recognized just before with their eyes.

[0125] A large value is set as the weighting parameter α used in updating the potential based on the velocity vector, and adjustment is performed so that the moving velocity is more easily reflected in the updating of the potential.

[0126] In this way, in the no-entry area setting unit 53 of the control device 11, the weighting parameters are adjusted using the characteristics of the moving person, such as age and line of sight, as additional information, and the adjusted weighting parameters are used to update the potential. By using the additional information, the no-entry area setting unit 53 can reflect a more accurate and detailed cognitive state in the setting of the no-entry area.

[0127] <Fifth embodiment: Three-dimensional response by considering human posture> In addition to processing using speed, acceleration, and jerk on the XY plane, similar processing can be performed in the vertical direction (Z axis direction) to generate a map of no-entry areas in three-dimensional space.

[0128] In this case, in the people flow measurement phase, in addition to detecting the position on the XY plane, the bone shape in the 3D space is estimated. Based on the bone shape estimation result, specific parts of the moving person, such as the head and feet, are detected, as shown in Fig. 17.

[0129] In the example of Fig. 17, the positions of the head and toes on the YZ plane are detected. In the people flow measurement phase, people flow data including the speed, acceleration, and jerk of the head and the speed, acceleration, and jerk of the toes are measured based on such detection results.

[0130] Based on such people flow data, the cognitive state of people is estimated, and the potential is updated in the same way as processing on the XY plane, so that, for example, avoidance behavior such as lowering one's head when there is an obstacle above one's head is reflected in the update of the potential of each area in the three-dimensional space. Note that the potential of area #25 in Figure 17 between the head and toes is updated so that it becomes a passable area.

[0131] In this way, it is possible to update the potential of each area in the three-dimensional space based on the estimated result of the person's cognitive state. By using the no-entry area map in the three-dimensional space, it is possible to apply this technology to the behavior plan when an aircraft such as a drone moves, and the behavior plan of a robot arm, etc. In other words, this technology is applicable not only to the behavior plan of the mobile robot 1 that moves on a plane, but also to the behavior plan of a robot that moves in the three-dimensional space.

[0132] <Other> Although the mobile robot 1 is assumed to be a robot that moves using tires, the present technology can also be applied to cases where a bipedal humanoid robot or a quadrupedal animal-type robot is used. In other words, the present technology can be applied to the behavior planning of various mobile objects including robots, drones, robot arms, and automobiles.

[0133] Although the parameters of each area that are updated based on the estimation results of a person's cognitive state are assumed to be potentials used in the potential method, other parameters used in the action plan may also be set based on the estimation results of a person's cognitive state.

[0134] In the above, the processing of the people flow measurement phase and the no-entry area setting phase is performed by the control device 11, and the processing of the traveling phase is performed by the mobile robot 1, but the combination of devices responsible for processing each phase can be changed arbitrarily.

[0135] 18 to 20 are block diagrams showing other configuration examples of the information processing system.

[0136] 18, a people flow measurement unit 51 and a people flow data storage unit 52 are provided in the control device 11, and a no-entry area setting unit 53, no-entry area map storage unit 54, action planning unit 111, and movement control unit 112 are provided in the mobile robot 1. The information processing system shown in FIG. 18 is configured for a case where the people flow measurement phase is processed by the control device 11, and the no-entry area setting phase and the traveling phase are processed by the mobile robot 1.

[0137] 19, a people flow measurement unit 51 and a people flow data storage unit 52 are provided in the measurement device, a no-entry area setting unit 53 and a no-entry area map storage unit 54 are provided in the control device 11, and an action planning unit 111 and a movement control unit 112 are provided in the mobile robot 1. The information processing system shown in FIG. 19 is configured for a case in which the people flow measurement phase is processed by the measurement device, the no-entry area setting phase is processed by the control device 11, and the traveling phase is processed by the mobile robot 1.

[0138] In this way, the processing of the people flow measurement phase can be performed by a device other than the mobile robot 1 and the control device 11.

[0139] In the example of Fig. 20, a people flow measurement unit 51, a people flow data storage unit 52, an entry-prohibited area setting unit 53, an entry-prohibited area map storage unit 54, an action planning unit 111, and a movement control unit 112 are provided in the mobile robot 1. The information processing system shown in Fig. 20 is configured when all phases, namely, the people flow measurement phase, the entry-prohibited area setting phase, and the running phase, are processed by the mobile robot 1. The mobile robot 1 functions as an information processing device that processes all phases, namely, the people flow measurement phase, the entry-prohibited area setting phase, and the running phase.

[0140] About the Program The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0141] The program to be installed is provided by being recorded on removable medium 31 shown in FIG. 6, which is made up of an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or a semiconductor memory. Alternatively, the program may be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be installed in advance in ROM 22 or storage unit 28.

[0142] The program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0143] The effects described herein are merely examples and are not limiting, and other effects may also be present.

[0144] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0145] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0146] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0147] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0148] <Examples of configuration combinations> The present technology can also be configured as follows.

[0149] (1) A parameter setting unit is provided that sets parameters for each area used in the action plan of the moving object based on the cognitive state of people during movement estimated from people flow data. Information processing device. (2) The parameter setting unit estimates a person's cognitive state based on a trajectory of movement and a speed included in the people flow data. The information processing device according to (1). (3) The parameter setting unit estimates a person's cognitive state based on a trajectory of movement and an acceleration and a jerk included in the people flow data. The information processing device according to (2). (4) The parameter setting unit updates the parameters of an area in a component direction of a velocity vector based on the velocity vector represented by the people flow data. The information processing device according to (3). (5) The parameter setting unit updates the parameters of an area in a component direction of an acceleration vector based on the acceleration vector represented by the people flow data. The information processing device according to (4). (6) The parameter setting unit updates the parameters of an area in a direction opposite to a component direction of the jerk vector based on the jerk vector represented by the people flow data. The information processing device according to (5). (7) The parameter is a potential used when the action plan is performed by a potential method, The parameter setting unit is updating the potential based on the velocity vector and updating the potential based on the acceleration vector in such a manner as to decrease the potential; The potential is updated based on the jerk vector so as to increase the potential. The information processing device according to (6). (8) The parameter setting unit sets the parameters based on the cognitive state of moving people estimated from the people flow data when the number of people present within a certain range is less than a predetermined number. 8. The information processing device according to any one of (1) to (7). (9) The parameter setting unit sets weights representing degrees of update of the parameters based on the velocity vector, the acceleration vector, and the jerk vector, based on a shape of a space in which a person moves. The information processing device according to (6) or (7). (10) The parameter setting unit adjusts the degree of update of the parameters based on additional information according to characteristics of the person who is moving. The information processing device according to any one of (1) to (9). (11) The parameter setting unit estimates a cognitive state of a person based on a trajectory of movement of a specific part of the person in a three-dimensional space and a speed and acceleration included in the people flow data, and sets the parameters of each area constituting the three-dimensional space. The information processing device according to any one of (1) to (10). (12) The vehicle further includes a people flow measurement unit that detects moving people based on sensor data and generates the people flow data. The information processing device according to any one of (1) to (11). (13) The apparatus further includes an action planning unit that performs the action plan based on map information, which is information on the area in which the parameters are set. 13. The information processing device according to any one of (1) to (12). (14) An information processing device, The parameters of each area used in the movement planning of moving objects are set based on the cognitive state of people while moving, which is estimated from people flow data. Information processing methods. (15) On the computer, The parameters of each area used in the movement planning of moving objects are set based on the cognitive state of people while moving, which is estimated from people flow data. A program for executing a process. [Explanation of symbols]

[0150] 1-1 to 1-3 Mobile robot, 11 control device, 51 people flow measurement unit, 52 people flow data storage unit, 53 no-entry area setting unit, 54 no-entry area map storage unit, 101 control unit, 102 movement unit, 103 camera, 104 sensor, 105 communication unit, 111 action planning unit, 112 movement control unit

Claims

1. The parameter setting unit sets parameters for each area, which are used for a behavior plan for a moving object moving in the space after a person has moved based on people flow data including the trajectories of the respective movements of a plurality of people moving in the space, and indicates that an area in which at least a person has not entered is a no-entry area. Information processing device.

2. The parameter setting unit estimates a person's cognitive state based on a trajectory of movement and a speed included in the people flow data. The information processing device according to claim 1 .

3. The parameter setting unit estimates a person's cognitive state based on a trajectory of movement and an acceleration and a jerk included in the people flow data. The information processing device according to claim 2 .

4. The parameter setting unit updates the parameters of an area in a component direction of a velocity vector based on the velocity vector represented by the people flow data. The information processing device according to claim 3 .

5. The parameter setting unit updates the parameters of an area in a component direction of an acceleration vector based on the acceleration vector represented by the people flow data. The information processing device according to claim 4.

6. The parameter setting unit updates the parameters of an area in a direction opposite to a component direction of the jerk vector based on the jerk vector represented by the people flow data. The information processing device according to claim 5 .

7. The parameter is a potential used when the action plan is performed by a potential method, The parameter setting unit is updating the potential based on the velocity vector and updating the potential based on the acceleration vector in such a manner as to decrease the potential; The potential is updated based on the jerk vector so as to increase the potential. The information processing device according to claim 6.

8. The parameter setting unit sets the parameters based on the cognitive state of moving people estimated from the people flow data when the number of people present within a certain range is less than a predetermined number. The information processing device according to claim 1 .

9. The parameter setting unit sets weights representing degrees of update of the parameters based on the velocity vector, the acceleration vector, and the jerk vector, based on a shape of the space in which a person moves. The information processing device according to claim 6.

10. The parameter setting unit estimates a cognitive state of a person based on a trajectory of movement of a specific part of the person in a three-dimensional space and a speed and acceleration included in the people flow data, and sets the parameters of each area constituting the three-dimensional space. The information processing device according to claim 1 .

11. The vehicle further includes a people flow measurement unit that detects moving people based on sensor data and generates the people flow data. The information processing device according to claim 1 .

12. The apparatus further includes an action planning unit that performs the action plan based on map information, which is information on the area in which the parameters are set. The information processing device according to claim 1 .

13. An information processing device, Based on people flow data including the trajectories of the movements of each of a plurality of people moving through a space, parameters are set for each area, which are used for a behavior plan for a moving object moving through the space after the people have moved, and which indicate that an area into which at least one person has not entered is a no-entry area. Information processing methods.

14. On the computer, Based on people flow data including the trajectories of the movements of each of a plurality of people moving through a space, parameters are set for each area, which are used for a behavior plan for a moving object moving through the space after the people have moved, and which indicate that an area into which at least one person has not entered is a no-entry area. A program for executing a process.

Images