Information processing methods, mobile bodies and programs

Abstract

This technology relates to an information processing method, a moving body, and a program that enables appropriate tracking of the position of an object. The information processing method according to this technology includes: identifying an object to be tracked from objects identified based on sensor data from sensors detecting objects in the surrounding three-dimensional space; tracking the position of the object to be tracked in the three-dimensional space; and controlling a threshold for object identification based on the crowding around the sensors. This technology can be applied, for example, to robots that follow human movement.

Description

Technical Field

[0001] This technology relates to information processing methods, moving bodies, and programs, and more specifically to information processing methods, moving bodies, and programs capable of appropriately tracking the position of an object. Background Technology

[0002] A system is known that identifies objects, such as people, appearing in images captured by a camera device and provides users with information about crowding in a three-dimensional space, such as in a commercial facility (see, for example, Patent Document 1). The technique for detecting people appearing in images captured by a camera device is also used to track the position of people moving in three-dimensional space. When tracking a person, the person to be tracked is identified from among the people identified in the image.

[0003] Citation List

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-173775 Summary of the Invention

[0006] The problem to be solved by the present invention

[0007] In crowded 3D spaces, mismatches may occur, such as misidentifying a person different from the one being tracked from among people identified in an image. To prevent mismatches, if the threshold used for person identification is reduced, the person being tracked may be lost when the 3D space is not crowded.

[0008] This technology was developed in view of such circumstances and can properly track the position of objects.

[0009] Solution to the problem

[0010] The information processing method according to a first aspect of the present technology includes: identifying an object to be tracked from objects identified based on sensor data from a sensor that detects objects in a surrounding three-dimensional space, and tracking the position of the object to be tracked in the three-dimensional space; and controlling a threshold for identifying the object based on the degree of crowding around the sensor.

[0011] The moving body according to a second aspect of the present technology includes: a sensor that detects objects in the surrounding three-dimensional space; a tracking unit that identifies objects to be tracked from objects identified based on sensor data from the sensor and tracks the position of the objects to be tracked in the three-dimensional space; a tracking control unit that controls a threshold for identifying objects based on the degree of crowding around the sensor; and a movement control unit that controls the movement of the moving body based on the position of the objects to be tracked.

[0012] A procedure according to a third aspect of the present technology causes a computer to perform a process comprising: identifying an object to be tracked from objects identified based on sensor data from a sensor that detects objects in a surrounding three-dimensional space, and tracking the position of the object to be tracked in the three-dimensional space; and controlling a threshold for identifying the object based on the degree of crowding around the sensor.

[0013] According to the first and third aspects of this technology, an object to be tracked is identified from objects identified based on sensor data from a sensor that detects objects in the surrounding three-dimensional space, and the position of the object to be tracked in the three-dimensional space is tracked; and a threshold for identifying the object is controlled based on the degree of crowding around the sensor.

[0014] According to a second aspect of the present technology, an object in the surrounding three-dimensional space is detected by a sensor; an object to be tracked is identified from the objects identified by the sensor based on sensor data by a tracking unit, and the tracking unit tracks the position of the object to be tracked in the three-dimensional space; and a threshold for identifying the object is controlled by a tracking control unit based on the degree of congestion around the sensor, and the tracking control unit controls the movement of the moving body based on the position of the object to be tracked. Attached Figure Description

[0015] [ Figure 1 [Illustration 1] is a diagram showing an example configuration of the appearance of a robot according to an embodiment of the present technology.

[0016] [ Figure 2 [ ] is a block diagram showing an example of robot configuration.

[0017] [ Figure 3 ] is a diagram showing an example of an object map.

[0018] [ Figure 4 [ ] is a diagram showing an example of a target range used to calculate the degree of congestion.

[0019] [ Figure 5 [ ] is a diagram illustrating human identification methods.

[0020] [ Figure 6 [This is a flowchart illustrating the process by which the computing unit calculates the target range for calculating the degree of congestion.]

[0021] [ Figure 7 This is a flowchart illustrating the process by which the computing unit calculates the level of crowding and tracks the location of the person to be tracked.

[0022] [ Figure 8 ] is shown in Figure 7 The flowchart for the human tracking process performed in step S25.

[0023] [ Figure 9[ ] is a block diagram illustrating an example configuration of computer hardware. Detailed Implementation

[0024] The modes used to implement this technology will be described below. The descriptions will be given in the following order.

[0025] 1. Robot configuration

[0026] 2. Operation of the computing unit

[0027] <1. Robot Configuration>

[0028] Figure 1 This is a diagram illustrating an example configuration of the appearance of robot 1 according to an embodiment of the present technology.

[0029] like Figure 1 As shown, robot 1 is a mobile body with a humanoid upper body and a wheeled locomotion mechanism. A head unit 21, shaped like a flattened sphere, is mounted on the body unit 11. Two vision sensors 12A, with shapes mimicking human eyes, are positioned in front of the head unit 12. Each of the vision sensors 12A is a sensor that detects objects in the surrounding three-dimensional space and includes a camera device or light detection and ranging (or laser imaging detection and ranging) (LiDAR).

[0030] Each of the body units 11 is provided at the upper end as an arm unit consisting of a multi-degree-of-freedom manipulator, and a hand unit is provided at the distal end of each arm unit.

[0031] A vehicle-shaped mobile body unit 13 is provided at the lower end of the body unit 11. The robot 1 can move by rotating the wheels provided on the left and right sides of the mobile body unit 13 or by turning the wheels in different directions.

[0032] For example, a computing unit for controlling the operation of robot 1 is provided inside body unit 11. The computing unit tracks the position of the object to be tracked (e.g., a person) based on sensor data such as vision sensor 12A, and controls the moving body unit 13 to make robot 1 follow the object.

[0033] Note that robot 1 may not be configured as follows. Figure 1 The robot shown is not a dual-armed robot, but is configured as a single-armed robot (with a hand unit) or a robot without arms. Furthermore, the body unit 11 can be mounted on legs instead of the vehicle (mobile body unit 13).

[0034] Figure 2 This is a block diagram showing an example configuration of robot 1.

[0035] like Figure 2As shown, robot 1 includes a vision sensor 12A, a storage unit 21, a computing unit 22, and an actuator 23.

[0036] Storage unit 21 stores an object map, the object map Figure 3 The robot 1 is given instructions regarding the shape, position, and speed of movement of people or other objects present around it.

[0037] Figure 3 This is a diagram showing an example of an object map.

[0038] like Figure 3 As shown, the object map is, for example, a map in a view overlooking the perimeter of robot 1 from above. Figure 3 In the example, people P1 to P3 and objects Obj1 to Obj4 are in front of robot 1.

[0039] For example, an object map is generated based on sensor data from the vision sensor 12A. Specifically, the shape, position, and speed of movement of each person or object appearing in the image captured by the camera device used as the vision sensor 12A are registered on the object map. Furthermore, the shape, position, and speed of movement of people or objects detected by LiDAR, which is used as the vision sensor 12A, are also registered on the object map.

[0040] For example, the image capture range (sensing range) of the camera device and the scanning range (sensing range) of the LiDAR are also registered on the object map. Figure 3 In the diagram, the image capture range of the camera device is surrounded by solid lines, while the scanning range of the LiDAR is surrounded by dashed lines. The image capture range of the camera device includes people P1, people P2, and object Obj3, while the scanning range of the LiDAR includes people P1 to P3 and objects Obj1 to Obj4.

[0041] Return to Figure 2 The computing unit 22 identifies the person to be tracked from the people identified by the sensor data based on the vision sensor 12A, tracks the position of the person to be tracked in three-dimensional space, and controls the threshold for human identification based on the degree of crowding around the robot 1 (vision sensor 12A).

[0042] The calculation unit 22 includes an identification unit 41, a tracking unit 42, a target range calculation unit 43, a congestion level calculation unit 44, a threshold calculation unit 45, a user interface (UI) control unit 46, an action plan unit 47, and a movement control unit 48.

[0043] The identification unit 41 identifies the location of people around the robot 1 based on the sensor data of the vision sensor 12A, and provides the results of the identification of people to the tracking unit 42.

[0044] The tracking unit 42 tracks the position of the person to be tracked based on the recognition result of the person identified by the recognition unit 41, and updates the position and movement speed of the person to be tracked on the object map. Specifically, the tracking unit 42 identifies the person to be tracked from the people identified by the recognition unit 41 (people included in the image capture range of the camera device or the scanning range of the LiDAR), and registers the current position and movement speed of the person to be tracked on the object map.

[0045] More specifically, tracking unit 42 uses a particle filter or Kalman filter to estimate the current position of the person to be tracked. Tracking unit 42 searches for the person to be tracked from among those identified by identification unit 41 (whose positions are within a predetermined distance around the estimated position of the person to be tracked). Hereinafter, the range for searching for the person to be tracked is also referred to as the search range, and the size of the search range is also referred to as a threshold for person identification. The threshold for person identification indicates, for example, the radius of the search range.

[0046] The target range calculation unit 43 obtains information from the storage unit 21 indicating the location and movement speed of the person to be tracked and the image capture range of the camera device, and calculates the target range for calculating the degree of crowding based on this information.

[0047] Figure 4 This is a diagram illustrating an example of a target range used to calculate congestion levels. Figure 4 In this section, a description will be given assuming that eight people, P11 to P18, exist around robot 1. Furthermore, by... Figure 4 The black dot in the image indicates the person P15, who is set as the person to be tracked.

[0048] exist Figure 4 In example A, the target range calculation unit 43 sets a rectangular region A1, which has a rectangular shape when viewing the periphery of robot 1 from above, as the target range for calculating the degree of crowding. Region A1 is the rectangular region with the smallest area among the rectangular regions (each rectangular region surrounds all persons P14 to P16 included in the image capture range of the camera device). Note that a circular region with the smallest area among the circular regions (each circular region surrounds all persons included in the image capture range of the camera device) can be set as the target range for calculating the degree of crowding.

[0049] exist Figure 4 In example B, the target range calculation unit 43 sets a circular region A2, centered on the person P15 to be tracked, as the periphery of robot 1 when viewed from above, as the target range for calculating the crowding level. For example, the radius a of region A2 is preset. Region A2 includes four people P14 to P17.

[0050] exist Figure 4In the example in C, the target range calculation unit 43 sets a region A3, which has an elliptical shape centered on the person P15 to be tracked, as the periphery of the robot 1 when viewed from above, as the target range for calculating the crowding level. Region A3 is an elliptical region formed taking into account the moving speed of the person P15 to be tracked. Figure 3 In the example, person P15 moves in the direction indicated by the white arrow. For example, the radius 'a' in the direction perpendicular to the movement direction of person P15 in region A3 is preset, and the radius 'b' in the direction parallel to the movement direction of person P15 is set to (a + movement speed). (1 second). Area A3 includes four people, P14 through P17.

[0051] Target range calculation unit 43 provides information indicating the target range for calculating congestion level as described above. Figure 3 The crowding level calculation unit 44 and UI control unit 46 are included. Note that the target range for calculating the crowding level can be not only an area with the shape described above, but also an area with other shapes (such as polygonal shapes or shapes obtained by cutting off a part of a circle or ellipse) when viewed from above around the periphery of the robot 1.

[0052] The congestion level calculation unit 44 calculates the congestion level around robot 1 based on, for example, the number of people included in the target range calculated by the target range calculation unit 43. Specifically, the congestion level calculation unit 44 refers to the object map stored in the storage unit 21 to count the number of people included in the target range, and sets the value obtained by dividing the number of people included in the target range by the area of ​​the target range as the congestion level around robot 1. The congestion level calculation unit 44 provides the congestion level around robot 1 to the threshold calculation unit 45.

[0053] Note that in robot 1, a range of congestion levels (maximum and minimum values) is preset. If the congestion level calculated by the congestion level calculation unit 44 exceeds the maximum value, the congestion level value is considered the maximum value. If the calculated congestion level is less than the minimum value, the congestion level value is considered the minimum value.

[0054] The threshold calculation unit 45 calculates a threshold for human identification based on the crowding level calculated by the crowding level calculation unit 44. In robot 1, a range (maximum and minimum value) for the threshold for human identification is preset. The threshold calculation unit 45 calculates the threshold for human identification by linearly mapping the crowding level. When the crowding level and the threshold for human identification are, for example, proportional, the threshold calculation unit 45 calculates the threshold for human identification using the following equation (1).

[0055] [Mathematical Expression 1]

[0056]

[0057] …(1)

[0058] For example, if the crowding level ranges from 0 to 4, and the threshold for human identification ranges from 0.7 m to 0.4 m, then the threshold = 0.7 - (crowding level - 0). 0.3 / 4. Here, the threshold for human identification is at its minimum when the crowding level is at its maximum, and at its maximum when the crowding level is at its minimum. In other words, the higher the crowding level, the lower the threshold for human identification.

[0059] The threshold calculation unit 45 provides the calculated threshold to the tracking unit 42. The threshold calculation unit 45 acts as a tracking control unit, which controls the threshold used for human identification based on the crowding level around the robot 1 by the tracking unit 42. The tracking unit 42 identifies the person to be tracked by using the threshold calculated by the threshold calculation unit 45.

[0060] Figure 5 This is a diagram illustrating methods for human identification.

[0061] First, the tracking unit 42 is based on Figure 5 The position of person P15 at time t, as shown in the lower part, is estimated by... Figure 5 The dashed line in the diagram indicates the position of person P15 at time t+1. Next, tracking unit 42 uses the sensor data acquired at time t+1 to determine the position of the person identified by recognition unit 41. Figure 5 In the example, people were identified. to The location.

[0062] Next, the tracking unit 42 tracks people identified by the identification unit 41 and included within a search range A11 centered on the estimated location of person P15 at time t+1 and calculated by the threshold calculation unit 45. to Searching for person P15. In Figure 5 In the example, because only people Included in the search range A11, therefore the tracking unit 42 will track the person. P15 is identified as the person to be tracked.

[0063] Note that instead of setting one person as the tracking target, you can set multiple people as the tracking target and track the location of each person simultaneously.

[0064] Return to Figure 2For example, the UI control unit 46 displays the UI on a display unit located in the controller of the robot 1, and accepts user operations via the UI.

[0065] For example, the UI control unit 46 receives a designation of the person to be followed (as the object to be followed by the robot 1) via the UI. The UI control unit 46 notifies the tracking unit 42 of the person to be followed as designated by the user.

[0066] Furthermore, for example, the UI control unit 46 presents the user with a target range for calculating the level of congestion. The user can specify the shape of the target range (e.g., a rectangular shape, a circular shape, or an elliptical shape) while viewing the presented target range for calculating the level of congestion.

[0067] Action planning unit 47 refers to the object map stored in storage unit 21 to obtain the location of the person to be followed (the person to be tracked), and creates a motion plan for robot 1 based on the location of the person to be followed. For example, action planning unit 47 obtains the route for following the person to be followed and creates a motion plan so that robot 1 moves along the route. Action planning unit 47 provides the created motion plan to motion control unit 48.

[0068] The motion control unit 48 controls the movement of the robot 1 (the machine itself) to implement the action plan created by the action planning unit 47. For example, the motion control unit 48 controls the actuator 23 provided in the motion body unit 13, so that the robot 1 moves along the route obtained by the action planning unit 47.

[0069] Note that some or all of the configuration of the computing unit 22 can be implemented by a controller that is separately configured from the robot 1. For example, the recognition unit 41, tracking unit 42, target range calculation unit 43, crowding level calculation unit 44, threshold calculation unit 45, and UI control unit 46 are implemented by the controller. In addition, the storage unit 21 can be located in a device other than the robot 1, for example, in the cloud.

[0070] <2. Operations of the Computation Unit>

[0071] Next, we will refer to Figure 6 The flowchart in the diagram describes the computation unit 22, which has the configuration described above, performing a process for calculating the target range of congestion levels.

[0072] In step S1, the UI control unit 46 controls the display unit, such as a controller, to display candidates for the person to be followed and presents the candidates to the user. Here, the UI control unit 46, for example, presents the person identified by the recognition unit 41 as a candidate for the person to be followed to the user, and receives the user's selection of the person to be followed. The user can select the person to follow from the presented candidates.

[0073] In step S2, the UI control unit 46 determines whether the user has selected the person to follow.

[0074] If it is determined in step S2 that the user has not selected anyone to follow, the process returns to step S1, and the candidates for people to follow are displayed continuously until the user selects someone to follow.

[0075] On the other hand, if it is determined in step S2 that the user has selected the person to follow, the target range calculation unit 43 calculates the target range for calculating the degree of crowding in step S3.

[0076] In step 4, the UI control unit 46 controls the display unit of the controller to display the target range for calculating the congestion level and presents the target range to the user. After displaying the target range for calculating the congestion level, the process returns to step S3, and the target range for calculating the congestion level is calculated again, for example, if there are people around the robot 1 or people to be followed are moving.

[0077] Next, we will refer to Figure 7 The flowchart in the diagram describes the process by which calculation unit 22 calculates the level of crowding and tracks the location of the people to be tracked.

[0078] In step S21, the identification unit 41 identifies the location of people present around the robot 1 based on sensor data from the vision sensor 12A. For example, it identifies the location of a person appearing in an image captured by a camera device used as the vision sensor 12A, and projects the person's location in the image onto their location in three-dimensional space. Frame numbers, starting from, for example, 0, are assigned to each frame of the image captured by the camera device.

[0079] In step S22, the tracking unit 42 determines whether the frame number of the frame in which a person is identified (the current frame) is 1 or greater.

[0080] If the frame number of the current frame is determined to be 0 in step S22, the position of the person appearing in the current frame in the three-dimensional space is registered on the object map, and the process returns to step S21.

[0081] On the other hand, if it is determined in step S22 that the frame number of the current frame is 1 or greater, the congestion calculation unit 44 calculates the congestion level around robot 1 in step S23. Specifically, the congestion calculation unit 44 refers to an object map that registers the positions of people appearing in past frames in three-dimensional space, counts the number of people included in the target area, and divides the number of people by the area of ​​the target area to calculate the congestion level.

[0082] Note that when calculating the target range for calculating the degree of crowding based on the image capture range of the camera device, the crowding degree calculation unit 44 can also refer to the positions of people identified in the current frame to count the number of people included in the target range.

[0083] In step S24, the threshold calculation unit 45 calculates a threshold for human identification based on the degree of crowding around the robot 1.

[0084] In step S25, tracking unit 42 performs person tracking processing. The threshold calculated in step S24 is used in the person tracking processing to identify the person to be tracked from among the people identified in the current frame in step S21. (See below for further details.) Figure 8 Describe the details of human tracking and processing.

[0085] After the person tracking process is performed in step S25, the process returns to step S21 and the subsequent processes are repeated.

[0086] Next, we will refer to Figure 8 The flowchart description in Figure 7 The person tracking process is performed in step S25.

[0087] In step S41, the tracking unit 42 obtains the result of the identification of the person from the identification unit 41.

[0088] In step S42, the tracking unit 42 estimates the moving speed and current position of the person to be tracked. Specifically, the tracking unit 42 creates a motion model of the person to be tracked based on the position of the person to be tracked at a time in a past frame, and uses the motion model to estimate, for example, the position of the person to be tracked at a time in the current frame (current position). For example, the motion model generated based on the position of the person to be tracked at time t-1 and the position of the person to be tracked at time t is used to estimate the position of the person to be tracked at time t+1 in the current frame.

[0089] In step S43, the tracking unit 42 identifies the person to be tracked at the current frame time. Specifically, the tracking unit 42 searches for the person to be tracked among the people identified in the current frame who exist within a range centered on the estimated location of the person to be tracked within a threshold.

[0090] In step S44, the tracking unit 42 registers the current location and movement speed of the person to be tracked on the object map. Here, for example, the locations of people other than the person to be tracked are also registered on the object map.

[0091] After registering the current location and movement speed of the person to be tracked on the object map, the process returns to... Figure 7 Step S25 in the process, and then perform subsequent processing.

[0092] As described above, in the computing unit 22 of this technology, the object to be tracked is identified from objects (such as people) identified based on sensor data from the visual sensor 12A that detects objects in the surrounding three-dimensional space, and the position of the object to be tracked in the three-dimensional space is tracked. Furthermore, in the computing unit 22 of this technology, the threshold used for object identification is controlled based on the degree of crowding around the visual sensor 12A (robot 1).

[0093] For example, when the area around robot 1 is crowded, computing unit 22 decreases the threshold for human identification, thereby preventing mismatches, where a person different from the person to be tracked is identified as the person to be tracked from among the people identified in the image. Furthermore, for example, when the area around robot 1 is not crowded, computing unit 22 increases the threshold for human identification, thereby preventing the person to be tracked from being lost.

[0094] As described above, by controlling the threshold used for human identification based on the degree of crowding, the computing unit 22 can prevent both mismatch and loss of the person to be followed, and can properly track the position of the person to be followed.

[0095] <Computer Configuration Example>

[0096] The series of processes described above can be executed by hardware or by software. In the case where the series of processes are executed by software, the program constituting the software is installed from the program recording medium onto a computer, general-purpose personal computer, or similar hardware integrated into dedicated hardware.

[0097] Figure 9 This is a block diagram illustrating an example configuration of computer hardware that performs the series of processes described above through a program.

[0098] The central processing unit (CPU) 501, read-only memory (ROM) 502 and random access memory (RAM) 503 are interconnected via bus 504.

[0099] The input / output interface 505 is also connected to the bus 504. Input units 506, including a keyboard, mouse, etc., and output units 507, including a display, speakers, etc., are connected to the input / output interface 505. Furthermore, storage units 508, including hard disks, non-volatile memory, etc., communication units 509, including network interfaces, etc., and drivers 510 that drive removable media 511 are connected to the input / output interface 505.

[0100] In a computer configured as described above, for example, CPU 501 loads a program stored in storage unit 508 into RAM 503 via input / output interface 505 and bus 504, and executes the program to perform the series of processes described above.

[0101] The program to be executed by the CPU 501 is provided, for example, in a state recorded in the removable medium 511 or via a wired or wireless transmission medium (such as a local area network, the Internet, or digital broadcasting), and is installed in the storage unit 508.

[0102] Note that a program to be executed by a computer can be a program that performs processing in the time sequence described herein, or it can be a program that performs processing in parallel or at necessary time points (such as when a call is made).

[0103] Note that the effects described in this article are merely examples and are not limited to any particular effect; other effects may also exist.

[0104] The implementation of this technology is not limited to the implementation described above, and various changes can be made to them without departing from the scope of this technology.

[0105] For example, this technology can be configured as cloud computing, where multiple devices or apparatuses share a single function and perform processing collaboratively over a network.

[0106] Furthermore, each of the steps described in the flowchart above can be performed by a single device or apparatus, or can be shared and performed by multiple devices or apparatuses.

[0107] Furthermore, in cases where a step includes multiple processes, the multiple processes included in a step may be performed by a single device or apparatus, or by multiple devices or apparatuses in a shared manner.

[0108] <Example of configuration combinations>

[0109] This technology can also have the following configurations.

[0110] (1) An information processing method, comprising:

[0111] Identify the object to be tracked from objects identified by sensors that detect objects in three-dimensional space, and track the position of the object to be tracked in three-dimensional space; and

[0112] The threshold used to identify objects is controlled based on the level of crowding around the sensor.

[0113] (2) According to the information processing method in (1) above, where,

[0114] Identify the objects to be tracked from those included in the search range among the objects identified based on sensor data, and

[0115] The threshold indicates the size of the search range.

[0116] (3) Based on the information processing method in (2) above, where,

[0117] The search range is the area centered on the estimated location of the object, and

[0118] The threshold indicates the radius of the search range.

[0119] (4) Based on any one of the information processing methods in (1) to (3) above, where,

[0120] The threshold is calculated by linearly mapping the degree of crowding.

[0121] (5) The information processing method based on any one of (1) to (4) above also includes:

[0122] The degree of congestion is calculated based on the number of objects included in the target range used to calculate the degree of congestion.

[0123] (6) Based on the information processing method in (5) above, where,

[0124] The target area used to calculate the degree of crowding is an area that has a polygonal shape, a circular shape, an elliptical shape, or a shape obtained by cutting off a part of a circle or ellipse when viewed from above in three-dimensional space.

[0125] (7) Based on the information processing methods in (5) or (6) above, it also includes:

[0126] The target range for calculating the degree of congestion is calculated based on the location of the object to be tracked, the speed at which the object moves, or the sensing range of the sensor.

[0127] (8) The information processing method based on any one of (5) to (7) above also includes:

[0128] The target range used to calculate the level of congestion is presented to the user.

[0129] (9) The information processing method based on any one of (1) to (8) above also includes:

[0130] The system presents the objects identified based on sensor data as candidates for tracking to the user and receives the user's selection of the objects to be tracked.

[0131] (10) According to any one of the information processing methods in (1) to (9) above, where,

[0132] Tracking the position of a person as an object in three-dimensional space, and

[0133] The threshold for identifying people is controlled based on the degree of crowding.

[0134] (11) According to any one of the information processing methods in (1) to (10) above, where,

[0135] The sensor includes at least one of a camera device and a LiDAR.

[0136] (12) According to any one of the information processing methods in (1) to (11) above, where,

[0137] The sensor is mounted on a moving body that moves based on the position of the object to be tracked in three-dimensional space.

[0138] (13) A mobile body, comprising:

[0139] Sensors that detect objects in three-dimensional space;

[0140] The tracking unit identifies the object to be tracked from the objects identified by sensor-based sensor data and tracks the position of the object to be tracked in three-dimensional space.

[0141] The tracking control unit controls the threshold used for object identification based on the level of crowding around the sensor; and

[0142] The motion control unit controls the movement of the moving body based on the position of the object to be tracked.

[0143] (14) A program for causing a computer to perform a process, the process comprising:

[0144] Identify the object to be tracked from objects identified by sensors that detect objects in three-dimensional space, and track the position of the object to be tracked in three-dimensional space; and

[0145] The threshold used to identify objects is controlled based on the level of crowding around the sensor.

[0146] List of reference numerals

[0147] 1 robot

[0148] 11 body units

[0149] 12 head units

[0150] 12A vision sensor

[0151] 13 moving body units

[0152] 21 storage units

[0153] 22 computing units

[0154] 23 actuators

[0155] 41 Identification Unit

[0156] 42 tracking units

[0157] 43 Target Range Calculation Unit

[0158] 44 Congestion Calculation Unit

[0159] 45 threshold calculation units

[0160] 46 UI control unit

[0161] 47 Action Plan Units

[0162] 48 Motion Control Unit

Claims

1. An information processing method, comprising: Identify the object to be tracked from the objects identified by the sensor data of the sensor that detects objects in three-dimensional space, and track the position of the object to be tracked in the three-dimensional space; as well as The threshold used to identify the object is controlled based on the level of crowding around the sensor.

2. The information processing method according to claim 1, wherein, Identify the object to be tracked from among the objects included in the search range identified based on the sensor data, and The threshold indicates the size of the search range.

3. The information processing method according to claim 2, wherein, The search range is the range centered on the estimated location of the object, and The threshold indicates the radius of the search range.

4. The information processing method according to claim 1, wherein, The threshold is calculated by linearly mapping the degree of congestion.

5. The information processing method according to claim 1, further comprising: The degree of congestion is calculated based on the number of objects included in the target range used to calculate the degree of congestion.

6. The information processing method according to claim 5, wherein, The target range used to calculate the degree of crowding is an area that has a polygonal shape, a circular shape, an elliptical shape, or a shape obtained by cutting off a portion of a circle or ellipse when viewed from above in the three-dimensional space.

7. The information processing method according to claim 5 further includes: The target range for calculating the degree of congestion is calculated based on the location of the object to be tracked, the moving speed of the object to be tracked, or the sensing range of the sensor.

8. The information processing method according to claim 5, further comprising: The target range used to calculate the level of congestion is presented to the user.

9. The information processing method according to claim 1, further comprising: The system presents the objects identified based on the sensor data as candidates for the objects to be tracked to the user, and receives the user's selection of the objects to be tracked.

10. The information processing method according to claim 1, wherein, Tracking the position of the person as the object in the three-dimensional space, and The threshold used to identify the person is controlled based on the degree of crowding.

11. The information processing method according to claim 1, wherein, The sensor includes at least one of a camera device and a LiDAR.

12. The information processing method according to claim 1, wherein, The sensor is mounted on a moving body that moves based on the position of the object to be tracked in the three-dimensional space.

13. A mobile body, comprising: Sensor, which detects objects in three-dimensional space; A tracking unit identifies an object to be tracked from objects identified based on sensor data from the sensor, and tracks the position of the object to be tracked in the three-dimensional space. A tracking control unit that controls a threshold for identifying the object based on the degree of crowding around the sensor; as well as A motion control unit that controls the movement of the moving body based on the position of the object to be tracked.

14. A program for causing a computer to perform a process, said process comprising: Identify the object to be tracked from the objects identified by the sensor data of the sensor that detects objects in three-dimensional space, and track the position of the object to be tracked in the three-dimensional space; as well as The threshold used to identify the object is controlled based on the level of crowding around the sensor.

Images