Information processing apparatus, information processing method, and program

Abstract

This information processing device comprises a fusing unit that updates a second cluster group by fusing each first cluster included in a first cluster group obtained by clustering first point group data acquired by a sensor to each second cluster included in the second cluster group which is generated on the basis of second point group data acquired more in the past than the first point group data.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, a technique for representing the surrounding environment as an aggregate of minute planes based on sensing results by a sensor or the like has been proposed (for example, Non-Patent Document 1).

[0003] For example, a technique for representing the surface of an object existing in the surrounding environment as point cloud data by acquiring the distance and azimuth to the object existing in the surrounding environment using a distance sensor or the like has been proposed.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

[0005] Data representing such a surrounding environment is desirably updated rapidly as the surrounding environment changes.

[0006] Therefore, it is desirable to provide an information processing apparatus, an information processing method, and a program capable of updating data representing the environment with higher efficiency.

[0007] An information processing apparatus according to an embodiment of the present disclosure includes a fusion unit that updates the second cluster group by fusing each of the first clusters included in the first cluster group obtained by clustering the first point cloud data acquired by a sensor with each of the second clusters included in the second cluster group generated based on the second point cloud data acquired earlier than the first point cloud data. The shapes of the first cluster and the second cluster are oblate ellipsoids, and the fusion part fuses each of the first clusters with each of the second clusters based on the normal direction to the flat plane of the oblate ellipsoid and the thickness in the normal direction of the oblate ellipsoid, as the shapes of the first cluster and the second cluster.

[0008] An information processing method according to an embodiment of the present disclosure includes updating a second cluster group by using an arithmetic processing unit to fuse each of the first clusters included in the first cluster group obtained by clustering the first point cloud data acquired by a sensor with each of the second clusters included in the second cluster group generated based on the second point cloud data acquired earlier than the first point cloud data. The shapes of the first cluster and the second cluster are oblate ellipsoids, and based on the normal direction to the flat plane of the oblate ellipsoid and the thickness in the normal direction of the oblate ellipsoid, as the shapes of the first cluster and the second cluster, each of the first clusters is fused with each of the second clusters.

[0009] A program according to an embodiment of the present disclosure causes a computer to function as a fusion unit that updates a second cluster group by fusing each of the first clusters included in the first cluster group obtained by clustering the first point cloud data acquired by a sensor with each of the second clusters included in the second cluster group generated based on the second point cloud data acquired earlier than the first point cloud data. The shapes of the first cluster and the second cluster are oblate ellipsoids, and the fusion part fuses each of the first clusters with each of the second clusters based on the normal direction to the flat plane of the oblate ellipsoid and the thickness in the normal direction of the oblate ellipsoid, as the shapes of the first cluster and the second cluster.

[0010] In the information processing apparatus, information processing method, and program according to an embodiment of the present disclosure, each cluster of the first cluster group based on the first point cloud data acquired by a sensor is fused with each cluster of the second cluster group based on the second point cloud data acquired earlier than the first point cloud data, whereby the second cluster group can be updated. Thereby, for example, the surrounding environment can be represented using a cluster group obtained by compressing point cloud data acquired by a sensor through clustering, and the cluster group can be updated by sequentially fusing the cluster groups in time series.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments in the present disclosure will be described in detail with reference to the drawings. The embodiments described below are a specific example of the present disclosure, and the technology according to the present disclosure is not limited to the following aspects. Also, regarding the arrangement, dimensions, dimensional ratios, etc. of each component of the present disclosure, they are not limited to the states shown in each figure.

[0013] Note that the description will be made in the following order. 1. First Embodiment 1.1. Configuration Example 1.2. Operation Example 1.3. Modification Example 2. Second Embodiment 2.1. Configuration Example 2.2. Modification Example 3. Hardware Configuration Example

[0014] <1. First Embodiment> (1.1. Configuration Example) First, with reference to FIG. 1, a mobile body including an information processing apparatus according to the first embodiment of the present disclosure will be described. FIG. 1 is a block diagram showing an example of the functional configuration of a mobile body 1 including an information processing apparatus 10 according to the present embodiment.

[0015] As shown in FIG. 1, the mobile body 1 includes a sensor unit 20, an information processing apparatus 10 according to the present embodiment, a drive control unit 31, and a movement mechanism 32. Further, the information processing apparatus 10 according to the present embodiment includes a plane estimation unit 110, a cluster group update unit 120, an object recognition unit 130, a self-position estimation unit 140, an action planning unit 150, and a coordinate system conversion unit 160.

[0016] The mobile body 1 is, for example, a robot or a drone equipped with a moving mechanism 32 and capable of autonomous movement. The information processing device 10 provided in the mobile body 1 can, for example, construct an environmental map based on the data acquired by the sensor unit 20 and plan the movement route of the mobile body 1 based on the constructed environmental map. The movement route planned by the information processing device 10 can be output to, for example, the drive control unit 31 that controls the moving mechanism 32 of the mobile body 1, so that the mobile body 1 can be moved along the planned movement route.

[0017] Needless to say, the information processing device 10 may be provided inside the mobile body 1 or may be provided outside the mobile body 1. When the information processing device 10 is provided inside the mobile body 1, the information processing device 10 can output a route plan to the drive control unit 31 of the mobile body 1 via internal wiring. Also, when the information processing device 10 is provided outside the mobile body 1, the information processing device 10 can transmit a route plan to the drive control unit 31 of the mobile body 1 via wireless communication or the like.

[0018] The sensor unit 20 includes sensors that sense the environment around the mobile body 1 and outputs the sensing result as point cloud data that is a set of points.

[0019] For example, the sensor unit 20 may include a distance measuring sensor that measures the distance to an object such as an ultrasonic sensor (Sound Navigation And Ranging: SONAR), a ToF (Time of Flight) sensor, a RADAR (Radio Detecting And Ranging), or a LiDAR (Light Detection And Ranging) sensor. In such a case, the sensor unit 20 can generate point cloud data by converting the measurement points into points in a three-dimensional coordinate system based on the information on the distance and azimuth to the measurement points acquired from the distance measuring sensor.

[0020] Alternatively, the sensor unit 20 may include an imaging device that acquires an image of the environment around the moving body 1, such as a stereo camera, a monocular camera, a color camera, an infrared camera, a spectroscopic camera, or a polarization camera. In such a case, the sensor unit 20 can estimate the depth of the image points included in the captured image based on the captured image, and generate point cloud data by converting the image points into points in a three-dimensional coordinate system based on the information regarding the estimated depth.

[0021] Note that the sensor unit 20 may be provided in another device or object outside the moving body 1 as long as it can sense the environment around the moving body 1. For example, the sensor unit 20 may be provided on the ceiling, wall, or floor of the space where the moving body 1 exists.

[0022] The plane estimation unit 110 generates a cluster group by clustering the point groups included in the point cloud data acquired by the sensor unit 20. The generated cluster group corresponds to a group of micro planes that constitute the surface of the object existing around the moving body 1.

[0023] With reference to FIGS. 2 and 3, the clustering of the point cloud data and the clusters included in the cluster group will be described in more detail. FIG. 2 is an explanatory diagram for explaining the point cloud data and the clustering of the point cloud data. FIG. 3 is an explanatory diagram schematically showing the shape of the clusters corresponding to the micro planes that constitute the object surface.

[0024] For example, as shown in FIG. 2, the plane estimation unit 110 can generate a cluster group including a plurality of clusters CL by clustering a point group PC that is a set of points P in a three-dimensional coordinate system. Each of the clusters CL can be represented, for example, by a flattened ellipsoid based on the center coordinates and the probability distribution shape of the cluster CL as shown in FIG. 3. Specifically, the cluster CL may be represented by a flattened ellipsoid centered on the average value of the coordinates of the points included in the cluster CL and having the covariance of the coordinates of the points included in the cluster CL as the thickness. Such a cluster CL of a flattened ellipsoid can use a vector NL indicating the normal direction with respect to the flat plane and the thickness in the normal direction of the flattened ellipsoid as representative parameters of the shape. That is, the plane estimation unit 110 can represent a microplane group constituting the object surface existing around the moving body 1 with a cluster group including clusters represented by flattened ellipsoids.

[0025] More specifically, the plane estimation unit 110 may divide the point group included in the point group data into a plurality of partial point groups according to the following procedure, and perform clustering on each of the divided partial point groups to generate a cluster group corresponding to the microplane group constituting the object surface.

[0026] Specifically, the plane estimation unit 110 first determines whether to divide the point group included in the point group data based on a predetermined division condition. When the point group satisfies the predetermined division condition, the plane estimation unit 110 divides the point group into approximately half to generate two partial point groups. Next, the plane estimation unit 110 repeatedly divides the generated partial point groups into approximately half until the above division condition is no longer satisfied. Subsequently, the plane estimation unit 110 determines whether to perform clustering on each of the partial point groups divided until the above division condition is no longer satisfied based on a predetermined clustering condition. Thereafter, the plane estimation unit 110 can generate a cluster group corresponding to the microplane group constituting the object surface by performing clustering on each of the partial point groups that satisfy the predetermined clustering condition.

[0027] Here, the above-mentioned predetermined division condition may include a condition that the number of points included in the point cloud is equal to or greater than a predetermined number. Further, the predetermined division condition may include a condition that when setting the smallest rectangular parallelepiped region (also referred to as a bounding box) that encloses all the points of the point cloud in a three-dimensional coordinate system, the length of the longest side of the rectangular parallelepiped region is equal to or greater than a predetermined length. Further, the predetermined division condition may include a condition that the density of points in the point cloud is equal to or greater than a predetermined value. The density of points in the point cloud may be, for example, a value obtained by dividing the number of points included in the point cloud by the size of the above-mentioned bounding box. Further, the predetermined division condition may include a condition related to the shape of the point cloud. Furthermore, the predetermined division condition may include a condition obtained by combining a plurality of the above-mentioned conditions.

[0028] Further, the predetermined clustering condition may include a condition that the number of points included in the point cloud is equal to or greater than a predetermined number. Further, the predetermined clustering condition may include a condition that the length of the longest side of the above-mentioned bounding box is equal to or greater than a predetermined length. Further, the predetermined clustering condition may include a condition that the density of points in the above-mentioned point cloud is equal to or greater than a predetermined value. Further, the predetermined clustering condition may include a condition related to the shape of the point cloud. Furthermore, the predetermined clustering condition may include a condition obtained by combining a plurality of the above-mentioned conditions.

[0029] Note that the predetermined division condition and the predetermined clustering condition can be appropriately set with optimal conditions based on the characteristics of the point cloud and the like.

[0030] According to this, since the plane estimation unit 110 performs clustering on each of the point clouds after dividing until the predetermined division condition is no longer satisfied, the size of each of the point clouds to be the object of clustering can be reduced. Therefore, the plane estimation unit 110 can further shorten the processing time of clustering. In addition, since the plane estimation unit 110 can exclude point clouds that do not satisfy the predetermined clustering condition and are likely to be inaccurate from the object of clustering, the accuracy of clustering can be further improved.

[0031] The coordinate system conversion unit 160 converts the position coordinates of each cluster in the cluster group generated by the plane estimation unit 110 into relative coordinates with respect to the environmental map. Specifically, the coordinate system conversion unit 160 converts the position coordinates of each cluster into relative coordinates with respect to the environmental map based on the self-position of the moving body 1 estimated by the self-position estimation unit 140 described later. For example, the sensor unit 20 mounted on the moving body 1 has its position fluctuating as the moving body 1 moves. Therefore, the coordinate system conversion unit 160 converts the position coordinates of each cluster based on the point cloud data acquired by the sensor unit 20 into relative coordinates with respect to the environmental map based on the self-position of the moving body 1 on which the sensor unit 20 is mounted. Thereby, even when the position of the sensor unit 20 fluctuates, the information processing apparatus 10 can handle the cluster group based on the point cloud data acquired by the sensor unit 20 on the same coordinates.

[0032] The cluster group update unit 120 updates the position and shape of each cluster in the cluster group corresponding to the microplane group constituting the object surface by using the point cloud data acquired by the sensor unit 20 at any time. Specifically, the cluster group update unit 120 may update the second cluster group by fusing the cluster group (referred to as the first cluster group) obtained by clustering the point cloud data (referred to as the first point cloud data) acquired by the sensor unit 20 with the cluster group (referred to as the second cluster group) generated based on the point cloud data (referred to as the second point cloud data) acquired earlier than the first point cloud data.

[0033] That is, the cluster group update unit 120 can update the position and shape of each cluster in the second cluster group by compressing the information of the point cloud data acquired by the sensor unit 20 through clustering and sequentially fusing the information-compressed cluster group (the first cluster group) with the cluster group (the second cluster group) generated in the past in time series.

[0034] According to this, since the cluster group update unit 120 can use the clusters included in the cluster group as the data handling unit instead of the points included in the point cloud data, the amount of data to be handled and the amount of data calculation can be reduced. Therefore, the information processing apparatus 10 can reflect the point cloud data acquired by the sensor unit 20 in the environmental map with higher efficiency.

[0035] More specifically, the cluster group update unit 120 includes a determination unit 121 and a fusion unit 122. Hereinafter, with reference to FIGS. 4 to 7, the functions of the determination unit 121 and the fusion unit 122 will be described. FIG. 4 is an explanatory diagram for explaining an example of a method for determining a cluster to be fused. FIG. 5 is an explanatory diagram for explaining another example of a method for determining a cluster to be fused. FIG. 6 is an explanatory diagram for explaining a method for estimating the normal direction and thickness of a first cluster and a second cluster to be fused. FIG. 7 is an explanatory diagram showing the normal direction, thickness, and shape of a fused cluster obtained by fusing the first cluster and the second cluster. In FIGS. 4 to 7, each of the clusters represented by the flat ellipsoid as shown in FIG. 3 is represented by an elliptical shape that is a longitudinal cross-section of the flat ellipsoid (that is, a cross-sectional view in the normal direction of the cluster).

[0036] The determination unit 121 determines a combination to be fused between a cluster (referred to as a first cluster) included in the first cluster group and a cluster (referred to as a second cluster) included in the second cluster group.

[0037] As shown in FIG. 4, the determination unit 121 may determine a combination of the first cluster LC to be fused and the second cluster HC based on the positions and shapes of the first cluster LC included in the first cluster group and the positions and shapes of the second cluster HC included in the second cluster group. Specifically, the determination unit 121 determines, based on the angle nθ formed by the normal direction with respect to the flat surface of each of the first cluster LC and the second cluster HC, the normalized distance nd (distance normalized by the cluster thickness) in the normal direction of each of the first cluster LC and the second cluster HC, and the normalized distance cd (distance normalized by the minor axis length of the cluster) in the flat surface direction of each of the first cluster LC and the second cluster HC, a combination of the first cluster LC to be fused and the second cluster HC.

[0038] For example, the determination unit 121 may determine a combination of the first cluster LC and the second cluster HC whose angle nθ formed by the normal direction, the distance nd in the normal direction, and the distance cd in the flat surface direction are each below their respective threshold values as a combination of clusters to be fused. Alternatively, the determination unit 121 may determine a combination of the first cluster LC and the second cluster HC whose angle nθ formed by the normal direction, the distance nd in the normal direction, and the distance cd in the flat surface direction satisfy a predetermined relational expression as a combination of clusters to be fused.

[0039] Further, as shown in FIG. 5, the determination unit 121 may determine a combination of the first cluster LC to be fused and the second cluster HC based on the normalized distance between the first cluster LC included in the first cluster group and the second cluster HC included in the second cluster group. Specifically, the determination unit 121 may determine a combination of the first cluster LC to be fused and the second cluster HC based on the distance normalized by the covariance. For example, the distance normalized by the covariance (Mahalanobis distance) has the center point of the first cluster LC as μ j , the covariance of the first cluster LC as Σ j , the center point of the second cluster HC as μ i , and the covariance of the second cluster HC as Σ i , and can be expressed by the following set of equations 1.

[0040]

Number

[0041] Note that if the determination unit 121 satisfies the determination conditions for the clusters to be fused, it may determine one second cluster HC as the cluster to be fused for a plurality of first clusters LC, or may determine one first cluster LC as the cluster to be fused for a plurality of second clusters HC.

[0042] In addition, the second cluster HC may not have a first cluster LC determined to be a fusion target. As will be described later, a second cluster HC that has not been determined to be a fusion target for the first cluster LC for a predetermined period or more may be discarded when the second cluster group is updated by fusion with the first cluster group.

[0043] Conversely, the first cluster LC may not have a second cluster HC determined to be a fusion target. In such a case, the first cluster LC will be added to the second cluster group without being fused with the second cluster HC.

[0044] The fusion unit 122 fuses the first cluster LC and the second cluster HC determined to be fusion targets based on the shapes of the first cluster LC and the second cluster HC.

[0045] As shown in FIG. 6, the fusion unit 122 first highly accurately estimates the normal direction with respect to the flat surface of the first cluster LC and the thickness in the normal direction (also referred to as the normal direction and thickness LNL of the first cluster LC), and the normal direction with respect to the flat surface of the second cluster HC and the thickness in the normal direction (also referred to as the normal direction and thickness HNL of the second cluster HC). Thereafter, the fusion unit 122 uses the highly accurately estimated normal direction and thickness LNL of the first cluster LC and the normal direction and thickness HNL of the second cluster HC to fuse the first cluster LC and the second cluster HC.

[0046] Specifically, the fusion unit 122 can accurately estimate the normal direction and thickness LNL of the first cluster LC and the normal direction and thickness HNL of the second cluster HC by using a Kalman filter.

[0047] For example, the fusion unit 122 may define the parameters of the Kalman filter by using the cluster components shown below, and estimate the normal direction and thickness LNL of the first cluster LC and the normal direction and thickness HNL of the second cluster HC based on the Kalman filter prediction equation shown in Equation Group 2 and the Kalman filter update equation shown in Equation Group 3. x t : Normal vector of the second cluster HC (σ x ) 2 t : Thickness in the normal direction of the second cluster HC z t : Normal vector of the first cluster LC σ z 2 : Thickness in the normal direction of the first cluster LC σ w 2 : System noise (hyperparameter)

[0048]

Number

[0049]

Number

[0050] Thereafter, as shown in FIG. 7, the fusion unit 122 derives the normal direction and thickness UNL of the fused cluster UC based on the estimated normal direction and thickness LNL of the first cluster LC and the normal direction and thickness HNL of the second cluster HC. According to this, the fusion unit 122 can update the position and shape of the second cluster HC included in the second cluster group in the fused cluster UC obtained by fusing the first cluster LC included in the first cluster group and the second cluster HC included in the second cluster group.

[0051] Specifically, the fusion unit 122 may fuse the first cluster LC and the second cluster HC through a simple fusion process based on the cluster shape. For example, the center point of the first cluster LC is μ j , the covariance of the first cluster LC is Σ j , the center point of the second cluster HC is μ i , the covariance of the second cluster HC is Σ i , then the center point μ merged and the covariance Σ merged of the fused cluster UC can be represented by the following set of equations 4.

[0052]

Equation

[0053] Alternatively, the fusion unit 122 may fuse the first cluster LC and the second cluster HC through a fusion process that includes a pair of fused clusters based on the cluster shape. For example, the center point of the first cluster LC is μ j , the covariance of the first cluster LC is Σ j , the center point of the second cluster HC is μ i , the covariance of the second cluster HC is Σ i , then the center point μ merged and the covariance Σ merged of the fused cluster UC can be represented by the following set of equations 5.

[0054]

Equation

[0055] Alternatively, the fusion unit 122 may fuse the first cluster LC and the second cluster HC through a fusion process using cluster weights. Specifically, the fusion unit 122 may use the number of point groups in the point group data used for generating the first cluster group and the second cluster group as the cluster weights to fuse the first cluster LC and the second cluster HC. For example, the center point of the first cluster LC is μ j, let the covariance of the first cluster LC be Σ j , and let the center point of the second cluster HC be μ i , and let the covariance of the second cluster HC be Σ i , then the center point μ merged , and the covariance Σ merged of the fused cluster UC can be expressed by the following set of equations 6. Here, w i is the number of point clouds in the point cloud data used for generating the first cluster group containing the first cluster LC, and w j is the number of point clouds in the point cloud data used for generating the second cluster group containing the second cluster HC.

[0056]

Equation

[0057] Furthermore, the fusion unit 122 may fuse the first cluster LC and the second cluster HC through a fusion process using cluster reliability. Specifically, the fusion unit 122 may fuse the first cluster LC and the second cluster HC using the reliability of the sensor unit 20 when acquiring the point cloud data used for generating the first cluster group and the second cluster group as the cluster weight. For example, let the center point of the first cluster LC be μ j , and the covariance of the first cluster LC be Σ j , and let the center point of the second cluster HC be μ i , and the covariance of the second cluster HC be Σ i , then the center point μ merged , and the covariance Σ merged of the fused cluster UC can be expressed by the following set of equations 7. Here, c i is the reliability when acquiring the point cloud data used for generating the first cluster group containing the first cluster LC, and c j is the reliability when acquiring the point cloud data used for generating the second cluster group containing the second cluster HC.

[0058]

Equation

[0059] Here, for the second cluster HC included in the second cluster group that was not a fusion target with the first cluster LC, the fusion unit 122 may continue to hold it in the second cluster group until a predetermined period. After that, for the second cluster HC that was not a fusion target with the first cluster LC for a period longer than the predetermined period, the fusion unit 122 may discard it when updating the second cluster group. Also, for the first cluster LC included in the first cluster group that was not a fusion target with the second cluster HC, the fusion unit 122 may directly add it to the second cluster group.

[0060] The object recognition unit 130 recognizes an object corresponding to one or more grouped clusters by grouping one or more clusters included in the cluster group updated by the cluster group update unit 120. For example, the object recognition unit 130 may recognize an object existing around the moving body 1 based on the shape, arrangement, or mutual interval of each cluster included in the cluster group. Alternatively, the object recognition unit 130 may recognize an object existing around the moving body 1 by associating the shape and arrangement of each cluster included in the cluster group with the captured image acquired by the imaging device included in the sensor unit 20.

[0061] Furthermore, the object recognition unit 130 constructs an environmental map representing each cluster using the center coordinates and the probability distribution shape based on the recognized object. Note that the object recognition unit 130 may construct an environmental map in a two-dimensional plane or a three-dimensional space. By constructing an environmental map using a cluster represented by an oblate ellipsoid, the object recognition unit 130 can reduce the data amount of the environmental map compared to the case of constructing an environmental map using a so-called grid map. According to this, the object recognition unit 130 can construct an environmental map in a short time even with fewer computing resources.

[0062] The self-position estimation unit 140 estimates the position of the mobile body 1 in the environmental map based on the information about the environment around the mobile body 1 acquired by the sensor unit 20. Further, the self-position estimation unit 140 generates self-position data indicating the estimated position of the mobile body 1 and outputs it to the coordinate system conversion unit 160 and the action planning unit 150.

[0063] Alternatively, the self-position estimation unit 140 may estimate the position of the mobile body 1 based on the sensing results of sensors that measure the state of the mobile body 1. For example, the self-position estimation unit 140 may estimate the position of the mobile body 1 by calculating the moving direction and moving distance of the mobile body 1 based on the sensing results of encoders provided at each joint of the legs included in the moving mechanism 32 of the mobile body 1. For example, the self-position estimation unit 140 may estimate the position of the mobile body 1 by calculating the moving direction and moving distance of the mobile body 1 based on the sensing results of encoders provided on each wheel included in the moving mechanism 32 of the mobile body 1. For example, the self-position estimation unit 140 may estimate the position of the mobile body 1 by calculating the moving direction and moving distance of the mobile body 1 based on the sensing results of an IMU (Inertial Measurement Unit) having a three-axis gyro sensor and a three-direction accelerometer provided on the mobile body 1.

[0064] Furthermore, the self-position estimation unit 140 may estimate the position of the mobile body 1 based on information acquired by other sensors such as a GNSS (Global Navigation Satellite System) sensor.

[0065] The action planning unit 150 creates an action plan for the mobile body 1 by grasping the situation around the mobile body 1 based on the information acquired from the sensor unit 20 and the like. Specifically, the action planning unit 150 may plan a movement route to the destination of the mobile body 1 based on the environmental map around the mobile body 1 and the self-position data of the mobile body 1.

[0066] The drive control unit 31 controls the drive of the movement mechanism 32 to move the moving body 1 along the movement path planned by the action planning unit 150. For example, the drive control unit 31 may control the movement mechanism 32 so that the difference between the position of the moving body 1 on the movement path planned by the action planning unit 150 at a predetermined time and the actual position of the moving body 1 becomes small, thereby moving the moving body 1 along the movement path.

[0067] The movement mechanism 32 is, for example, a mechanism that enables the moving body 1 to move on the ground, on water, underwater, or in the air. For example, the movement mechanism 32 may be a movement mechanism that enables travel on the ground such as two-wheeled or four-wheeled wheels, a movement mechanism that enables walking on the ground such as two-legged or four-legged legs, a movement mechanism that enables flight in the air such as a propeller or a rotary wing, or a movement mechanism that enables movement on water or underwater such as a screw.

[0068] According to the above configuration, the information processing apparatus 10 according to the present embodiment can compress the point cloud data obtained by sensing the surrounding environment by clustering, and can represent the surrounding environment by the clustered cluster group. According to this, the information processing apparatus 10 can reduce the amount of data to be handled and the calculation load.

[0069] Further, the information processing apparatus 10 according to the present embodiment can sequentially fuse clusters using the normal direction and thickness estimated with higher accuracy by estimating the normal direction and thickness of the cluster by a Kalman filter. According to this, the information processing apparatus 10 can create an environmental map in which the environment around the moving body 1 is represented with higher accuracy.

[0070] Therefore, the information processing apparatus 10 according to the present embodiment can shorten the processing time of the recognition process of the surrounding environment, and thus can improve the response speed of the moving body 1. In addition, since the information processing apparatus 10 can improve the recognition accuracy with respect to the surrounding environment, it is possible to plan an efficient movement path with less margin for the objects existing in the surrounding environment.

[0071] (1.2. Operation Example) Next, with reference to FIG. 8, the flow of the operation of the information processing apparatus 10 according to the present embodiment will be described. Hereinafter, the flow of the operation will be described focusing on the cluster group update unit 120 among the components included in the information processing apparatus 10. FIG. 8 is a flowchart showing an example of the operation of the cluster group update unit 120.

[0072] As shown in FIG. 8, first, the determination unit 121 acquires the data of the first cluster group obtained by clustering the point cloud data acquired by the sensor unit 20 (S101). Next, the determination unit 121 searches for and determines each of the first clusters included in the first cluster group to be fused and the second clusters included in the second cluster group (S102).

[0073] Here, the second cluster group is a cluster group generated earlier than the first cluster group. That is, the second cluster group is a cluster group generated by clustering the second point cloud data acquired earlier than the first point cloud data used for generating the first cluster group, or a cluster group obtained by further updating the cluster group.

[0074] Specifically, the determination unit 121 determines whether or not there is a second cluster to be fused for each of the first clusters included in the first cluster group (S103). The first clusters (S103 / Yes) for which there is a second cluster to be fused are fused with the second clusters to be fused in the step of cluster fusion processing (S108) after the estimation of the normal direction and thickness (S107) described later.

[0075] On the other hand, the first clusters (S103 / No) for which there is no second cluster to be fused are directly added to the second cluster group in the step of cluster addition (S109).

[0076] Further, the determination unit 121 determines whether or not there is a first cluster to be fused for each of the second clusters included in the second cluster group (S104). For a second cluster for which there is a first cluster to be fused (S104 / Yes), after estimating the normal direction and thickness (S107), it is fused with the first cluster to be fused in the step of fusing the subsequent-stage clusters (S108).

[0077] On the other hand, for a second cluster for which there is no first cluster to be fused (S104 / No), it is further determined whether or not it is outside the fusion target for a predetermined time or more (S105). If the second cluster is outside the fusion target for a predetermined time or more (S105 / Yes), the corresponding second cluster is discarded (S106). On the other hand, if the second cluster is outside the fusion target for less than the predetermined time, the corresponding second cluster is not discarded and is added to the second cluster group as it is in the step of cluster addition (S109).

[0078] Thereafter, the fusion unit 122 estimates the normal direction and thickness of the first cluster and the second cluster that are the fusion targets using a Kalman filter, respectively (S107). Next, the fusion unit 122 fuses the first cluster and the second cluster based on the estimated normal direction and thickness (S108). Further, the fusion unit 122 updates the second cluster group by adding the first cluster and the second cluster that bypassed the fusion process to the second cluster after fusion (S109) (S110).

[0079] Thereafter, the cluster group update unit 120 newly acquires the first cluster group obtained by clustering the point cloud data newly acquired by the sensor unit 20 (S101), and repeatedly executes the update of the second cluster group.

[0080] According to the above operation, the information processing apparatus 10 according to the present embodiment can improve the recognition accuracy of the surrounding environment by sequentially fusing the cluster groups representing the surrounding environment in time series.

[0081] (1.3. Modification example) Next, with reference to FIGS. 9 to 14, a mobile body including an information processing apparatus according to a modification of the present embodiment will be described. FIG. 9 is a block diagram showing an example of the functional configuration of a mobile body 1A including an information processing apparatus 10A according to this modification.

[0082] As shown in FIG. 9, in the information processing apparatus 10A according to this modification, the self-position estimation unit 140A can estimate the self-position of the mobile body 1A with higher accuracy by further using the first cluster group and the second cluster group. Specifically, the self-position estimation unit 140A may estimate the self-position of the mobile body 1A based on the flowchart shown in FIG. 10. FIG. 10 is a flowchart showing the flow of estimating the self-position of the mobile body 1A by the self-position estimation unit 140A.

[0083] As shown in FIG. 10, first, the self-position estimation unit 140A sets a reference for the map coordinates of the environmental map (S201). Specifically, the self-position estimation unit 140 may estimate the initial position of the mobile body 1A in the absolute coordinates on the earth acquired by the GNSS sensor, and use the absolute coordinates of the mobile body 1A as the reference for the map coordinates (earth-centered coordinate system). Alternatively, the self-position estimation unit 140A may use the initial position of the mobile body 1A as the zero point as the reference for the map coordinates (relative coordinate system).

[0084] Subsequently, the self-position estimation unit 140A estimates the self-position and orientation of the mobile body 1A based on the sensing results of, for example, an IMU or the like (S202). Specifically, the self-position estimation unit 140A may estimate the self-position and orientation of the mobile body 1A using the sensing results of the IMU during the period until the point cloud data is acquired by the sensor unit 20 with reference to the position and orientation of the initial position of the mobile body 1A. Alternatively, the self-position estimation unit 140A may estimate the self-position and orientation of the mobile body 1A using the sensing results of the IMU for the displacement from time t - 1 to time t with reference to the position and orientation of the mobile body 1A at time t - 1 (one time step before).

[0085] Furthermore, the self-position estimation unit 140A may estimate the self-position and attitude of the mobile body 1A using the amount of movement of the mobile body 1A estimated from an encoder or the like included in the movement mechanism 32 of the mobile body 1. Also, the self-position estimation unit 140A may correct the result of self-position estimation using geomagnetic data or GNSS data.

[0086] Next, the self-position estimation unit 140A performs self-position estimation of the mobile body 1A using the cluster group, with the position and attitude (time t) of the mobile body 1A estimated in S202 as the initial values (S203). Specifically, the self-position estimation unit 140A performs matching between the cluster groups, using the first cluster group based on the point cloud data acquired at time t as the source and the second cluster group that was fusion-processed one time step before as the target, thereby enabling more accurate estimation of the self-position and attitude of the mobile body 1A. After that, the self-position estimation unit 140A outputs the self-position and attitude of the mobile body 1A at time t estimated with high accuracy (S204).

[0087] Note that the self-position estimation unit 140A may skip the step of self-position estimation using the IMU (S202) and execute only the step of self-position estimation using the cluster group (S203). In such a case, the self-position estimation unit 140A may use the position and attitude of the mobile body 1A at the time one time step before (time t - 1) as the initial values for self-position estimation using the cluster group.

[0088] With reference to FIGS. 11 to 14, the first to fourth examples of the method for matching between cluster groups performed by the self-position estimation unit 140A will be described in more detail.

[0089] FIG. 11 is an explanatory diagram for explaining a first example of self-position estimation of the mobile body 1A based on the first cluster group and the second cluster group. The self-position estimation unit 140A can estimate the self-position and attitude of the mobile body 1A by performing matching between the cluster groups using Point To Plane ICP of the cluster groups. The first example of self-position estimation is a method suitable when the surface of the object existing in the external environment is mainly composed of planes.

[0090] As shown in FIG. 11, first, the self-position estimation unit 140A defines the specifications of the cluster group as follows. p1, p2,... p n : Center point of the source cluster group q1, q2,... q n : Center point of the target cluster group n1, n2,... n n : Normal of the target cluster group

[0091] Here, the self-position estimation unit 140A estimates R and t such that the loss function E represented by the following mathematical formula 8 is minimized. Since the rotation matrix R corresponds to the estimated attitude of the moving body 1A and the translation matrix t corresponds to the estimated position of the moving body 1A, the self-position estimation unit 140A can estimate the self-position and attitude of the moving body 1A by estimating R and t.

[0092]

Equation

[0093] FIG. 12 is an explanatory diagram for explaining a second example of the self-position estimation of the moving body 1A based on the first cluster group and the second cluster group. The self-position estimation unit 140A can estimate the self-position and attitude of the moving body 1A by performing matching between the cluster groups using the Symmetric ICP of the cluster group. The second example of the self-position estimation is a suitable method when the object surface existing in the external environment is mainly a curved surface.

[0094] As shown in FIG. 12, first, the self-position estimation unit 140A defines the specifications of the cluster group as follows. p1, p2,... p n : Center point of the source cluster group n1 p 、n2 p 、... n n p : Normal of the source cluster group q1, q2,... q n : Center point of the target cluster group n1 q and n2 q ...n n q : Normal of the target cluster group

[0095] Here, the self - position estimation unit 140A estimates R and t such that the loss function E represented by the following mathematical formula 9 is minimized. Since the rotation matrix R corresponds to the estimated attitude of the moving body 1A and the translation matrix t corresponds to the estimated position of the moving body 1A, the self - position estimation unit 140A can estimate the self - position and attitude of the moving body 1A by estimating R and t.

[0096]

Equation

[0097] FIG. 13 is an explanatory diagram for explaining a third example of self - position estimation of the moving body 1A based on the first cluster group and the second cluster group. The self - position estimation unit 140A can estimate the self - position and attitude of the moving body 1A by performing matching between the cluster groups using Point To Plane ICP considering the covariance of the cluster groups. The third example of self - position estimation is a suitable method when the object surface existing in the external environment is mainly flat and the covariance is large.

[0098] As shown in FIG. 13, first, the self - position estimation unit 140A defines the specifications of the cluster group as follows. p1, p2,...p n : Center point of the source cluster group q1, q2,...q n : Center point of the target cluster group n1, n2,...n n : Normal of the target cluster group C1, C2,...C n : Covariance of the target cluster group

[0099] Here, the self-position estimation unit 140A estimates R and t such that the loss function E represented by the following mathematical formula 10 is minimized. Since the rotation matrix R corresponds to the estimated attitude of the moving body 1A and the translation matrix t corresponds to the estimated position of the moving body 1A, the self-position estimation unit 140A can estimate the self-position and attitude of the moving body 1A by estimating R and t.

[0100]

Number

[0101] FIG. 14 is an explanatory diagram for explaining a fourth example of the self-position estimation of the moving body 1A based on the first cluster group and the second cluster group. The self-position estimation unit 140A can estimate the self-position and attitude of the moving body 1A by performing matching between the cluster groups using Symmetric ICP that takes into account the covariance of the cluster groups. The fourth example of self-position estimation is a method suitable when the object surface existing in the external environment is mainly a curved surface and the covariance is large.

[0102] As shown in FIG. 14, first, the self-position estimation unit 140A defines the specifications of the cluster groups as follows. p1, p2,... p n : Center point of the source cluster group n1 p 、n2 p 、... n n p : Normal of the source cluster group C1 p 、C2[[ID=,34]] p 、... C n p : Covariance of the source cluster group q1, q2,... q n : Center point of the target cluster group n1 q に、n2 q 、... n<, n q : Normal of the target cluster group C1: q 、C2 q 、... C nq : Covariance of the target cluster group

[0103] Here, the self-position estimation unit 140A estimates R and t such that the loss function E represented by the following mathematical formula 11 is minimized. Since the rotation matrix R corresponds to the estimated posture of the moving body 1A and the translation matrix t corresponds to the estimated position of the moving body 1A, the self-position estimation unit 140A can estimate the self-position and posture of the moving body 1A by estimating R and t.

[0104]

Equation

[0105] The above method of matching cluster groups can also be performed in combination with each other. Further, the self-position estimation unit 140A may estimate the self-position and posture of the moving body 1A by collectively performing the self-position estimation executed in steps S202 and S203 using the method of Bundle Ajustment.

[0106] <2. Second Embodiment> (2.1. Configuration Example) Next, with reference to FIG. 15, a moving body including the information processing apparatus according to the second embodiment of the present disclosure will be described. FIG. 15 is a block diagram showing an example of the functional configuration of the moving body 2 including the information processing apparatus 11 according to the present embodiment.

[0107] As shown in FIG. 15, the moving body 2 includes a sensor unit 20 including a plurality of sensors 21 and 22, the information processing apparatus 11 according to the present embodiment, a drive control unit 31, and a moving mechanism 32. The information processing apparatus 11 includes a plurality of plane estimation units 111 and 112, a plurality of coordinate system conversion units 161 and 162, a cluster group update unit 120A, an object recognition unit 130, a self-position estimation unit 140, and an action planning unit 150.

[0108] Note that the object recognition unit 130, the self-position estimation unit 140, the action planning unit 150, the drive control unit 31, and the movement mechanism 32 are substantially the same as the respective configurations described in the first embodiment, and thus the description thereof is omitted here.

[0109] The mobile body 2 including the information processing apparatus 11 according to the present embodiment includes a sensor unit 20 including a plurality of sensors 21 and 22 that sense the environment around the mobile body 2. The sensor unit 20 includes a plurality of sensors 21 and 22 that are the same type or different types of sensors, and outputs the sensing result of the environment around the mobile body 2 as point cloud data. Specifically, the plurality of sensors 21 and 22 may be ranging sensors such as ultrasonic sensors, ToF sensors, RADAR, or LiDAR sensors, or imaging devices such as stereo cameras, monocular cameras, color cameras, infrared cameras, spectroscopic cameras, or polarization cameras. For example, the plurality of sensors 21 and 22 may be a combination of one system of stereo cameras and one system of ToF sensors.

[0110] The point cloud data acquired by the plurality of sensors 21 and 22 is individually clustered by the plane estimation units 111 and 112, and then converted into the same coordinate system by the coordinate system conversion units 161 and 162. That is, the point cloud data acquired by the sensor 21 is converted into a cluster group by being clustered by the plane estimation unit 111, and the coordinate system is converted into the coordinate system of the environmental map by the coordinate system conversion unit 161. Further, the point cloud data acquired by the sensor 22 is converted into a cluster group by being clustered by the plane estimation unit 112, and the coordinate system is converted into the coordinate system of the environmental map by the coordinate system conversion unit 162.

[0111] Specifically, the coordinate system conversion units 161 and 162 convert the position coordinates of each cluster included in the cluster group into relative coordinates with respect to the environmental map based on the self-position of the mobile body 2 in order to correct the positions of the sensors 21 and 22 that vary as the mobile body 2 moves. Further, the coordinate system conversion units 161 and 162 convert the coordinate systems of the cluster groups based on the point cloud data acquired by each of the sensors 21 and 22 into the same coordinate system in order to correct the difference in the reference positions of the sensors 21 and 22 with different mounting positions.

[0112] According to the coordinate system conversion units 161 and 162, even the point cloud data acquired by different types of sensors, namely sensors 21 and 22, can be converted into the same coordinate system. Therefore, the cluster group update unit 120A can sequentially fuse in time series the cluster groups obtained by clustering the point cloud data output from the sensor unit 20 regardless of the types of sensors included in the sensor unit 20.

[0113] The cluster group update unit 120A is different from the cluster group update unit 120 described in the first embodiment in that it further includes a sequencing unit 123.

[0114] The sequencing unit 123 rearranges in time series the first cluster group obtained by clustering the point cloud data acquired by each of the sensors 21 and 22. Specifically, regardless of which of the sensors 21 and 22 the point cloud data serving as the basis of the first cluster group is acquired from, the sequencing unit 123 rearranges the first cluster group in time series based on the time when the point cloud data is acquired. According to this, the determination unit 121 and the fusion unit 122 subsequent to the sequencing unit 123 can fuse the first cluster group reflecting the temporal change of the environment around the moving body 2 into the second cluster group in time series.

[0115] In the information processing apparatus 11 according to the present embodiment, the data of the first cluster group obtained by clustering the point cloud data acquired by the plurality of sensors 21 and 22 are separately input to the cluster group update unit 120A. Therefore, in the information processing apparatus 11, a sequencing unit 123 for rearranging the first cluster group in time series across the plurality of sensors 21 and 22 is provided. According to this, the determination unit 121 and the fusion unit 122 subsequent to the sequencing unit 123 can update the second cluster group by fusing the first cluster group into the second cluster group as described in the first embodiment in the order rearranged by the sequencing unit 123.

[0116] Since the determination unit 121 and the fusion unit 122 are substantially the same as each configuration described in the first embodiment, the description thereof is omitted here.

[0117] The information processing apparatus 11 according to the present embodiment can reflect all the point cloud data acquired by the plurality of sensors 21 and 22 in the environmental map representing the environment around the moving body 2, and thus can construct a more accurate environmental map.

[0118] In FIG. 15, an example in which the sensor unit 20 includes two systems of sensors 21 and 22 is shown. However, the technology according to the present embodiment is not limited to the above example. The number of sensors included in the sensor unit 20 is not particularly defined as long as it is two or more.

[0119] (2.2. Modification example) Next, with reference to FIG. 16, a moving body including the information processing apparatus according to the modification example of the present embodiment will be described. FIG. 16 is a block diagram showing an example of the functional configuration of a moving body 2A including the information processing apparatus 11A according to the present modification example.

[0120] As shown in FIG. 16, in the information processing apparatus 11A according to the modification example, similar to the information processing apparatus 10A according to the modification example of the first embodiment, the self-position estimation unit 140A can estimate the self-position of the moving body 2A with higher accuracy by further using the first cluster group and the second cluster group. Since the specific operation of the self-position estimation unit 140A is substantially the same as that of the modification example of the first embodiment, the detailed description thereof is omitted here.

[0121] <3. Hardware configuration example> Furthermore, with reference to FIG. 17, the hardware configurations of the information processing apparatuses 10, 10A, 11, and 11A according to an embodiment of the present disclosure will be described. FIG. 17 is a block diagram showing an example of the hardware configurations of the information processing apparatuses 10, 10A, 11, and 11A according to the present embodiment.

[0122] The functions of the information processing apparatuses 10, 10A, 11, and 11A according to the present embodiment are realized by the cooperation of software and the hardware described below. For example, the functions of the above-described plane estimation units 110, 111, 112, coordinate system conversion units 160, 161, 162, cluster group update units 120, 120A, object recognition unit 130, self-position estimation units 140, 140A, and action planning unit 150 may be executed by the CPU 901.

[0123] As shown in FIG. 17, the information processing apparatuses 10, 10A, 11, and 11A include a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 903, and a RAM (Random Access Memory) 905.

[0124] Further, the information processing apparatuses 10, 10A, 11, and 11A may further include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925. Further, the information processing apparatuses 10, 10A, 11, and 11A may have another processing circuit such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) instead of or together with the CPU 901.

[0125] The CPU 901 functions as an arithmetic processing device or a control device, and controls the overall operation of the information processing apparatuses 10, 10A, 11, and 11A according to various programs recorded in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 927. The ROM 903 stores programs used by the CPU 901, arithmetic parameters, and the like. The RAM 905 temporarily stores programs used in the execution of the CPU 901 and parameters used in the execution thereof.

[0126] The CPU 901, ROM 903, and RAM 905 are interconnected by a host bus 907 configured by an internal bus such as a CPU bus. Further, the host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909.

[0127] The input device 915 is a device that receives input from a user, such as a mouse, keyboard, touch panel, button, switch, or lever. The input device 915 may be a microphone that detects the user's voice. Also, the input device 915 may be, for example, a remote control device that uses infrared rays or other radio waves, or an external connection device 929 corresponding to the operation of the information processing device 10.

[0128] The input device 915 further includes an input control circuit that outputs an input signal generated based on the information input by the user to the CPU 901. The user can input various data or give instructions for processing operations to the information processing devices 10, 10A, 11, 11A by operating the input device 915.

[0129] The output device 917 is a device capable of visually or auditorily presenting the information acquired or generated by the information processing devices 10, 10A, 11, 11A to the user. The output device 917 may be, for example, a display device such as an LCD (Liquid Crystal Display), PDP (Plasma Display Panel), OLED (Organic Light Emitting Diode) display, hologram, or projector. Also, the output device 917 may be a sound output device such as a speaker or headphones, or a printing device such as a printer device. The output device 917 may output the information obtained by the processing of the information processing devices 10, 10A, 11, 11A as video such as text or images, or as sound such as voice or audio.

[0130] Storage device 919 is a data storage device configured as an example of the storage units of information processing devices 10, 10A, 11, and 11A. The storage device 919 may be configured by, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 can store programs executed by the CPU 901, various data, or various data acquired from the outside.

[0131] Drive 921 is a device for reading from or writing to a removable recording medium 927 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built in or externally attached to the information processing devices 10, 10A, 11, and 11A. For example, the drive 921 can read the information recorded on the attached removable recording medium 927 and output it to the RAM 905. Also, the drive 921 can write a recording to the attached removable recording medium 927.

[0132] Connection port 923 is a port for directly connecting an external connection device 929 to the information processing devices 10, 10A, 11, and 11A. The connection port 923 may be, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, or a SCSI (Small Computer System Interface) port. Also, the connection port 923 may be an RS-232C port, an optical audio terminal, or an HDMI (registered trademark) (High-Definition Multimedia Interface) port. When the connection port 923 is connected to the external connection device 929, various data can be transmitted and received between the information processing devices 10, 10A, 11, and 11A and the external connection device 929.

[0133] The communication device 925 is a communication interface composed of, for example, a communication device for connecting to a communication network 931. The communication device 925 may be, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). Further, the communication device 925 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

[0134] The communication device 925 can transmit and receive signals, etc. using a predetermined protocol such as TCP / IP, for example, between the Internet or other communication devices. The communication network 931 connected to the communication device 925 is a network connected by wire or wirelessly, and may be, for example, an Internet communication network, a home LAN, an infrared communication network, a radio wave communication network, or a satellite communication network.

[0135] Note that a program for causing hardware such as the CPU 901, ROM 903, and RAM 905 built in a computer to exhibit functions equivalent to those of the information processing apparatuses 10, 10A, 11, and 11A described above can also be created. Further, a computer-readable recording medium recording the program can also be provided.

[0136] As described above, the technology according to the present disclosure has been described by way of the first and second embodiments and modified examples. However, the technology according to the present disclosure is not limited to the above-described embodiments and the like, and various modifications are possible. For example, the technology according to the present disclosure can be applied not only to path planning in a three-dimensional space environmental map but also to path planning in a two-dimensional plane environmental map.

[0137] Furthermore, not all of the configurations and operations described in each embodiment are essential to the configuration and operation of the present disclosure. For example, among the components in each embodiment, components not described in the independent claims indicating the highest-level concept of the present disclosure should be understood as optional components. That is, the information processing apparatus according to the first and second embodiments of the present disclosure only needs to include at least a fusion unit.

[0138] The terms used throughout this specification and the appended claims should be construed as "non-limiting" terms. For example, the terms "comprising" or "included" should be construed as not being limited to the manner described as being included. The term "having" should be construed as not being limited to the manner described as having.

[0139] The terms used in this specification include terms used merely for convenience of explanation and not for the purpose of limiting the configuration and operation. For example, terms such as "right", "left", "up", and "down" merely indicate directions on the drawing being referred to. Also, the terms "inside" and "outside" merely indicate the direction toward the center of the element of interest and the direction away from the center of the element of interest, respectively. The same applies to similar terms and terms of the same purport.

[0140] Note that the technology according to the present disclosure can also adopt the following configuration. According to the technology according to the present disclosure having the following configuration, for example, by sequentially fusing in time series a cluster group obtained by clustering point cloud data acquired by a sensor with a cluster group generated in the past, the cluster group corresponding to the microplane group on the surface of an object existing in the surrounding environment can be sequentially updated. Therefore, the technology according to the present disclosure can update the environmental map representing the surrounding environment with higher efficiency. The effects achieved by the technology according to the present disclosure are not necessarily limited to the effects described herein, and any effect described in the present disclosure may be applicable. (1) A fusion unit that updates the second cluster group by fusing each of the first clusters included in the first cluster group obtained by clustering the first point cloud data acquired by the sensor with each of the second clusters included in the second cluster group generated based on the second point cloud data acquired prior to the first point cloud data. An information processing apparatus. (2) The fusion unit fuses each of the first clusters with each of the second clusters based on at least one of the shapes of the first cluster and the second cluster, the weights based on the number of point clouds of the point cloud data used for generating the first cluster and the second cluster, or the reliability of the sensor when acquiring the first point cloud data and the second point cloud data. The information processing apparatus according to (1) above. (3) The shapes of the first cluster and the second cluster are oblate ellipsoids. The fusion unit fuses each of the first clusters with each of the second clusters based on the normal direction with respect to the flat plane of the oblate ellipsoid and the thickness in the normal direction of the oblate ellipsoid. The information processing apparatus according to (2) above. (4) The fusion unit estimates the shapes of the first cluster and the second cluster to be fused based on a Kalman filter. The information processing apparatus according to (3) above. (5) The information processing apparatus according to any one of (1) to (4) above, further comprising a determination unit that determines each of the second clusters to be fused for each of the first clusters. (6) The shapes of the first cluster and the second cluster are oblate ellipsoids. The determination unit determines each of the second clusters to be fused for each of the first clusters based on the distance and angle in the normal direction with respect to the flat plane of the oblate ellipsoid, and the distance in the plane direction of the oblate ellipsoid. The information processing apparatus according to (5) above. flat <00006... (7) The first point cloud data is acquired by a plurality of sensors. The fusion unit sequentially fuses a plurality of the first cluster groups obtained by clustering the first point cloud data acquired by the plurality of sensors into the second cluster group in time series, and the information processing apparatus according to any one of (1) to (6) above. (8) The plurality of sensors include sensors of the same type or different types, and the information processing apparatus according to (7) above. (9) The plurality of the first cluster groups are coordinate-transformed into the same coordinate system, and the information processing apparatus according to (7) or (8) above. (10) The sensor is mounted on a moving body on which an action plan is created based on an environmental map, The information processing apparatus according to any one of (1) to (9) above further includes a coordinate conversion unit that converts the position coordinates of the first cluster included in the first cluster group into relative coordinates with respect to the environmental map. (11) The information processing apparatus further includes a self-position estimation unit that estimates the self-position of the moving body, The coordinate system The conversion unit converts the position coordinates of the first cluster into the relative coordinates with respect to the environmental map based on the estimated self-position of the moving body, and the information processing apparatus according to (10) above. (12) The self-position estimation unit estimates the self-position of the moving body based on the first cluster and the second cluster, and the information processing apparatus according to (11) above. (13) The fusion unit discards a second cluster in which the first cluster has not been fused for a predetermined period or more among the second clusters included in the second cluster group, and updates the second cluster group, and the information processing apparatus according to any one of (1) to (12) above. (14) The first point cloud data and the second point cloud data are data representing a three-dimensional space, and the information processing apparatus according to any one of (1) to (13) above. (15) The first cluster group is generated by clustering partial point groups obtained by dividing the point groups included in the first point group data until a predetermined division condition is no longer satisfied, and the information processing apparatus according to any one of (1) to (14) above. (16) The predetermined division condition includes at least one or more of a condition related to the number of points included in the point group, a condition related to the size of the region surrounding the point group, a condition related to the density of the points included in the point group, or a condition related to the shape of the point group, and the information processing apparatus according to (15) above. (17) The first cluster group is generated by clustering only the partial point groups that satisfy a predetermined clustering condition, and the information processing apparatus according to (15) or (16) above. (18) The predetermined clustering condition includes at least one or more of a condition related to the number of points included in the point group, a condition related to the size of the region surrounding the point group, a condition related to the density of the points included in the point group, or a condition related to the shape of the point group, and the information processing apparatus according to (17) above. (19) Using an arithmetic processing unit, Each of the first clusters included in the first cluster group obtained by clustering the first point group data acquired by the sensor is fused with each of the second clusters included in the second cluster group generated based on the second point group data acquired earlier than the first point group data, and updating the second cluster group. Information processing method. (20) A computer Function as a fusion unit that updates the second cluster group by fusing each of the first clusters included in the first cluster group obtained by clustering the first point group data acquired by the sensor with each of the second clusters included in the second cluster group generated based on the second point group data acquired earlier than the first point group data. Program.

[0141] This application claims priority based on Japanese Patent Application No. 2020-118065, filed with the Japan Patent Office on July 8, 2020, and incorporates by reference all of the contents of this application into this application.

[0142] Those skilled in the art can conceive of various modifications, combinations, sub-combinations, and changes according to design requirements and other factors, and it is understood that they are included within the scope of the appended claims and the scope of their equivalents.

Claims

1. A fusion unit that updates the second cluster group by fusing each of the first clusters included in the first cluster group obtained by clustering the first point cloud data acquired by a sensor with each of the second clusters included in the second cluster group generated based on the second point cloud data acquired earlier than the first point cloud data; The shapes of the first cluster and the second cluster are flat ellipsoids; The fusion unit fuses each of the first clusters with each of the second clusters based on the normal direction to the flat plane of the flat ellipsoid and the thickness in the normal direction of the flat ellipsoid as the shapes of the first cluster and the second cluster; An information processing apparatus.

2. The fusion unit further fuses each of the first clusters with each of the second clusters based on a weight based on the number of point clouds of the point cloud data used for generating the first cluster and the second cluster, or at least one of the reliabilities of the sensors when the first point cloud data and the second point cloud data are acquired. The information processing apparatus according to claim 1.

3. The fusion unit estimates the shapes of the first cluster and the second cluster to be fused based on a Kalman filter. The information processing apparatus according to claim 1.

4. The information processing apparatus according to claim 1, further comprising a determination unit that determines each of the second clusters to be fused for each of the first clusters.

5. The determination unit determines each of the second clusters to be fused for each of the first clusters based on the distance and angle in the normal direction to the flat plane of the flat ellipsoid and the distance in the flat plane direction of the flat ellipsoid. The information processing apparatus according to claim 4.

6. The first point cloud data is acquired by a plurality of sensors; The fusion unit sequentially fuses the plurality of first cluster groups obtained by clustering the first point cloud data acquired by the plurality of sensors with the second cluster group in time series. The information processing apparatus according to claim 1.

7. The plurality of sensors include sensors of the same type or different types. The information processing apparatus according to claim 6.

8. The plurality of first cluster groups are coordinate-transformed into the same coordinate system. The information processing apparatus according to claim 6.

9. The sensor is mounted on a moving body on which an action plan is created based on an environmental map. The information processing apparatus according to claim 1, further comprising a coordinate system conversion unit that converts the position coordinates of the first cluster included in the first cluster group into relative coordinates with respect to the environmental map.

10. further comprising a self-position estimation unit that estimates the self-position of the moving body, The coordinate system conversion unit converts the position coordinates of the first cluster into the relative coordinates with respect to the environmental map based on the estimated self-position of the moving body. The information processing apparatus according to claim 9.

11. The self-position estimation unit estimates the self-position of the moving body based on the first cluster and the second cluster. The information processing apparatus according to claim 10.

12. The fusion unit discards the second clusters in the second cluster group that have not been fused with the first cluster for a predetermined period or more, and updates the second cluster group. The information processing apparatus according to claim 1.

13. The first point cloud data and the second point cloud data are data representing a three-dimensional space. The information processing apparatus according to claim 1.

14. The first cluster group is generated by clustering partial point clouds obtained by dividing the point clouds included in the first point cloud data until a predetermined division condition is no longer satisfied. The information processing apparatus according to claim 1.

15. The predetermined division condition includes at least one or more of a condition related to the number of points included in the point cloud, a condition related to the size of the region surrounding the point cloud, a condition related to the density of the points included in the point cloud, or a condition related to the shape of the point cloud. The information processing apparatus according to claim 14.

16. The first cluster group is generated by clustering only the partial point clouds that satisfy a predetermined clustering condition. The information processing apparatus according to claim 14.

17. The predetermined clustering condition includes at least one or more of a condition related to the number of points included in the point cloud, a condition related to the size of the region surrounding the point cloud, a condition related to the density of the points included in the point cloud, or a condition related to the shape of the point cloud. The information processing apparatus according to claim 16.

18. Using an arithmetic processing unit Each of the first clusters included in the first cluster group obtained by clustering the first point cloud data acquired by the sensor is fused with each of the second clusters included in the second cluster group generated based on the second point cloud data acquired earlier than the first point cloud data, thereby updating the second cluster group, The shapes of the first cluster and the second cluster are flat ellipsoids, Based on the normal direction to the flat plane of the flat ellipsoid and the thickness in the normal direction of the flat ellipsoid, each of the first clusters is fused with each of the second clusters as the shapes of the first cluster and the second cluster Information processing method.

19. A computer, Each of the first clusters included in the first cluster group obtained by clustering the first point cloud data acquired by the sensor is fused with each of the second clusters included in the second cluster group generated based on the second point cloud data acquired earlier than the first point cloud data, thereby functioning as a fusion unit that updates the second cluster group, The shapes of the first cluster and the second cluster are flat ellipsoids, Based on the normal direction to the flat plane of the flat ellipsoid and the thickness in the normal direction of the flat ellipsoid, the fusion unit fuses each of the first clusters with each of the second clusters as the shapes of the first cluster and the second cluster Program.

Images