Information processing method and information processing device

Abstract

Provided is an information processing method comprising processing of: obtaining a plurality of pieces of point information including positional information of a point sensed by at least one point group detection sensor (S101); searching for a second point within a predetermined range from a first point in the point group indicated by the plurality of pieces of point information (S102); calculating at least one of a first difference which is a difference in height between the first point and the second point, or a second difference which is an angle formed by a line connecting a reference point with the first point and a line connecting the reference point with the second point (S103); determining, on the basis of at least one of the first difference and the second difference, whether the second point is to be subjected to a subsequent process (S104); and outputting the point information of the second point determined to be subjected to the subsequent process (S105).

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method and an information processing device. [Background technology]

[0002] With the expected widespread use of self-driving vehicles, point cloud detection sensors such as LiDAR (Light Detection and Ranging) are being used as candidates for external recognition sensors to be mounted on self-driving vehicles. Object detection using such point cloud detection sensors generally uses a method that uses a gradient or a normal vector of the gradient relative to the horizontal direction obtained from multiple points sensed by the point cloud detection sensor. For example, Patent Document 1 discloses an object detection method using such a method.

[0003] However, while the vehicle is traveling, a deviation in the position or angle of the point cloud detection sensor (hereinafter simply referred to as deviation of the point cloud detection sensor) may occur due to vibration, distortion of the vehicle, etc., which may cause malfunction in subsequent processing. For example, Patent Document 2 discloses a method of measuring the amount of deviation of the point cloud detection sensor and correcting the point cloud detection sensor based on the measured amount of deviation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-50035 [Patent Document 2] Japanese Patent Publication No. 2020-46411 Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is difficult to accurately measure the deviation of the point cloud detection sensor as disclosed in Patent Document 2. Therefore, there is a risk that the deviation of the point cloud detection sensor may cause malfunction in subsequent processing using the point cloud information.

[0006] Therefore, the present disclosure provides an information processing method and the like that can suppress malfunctions in subsequent processing caused by deviations in the point cloud detection sensor without measuring the deviations. [Means for solving the problem]

[0007] The information processing method according to the present disclosure is an information processing method executed by a computer, and includes a process of acquiring a plurality of point information including position information of points sensed by at least one point cloud detection sensor provided on a moving body, searching for a second point within a predetermined range from a first point in the point cloud indicated by the plurality of point information, calculating at least one of a first difference which is the difference in elevation between the first point and the second point, or a second difference which is the angle formed between a line connecting a reference point and the first point and a line connecting the reference point and the second point, determining whether or not to subject the second point to subsequent processing based on at least one of the first difference or the second difference, and outputting point information of the second point determined to be subject to subsequent processing.

[0008] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0009] According to the information processing method and the like according to one aspect of the present disclosure, it is possible to suppress malfunctions in subsequent processing caused by deviations of the point cloud detection sensor without measuring the deviations. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining the principle of obstacle detection by a point cloud detection sensor. [Figure 2] FIG. 2 is a diagram for explaining erroneous detection of an obstacle due to positional deviation of the point cloud detection sensor. [Figure 3]FIG. 3 is a diagram for explaining erroneous detection of an obstacle due to an angle deviation of the point cloud detection sensor. [Figure 4] FIG. 4 is a block diagram illustrating an example of an information processing device according to the embodiment. [Figure 5] FIG. 5 is a flowchart illustrating an example of an information processing method according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of an operation of the information processing device according to the embodiment when detecting an obstacle. [Figure 7] FIG. 7 is a flowchart showing a specific example of the operation of the neighboring point search unit according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of neighboring points and non-neighboring points. [Figure 9] FIG. 9 is a flowchart showing a specific example of the operation of the elevation difference determining unit according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of valid neighboring points and invalid neighboring points based on the determination of the elevation difference. [Figure 11] FIG. 11 is a flowchart showing a specific example of the operation of the elevation angle difference determining unit according to the embodiment. [Figure 12] FIG. 12 is a diagram showing an example of valid neighboring points and invalid neighboring points based on the determination of the elevation angle difference. [Figure 13] FIG. 13 is a flowchart illustrating a specific example of the operation of the gradient calculation feasibility determination unit according to the embodiment. [Figure 14] FIG. 14 is a flowchart illustrating a specific example of the operation of the gradient calculation unit according to the embodiment. [Figure 15] FIG. 15 is a diagram for explaining a method for calculating the gradient between two points. [Figure 16] FIG. 16 is a flowchart showing a specific example of the operation of the obstacle point determination unit according to the embodiment. [Figure 17] FIG. 17 is a flowchart illustrating a specific example of the operation of the notification unit according to the embodiment. [Figure 18] FIG. 18 is a diagram showing an example in which the current ratio is an outlier compared to the past ratios. [Figure 19] FIG. 19 is a flowchart showing an example of an operation of the information processing device according to the first modification when detecting an obstacle. [Figure 20] FIG. 20 is a flowchart showing an example of an operation of the information processing device according to the second modification when detecting an obstacle. DETAILED DESCRIPTION OF THE INVENTION

[0011] As described above, it is difficult to accurately measure the deviation of the point cloud detection sensor as disclosed in Patent Document 2. Therefore, there is a risk that the deviation of the point cloud detection sensor may cause malfunction in subsequent processing using the point cloud information.

[0012] If the misalignment of the point cloud detection sensor were to be measured, a device such as a vibration detector would be required, and the calculation load would also be large. In other words, with the conventional technology, there are issues such as the cost and amount of calculation required to suppress malfunctions in subsequent processing caused by the misalignment of the point cloud detection sensor.

[0013] Therefore, an information processing method according to one embodiment of the present disclosure is an information processing method executed by a computer, which includes a process of acquiring a plurality of point information including position information of points sensed by at least one point cloud detection sensor provided on a moving body, searching for a second point within a predetermined range from a first point in the point cloud indicated by the plurality of point information, calculating at least one of a first difference which is the difference in elevation between the first point and the second point, or a second difference which is the angle between a line connecting a reference point and the first point and a line connecting the reference point and the second point, determining whether or not to subject the second point to subsequent processing based on at least one of the first difference or the second difference, and outputting point information of the second point determined to be subject to subsequent processing.

[0014] For example, if a positional or angular deviation occurs in a point cloud detection sensor due to vibration, the position information of a point sensed by the point cloud detection sensor may differ from the actual position. In this case, subsequent processing, such as processing using a gradient, may malfunction, such as falsely detecting an obstacle as if a steep gradient exists when in fact there is no steep gradient. However, the positional or angular deviation of a point cloud detection sensor caused by vibrations associated with the movement of a moving object is often small. Therefore, the elevation difference between two points that may be detected as a steep gradient due to the positional or angular deviation of the point cloud detection sensor, or the angles formed by the lines connecting the reference point and each of the two points, are often also small. Therefore, in the present disclosure, whether to subject a second point to subsequent processing is determined based on at least one of a first difference, which is the elevation difference between a first point and a second point within a predetermined range sensed by the point cloud detection sensor, and a second difference, which is the angle (also called the elevation difference) formed by the line connecting the reference point and the first point and the line connecting the reference point and the second point. This makes it possible to exclude the second point, which is erroneously detected as having a steep gradient due to a positional or angular deviation of the point cloud detection sensor even though the first or second difference is minute, from being subjected to subsequent processing. Therefore, it is possible to suppress malfunctions in subsequent processing caused by the deviation without measuring the deviation of the point cloud detection sensor.

[0015] Furthermore, the distance between the reference point and the first point or the second point may be calculated, and whether or not to calculate at least one of the first difference or the second difference may be determined depending on the distance between the reference point and the first point or the second point.

[0016] The farther away from the reference point, the wider the intervals between the sensed point clouds become. If the second point in such a point cloud is not subject to subsequent processing, the number of points that can be used for subsequent processing at positions far from the reference point will be reduced, making it difficult to perform subsequent processing. Therefore, when the distance between the reference point and the first point or the second point is far, at least one of the first difference or the second difference is not calculated, that is, a determination is not made to exclude the second point from subsequent processing. This allows the second point far from the reference point to be subject to subsequent processing, thereby preventing positions far from the reference point from being unable to be processed further.

[0017] Furthermore, the at least one point cloud detection sensor may be a plurality of point cloud detection sensors, and further, it may be determined whether the point cloud detection sensor that sensed the first point and the point cloud detection sensor that sensed the second point are the same, and if the point cloud detection sensor that sensed the first point and the point cloud detection sensor that sensed the second point are not the same, at least one of the first difference or the second difference may be calculated.

[0018] When sensing is performed using multiple point cloud detection sensors, the first and second points sensed by different point cloud detection sensors are more susceptible to misalignment of the point cloud detection sensors than the first and second points sensed by the same point cloud detection sensor, resulting in malfunctions in subsequent processing. Therefore, when the point cloud detection sensor that sensed the first point and the point cloud detection sensor that sensed the second point are not the same, at least one of the first difference and the second difference is calculated. In other words, when the point cloud detection sensor that sensed the first point and the point cloud detection sensor that sensed the second point are the same, at least one of the first difference and the second difference is not calculated, i.e., a determination is not made to exclude the second point from subsequent processing. This allows as many second points as possible, which are less likely to cause malfunctions in subsequent processing, to be targeted for subsequent processing.

[0019] Furthermore, the method may further calculate a proportion of the second points that were not determined to be the target of the subsequent processing, store a history of the calculated proportion for each scene, determine whether or not there is a history in which the proportion is different for the same scene, and output a notification if there is a history in which the proportion is different. For example, the scene may include a position, and the history may be stored for each position or attribute of a position sensed by the at least one point cloud detection sensor, and determine whether or not there is a history in which the proportion is different for a scene in which the position or attribute of the position matches.

[0020] If there are histories with different proportions of scenes that were not determined to be the target of subsequent processing despite being the same scene (for example, scenes with matching positions or position attributes), there is a risk that some kind of abnormality has occurred in the point cloud detection sensor, etc. Therefore, by outputting a notification in such a case, it is possible to make the occupants or managers of the mobile body aware of the abnormality in the point cloud detection sensor, etc.

[0021] Furthermore, mobile object information indicating characteristics of the mobile object may be acquired, and the predetermined range may be determined based on the mobile object information.

[0022] In this way, by determining the predetermined range based on the characteristics of the moving object, it is possible to determine a predetermined range that is suitable for the moving object. In other words, it is possible to search for a second point that is likely to affect the movement of the moving object. Therefore, it is possible to optimize the amount of calculation and suppress malfunctions in subsequent processing.

[0023] The moving body information may include at least one of the height and width of the moving body, and the predetermined range may be determined based on at least one of the height and width of the moving body.

[0024] Point information in a range that exceeds the height or width of the moving object may not be effective information for subsequent processing. Therefore, by determining the predetermined range for searching the second point based on the height or width of the moving object, it is possible to effectively utilize point information of the second point that is determined to be the target of subsequent processing.

[0025] In addition, the determination of whether or not to subject the second point to the subsequent processing may be made based on whether or not the first difference is greater than or equal to a threshold, and the moving body information may include vibration characteristics of the moving body or the moving body's ability to climb steps, and the threshold of the first difference may be determined based on the vibration characteristics of the moving body or the moving body's ability to climb steps.

[0026] By setting the maximum positional deviation of the point cloud detection sensor that may occur due to vibration of the moving body as the threshold, it is possible to exclude a second point, whose elevation difference from the first point is such that it is caused by vibration of the moving body and may cause a malfunction, from being subjected to further processing. Alternatively, by setting the maximum elevation difference that the moving body cannot overcome as the threshold, it is possible to exclude a second point, whose elevation difference from the first point is such that it is possible for the moving body to overcome it and therefore does not pose a problem if it is not subjected to further processing, from being subjected to further processing. Furthermore, it is conceivable to use a noise removal filter that uses a moving average or the like to exclude the second point from further processing, but the moving average parameters are difficult to adjust due to the time window width. On the other hand, in this embodiment, intuitive parameters such as the moving body's ability to overcome steps can be used, making it easy to adjust the threshold. Furthermore, the larger the threshold value of the first difference, the lower the false detection rate and the higher the non-detection rate (in other words, the smaller the threshold value of the first difference, the higher the false detection rate and the lower the non-detection rate), so by being able to adjust the threshold value of the first difference, it is possible to control the false detection rate and the non-detection rate according to the capabilities of the mobile object or the usage environment, etc.

[0027] In addition, the determination of whether or not to subject the second point to the subsequent processing may be made based on whether or not the second difference is greater than or equal to a threshold value, and the moving body information may include vibration characteristics of the moving body, and the threshold value of the second difference may be determined based on the vibration characteristics of the moving body.

[0028] By setting the maximum angle deviation of the point cloud detection sensor that can occur due to vibration of the mobile body as the threshold, it is possible to prevent the second point, which has an elevation angle difference that is caused by vibration of the mobile body and that could cause a malfunction, from being subjected to subsequent processing. Also, the larger the threshold for the second difference, the lower the false detection rate and the higher the non-detection rate (in other words, the smaller the threshold for the second difference, the higher the false detection rate and the lower the non-detection rate), so by being able to adjust the threshold for the second difference, it is possible to control the false detection rate and the non-detection rate according to the capabilities of the mobile body, the usage environment, etc.

[0029] The subsequent processing may include an object detection process.

[0030] This makes it possible to suppress erroneous detection of an object caused by a deviation of the point cloud detection sensor without measuring the deviation.

[0031] An information processing device according to one embodiment of the present disclosure includes an acquisition unit that acquires a plurality of point information including position information of points sensed by at least one point cloud detection sensor provided on a moving body; a search unit that searches for a second point within a predetermined range from a first point in the point cloud indicated by the plurality of point information; a calculation unit that calculates at least one of a first difference, which is the difference in elevation between the first point and the second point, or a second difference, which is the angle formed between a line connecting a reference point and the first point and a line connecting the reference point and the second point; a determination unit that determines whether or not to subject the second point to subsequent processing based on at least one of the first difference or the second difference; and an output unit that outputs point information of the second point determined to be subject to subsequent processing.

[0032] This makes it possible to provide an information processing device that can suppress malfunctions in subsequent processing caused by deviations of the point cloud detection sensor without measuring the deviations.

[0033] Before describing the embodiments, the detection of an obstacle by a point cloud detection sensor and erroneous detection of an obstacle when a deviation occurs in the point cloud detection sensor will be described below.

[0034] Fig. 1 is a diagram for explaining the principle of obstacle detection by a point cloud detection sensor, which shows LiDAR1 and LiDAR2 as two point cloud detection sensors.

[0035] LiDAR is a sensor that can irradiate an observation area with laser light and calculate the position of a detection point from the optical path length of the scattered light and the angle at which the laser light is irradiated. For example, multiple point information including position information of points sensed by LiDAR can be used to detect objects such as obstacles.

[0036] As shown in Figure 1, the absolute value of the gradient θ of the line connecting points a1 and a2 sensed by LiDAR2 relative to the horizontal direction is smaller than a threshold value. This threshold value is an angle for determining whether the object causing the gradient θ is an obstacle or not. From points a1 and a2 obtained by LiDAR2, it is possible to detect that there is no obstacle in this area. On the other hand, the absolute value of the gradient θ of the line connecting points a3 and a4 sensed by LiDAR1 and LiDAR2 relative to the horizontal direction is greater than or equal to the threshold value. From points a3 and a4 sensed by LiDAR1 and LiDAR2, it is possible to detect that there is an obstacle in this area.

[0037] For example, a point cloud detection sensor such as LiDAR is provided on a moving object such as a vehicle and is used to detect obstacles around the vehicle. The vehicle vibrates when moving, which causes the point cloud detection sensor provided on the vehicle to vibrate as well, resulting in a positional or angular deviation of the point cloud detection sensor. The effect of a positional deviation of the point cloud detection sensor is explained using FIG. 2, and the effect of an angular deviation of the point cloud detection sensor is explained using FIG. 3.

[0038] FIG. 2 is a diagram for explaining erroneous detection of an obstacle due to positional deviation of the point cloud detection sensor.

[0039] Assume that the point cloud detection sensor vibrates in the up-down direction (e.g., vertical direction), and points b1 and b2 are sensed by LiDAR1 at the moment LiDAR1 shifts downward as shown in the upper part of Fig. 2. The position information of the points sensed by LiDAR1 and LiDAR2 is calculated assuming that the relative positional relationship between LiDAR1 and LiDAR2 is the same as the positional relationship before LiDAR1 shifted downward as shown in the lower part of Fig. 2, so points b1 and b2 are calculated to be located at points b3 and b4, which are higher than the actual positions. Therefore, in the area enclosed by the dashed line in the lower part of Fig. 2, the absolute value of the gradient θ is determined to be equal to or greater than the threshold, resulting in erroneous detection of an obstacle.

[0040] FIG. 3 is a diagram for explaining erroneous detection of an obstacle due to an angle deviation of the point cloud detection sensor.

[0041] Assume that the point cloud detection sensor vibrates, and at the moment LiDAR1 rotates (spins) as shown in the upper part of Fig. 3, points c1, c2, and c3 are sensed by LiDAR1. The position information of the points sensed by LiDAR1 and LiDAR2 is calculated assuming that the relative positional relationship between LiDAR1 and LiDAR2 is the same as the positional relationship before LiDAR1 rotated, as shown in the lower part of Fig. 3, and therefore points c1, c2, and c3 are calculated to be located at points c4, c5, and c6, respectively, diagonally above their actual positions. Therefore, in the area enclosed by the dashed line in the lower part of Fig. 3, the absolute value of the gradient θ is determined to be equal to or greater than the threshold, resulting in erroneous detection of an obstacle.

[0042] Therefore, we will specifically explain, with reference to the drawings, an information processing method and information processing device that can suppress malfunctions (hereinafter, for example, false detection) in subsequent processing such as object detection processing that is performed after point cloud sensing by a point cloud detection sensor.

[0043] The embodiments described below are all comprehensive or specific examples, and the numerical values, shapes, materials, components, arrangement and connection of the components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure.

[0044] (Embodiment) FIG. 4 is a block diagram showing an example of the information processing device 100 according to the embodiment.

[0045] The information processing device 100 is a device for performing subsequent processing using multiple point information including position information of points sensed by a point cloud detection sensor. The subsequent processing includes, for example, object detection processing (also referred to as obstacle detection processing). Note that the subsequent processing may include distance detection processing or shape detection processing instead of or in addition to the object detection processing, and multiple processes may be performed as the subsequent processing. The information processing device 100 includes a point information acquisition unit 101, a point information integration unit 102, a noise removal unit 103, a neighboring point search unit 104, an elevation difference determination unit 105, an elevation angle difference determination unit 106, a gradient calculation feasibility determination unit 107, a gradient calculation unit 108, an obstacle point determination unit 109, an obstacle point cloud output unit 110, a threshold setting unit 111, a storage unit 112, and a notification unit 113.

[0046] The information processing device 100 is a computer including a processor, a memory, etc. The memory is a read-only memory (ROM) and a random access memory (RAM), etc., and can store a program executed by the processor. The point information acquisition unit 101, the point information integration unit 102, the noise removal unit 103, the neighboring point search unit 104, the elevation difference determination unit 105, the elevation angle difference determination unit 106, the gradient calculation feasibility determination unit 107, the gradient calculation unit 108, the obstacle point determination unit 109, the obstacle point cloud output unit 110, the threshold setting unit 111, and the notification unit 113 are realized by a processor or the like that executes a program stored in the memory. The storage unit 112 may be realized by the same memory as the memory in which the program is stored, or by a different memory.

[0047] The information processing device 100 is, for example, a device mounted on a mobile object. The mobile object is, for example, a vehicle, but may also be an unmanned aerial vehicle, a robot, a ship, or the like. The information processing device 100 does not have to be mounted on a mobile object and may also be a server. Furthermore, the components constituting the information processing device 100 may be distributed and arranged across multiple servers.

[0048] The point information acquisition unit 101 is an example of an acquisition unit that acquires, from at least one point cloud detection sensor, a plurality of pieces of point information including position information of points sensed by the at least one point cloud detection sensor. The point information acquisition unit 101 may acquire the plurality of pieces of point information directly from the at least one point cloud detection sensor, or may acquire the plurality of pieces of point information from the at least one point cloud detection sensor via another device. The point information acquisition unit 101 may also acquire position information of the at least one point cloud detection sensor. The position information of the at least one point cloud detection sensor may be stored in advance in the information processing device 100.

[0049] For example, at least one point cloud detection sensor is provided on a moving body. Here, the at least one point cloud detection sensor is assumed to be a plurality of point cloud detection sensors, and the plurality of point cloud detection sensors are provided on the moving body. While the moving body is traveling, a deviation in the position or angle of the point cloud detection sensor may occur due to vibration, distortion of the moving body, or the like.

[0050] The point information integration unit 102 integrates a plurality of pieces of point information obtained from each of the plurality of point cloud detection sensors.

[0051] The noise removal unit 103 removes noise contained in the plurality of point information. Specifically, the noise removal unit 103 removes points that are spatially or temporally isolated as noise. For example, if a certain number of points or more do not exist within a certain distance around a certain point, the noise removal unit 103 deletes point information corresponding to the point from the plurality of point information. Furthermore, for example, if a certain number of points or more do not exist within a certain distance around a certain point within a certain time period in the past, the noise removal unit 103 deletes point information corresponding to the point from the plurality of point information.

[0052] The neighboring point search unit 104 is an example of a search unit that searches for a second point (e.g., a neighboring point of the first point) within a predetermined range from a first point in a point cloud indicated by a plurality of point information. The predetermined range is determined, for example, by the threshold setting unit 111. The neighboring point search unit 104 searches for at least one second point. That is, the neighboring point search unit 104 may search for only one second point, or may search for two or more second points. When calculating a three-dimensional gradient in subsequent processing, two or more second points are required for the first point.

[0053] The elevation difference determination unit 105 calculates a first difference, which is the difference in elevation between the first point and the second point, and determines whether or not to subject the second point to subsequent processing based on the first difference. For example, the elevation difference determination unit 105 determines whether or not to subject the second point to subsequent processing depending on whether or not the first difference is equal to or greater than a threshold. The threshold is determined, for example, by the threshold setting unit 111. When two or more second points are searched for, the elevation difference determination unit 105 calculates a first difference for each of the two or more searched second points and determines whether or not to subject the second point to subsequent processing.

[0054] The elevation angle difference determination unit 106 calculates a second difference, which is the angle (also referred to as an elevation angle difference) between a line connecting the reference point and the first point and a line connecting the reference point and the second point, and determines whether or not to subject the second point to subsequent processing based on the second difference. The reference point may be any point in the space in which the moving body exists. The reference point may also be a point based on the moving body. For example, the reference point may be a point on the moving body, or a point outside the moving body that has a predetermined relationship with the moving body. For example, the point on the moving body may be the position of at least one point cloud detection sensor provided on the moving body, a position calculated from the positions of multiple point cloud detection sensors (e.g., an intermediate position), or the center of gravity of the moving body. The point outside the moving body that has a predetermined relationship with the moving body may be, for example, a position spaced a predetermined distance above the moving body. An example in which the reference point is the position of at least one point cloud detection sensor will be described below. If the mobile object is equipped with one point cloud detection sensor, the elevation angle difference determination unit 106 calculates the second difference using the one point cloud detection sensor as a reference point. If the mobile object is equipped with multiple point cloud detection sensors, the elevation angle difference determination unit 106 calculates the second difference using any one of the multiple point cloud detection sensors as a reference point. For example, the elevation angle difference determination unit 106 determines whether or not to subject the second point to subsequent processing depending on whether the second difference is equal to or greater than a threshold. The threshold is determined, for example, by the threshold setting unit 111. If two or more second points are searched for, the elevation angle difference determination unit 106 calculates the second difference for each of the two or more searched second points and determines whether or not to subject the second point to subsequent processing.

[0055] The elevation difference determination unit 105 and the elevation angle difference determination unit 106 are an example of a calculation unit that calculates at least one of a first difference, which is the difference in elevation between the first point and the second point, and a second difference, which is the angle formed between a line connecting the reference point and the first point and a line connecting the reference point and the second point. The elevation difference determination unit 105 and the elevation angle difference determination unit 106 are also an example of a determination unit that determines whether or not to subject the second point to subsequent processing based on at least one of the first difference and the second difference.

[0056] The gradient calculation feasibility determination unit 107 is an example of an output unit that outputs point information of second points determined to be targets for subsequent processing to a processing unit that performs subsequent processing. When two or more second points are searched for, the gradient calculation feasibility determination unit 107 outputs point information of second points determined to be targets for subsequent processing among the two or more searched second points.

[0057] The gradient calculation unit 108 and the obstacle point determination unit 109 are an example of a processing unit that performs obstacle detection processing as subsequent processing.

[0058] The gradient calculation unit 108 calculates the gradient from the position of the first point and the position of the second point output from the gradient calculation possibility determination unit 107.

[0059] If the absolute value of the gradient calculated by the gradient calculation unit 108 is equal to or greater than a threshold, the obstacle point determination unit 109 determines that the first point and the second point are obstacle points corresponding to an obstacle. The threshold is set by the threshold setting unit 111, for example.

[0060] The processes in the neighbor point search unit 104, elevation difference determination unit 105, elevation angle difference determination unit 106, gradient calculation feasibility determination unit 107, gradient calculation unit 108, and obstacle point determination unit 109 are repeatedly performed for each point in the point cloud indicated by the plurality of point information sensed by at least one point cloud detection sensor. Specifically, if the point cloud indicated by the plurality of point information contains 100 points, first, one of the 100 points is selected as a first point, and the other 99 points are searched for as candidates for the second point. From among the searched second points, a second point determined to be the target of subsequent processing is output, and obstacle detection is performed for the combination of this first point and second point. Next, another point from the 100 points is selected as a first point, and similar processing is performed. This is repeated for each of the 100 points.

[0061] The obstacle point cloud output unit 110 outputs all points determined to be obstacle points as an obstacle point cloud, thereby enabling the moving object or the like to perform processing such as automatically avoiding the obstacle.

[0062] The threshold setting unit 111 acquires mobile object information indicating the characteristics of the mobile object, and determines the predetermined range based on the mobile object information. The mobile object information may be stored in the storage unit 112 or acquired from an external device.

[0063] For example, the threshold setting unit 111 acquires moving object information including at least one of the height or width of the moving object, and determines a predetermined range based on at least one of the height or width of the moving object. Point information in a range that exceeds the height or width of the moving object may not be effective information for subsequent processing. Therefore, by determining the predetermined range for searching for the second point based on the height or width of the moving object, it is possible to effectively utilize point information of the second point determined to be subject to subsequent processing.

[0064] Furthermore, the threshold setting unit 111 acquires mobile object information including the vibration characteristics of the mobile object or the step-clearing ability of the mobile object, and determines the threshold for the first difference based on the vibration characteristics of the mobile object or the step-clearing ability of the mobile object. The step-clearing ability indicates the height from the ground surface that the mobile object can climb. Note that the step-clearing ability may be estimated from the climbing ability. Specifically, the elevation difference may be calculated from the inclination angle indicated by the climbing ability and a predetermined horizontal distance (for example, a predetermined distance in the neighboring point search), and the calculated elevation difference may be determined as the threshold for the first difference. By setting the maximum positional deviation of the point cloud detection sensor that may occur due to the vibration of the mobile object as the threshold, it is possible to prevent a second point, which has an elevation difference from the first point that is caused by the vibration of the mobile object and may cause erroneous detection, from being subjected to subsequent processing. Alternatively, by setting the maximum elevation difference that the moving object cannot overcome as the threshold, it is possible to exclude a second point, whose elevation difference from the first point is such that the moving object can overcome it and therefore does not need to be subjected to further processing, from the target of further processing. Furthermore, a noise removal filter using a moving average or the like could be used to exclude the second point from the target of further processing, but the moving average parameter is difficult to adjust due to the time window width. On the other hand, this aspect allows for the use of intuitive parameters such as the moving object's ability to overcome steps, making it easy to adjust the threshold. Furthermore, the larger the threshold for the first difference, the lower the false detection rate and the higher the non-detection rate (in other words, the smaller the threshold for the first difference, the higher the false detection rate and the lower the non-detection rate). Therefore, by adjusting the threshold for the first difference, it is possible to control the false detection rate and the non-detection rate according to the moving object's ability, the usage environment, etc.

[0065] Furthermore, the threshold setting unit 111 acquires mobile object information including the vibration characteristics of the mobile object, and determines the threshold for the second difference based on the vibration characteristics of the mobile object. By setting the threshold to the maximum angle deviation of the point cloud detection sensor that may occur due to the vibration of the mobile object, it is possible to prevent a second point, which has an elevation angle difference that may occur due to the vibration of the mobile object and may cause a false detection, from being subjected to subsequent processing. Furthermore, the larger the threshold for the second difference, the lower the false detection rate and the higher the non-detection rate (in other words, the smaller the threshold for the second difference, the higher the false detection rate and the lower the non-detection rate). Therefore, by adjusting the threshold for the second difference, it is possible to control the false detection rate and the non-detection rate according to the capabilities of the mobile object, the usage environment, etc.

[0066] Furthermore, the threshold setting unit 111 sets the gradient at which an obstacle should be determined as a threshold.

[0067] The elevation difference determination unit 105 and the elevation angle difference determination unit 106 calculate the proportion of second points that were not determined to be the target of subsequent processing, and the storage unit 112 stores a history of the calculated proportions for each scene (specifically, for each time and place). For example, the history of the proportions is stored for each position or attribute of a position sensed by at least one point cloud detection sensor. The attribute of a position is, for example, an attribute that can be distinguished according to the degree of vibration that occurs during movement, and specifically, an attribute such as an ordinary road or an expressway.

[0068] The notification unit 113 outputs a notification when a history with a different proportion of second points that were not determined to be the target of subsequent processing exists in the storage unit 112. Details of the notification unit 113 will be described later.

[0069] Next, the operation of the information processing device will be described with reference to FIG.

[0070] Fig. 5 is a flowchart showing an example of an information processing method according to the embodiment. Note that, since the information processing method according to the embodiment is a method executed by information processing device 100 (computer), Fig. 5 is also a flowchart showing an example of the operation of information processing device 100 according to the embodiment.

[0071] First, the point information acquisition unit 101 acquires a plurality of point information including position information of points sensed by at least one point cloud detection sensor (step S101).

[0072] Next, the neighboring point search unit 104 searches for a second point within a predetermined range from the first point in the point group indicated by the plurality of point information (step S102).

[0073] Next, the elevation difference determination unit 105 and the elevation angle difference determination unit 106 calculate at least one of a first difference, which is the difference in elevation between the first point and the second point, or a second difference, which is the angle between a line connecting the reference point and the first point and a line connecting the reference point and the second point (step S103).

[0074] Next, the elevation difference determining unit 105 and the elevation angle difference determining unit 106 determine whether or not the second point is to be subjected to subsequent processing based on at least one of the first difference and the second difference (step S104).

[0075] Then, the gradient calculation possibility determination unit 107 outputs point information of the second point determined to be the target of subsequent processing (step S105).

[0076] The operation of the information processing device 100 when detecting an obstacle will be described in detail below.

[0077] FIG. 6 is a flowchart showing an example of the operation of the information processing device 100 according to the embodiment when detecting an obstacle.

[0078] The information processing device 100 performs the processes from step S202 to step S210 for each of all points sensed by a plurality of point cloud detection sensors within an area where obstacle determination is to be performed (step S201). The information processing device 100 selects one point from among all points within the area where obstacle determination is to be performed.

[0079] The neighboring point search unit 104 sets the selected point as the first point and searches for a neighboring point (i.e., a second point) within a predetermined range from the first point (step S202). Hereinafter, the first point will be referred to as the target point, and the second point will be referred to as the neighboring point. Details of the operation of the neighboring point search unit 104 in step S202 will be described with reference to FIGS. 7 and 8.

[0080] FIG. 7 is a flowchart showing a specific example of the operation of the neighboring point searching unit 104 according to the embodiment.

[0081] The neighboring point search unit 104 performs the processes from step S302 to step S304 for each of all points other than the target point among all points within the area where obstacle determination is performed (step S301). The neighboring point search unit 104 selects one point (called the other point) from all points other than the target point.

[0082] The neighboring point search unit 104 calculates the distance between the target point and the selected other points (step S302). Since each point is point information including position information, the distance between points can be calculated from the position information of each point.

[0083] Next, the neighboring point searching unit 104 determines whether the calculated distance is equal to or greater than a threshold value (step S303). As described above, the threshold value is a value corresponding to a predetermined range determined by the threshold value setting unit 111 based on the moving object information including at least one of the height and width of the moving object.

[0084] If the calculated distance is less than the threshold value (No in step S303), neighboring point searching unit 104 determines that the selected other point is a neighboring point (step S304).

[0085] On the other hand, if the calculated distance is equal to or greater than the threshold (Yes in step S303), neighboring point searching unit 104 does not determine that the other selected points are neighboring points.

[0086] Then, the neighboring point search unit 104 selects points that have not yet been selected from among all points other than the target point as other points, and performs the processes from step S302 to step S304 on the other points. The neighboring point search unit 104 performs the processes from step S302 to step S304 on all points other than the target point, and searches for one or more neighboring points from among all points other than the target point.

[0087] FIG. 8 is a diagram illustrating an example of neighboring points and non-neighboring points.

[0088] As shown in Figure 8, points whose distance from the target point is d less than the threshold D are determined to be nearby points, while points whose distance from the target point is greater than or equal to the threshold D are not determined to be nearby points and are non-neighboring points.

[0089] The neighboring point search unit 104 may remap all points within the area where obstacle determination is performed onto a spherical grid, a cylindrical grid, or a polygonal grid centered on the moving object, and set points on adjacent grids or nearby grids as neighboring points. This reduces the calculation load during neighboring point search.

[0090] 6, the elevation difference determination unit 105 performs a determination based on the elevation difference between the target point and each of the one or more neighboring points searched for by the neighboring point search unit 104 (step S203). Details of the operation of the elevation difference determination unit 105 in step S203 will be described with reference to FIGS. 9 and 10.

[0091] FIG. 9 is a flowchart showing a specific example of the operation of the elevation difference determining unit 105 according to the embodiment.

[0092] The elevation difference determination unit 105 performs the processes from step S402 to step S404 for each of all neighboring points found for the target point (step S401). The elevation difference determination unit 105 selects one neighboring point from all the neighboring points found.

[0093] The elevation difference determination unit 105 calculates the elevation difference between the target point and the selected neighboring point (step S402). Since each point is point information including position information, the elevation difference between points can be calculated from the position information of each point.

[0094] The elevation difference determination unit 105 determines whether the calculated elevation difference is equal to or greater than a threshold value (step S403). As described above, the threshold value is a threshold value determined by the threshold value setting unit 111 based on the moving object information including the vibration characteristics of the moving object or the moving object's ability to climb over steps. In step S403, the elevation difference determination unit 105 determines whether to target the nearby point for obstacle detection processing based on the elevation difference.

[0095] If the calculated elevation difference is less than the threshold value (No in step S403), the elevation difference determination unit 105 excludes the selected neighboring point from the obstacle detection processing target (step S404). That is, the elevation difference determination unit 105 determines that the neighboring point whose elevation difference with the target point is less than the threshold value is not to be the obstacle detection processing target.

[0096] On the other hand, if the calculated elevation difference is equal to or greater than the threshold (Yes in step S403), the elevation difference determination unit 105 does not exclude the selected neighboring point from the obstacle detection process. In other words, the elevation difference determination unit 105 determines that the neighboring point whose elevation difference from the target point is equal to or greater than the threshold is to be the obstacle detection process target.

[0097] Then, the elevation difference determination unit 105 selects a neighboring point that has not yet been selected from all neighboring points, and performs the processes from step S402 to step S404 on that neighboring point. The elevation difference determination unit 105 performs the processes from step S402 to step S404 on all neighboring points for the target point, and excludes from the obstacle detection process any neighboring point whose elevation difference from the target point is less than a threshold value.

[0098] FIG. 10 is a diagram showing an example of valid neighboring points and invalid neighboring points based on the determination of the elevation difference.

[0099] As shown in Figure 10, it can be seen that neighboring points whose elevation difference from the target point is less than the threshold H are determined to be neighboring points that have been excluded from the obstacle detection process (i.e., invalid neighboring points), and neighboring points whose elevation difference from the target point is h that is greater than or equal to the threshold H are determined to be neighboring points that have not been excluded from the obstacle detection process (i.e., valid neighboring points).

[0100] 6, the elevation difference determination unit 105 extracts neighboring points whose elevation difference from the target point is equal to or greater than a threshold (step S204). That is, as described in FIG. 9, the elevation difference determination unit 105 extracts, for the obstacle detection process, neighboring points that have not been excluded from the obstacle detection process.

[0101] Next, the elevation angle difference determination unit 106 performs a determination based on the elevation angle difference between the target point and the neighboring point for each of the one or more neighboring points searched for by the neighboring point search unit 104 (step S205). Details of the operation of the elevation angle difference determination unit 106 in step S205 will be described with reference to FIGS. 11 and 12.

[0102] FIG. 11 is a flowchart showing a specific example of the operation of the elevation angle difference determining unit 106 according to the embodiment.

[0103] The elevation angle difference determination unit 106 performs the processes from step S502 to step S504 for each of all neighboring points found for the target point (step S501). The elevation angle difference determination unit 106 selects one neighboring point from all the neighboring points found.

[0104] The elevation angle difference determination unit 106 calculates the elevation angle difference between the target point and the selected neighboring point relative to the reference point (step S502). As described above, the position of any point cloud detection sensor among the multiple point cloud detection sensors provided on the moving object becomes the reference point. Furthermore, since each point is point information including position information, the elevation angle difference between each point relative to the reference point can be calculated from the position information of each point.

[0105] The elevation angle difference determination unit 106 determines whether the calculated elevation angle difference is equal to or greater than a threshold value (step S503). As described above, the threshold value is determined by the threshold value setting unit 111 based on the moving body information including the vibration characteristics of the moving body. In step S503, the elevation angle difference determination unit 106 determines whether to subject the nearby point to obstacle detection processing based on the elevation angle difference.

[0106] If the calculated elevation angle difference is less than the threshold value (No in step S503), the elevation angle difference determination unit 106 excludes the selected neighboring point from the obstacle detection processing target (step S504). That is, the elevation angle difference determination unit 106 determines that the neighboring point whose elevation angle difference with respect to the reference point and the target point is less than the threshold value will not be the obstacle detection processing target.

[0107] On the other hand, if the calculated elevation angle difference is equal to or greater than the threshold (Yes in step S503), the elevation angle difference determination unit 106 does not exclude the selected neighboring point from the obstacle detection process. That is, the elevation angle difference determination unit 106 determines that the neighboring point, whose elevation angle difference with respect to the reference point is equal to or greater than the threshold, is to be the obstacle detection process target.

[0108] The elevation angle difference determination unit 106 then selects a nearby point that has not yet been selected from all nearby points, and performs the processes from step S502 to step S504 on that nearby point. The elevation angle difference determination unit 106 performs the processes from step S502 to step S504 on all nearby points relative to the target point, and excludes nearby points whose elevation angle difference from the target point is less than a threshold value from the obstacle detection process.

[0109] FIG. 12 is a diagram showing an example of valid neighboring points and invalid neighboring points based on the determination of the elevation angle difference.

[0110] As shown in FIG. 12, a neighboring point whose elevation angle difference with respect to the reference point and the target point is less than the threshold value Ψ is determined to be a neighboring point excluded from the obstacle detection process (i.e., an invalid neighboring point), and a neighboring point whose elevation angle difference with respect to the target point is ψ equal to or greater than the threshold value Ψ is determined to be a neighboring point that is not excluded from the obstacle detection process (i.e., a valid neighboring point).

[0111] 6, the elevation angle difference determination unit 106 extracts nearby points whose elevation angle difference between the target point and the reference point is equal to or greater than a threshold (step S206). That is, as described in FIG. 11, the elevation angle difference determination unit 106 extracts nearby points that have not been excluded from the obstacle detection processing targets, for the obstacle detection processing.

[0112] Note that steps S203 and S204 and steps S205 and S206 may be executed in reverse order or may be executed in parallel. Furthermore, when steps S203, S204, S205, and S206 are executed in this order, the elevation angle difference determination unit 106 may not make a determination based on the elevation angle difference for each of the one or more neighboring points searched for by the neighboring point search unit 104, but may make a determination based on the elevation angle difference for each of the neighboring points extracted by the elevation angle difference determination unit 105. Furthermore, when steps S205, S206, S203, and S204 are executed in this order, the elevation difference determination unit 105 may not make a determination based on the elevation difference for each of the one or more neighboring points searched for by the neighboring point search unit 104, but may make a determination based on the elevation difference for each of the neighboring points extracted by the elevation angle difference determination unit 106.

[0113] Next, the gradient calculation feasibility determination unit 107 determines whether there are a sufficient number of neighboring points extracted by the elevation difference determination unit 105 and the elevation angle difference determination unit 106 that have not been excluded from the obstacle detection process (step S207). The presence of a sufficient number of neighboring points means that there are a sufficient number of neighboring points required for gradient calculation. The operation of the gradient calculation feasibility determination unit 107 in step S207 will be described in detail with reference to FIG. 13.

[0114] FIG. 13 is a flowchart showing a specific example of the operation of the gradient calculation feasibility determination unit 107 according to the embodiment.

[0115] The gradient calculation feasibility determination unit 107 counts the number of nearby points to be subjected to obstacle detection processing (step S601). For example, if the nearby points extracted by the elevation difference determination unit 105 and the nearby points extracted by the elevation angle difference determination unit 106 overlap, the gradient calculation feasibility determination unit 107 prevents the overlapping nearby points from being counted twice.

[0116] The gradient calculation feasibility determination unit 107 determines whether the counted number of points is equal to or greater than a threshold value (step S602). When calculating a two-dimensional gradient, the threshold value may be set to 1 because the gradient can be calculated using the target point and at least one neighboring point. When calculating a three-dimensional gradient, the threshold value may be set to 2 because the gradient can be calculated using the target point and at least two neighboring points.

[0117] If the number of counted points is equal to or greater than the threshold value (Yes in step S602), in other words, if there are a sufficient number of nearby points that have not been excluded from the obstacle detection processing (Yes in step S207 in FIG. 6), the gradient calculation feasibility determination unit 107 outputs the point information determined to be the target of the obstacle detection processing to the processing unit that performs the obstacle detection processing, and the processing of step S208 in FIG. 6 is performed.

[0118] If the number of counted points is less than the threshold value (No in step S602), in other words, if there are not enough nearby points that have not been excluded from the obstacle detection process (No in step S207 in Figure 6), the currently selected target point and its nearby points are not subject to the obstacle detection process, and a point that has not yet been selected among all points within the area where obstacle determination is to be performed is selected as the target point, and the loop of step S201 in Figure 6 is performed.

[0119] 6, if there are a sufficient number of neighboring points that have not been excluded from the obstacle detection process (Yes in step S207), the gradient calculation unit 108 performs gradient calculation (step S208). Details of the operation of the gradient calculation unit 108 in step S208 will be described with reference to FIGS. 14 and 15.

[0120] FIG. 14 is a flowchart showing a specific example of the operation of the gradient calculation unit 108 according to the embodiment.

[0121] The gradient calculation unit 108 performs the process of step S702 for each of all neighboring points (i.e., valid neighboring points) that are the object of obstacle detection processing (step S701). The gradient calculation unit 108 selects one neighboring point from all the valid neighboring points.

[0122] The gradient calculation unit 108 calculates the gradient between the target point and the selected valid neighboring point (step S702). The method of calculating the gradient between two points will be described with reference to FIG.

[0123] FIG. 15 is a diagram for explaining a method for calculating the gradient between two points.

[0124] As shown in Figure 15, the gradient g between the target point and a valid neighboring point can be calculated by dividing the elevation difference h between the target point and the valid neighboring point by the horizontal distance p between the target point and the valid neighboring point.

[0125] 14, the gradient calculation unit 108 selects a neighboring point that has not yet been selected from all valid neighboring points, and performs the process of step S702 for that neighboring point. That is, the gradient calculation unit 108 calculates the gradient between the target point and each of all valid neighboring points.

[0126] Then, the gradient calculation unit 108 uses the gradient with the largest absolute value among all the calculated gradients between two points as the gradient for the target point currently selected in step S201 (step S703). Note that the gradient calculation unit 108 may also use the gradient between the target point and the nearest neighbor point among all valid neighbor points as the gradient for the target point currently selected in step S201. Alternatively, the gradient calculation unit 108 may use the average value of all the calculated gradients between two points as the gradient for the target point currently selected in step S201.

[0127] Returning to the explanation in Fig. 6, the obstacle point determination unit 109 determines whether or not the gradient adopted by the gradient calculation unit 108 is large (step S209). Details of the operation of the obstacle point determination unit 109 in step S209 will be explained using Fig. 16.

[0128] FIG. 16 is a flowchart showing a specific example of the operation of the obstacle point determination unit 109 according to the embodiment.

[0129] The obstacle point determination unit 109 compares the absolute value of the adopted gradient with a threshold value (step S801), and determines whether the absolute value of the gradient is equal to or greater than the threshold value (step S802). The threshold value is set by the threshold value setting unit 111, as described above.

[0130] If the absolute value of the gradient is equal to or greater than the threshold value (Yes in step S802), in other words, if the gradient adopted by the gradient calculation unit 108 is large (Yes in step S209 in FIG. 6), the obstacle point determination unit 109 performs the processing of step S210 in FIG. 6.

[0131] If the absolute value of the gradient is less than the threshold value (No in step S802), in other words, if the gradient adopted by the gradient calculation unit 108 is small (No in step S209 in FIG. 6), the obstacle point determination unit 109 does not determine that the currently selected target point and its neighboring points are obstacle points corresponding to an obstacle, and instead selects as target points a point that has not yet been selected among all points within the area where obstacle determination is to be performed, and the loop of step S201 in FIG. 6 is performed.

[0132] Returning to the explanation in FIG. 6, if the gradient adopted by the gradient calculation unit 108 is large (Yes in step S209), the obstacle point determination unit 109 determines that the target point selected in step S201 and its neighboring points are obstacle points corresponding to an obstacle (step S210).

[0133] Then, the processing from step S202 is performed using the points that have not yet been selected among all points within the area where obstacle determination is to be performed as target points, and an obstacle point group is extracted from all points within the area where obstacle determination is to be performed.

[0134] Next, the details of the operation of the notification unit 113 will be described with reference to FIGS.

[0135] FIG. 17 is a flowchart showing a specific example of the operation of notification unit 113 according to the embodiment.

[0136] The notification unit 113 calculates the proportion of neighboring points that were not determined to be targets of the obstacle detection process in the obstacle detection process currently performed in a certain scene (step S901). Specifically, for each target point, the proportion of all neighboring points that were determined to be invalid neighboring points is calculated, and the average is calculated.

[0137] The notification unit 113 compares the currently calculated ratio with the past ratio (step S902). The past ratios are stored in the storage unit 112, and for example, the notification unit 113 compares the currently calculated ratio with the past ratio for the same scene as this time (specifically, a scene with a matching position or position attribute).

[0138] The notification unit 113 determines whether the current ratio is an outlier compared to past ratios (step S903). An example of a case where the current ratio is an outlier compared to past ratios will be described with reference to FIG.

[0139] Fig. 18 is a diagram showing an example where the current ratio is an outlier compared to past ratios. Fig. 18 shows the distribution of the ratio of nearby points that were not determined to be targets for obstacle detection processing when a mobile object traveled through the same point A. This distribution can be calculated from the history of the ratio of nearby points that were not determined to be targets for obstacle detection processing.

[0140] As shown in Figure 18, when a moving object has traveled through point A in the past, the percentage of points determined to be invalid nearby points is low, but when the moving object travels through the same point A this time, the percentage of points determined to be invalid nearby points is high. In other words, it can be seen that the current percentage is an outlier compared to the past percentages.

[0141] 17, if the current ratio is an outlier compared to the past ratios (Yes in step S903), the notification unit 113 issues a maintenance alert (step S904). If the mobile object travels through the same point A both now and in the past, and the current ratio is an outlier compared to the past ratio, this may indicate that some kind of abnormality has occurred in the point cloud detection sensor, etc. If the current ratio is not an outlier compared to the past ratios (No in step S903), the notification unit 113 terminates the process without issuing a notification.

[0142] In Figure 18, an example is described in which a ratio comparison is performed for the same location, but the ratio comparison does not have to be performed for the same location, and may be performed for the same attribute (for example, a general road or a highway, etc.).

[0143] In this way, if there are histories with different proportions of cases where the object was not determined to be subject to the obstacle detection process despite the same scene (for example, a scene with matching positions or position attributes) (for example, if the proportion in the current history is an outlier compared to the proportions in past history), there is a risk that some kind of abnormality has occurred in the point cloud detection sensor, etc. Therefore, by outputting a notification in such a case, it is possible to make the occupant or manager of the mobile body aware of the abnormality in the point cloud detection sensor, etc.

[0144] The information processing device 100 may calculate the distance between the point cloud detection sensor (reference point) and the target point or a nearby point, and determine whether to calculate at least one of the height difference and the elevation angle difference depending on the distance between the reference point and the target point or the nearby point. This will be described with reference to FIG. 19.

[0145] Fig. 19 is a flowchart showing an example of the operation of the information processing device 100 according to Modification 1 when detecting an obstacle. Note that steps S201 to S210 in Fig. 19 are the same as those in Fig. 6, and therefore description thereof will be omitted.

[0146] The information processing device 100 calculates the distance between the reference point and the target point or the searched neighboring point, and determines whether the distance is close (for example, whether it is equal to or less than a threshold) (step S211). If the distance between the reference point and the target point or the searched neighboring point is close (Yes in step S211), the processes from step S203 to step S206 are performed, and if the distance between the reference point and the target point or the searched neighboring point is far (No in step S211), the processes from step S203 to step S206 are not performed. Note that, since the target point and the neighboring point are at approximately the same distance from the reference point, the distance to the reference point may be calculated for either the target point or the neighboring point.

[0147] The farther away from the reference point, the wider the intervals between the sensed point clouds become. If nearby points in such a point cloud are not subject to obstacle detection processing, the number of points that can be used for obstacle detection processing at positions far from the reference point will be reduced, making it difficult to perform obstacle detection processing. Therefore, when the distance between the reference point and the target point or its nearby points is far, at least one of the elevation difference or elevation angle difference is not calculated, that is, a determination is not made to exclude nearby points from being subject to obstacle detection processing. This allows nearby points far from the reference point to be subject to obstacle detection processing, thereby preventing the inability to perform obstacle detection processing at positions far from the reference point.

[0148] Furthermore, the information processing device 100 may determine whether the point cloud detection sensor that sensed the target point and the point cloud detection sensor that sensed the neighboring points are the same, and if the point cloud detection sensor that sensed the target point and the point cloud detection sensor that sensed the neighboring points are not the same, calculate at least one of the height difference or the elevation angle difference. This will be described with reference to FIG. 20.

[0149] Fig. 20 is a flowchart showing an example of the operation of the information processing device 100 according to Modification 2 when detecting an obstacle. Note that steps S201 to S210 in Fig. 20 are the same as those in Fig. 6, and therefore description thereof will be omitted.

[0150] The information processing device 100 determines whether the searched neighboring point was observed by a point cloud detection sensor other than the point cloud detection sensor that observed the target point (step S212). If the searched neighboring point was observed by another point cloud detection sensor (Yes in step S212), the processes from step S203 to step S206 are performed, and if the searched neighboring point was not observed by another point cloud detection sensor (No in step S211), the processes from step S203 to step S206 are not performed.

[0151] When sensing is performed using multiple point cloud detection sensors, target points and neighboring points sensed by different point cloud detection sensors are more susceptible to misalignment of the point cloud detection sensors than target points and neighboring points sensed by the same point cloud detection sensor, making them more susceptible to false detection in the obstacle detection process. Therefore, when the point cloud detection sensor sensing the target point and the point cloud detection sensor sensing the neighboring points are different, at least one of the elevation difference and the elevation angle difference is calculated. In other words, when the point cloud detection sensor sensing the target point and the point cloud detection sensor sensing the neighboring points are the same, at least one of the elevation difference and the elevation angle difference is not calculated, i.e., a determination is not made to exclude the neighboring points from the obstacle detection process. This allows as many second points as possible, which are less likely to cause false detection in the obstacle detection process, to be targeted in the obstacle detection process.

[0152] As described above, if a positional or angular deviation occurs in a point cloud detection sensor due to, for example, vibration, the position information of a point sensed by the point cloud detection sensor may differ from the actual position. In this case, for example, an obstacle detection process that uses a gradient may erroneously detect a steep gradient when there is no actual steep gradient (e.g., erroneous detection of an obstacle). However, the positional or angular deviation of a point cloud detection sensor caused by vibrations or the like associated with the movement of a mobile object is often small. Therefore, the elevation difference between two points that may be detected as a steep gradient due to a positional or angular deviation of the point cloud detection sensor, or the elevation angle difference between the two points relative to a reference point, is often also small. Therefore, in the present disclosure, whether to target a nearby point for obstacle detection processing is determined based on at least one of the elevation difference between a target point and a nearby point within a predetermined range sensed by the point cloud detection sensor or the elevation angle difference between the target point and the nearby point relative to the reference point. This makes it possible to exclude nearby points that are erroneously detected as a steep gradient due to a positional or angular deviation of the point cloud detection sensor from being targeted for obstacle detection processing. Therefore, it is possible to suppress erroneous detection in the obstacle detection process caused by the deviation of the point cloud detection sensor without measuring the deviation.

[0153] (Other embodiments) While the information processing method and information processing device 100 according to one or more aspects of the present disclosure have been described above based on the embodiments, the present disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to each embodiment and forms constructed by combining components of different embodiments may also be included within the scope of one or more aspects of the present disclosure.

[0154] For example, in the above embodiment, an example has been described in which the information processing device 100 includes both the elevation difference determination unit 105 and the elevation angle difference determination unit 106, but it is not necessary to include either one of them. In other words, it is not necessary to calculate both the first difference and the second difference; it is sufficient to calculate at least one of them. For example, when the first difference is calculated, it is determined based on the first difference whether or not to subject the second point to subsequent processing, and when the second difference is calculated, it is determined based on the second difference whether or not to subject the second point to subsequent processing.

[0155] For example, in the above embodiment, an example has been described in which the information processing device 100 includes the storage unit 112 and the notification unit 113, but the information processing device 100 may not include the storage unit 112 and the notification unit 113. That is, the percentage of second points that were not determined to be subject to subsequent processing may not be calculated, the history of the percentages may not be stored for each scene, and notifications such as maintenance alerts may not be output.

[0156] For example, in the above embodiment, an example has been described in which the information processing device 100 includes the threshold setting unit 111, but the information processing device 100 may not include the threshold setting unit 111. That is, the predetermined range may not be determined based on moving body information including at least one of the height or width of the moving body, and the threshold for the first difference or the second difference may not be determined based on moving body information including the vibration characteristics of the moving body or the step-climbing ability of the moving body.

[0157] The threshold values ​​for the first difference and the second difference may be set based on a preliminary test. For example, a test such as a vibration test or a running test may be performed, and the allowable height difference and the allowable elevation angle difference may be set based on the test results.

[0158] For example, the present disclosure can be realized as a program for causing a processor to execute steps included in an information processing method. Furthermore, the present disclosure can be realized as a non-transitory computer-readable recording medium, such as a CD-ROM, on which the program is recorded.

[0159] For example, when the present disclosure is realized as a program (software), each step is performed by running the program using hardware resources such as a computer's CPU, memory, input / output circuits, etc. In other words, each step is performed by the CPU acquiring data from memory or input / output circuits, etc., performing calculations on the data, and outputting the calculation results to memory or input / output circuits, etc.

[0160] In the above embodiment, each component included in the information processing device 100 may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0161] Some or all of the functions of the information processing device 100 according to the above-described embodiment are typically realized as an LSI, which is an integrated circuit. These may be individually integrated into single chips, or some or all of them may be integrated into a single chip. Furthermore, the integrated circuit is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array), which can be programmed after LSI manufacture, or a reconfigurable processor, which can reconfigure the connections and settings of circuit cells within an LSI, may also be used.

[0162] Furthermore, various modifications made to the embodiments of the present disclosure within the scope that would occur to a person skilled in the art are also included in the present disclosure, as long as they do not deviate from the gist of the present disclosure. [Industrial Applicability]

[0163] The present disclosure can be applied to a device that performs processing using point information sensed by a point cloud detection sensor such as a LiDAR. [Explanation of symbols]

[0164] 100 Information processing device 101 points information acquisition part 102 Point Information Integration Department 103 Noise removal section 104 Nearby point search unit 105 Height difference determination section 106 Elevation angle difference determination section 107 Gradient calculation possibility determination unit 108 Gradient calculation unit 109 Obstacle point determination unit 110 Obstacle point cloud output unit 111 Threshold setting unit 112 Storage section 113 Notification Department

Claims

1. 1. A computer-implemented information processing method, comprising: acquiring a plurality of pieces of point information including position information having height information of points sensed by at least one point cloud detection sensor provided in the moving body; calculating a distance between a first point in the point cloud indicated by the plurality of point information and one or more other points other than the first point in the point cloud indicated by the plurality of point information, and searching for a point among the one or more other points for which the calculated distance is less than a threshold value as a second point within a predetermined range from the first point; calculating a first difference which is a difference in elevation between the first point and the second point; determining whether the first difference is equal to or greater than a threshold, and if the first difference is less than the threshold, determining that the second point is not to be subjected to subsequent processing, which is object detection processing, distance detection processing, or shape detection processing, and if the first difference is equal to or greater than the threshold, determining that the second point is to be subjected to the subsequent processing; outputting point information of the second point determined to be a target of the subsequent processing; Information processing methods.

2. moreover, calculating a distance between the first point or the second point and a reference point which is a point on the moving body or a point outside the moving body that has a predetermined relationship with the moving body; It is determined whether the distance between the reference point and the first point or the second point is equal to or less than a threshold value, and if the distance is equal to or less than the threshold value, the first difference is calculated, and if the distance is greater than the threshold value, the first difference is not calculated. The information processing method according to claim 1 .

3. the at least one point cloud detection sensor is a plurality of point cloud detection sensors, moreover, determining whether the point cloud detection sensor that sensed the first point and the point cloud detection sensor that sensed the second point are the same; Calculating the first difference when the point cloud detection sensor that sensed the first point and the point cloud detection sensor that sensed the second point are not the same.

3. The information processing method according to claim 1 or 2.

4. The threshold value of the first difference is a height from the ground surface that the moving body can overcome, or a maximum positional deviation of the point cloud detection sensor that can be caused by vibration of the moving body. The information processing method according to any one of claims 1 to 3.

5. An information processing method executed by a computer, comprising: acquiring a plurality of pieces of point information including position information having height information of points sensed by at least one point cloud detection sensor provided in the moving body; calculating a distance between a first point in the point cloud indicated by the plurality of point information and one or more other points other than the first point in the point cloud indicated by the plurality of point information, and searching for a point among the one or more other points for which the calculated distance is less than a threshold value as a second point within a predetermined range from the first point; calculating a second difference, which is the angle between a line connecting the first point and a reference point, which is a point on the moving body or a point outside the moving body that has a predetermined relationship with the moving body, and a line connecting the reference point and the second point; determining whether the second difference is equal to or greater than a threshold, and if the second difference is less than the threshold, determining that the second point is not to be subjected to subsequent processing, which is object detection processing, distance detection processing, or shape detection processing, and if the second difference is equal to or greater than the threshold, determining that the second point is to be subjected to the subsequent processing; outputting point information of the second point determined to be a target of the subsequent processing; Information processing methods.

6. Furthermore, Calculating the distance between the reference point and the first point or the second point; It is determined whether the distance between the reference point and the first point or the second point is equal to or less than a threshold value, and if the distance is equal to or less than the threshold value, the second difference is calculated, and if the distance is greater than the threshold value, the second difference is not calculated. The information processing method according to claim 5 .

7. The at least one point cloud detection sensor is a plurality of point cloud detection sensors, moreover, determining whether the point cloud detection sensor that sensed the first point and the point cloud detection sensor that sensed the second point are the same; Calculating the second difference when the point cloud detection sensor that sensed the first point and the point cloud detection sensor that sensed the second point are not the same.

7. The information processing method according to claim 5 or 6.

8. The second difference threshold is a maximum angle deviation of the point cloud detection sensor that can be caused by vibration of the moving body. The information processing method according to any one of claims 5 to 7.

9. moreover, Calculating a ratio of the second points that were not determined to be the target of the subsequent processing; storing a history of the calculated ratio for each scene; determining whether there are any histories in which the ratios are different in the same scene; If there is a history in which the ratio is different, output a notification; the scene includes a location; the history is stored for each position or attribute of a position sensed by the at least one point cloud detection sensor, In a scene in which the position or the attribute of the position matches, it is determined whether or not there is a history in which the ratio is different. The information processing method according to any one of claims 1 to 8.

10. The predetermined range is a range that does not exceed the height or width of the moving object. The information processing method according to any one of claims 1 to 9.

11. an acquisition unit that acquires a plurality of point information including position information having height information of points sensed by at least one point cloud detection sensor provided in the moving body; a search unit that calculates a distance between a first point in the point cloud indicated by the plurality of point information and one or more other points other than the first point in the point cloud indicated by the plurality of point information, and searches for a point among the one or more other points whose calculated distance is less than a threshold value as a second point within a predetermined range from the first point; a calculation unit that calculates a first difference that is a difference in elevation between the first point and the second point; a determination unit that determines whether the first difference is equal to or greater than a threshold, and determines that the second point is not to be subjected to subsequent processing, which is object detection processing, distance detection processing, or shape detection processing, if the first difference is less than the threshold, and determines that the second point is to be subjected to the subsequent processing, which is object detection processing, distance detection processing, or shape detection processing, if the first difference is equal to or greater than the threshold; an output unit that outputs point information of the second point determined to be a target of the subsequent processing. Information processing device.

12. An acquisition unit that acquires a plurality of point information including position information having height information of points sensed by at least one point cloud detection sensor provided on a moving body; a search unit that calculates a distance between a first point in the point cloud indicated by the plurality of point information and one or more other points other than the first point in the point cloud indicated by the plurality of point information, and searches for a point among the one or more other points whose calculated distance is less than a threshold value as a second point within a predetermined range from the first point; a calculation unit that calculates a second difference, which is an angle between a line connecting the first point and a reference point, which is a point on the moving body or a point outside the moving body that has a predetermined relationship with the moving body, and a line connecting the reference point and the second point; a determination unit that determines whether the second difference is equal to or greater than a threshold, and determines that the second point is not to be subjected to subsequent processing, which is object detection processing, distance detection processing, or shape detection processing, if the second difference is less than the threshold, and determines that the second point is to be subjected to the subsequent processing, which is object detection processing, distance detection processing, or shape detection processing, if the second difference is equal to or greater than the threshold; an output unit that outputs point information of the second point determined to be a target of the subsequent processing. Information processing device.

Images