Information processing device, control method, program, and storage medium

Abstract

To provide an information processing device capable of appropriately processing point group information output from a measurement device even when there is a variation in the measurement device, a control method, a program, and a storage medium storing the program.SOLUTION: An information processing device functionally has acquisition means, detection means, and processing means. The acquisition means acquires point group information, which is a set of data representing points measured by a measurement device for each measurement direction. The detection means detects a variation greater than or equal to a predetermined degree of the measurement device. The processing means determines a search range for searching for a point of the point group information obtained at a previous processing time corresponding to each point of the point group information obtained at a current processing time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a technique for processing measured data. [Background technology]

[0002] A distance measuring device has been known that irradiates a measurement object with light, detects the light reflected from the measurement object, and calculates the distance to the measurement object based on the time difference between the time when the light is irradiated to the measurement object and the time when the light reflected from the measurement object is detected. Patent Document 1 also discloses a forward vehicle recognition device that changes a lighting pattern for projecting a light projection pattern depending on the detection state of the light projection pattern, and detects the distance and inclination to a forward vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-082750 Summary of the Invention [Problem to be solved by the invention]

[0004] When a vehicle equipped with a measuring device such as a lidar passes over a road with bumps, the inclination of the measuring device fluctuates significantly, making it difficult to match the point cloud information output by the measuring device before the change with the point cloud information output by the measuring device after the change.

[0005] The present disclosure provides an information processing device, a control method, a program, and a storage medium storing the program, which are capable of suitably processing point cloud information output by a measurement device even when there is a change in the measurement device. [Means for solving the problem]

[0006] The claimed invention is an acquisition means for acquiring point cloud information, which is a collection of data representing points measured by the measurement device for each measurement direction; a detection means for detecting fluctuations of the measuring device that are equal to or greater than a predetermined level; a processing means for determining a search range for searching for points of the second point cloud information, which is the point cloud information obtained at the previous processing time, corresponding to each point of the first point cloud information, which is the point cloud information obtained at the current processing time, based on whether or not the variation is detected; The information processing device has the following.

[0007] The claimed invention also includes: Acquire point cloud information, which is a collection of data representing points measured by the measurement device in each measurement direction; Detecting fluctuations of the measuring device that are greater than or equal to a predetermined degree; determining a search range for searching for points of the second point cloud information, which is the point cloud information obtained at the previous processing time, corresponding to each of the points of the first point cloud information, which is the point cloud information obtained at the current processing time, based on whether or not the variation is detected; It is a control method.

[0008] The claimed invention also includes: Acquire point cloud information, which is a collection of data representing points measured by the measurement device in each measurement direction; Detecting fluctuations of the measuring device that are greater than or equal to a predetermined degree; This is a program that causes a computer to execute a process of determining a search range for searching for points of the second point cloud information, which is the point cloud information obtained at the previous processing time, that correspond to each of the points of the first point cloud information, which is the point cloud information obtained at the current processing time, based on whether or not the variation is detected. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows a schematic configuration of a lidar according to a first embodiment. [Figure 2] 3 is an example of a flowchart according to the first embodiment. [Figure 3]10A shows an example of a search range when pitch fluctuation is not taken into consideration, and FIG. 10B shows an example of a search range that is set when fluctuation is detected in the fluctuation detection process. [Figure 4] 1 shows a schematic configuration of a lidar according to a second embodiment. [Figure 5] 10 is an example of a flowchart according to a second embodiment. [Figure 6] FIG. 10 is a configuration diagram of a lidar system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] In a preferred embodiment of the present invention, an information processing device includes an acquisition means for acquiring point cloud information, which is a set of data representing points measured by a measurement device for each measurement direction, a detection means for detecting fluctuations of the measurement device that are equal to or greater than a predetermined degree, and a processing means for determining, based on whether or not the fluctuations are detected, a search range in which to search for points in the second point cloud information, which is the point cloud information obtained at a previous processing time, that correspond to each of the points in the first point cloud information, which is the point cloud information obtained at the current processing time. In this aspect, the information processing device can suitably set a search range that allows accurate correspondence between the first point cloud information corresponding to the current processing time and the second point cloud information corresponding to the previous processing time, even if fluctuations occur in the measurement device.

[0011] In one aspect of the information processing device, when the variation is detected, the processing means expands the search range in the direction of the variation, thereby enabling the information processing device to set a search range that allows accurate correspondence between the first point cloud information and the second point cloud information based on the direction of the variation.

[0012] In another aspect of the information processing device, the detection means detects the fluctuation in at least one of a pitch direction, a yaw direction, and a roll direction of the measurement device. In this way, the information processing device can set a search range corresponding to the fluctuation in the pitch direction, the yaw direction, or the roll direction of the measurement device.

[0013] In another aspect of the information processing device, the processing means detects the direction in which the fluctuation occurred among the positive direction and the negative direction of the pitch direction, the yaw direction, and the roll direction, and expands the search range in the direction opposite to the detected direction. This allows the information processing device to expand the search range in a necessary direction when a fluctuation is detected.

[0014] In another aspect of the information processing device, the detection means detects the variation based on the first point cloud information and the second point cloud information. In a preferred example, the detection means detects the variation based on a comparison result between the first point cloud information and the second point cloud information based on ground points, which are the points representing the ground. In another preferred example, the detection means detects the variation based on a comparison result between the first point cloud information and the second point cloud information, which are based on the points representing a measurement distance equal to or greater than a predetermined distance. These aspects enable the information processing device to preferably detect variation in the measurement device.

[0015] In another aspect of the information processing device, the detection means detects the fluctuation based on a detection signal output by a sensor provided on the measuring device or a mobile object on which the measuring device is provided. This aspect also enables the information processing device to preferably detect fluctuations of the measuring device.

[0016] In another preferred embodiment of the present invention, a control method is executed by an information processing device, which acquires point cloud information, which is a collection of data representing points measured by a measurement device for each measurement direction, detects fluctuations of the measurement device that are equal to or greater than a predetermined degree, and, based on whether the fluctuations are detected, determines a search range in which to search for points in second point cloud information, which is the point cloud information obtained at a previous processing time, that correspond to each of the points in first point cloud information, which is the point cloud information obtained at a current processing time. By executing this control method, the information processing device can suitably set a search range that enables accurate correspondence between the first point cloud information corresponding to the current processing time and the second point cloud information corresponding to the previous processing time, even if fluctuations occur in the measurement device.

[0017] In another preferred embodiment of the present invention, a program causes a computer to execute a process of acquiring point cloud information, which is a collection of data representing points measured by a measurement device for each measurement direction, detecting fluctuations of the measurement device that are equal to or greater than a predetermined degree, and determining a search range for searching for points of second point cloud information, which is the point cloud information obtained at a previous processing time, that correspond to each of the points of first point cloud information, which is the point cloud information obtained at a current processing time, based on whether or not the fluctuations are detected. By executing this program, the computer can preferably set a search range that allows accurate correspondence between the first point cloud information corresponding to the current processing time and the second point cloud information corresponding to the previous processing time, even if fluctuations occur in the measurement device. Preferably, the program is stored in a storage medium. [Example]

[0018] Preferred embodiments of the present invention will now be described with reference to the drawings.

[0019] <First Example> (1) Device configuration 1 shows a schematic configuration of a LIDAR 100 according to a first embodiment. The LIDAR 100 is mounted on a vehicle that provides driving assistance such as autonomous driving, for example. The LIDAR 100 emits laser light over a predetermined angular range in the horizontal and vertical directions and receives light that is reflected by an object and returns (also referred to as "reflected light"), thereby discretely measuring the distance from the LIDAR 100 to the object and generating point cloud information that indicates the three-dimensional position of the object.

[0020] As shown in FIG. 1, the lidar 100 mainly includes a transmitter 1, a receiver 2, a beam splitter 3, a scanner 5, a piezoelectric sensor 6, a controller 7, and a memory 8.

[0021] The transmitter 1 is a light source that emits pulsed laser light toward the beam splitter 3. The transmitter 1 includes, for example, an infrared laser light emitting element. The transmitter 1 is driven based on a drive signal “Sg1” supplied from the controller 7.

[0022] The receiver 2 is, for example, an avalanche photodiode, generates a detection signal “Sg2” corresponding to the amount of received light, and supplies the generated detection signal Sg2 to the controller 7.

[0023] The beam splitter 3 transmits the pulsed laser light emitted from the transmitter 1. The beam splitter 3 also reflects the light reflected by the scanner 5 towards the receiver 2.

[0024] The scanner 5 is, for example, an electrostatically driven mirror (MEMS mirror), and its tilt (i.e., the angle of optical scanning) changes within a predetermined range based on the drive signal "Sg3" supplied from the control unit 7. The scanner 5 reflects the laser light that has passed through the beam splitter 3 toward the outside of the LIDAR 100, and also reflects reflected light that enters from the outside of the LIDAR 100 toward the beam splitter 3. In addition, a point measured by irradiating the laser light within the measurement range of the LIDAR 100, or the measurement data thereof, is also referred to as a "measured point."

[0025] The scanner 5 is also provided with a piezoelectric sensor 6. The piezoelectric sensor 6 detects distortion caused by stress of a torsion bar that supports the mirror portion of the scanner 5. The piezoelectric sensor 6 supplies the generated detection signal "Sg4" to the control unit 7. The detection signal Sg4 is used to detect the orientation of the scanner 5.

[0026] The memory 8 is composed of various types of volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The memory 8 stores programs required for the control unit 7 to execute predetermined processes. The memory 8 also stores various parameters referenced by the control unit 7. The memory 8 also stores point cloud information for the latest predetermined number of frames generated by the control unit 7.

[0027] The control unit 7 includes various processors, such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The control unit 7 executes a program stored in the memory 8 to perform predetermined processing. The control unit 7 is an example of a computer that executes a program. The control unit 7 is not limited to being realized by software according to a program, but may be realized by any combination of hardware, firmware, and software. The control unit 7 may also be a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller, or may be an ASSP (Application Specific Standard Produce), ASIC (Application Specific Integrated Circuit), or the like.

[0028] The control unit 7 functionally includes a transmission driving block 70, a scanner driving block 71, a point cloud information generation block 72, and a point cloud information processing block 73.

[0029] The transmission drive block 70 outputs a drive signal Sg1 that drives the transmission unit 1. The drive signal Sg1 includes information for controlling the emission time of a laser light-emitting element included in the transmission unit 1 and the emission intensity of the laser light-emitting element. The transmission drive block 70 controls the emission intensity of the laser light-emitting element included in the transmission unit 1 based on the drive signal Sg1.

[0030] The scanner driving block 71 outputs a driving signal Sg3 for driving the scanner 5. This driving signal Sg3 includes a horizontal driving signal corresponding to the resonance frequency of the scanner 5 and a vertical driving signal for vertical scanning. The scanner driving block 71 also monitors a detection signal Sg4 output from the piezo sensor 6 to detect the scanning angle of the scanner 5 (i.e., the emission direction of the laser light).

[0031] Based on the detection signal Sg2 supplied from the receiving unit 2, the point cloud information generation block 72 generates point cloud information indicating the distance (measurement distance) to an object irradiated with laser light for each measurement direction (i.e., the emission direction of the laser light) with the lidar 100 as the reference point. In this case, the point cloud information generation block 72 calculates the time from when the laser light is emitted until the receiving unit 2 detects the reflected light as the time of flight of light. Then, the point cloud information generation block 72 generates point cloud information indicating a set of points corresponding to a combination of the measurement distance according to the calculated time of flight and the emission direction of the laser light corresponding to the reflected light received by the receiving unit 2, and supplies the generated point cloud information to the point cloud information processing block 73. Hereinafter, the point cloud information obtained by one scan of all measurement points will be referred to as point cloud information for one frame. Here, the point cloud information can be considered as an image in which each measurement direction is a pixel and the measurement distance in each measurement direction is a pixel value. In this case, the emission direction of the laser light varies depending on the elevation and depression angles of the vertically arranged pixels, and the emission direction of the laser light varies depending on the horizontally arranged pixels. Then, for each pixel, a coordinate value in a three-dimensional coordinate system based on the LIDAR 100 is calculated based on the corresponding pair of emission direction and measurement distance.

[0032] The point cloud information processing block 73 removes noise data (false alarm data) generated by erroneously detecting an object in the point cloud information. Hereinafter, measurement points corresponding to data generated by detecting an actual object will be referred to as “valid points,” and measurement points other than valid points (i.e., measurement points corresponding to noise data) will be referred to as “invalid points.” The point cloud information processing block 73 searches for corresponding measurement points in a frame of point cloud information obtained at the current processing time (also referred to as “current frame”) and a frame of point cloud information obtained in the past (also referred to as “past frame”), and determines whether the point is valid based on the search results. Furthermore, in this case, if the point cloud information processing block 73 detects a fluctuation in the pitch direction of the lidar 100, it expands the search range (search range) to accurately determine whether the point is valid regardless of whether the lidar 100 is fluctuating. The current frame is an example of “first point cloud information,” and the past frame is an example of “second point cloud information.” The point cloud information processing block 73 is an example of an "acquisition means," a "detection means," and a "processing means." Furthermore, the LIDAR 100 excluding the point cloud information processing block 73 is an example of a "measurement device."

[0033] As a mode of removing noise data, the point cloud information processing block 73 may add flag information to the point cloud information, indicating whether each measured point is a valid point or not, instead of deleting the noise data from the point cloud information. The point cloud information processing block 73 may also supply the point cloud information after noise data removal processing to an external device outside the LIDAR 100, or to another processing block within the LIDAR 100 that performs obstacle detection, etc. In the former case, the point cloud information may be output to, for example, a device (also referred to as a "driving assistance device") that controls driving assistance such as automatic driving of a vehicle. In this case, for example, the vehicle is controlled to at least avoid obstacle points based on the point cloud information. The driving assistance device may be, for example, an ECU (Electronic Control Unit) of the vehicle, or an on-board device such as a car navigation device electrically connected to the vehicle. The point cloud information processing block 73 stores the point cloud information for each frame in the memory 8 in association with time information indicating the processing time for each frame. Furthermore, the lidar 100 is not limited to a scanning type lidar that scans a field of view with laser light, but may also be a flash type lidar that generates three-dimensional data by irradiating a diffused laser light within the field of view of a two-dimensional array sensor.

[0034] (2) Processing Overview 2 is an example of a flowchart showing a procedure for processing related to point cloud information (point cloud information processing). The LIDAR 100 repeatedly executes the point cloud information processing for each cycle of generating point cloud information for one frame.

[0035] First, the point cloud information generation block 72 generates point cloud information based on the detection signal Sg2 (step S01). In this case, the point cloud information generation block 72 generates point cloud information of the current frame based on the detection signal Sg2 generated by one scan of the scanning target range of the LIDAR 100.

[0036] Next, the point cloud information processing block 73 executes a noise removal process, which is a process for removing noise data from the point cloud information generated in step S01 (step S02). In this case, for example, the point cloud information processing block 73 determines whether each measured point in the current frame is a valid point based on the current frame and the previous frame. For example, the point cloud information processing block 73 sets a range (also referred to as a "search range") for searching for corresponding measured points in the current frame and the previous frame for each valid point in the previous frame. Then, the point cloud information processing block 73 searches for a measured point in the current frame within the search range whose measured distance differs from that of a valid point in the target previous frame within a predetermined distance difference. Then, the point cloud information processing block 73 sets as valid points those measured points in the current frame whose measured distances are similar to those of a predetermined number or more of valid points in the previous frame, and sets the others as invalid points. Since the search range here does not take into account the presence or absence of fluctuation of the Rider 100 in the pitch direction (corresponding to the normal search range described later), if there is fluctuation of the Rider 100 in the pitch direction, there will be measurement points that are determined to be invalid points even though they are valid points. Such measurement points will be corrected to valid points by the pitch fluctuation response processing described later.

[0037] The point cloud information processing block 73 may perform any noise removal process based on the point cloud information of the current frame, regardless of the point cloud information of the past frame. For example, the point cloud information processing block 73 may regard as invalid points any measurement point where the intensity of reflected light received by the receiving unit 2 is less than a predetermined threshold. In another example, the point cloud information processing block 73 may perform clustering of measurement points using Euclidean distance based on the measurement distance, and may regard as invalid points any measurement points that belong to clusters with less than a predetermined number of elements.

[0038] Next, the point cloud information processing block 73 executes a fluctuation detection process, which is a process for detecting fluctuations in the pitch direction of the lidar 100, based on the point cloud information (step S03). As a result, when the lidar 100 fluctuates in the pitch direction due to the vehicle passing over a bump, a step, or the like, the point cloud information processing block 73 accurately detects the fluctuation.

[0039] Then, if a fluctuation in the pitch direction is detected by the fluctuation detection process (step S04; Yes), the point cloud information processing block 73 executes pitch fluctuation response processing, which is processing for determining valid points assuming a fluctuation in the pitch direction of the lidar 100 (step S05). In this case, the point cloud information processing block 73 sets a search range for searching for corresponding measured points between the current frame and the past frame, and sets some of the measured points determined to be invalid points in step S02 as valid points based on the search results in that search range. On the other hand, if the point cloud information processing block 73 does not detect a fluctuation in the pitch direction by the fluctuation detection process (step S04; No), it ends the processing of the flowchart.

[0040] (3) Fluctuation detection processing Next, a specific example of the change detection process based on point cloud information executed by the point cloud information processing block 73 in step S03 will be described.

[0041] The point cloud information processing block 73 matches valid points in a past frame (for example, a past frame one processing time before) with valid points in the current frame that were determined to be valid points in the noise removal process. Then, based on the matching result (collation result), the point cloud information processing block 73 determines that there has been a fluctuation in the pitch direction of the lidar 100 if the fluctuation range in the pitch direction from the past frame (i.e., the number of lines in the horizontal scan or the number of pixels that have changed in the vertical direction when the point cloud information is considered as an image) is equal to or greater than a predetermined range.

[0042] Here, the point cloud information processing block 73 may use individual valid points, measured points representing the ground (also referred to as "ground points"), clusters based on clustering of valid points, or detected objects if object detection processing (such as instance segmentation) has been performed. In this case, the point cloud information processing block 73 may perform matching of the matching targets based on any matching method (optimization method) and calculate the above-mentioned fluctuation range. Note that when performing the above-mentioned matching based on ground points, the point cloud information processing block 73 may, for example, estimate a plane representing the ground based on the valid points determined in the noise removal processing of step S02, and determine valid points that are located at a position higher than the plane by a predetermined threshold or more as obstacle points, and determine other valid points as ground points. Then, the point cloud information processing block 73 calculates the fluctuation range between the ground points in the current frame and the ground points in the previous frame.

[0043] Furthermore, the point cloud information processing block 73 may define the space in which the above-described matching is performed as the entire field of view of the LIDAR 100, or may define it as a space that is at least a predetermined distance away. Generally, when the LIDAR 100 moves in the pitch direction, the more distant the object, the greater the range of the above-described movement in the pitch direction. Therefore, the point cloud information processing block 73 may perform the above-described matching on valid points (or clusters or objects) whose measurement distance is at least a predetermined distance.

[0044] Furthermore, if tracking has been performed on an object-by-object or cluster-by-cluster basis, the point cloud information processing block 73 may detect a change in the pitch direction of the LIDAR 100 based on the tracking results. In this case, the point cloud information processing block 73 predicts the position of an object or cluster in the current frame from a past frame, and determines that a change in the pitch direction of the LIDAR 100 has occurred if the deviation in the pitch direction between the predicted position and the actual position in the current frame is equal to or greater than a predetermined amount.

[0045] (4) Search range settings Next, the pitch fluctuation response processing executed in step S05 will be described in detail. In the pitch fluctuation response processing, when a fluctuation is detected in the fluctuation detection processing, the point cloud information processing block 73 searches for a measured point in the current frame that corresponds to an effective point in the past frame based on a search range expanded in the vertical direction corresponding to the pitch direction in which the fluctuation occurred. As a result, even if the lidar 100 fluctuates in the pitch direction when the vehicle passes over a bump, step, etc., the point cloud information processing block 73 searches for a measured point that corresponds between the past frame and the current frame and accurately determines the effective point.

[0046] Fig. 3(A) shows an example of a search range when pitch fluctuation is not taken into consideration (also called a "normal search range"). Points Pi, Pi1 to Pi6 in Fig. 3(A) are measurement points represented on a virtual two-dimensional plane facing the rider 100.

[0047] The point cloud information processing block 73 sets a normal search range in the noise removal process of step S02 and searches for measured points in the current frame that correspond to valid points in the past frame. Specifically, the point cloud information processing block 73 sequentially sets each valid point in the past frame as a point of interest Pi and sets a normal search range for the set point of interest Pi. The point cloud information processing block 73 then compares the measured distances of points Pi1 to Pi6 in the current frame that exist within the normal search range with the measured distance of point of interest Pi in the past frame. The point cloud information processing block 73 then sets as valid points those measured points in the current frame that are close in measurement distance to a predetermined number or more valid points in the past frame, and sets the other measured points as invalid points.

[0048] Note that the measurement directions of points Pi1 to Pi6 within the normal search range are similar to the measurement direction of the attention point Pi in the horizontal and vertical directions. Specifically, points Pi2 and Pi3 are located on the left and right of the attention point Pi on one line, and point Pi6 is located at the same position on the second line from the attention point Pi. Points Pi1 and Pi4 are located on the left and right of the attention point Pi on one line below, and point Pi5 is located at the same position on the second line below the attention point Pi.

[0049] 3(B) shows an example of a search range (also referred to as a "varied search range") that is set when a variation is detected in the variation detection process. In this case, the point cloud information processing block 73 expands the varied search range beyond the normal search range in the vertical direction corresponding to the pitch direction in which the variation of the lidar 100 occurred. Specifically, the varied search range is set to a range that includes the normal search range and also includes points Pi8 and Pi9 three lines above the target point Pi, point Pi12 four lines above, points Pi7 and Pi10 three lines below, and point Pi11 four lines below.

[0050] In the pitch fluctuation accommodation processing, the point cloud information processing block 73 sequentially sets valid points in past frames as attention points Pi, and searches for measured points in the current frame whose measurement distance is within a predetermined distance difference from the measurement distance of the attention point Pi in the past frame based on the variation search range. As a result of the search, the point cloud information processing block 73 sets, as valid points, measured points in the current frame whose measurement distance is within a predetermined distance difference from a predetermined number or more of valid points in the past frames (specifically, points determined to be invalid points in the noise removal processing). The point cloud information processing block 73 may perform the pitch fluctuation accommodation processing using a past frame corresponding to the processing time one frame before the current frame, or may perform the pitch fluctuation accommodation processing using past frames corresponding to multiple processing times. In this case, the point cloud information processing block 73 may set, as valid points, measured points in the current frame corresponding to valid points in any of the past frames.

[0051] In this way, when a fluctuation is detected in the fluctuation detection process, the point cloud information processing block 73 performs pitch fluctuation response processing based on a fluctuating search range that is expanded from the normal search range in the vertical direction corresponding to the pitch direction in which the fluctuation occurred. As a result, when the lidar 100 fluctuates in the pitch direction when the vehicle passes over a bump, step, etc., the point cloud information processing block 73 can accurately search for corresponding measurement points in the past frame and the current frame and accurately determine the valid points.

[0052] Furthermore, the point cloud information processing block 73 may set a variable search range depending on whether the lidar 100 has changed in the positive direction (e.g., the direction in which the elevation angle increases) or the negative direction (the direction in which the depression angle increases) in the pitch direction (i.e., depending on the change in the change over time). For example, when the point cloud information processing block 73 detects that the lidar 100 has changed in the direction in which the elevation angle increases, the point cloud information processing block 73 sets a variable search range by expanding the normal search range downward because each object moves relatively downward in the current frame. Similarly, when the point cloud information processing block 73 detects that the lidar 100 has changed in the direction in which the depression angle increases, the point cloud information processing block 73 sets a variable search range by expanding the normal search range upward. In this way, the point cloud information processing block 73 sets a variable search range by expanding the normal search range in the direction opposite to the direction of the change (here, the positive or negative direction in the pitch direction). This makes it possible to set a search range according to the change in the lidar 100.

[0053] (5) Variations Next, preferred modifications of the above-described embodiment will be described. The following modifications may be applied to the above-described embodiment in combination.

[0054] (Variation 1) In addition to or instead of the process of detecting fluctuations of the LIDAR 100 in the pitch direction, the point cloud information processing block 73 may detect fluctuations of the LIDAR 100 in the yaw direction or roll direction. In this case, when the point cloud information processing block 73 detects such fluctuations, it executes processing similar to the pitch fluctuation response processing using the past frame and the current frame based on a fluctuation search range that is an expanded search range in a direction corresponding to the direction of the detected fluctuation. This allows the point cloud information processing block 73 to accurately determine the valid point of the current frame even if fluctuations of the LIDAR 100 occur in directions other than the pitch direction.

[0055] (Variation 2) The use of the above-mentioned search range is not limited to determining whether or not there is a correspondence with a valid point in a past frame in the noise removal process. For example, when using a search range in object detection (including tracking), the point cloud information processing block 73 may expand the search range according to fluctuations in the LIDAR 100.

[0056] For example, when tracking a tracked object, the point cloud information processing block 73 sets a search range based on the predicted position of the tracked object or cluster in the current frame based on the detection result of the tracked object in a past frame using any tracking technology, and searches for the tracked object within that search range. In this case, the point cloud information processing block 73 expands the search range (varied search range) when a variation is detected in the variation detection process beyond the normal search range. This allows the point cloud information processing block 73 to properly perform tracking even if a variation occurs in the lidar 100 due to the vehicle passing over a bump or step.

[0057] <Second Example> 4 shows a schematic configuration of a LIDAR 100A according to the second embodiment. The LIDAR 100A differs from the LIDAR 100 according to the first embodiment in that the LIDAR 100A performs fluctuation detection processing based on information output by a sensor 9, which is a sensor provided in the LIDAR 100A or in the vehicle on which the LIDAR 100A is mounted. Hereinafter, the same components of the LIDAR 100A as those in the LIDAR 100 according to the first embodiment will be appropriately designated by the same reference numerals, and their description will be omitted.

[0058] The sensor 9 is a sensor provided on the vehicle on which the LIDAR 100A is mounted or on the LIDAR 100A, and detects fluctuations of the vehicle or the LIDAR 100A. The sensor 9 may be an acceleration sensor or any other sensor used for vibration detection. The sensor 9 supplies a detection signal Sg5 to the point cloud information processing block 73.

[0059] The point cloud information processing block 73 executes a fluctuation detection process, which is a process for detecting fluctuations in the pitch direction of the rider 100A that are equal to or greater than a predetermined degree, based on the detection signal Sg5. For example, the point cloud information processing block 73 detects that the rider 100A has fluctuated in the pitch direction when the degree of vibration indicated by the detection signal Sg5 is equal to or greater than a predetermined degree. Then, when a fluctuation is detected in the fluctuation detection process, the point cloud information processing block 73 performs a noise removal process using a fluctuating search range, and when a fluctuation is not detected in the fluctuation detection process, the point cloud information processing block 73 performs a noise removal process using a normal search range.

[0060] 5 is an example of a flowchart showing the procedure of point cloud information processing according to Example 2. The LIDAR 100A repeatedly executes point cloud information processing for each cycle of generating point cloud information for one frame.

[0061] First, the point cloud information generation block 72 generates point cloud information based on the detection signal Sg2 (step S11). Next, the point cloud information processing block 73 executes a fluctuation detection process based on the detection signal Sg5 output by the sensor 9 (step S12). As a result, when the lidar 100 fluctuates in the pitch direction due to the vehicle passing over a bump, a step, or the like, the point cloud information processing block 73 accurately detects the fluctuation.

[0062] If a pitch fluctuation is detected by the fluctuation detection process (step S13; Yes), the point cloud information processing block 73 executes pitch fluctuation-adaptive noise removal processing, which is a process for removing noise data based on past and current frames assuming pitch fluctuation (step S14). In this case, the point cloud information processing block 73 sets a fluctuation search range for each valid point in the past frame and searches for measurement points in the current frame whose measurement distance is within a predetermined distance. The point cloud information processing block 73 then sets as valid points those measurement points in the current frame whose measurement distance is similar to that of a predetermined number or more valid points in the past frame, and sets the others as invalid points. The point cloud information processing block 73 also executes processes such as setting as invalid points those measurement points whose intensity of reflected light received by the receiving unit 2 is below a predetermined threshold, and clustering the measurement points to set as invalid points those measurement points that belong to clusters with fewer than a predetermined number of elements.

[0063] On the other hand, if no fluctuation in the pitch direction is detected by the fluctuation detection process (step S13; No), the point cloud information processing block 73 executes normal noise removal processing (step S15). In this case, the point cloud information processing block 73 executes the same noise removal processing as step S02 in the first embodiment.

[0064] <Third Example> 6 is a configuration diagram of a LIDAR system according to the third embodiment. In the third embodiment, the point cloud information processing block 73 of the control unit 7 and functions corresponding to the point cloud information processing block 73 are provided in a device separate from the LIDAR 100X. Hereinafter, elements of the third embodiment that are the same as those in the first or second embodiment will be appropriately designated by the same reference numerals, and their description will be omitted.

[0065] The LIDAR system according to the third embodiment includes a LIDAR 100X and an information processing device 200. In this case, the LIDAR 100X supplies the point cloud information generated by the point cloud information generation block 72 to the information processing device 200.

[0066] The information processing device 200 has a control unit 7A and a memory 8. The memory 8 stores information necessary for the control unit 7A to execute processing. The control unit 7A functionally has a point cloud information acquisition block 72A and a point cloud information processing block 73. The point cloud information acquisition block 72A receives point cloud information generated by the point cloud information generation block 72 of the LIDAR 100X and supplies the received point cloud information to the point cloud information processing block 73. The point cloud information processing block 73 performs the same processing on the point cloud information supplied from the point cloud information acquisition block 72A as the point cloud information processing block 73 of the above-described embodiment.

[0067] The information processing device 200 may be realized by a driving assistance device. Furthermore, information on parameters necessary for processing may be stored in another device having a memory that can be referenced by the information processing device 200. Even with the configuration of this modified example, the information processing device 200 can accurately remove noise from the point cloud information generated by the lidar 100X.

[0068] As described above, the control unit 7 of the LIDAR 100 according to the first or second embodiment or the control unit 7A of the information processing device 200 according to the third embodiment functions as an information processing device of the present invention and functionally includes an acquisition unit, a detection unit, and a processing unit. The acquisition unit acquires point cloud information, which is a collection of data representing points measured by the measurement device for each measurement direction. The detection unit detects fluctuations of the measurement device that are greater than or equal to a predetermined degree. Based on whether or not fluctuations are detected, the processing unit determines a search range in which to search for points of point cloud information obtained at the previous processing time that correspond to each point of the point cloud information obtained at the current processing time. This makes it possible to recognize the correspondence between each point included in the point cloud information at the current processing time and the previous processing time, even if fluctuations occur in the measurement device, and to appropriately perform noise removal processing, tracking processing, and the like.

[0069] In the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a controller or the like that is a computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)).

[0070] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent documents and other documents are incorporated herein by reference. [Explanation of symbols]

[0071] 1. Transmitter 2. Receiving section 3 Beam splitter 5. Scanner 6 Piezo Sensors 7, 7A control section 8. Memory 100, 100X Lidar 200 Information processing device

Claims

[Claim 1] an acquisition means for acquiring point cloud information, which is a collection of data representing points measured by the measurement device for each measurement direction; a detection means for detecting fluctuations of the measuring device that are equal to or greater than a predetermined level; a processing means for determining a search range for searching for points of the second point cloud information, which is the point cloud information obtained at the previous processing time, corresponding to each of the points of the first point cloud information, which is the point cloud information obtained at the current processing time, based on whether or not the variation is detected; An information processing device having the above.

Images