Information Processing Apparatus, Control Method, Program, and Storage Medium

Abstract

This information processing device functionally comprises an acquisition means, a detection means, and a processing means. The acquisition means acquires point group information, which is an aggregate of data representing points measured by a measuring device per each measuring direction. The detection means detects a fluctuation, of a specified degree or more, of the measuring device. On the basis of the presence or absence of a detection of the fluctuation, the processing means determines a searching range in which to search for a point from point group information obtained at a prior processing time corresponding to each point from point group information obtained at a current processing time.

Description

Technical Field

[0001] The present disclosure relates to a technique for processing measured data.

Background Art

[0002] Conventionally, a distance measuring device is known that irradiates an object to be measured with light, detects the reflected light from the object to be measured, and calculates the distance to the object to be measured based on the time difference between the timing of irradiating the object to be measured with light and the timing of detecting the reflected light from the object to be measured. Further, Patent Document 1 discloses a forward vehicle recognition device that changes the lighting pattern for projecting the projection pattern according to the detection state of the projection pattern and detects the distance and inclination from the forward vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a vehicle equipped with a measuring device such as a lidar passes through a road with a step, the inclination of the measuring device fluctuates greatly, making it difficult to associate the point cloud information output by the measuring device before the fluctuation with the point cloud information output by the measuring device after the fluctuation.

[0005] As an example of the problems to be solved by the present invention, the above is mentioned. The main object of the present disclosure is to provide an information processing device, a control method, a program, and a storage medium storing the program that can preferably process the point cloud information output by the measuring device even when there is a change in the measuring device.

Means for Solving the Problems

[0006] The invention described in the claims is An acquisition means for acquiring point cloud information which is a set of data representing points measured by a measuring device for each measurement direction, A detection means for detecting fluctuations in the measuring device equal to or greater than a predetermined degree, Based on the presence or absence of the detection of the fluctuations, a search range for searching for points of second point cloud information which is the point cloud information obtained at a pre-processing time corresponding to each point of the first point cloud information which is the point cloud information obtained at the current processing time is determined Determination means; Search means for searching, based on the measurement distance by the measurement device for each point of the first point group information and the measurement distance by the measurement device for the points of the second point group information existing within the determined search range for each point of the first point group information, for points corresponding to each point of the first point group information from the points of the second point group information existing within the search range; An information processing apparatus comprising the above.

[0007] Also, the invention according to the claim is A computer acquires point cloud information which is a set of data representing points measured by a measuring device for each measurement direction, detects fluctuations in the measuring device equal to or greater than a predetermined degree, Based on the presence or absence of the detection of the fluctuations, a search range for searching for points of second point cloud information which is the point cloud information obtained at a pre-processing time corresponding to each point of the first point cloud information which is the point cloud information obtained at the current processing time is determined and searching, based on the measurement distance by the measurement device for each point of the first point group information and the measurement distance by the measurement device for the points of the second point group information existing within the determined search range for each point of the first point group information, for points corresponding to each point of the first point group information from the points of the second point group information existing within the search range; It is a control method.

[0008] Also, the invention according to the claim is A computer acquires point cloud information which is a set of data representing points measured by a measuring device for each measurement direction, detects fluctuations in the measuring device equal to or greater than a predetermined degree, Based on the presence or absence of the detection of the fluctuations, a search range for searching for points of second point cloud information which is the point cloud information obtained at a pre-processing time corresponding to each point of the first point cloud information which is the point cloud information obtained at the current processing time is determined and searching, based on the measurement distance by the measurement device for each point of the first point group information and the measurement distance by the measurement device for the points of the second point group information existing within the determined search range for each point of the first point group information, for points corresponding to each point of the first point group information from the points of the second point group information existing within the search range It is a program for causing a computer to execute processing.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0010] In a preferred embodiment of the present invention, the information processing apparatus includes an acquisition unit that acquires point cloud information, which is a set of data representing points measured by the measurement apparatus for each measurement direction, a detection unit that detects a variation of the measurement apparatus equal to or greater than a predetermined degree, and a processing unit that determines a search range for searching for points of second point cloud information, which is the point cloud information obtained at a pre-processing time, corresponding to each point of the first point cloud information, which is the point cloud information obtained at the current processing time, based on the presence or absence of the detection of the variation. In this aspect, the information processing apparatus can preferably set a search range that enables accurate association between the first point cloud information corresponding to the current processing time and the second point cloud information corresponding to the pre-processing time even when a variation occurs in the measurement apparatus.

[0011] In one aspect of the information processing apparatus, when the variation is detected, the processing unit expands the search range in the direction in which the variation occurred. Thereby, the information processing apparatus can set a search range that enables accurate association between the first point cloud information and the second point cloud information based on the direction in which the variation occurred.

[0012] In another aspect of the information processing apparatus, the detection means detects the variation in at least one of the pitch direction, yaw direction, or roll direction of the measuring device. In this way, the information processing apparatus can set a search range corresponding to the variation in the pitch direction, yaw direction, or roll direction of the measuring device.

[0013] In another aspect of the information processing apparatus, the processing means detects the direction in which the variation has occurred among the positive or negative directions of the pitch direction, yaw direction, or roll direction, and expands the search range in the direction opposite to the detected direction. Thereby, when the information processing apparatus detects a variation, it can expand the search range in the necessary direction.

[0014] In another aspect of the information processing apparatus, the detection means detects the variation based on the first point group information and the second point group information. In a preferred example, the detection means detects the variation based on the collation result between the first point group information and the second point group information based on the ground points that are the points representing the ground. In another preferred example, the detection means detects the variation based on the collation result between the first point group information and the second point group information based on the points representing a measurement distance of a predetermined distance or more. With these aspects, the information processing apparatus can suitably detect the variation of the measuring device.

[0015] In another aspect of the information processing apparatus, the detection means detects the variation based on a detection signal output by a sensor provided in the measuring device or a moving body on which the measuring device is provided. Also with this aspect, the information processing apparatus can suitably detect the variation of the measuring device.

[0016] In another preferred embodiment of the present invention, there is provided a control method executed by an information processing apparatus, which includes: obtaining point cloud information that is a set of data representing points measured by a measuring device for each measurement direction; detecting a variation of the measuring device equal to or greater than a predetermined degree; and determining a search range for searching for points of second point cloud information, which is the point cloud information obtained at a pre-processing time, corresponding to each point of first point cloud information, which is the point cloud information obtained at a current processing time, based on whether the variation is detected. By executing this control method, the information processing apparatus can suitably set a search range that enables accurate association between the first point cloud information corresponding to the current processing time and the second point cloud information corresponding to the pre-processing time even when a variation occurs in the measuring device.

[0017] In another preferred embodiment of the present invention, a program causes a computer to execute a process of obtaining point cloud information that is a set of data representing points measured by a measuring device for each measurement direction, detecting a variation of the measuring device equal to or greater than a predetermined degree, and determining a search range for searching for points of second point cloud information, which is the point cloud information obtained at a pre-processing time, corresponding to each point of first point cloud information, which is the point cloud information obtained at a current processing time, based on whether the variation is detected. By executing this program, the computer can suitably set a search range that enables accurate association between the first point cloud information corresponding to the current processing time and the second point cloud information corresponding to the pre-processing time even when a variation occurs in the measuring device. Preferably, the above program is stored in a storage medium.

Example

[0018] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0019] <First Embodiment> (1) Device Configuration FIG. 1 shows a schematic configuration of the lidar 100 according to the first embodiment. The lidar 100 is mounted on a vehicle that performs driving assistance such as autonomous driving, for example. The lidar 100 irradiates laser light within a predetermined angular range in the horizontal and vertical directions, and receives the light (also referred to as "reflected light") reflected by an object and returned. By doing so, the lidar 100 discretely measures the distance to the object and generates point cloud information indicating the three-dimensional position of the object.

[0020] As shown in FIG. 1, the lidar 100 mainly includes a transmission unit 1, a reception unit 2, a beam splitter 3, a scanner 5, a piezo sensor 6, a control unit 7, and a memory 8.

[0021] The transmission unit 1 is a light source that emits pulsed laser light toward the beam splitter 3. The transmission unit 1 includes, for example, an infrared laser light emitting element. The transmission unit 1 is driven based on a drive signal "Sg1" supplied from the control unit 7.

[0022] The reception unit 2 is, for example, an avalanche photodiode, generates a detection signal "Sg2" corresponding to the amount of received light, and supplies the generated detection signal Sg2 to the control unit 7.

[0023] The beam splitter 3 transmits the pulsed laser light emitted from the transmission unit 1. Further, the beam splitter 3 reflects the reflected light reflected by the scanner 5 toward the reception unit 2.

[0024] The scanner 5 is, for example, a mirror (MEMS mirror) of an electrostatic drive type, and based on a drive signal "Sg3" supplied from the control unit 7, the inclination (i.e., the angle of optical scanning) changes within a predetermined range. Then, the scanner 5 reflects the laser light transmitted through the beam splitter 3 toward the outside of the lidar 100, and reflects the reflected light incident from the outside of the lidar 100 toward the beam splitter 3. Also, a point measured by irradiating laser light within the measurement range of the lidar 100 or its measurement data is also referred to as a "measured point".

[0025] In addition, the scanner 5 is provided with a piezo sensor 6. The piezo sensor 6 detects the distortion caused by the stress of the torsion bar that supports the mirror part of the scanner 5. The piezo sensor 6 supplies the generated detection signal "Sg4" to the control unit 7. The detection signal Sg4 is used for detecting the orientation of the scanner 5.

[0026] The memory 8 is composed of various volatile memories and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The memory 8 stores programs necessary for the control unit 7 to execute predetermined processes. In addition, the memory 8 stores various parameters referred to by the control unit 7. Further, the memory 8 stores the point cloud information for the latest predetermined number of frames generated by the control unit 7.

[0027] The control unit 7 includes various processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The control unit 7 executes a predetermined process by executing the program stored in the memory 8. The control unit 7 is an example of a computer that executes a program. Note that the control unit 7 is not limited to being realized by software by a program, and may be realized by any combination of hardware, firmware, and software. Further, the control unit 7 may be a user-programmable integrated circuit such as an FPGA (field-programmable gate array) or a microcontroller, or may be an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or the like.

[0028] Functionally, the control unit 7 has a transmission drive block 70, a scanner drive block 71, a point cloud information generation block 72, and a point cloud information processing block 73.

[0029] The transmission drive block 70 outputs a drive signal Sg1 for driving the transmission unit 1. The drive signal Sg1 includes information for controlling the emission time of the laser light emitting element included in the transmission unit 1 and the emission intensity of the laser light emitting element. The transmission drive block 70 controls the emission intensity of the laser light emitting element included in the transmission unit 1 based on the drive signal Sg1.

[0030] The scanner drive block 71 outputs a drive signal Sg3 for driving the scanner 5. This drive signal Sg3 includes a horizontal drive signal corresponding to the resonance frequency of the scanner 5 and a vertical drive signal for vertical scanning. Also, the scanner drive block 71 detects the scanning angle of the scanner 5 (i.e., the emission direction of the laser light) by monitoring the detection signal Sg4 output from the piezo sensor 6.

[0031] Based on the detection signal Sg2 supplied from the reception unit 2, the point cloud information generation block 72 generates point cloud information indicating the distance (measured distance) from the lidar 100 as a reference point to the object irradiated with the laser light for each measurement direction (i.e., the emission direction of the laser light). In this case, the point cloud information generation block 72 calculates the time from when the laser light is emitted until the reception unit 2 detects the reflected light as the time of flight (Time of Flight) of the light. Then, the point cloud information generation block 72 generates point cloud information indicating a set of points corresponding to the pair of the measured distance corresponding to the calculated time of flight and the emission direction of the laser light corresponding to the reflected light received by the reception unit 2, and supplies the generated point cloud information to the point cloud information processing block 73. Hereinafter, the point cloud information obtained by one scan for all the measured points is regarded as the point cloud information for one frame. Here, the point cloud information can be regarded as an image in which each measurement direction is a pixel and the measured distance in each measurement direction is a pixel value. In this case, the emission direction of the laser light in the elevation angle is different in the vertical arrangement of the pixels, and the emission direction of the laser light in the horizontal angle is different in the horizontal arrangement of the pixels. And for each pixel, coordinate values in a three-dimensional coordinate system with the lidar 100 as a reference are obtained based on the corresponding pair of the emission direction and the measured distance.

[0032] The point cloud information processing block 73 removes noise data (false alarm data) generated by erroneously detecting an object in the point cloud information. Hereinafter, the measured points corresponding to the data generated by detecting an actual object are referred to as "valid points", and the measured points other than the valid points (i.e., the measured points corresponding to the noise data) are referred to as "invalid points". The point cloud information processing block 73 searches for corresponding measured points between the frame of the point cloud information obtained at the current processing time (also referred to as the "current frame") and the frame of the point cloud information obtained in the past (also referred to as the "past frame"), and based on the search result, determines whether it is a valid point or not. Further, in this case, when the point cloud information processing block 73 detects a variation in the pitch direction of the lidar 100, it expands the search range (search range) to accurately determine valid points regardless of the presence or absence of the variation of the lidar 100. The current frame is an example of the "first point cloud information", and the past frame is an example of the "second point cloud information". The point cloud information processing block 73 is an example of the "acquisition means", the "detection means", and the "processing means". Also, the lidar 100 excluding the point cloud information processing block 73 is an example of the "measurement device".

[0033] In addition, as a mode of removing noise data, instead of deleting the noise data from the point cloud information, the point cloud information processing block 73 may add flag information indicating whether each measured point is a valid point to the point cloud information. Further, the point cloud information processing block 73 may supply the point cloud information after the noise data removal process to an external device existing outside the lidar 100, or may supply it to another processing block in the lidar 100 that performs obstacle detection or the like. In the former case, the point cloud information may be output to, for example, a device that controls driving assistance such as automatic driving of a vehicle (also referred to as a "driving assistance device"). In this case, for example, vehicle control is performed to at least avoid obstacle points based on the point cloud information. The driving assistance device may be, for example, an ECU (Electronic Control Unit) of the vehicle, or may be an in-vehicle device such as a car navigation device electrically connected to the vehicle. Further, the point cloud information processing block 73 stores the point cloud information for each frame in the memory 8 in association with time information indicating the processing time for each frame. Further, the lidar 100 is not limited to a scanning type lidar that scans laser light over the field of view, and may be a flash type lidar that generates three-dimensional data by diffusely irradiating laser light over the field of view of a two-dimensional array sensor.

[0034] (2) Processing Outline FIG. 2 is an example of a flowchart showing the procedure of processing related to point cloud information (point cloud information processing). The lidar 100 repeatedly executes the point cloud information processing at each cycle of generating the point cloud information for one frame.

[0035] First, the point cloud information generation block 72 generates point cloud information based on the detection signal Sg2 (step S01). In this case, the point cloud information generation block 72 generates the point cloud information for the current frame based on the detection signal Sg2 generated by one scan in the scanning target range of the lidar 100.

[0036] Next, the point cloud information processing block 73 executes a noise removal process, which is a process of removing noise data from the point cloud information generated in step S01 (step S02). In this case, for example, the point cloud information processing block 73 determines whether each measured point in the current frame is a valid point based on the current frame and the past frame. For example, the point cloud information processing block 73 sets a range (also referred to as the "search range") for searching for corresponding measured points in the current frame and the past frame for each valid point in the past frame. Then, the point cloud information processing block 73 searches for the measured points in the current frame whose difference in measurement distance from the valid points in the target past frame within the search range is within a predetermined distance difference. Then, the point cloud information processing block 73 sets the measured points in the current frame whose measurement distances are approximated to a predetermined number or more of valid points in the past frame as valid points, and sets the others as invalid points. Since the search range here does not consider the presence or absence of fluctuations of the lidar 100 in the pitch direction (corresponding to the normal search range described later), when there are fluctuations of the lidar 100 in the pitch direction, there are measured points that are determined to be invalid points despite being valid points. Such measured points will be corrected to valid points by the pitch fluctuation corresponding process described later.

[0037] Note that the point cloud information processing block 73 may execute an arbitrary noise removal process based on the point cloud information of the current frame regardless of the point cloud information of the past frame. For example, the point cloud information processing block 73 regards the measured points whose intensity of the reflected light received by the receiving unit 2 is less than a predetermined threshold as invalid points. In another example, the point cloud information processing block 73 may perform clustering of the measured points in terms of Euclidean distance based on the measurement distance, and regard the measured points belonging to clusters with a number of elements less than a predetermined number as invalid points.

[0038] Next, the point cloud information processing block 73 executes a fluctuation detection process, which is a process of detecting fluctuations in the pitch direction of the lidar 100, based on the point cloud information (step S03). Thereby, the point cloud information processing block 73 accurately detects the fluctuations when the lidar 100 fluctuates in the pitch direction due to the vehicle passing over bumps, steps, etc.

[0039] When the variation in the pitch direction is detected by the variation detection process (step S04; Yes), the point cloud information processing block 73 executes a pitch variation corresponding process, which is a determination process of valid points on the premise of the variation in the pitch direction of the lidar 100 (step S05). In this case, the point cloud information processing block 73 sets a search range for searching for corresponding measured points between the current frame and the past frame, and based on the search result within the search range, sets some of the measured points determined as invalid points in step S02 as valid points. On the other hand, when the variation in the pitch direction is not detected by the variation detection process (step S04; No), the point cloud information processing block 73 ends the flowchart process.

[0040] (3) Change Detection Processing Next, a specific example of the variation detection process based on the point cloud information executed by the point cloud information processing block 73 in step S03 will be described.

[0041] The point cloud information processing block 73 performs matching between the valid points in the past frame (for example, the past frame one processing time before) and the valid points in the current frame determined as valid points in the noise removal process. Then, based on the result of the matching (collation result), when the variation width in the pitch direction from the past frame (that is, the number of horizontal scan lines or the number of variation pixels in the vertical direction when the point cloud information is regarded as an image) is equal to or greater than a predetermined width, it is determined that there is a variation in the pitch direction of the lidar 100.

[0042] Here, the point cloud information processing block 73 may use the object to be matched (the object to be collated) as individual valid points, as measured points representing the ground (also referred to as "ground points"), as clusters based on clustering of valid points, or as detected objects when object detection processing (such as instance segmentation) is performed. In this case, the point cloud information processing block 73 may perform the matching of the collation object based on an arbitrary matching method (optimization method) and calculate the above-described variation width. Note that when the point cloud information processing block 73 performs the above-described matching based on ground points, for example, it estimates a plane representing the ground based on the valid points determined in the noise removal process of step S02, determines valid points existing at positions higher than a predetermined threshold value from the plane as obstacle points, and determines the other valid points as ground points. Then, the point cloud information processing block 73 calculates the variation width between the ground points in the current frame and the ground points in the past frame.

[0043] Also, the point cloud information processing block 73 may define the space for performing the above-described matching as the entire field of view of the lidar 100, or may define it as a space at a distance of a predetermined distance or more. Generally, when there is a variation in the lidar 100 in the pitch direction, the above-described variation width in the pitch direction becomes larger for more distant objects. Therefore, the point cloud information processing block 73 may perform the above-described matching on valid points (or clusters or objects) whose measurement distance is a predetermined distance or more, for example.

[0044] Also, when the point cloud information processing block 73 is performing tracking on an object unit or a cluster unit, it may detect the variation in the pitch direction of the lidar 100 based on the result of the tracking. In this case, the point cloud information processing block 73 predicts the position of the object or cluster from the past frame to the current frame, and determines that there is a variation in the pitch direction of the lidar 100 when the deviation width in the pitch direction between the predicted position and the actual position in the current frame is a predetermined width or more.

[0045] (4) Setting of Search Range Next, the details of the pitch variation corresponding process executed in step S05 will be described. In the pitch variation corresponding process, when the point cloud information processing block 73 detects a variation in the variation detection process, it searches for the valid points of the past frame and the measured points of the current frame corresponding thereto based on the search range expanded in the vertical direction corresponding to the pitch direction in which the variation occurred. Thereby, even when the pitch of the lidar 100 varies in the pitch direction when the vehicle passes over a bump or a step, etc., the point cloud information processing block 73 searches for the corresponding measured points between the past frame and the current frame, and accurately executes the determination of the valid points.

[0046] Fig. 3(A) shows an example of the search range (also referred to as the "normal search range") when pitch variation is not considered. Each point Pi, Pi1~Pi6 in Fig. 3(A) is a measured point represented on a virtual two-dimensional plane facing the lidar 100.

[0047] The point cloud information processing block 73 sets the normal search range in the noise removal process of step S02, etc., and searches for the measured points of the current frame corresponding to the valid points of the past frame. Specifically, the point cloud information processing block 73 sequentially sets each valid point of the past frame as the attention point Pi, and sets the normal search range for the set attention point Pi. Then, the point cloud information processing block 73 compares the measured distances of each point Pi1~Pi6 in the current frame existing within the normal search range with the measured distance of the attention point Pi in the past frame. Then, the point cloud information processing block 73 sets the measured points of the current frame with a measured distance close to a predetermined number or more of valid points of the past frame as valid points, and sets the other measured points as invalid points.

[0048] Note that the points Pi1~Pi6 within the normal search range have measurement directions similar to the measurement direction of the attention point Pi in the horizontal direction (lateral direction) and the vertical direction (perpendicular direction). Specifically, points Pi2 and Pi3 exist on the left and right of the same line as the attention point Pi, and point Pi6 exists at the same position on the second line of the attention point Pi. Points Pi1 and Pi4 exist on the left and right of one line below the attention point Pi, and point Pi5 exists at the same position on the second line below the attention point Pi.

[0049] FIG. 3(B) shows an example of a search range (also referred to as a “search range with variation”) set when variation is detected in the change detection process. In this case, the point cloud information processing block 73 expands the search range with variation in the vertical direction corresponding to the pitch direction in which the variation of the lidar 100 has occurred, as compared with the normal search range. Specifically, the search range with variation includes the normal search range, and is set to a range including the points Pi8 and Pi9 on the 3 lines of the point of interest Pi, the point Pi12 on the 4 lines, the points Pi7 and Pi10 on the 3 lines below, and the point Pi11 on the 4 lines below.

[0050] Then, in the pitch variation corresponding process, the point cloud information processing block 73 sequentially sets the valid points of the past frame as the point of interest Pi, and searches for the measured points of the current frame having a measured distance within a predetermined distance difference from the measured distance of the point of interest Pi of the past frame, based on the search range with variation. Then, as a result of the above search, the point cloud information processing block 73 sets, as valid points, the measured points of the current frame having a measured distance within a predetermined distance difference from a predetermined number or more of valid points of the past frame (specifically, the points determined to be invalid points in the noise removal process). Note that the point cloud information processing block 73 may perform the above-described pitch variation corresponding process using the past frame corresponding to the processing time one frame before the current frame, or may perform the above-described pitch variation corresponding process using the past frames corresponding to a plurality of processing times. In this case, the point cloud information processing block 73 may set, as valid points, the measured points of the current frame corresponding to the valid points of any of the past frames.

[0051] As described above, when the point cloud information processing block 73 detects variation in the change detection process, the point cloud information processing block 73 performs a pitch variation corresponding process based on a search range with variation in which the vertical direction corresponding to the pitch direction in which the variation has occurred is expanded as compared with the normal search range. Thereby, when the lidar 100 varies in the pitch direction when the vehicle passes over a bump, a step, or the like, the point cloud information processing block 73 can accurately search for the corresponding measured points between the past frame and the current frame, and accurately execute the determination of valid points.

[0052] Further, the point cloud information processing block 73 may set a search range with variation according to whether the variation occurs in the positive direction (e.g., the direction in which the elevation angle increases) or the negative direction (the direction in which the depression angle increases) in the pitch direction (i.e., according to the temporal change of the variation). For example, when the point cloud information processing block 73 detects that the lidar 100 has varied in the direction in which the elevation angle increases, since each object moves relatively downward in the current frame, it sets a search range with variation in which the normal search range is expanded downward. Similarly, when the point cloud information processing block 73 detects that the lidar 100 has varied in the direction in which the depression angle increases, it sets a search range with variation in which the normal search range is expanded upward. In this way, the point cloud information processing block 73 sets a search range with variation in which the normal search range is expanded in the direction opposite to the direction of the variation (here, the positive or negative direction in the pitch direction). Thereby, a search range corresponding to the variation of the lidar 100 can be set.

[0053] (5) Modification Example Next, a modification suitable for the above-described embodiment will be described. The following modifications may be applied to the above-described embodiment in combination.

[0054] (Modification 1) In addition to, or instead of, the process of detecting the variation of the lidar 100 in the pitch direction, the point cloud information processing block 73 may detect the variation of the lidar 100 in the yaw direction or the roll direction. In this case, when the point cloud information processing block 73 detects such a variation, it executes a process similar to the pitch variation corresponding process using the past frame and the current frame based on a search range with variation in which the search range is expanded in the direction corresponding to the detected variation direction. Thereby, even when a variation of the lidar 100 other than in the pitch direction occurs, the point cloud information processing block 73 can accurately determine the valid points in the current frame.

[0055] (Modification 2) The use of the above search range is not limited to determining the presence or absence of correspondence with valid points of past frames in noise removal processing. For example, when the point cloud information processing block 73 uses the search range in object detection (including tracking), the search range may be expanded according to the variation of the lidar 100.

[0056] For example, when performing tracking of a tracking target, the point cloud information processing block 73 sets a search range based on the predicted position of the object or cluster of the tracking target in the current frame based on any tracking technology, based on the detection result of the tracking target in the past frame, and performs a search for the tracking target within the search range. In this case, when the point cloud information processing block 73 detects a variation in the variation detection processing, the search range (variation search range) is expanded more than the normal search range. Thereby, the point cloud information processing block 73 can appropriately execute tracking even when a variation occurs in the lidar 100 due to the vehicle passing over a bump or a step.

[0057] <Second Embodiment> FIG. 4 shows a schematic configuration of the lidar 100A according to the second embodiment. The lidar 100A is different from the lidar 100 according to the first embodiment in that the lidar 100A performs variation detection processing based on information output by a sensor 9 which is a sensor provided in the vehicle on which the lidar 100A is mounted or the lidar 100A. Hereinafter, for the components of the lidar 100A that are the same as those of the lidar 100 according to the first embodiment, the same reference numerals will be appropriately assigned and the description thereof will be omitted.

[0058] The sensor 9 is a sensor provided in the vehicle on which the lidar 100A is mounted or the lidar 100A, and detects variations in the vehicle or the lidar 100A. The sensor 9 may be an acceleration sensor or any other arbitrary sensor used for vibration detection. The sensor 9 supplies a detection signal Sg5 to the point cloud information processing block 73.

[0059] Based on the detection signal Sg5, the point cloud information processing block 73 executes a variation detection process, which is a process of detecting variations in the pitch direction of the lidar 100A at a predetermined degree or more. For example, when the degree of vibration indicated by the detection signal Sg5 is equal to or greater than a predetermined degree, the point cloud information processing block 73 detects that the lidar 100A has varied in the pitch direction. Then, when the point cloud information processing block 73 detects a variation in the variation detection process, it performs noise removal processing using the variation search range, and when it does not detect a variation in the variation detection process, it performs noise removal processing using the normal search range.

[0060] FIG. 5 is an example of a flowchart showing the procedure of point cloud information processing according to the second embodiment. The lidar 100A repeatedly executes point cloud information processing at each cycle of generating point cloud information for one frame.

[0061] First, the point cloud information generation block 72 generates point cloud information based on the detection signal Sg2 (step S11). Next, the point cloud information processing block 73 executes a variation detection process based on the detection signal Sg5 output by the sensor 9 (step S12). As a result, the point cloud information processing block 73 accurately detects the variation when the lidar 100 varies in the pitch direction due to the vehicle passing over bumps, steps, etc.

[0062] When the variation in the pitch direction is detected by the variation detection process (step S13; Yes), the point cloud information processing block 73 executes a pitch variation corresponding noise removal process, which is a noise data removal process based on the past frame and the current frame assuming the variation in the pitch direction (step S14). In this case, the point cloud information processing block 73 sets a variation search range for each valid point in the past frame and searches for the measured points in the current frame whose measured distance is within a predetermined distance. Then, the point cloud information processing block 73 sets the measured points in the current frame whose measured distance is approximated to a predetermined number or more of valid points in the past frame as valid points, and sets the others as invalid points. In addition, the point cloud information processing block 73 executes a process of setting the measured points whose intensity of the reflected light received by the receiving unit 2 is less than a predetermined threshold value as invalid points, a process of clustering the measured points, and setting the measured points belonging to a cluster with less than a predetermined number of elements as invalid points, etc.

[0063] On the other hand, when the variation in the pitch direction is not detected by the variation detection process (step S13; No), the point cloud information processing block 73 executes a normal noise removal process (step S15). In this case, the point cloud information processing block 73 executes the same noise removal process as step S02 of the first embodiment.

[0064] <Third Embodiment> FIG. 6 is a configuration diagram of a lidar system according to the third embodiment. In the third embodiment, a device different from the lidar 100X has the functions corresponding to the point cloud information processing block 73 and the point cloud information processing block 73 of the control unit 7. Hereinafter, for the elements of the third embodiment that are the same as those of the first embodiment or the second embodiment, the same reference numerals will be appropriately given and the description thereof will be omitted.

[0065] The lidar system according to the third embodiment includes a lidar 100X and an information processing device 200. In this case, the lidar 100X supplies the point cloud information generated by the point cloud information generation block 72 to the information processing device 200.

[0066] The information processing apparatus 200 includes a control unit 7A and a memory 8. The memory 8 stores information necessary for the control unit 7A to execute processing. Functionally, the control unit 7A includes a point cloud information acquisition block 72A and a point cloud information processing block 73. The point cloud information acquisition block 72A receives the point cloud information generated by the point cloud information generation block 72 of the lidar 100X and supplies the received point cloud information to the point cloud information processing block 73. The point cloud information processing block 73 executes the same processing as the point cloud information processing block 73 of the above-described embodiment on the point cloud information supplied from the point cloud information acquisition block 72A.

[0067] Note that the information processing apparatus 200 may be realized by a driving assistance apparatus. Also, the information on the parameters necessary for the processing may be stored by another apparatus having a memory that can be referenced by the information processing apparatus 200. Even with the configuration of this modification example, the information processing apparatus 200 can accurately execute noise removal of the point cloud information generated by the lidar 100X.

[0068] As described above, the control unit 7 of the lidar 100 according to the first embodiment or the second embodiment or the control unit 7A of the information processing apparatus 200 according to the third embodiment functions as the information processing apparatus in the present invention and functionally includes an acquisition unit, a detection unit, and a processing unit. The acquisition unit acquires point cloud information that is a set of data representing points measured by the measuring device for each measurement direction. The detection unit detects fluctuations of a predetermined degree or more in the measuring device. The processing unit determines a search range for searching for points of the point cloud information obtained at the previous processing time corresponding to each of the points of the point cloud information obtained at the current processing time based on the presence or absence of the detection of fluctuations. Thereby, even when fluctuations occur in the measuring device, it becomes possible to recognize the correspondence of each point included in the point cloud information at the current processing time and the previous processing time and appropriately execute noise removal processing, tracking processing, and the like.

[0069] In the above-described embodiments, the program can be stored using various types of non-transitory computer readable media and supplied to a controller or the like which is a computer. The non-transitory computer readable media include various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memories (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).

[0070] The present invention has been described with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. That is, the present invention naturally includes various modifications and corrections that those skilled in the art could make in accordance with the entire disclosure including the claims and the technical idea. Also, each disclosure of the above-cited patent documents and the like is incorporated herein by reference.

Explanation of Reference Numerals

[0071] 1 Transmitting unit 2 Receiving unit 3 Beam splitter 5 Scanner 6 Piezo sensor 7, 7A Control unit 8 Memory 100, 100X Lidar 200 Information processing apparatus

Claims

1. an acquisition means for acquiring point cloud information that is a set of data representing points measured by the measuring device for each measurement direction; a detection means for detecting a variation of the measuring device equal to or greater than a predetermined degree; a determination means for determining a search range for searching for points of second point cloud information, which is the point cloud information obtained at a pre-processing time, corresponding to each point of the first point cloud information obtained at the current processing time, based on the presence or absence of the detection of the variation; a search means for searching for points corresponding to each point of the first point cloud information from the points of the second point cloud information existing within the search range based on the measurement distance by the measuring device of each point of the first point cloud information and the measurement distance by the measuring device of the points of the second point cloud information existing within the search range determined for each point of the first point cloud information; an information processing apparatus having the above.

2. The information processing apparatus according to claim 1, wherein when the variation is detected, the determination means expands the search range in the direction in which the variation has occurred.

3. The information processing apparatus according to claim 1 or 2, wherein the detection means detects the variation in at least any one of the pitch direction, yaw direction, or roll direction of the measuring device.

4. The information processing apparatus according to claim 3, wherein the determination means detects the direction in which the variation has occurred among the positive direction or negative direction of the pitch direction, yaw direction, or roll direction, and expands the search range in the direction opposite to the detected direction.

5. The information processing apparatus according to any one of claims 1 to 4, wherein the detection means detects the variation based on the first point cloud information and the second point cloud information.

6. The detection means detects the variation based on a collation result between the first point cloud information and the second point cloud information based on ground points that are the points representing the ground. The information processing apparatus according to claim 5.

7. The information processing apparatus according to claim 5, wherein the detection means detects the variation based on a collation result between the first point cloud information and the second point cloud information based on points representing a measurement distance equal to or greater than a predetermined distance.

8. The information processing apparatus according to any one of claims 1 to 7, wherein the detection means detects the variation based on a detection signal output from a sensor provided in the measuring device or a moving body on which the measuring device is provided.

9. A control method executed by an information processing apparatus, comprising: acquiring point cloud information that is a set of data representing points measured by the measuring device for each measurement direction; Detect fluctuations in the measurement device equal to or greater than a predetermined degree, Based on the presence or absence of detection of the fluctuations, determine a search range for searching for points of second point cloud information, which is the point cloud information obtained at a pre-processing time, corresponding to each point of first point cloud information, which is the point cloud information obtained at the current processing time, Based on the measurement distance by the measurement device for each point of the first point cloud information and the measurement distance by the measurement device for the points of the second point cloud information existing within the determined search range for each point of the first point cloud information, search for points corresponding to each point of the first point cloud information from the points of the second point cloud information existing within the search range, Control method.

10. Obtain point cloud information, which is a set of data representing points measured by a measurement device for each measurement direction, Detect fluctuations in the measurement device equal to or greater than a predetermined degree, Based on the presence or absence of detection of the fluctuations, determine a search range for searching for points of second point cloud information, which is the point cloud information obtained at a pre-processing time, corresponding to each point of first point cloud information, which is the point cloud information obtained at the current processing time, A program that causes a computer to execute a process of searching for points corresponding to each point of the first point cloud information from the points of the second point cloud information existing within the search range, based on the measurement distance by the measurement device for each point of the first point cloud information and the measurement distance by the measurement device for the points of the second point cloud information existing within the determined search range for each point of the first point cloud information.

11. A storage medium storing the program according to Claim 10.

Images