LIDAR noise removal device and LIDAR noise removal method

Abstract

The present invention provides a LIDAR noise removal device and method. The device includes a LIDAR detection information processor that processes LIDAR detection information received from a vehicle's LIDAR. A sun position acquirer acquires the azimuth and elevation of the sun relative to the vehicle's direction of travel. A ROI selector selects a ROI corresponding to the sun from an image in front of the vehicle based on the azimuth and elevation, and compares the brightness of the selected ROI with a threshold. When the brightness of the ROI exceeds the threshold, a noise region selector selects a noise region corresponding to the ROI from the LIDAR detection information based on the azimuth and elevation, and a noise remover removes noise points from the selected noise region.

Description

Technical Field

[0001] The present invention relates to a light detection and ranging (LIDAR) noise removal device, and more particularly, to a LIDAR noise removal device and a LIDAR noise removal method capable of removing LIDAR noise caused by sunlight. Background Art

[0002] Generally, light detection and ranging (LIDAR) is a sensor configured to detect surrounding objects by emitting light and receiving light reflected from the object, and can use light with a wavelength of about 905nm or about 1550nm. However, when sunlight, which includes light components of all wavelengths, is irradiated on the LIDAR, noise may be generated at the target point of the detection signal in the LIDAR. Such noise may be generated irregularly in the direction of the sun and may adversely affect the detection accuracy of the object when the object exists in the direction of the sun.

[0003] When noise is combined with an object, it can adversely affect the object's size, position, and heading information, thereby reducing LIDAR recognition performance. Therefore, it is necessary to develop a LIDAR noise removal device that can estimate the direction and position components relative to the sun to remove sunlight noise and effectively extract objects. Summary of the Invention

[0004] Therefore, the present invention is directed to a LIDAR noise removal device and a LIDAR noise removal method thereof, which substantially eliminate one or more problems caused by limitations and disadvantages of the prior art. An object of the present invention is to provide a LIDAR noise removal device and a LIDAR noise removal method thereof, which can predict the direction and position relative to the sun based on global positioning system (GPS) information and image information to effectively remove noise points caused by sunlight without losing object information.

[0005] The objects of the present invention designed to solve the problems are not limited to the above objects, and other unmentioned objects will be clearly understood by those skilled in the art based on the following detailed description of the present invention.

[0006] To achieve these objectives and other advantages, and in accordance with the purposes of the present invention as embodied and broadly described herein, a LIDAR noise removal apparatus may include: a LIDAR detection information processor configured to process LIDAR detection information received from a LIDAR of a vehicle; a sun position acquirer configured to acquire an azimuth and elevation angle of the sun relative to a traveling direction of the vehicle; a region of interest (ROI) selector configured to select an ROI corresponding to the sun from an image in front of the vehicle based on the acquired azimuth and elevation angles of the sun, and compare the brightness of the selected ROI with a threshold to determine whether the brightness of the ROI exceeds the threshold; a noise region selector configured to select a noise region corresponding to the ROI from the LIDAR detection information based on the azimuth and elevation angles of the sun when the brightness of the ROI exceeds the threshold; and a noise remover configured to remove noise points in the selected noise region.

[0007] In another aspect of the present invention, a LIDAR noise removal method of a LIDAR noise removal device that receives LIDAR detection information from a LIDAR of a vehicle may include: processing the LIDAR detection information received from the LIDAR of the vehicle; obtaining an azimuth and elevation of the sun relative to a traveling direction of the vehicle; selecting a region of interest (ROI) corresponding to the sun from an image in front of the vehicle based on the obtained azimuth and elevation of the sun; comparing the brightness of the selected ROI with a threshold to determine whether the brightness of the ROI exceeds the threshold; when the brightness of the ROI exceeds the threshold, selecting a noise region corresponding to the ROI from the LIDAR detection information based on the azimuth and elevation of the sun; and removing noise points in the selected noise region.

[0008] In another aspect of the present invention, a non-transitory computer-readable recording medium storing a program for executing a LIDAR noise removal method of a LIDAR noise removal apparatus is provided, the program being capable of executing processing included in the LIDAR noise removal method.

[0009] In another aspect of the present invention, a vehicle may include: a LIDAR configured to sense information of surrounding objects of the vehicle; a camera configured to acquire an image in front of the vehicle; and a LIDAR noise removal device configured to remove noise points corresponding to sunlight incident on the LIDAR, wherein the LIDAR noise removal device may include: a LIDAR detection information processor configured to process LIDAR detection information received from the LIDAR; a sun position acquirer configured to acquire an azimuth and elevation angle of the sun relative to a driving direction of the vehicle; a region of interest (ROI) selector configured to select an ROI corresponding to the sun from the image in front of the vehicle based on the acquired azimuth and elevation angle of the sun, and compare the brightness of the selected ROI with a threshold to determine whether the brightness of the ROI exceeds the threshold; a noise region selector configured to select a noise region corresponding to the ROI from the LIDAR detection information based on the azimuth and elevation angle of the sun when the brightness of the ROI exceeds the threshold; and a noise remover configured to remove noise points in the selected noise region.

[0010] It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this application; the drawings illustrate embodiments of the present invention and together with the description serve to explain the principles of the present invention. In the drawings:

[0012] Figure 1 is a schematic diagram illustrating a vehicle including a LIDAR noise removal apparatus according to an exemplary embodiment of the present invention;

[0013] Figure 2 is a schematic diagram illustrating noise points generated by sunlight according to an exemplary embodiment of the present invention;

[0014] Figure 3 is a schematic diagram illustrating an ROI image corresponding to the sun extracted from a front image of a vehicle according to an exemplary embodiment of the present invention;

[0015] Figure 4 is a schematic diagram illustrating LIDAR detection information on an XY coordinate plane according to an exemplary embodiment of the present invention, on which a vehicle point and a noise point caused by sunlight exist in the same direction;

[0016] Figure 5is a schematic diagram illustrating LIDAR detection information on a YZ coordinate plane according to an exemplary embodiment of the present invention, on which a vehicle point and a noise point caused by sunlight exist in the same direction;

[0017] Figure 6 is a schematic diagram illustrating layers of a LIDAR and an elevation angle of the sun according to an exemplary embodiment of the present invention;

[0018] Figure 7A and Figure 7B is a schematic diagram illustrating the FOV of a LIDAR and the azimuth angle of the sun according to an exemplary embodiment of the present invention;

[0019] Figure 8 is a block diagram illustrating a LIDAR noise removal apparatus according to an exemplary embodiment of the present invention; and

[0020] Figure 9 is a flowchart illustrating a LIDAR noise removal method of a LIDAR noise removal apparatus according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0021] It should be understood that the term "vehicle" or "vehicular" or other similar terms used in this document generally include motor vehicles, such as passenger vehicles including sport utility vehicles (SUVs), buses, trucks, various commercial vehicles, ships including various boats, vessels, aircraft, etc., and include hybrid vehicles, electric vehicles, internal combustion engine vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles and other alternative fuel vehicles (e.g., fuels derived from non-fossil energy).

[0022] Although the exemplary embodiments are described as using multiple units to perform the exemplary processes, it is understood that the exemplary processes can also be performed by one or more modules. In addition, it is understood that the term "controller / control unit" refers to a hardware device that includes a memory and a processor. The memory is configured to store the modules, and the processor is specifically configured to execute the modules to perform one or more processes described further below.

[0023] Furthermore, the control logic of the present invention can be implemented as a non-volatile computer-readable medium, which is a computer-readable medium containing executable program instructions executed by a processor, controller / control unit, etc. Examples of computer-readable media include, but are not limited to, ROM, RAM, compact disc (CD)-ROMs, magnetic tapes, floppy disks, flash drives, smart cards, and optical data storage devices. The computer-readable recording medium can also be distributed among network-connected computer systems so that the computer-readable media is stored and executed in a distributed manner, such as by a telematics server or a controller area network (CAN).

[0024] The terms used in this article are only used to describe specific embodiments and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates that the plural forms are not included. It will also be further understood that when the terms "include" and / or "comprising" are used in this specification, it is indicated that the features, values, steps, operations, elements and / or parts are present, but the presence or addition of one or more other features, values, steps, operations, elements, parts and / or groups thereof are not excluded. As used herein, the term "and / or" includes any and all combinations of one or more related enumeration items.

[0025] Unless otherwise stated or apparent from the context, as used herein, the term "about" is understood to mean within the normal tolerance range in the art, for example, within 2 standard deviations of the mean. "About" can be understood to mean within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, 0.1%, 0.05%, or 0.01% of the specified value. Unless the context indicates otherwise, the term "approximately" modifies all numerical values ​​provided herein.

[0026] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are shown in the accompanying drawings. The following exemplary embodiments are given by way of example so that those skilled in the art can fully understand the concept of the present invention. Therefore, the present invention is not limited to the following exemplary embodiments and can be implemented in various other forms. In order to clearly describe the present invention, parts that are not relevant to the description of the present invention have been omitted from the accompanying drawings. Throughout this specification, the same reference numerals will be used to represent the same or similar parts as much as possible.

[0027] In the following, reference will be made to Figures 1 to 9 A LIDAR noise removal apparatus and a LIDAR noise removal method thereof applicable to exemplary embodiments of the present invention are described in detail.

[0028] Figure 1: is a schematic diagram showing a vehicle including a LIDAR noise removal apparatus according to an exemplary embodiment of the present invention. Figure 1 As shown, the vehicle 1 may include: a LIDAR 210, a camera 220 and a LIDAR noise removal device 100; the LIDAR 210 is configured to sense information of objects around the vehicle 1; the camera 220 is configured to obtain an image in front of the vehicle 1; and the LIDAR noise removal device 100 is configured to remove noise points corresponding to sunlight incident on the LIDAR 210.

[0029] Specifically, the light detection and ranging (LIDAR) can be a multi-channel LIDAR having a front LIDAR and a front side LIDAR of the vehicle 1, but is not limited thereto. For example, the multi-channel LIDAR can include multiple layers, each layer having a specified angle. The LIDAR can be configured to sense objects around the vehicle 1 and generate LIDAR detection information of a LIDAR sensor coordinate library, which can be coordinate-converted into a coordinate library of a front image acquired from the camera 220.

[0030] The LIDAR noise removal device 100 can be configured to select a region of interest (ROI) corresponding to the sun from the front image of the vehicle 1 based on the azimuth and elevation of the sun, compare the brightness of the selected ROI with a threshold, and when the brightness of the ROI exceeds the threshold, select a noise area corresponding to the ROI from the LIDAR detection information, and remove noise points in the selected noise area.

[0031] On the other hand, when the brightness of the ROI is less than or equal to the threshold, the LIDAR noise removal apparatus 100 can be configured to assign an object point flag to all points in the noise region corresponding to the ROI, modify the marking parameters of the noise region, and distinguish between object points and noise points in the noise region, so that overlapping objects in the noise region are not removed and only the distinguished noise points are removed. Specifically, the LIDAR noise removal apparatus 100 can be configured to distinguish between object points and noise points in the noise region corresponding to the ROI by modifying the marking parameters to reduce the distance threshold for the distance between points in the noise region.

[0032] In other words, when the brightness of the ROI exceeds a threshold, the LIDAR noise removal apparatus 100 can be configured to recognize that sunlight is shining on the vehicle's LIDAR and remove noise points in the noise region corresponding to the ROI. In some cases, when the brightness of the ROI exceeds a threshold, the LIDAR noise removal apparatus 100 can be configured to recognize that sunlight is shining on the vehicle's LIDAR and that there are no objects in the noise region corresponding to the ROI.

[0033] Furthermore, when the brightness of the ROI is less than or equal to a threshold, the LIDAR noise removal device 100 can be configured to assign an object point flag to all points in the noise region corresponding to the ROI to prevent objects present in the noise region from being identified as noise and removed as noise, and to modify the flag parameters of the noise region. Alternatively, the LIDAR noise removal device 100 can be configured to receive LIDAR detection information from the LIDAR 210 of the vehicle 1 and parse the received LIDAR detection information for processing.

[0034] Then, the LIDAR noise removal device 100 can be configured to obtain the position information, heading information, and time information of the vehicle 1, and extract the azimuth and elevation angle of the sun relative to the travel direction of the vehicle 1 based on the obtained position information, heading information, and time information of the vehicle 1. Specifically, the LIDAR noise removal device 100 can be configured to obtain the position information, heading information, and time information of the vehicle 1 from a global positioning system (GPS), but is not limited thereto.

[0035] Furthermore, the LIDAR noise removal apparatus 100 may be configured to acquire an image in front of the vehicle 1, convert the azimuth and elevation of the sun into LIDAR coordinates and coordinates of the image in front of the vehicle, and select a ROI corresponding to the sun from the image in front of the vehicle. Specifically, the LIDAR noise removal apparatus 100 may be configured to acquire an image in front of the vehicle 1 from the camera 220 (or other imaging device) of the vehicle 1, but is not limited thereto. When selecting the ROI corresponding to the sun from the image in front of the vehicle 1, the LIDAR noise removal apparatus 100 may be configured to calculate the brightness of the selected ROI and determine whether the calculated brightness of the ROI exceeds a threshold.

[0036] For example, the LIDAR noise removal device 100 may be configured to convert the image of the selected ROI into a grayscale image and calculate the brightness of the ROI using the binary value of the converted image, but the present invention is not limited thereto. Furthermore, the LIDAR noise removal device 100 may be configured to change the threshold value according to the altitude of the sun. For example, the LIDAR noise removal device 100 may be configured to increase the threshold value when the altitude of the sun is approximately 10° or less relative to the horizon.

[0037] This is due to the fact that when the sun's altitude is approximately 10° or less relative to the horizon, even if the sun is behind the object, an object existing in front of the sun may still be misidentified as noise caused by the sun and removed as such when the brightness of the ROI corresponding to the sun in the front image is greater than a threshold. Therefore, when the sun's altitude is approximately 10° or less relative to the horizon, the LIDAR noise removal apparatus 100 can be configured to set the threshold to a higher value (e.g., an increased value), thereby preventing the object from being misidentified as noise caused by the sun and removed as such even when the brightness of the ROI of the object between the sun and the vehicle is greater than the threshold.

[0038] When the brightness of the ROI exceeds a threshold, the LIDAR noise removal device 100 can be configured to recognize that sunlight is irradiating the vehicle, extract the LIDAR 210 layer corresponding to the sun's altitude, the angle corresponding to the sun's azimuth, and the filter value, and select the noise region corresponding to the ROI from the LIDAR detection information based on the extracted layer, angle, and filter value. The filter value can be calculated and updated in real time based on changes in time and the vehicle's heading direction.

[0039] In addition, when selecting a noise region corresponding to an ROI, the LIDAR noise removal apparatus 100 can be configured to assign a noise point flag to points in the noise region, identify the points assigned the noise point flag as noise points, and remove the identified noise points. On the other hand, when the brightness of the ROI is less than or equal to a threshold, the LIDAR noise removal apparatus 100 can be configured to assign an object point flag to all points in the noise region corresponding to the ROI and modify the labeling parameters of the noise region.

[0040] Specifically, the LIDAR noise removal device 100 can be configured to modify the marker parameters to reduce the distance threshold for the distance between points in the noise region corresponding to the ROI. Furthermore, the LIDAR noise removal device 100 can be configured to check the number of object points of the marker object in the noise region corresponding to the ROI and perform filtering to identify the object based on the number of object points of the marker object. For example, when the number of object points of the marker object is two or fewer, the LIDAR noise removal device 100 can be configured to determine the object point as a noise point, thereby removing the object.

[0041] As described above, according to the present invention, the LIDAR noise removal device can be configured to predict the direction and position relative to the sun based on global positioning system (GPS) information and image information, thereby effectively removing noise points caused by sunlight without losing object information. Furthermore, according to the present invention, the LIDAR noise removal device can be configured to effectively remove noise caused by sunlight, thereby preventing the noise from affecting the size of objects or being erroneously detected as objects during the processing of LIDAR signals.

[0042] In other words, according to the present invention, the LIDAR noise removal device can be configured to identify and track the vehicle's direction of travel and the direction and angle of the sun to remove noise caused by sunlight, thereby ensuring optimal object recognition logic performance. Furthermore, according to the present invention, the LIDAR noise removal device can be configured to remove irregular solar noise to more accurately identify objects.

[0043] Figure 2 is a schematic diagram showing noise points generated by sunlight. Figure 2 As shown, the LIDAR noise removal device of the present invention can be configured to receive LIDAR detection information from the LIDAR of the vehicle 1 and parse the received LIDAR detection information to process it.

[0044] Specifically, the LIDAR detection information may include noise points 10 that are irregularly generated in the direction of the sun 3. When an object 2, such as a surrounding vehicle, is present in the direction of the sun 3, the noise points 10 in the LIDAR detection information may adversely affect the detection accuracy of the object point 20. Therefore, the LIDAR noise removal apparatus of the present invention may be configured to estimate the direction and position components relative to the sun 3 to remove the noise points 10 caused by sunlight and effectively extract the object 2.

[0045] Figure 3 : is a schematic diagram showing an ROI image corresponding to the sun extracted from the front image of the vehicle. Figure 3 As shown, the LIDAR noise removal device of the present invention can obtain a front image 30 of a vehicle, convert the azimuth and elevation angles of the sun into coordinates of the front image 30 , and select a ROI 40 corresponding to the sun from the front image 30 .

[0046] Specifically, when selecting ROI 40 corresponding to the sun from the vehicle's front image 30, the LIDAR noise removal device can be configured to calculate the brightness of the selected ROI 40 and compare the calculated brightness of the ROI 40 with a threshold value to determine whether the object 2 is present between the sun and the vehicle. For example, the LIDAR noise removal device can be configured to convert the image of the selected ROI into a grayscale image and calculate the brightness of the ROI using the binary value of the converted image, but is not limited thereto.

[0047] The LIDAR noise removal device can be configured to determine that there is no object between the sun and the vehicle when the brightness of ROI 40 is greater than a threshold, and can be configured to determine that Object 2 is present between the sun and the vehicle when the brightness of ROI 40 is less than or equal to the threshold. When the sun's altitude is too low, approximately 10 degrees or less from the horizon, Object 2 may still be present between the sun and the vehicle even if the brightness of ROI 40 in the front image is greater than the threshold. In this case, the LIDAR noise removal device can be configured to increase the threshold above the reference threshold.

[0048] This is due to the fact that when the sun's altitude is approximately 10° or less relative to the horizon, even if the sun is behind object 2, object 2 existing in front of the sun may still be misidentified as noise caused by the sun and removed as such when the brightness of the ROI corresponding to the sun in the front image is greater than the threshold. Therefore, when the sun's altitude is approximately 10° or less relative to the horizon, the LIDAR noise removal device can be configured to set the threshold to a higher value (e.g., an increased value), thereby preventing object 2 from being misidentified as noise caused by the sun and removed as such even when the brightness of the ROI of object 2 is greater than the threshold between the sun and the vehicle.

[0049] The LIDAR noise removal device of the present invention can be configured to use GPS location information and the vehicle's heading information to calculate the position of the sun relative to the vehicle's direction of travel, select the range of the noise candidate group based on the calculated direction (including horizontal and vertical directions), and then use the vehicle's front image to determine whether sunlight is irradiating the vehicle. Specifically, when the sun appears in the vehicle's front image, the LIDAR noise removal device can be configured to select a ROI with an azimuth and altitude relative to the sun and compare the brightness of the selected ROI with a threshold.

[0050] The LIDAR noise removal device can then be configured to examine the relationship between points in the LIDAR detection information based on the comparison result between the ROI brightness and the threshold value to more accurately remove noise caused by sunlight. When the sun's altitude is very similar to the horizon and is located on the same line as object 2, that is, when the sun's altitude is approximately 10° or less relative to the horizon, the LIDAR noise removal device can be configured to set the ROI brightness threshold to a higher value.

[0051] This is based on the fact that the brightness of the ROI in the front image may be high even when the sun is behind the object 2. Normally, due to the characteristics of light, when the sun appears behind the object, the light does not reach the LIDAR, and the LIDAR can only measure the object point information.

[0052] Figure 4 is a schematic diagram showing LIDAR detection information on an XY coordinate plane, on which a vehicle point and a noise point caused by sunlight exist in the same direction. Figure 5 is a schematic diagram showing LIDAR detection information on the YZ coordinate plane, on which a vehicle point and a noise point caused by sunlight exist in the same direction.

[0053] like Figure 4 and Figure 5 As shown, the LIDAR noise removal apparatus of the present invention can be configured to extract a LIDAR layer corresponding to the altitude of the sun, an angle corresponding to the azimuth of the sun, and a filter value, and select a noise region corresponding to the ROI from the LIDAR detection information 80 based on the extracted layer, angle, and filter value.

[0054] Specifically, LIDAR detection information 80 shows noise points caused by sunlight along the direction of the sun. Furthermore, when an object is between the sun and the vehicle, a point overlap region 82 exists, comprising the object points of the object and noise points caused by sunlight adjacent to and surrounding the object. Therefore, the LIDAR noise removal device of the present invention can be configured to compare the brightness of the ROI with a threshold. When the brightness of the ROI is less than or equal to the threshold, it identifies the presence of an object between the sun and the vehicle and modifies the marker parameters of the noise region corresponding to the ROI.

[0055] In other words, when the brightness of the ROI is less than or equal to a threshold, the LIDAR noise removal device can be configured to detect the presence of an object between the sun and the vehicle, assign an object point flag to all points in the noise region corresponding to the ROI, and modify a marking parameter of the noise region. Specifically, the LIDAR noise removal device can be configured to modify the marking parameter to reduce the distance threshold for the distance between points in the noise region corresponding to the ROI.

[0056] For example, the LIDAR noise removal device can be configured to calculate the distance between points in the noise region corresponding to the ROI and compare the calculated distance with a distance threshold to determine whether the points are connected to the object or far away from the object. In addition, when the distance threshold of the distance between points in the noise region corresponding to the ROI decreases, the irregular noise caused by sunlight can be marked as a different object due to the reduced threshold distance.

[0057] Specifically, when there are approximately two or fewer object points in a marker object, the LIDAR noise removal device can be configured to determine that the object is noise and thus delete it. That is, the LIDAR noise removal device can be configured to check the number of object points of the marker object in the noise region corresponding to the ROI, and perform filtering based on the number of object points of the marker object to identify the object.

[0058] Therefore, when there is an object between the sun and the vehicle, the LIDAR noise removal device of the present invention can be configured to distinguish between the object point and the noise point and remove the distinguished noise point by marking in the point overlap area 82 including the object point of the object and the noise point caused by the sunlight distributed adjacent to and around the object.

[0059] Figure 6 is a schematic diagram showing the layers of LIDAR and the elevation angle of the sun, Figure 7A and 7B : is a schematic diagram showing the FOV of LIDAR and the azimuth angle of the sun. Figure 6 As shown, the LIDAR 210 can be configured to sense information of objects around the vehicle and send the LIDAR detection information to the LIDAR noise removal device.

[0060] Specifically, the LIDAR 210 may include: Figure 7A The front side LIDAR 212 and Figure 7B The multi-channel LIDAR of the front LIDAR 214 in FIG. 2 may be a multi-channel LIDAR, but is not limited thereto. For example, the multi-channel LIDAR may include multiple layers 74, each layer having a specified angle. Figure 6, the present invention can predict the elevation angle 72 of the sun based on the layer 74 of the LIDAR 210. In addition, as Figure 7A and Figure 7B As shown, the present invention can predict the azimuth angle 76 of the sun from the field of view (FOV) 78 of each of the LIDAR 212 and the LIDAR 214. The LIDAR 210 can be configured to sense objects around the vehicle and generate LIDAR detection information in a LIDAR sensor coordinate library, which can be coordinate-converted to a coordinate library of a front image of the vehicle.

[0061] The present invention can use the elevation and azimuth angles of the sun corresponding to the vehicle's heading information to predict the expected range of the noise signal, select an ROI based on the predicted range, and remove noise for the LIDAR layer and azimuth corresponding to the sun where noise due to sunlight is expected. The present invention can utilize the following methods: excluding points in the noise region due to sunlight, reducing the distance threshold between points during clustering of the noise region due to sunlight, and excluding sparse points from candidate targets without marking them.

[0062] Furthermore, the present invention can utilize a grid or voxel map during the clustering process. During the clustering process, the present invention can select a representative point in a voxel, calculate the distance between the representative point and the representative point in an adjacent voxel, and compare the calculated distance with a distance threshold to determine whether the corresponding object is a connected object or a distant object. As the distance threshold decreases, irregular solar noise can be marked as a different object. Specifically, when there are two or fewer object points in a marked object, the present invention can determine that the object is noise and therefore delete it.

[0063] Figure 8 : is a block diagram showing a LIDAR noise removal apparatus according to an exemplary embodiment of the present invention. Figure 8 As shown, the LIDAR noise removal device 100 of the present invention may include: a LIDAR detection information processor 110, a sun position acquirer 120, a ROI selector 130, a noise region selector 140, a noise remover 150, and an object identifier 160. Each component may be operated by a process of an overall controller.

[0064] Specifically, the LIDAR detection information processor 110 may be configured to process LIDAR detection information received from the vehicle's LIDAR. In other words, the LIDAR detection information processor 110 may be configured to receive LIDAR detection information from the vehicle's LIDAR and parse the received LIDAR detection information for processing. The sun position acquirer 120 may be configured to acquire the azimuth and elevation angles of the sun relative to the vehicle's travel direction.

[0065] The solar position acquirer 120 may be configured to acquire the vehicle's position information, heading information, and time information (using various sensors), and extract the azimuth and elevation angles of the sun relative to the vehicle's travel direction based on the acquired vehicle's position information, heading information, and time information. For example, the solar position acquirer 120 may be configured to acquire the vehicle's position information, heading information, and time information from a global positioning system (GPS).

[0066] The ROI selector 130 may be configured to select a region of interest (ROI) corresponding to the sun from the front image of the vehicle based on the acquired azimuth and elevation of the sun, and compare the brightness of the selected ROI with a threshold to determine whether the brightness of the ROI exceeds the threshold. Specifically, the ROI selector 130 may be configured to acquire the front image of the vehicle, convert the azimuth and elevation of the sun into coordinates of the LIDAR and the coordinates of the front image, and select the ROI corresponding to the sun from the front image.

[0067] For example, the ROI selector 130 may be configured to acquire an image of the vehicle's front from a forward-looking camera. Furthermore, the ROI selector 130 may be configured to calculate the brightness of the selected ROI and determine whether the calculated brightness of the ROI exceeds a threshold. Specifically, the ROI selector 130 may be configured to convert the image of the selected ROI into a grayscale image and calculate the brightness of the ROI using the binary value of the converted image.

[0068] In some cases, ROI selector 130 can be configured to change the threshold based on changes in the sun's altitude. For example, ROI selector 130 can be configured to increase the threshold when the sun's altitude is approximately 10° or less relative to the horizon. This is due to the fact that when the sun's altitude is approximately 10° or less relative to the horizon, even if the sun is behind the object, objects in front of the sun may still be misidentified as noise caused by the sun and removed as such if the brightness of the ROI corresponding to the sun in the front image is greater than the threshold.

[0069] Therefore, when the sun's altitude relative to the horizon is approximately 10° or less, the ROI selector 130 can be configured to set the threshold to a higher value, thereby preventing the object from being misidentified as noise caused by the sun and removed as such even when the brightness of the ROI in which the object exists between the sun and the vehicle is greater than the threshold. The noise region selector 140 can be configured to select a noise region corresponding to the ROI from the LIDAR detection information based on the azimuth and elevation angles of the sun when the brightness of the ROI exceeds the threshold.

[0070] When the brightness of the ROI exceeds a threshold, the noise region selector 140 may be configured to recognize that sunlight is shining on the vehicle. Furthermore, when the brightness of the ROI exceeds a threshold, the noise region selector 140 may be configured to extract a LIDAR layer corresponding to the altitude of the sun, an angle corresponding to the azimuth of the sun, and a filter value, and select a noise region corresponding to the ROI from the LIDAR detection information based on the extracted layer, angle, and filter value.

[0071] For example, the filter value may be calculated and updated in real time based on changes in time and the vehicle's heading. Furthermore, in response to determining that the sun and the vehicle maintain a line of sight (LOS) and that there is no object information between the two, the noise region selector 140 may be configured to extract a LIDAR layer corresponding to the sun's altitude and an angle corresponding to the sun's azimuth.

[0072] Furthermore, the noise remover 150 may be configured to remove noise points in a selected noise region. Specifically, when a noise region corresponding to an ROI is selected, the noise remover 150 may be configured to assign a noise point flag to each point in the noise region. In other words, the noise remover 150 may be configured to identify the points to which the noise point flag is assigned as noise points and remove the identified noise points.

[0073] Object identifier 160 can be configured to identify objects from LIDAR detection information. When the brightness of an ROI is less than or equal to a threshold, object identifier 160 can be configured to assign an object point designation to all points in the noise region corresponding to the ROI and modify the marking parameters for the noise region. For example, object identifier 160 can be configured to modify the marking parameters to reduce the distance threshold for distances between points in the noise region corresponding to the ROI.

[0074] Furthermore, the object identifier 160 may be configured to check the number of object points of the marker object in the noise region corresponding to the ROI, and perform filtering based on the number of object points of the marker object to identify the object. For example, when the number of object points of the marker object is approximately two or less, the object identifier 160 may be configured to determine the object point as a noise point, thereby removing the object.

[0075] Figure 9 1 is a flow chart illustrating a LIDAR noise removal method of a LIDAR noise removal device according to an exemplary embodiment of the present invention. The method described below can be executed by processing of a controller. Figure 9 As shown, the present invention can receive LIDAR detection information from the LIDAR of the vehicle (S10). In response to receiving the LIDAR detection information, the present invention can parse the received LIDAR detection information to process it (S20).

[0076] Then, the present invention can obtain the vehicle's position information, forward direction information, and time information, and extract the azimuth and elevation angle of the sun relative to the vehicle's travel direction based on the obtained vehicle's position information, forward direction information, and time information (S30). The vehicle's position information, forward direction information, and time information can be obtained from a global positioning system (GPS).

[0077] Furthermore, the present invention can acquire an image in front of the vehicle (S40). Specifically, the image in front of the vehicle can be acquired from a camera or other imaging device of the vehicle. The present invention can then convert the azimuth and elevation angles of the sun into coordinates of the LIDAR and the coordinates of the front image, and select a region of interest (ROI) corresponding to the sun from the front image based on the converted azimuth and elevation angles of the sun (S50).

[0078] The present invention may then calculate the brightness of the selected ROI (S60). Specifically, the present invention may convert the image of the selected ROI into a grayscale image and use the binary value of the converted image to calculate the brightness of the ROI. The present invention may determine whether the calculated brightness of the ROI exceeds a threshold (S70). Specifically, the threshold may vary based on the altitude of the sun. For example, when the altitude of the sun is approximately 10° or less from the horizon, the threshold may be increased to above a reference threshold.

[0079] When the brightness of the ROI exceeds a threshold, the present invention can select a noise region corresponding to the ROI from the LIDAR detection information based on the azimuth and elevation of the sun (S80). When the brightness of the ROI exceeds a threshold, the present invention can identify that sunlight is irradiating the vehicle.

[0080] Furthermore, when the brightness of an ROI exceeds a threshold, the present invention can extract the LIDAR layer corresponding to the sun's altitude, the angle corresponding to the sun's azimuth, and the filter value. Based on the extracted layer, angle, and filter value, the noise region corresponding to the ROI can be selected from the LIDAR detection information. For example, the filter value can be calculated and updated in real time based on changes in time and the vehicle's heading.

[0081] In addition, the present invention can remove noise points in a selected noise region (S90). When a noise region corresponding to an ROI is selected, the present invention can assign a noise point flag to points in the noise region, identify the points assigned the noise point flag as noise points, and remove the identified noise points. On the other hand, when the brightness of the ROI is less than or equal to a threshold, the present invention can assign an object point flag to all points in the noise region corresponding to the ROI and modify the marking parameters of the noise region (S110). When modifying the marking parameters for the noise region corresponding to the ROI, the present invention can modify the marking parameters to reduce a distance threshold for the distance between points in the noise region.

[0082] The present invention can then identify the object based on the LIDAR detection information (S100). Specifically, the present invention can check the number of object points of the marker object in the noise region corresponding to the ROI, and perform filtering based on the number of object points of the marker object to identify the object. For example, when the number of object points of the marker object is two or less, the present invention can determine the object point as a noise point, thereby removing the object.

[0083] Furthermore, the present invention determines whether the current state is a state in which the LIDAR noise removal operation ends (S120). In response to determining that the current state is a state in which the LIDAR noise removal operation ends, the present invention may end the LIDAR noise removal operation. Alternatively, the present invention may provide a non-volatile computer-readable recording medium storing a program for executing a LIDAR noise removal method for a LIDAR noise removal apparatus, wherein the program may execute the processes included in the LIDAR noise removal method.

[0084] As described above, according to the present invention, the direction and position relative to the sun can be predicted based on Global Positioning System (GPS) information and image information, effectively removing noise points caused by sunlight without losing object information. Furthermore, according to the present invention, noise caused by sunlight can be effectively removed, preventing the noise from affecting the size of objects or preventing them from being mistakenly detected as objects during LIDAR signal processing.

[0085] In other words, according to the present invention, the vehicle's direction of travel and the direction and angle of the sun can be identified and tracked to remove noise caused by sunlight, thereby ensuring optimal object recognition logic performance. In addition, according to the present invention, irregular solar noise can be removed to more accurately identify objects.

[0086] The LIDAR noise removal method of the present invention can be implemented as computer-readable code on a program storage medium. Non-volatile computer-readable media can be any type of recording device that stores data in a computer-readable manner. Non-volatile computer-readable media can include, for example, hard disk drives (HDDs), solid-state drives (SSDs), silicon disk drives (SDDs), read-only memories (ROMs), random-access memories (RAMs), compact disk read-only memories (CD-ROMs), magnetic tapes, floppy disks, and optical data storage devices.

[0087] Those skilled in the art will appreciate that the effects obtainable by the present invention are not limited to those that have been specifically described above, and other effects of the present invention will be more clearly understood from the above detailed description.

[0088] The above detailed description should not be interpreted as limiting the present invention in any way, but should be considered by way of example. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all equivalent modifications made without departing from the scope of the present invention should be understood to be included in the appended claims.

Claims

1. A LIDAR noise removal device, comprising: a LIDAR detection information processor configured to process LIDAR detection information received from the vehicle's LIDAR; a sun position acquirer configured to acquire an azimuth and elevation angle of the sun relative to a travel direction of the vehicle; a region of interest selector configured to select a region of interest corresponding to the sun from a front image acquired by a camera of the vehicle based on the acquired azimuth and elevation angle of the sun, and compare a brightness of the selected region of interest with a threshold value to determine whether the brightness of the region of interest exceeds the threshold value; a noise region selector configured to select a noise region corresponding to the region of interest from the LIDAR detection information based on the azimuth and elevation angles of the sun when the brightness of the region of interest exceeds a threshold; a noise remover configured to remove noise points in a selected noise region; An object identifier is configured to assign object point labels to all points in a noise region corresponding to the region of interest when the brightness of the region of interest is less than or equal to a threshold, and to modify labeling parameters of the noise region corresponding to the region of interest to reduce the distance threshold for distances between points in the noise region corresponding to the region of interest.

2. The LIDAR noise removal device according to claim 1, wherein: The LIDAR detection information processor is configured to receive LIDAR detection information from the vehicle's LIDAR and parse the received LIDAR detection information to process it.

3. The LIDAR noise removal device according to claim 1, wherein: The sun position acquirer is configured to acquire the vehicle's position information, forward direction information, and time information, and extract the azimuth and elevation angles of the sun relative to the vehicle's travel direction based on the acquired vehicle's position information, forward direction information, and time information.

4. The LIDAR noise removal device according to claim 1, wherein: The region of interest selector is configured to: obtain a front image of the vehicle, convert the azimuth and elevation angles of the sun into coordinates of the LIDAR and the front image, and select a region of interest corresponding to the sun from the front image.

5. The LIDAR noise removal device according to claim 1, wherein: The noise region selector is configured to recognize that sunlight is irradiating the vehicle when the brightness of the region of interest exceeds a threshold.

6. The LIDAR noise removal device according to claim 1, wherein: The noise region selector is configured to extract, when the brightness of the region of interest exceeds a threshold, a LIDAR layer corresponding to the elevation angle of the sun, an angle corresponding to the azimuth of the sun, and a filter value, and select a noise region corresponding to the region of interest from the LIDAR detection information based on the extracted layer, angle, and filter value.

7. The LIDAR noise removal device according to claim 1, wherein: The object identifier is configured to identify an object from LIDAR detection information.

8. The LIDAR noise removal device according to claim 7, wherein: The object identifier is configured to distinguish between object points and noise points in the noise region based on the modified marking parameters when the brightness of the region of interest is less than or equal to a threshold value, and remove only the distinguished noise points.

9. The LIDAR noise removal device according to claim 7, wherein: The object identifier is configured to check the number of object points of the marker object in a noise region corresponding to the region of interest, and perform filtering based on the number of object points of the marker object to identify the object.

10. A LIDAR noise removal method for a LIDAR noise removal device, wherein the LIDAR noise removal device receives LIDAR detection information from a vehicle's LIDAR, the method comprising: processing, by a processor, LIDAR detection information received from the vehicle's LIDAR; The processor obtains the azimuth and elevation of the sun relative to the vehicle's travel direction; The processor selects a region of interest corresponding to the sun from a front image acquired by a camera of the vehicle based on the acquired azimuth and elevation of the sun; Comparing, by the processor, the brightness of the selected region of interest with a threshold value to determine whether the brightness of the region of interest exceeds the threshold value; When the brightness of the region of interest exceeds a threshold, the processor selects a noise region corresponding to the region of interest from the LIDAR detection information based on the azimuth and elevation angles of the sun; removing noise points in the selected noise area by a processor; When the brightness of the region of interest is less than or equal to a threshold, the processor assigns an object point flag to all points in the noise region corresponding to the region of interest, and modifies a marking parameter of the noise region corresponding to the region of interest to reduce the distance threshold for the distance between points in the noise region corresponding to the region of interest.

11. The LIDAR noise removal method according to claim 10, wherein: Processing LIDAR detection information includes: Receiving, by the processor, LIDAR detection information from the vehicle's LIDAR; The processor parses the received LIDAR detection information to process it.

12. The LIDAR noise removal method according to claim 10, wherein: Obtaining the sun's azimuth and elevation angles includes: The processor obtains the vehicle's location information, heading information, and time information; The processor extracts the azimuth and elevation of the sun relative to the vehicle's travel direction based on the acquired vehicle's position information, forward direction information, and time information.

13. The LIDAR noise removal method according to claim 10, wherein: Select areas of interest include: The processor acquires a front image of the vehicle; The processor converts the azimuth and elevation of the sun into the coordinates of the LIDAR and the coordinates of the front image; A processor selects a region of interest corresponding to the sun from the front image based on the converted azimuth and elevation of the sun.

14. The LIDAR noise removal method according to claim 10, wherein: Selecting the noise region corresponding to the region of interest includes recognizing that sunlight is irradiating the vehicle when brightness of the region of interest exceeds a threshold.

15. The LIDAR noise removal method according to claim 10, further comprising: When the brightness of the region of interest is less than or equal to a threshold, distinguishing, by the processor, between object points and noise points in the noise region based on the modified marking parameters; The processor identifies the object from the LIDAR detection information.

16. The LIDAR noise removal method according to claim 15, wherein: Identifying the object from the LIDAR detection information includes checking the number of object points of the marker object in a noise region corresponding to the region of interest, and performing filtering based on the number of object points of the marker object to identify the object. 17 . A non-volatile computer-readable recording medium storing a program for executing the method according to claim 10 .

18. A vehicle comprising: LIDAR configured to sense information about surrounding objects of the vehicle; a camera configured to capture an image of the front of the vehicle; as well as A LIDAR noise removal device configured to remove noise points corresponding to sunlight incident on the LIDAR, Wherein, the LIDAR noise removal device includes: a LIDAR detection information processor configured to process LIDAR detection information received from the LIDAR; a sun position acquirer configured to acquire an azimuth and elevation angle of the sun relative to a travel direction of the vehicle; a region of interest selector configured to select a region of interest corresponding to the sun from the front image of the vehicle based on the acquired azimuth and elevation of the sun, and compare the brightness of the selected region of interest with a threshold to determine whether the brightness of the region of interest exceeds the threshold; a noise region selector configured to select a noise region corresponding to the region of interest from the LIDAR detection information based on the azimuth and elevation angles of the sun when the brightness of the region of interest exceeds a threshold; and a noise remover configured to remove noise points in a selected noise area, An object identifier is configured to assign object point labels to all points in a noise region corresponding to the region of interest when the brightness of the region of interest is less than or equal to a threshold, and to modify labeling parameters of the noise region corresponding to the region of interest to reduce the distance threshold for distances between points in the noise region corresponding to the region of interest.

Images