LiDAR-Based Atmospheric Filtering System and Method

Abstract

The present invention provides a system and method for filtering the atmospheric state from LiDAR point clouds. 【Solution means】This method includes the step of generating at least one point cloud using a LiDAR system including a processor. This method further includes the steps of, using the processor, within one of the at least one point clouds, identifying and separating one or more ground points indicating a ground portion in the environment of the point cloud; removing the ground portion from the point cloud to generate an initially processed point cloud; within the initially processed point cloud, identifying and separating one or more atmospheric state points indicating one or more atmospheric states in the environment of the initially processed point cloud; and removing the atmospheric state points from the initially processed point cloud to generate a finally processed point cloud.

Description

Technical Field

[0001] This application claims the priority of U.S. Application No. 17 / 806,804, filed on June 14, 2022, the content of which is incorporated herein by reference.

[0002] Embodiments of the present disclosure relate to object detection by Light Detection and Ranging (LiDAR), and more particularly, to systems and methods for filtering atmospheric conditions from LiDAR point clouds.

Background Art

[0003] Autonomous vehicles or other self-driving vehicles need to be able to detect one or more objects and / or potential hazards in the environment in order to safely and efficiently navigate through the vehicle's environment and prevent possible collisions. To detect these objects and potential hazards, self-driving vehicles are often equipped with one or more environmental sensing technologies, such as photographic imaging systems and technologies (such as cameras), radio detection and ranging (RADAR) systems and technologies, and light detection and ranging (LiDAR) systems and technologies.

[0004] A LiDAR sensor is configured to emit light that hits a substance (e.g., an object) in its vicinity. When the light contacts the substance, it is deflected. A portion of the deflected light is reflected back to the LiDAR sensor. The LiDAR sensor is configured to measure data regarding the reflected light (e.g., the distance the light has traveled, the length of time it takes for the light to travel from the LiDAR sensor to the LiDAR sensor, the intensity of the light returning to the LiDAR sensor, etc.). Subsequently, by using this data to generate a point cloud of a part or all of the environment around the LiDAR sensor, an object map of the objects in the environment can generally be recreated.

[0005] When used in a vehicle, a LiDAR sensor can be used to detect one or more objects in the vehicle's environment. Since LiDAR uses data points collected using light reflected from one or more objects, the reliability of LiDAR, particularly with regard to its use in autonomous vehicles, can be affected by various atmospheric conditions such as rain and fog. The reason is that these conditions absorb and reflect light, causing the readings regarding the presence and / or size of objects in the vehicle's environment to be incorrect. Due to at least this drawback, LiDAR is often used in combination with other technologies such as photographic imaging systems and technologies, RADAR systems and technologies.

[0006] In existing technologies, through a density-based approach, an attempt is made to analyze the LiDAR point cloud to determine the point density of each point within the point cloud in order to adjust this limitation of LiDAR. Points with low density in the LiDAR point cloud (i.e., points whose distance from other points is greater than a threshold distance) are selected, and points with high density (i.e., points whose distance from other points is less than the threshold distance) are retained within the point cloud. One problem with this method is that due to some atmospheric conditions such as rain clouds, the density of points within the LiDAR point cloud becomes high, and there is a possibility that the LiDAR system will predict incorrect objects.

[0007] In other existing technologies, in combination with density analysis of the entire point cloud, the intensity of each point within the point cloud is analyzed. However, an object such as a vehicle can generate both high-intensity points and low-intensity points within the LiDAR point cloud. Therefore, this method also causes the identification of inappropriate objects or the failure to identify objects within the environment of the point cloud.

[0008] Another problem with existing technology systems and methods is the handling of the ground within the LiDAR point cloud. In existing technologies, since the ground is not identified, separated, and removed from the LiDAR point cloud, it becomes difficult to identify objects and / or atmospheric conditions along the relative boundary of the ground portion. Summary of the Invention Problems to be Solved by the Invention

[0009] Therefore, at least for these reasons, there is a need for a system and method for more accurately distinguishing the atmospheric state and objects within a LiDAR point cloud.

Means for Solving the Problem

[0010] According to an embodiment of the present disclosure, a method for filtering the atmospheric state from a light detection and ranging (LiDAR) point cloud is provided. This method may include the step of generating at least one point cloud using a LiDAR system including a processor. This method includes, using the processor, within one of the at least one point cloud, identifying and separating one or more ground points indicating a ground portion within the environment of the point cloud; removing the ground portion from the point cloud to generate a pre-processed point cloud; within the pre-processed point cloud, identifying and separating one or more atmospheric state points indicating one or more atmospheric states within the environment of the pre-processed point cloud; and further including the step of removing the atmospheric state points from the pre-processed point cloud to generate a final processed point cloud.

[0011] According to an embodiment of the present disclosure, the step of identifying and separating one or more atmospheric state points includes identifying one or more high-intensity points and one or more low-intensity points within the pre-processed point cloud, and the positions of each of the one or more high-intensity points and the positions of each of the one or more low-intensity points; for each of the one or more low-intensity points, determining an average distance to one or more high-intensity points among the one or more high-intensity points; and for each of the one or more low-intensity points, when the average distance is greater than a threshold distance, classifying the low-intensity point as an atmospheric state point.

[0012] According to an embodiment of the present disclosure, the step of identifying one or more high-intensity points and one or more low-intensity points in the pre-processed point cloud may include determining an intensity score for each point in the pre-processed point cloud, classifying each point in the pre-processed point cloud with an intensity score exceeding an intensity threshold as a high-intensity point, and classifying each point in the pre-processed point cloud with an intensity score below the intensity threshold as a low-intensity point.

[0013] According to an embodiment of the present disclosure, this method may further include classifying all points that were not removed from the final processed point cloud as obstacle points indicating one or more objects in the environment of the final processed point cloud.

[0014] According to an embodiment of the present disclosure, the one or more atmospheric conditions may include one or more of rain and fog.

[0015] According to an embodiment of the present disclosure, the at least one point cloud includes an initial point cloud and a subsequent point cloud, and the step of identifying and separating the one or more ground points may include identifying and separating one or more ground points in the initial point cloud, and identifying and separating one or more ground points in the subsequent point cloud by comparing one or more ground points in the initial point cloud with one or more points in the subsequent point cloud.

[0016] According to an embodiment of the present disclosure, the step of identifying and separating the one or more ground points may include comparing the point cloud with one or more secondary scans of the environment of the point cloud.

[0017] According to an embodiment of the present disclosure, the one or more secondary scans may include one or more of one or more two-dimensional (2D) camera images of the environment and one or more radio detection and ranging (RADAR) scans of the environment.

[0018] According to an embodiment of the present disclosure, a system for filtering the atmospheric state from a LiDAR point cloud is provided. The system may include a vehicle and a LiDAR system coupled to the vehicle. The LiDAR system may include one or more LiDAR sensors and a processor. The processor is configured to generate at least one point cloud of the environment, identify and separate one or more ground points indicating a ground portion in the environment of the point cloud within one of the at least one point clouds, remove the ground portion from the point cloud to generate a pre-processed point cloud, identify and separate one or more atmospheric state points indicating one or more atmospheric states in the environment of the pre-processed point cloud within the pre-processed point cloud, and remove the atmospheric state points from the pre-processed point cloud to generate a final processed point cloud.

[0019] According to an embodiment of the present disclosure, the step of identifying and separating one or more atmospheric state points may include identifying one or more high-intensity points and one or more low-intensity points within the pre-processed point cloud, and positions of each of the one or more high-intensity points and positions of each of the one or more low-intensity points, determining, for each of the one or more low-intensity points, an average distance to one or more of the one or more high-intensity points, and classifying, for each of the one or more low-intensity points, the low-intensity point as an atmospheric state point if the average distance is greater than a threshold distance.

[0020] According to an embodiment of the present disclosure, the step of identifying one or more high-intensity points and one or more low-intensity points within the pre-processed point cloud may include determining an intensity score for each point of the pre-processed point cloud, classifying, as high-intensity points, each point of the pre-processed point cloud whose intensity score exceeds an intensity threshold, and classifying, as low-intensity points, each point of the pre-processed point cloud whose intensity score is below the intensity threshold.

[0021] According to an embodiment of the present disclosure, the processor may be further configured to execute a step of classifying all points that were not removed from the final processed point cloud as obstacle points indicating one or more objects within the environment of the final processed point cloud.

[0022] According to an embodiment of the present disclosure, the at least one point cloud includes an initial point cloud and a subsequent point cloud, and the step of identifying and separating the one or more ground points may include identifying and separating one or more ground points within the initial point cloud, and identifying and separating one or more ground points within the subsequent point cloud by comparing the one or more ground points within the initial point cloud with the one or more points within the subsequent point cloud.

[0023] According to an embodiment of the present disclosure, the system may further include one or more secondary scanners configured to scan the environment of the point cloud and generate one or more secondary scans of the environment of the point cloud, and the step of identifying and separating the one or more ground points may include comparing the point cloud with the one or more secondary scans of the environment of the point cloud.

[0024] According to an embodiment of the present disclosure, the one or more secondary scanners may include one or more of one or more cameras and one or more RADAR scanners.

[0025] According to other embodiments of the present disclosure, a system is provided. The system includes at least one LiDAR system coupled to a vehicle and configured to generate one or more point clouds of the environment, and a computing device including a processor and a memory, coupled to the vehicle and configured to store programming instructions. When the programming instructions are executed by the processor, the processor is caused to identify and separate one or more ground points indicating a ground portion in the environment of the point cloud within one of the one or more point clouds, remove the ground portion from the point cloud to generate an initially processed point cloud, identify and separate one or more atmospheric state points indicating one or more atmospheric states in the environment of the initially processed point cloud within the initially processed point cloud, and remove the atmospheric state points from the initially processed point cloud to generate a finally processed point cloud.

[0026] According to an embodiment of the present disclosure, when the processor may identify and separate the one or more atmospheric state points, the programming instructions further cause the processor to identify one or more high-intensity points and one or more low-intensity points within the initially processed point cloud, and the positions of each of the one or more high-intensity points and the positions of each of the one or more low-intensity points, determine, for each of the one or more low-intensity points, an average distance to one or more high-intensity points among the one or more high-intensity points, and classify, for each of the one or more low-intensity points, the low-intensity point as an atmospheric state point if the average distance is greater than a threshold distance.

[0027] According to an embodiment of the present disclosure, when the processor identifies one or more high-intensity points and one or more low-intensity points in the initially processed point cloud, the programming instructions further cause the processor to determine an intensity score for each point in the initially processed point cloud, classify each point in the initially processed point cloud having an intensity score exceeding an intensity threshold as a high-intensity point, and classify each point in the initially processed point cloud having an intensity score below the intensity threshold as a low-intensity point.

[0028] According to an embodiment of the present disclosure, the programming instructions may further cause the processor to classify all points not removed from the finally processed point cloud as obstacle points indicating one or more objects in the environment of the finally processed point cloud.

[0029] According to an embodiment of the present disclosure, the at least one point cloud includes an initial point cloud and a subsequent point cloud. When the processor identifies and separates the one or more ground points, the programming instructions further cause the processor to identify and separate one or more ground points in the initial point cloud, and identify and separate one or more ground points in the subsequent point cloud by comparing the one or more ground points in the initial point cloud with one or more points in the subsequent point cloud.

Brief Description of the Drawings

[0030]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0031] The terms used in this specification are for the purpose of describing particular embodiments and are not intended to limit the present disclosure. As used in this specification, the singular forms "a", "one" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. These terms are only intended to distinguish one component from another and do not limit the nature, order or sequence of the components. The terms "comprises" and / or "comprising", when used in this specification, specify the presence of the features, integers, steps, operations, elements and / or components referred to, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. As used in this specification, the term "and / or" includes any and all combinations of one or more of the associated listed items. Throughout this specification, unless expressly stated to the contrary, the word "comprise" and variations such as "comprises" or "comprising" mean that the recited elements are included, but do not mean that other elements are excluded. Also, the terms "unit", "-er", "-or" and "module" described in this specification mean a unit that processes at least one function and operation, and can be implemented by hardware components, software components and combinations thereof.

[0032] In this document, when terms such as "first" and "second" are used to modify a noun, such use is merely for the purpose of distinguishing one item from another and does not require a consecutive order unless otherwise specified. Further, when terms indicating relative positions such as "vertical" and "horizontal" or "front" and "back" are used, those terms are relative to each other and need not be absolute, and shall refer to only one of the possible positions of the device associated with those terms depending on the orientation of the device.

[0033] "Electronic device" or "computing device" refers to a device that includes a processor and a memory. Each device may have its own processor and / or memory, or may share the processor and / or memory with other devices, such as in a virtual machine or container configuration. The memory contains programming instructions that, when executed by the processor, cause the electronic device to perform one or more operations according to the programming instructions.

[0034] Terms such as "memory", "memory device", "computer-readable storage medium", "data store", "data storage facility", etc. each refer to a non-transitory device in which computer-readable data, programming instructions, or both are stored. Unless otherwise specified, terms such as "memory", "memory device", "computer-readable storage medium", "data store", "data storage facility", etc. are intended to include single-device embodiments, embodiments in which multiple memory devices store a set of data or instructions together or collectively, and individual sectors within such devices.

[0035] The terms "processor" and "processing device" refer to the hardware components of an electronic device configured to execute programming instructions. Unless otherwise specified, the singular terms "processor" or "processing device" are intended to include both embodiments of a single processing device and embodiments in which multiple processing devices execute a process together or collectively.

[0036] The term "module" refers to a set of computer-readable programming instructions that, when executed by a processor, cause the processor to perform a specific function.

[0037] The term "vehicle" or other similar terms refer to any suitable motor vehicle driven by any appropriate power source and capable of transporting one or more passengers and / or cargo. The term "vehicle" includes, but is not limited to, autonomous vehicles (i.e., vehicles that do not require a human operator and / or require limited operation by a human operator through boarding or remote control), automobiles (e.g., passenger cars, trucks, sport utility vehicles, vans, buses, commercial vehicles, class 8 trucks, etc.), boats, drones, trains, and the like.

[0038] Exemplary embodiments are described as using multiple units to execute an exemplary process, but it should be understood that the exemplary process may be executed by one or more modules. Further, it should be understood that the term "controller / control unit" refers to a hardware device that includes a memory and a processor and is specifically programmed to execute the processes described herein. The memory is configured to store the modules, and the processor is specifically configured to cause the one or more processes further described below to be executed by the modules.

[0039] Furthermore, the control logic of the present disclosure may be embodied as a non-transitory computer-readable medium on a computer-readable medium including executable programming instructions executed by a processor, a controller, or the like. Examples of computer-readable media include, but are not limited to, ROM, RAM, compact disc (CD)-ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage devices. Since the computer-readable medium can also be distributed to a computer system connected to a network, it may be stored and executed in a distributed manner by a telematics server or a controller area network (CAN), or the like.

[0040] Unless otherwise specified or clear from the context, as used herein, the term "about" is understood to be within the normal acceptable range in the art, e.g., within two standard deviations of the average value. About can be understood to be within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, 0.1%, 0.05%, or 0.01% of the recited value.

[0041] Hereinafter, with reference to the drawings, some embodiments of the present disclosure will be described in detail. In the drawings, the same reference numerals are used throughout to indicate the same or equivalent elements. Also, detailed descriptions of well-known features or functions are omitted in order not to unnecessarily obscure the gist of the present disclosure.

[0042] Hereinafter, a system and method for filtering the atmospheric state from a light detection and ranging (LiDAR) point cloud according to an embodiment of the present disclosure will be described with reference to the accompanying drawings.

[0043] Now, referring to FIG. 1, a LiDAR-equipped vehicle 105 on a road 110 according to various embodiments of the present disclosure is illustratively shown.

[0044] According to various embodiments, vehicle 105 includes one or more sensors, such as, among suitable sensors, in particular one or more LiDAR sensors 115, one or more radio detection and ranging (RADAR) sensors 120, and one or more cameras 125. According to various embodiments, the one or more sensors may communicate electronically with one or more computing devices 130. The computing device 130 may be separate from the one or more sensors and / or may be incorporated into the one or more sensors. Vehicle 105 may include a LiDAR system that includes one or more LiDAR sensors 115 and one or more computing devices 130.

[0045] In the example of FIG. 1, the LiDAR sensor 115 is configured to emit light that strikes a substance (e.g., obstacle 150) in the environment of vehicle 105. The light is deflected when it contacts the substance. A portion of the deflected light is reflected back to the LiDAR sensor 115. The LiDAR sensor 115 may be configured to measure data regarding the reflected light (e.g., things understood by those skilled in the art such as the distance the light propagated, the length of time it took for the light to propagate from the LiDAR sensor 115 to the LiDAR sensor 115, the intensity of the light returning to the LiDAR sensor 115). Subsequently, an object map of the obstacle 150 in the environment may generally be recreated by generating a point cloud of part or all of the environment around vehicle 105 using this data.

[0046] According to various embodiments, the LiDAR sensor 115 may be coupled to the vehicle 105 and configured to generate one or more point clouds of the environment surrounding the vehicle 105. The environment may completely surround the vehicle 105 or may include a portion around the vehicle 105.

[0047] According to various embodiments, the computing device 130 may include a processor 135 and / or a memory 140. The memory 140 may be configured to store programming instructions, which when executed by the processor 135, cause the processor 135 to perform tasks such as, for example, generating one or more point clouds, identifying and separating one or more ground points within the point cloud, removing the ground portion from the point cloud to generate a pre-processed point cloud, identifying and separating one or more atmospheric state points within the pre-processed point cloud, and removing the atmospheric state points from the pre-processed point cloud to generate a final processed point cloud. According to various embodiments, the one or more ground points indicate a ground portion within the environment of the point cloud, and the one or more atmospheric state points indicate one or more atmospheric states within the environment of the processed point cloud. The atmospheric state may include, for example, rain, fog, smoke, smog, snow, dust, and / or other suitable forms of atmospheric states.

[0048] According to various embodiments, the task of identifying and separating one or more atmospheric state points may include identifying one or more high-intensity points and one or more low-intensity points within the pre-processed point cloud, and the positions of each of the one or more high-intensity points and the positions of each of the one or more low-intensity points. An intensity score for each point in the pre-processed point cloud may be determined. According to various embodiments, each of these intensity scores may be compared with an intensity threshold. The intensity threshold may be dynamic and / or pre-determined.

[0049] According to various embodiments, each point in the pre-processed point cloud with an intensity score exceeding the intensity threshold may be classified as a high-intensity point, and each point in the pre-processed point cloud with an intensity score below the intensity threshold may be classified as a low-intensity point. Thus, the point cloud may include low-intensity points, high-intensity points, and / or a mixture of high-intensity points and low-intensity points.

[0050] According to various embodiments, the processor 135 may be configured to determine, for each of one or more low-intensity points within a point cloud, an average distance to one or more high-intensity points among the one or more high-intensity points. For each of the one or more low-intensity points, if the average distance is greater than a threshold distance, the low-intensity point may be classified as an atmospheric condition point. The distance threshold may be dynamic and / or pre-determined.

[0051] The vehicle 105 may include a sensing system 200, as shown, for example, in FIG. 2. The sensing system 200 may be configured to assist the vehicle 105 in identifying / perceiving one or more obstacles 150 within its environment.

[0052] According to various embodiments, the sensing system 200 may include one or more sensors 205, including, for example, a LiDAR sensor 115, a RADAR sensor 120, a camera 125, etc. The one or more sensors 205 may be disposed at any suitable location along the vehicle 105 (e.g., front, side, rear, top, bottom, etc.).

[0053] According to various embodiments, the LiDAR sensor 115 may communicate electronically with one or more LiDAR annotators 210. The LiDAR annotator 210 may be configured to annotate one or more point clouds for use in obstacle detection in the path, potential paths, and / or environment of the vehicle 105 (e.g., using an obstacle detector 215).

[0054] According to various embodiments, the LiDAR annotator 210 may operate in conjunction with a ground model 220 configured to model a ground portion of one or more LiDAR point clouds. The ground portion of the LiDAR point cloud is typically high-intensity, high-density points within the point cloud. To more accurately detect atmospheric points along or near the ground portion of the point cloud, the systems and methods of the present disclosure may be configured to remove the ground portion before applying the analysis of a three-dimensional (3D) object detector 225 to the point cloud.

[0055] According to various embodiments, the LiDAR sensor 115 and / or the LiDAR annotator 210 may provide data that can be analyzed by the obstacle detector 215 to detect one or more obstacles in the path, potential paths, and / or environment of the vehicle 105. These obstacles (e.g., obstacle 150 shown in FIG. 1) may be transmitted to the planning module 230 that plans the trajectory of the vehicle 105. According to various embodiments, the obstacle detector 215 may include an environmental subtraction module 235 configured to subtract from the obstacle detector 215 areas of the environment where environmental conditions (e.g., rain, fog, and / or other weather conditions) are identified as obstacles 150.

[0056] According to various embodiments, as would be understood by one of ordinary skill in the art, when identifying and / or separating one or more ground points, the perception system 200 may be configured to compare the LiDAR point cloud with one or more secondary scans of the environment such as, for example, one or more two-dimensional (2D) camera images of the environment, RADAR scans of the environment, and / or other suitable maps incorporating surface features of the road and / or other suitable methods for determining the road surface. Alternatively, the LiDAR point cloud itself may be analyzed to determine the road surface. According to various embodiments, one or more of the LiDAR sensor 115, the RADAR sensor 120, and the camera 125 may supply data for analysis by a 2D object detection module 240 (for detecting objects in 2D space) and / or a 3D object detection module 245 (for detecting objects in 3D space). The 3D object detection module 245 may further be used to determine ground points within the point cloud. Also, a 2D / 3D conversion module 250 may be used to further convert 2D images to 3D for 3D analysis.

[0057] According to various embodiments, data from one or more sensors 205 and / or data from object detection modules 240, 245 may be sent to a fusion tracker module 255 configured to fuse data from multiple sensors and / or sensor types and / or ground data while tracking an object or obstacle from one analysis cycle to another. According to various embodiments, the fusion tracker module 255 may be configured to analyze previous sensor data analysis results against new sensor data to maintain and / or modify object and / or obstacle data to improve obstacle determination accuracy. The results of the fusion tracker module 255 may be sent to a planning module 230 used to plan the trajectory of the vehicle 105.

[0058] Referring now to FIG. 3, an exemplary flowchart of a method 300 for identifying, separating, and filtering ground points and atmospheric state points in a LiDAR point cloud according to various embodiments of the present disclosure is shown.

[0059] At 305, one or more LiDAR point clouds are generated using a LiDAR system and a computer processor that includes one or more LiDAR sensors and at least one computer memory. According to various embodiments, the LiDAR point clouds within the one or more LiDAR point clouds are initial LiDAR point clouds that represent all or a portion of the surrounding environment of the vehicle. The LiDAR point clouds may be obtained, for example, from a combination of different types of LiDAR sensors. For example, a scanning LiDAR sensor, a spinning LiDAR sensor, a flash LiDAR sensor, and / or other suitable types of LiDAR sensors may be combined within the LiDAR point clouds.

[0060] At 310, one or more ground points within the initial point cloud are identified and separated. The one or more ground points are identified using any suitable means such as, for example, an analysis comparing the initial LiDAR point cloud with other sensor data (camera data, RADAR data, etc.), an analysis comparing the LiDAR point cloud with one or more previous LiDAR point cloud analysis results, an analysis of the point intensity and / or point density within the initial LiDAR point cloud (based on a threshold to determine the intensity and / or density of known ground points), and / or other suitable means. According to various embodiments, the one or more ground points may indicate a ground portion within the environment of the point cloud.

[0061] According to various embodiments, at least one point cloud includes an initial point cloud and a subsequent point cloud, and the step of identifying and separating one or more ground points includes the step of identifying and separating one or more ground points within the initial point cloud and the step of identifying and separating one or more ground points within the subsequent point cloud by comparing the one or more ground points within the initial point cloud with one or more points within the subsequent point cloud.

[0062] At 315, one or more ground points are removed from the initial LiDAR point cloud to generate an initial processed point cloud.

[0063] At 320, the initial processed point cloud is analyzed to identify and separate one or more atmospheric condition points within the initial processed point cloud. According to various embodiments, the initial processed point cloud may be a set of non-ground points having returns from a LiDAR scan. The one or more atmospheric condition points indicate one or more atmospheric conditions within the environment of the processed point cloud. The atmospheric conditions may include, for example, rain, fog, smoke, smog, snow, dust, and / or other suitable forms of atmospheric conditions. At 325, one or more atmospheric condition points are removed from the initial processed point cloud to generate a final processed point cloud. At 330, the remaining points within the final processed point cloud are classified as obstacle points indicating one or more obstacles. Steps 320, 325 are shown in and described in more detail in FIG. 4.

[0064] In 335, the trajectory of the vehicle is planned, and the finally processed point cloud and obstacle data are incorporated into the trajectory so that the vehicle can avoid one or more classified obstacles.

[0065] Figure 4 is an exemplary flowchart of a method for identifying and separating (step 320) and filtering (step 325) atmospheric state points within a LiDAR pre-processed point cloud according to various embodiments of the present disclosure.

[0066] In 405, non-ground points are identified from the pre-processed point cloud. In 410, it is determined whether each non-ground point is of high intensity. According to various embodiments, in order to determine whether a non-ground point is of high intensity, a density score may be assigned to the non-ground point based on the LiDAR data of that point in the point cloud. For each non-ground point, if the intensity score is greater than a predetermined intensity threshold score, the non-ground point is classified as a high-intensity point. For each non-ground point, if the intensity score is less than a predetermined intensity threshold score, the non-ground point is classified as a low-intensity point. In 420, each non-ground point (high-intensity point) classified as being of high intensity is retained within the point cloud.

[0067] For each non-ground point (low-intensity point) classified as having a low intensity, the position of the low-intensity point and the average distance from the low-intensity point to nearby high-intensity points are determined. At 415, for each low-intensity point that is non-ground, it is determined whether the low-intensity point is close to a high-intensity point. If the average distance to the closest number of high-intensity points from the low-intensity point is less than the threshold distance, the low-intensity point is considered to be close to the high-intensity point. Otherwise, the low-intensity point is not considered to be close to the high-intensity point. According to various embodiments, the threshold distance may be a distance selected from a length of about 10 cm to 3 m. However, it should be noted that other appropriate threshold distances may be used on the premise of maintaining the spirit and functionality of the present disclosure. According to various embodiments, when the threshold distance is selectively changed, the number of nearby high-intensity points used to calculate the threshold distance may also increase. In other embodiments, the initial distance measurement may be used to search only for points within a designated point group around the low-intensity point under consideration. Thereby, the number of high-intensity points for which it is necessary to calculate the distance between the low-intensity point and the high-intensity point can be limited.

[0068] At 420, if it is determined that the low-intensity point is close to the high-intensity point, the low-intensity point is retained in the point group. At 425, if it is determined that the low-intensity point is not close to the high-intensity point, the low-intensity point is classified as an atmospheric point and deleted from the point group.

[0069] According to some embodiments, the setting of the intensity threshold and the threshold distance may be dynamic based on the state detected in the environment. For example, when the intensity is close to the intensity threshold, in a situation where the intensity scores of the points are more uniformly high, the threshold distance may be set to a higher value. Conversely, when the intensity score is low within the scan, the threshold distance for the intensity score close to the intensity threshold becomes low, and more points may be captured as part of the obstacle 150.

[0070] Referring now to FIG. 5, a diagram of an exemplary architecture of a computing device 500 is provided. The computing device 130 of FIG. 1 may be the same as or similar to the computing device 500. Thus, for example, to understand the computing device 130 of FIG. 1, the description of the computing device 500 is sufficient.

[0071] The computing device 500 may include more or fewer components than those shown in FIG. 1. The hardware architecture of FIG. 5 represents one exemplary implementation of a representative computing device configured with one or more methods and means for filtering atmospheric conditions from a LiDAR point cloud, as described herein. Thus, the computing device 500 of FIG. 5 implements at least a portion of the methods described herein (e.g., method 300 of FIG. 3 and / or method 400 of FIG. 4).

[0072] Some or all components of the computing device 500 may be implemented as hardware, software, and / or a combination of hardware and software. Hardware includes, but is not limited to, one or more electronic circuits. Electronic circuits include, but are not limited to, passive elements (e.g., resistors and capacitors) and / or active elements (e.g., amplifiers and / or microprocessors). Passive elements and / or active elements may be adapted, arranged, and / or programmed to perform one or more of the methodologies, procedures, or functions described herein.

[0073] As shown in FIG. 5, computing device 500 includes a user interface 502, a central processing unit (“CPU”) 506, a system bus 510, a memory 512 connected to and accessible via the system bus 510 to other parts of the computing device 500, and a hardware entity 514 connected to the system bus 510. The user interface 502 may include input and output devices that facilitate interaction between the user and software for controlling the operation of the computing device 500. The input device includes, but is not limited to, a physical keyboard and / or a touch keyboard 550. The input device may be connected to the computing device 500 via a wired connection or a wireless connection (e.g., a Bluetooth® connection). The output device includes, but is not limited to, a speaker 552, a display 554, and / or a light emitting diode 556.

[0074] At least a part of the hardware entity 514 executes actions involving access to and use of the memory 512, and the memory 512 may be other suitable memory types such as random access memory (RAM), disk driver and / or compact disc read-only memory (CD-ROM). The hardware entity 514 may include a disk drive unit 516 that includes a computer-readable storage medium 518 storing one or more sets of instructions 520 (such as programming instructions such as software code, but not limited thereto) configured to implement one or more of the methodologies, procedures, or functions described herein. The instructions 520 may be wholly or at least partially present in the memory 512 and / or within the CPU 506 while being executed by the computing device 500. The memory 512 and the CPU 506 may constitute a machine-readable medium. As used herein, the term "machine-readable medium" refers to a single medium or multiple media (such as a centralized or distributed database and / or associated cache and server) storing one or more sets of the instructions 520. As used herein, the term "machine-readable medium" also refers to any medium that can store, encode, or carry a set of instructions 520 executable by the computing device 500 and cause the computing device 500 to execute one or more of the methodologies of the present disclosure.

[0075] Referring now to FIG. 6, an exemplary vehicle system architecture 600 for a vehicle, according to various embodiments of the present disclosure, is provided.

[0076] The vehicle 105 of FIG. 1 may have the same or a similar system architecture as that shown in FIG. 6. Thus, the following description of the vehicle system architecture 600 is sufficient to understand the vehicle 105 of FIG. 1.

[0077] As shown in FIG. 6, a vehicle system architecture 600 includes an engine or motor or propulsion device (e.g., thruster) 602 and various sensors 604-618 that measure various parameters of the vehicle system architecture 600. In a gas-powered vehicle or hybrid vehicle equipped with a fuel-driven engine, the sensors 604-618 may include, for example, an engine temperature sensor 604, a battery voltage sensor 606, an engine revolutions per minute (RPM) sensor 608, and / or a throttle position sensor 610. If the vehicle is an electric vehicle or hybrid vehicle, the vehicle may include a battery monitoring system 612 (which measures the current, voltage, and / or temperature of the battery), a motor current sensor 614, a voltage sensor 616, and sensors such as a motor position sensor 618, such as a resolver and an encoder.

[0078] Operation parameter sensors common to both types of vehicles include, for example, position sensors 634 such as an accelerometer, a gyroscope, and / or an inertial measurement unit, a speed sensor 636, and / or an odometer sensor 638. The vehicle system architecture 600 may include a clock 642 that is used to determine vehicle time while the system is operating. The clock 642 may be encoded in the vehicle-mounted computing device 620, may be a separate device, and multiple clocks may be available.

[0079] The vehicle system architecture 600 may include various sensors that operate to collect information about the environment in which the vehicle is traveling. These sensors may include, for example, a position sensor 644 (e.g., a global positioning system (GPS) device), an object detection sensor such as one or more cameras 646, a LiDAR sensor system 648 and / or a radar and / or sonar system 650. The sensors may include environmental sensors 652 such as a precipitation sensor and / or an ambient temperature sensor. The object detection sensor enables the vehicle system architecture 600 to detect objects within a predetermined distance range in any direction of the vehicle 105, and the environmental sensor 652 collects data regarding the environmental conditions within the driving area of the vehicle.

[0080] During operation, information is transmitted from the sensors to the on-vehicle computing device 620. The on-vehicle computing device 620 may be configured to analyze data captured by the sensors and / or data received from a data provider, and may optionally be configured to control the operation of the vehicle system architecture 600 based on the analysis results. For example, the on-vehicle computing device 620 may control the brakes via a brake controller 622, control the direction via a steering controller 624, control the speed and acceleration via a throttle controller 626 (in the case of a gasoline-powered vehicle) or a motor speed controller 628 (such as a current level controller for an electric vehicle), and may be configured to control via a differential gear controller 630 (in the case of a vehicle with a transmission) and / or other controllers.

[0081] Geolocation information may be communicated from the location sensor 644 to the on-board computing device 620, after which the on-board computing device 620 may access a map of the environment corresponding to the location information to determine known fixed features of the environment such as roads, buildings, stop signs and / or stop / go signals. Image captured from the camera 646 and / or object detection information captured from sensors such as the LiDAR sensor system 648 are communicated from those sensors to the on-board computing device 620. The object detection information and / or the captured image are processed by the on-board computing device 620 to detect objects near the vehicle. Known or hereafter known techniques for performing object detection based on sensor data and / or captured images may be used in the embodiments disclosed herein.

[0082] The features and functions described above, as well as alternative means, may be combined in many other different systems or applications. Those skilled in the art can make various alternative means, modifications, changes or improvements, and each of those alternative means, modifications, changes or improvements is also considered to be included in the disclosed embodiments.

Claims

1. A method for filtering the atmospheric state from a point cloud of optical detection and ranging (LiDAR), comprising: generating at least one point cloud using a LiDAR system including a processor; using the processor, identifying and separating one or more ground points indicating a ground portion in the environment of the point cloud within one of the at least one point cloud; removing the ground portion from the point cloud to generate a pre-processed point cloud; identifying and separating one or more atmospheric state points indicating one or more atmospheric states in the environment of the pre-processed point cloud within the pre-processed point cloud; removing the atmospheric state points from the pre-processed point cloud to generate a final processed point cloud, and executing steps including the above steps.

2. The step of identifying and separating one or more atmospheric state points includes: identifying one or more high-intensity points and one or more low-intensity points within the pre-processed point cloud, and the positions of each of the one or more high-intensity points and the positions of each of the one or more low-intensity points; determining, for each of the one or more low-intensity points, an average distance to one or more high-intensity points among the one or more high-intensity points; classifying, for each of the one or more low-intensity points, the low-intensity point as an atmospheric state point if the average distance is greater than a threshold distance. The method according to claim 1.

3. The step of identifying one or more high-intensity points and one or more low-intensity points within the pre-processed point cloud includes: determining an intensity score for each point of the pre-processed point cloud; classifying, as high-intensity points, each point of the pre-processed point cloud whose intensity score exceeds an intensity threshold; classifying, as low-intensity points, each point of the pre-processed point cloud whose intensity score is below the intensity threshold. The method according to claim 2.

4. The method according to claim 1, further comprising classifying all points not removed from the final processed point cloud as obstacle points indicating one or more objects in the environment of the final processed point cloud.

5. The one or more atmospheric states include one or more of rain and fog. The method according to claim 1.

6. The at least one point cloud includes an initial point cloud and a subsequent point cloud, The step of identifying and separating the one or more ground points includes: identifying and separating one or more ground points within the initial point cloud; The method according to claim 1, comprising the step of identifying and separating one or more ground points in the subsequent point cloud by comparing one or more ground points in the initial point cloud with one or more points in the subsequent point cloud.

7. The method according to claim 1, wherein the step of identifying and separating the one or more ground points comprises comparing the point cloud with one or more secondary scans of the environment of the point cloud.

8. The method according to claim 7, wherein the one or more secondary scans include one or more of one or more two-dimensional camera images of the environment and one or more radio detection and ranging (RADAR) scans of the environment.

9. A system for filtering the atmospheric state from a light detection and ranging (LiDAR) point cloud, comprising: a vehicle; a LiDAR system coupled to the vehicle, the LiDAR system comprising: one or more LiDAR sensors; a processor, the processor being configured to: generate at least one point cloud of the environment; identify and separate one or more ground points indicating a ground portion in the environment of the point cloud within one of the at least one point cloud; remove the ground portion from the point cloud to generate an initially processed point cloud; identify and separate one or more atmospheric state points indicating one or more atmospheric states in the environment of the initially processed point cloud within the initially processed point cloud; remove the atmospheric state points from the initially processed point cloud to generate a finally processed point cloud.

10. The step of identifying and separating one or more atmospheric state points comprises: identifying one or more high-intensity points and one or more low-intensity points in the initially processed point cloud, and the positions of each of the one or more high-intensity points and the positions of each of the one or more low-intensity points; determining, for each of the one or more low-intensity points, an average distance to one or more of the one or more high-intensity points; classifying, for each of the one or more low-intensity points, the low-intensity point as an atmospheric state point if the average distance is greater than a threshold distance. The system according to claim 9.

11. The step of identifying one or more high-intensity points and one or more low-intensity points in the initially processed point cloud comprises: determining an intensity score for each point in the initially processed point cloud; classifying, as high-intensity points, each point in the initially processed point cloud whose intensity score exceeds an intensity threshold; The system according to claim 10, further comprising: classifying, as low-intensity points, each point in the initially processed point cloud whose intensity score is below the intensity threshold.

12. The system according to claim 9, wherein the processor is further configured to execute a step of classifying all points that were not removed from the finally processed point cloud as obstacle points indicating one or more objects within the environment of the finally processed point cloud.

13. The at least one point cloud includes an initial point cloud and a subsequent point cloud, The step of identifying and separating the one or more ground points includes: identifying and separating one or more ground points within the initial point cloud; The system according to claim 9, further comprising: identifying and separating one or more ground points within the subsequent point cloud by comparing the one or more ground points within the initial point cloud with one or more points within the subsequent point cloud.

14. scanning the environment of the point cloud, further comprising one or more secondary scanners configured to generate one or more secondary scans of the environment of the point cloud, The system according to claim 9, wherein the step of identifying and separating the one or more ground points includes comparing the point cloud with the one or more secondary scans of the environment of the point cloud.

15. The system according to claim 14, wherein the one or more secondary scanners include one or more of one or more cameras and one or more radio detection and ranging (RADAR) scanners.

16. at least one light detection and ranging (LiDAR) system coupled to a vehicle and configured to generate one or more point clouds of an environment; a computing device including a processor and a memory, coupled to the vehicle and configured to store programming instructions, the programming instructions, when executed by the processor, cause the processor to: identifying and separating, within one of the one or more point clouds, one or more ground points indicating a ground portion within the environment of the point cloud; removing the ground portion from the point cloud to generate an initially processed point cloud; identifying and separating, within the initially processed point cloud, one or more atmospheric state points indicating one or more atmospheric states within the environment of the initially processed point cloud; A system that executes a step of removing the atmospheric state points from the initially processed point cloud to generate a finally processed point cloud.

17. When the processor identifies and separates the one or more atmospheric state points, the programming instructions further cause the processor to Identify one or more high-intensity points and one or more low-intensity points in the initially processed point cloud, and the positions of each of the one or more high-intensity points and the positions of each of the one or more low-intensity points; For each of the one or more low-intensity points, determine the average distance to one or more of the high-intensity points among the one or more high-intensity points; The system according to claim 16, wherein, for each of the one or more low-intensity points, when the average distance is greater than a threshold distance, the low-intensity point is classified as an atmospheric state point.

18. When the processor identifies one or more high-intensity points and one or more low-intensity points in the initially processed point cloud, the programming instructions further cause the processor to Determine an intensity score for each point in the initially processed point cloud; Classify each point in the initially processed point cloud whose intensity score exceeds an intensity threshold as a high-intensity point; The system according to claim 17, wherein each point in the initially processed point cloud whose intensity score is below the intensity threshold is classified as a low-intensity point.

19. The programming instructions further cause the processor to classify all points that were not removed from the finally processed point cloud as obstacle points indicating one or more objects in the environment of the finally processed point cloud. The system according to claim 16.

20. The one or more point clouds include an initial point cloud and a subsequent point cloud. When the processor identifies and separates the one or more ground points, the programming instructions further cause the processor to Identify and separate one or more ground points in the initial point cloud; The system according to claim 16, wherein one or more ground points in the subsequent point cloud are identified and separated by comparing one or more ground points in the initial point cloud with one or more points in the subsequent point cloud.

Images