Method and apparatus for lidar data clustering based on multiple range image generation

Abstract

A LiDAR data clustering method based on multiple range image generation according to a first aspect of the present invention comprises: a step of classifying a plurality of point cloud data into a plurality of groups based on distance relationships between a plurality of point cloud data mapped to a single cell; a step of generating a plurality of range images corresponding to the plurality of groups; a step of performing a distance value-based array operation between point cloud data corresponding to neighboring cells using the plurality of range images; and a step of performing clustering of the plurality of point cloud data based on the array operation.

Description

Technology Field

[0001] The present invention relates to a LiDAR data clustering method and apparatus based on range image multiple generation. Background Technology

[0002] LiDAR (light detection and ranging) refers to a sensor that transmits near-infrared, visible, and ultraviolet light to an object and collects the object's three-dimensional information (x, y, z) by detecting the light rays that reflect back after hitting the object through a light sensor. Generally, LiDAR uses light rays of a fixed wavelength and measures the time it takes for the light to be reflected back and the intensity of the light rays.

[0003] Point cloud data acquired through LiDAR consists of a set of multiple points distributed in space. To detect or separate objects using this point cloud data, clustering processing to group individual points into meaningful cluster units is essential. Conventionally, clustering based on distance relationships between point cloud data has been primarily used, and this method has the advantage of a relatively simple computational structure.

[0004] However, in solid-state LiDAR, multiple point cloud data frequently overlap and map to the same orientation or cell location depending on the sensor structure or scanning method. In such cases, conventional methods use approaches that store only one representative point per pixel or select and process only a subset of the multiple points. As a result, problems arise where the remaining point cloud data corresponding to the same pixel is excluded during the clustering process, or points belonging to different objects are incorrectly grouped into the same cluster. The problem to be solved

[0005] The problem to be solved by the present invention includes providing a technology that groups a plurality of point cloud data mapped to a single cell, generates a plurality of range images corresponding to the number of groups, and then performs clustering on the point cloud data through range image-based array operations.

[0006] However, the problems that the present invention aims to solve are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below. means of solving the problem

[0007] A LiDAR data clustering method based on multiple range image generation according to a first aspect of the present invention comprises: a step of classifying a plurality of point cloud data into a plurality of groups based on distance relationships between a plurality of point cloud data mapped to a single cell; a step of generating a plurality of range images corresponding to the plurality of groups; a step of performing a distance value-based array operation between point cloud data corresponding to neighboring cells using the plurality of range images; and a step of performing clustering of the plurality of point cloud data based on the array operation.

[0008] The above distance relationship may include distance values ​​calculated based on the three-dimensional coordinate values ​​of each point cloud data.

[0009] In the step of classifying the above data into multiple groups, if the distance value is greater than a preset threshold, it can be classified into different groups.

[0010] Among the above multiple range images, one range image may correspond to each group.

[0011] In the step of generating the plurality of range images, the plurality of range images may be generated such that point cloud data classified into different groups among the point cloud data corresponding to the same cell is mapped to different range images.

[0012] The above array operation can be performed based on a three-dimensional array structure including the plurality of range images.

[0013] In the step of performing the above array operation, the neighbor cells included in the remaining range images can be searched based on the reference cell included in the reference range image among the plurality of range images.

[0014] In the step of performing the clustering above, a distance value calculated based on three-dimensional coordinate values ​​between reference point cloud data included in the reference cell and neighbor point cloud data included in the neighbor cell can be compared with a preset threshold.

[0015] In the step of performing the clustering above, if the distance value exceeds the threshold, the reference point cloud data and the neighbor point cloud data may be clustered into different clusters.

[0016] Each of the above multiple range images includes information regarding the azimuth angle, elevation angle, and image index, but in the step of performing the array operation, information corresponding to the elevation angle of each cloud point data is removed so that the multiple range images can be stored in a two-dimensional array form.

[0017] A LiDAR data clustering device according to a second aspect of the present invention comprises a memory for storing at least one instruction and a processor, wherein the at least one instruction is executed by the processor, the device classifies the plurality of point cloud data into a plurality of groups based on the distance relationship between the plurality of point cloud data mapped to a cell, generates a plurality of range images corresponding to the plurality of groups, performs a distance value-based array operation between the point cloud data corresponding to neighboring cells using the plurality of range images, and performs clustering of the plurality of point cloud data based on the array operation.

[0018] A computer-readable recording medium storing at least one computer-executable instruction according to a third aspect of the present invention, wherein the at least one instruction, when executed by a processor, comprises the steps of: classifying the plurality of point cloud data into a plurality of groups based on distance relationships between the plurality of point cloud data mapped to a cell; generating a plurality of range images corresponding to the plurality of groups; performing a distance value-based array operation between point cloud data corresponding to neighboring cells using the plurality of range images; and performing clustering of the plurality of point cloud data based on the array operation, thereby enabling the processor to perform the method.

[0019] A computer program stored in a computer-readable recording medium according to a fourth aspect of the present invention, the computer program comprises instructions for the processor to perform a method comprising: classifying a plurality of point cloud data into a plurality of groups based on distance relationships between a plurality of point cloud data mapped to a cell when executed by a processor; generating a plurality of range images corresponding to the plurality of groups; performing a distance value-based array operation between point cloud data corresponding to neighboring cells using the plurality of range images; and performing clustering of the plurality of point cloud data based on the array operation. Effects of the invention

[0020] According to one embodiment, point cloud data mapped and overlaid on the same cell can be utilized without loss, and accordingly, the clustering accuracy of the point cloud data can be improved. In addition, even when point cloud data corresponding to the same cell belong to different objects, the boundaries between objects can be distinguished more clearly.

[0021] Furthermore, the relationships between spatially adjacent point cloud data can be effectively reflected, and phenomena such as object shape distortion or separation can be reduced compared to conventional methods using single-range images. As a result, stable clustering results can be obtained even in complex environments.

[0022] In addition, by using an array operation method that removes information corresponding to elevation angles among the components of a range image and stores it in a 2D array form, the amount of computation and memory usage required for array operations can be reduced. Consequently, the applicability can be improved in autonomous driving environments or embedded systems that require real-time processing. Furthermore, clustering processing efficiency can be improved as unnecessary dimensional information is removed.

[0023] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below. Brief explanation of the drawing

[0024] FIG. 1 is a block diagram exemplarily illustrating a LiDAR data clustering device according to one embodiment. Figure 2 is a block diagram exemplifying the functions of a LiDAR data clustering program. FIG. 3 is a flowchart exemplarily illustrating a range image multi-generation-based LiDAR data clustering method according to one embodiment. Figure 4 illustrates, in an exemplary manner, the generation of a range image using azimuth and elevation information. Figures 5 and 6 exemplarily illustrate the operation of generating clusters based on the distance relationship between a reference cell and neighbor cells in a single-range image. FIG. 7 illustrates an exemplary range image generated according to one embodiment. FIGS. 8 and 9 exemplarily illustrate the limitations of a conventional LiDAR data clustering method. FIGS. 10 to 13 specifically illustrate a range image multiple generation-based LiDAR data clustering method according to one embodiment. FIGS. 14 and 19 exemplarily illustrate performing operations for clustering by optimizing a three-dimensional array into a two-dimensional array form. FIG. 20 exemplarily illustrates clustering results output according to a single-range image-based clustering method and a range image multi-generation-based LiDAR data clustering method according to one embodiment. Specific details for implementing the invention

[0025] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0026] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions in the embodiments of the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification.

[0027] First, the terms used in this specification will be briefly explained, and the invention will be described in detail.

[0028] The terms used in this specification have been selected to be as widely used as possible, taking into account the functions of the present invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0029] When a part of a specification is described as 'comprising' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0030] Additionally, the term "part" as used in the specification refers to software or hardware components, such as FPGAs or ASICs, and the "part" performs certain roles. However, the meaning of "part" is not limited to software or hardware. The "part" may be configured to reside in an addressable storage medium or configured to run one or more processors. Thus, by example, the "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts."

[0031] Below, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention.

[0032] FIG. 1 is a block diagram exemplarily illustrating a LiDAR data clustering device according to one embodiment.

[0033] As shown in FIG. 1, the LiDAR data clustering device (100) may include an input unit (110), an output unit (120), a processor (130), a memory (140), or a communication unit (160).

[0034] For convenience of explanation, the following description describes the LiDAR data clustering device (100) as an example that includes an input unit (110), an output unit (120), a processor (130), a memory (140), or a communication unit (160), but is not limited thereto. That is, each unit configuration may be provided outside the LiDAR data clustering device (100) and operate in a manner that interacts with the LiDAR data clustering device (100).

[0035] The input unit (110) may include a user interface for receiving commands, information, etc., used to control the LiDAR data clustering device (100). Additionally, the input unit (110) may be a hardware device (e.g., keyboard, mouse, touchpad, etc.) capable of directly receiving commands, information, etc., used to control the LiDAR data clustering device (100).

[0036] The input unit (110) may include a light receiving module capable of receiving a reflected light beam that is reflected from an object by a transmitted light beam.

[0037] In one embodiment, the input unit (110) can receive information required for a range image multi-generation-based LiDAR data clustering method. Specifically, information including point cloud data can be input through the input unit (110).

[0038] The output unit (120) may include a light transmission module that transmits a light beam around the LiDAR data clustering device (100).

[0039] The output unit (120) can provide information including cloud data, range images, or clustering results to the user as visual information through an interface.

[0040] The processor (130) can control the overall operation of the LiDAR data clustering device (100) to carry out the present invention.

[0041] The processor (130) can execute the LiDAR data clustering program (150) by loading information required for the execution of the LiDAR data clustering program (150) from memory (140).

[0042] The processor (130) can control the LiDAR data clustering device (100) to store data received from an external device through the communication unit (160) in the memory (140). Additionally, the processor (130) can control the LiDAR data clustering device (100) to transmit and receive information including cloud data, range images, or clustering results to and from an external device through the communication unit (160).

[0043] The processor (130) may refer to a processing unit such as a microprocessor, a central processing unit (CPU), a graphic processing unit (GPU), a tensor processing unit (TPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a microcontroller unit (MCU), but is not limited to the embodiments described above.

[0044] The memory (140) can store information necessary for the execution of the LiDAR data clustering program (150) and the LiDAR data clustering program (150). Additionally, the memory (140) can store processing results by the processor (130).

[0045] The LiDAR data clustering program (150) may mean software comprising computer-executable instructions programmed to perform the method according to the present invention.

[0046] The memory (140) can store information including cloud data, range images, or clustering results. Additionally, the memory (140) can store information received from an external device through the communication unit (160).

[0047] Memory (140) may refer to a computer-readable recording medium such as a magnetic medium such as a hard disk, floppy disk, and magnetic tape, an optical recording medium such as a CD-ROM and DVD, a magneto-optical medium such as a floptical disk, a random access memory such as a DRAM (dynamic random access memory), a SRAM (static random access memory), a hardware device specifically configured to store and execute program instructions such as a flash memory, but is not limited to the embodiments described above.

[0048] The communication unit (160) may be implemented to include a wired or wireless communication module. The wired or wireless communication module may be of various forms and is not limited to one of them.

[0049] The communication unit (160) may be a wireless communication module capable of performing wireless communication by adopting a communication method such as CDMA, GSM, W-CDMA, TD-SCDMA, WiBro, LTE, EPC, 5G, wireless LAN, Wi-Fi, Bluetooth, Zigbee, WFD (Wi-Fi Direct), UWB (Ultra Wide Band), infrared communication (IrDA; infrared data association), BLE (Bluetooth Low Energy) or NFC (Near Field Communication), but is not limited to the above-described embodiment.

[0050] The communication unit (160) may be a wired communication module capable of performing wired communication by adopting a communication method such as USB (universal serial bus), HDMI (high definition multimedia interface), RS-232 (recommended standard 232), LAN (Local Area Network), WAN (Wide Area Network), Internet, or telephone network, but is not limited to the above-described embodiment.

[0051] In addition, information input and output through the input unit (110) and output unit (120), information stored in the memory (140), and information transmitted and received through the communication unit (160) include all information related to the present invention and are not limited to the embodiments described above.

[0052] The function or operation of the LiDAR data clustering program (150) will be examined in detail through FIG. 2.

[0053] Figure 2 is a block diagram exemplifying the functions of a LiDAR data clustering program.

[0054] As shown in FIG. 2, the LiDAR data clustering program (150) may include a point cloud data classification unit (210), a range image generation unit (220), an array operation unit (230), and a clustering unit (240). The point cloud data classification unit (210), the range image generation unit (220), the array operation unit (230), and the clustering unit (240) are exemplary divisions of the functions of the LiDAR data clustering program (150) and are not limited thereto.

[0055] According to the embodiment, the functions of the point cloud data classification unit (210), range image generation unit (220), array operation unit (230), and clustering unit (240) can be merged / separated and can be implemented as a series of computer-executable instructions included in at least one program.

[0056] Each of the point cloud data classification unit (210), range image generation unit (220), array operation unit (230), and clustering unit (240) is stored in memory (140) and can perform the functions described below when executed by the aforementioned processor (130).

[0057] The point cloud data classification unit (210) can classify multiple point cloud data into multiple groups based on the distance relationship between multiple point cloud data mapped to one cell.

[0058] A cell can refer to a grid unit on a range image, that is, a data processing unit to which point cloud data is mapped.

[0059] Point cloud data may be a data set containing spatial coordinate information for multiple points measured by LiDAR.

[0060] In one embodiment, the aforementioned distance relationship may include a distance value calculated based on the three-dimensional coordinate values ​​of each point cloud data.

[0061] In one embodiment, the point cloud data classification unit (210) can classify point cloud data into different groups when the distance value is greater than a preset threshold.

[0062] The range image generation unit (220) can generate multiple range images corresponding to multiple groups.

[0063] A range image is a data structure composed of cells, and distance information of point cloud data may be stored in each cell. A range image may refer to an image comprising a data structure arranged in a two-dimensional grid form based on spatial orientation information of each point cloud data acquired from a lidar. Specifically, a range image may include a plurality of cells defined based on at least one of an azimuth angle and an elevation angle corresponding to each point cloud data relative to the lidar, and each cell may store the range value of the point cloud data corresponding to that cell or information based on the range value.

[0064] In one embodiment, point cloud data can be mapped into a range image in a two-dimensional coordinate system using direction information based on the lidar. Here, the direction information is a direction value defined according to the lidar's scan structure, and may include, for example, azimuth and elevation information defined corresponding to each point during the process in which the lidar acquires point cloud data. The azimuth may be an angle representing the scan direction on a horizontal plane relative to the lidar, and the elevation may be an angle representing the scan angle in a vertical direction relative to the lidar. These azimuth and elevation angles may be pre-set according to the lidar's scan structure or assigned to correspond to each point during the scanning process, and may be used as reference information indicating which direction in space each point was measured.

[0065] In one embodiment, the range image generation unit (220) can generate a plurality of range images such that one range image corresponds to each group.

[0066] Additionally, the range image generation unit (220) can generate multiple range images such that point cloud data classified into different groups among point cloud data corresponding to the same cell is mapped to different range images.

[0067] The array operation unit (230) can perform array operations based on distance values ​​between point cloud data corresponding to neighboring cells using a plurality of range images. The array operation can be performed based on a three-dimensional array structure including a plurality of range images.

[0068] In one embodiment, the array operation unit (230) can search for neighboring cells included in the remaining range images based on a reference cell included in the reference range image among a plurality of range images.

[0069] In one embodiment, each of the plurality of range images may include information regarding an azimuth angle, an elevation angle, and an image index. At this time, the array operation unit (230) may store the plurality of range images in a two-dimensional array form by removing information corresponding to the elevation angle of each cloud point data. Here, the azimuth angle, elevation angle, and image index included in the range image may be information for defining a coordinate system in which point cloud data is mapped on the range image.

[0070] The clustering unit (240) can perform clustering of multiple point cloud data based on array operations.

[0071] In one embodiment, the clustering unit (240) can compare a distance value calculated based on three-dimensional coordinate values ​​between reference point cloud data included in a reference cell and neighbor point cloud data included in a neighbor cell with a preset threshold. If the distance value exceeds the threshold, the clustering unit (240) can cluster the reference point cloud data and the neighbor point cloud data into different clusters.

[0072] FIG. 3 is a flowchart illustrating an exemplary LiDAR data clustering method based on range image multi-generation according to one embodiment. Here, the method illustrated in FIG. 3 may be executed by a LiDAR data clustering device (100). Furthermore, since the flowchart illustrated in FIG. 3 is merely exemplary, depending on the embodiment, each step may be executed in a different order than that described in the flowchart, or additional steps not described in the flowchart may be executed, or one or more of the steps described in the flowchart may not be executed.

[0073] As shown in FIG. 3, a LiDAR data clustering method based on multiple range image generation according to one embodiment comprises the steps of: classifying multiple point cloud data into multiple groups based on the distance relationship between multiple point cloud data mapped to one cell (S310); generating multiple range images corresponding to the multiple groups (S320); performing a distance value-based array operation between point cloud data corresponding to neighboring cells using the multiple range images (S330); and performing clustering of multiple point cloud data based on the array operation (S340).

[0074] Figure 4 illustrates, in an exemplary manner, the generation of a range image using azimuth and elevation information.

[0075] Referring to FIG. 4, point cloud data acquired from the lidar may include azimuth and elevation information in the lidar reference coordinate system. This azimuth and elevation information may be divided into predefined angle ranges according to the angle resolution of the lidar, and cell coordinates of the range image may be determined corresponding to each divided angle range.

[0076] Specifically, the azimuth of the point cloud data can be quantized to a first angle resolution and converted into a horizontal index of the range image, and the elevation can be quantized to a second angle resolution and converted into a vertical index of the range image. Accordingly, the range image can have a two-dimensional grid structure defined by the azimuth axis and the elevation axis. The aforementioned quantization of angles can be performed through the following formula.

[0077]

[0079] At this time, is the resolution for the azimuth (theta) direction, represents the resolution for the elevation (phi) direction.

[0080] Distance information of point cloud data corresponding to the azimuth and elevation ranges of the cell can be stored in each cell of the range image. In this case, one or more point cloud data may correspond to a single cell, and only some of the information among the corresponding point cloud data may be stored, or data selected according to specific criteria may be stored.

[0081] Figures 5 and 6 exemplarily illustrate the operation of generating clusters based on the distance relationship between a reference cell and neighbor cells in a single-range image.

[0082] Referring to FIGS. 5 and 6, clusters can be generated by comparing the distance relationship between point cloud data corresponding to a reference cell and an adjacent neighbor cell (neighbor cell), centered on a reference cell (current cell) included in a range image. In this case, the reference cell refers to a single cell selected on the range image, and the neighbor cells may include cells adjacent to the reference cell in the up-down, left-right, or diagonal directions.

[0083] The distance value between the point cloud data corresponding to a reference cell and the point cloud data corresponding to a neighboring cell can be calculated based on the 3D coordinate values ​​of each point cloud data. The calculated distance value can be compared with a preset threshold value, and if the distance value is less than or equal to the threshold value, the reference cell and the neighboring cell can be determined to belong to the same cluster. In this case, the threshold value (ε) can be determined according to the following mathematical formula.

[0084]

[0085] At this time, the resolution is as described above or and the threshold is or R represents the range of the reference cell data. That is, the threshold value can be determined based on the angular resolution and distance value used to generate the range image. For example, the threshold value can be set by considering the distance value of the point cloud data corresponding to the reference cell and the azimuth or elevation resolution of the sensor. Accordingly, even if the point cloud data has the same distance difference, different clustering results may be derived depending on the angular resolution.

[0086] In addition, as shown in Fig. 6, if the distance value between the point cloud data corresponding to the reference cell and the neighbor cell exceeds a threshold, the reference cell and the neighbor cell can be classified into different clusters.

[0087] FIG. 7 illustrates an exemplary range image generated according to one embodiment.

[0088] Referring to Fig. 7, three-dimensional point cloud data acquired from the lidar can be converted into a range image.

[0089] In Fig. 7(a), point cloud data acquired based on LiDAR is shown distributed in three-dimensional space, and each point can be displayed in a different color depending on the distance value (range) from the LiDAR.

[0090] Figure 7(b) illustrates a range image generated by arranging point cloud data in cell units based on azimuth and elevation information. The range image has a two-dimensional grid structure defined in the azimuth and elevation directions, and distance information of the point cloud data corresponding to that cell can be stored in each cell. At this time, each cell on the range image can be represented in a different color depending on the distance value of the point cloud data.

[0091] The illustrated range image is generated based on point cloud data acquired at a single point in time, and point cloud data corresponding to the same azimuth and elevation ranges can be mapped to the same cell. Accordingly, one or more point cloud data can correspond to a single cell, and distance information of some of this data or representative data can be stored in the cell.

[0092] FIGS. 8 and 9 exemplarily illustrate the limitations of a conventional LiDAR data clustering method.

[0093] Referring to Fig. 8(a), it can be seen that when a solid-state LiDAR is applied, the angular resolution of the sensor may be set differently depending on the spatial region. For example, the angular resolution in the azimuth or elevation direction may be applied differently depending on the region of interest and the region of non-interest, and accordingly, the spatial resolution of the cell may not be uniform even within the same range image. Due to these characteristics, when generating a range image based on a single fixed angular resolution, a phenomenon may occur in which multiple point cloud data are superimposed and mapped to a single cell in a specific region.

[0094] Additionally, referring to the range image shown in Fig. 8(b), a relatively high angular resolution is applied in the region of interest, so the point cloud data is represented densely, whereas in the region of non-interest, the point cloud data may be distributed sparsely due to the low angular resolution. Such a difference in resolution can cause an imbalance in the distribution of cell-unit data on the range image.

[0095] Referring to FIG. 9, it can be seen that multiple point cloud data are mapped to the same cell. In this case, in a conventional clustering method, a method may be applied in which only one of the point cloud data corresponding to a single cell is selected and used for cluster determination, and as a result, the remaining point cloud data corresponding to the same cell may be excluded during the cluster determination process. As illustrated in FIG. 9, in this processing method, only the point cloud data closest to the reference cell is used, and other point cloud data corresponding to the same cell may be excluded from the cluster determination target.

[0096] FIGS. 10 to 13 specifically illustrate a range image multiple generation-based LiDAR data clustering method according to one embodiment.

[0097] Referring to FIG. 10, point cloud data acquired from the lidar can be mapped to cell coordinates of a range image based on the sensor's azimuth and elevation information. In this case, the range image has a two-dimensional grid structure defined in the azimuth direction and the elevation direction, and point cloud data corresponding to the corresponding angle range can be mapped to each cell. One or more point cloud data can be mapped to the same cell.

[0098] Referring to FIG. 11, for multiple point cloud data mapped to the same cell, distance values ​​between each point cloud data can be compared. The distance values ​​can be calculated based on the three-dimensional coordinate values ​​of each point cloud data, and the calculated distance values ​​can be compared with a preset threshold value. At this time, the threshold value can be set based on the angular resolution used to generate the range image and the reference distance value. For example, if the distance value between data corresponding to a vehicle and data corresponding to a tree is determined to be greater than the threshold value, the two data can be classified into different groups.

[0099] Referring to FIG. 12, among multiple point cloud data mapped to the same cell, point cloud data with distance values ​​below a threshold can be classified into the same group, while point cloud data with distance values ​​exceeding the threshold can be classified into different groups. This group classification can be performed repeatedly within the same cell. At this time, multiple range images can be generated corresponding to the number of multiple groups. For example, as a result of comparing distance values ​​for a vehicle and a tree in FIG. 12, two range images can be generated.

[0100] Referring to FIG. 13, multiple range images can be generated corresponding to multiple groups classified within the same cell. For example, point cloud data belonging to the first group may be stored in the first range image, and point cloud data belonging to the second group may be stored in the second range image. The multiple range images generated in this way may have different image indices and can be used for array operations and clustering processing in subsequent steps.

[0101] FIGS. 14 and 19 exemplarily illustrate performing operations for clustering by optimizing a three-dimensional array into a two-dimensional array form.

[0102] Referring to FIG. 14, a process of generating clusters by searching for neighboring cells (neighboring cells) based on a reference cell (current cell) based on multiple range images is illustrated. In this case, the reference cell refers to a single cell included in the reference range image, and the neighboring cells may include cells included in the reference range image as well as in other additionally generated range images. Accordingly, the range of neighboring cells searched around the reference cell can be expanded according to the number of generated range images.

[0103] Referring to FIG. 15, differences in the operational structures performed according to the array dimension in range image-based clustering are illustrated. FIG. 15 (a) illustrates 2D array-based clustering using a single range image, and FIG. 15 (b) illustrates 3D array-based clustering using multiple range images.

[0104] When using a single range image, the range image has a two-dimensional array structure defined in the azimuth and elevation directions, and the clustering operation can be performed by searching for cells adjacent to a reference cell within the two-dimensional array. In this case, the search targets are limited to cells within the same range image, and the array index can be traversed only within the two-dimensional range.

[0105] On the other hand, when using multiple range images, the range images can be stacked in the direction of the image index to form a three-dimensional array structure. In this case, the search for neighboring cells centered on a reference cell can be performed including the image index direction as well as the azimuth and elevation directions. Accordingly, array operations may include searching for additional dimensions compared to when performed based on a two-dimensional array.

[0106] In other words, in 3D array-based clustering, not only cells at the same location as the reference cell but also cells included in different range images can be included in the search target, and accordingly, array index access and distance comparison operations may be repeated in proportion to the number of images. Due to this structure, array operations involve more cell search paths and comparison operations compared to 2D array-based operations, which can lead to inefficient operations.

[0107] Referring to FIG. 16, a plurality of range images can be represented as a three-dimensional data structure including axes corresponding to azimuth, elevation, and image index. FIG. 16 (a) illustrates such a three-dimensional range image structure, and each range image may be composed of cells according to the azimuth and elevation axes and may have a stacked form in the direction of the image index.

[0108] In this case, multiple point cloud data can be mapped to a single cell corresponding to the same azimuth and elevation angles. Such point cloud data can be stored in different range images corresponding to different image indices, and accordingly, data corresponding to a single cell location may exist distributed in the direction of the image index.

[0109] Figure 16(b) illustrates a process of removing one axis from a three-dimensional range image structure and storing it in the form of a two-dimensional range image. The axis to be removed may be one of an azimuth, an elevation, or an image index; for example, an axis corresponding to the elevation may be removed. Accordingly, each range image may be stored in the form of a two-dimensional array corresponding to an azimuth and an image index.

[0110] In this process, point cloud data corresponding to the same azimuth angle can be arranged in an aligned state, and distance values ​​between multiple point cloud data mapped to the same cell location can be compared. Depending on the result of the distance value comparison, the multiple point cloud data can be stored in different range images or in the same range image.

[0111] That is, by removing one of the axes constituting the 3D range image structure, point cloud data distributed across multiple range images can be reconstructed into a 2D array. This 2D array structure can then be used for cell-unit operations for clustering.

[0112] Referring to FIG. 17, a plurality of range images can be represented as a three-dimensional data structure including axes corresponding to azimuth, elevation, and image index, and as described above, by removing one axis from this three-dimensional data structure, the range images can be stored in a two-dimensional array form.

[0113] Figure 17 (a) illustrates a case where the axis corresponding to the azimuth (theta) is removed. In this case, multiple point cloud data may exist within a cell corresponding to the same elevation angle, and as the number of objects corresponding to a single cell increases, the number of data stored at the same cell location may increase. Accordingly, the number of range images generated to separate point cloud data corresponding to the same cell may increase.

[0114] Figure 17 (b) illustrates a case where the axis corresponding to the elevation angle (phi) is removed. In this case, point cloud data corresponding to the same azimuth angle can be separated and stored in different image index directions, and a range image can be generated depending on the number of point cloud data corresponding within a single cell.

[0115] Meanwhile, the criteria used for clustering between point cloud data can be calculated based on the distance values ​​of each point cloud data. The distance value includes a distance component from the LiDAR, and the spatial location information of the point cloud data may be reflected in the said distance component. For example, the distance value calculated based on the 3D coordinate values ​​of the point cloud data may include not only the azimuth component but also spatial location information corresponding to the elevation angle. Therefore, even if the axis corresponding to the elevation angle is removed from the 3D range image structure and the data is stored in the form of a 2D range image, the elevation angle information may be indirectly reflected in the clustering judgment performed through the comparison of distance values ​​between point cloud data. Accordingly, clustering judgment based on the distance relationship between point cloud data can be performed even when the axis corresponding to the elevation angle is removed.

[0116] Referring to FIG. 18, point cloud data acquired from the lidar can be represented as a three-dimensional range image structure including axes corresponding to azimuth, elevation, and image index, and it can be seen that the axis corresponding to elevation is removed from the three-dimensional range image structure and stored as a two-dimensional data structure corresponding to azimuth and image index.

[0117] In this case, point cloud data is not sorted according to elevation (phi) values, but can be sorted based solely on azimuth (theta) values. That is, point cloud data corresponding to the same azimuth can be sorted and stored in a single column direction.

[0118] Among point cloud data corresponding to the same azimuth angle, the distance difference between each data point can be calculated. The distance difference can be calculated using a distance value derived based on the 3D coordinate values ​​of each point cloud data point, and the calculated distance difference can be compared with a preset threshold value. In this case, the threshold value can be set based on the angular resolution used to generate the range image and a reference distance value.

[0119] As illustrated in FIG. 18, among point cloud data corresponding to the same azimuth angle, data with a distance difference below a threshold can be stored in the same range image, and data with a distance difference exceeding the threshold can be stored in different range images. Accordingly, multiple range images can be generated based on distance relationships even if they correspond to the same azimuth angle.

[0120] Referring to FIG. 19, a process of performing clustering using a range image stored in the form of a two-dimensional array is illustrated. Neighboring cells are searched based on a reference cell, and the distance value between the point cloud data contained in the reference cell and the point cloud data contained in the neighboring cells can be compared with a threshold value. Based on the comparison result, the point cloud data can be clustered into the same cluster or different clusters.

[0121] FIG. 20 exemplarily illustrates clustering results output according to a single-range image-based clustering method and a range image multi-generation-based LiDAR data clustering method according to one embodiment.

[0122] Referring to FIG. 20, the clustering result output according to a single-range image-based clustering method is shown on the left, and the clustering result output according to a multiple-range image generation-based clustering method using multiple range images is shown on the right.

[0123] Referring to the single-range image-based LiDAR data clustering result of FIG. 20 (a), the result of clustering performed based on a structure in which only one point cloud data is stored in each cell of the range image is illustrated. In this case, when multiple point cloud data are mapped to a single cell, only some of the point cloud data corresponding to that cell may be included in the clustering target. Accordingly, other point cloud data corresponding to the same cell may not be included in the clustering result.

[0124] In the result of LiDAR data clustering based on range image multi-generation according to an embodiment of the present invention of FIG. 20(b), the result of clustering is shown in which a plurality of point cloud data mapped to the same cell are separated and stored in different range images according to distance relationships. In this case, a plurality of point cloud data corresponding to the same cell may each be included in the clustering target, and each point cloud data may be assigned to a cluster based on distance relationships.

[0125] As shown in FIG. 20, it can be seen that even when the same scene is used as input, different clustering results are obtained depending on the difference in the range image generation method and the in-cell data processing method.

[0126] As described above, according to one embodiment of the present invention, point cloud data mapped and superimposed on the same cell can be utilized without loss, and accordingly, the clustering accuracy of the point cloud data can be improved. In addition, even when point cloud data corresponding to the same cell belong to different objects, the boundaries between objects can be distinguished more clearly.

[0127] Furthermore, the relationships between spatially adjacent point cloud data can be effectively reflected, and phenomena such as object shape distortion or separation can be reduced compared to conventional methods using single-range images. As a result, stable clustering results can be obtained even in complex environments.

[0128] In addition, by using an array operation method that removes information corresponding to elevation angles among the components of a range image and stores it in a 2D array form, the amount of computation and memory usage required for array operations can be reduced. Consequently, the applicability can be improved in autonomous driving environments or embedded systems that require real-time processing. Furthermore, clustering processing efficiency can be improved as unnecessary dimensional information is removed.

[0129] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

[0130] Combinations of each block of the block diagram attached to the present invention and each step of the flowchart may be performed by computer program instructions. Since these computer program instructions may be loaded into an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions performed through the encoding processor of the computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or each step of the flowchart. Since these computer program instructions may also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory may also produce a manufactured item containing instruction means for performing the function described in each block of the block diagram or each step of the flowchart. Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that execute a computer or other programmable data processing equipment by performing a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

[0131] Additionally, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). In some embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps depicted consecutively may actually be performed substantially simultaneously, or the blocks or steps may be performed in reverse order according to the corresponding function.

[0132] Meanwhile, according to one embodiment of the present invention, the various embodiments described above may be implemented as software comprising instructions stored on a computer-readable recording medium using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented as the processor itself. According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0133] Meanwhile, a computer or a similar device may include a device according to the disclosed embodiments, which is capable of calling instructions stored from a storage medium and operating according to the called instructions. When said instructions are executed by a processor, the processor may perform a function corresponding to said instructions directly or by using other components under the control of said processor. The instructions may include code generated or executed by a compiler or an interpreter.

[0134] A computer-readable recording medium may be provided in the form of a non-transitory computer-readable recording medium. Here, "non-transitory" simply means that the storage medium does not contain a signal and is tangible, without distinguishing whether data is stored semi-permanently or temporarily on the storage medium. In this context, a non-transitory computer-readable medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short moment, such as registers, caches, or memory. Specific examples of non-transitory computer-readable media may include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0135] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0136] 100: LiDAR data clustering device 150: LiDAR Data Clustering Program

Claims

Claim 1 A LiDAR data clustering method based on multiple range image generation performed by a LiDAR data clustering device, comprising: a step of classifying multiple point cloud data into multiple groups based on distance relationships between multiple point cloud data mapped to a single cell; a step of generating multiple range images corresponding to the multiple groups; a step of performing a distance value-based array operation between point cloud data corresponding to neighboring cells using the multiple range images; and a step of performing clustering for the multiple point cloud data based on the array operation, wherein the array operation is performed based on a three-dimensional array structure including the multiple range images, and in the step of performing the array operation, the neighboring cells included in the remaining range images are searched based on a reference cell included in a reference range image among the multiple range images. Claim 2 A range image multi-generation based LiDAR data clustering method according to claim 1, wherein the distance relationship includes a distance value calculated based on the three-dimensional coordinate value of each point cloud data. Claim 3 A range image multi-generation based LiDAR data clustering method according to claim 2, wherein in the step of classifying the data into multiple groups, the distance value is classified into different groups when it is greater than a preset threshold. Claim 4 A LiDAR data clustering method based on multiple range image generation according to claim 1, characterized in that one range image corresponds to each group of the plurality of range images. Claim 5 A LiDAR data clustering method based on multiple range image generation according to claim 1, wherein the step of generating the plurality of range images generates the plurality of range images such that point cloud data classified into different groups among point cloud data corresponding to the same cell is mapped to different range images. Claim 6 delete Claim 7 delete Claim 8 A range image multi-generation based LiDAR data clustering method according to claim 1, wherein in the step of performing the clustering, a distance value calculated based on three-dimensional coordinate values ​​between reference point cloud data included in the reference cell and neighbor point cloud data included in the neighbor cell is compared with a preset threshold. Claim 9 In claim 8, the range image multi-generation-based LiDAR data clustering method wherein, in the step of performing the clustering, if the distance value exceeds the threshold, the reference point cloud data and the neighbor point cloud data are clustered into different clusters. Claim 10 A LiDAR data clustering method based on multiple range image generation according to claim 1, wherein each of the plurality of range images includes information regarding an azimuth angle, an elevation angle, and an image index, and in the step of performing the array operation, information corresponding to the elevation angle of each cloud point data is removed and the plurality of range images are stored in the form of a two-dimensional array. Claim 11 A LiDAR data clustering device comprising: a memory storing at least one instruction; and a processor, wherein the at least one instruction is executed by the processor, the device classifies the plurality of point cloud data into a plurality of groups based on the distance relationship between the plurality of point cloud data mapped to a single cell, generates a plurality of range images corresponding to the plurality of groups, performs a distance value-based array operation between point cloud data corresponding to neighboring cells using the plurality of range images, and performs clustering of the plurality of point cloud data based on the array operation, wherein the array operation is performed based on a three-dimensional array structure including the plurality of range images, and the processor performs the array operation by searching for the neighboring cells included in the remaining range images based on a reference cell included in a reference range image among the plurality of range images. Claim 12 A computer-readable recording medium storing at least one computer-executable instruction, wherein the at least one instruction, when executed by a processor, comprises: a step of classifying the plurality of point cloud data into a plurality of groups based on a distance relationship between the plurality of point cloud data mapped to a cell; a step of generating a plurality of range images corresponding to the plurality of groups; a step of performing a distance value-based array operation between point cloud data corresponding to neighboring cells using the plurality of range images; and a step of performing clustering of the plurality of point cloud data based on the array operation, wherein the array operation is performed based on a three-dimensional array structure including the plurality of range images, and in the step of performing the array operation, the processor performs a method of searching for the neighboring cells included in the remaining range images based on a reference cell included in a reference range image among the plurality of range images. Claim 13 A computer program stored on a computer-readable recording medium, wherein the computer program, when executed by a processor, comprises: a step of classifying the plurality of point cloud data into a plurality of groups based on a distance relationship between the plurality of point cloud data mapped to a single cell; a step of generating a plurality of range images corresponding to the plurality of groups; a step of performing a distance value-based array operation between point cloud data corresponding to neighboring cells using the plurality of range images; and a step of performing clustering of the plurality of point cloud data based on the array operation, wherein the array operation is performed based on a three-dimensional array structure including the plurality of range images, and wherein the step of performing the array operation includes instructions for the processor to perform a method of searching for the neighboring cells included in the remaining range images based on a reference cell included in a reference range image among the plurality of range images.

Images