Method and apparatus for processing 3D data by using point cloud data

Abstract

A method by which an apparatus uses a point cloud to process 3D data, according to one embodiment of the present invention, may comprise the operations of: detecting point cloud data from a sensor; determining whether points of the point cloud data are located within a designated distance from the sensor; sampling the point cloud data according to the determination result; and outputting the sampled point cloud data. Other various embodiments are possible.

Description

Method and device for processing 3D data using point cloud data

[0001] The present invention relates to a method and device for sampling point cloud data and processing the data into 3D data.

[0002] This application claims priority to Korean Patent Application No. 10-2024-0052189, filed on April 18, 2024, the entire contents of which are disclosed in the specification and drawings of the said application are incorporated herein by reference.

[0003] Meanwhile, the present invention was supported by the following national research and development project.

[0004] Assignment ID: 2710011453

[0005] Assignment Number: 0536-20240034

[0006] Ministry of Science and ICT

[0007] Project Management Agency Name: National Research Foundation of Korea

[0008] Research Project Name: Basic Research Project in Science and Engineering / Mid-career Researcher Support Project

[0009] Research Project Name: Hyper-Realistic Permanent Hybrid Telepresence Platform

[0010] Project Performing Institution Name: Seoul National University

[0011] Research period: March 1, 2024 - February 28, 2025

[0012] Assignment ID: 2710006536

[0013] Assignment Number: 0536-20240107

[0014] Ministry of Science and ICT

[0015] Project Management Agency Name: National Research Foundation of Korea

[0016] Research Project Name: Source Technology Development Project

[0017] Research Project Name: High-Precision, High-Reliability Teleinteraction Platform

[0018] Project Performing Institution Name: Seoul National University

[0019] Research Period: July 1, 2024 - December 31, 2024

[0020] Point cloud data detected by a light detection and ranging (LiDAR) sensor can be used to represent 3D data. In particular, in recently emerging technologies such as autonomous driving, drones, and unmanned robots, 3D data can be acquired in real time, converted into point cloud data, and objects detected using this converted point cloud data.

[0021] Deep learning techniques can typically be used to detect objects in point cloud data in three-dimensional space. However, most deep learning techniques for object detection in point cloud data require deep learning operations for each point contained in the point cloud data, resulting in long delays.

[0022] Therefore, to overcome the drawback of long delays, a subset of points can be selected from point cloud data and deep learning operations can be performed on these selected points. However, selecting a subset of points and performing deep learning operations on them requires time for point selection, and object detection performance and accuracy deteriorate as these points are used for object detection.

[0023] The technical problem to be solved by the present invention is to provide a method for processing 3D data using point cloud data, which detects point cloud data and performs resolution-based sampling or boundary-based sampling on the detected point cloud data based on the distance between points of the point cloud data and a sensor, and a device therefor.

[0024] The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0025] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention for achieving the above technical task may include an operation of detecting point cloud data from a sensor, an operation of determining whether points of the point cloud data are located within a specified distance from the sensor, an operation of sampling the point cloud data based on a result of the determination, and an operation of outputting the sampled point cloud data.

[0026] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of performing resolution-based sampling on the point cloud data when, as a result of the determination, the points are located within a specified distance from the sensor.

[0027] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of selecting a plurality of first points located within a specified distance value from the sensor among points of the point cloud data, and an operation of converting coordinates of the plurality of selected first points into a spherical coordinate system.

[0028] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of combining the converted points into bin units of specified pixel values ​​and an operation of checking points corresponding to each bin.

[0029] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of removing points of each identified bin except for the point located closest to the sensor.

[0030] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of converting each bin from which points have been removed into point cloud data.

[0031] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of performing boundary-based sampling on the point cloud data when, as a result of the determination, the points are not located within a specified distance from the sensor.

[0032] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of projecting points of the point cloud data into a distance image and an operation of calculating curvature values ​​for a plurality of pixels of the distance image.

[0033] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of identifying pixels that exhibit a curvature value greater than a specific value among the calculated curvature values, and an operation of identifying the identified pixels as a boundary line.

[0034] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of confirming the object direction of the object with respect to the confirmed boundary line and an operation of removing points of the distance image, excluding points having values ​​within a range confirmed as the object direction.

[0035] A method for processing 3D data using a point cloud in a device according to one embodiment of the present invention may further include an operation of converting a distance image from which points have been removed into point cloud data.

[0036] A device for processing 3D data using a point cloud according to one embodiment of the present invention may include a sensor for detecting point cloud data, a resolution-based sampler for performing resolution-based sampling on the point cloud data, a boundary-based sampler for performing boundary-based sampling on the point cloud data, a display for outputting point cloud data on which the resolution-based sampling or the boundary-based sampling has been performed, and a processor for determining whether points of the detected point cloud data are located within a specified distance from the sensor, controlling the resolution-based sampler to perform the resolution-based sampling if it is determined that the points are located within the specified distance from the sensor, and controlling the boundary-based sampler to perform the boundary-based sampling if it is determined that the points are not located within the specified distance from the sensor.

[0037] According to the present invention as described above, when sampling point cloud data, resolution-based sampling or boundary-based sampling can be performed in a complex manner depending on the distance between the sensor and the object.

[0038] In addition, the present invention has the effect of reducing the sampling delay speed by removing 80% of unnecessary points in point cloud data, such as points far from the boundary line or points that are excessively dense, when detecting the object, and performing resolution-based sampling or boundary-based sampling on the point cloud data from which the points have been removed.

[0039] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0040] FIG. 1 is a block diagram illustrating a configuration of a device for processing 3D data using a point cloud according to one embodiment of the present invention.

[0041] FIG. 2 is a flowchart illustrating an operation of processing 3D data using a point cloud according to one embodiment of the present invention.

[0042] FIG. 3 is a flowchart illustrating an operation of performing resolution-based sampling in a device according to one embodiment of the present invention.

[0043] FIG. 4 is a flowchart illustrating an operation of performing boundary-based sampling in a device according to one embodiment of the present invention.

[0044] FIG. 5 is a diagram illustrating a distance image configuration according to one embodiment of the present invention.

[0045] FIG. 6 is a diagram illustrating an example of point cloud data according to one embodiment of the present invention.

[0046] FIG. 7 is a graph showing the results of performing resolution-based sampling and boundary-based sampling on point cloud data in which points are located within a specified distance, according to one embodiment of the present invention.

[0047] FIG. 8 is a graph showing the results of performing resolution-based sampling or boundary-based sampling on point cloud data of an object according to the distance at which points are located, according to one embodiment of the present invention.

[0048] The purpose, technical configuration, and resulting operational effects of the present invention will be more clearly understood through the following detailed description based on the drawings attached to the specification of the present invention. Reference will now be made to the accompanying drawings, which will further describe embodiments of the present invention.

[0049] The embodiments disclosed herein should not be construed or used to limit the scope of the present invention. Those skilled in the art will readily appreciate that the descriptions of the embodiments herein, including the embodiments, have a wide range of applications. Therefore, any embodiments described in the detailed description of the present invention are intended to serve as illustrative examples to better illustrate the present invention and are not intended to limit the scope of the present invention to the embodiments.

[0050] The functional blocks depicted in the drawings and described below are merely examples of possible implementations. Other implementations may utilize other functional blocks without departing from the spirit and scope of the detailed description. Furthermore, while one or more functional blocks of the present invention are depicted as individual blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

[0051] Additionally, the expression "including certain components" is an "open" expression, simply indicating the presence of those components, and should not be construed as excluding additional components.

[0052] Furthermore, when it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

[0053] Hereinafter, detailed embodiments of the present invention will be described with reference to the drawings.

[0054] FIG. 1 is a block diagram illustrating a configuration of a device for processing 3D data using a point cloud according to one embodiment of the present invention.

[0055] Referring to FIG. 1, the device (100) may include a sensor (110), a multiplexer (120), a resolution-based sampler (130), a boundary-based sampler (140), and a display (150). In addition, the device (100) may include a processor (not shown), and may perform an operation of sampling and outputting point cloud data in the overall device (100) by the processor.

[0056] According to one embodiment of the present invention, the sensor (110) is a lidar sensor that can detect point cloud data for an outdoor scene (e.g., a road condition) or a specific object.

[0057] According to one embodiment of the present invention, the sensor (110) can output a laser beam and measure the output speed of the laser beam and the speed of the laser beam reflected from the object based on the output. Thereafter, the multiplexer (120) can measure the time from when the laser beam is output to when it is reflected based on the measured speeds.

[0058] According to one embodiment of the present invention, the multiplexer (120) can check the distance values ​​(range values) from the sensor (110) at various locations in the three-dimensional space based on the measured time, and check the point cloud data including points corresponding to the checked distance values.

[0059] According to one embodiment of the present invention, the processor determines whether the points are located within a specified distance (e.g., 20 to 40 m) from the sensor (110) based on the point cloud data, and controls resolution-based sampling or edge-based sampling to be performed on the detected point cloud data based on the determination result.

[0060] According to one embodiment of the present invention, if the processor determines that the points are located within a specified distance from the sensor (110) as a result of the determination, the processor may transmit the detected cloud data to a resolution-based sampler (130) so that resolution-based sampling may be performed. In addition, if the processor determines that the points are not located within a specified distance from the sensor (110), the processor may transmit the detected point cloud data to a boundary-based sampler (140) so that boundary-based sampling may be performed.

[0061] According to one embodiment of the present invention, the resolution-based sampler (130) may include a horizontal angle calculator (131) and a per angle sampler (132).

[0062] According to one embodiment of the present invention, the horizontal angle calculator (131) can select points within a specified distance value (e.g., 20 m) from among points in the point cloud data. Thereafter, the horizontal angle calculator (131) can determine an object located within a specified distance (20 m) from the sensor (110) as an object close to the sensor (110).

[0063] According to one embodiment of the present invention, the close object may have a larger horizontal angular range relative to the sensor (110) than an object farther from the sensor (110). Accordingly, the horizontal angle calculator (131) may detect the object using the points in the point cloud data for the close object as the number of points increases compared to the far object.

[0064] According to one embodiment of the present invention, the angle unit sampler (132) can convert the Cartesian coordinates (or Cartesian coordinates) of points selected from the horizontal angle calculator (131) into a spherical coordinate system. For example, the angle unit sampler (132) can confirm the coordinate values ​​of (x, y, z) as coordinate values ​​of (r, θ, Ø) by converting the coordinate system of the selected points into a spherical coordinate system.

[0065] According to one embodiment of the present invention, the resolution-based sampler (130) can combine points on the transformed spherical coordinate system into points in bin units of a specified pixel value (e.g., 50 pixels) and identify points corresponding to each of a plurality of bins.

[0066] According to one embodiment of the present invention, the resolution-based sampler (130) can identify bins having θ values ​​in the same range as the same bin. For example, the resolution-based sampler (130) can divide θ values ​​from 0° to 360° into several ranges. Thereafter, the resolution-based sampler (130) can determine bins having θ values ​​in a specific range (e.g., from 30° to 45°) as the same bin.

[0067] According to one embodiment of the present invention, the resolution-based sampler (130) can set the sampling degree by setting the bin size. For example, the resolution-based sampler (130) can set the bin size by changing the θ value.

[0068] According to one embodiment of the present invention, the resolution-based sampler (130) may set the number of the plurality of bins to less than 2650. At this time, the resolution-based sampler (130) may combine the points by adding points corresponding to the plurality of bins concurrently.

[0069] According to one embodiment of the present invention, the resolution-based sampler (130) can remove all points except one point from each bin. Thereafter, the resolution-based sampler (130) can convert the bins from which the points have been removed into point cloud data and transmit it to the display (150).

[0070] According to one embodiment of the present invention, a boundary-based sampler (140) may include a range image projector (141) and an edge detector (142).

[0071] According to one embodiment of the present invention, the range image projector (141) can project point cloud data into a range image in which the horizontal and vertical axes represent vertical and horizontal angles, respectively, and the value of each pixel represents the distance from each point to the sensor (110). For example, the projected range image may have a grid structure in which the positions of points are shifted from the point cloud data.

[0072] According to one embodiment of the present invention, the boundary detection unit (142) can calculate a curvature value for each pixel of the distance image.

[0073] According to one embodiment of the present invention, the point cloud data for which the boundary-based sampling is performed relates to a distant object, and the points of the point cloud data of the distant object can clearly show a distance difference between the points corresponding to the background and the points of the object that is greater than a specified value. Accordingly, the boundary detection unit (142) can calculate a curvature value from a distance image for the object based on the characteristics of the distant object that show a distance difference greater than a specified value.

[0074] Mathematical expression 1 is an expression for calculating a curvature value according to one embodiment of the present invention.

[0075]

[0076] Here, i and j are the horizontal and vertical indices, respectively, C is the curvature value, R is the number of pixels, and X i.j is the distance value of the pixel at position (ij).

[0077] According to one embodiment of the present invention, the boundary detection unit (142) can identify pixels in which the calculated curvature value exceeds a certain threshold value as boundaries. For example, the boundary detection unit (142) can set the threshold value low or high to set the degree of boundary-based sampling.

[0078] According to one embodiment of the present invention, the boundary detection unit (142) can identify the object direction for each boundary line.

[0079] According to one embodiment of the present invention, a boundary-based sampler (140) can remove points of point cloud data except for points having values ​​within a range specified in the object direction of each boundary line.

[0080] According to one embodiment of the present invention, the boundary-based sampler (140) can convert the distance image from which the points have been removed into point cloud data and transmit it to the display (150).

[0081] According to one embodiment of the present invention, the display (150) can receive point cloud data converted as resolution-based sampling or boundary-based sampling is performed from a resolution-based sampler (130) or a boundary-based sampler (140), and output the transmitted point cloud data.

[0082] FIG. 2 is a flowchart illustrating an operation of processing 3D data using a point cloud according to one embodiment of the present invention.

[0083] Referring to FIG. 2, in operation S110, the device (100) can detect point cloud data for an object from a sensor (110).

[0084] In operation S120, the device (100) can determine whether points of the point cloud data are located within a specified distance (e.g., 20 to 40 m) from the sensor (110) based on the point cloud data.

[0085] As a result of performing the above-described S120 operation, if it is determined that the points are located within a specified distance from the sensor (110), the device (100) can perform resolution-based sampling on the detected point cloud data in operation S130.

[0086] As a result of performing the above-described S120 operation, if it is determined that the points are not located within a specified distance from the sensor (110), the device (100) can perform boundary-based sampling on the detected point cloud data in operation S140.

[0087] As the aforementioned S130 operation or S140 operation is performed, the device (100) can output the sampled point cloud data through the display (150) in the S150 operation.

[0088] FIG. 3 is a flowchart illustrating an operation of performing resolution-based sampling in a device according to one embodiment of the present invention.

[0089] Referring to FIG. 3, the operation S130 for performing resolution-based sampling of FIG. 2 may include an operation for selecting points (S131), an operation for converting the selected points into a spherical coordinate system (S132), an operation for combining points into bin units (S133), and an operation for removing points for each bin (S134).

[0090] In operation S131, the device (100) can select points within a specified distance value (e.g., 20 m) among points of point cloud data.

[0091] According to one embodiment of the present invention, an object located within a specified distance (20 m) from a sensor (110) may be determined as an object close to the sensor (110). At this time, the close object may have a larger horizontal angular range with respect to the sensor (110) than an object far from the sensor (110). Accordingly, as the number of points in the point cloud data for a close object increases compared to a far object, the object can be detected using the points.

[0092] In operation S132, the device (100) can convert the Cartesian coordinates (or Cartesian coordinates) of the selected points into a spherical coordinate system. For example, by converting the coordinate system of the selected points into a spherical coordinate system, the device (100) can confirm the coordinate values ​​of (x, y, z) as the coordinate values ​​of (r, θ, Ø).

[0093] In operation S133, the device (100) can combine points on the transformed spherical coordinate system into points in bin units of a specified pixel value (e.g., 50 pixels), and check the points corresponding to each of a plurality of bins. For example, the plurality of bins can be determined to be the same bin if they have θ values ​​within a specified range.

[0094] According to one embodiment of the present invention, the device (100) can set the number of the plurality of bins to less than 2650. At this time, the device (100) can combine the points by adding the points corresponding to the plurality of bins concurrently.

[0095] In operation S134, the device (100) can remove all points except one point from each bin. Here, there are three ways to select a point: selecting the farthest point, selecting the closest point, and selecting randomly. Experimentally, the method of selecting the closest point shows the best performance, and the method of selecting randomly is the best in terms of computational speed. However, in this method, although the size of the distant points themselves becomes smaller, the area corresponding to each bin becomes wider, so when this method is applied, points with high importance may be lost. Therefore, a separate step of selecting points within a specific distance is required. Therefore, it is preferable to select the closest point.

[0096] Thereafter, the device (100) can output point cloud data consisting of bins from which the points have been removed.

[0097] FIG. 4 is a flowchart illustrating an operation of performing boundary-based sampling in a device according to one embodiment of the present invention.

[0098] Referring to FIG. 4, the operation S140 for performing boundary-based sampling of FIG. 2 may include an operation for projecting points into a distance image (S141), an operation for calculating curvature (S142), an operation for checking a boundary (S143), an operation for checking an object direction (S144), an operation for removing points (S145), and an operation for converting into cloud data (S146).

[0099] In operation S141, the device (100) can project points of point cloud data into a distance image.

[0100] According to one embodiment of the present invention, the distance image may be a data format in which the horizontal and vertical axes represent vertical and horizontal angles, respectively, and the value of each pixel represents the distance between each point of the point cloud data and the sensor (110). The distance image may be a grid structure in which the positions of the points are shifted from the point cloud data.

[0101] In operation S142, the device (100) can calculate a curvature value for each pixel of the distance image.

[0102] According to one embodiment of the present invention, the point cloud data for which the boundary-based sampling is performed relates to distant points, and the points of the distant point cloud data can clearly show a distance difference between points corresponding to the background and points corresponding to an object exceeding a specified value. Accordingly, the device (100) can calculate a curvature value from a distance image for the points based on the characteristics of the distant points for which a distance difference exceeding the specified value is shown.

[0103] In operation S143, the device (100) can identify pixels in which the calculated curvature value is greater than a specific value as boundaries.

[0104] In operation S144, the device (100) can check the object direction for each boundary line.

[0105] In operation S145, the device (100) can remove points of the point cloud data except for the points having values ​​within a range specified in the object direction of each boundary line.

[0106] In operation S146, the device (100) can convert the distance image into point cloud data.

[0107] FIG. 5 is a diagram illustrating a distance image configuration according to one embodiment of the present invention.

[0108] According to one embodiment of the present invention, the device (100) can detect first point cloud data of a first object located within a specified distance from the sensor (110) using the sensor (110), and can detect second point cloud data of a second object located outside the specified distance from the sensor (110). Thereafter, the device (100) can convert the detected first and second point clouds into the form of a first distance image (510) and a second distance image (520).

[0109] Referring to FIG. 5, the first and second distance images (510, 520) may include points representing distance values ​​of objects (511, 521) and backgrounds (512, 522) from the sensor (110), respectively. For example, the device (100) may convert the first and second distance images (510, 520) so that the points are expressed in black when the distance value is smaller or in white when the distance value is larger, corresponding to the distance values ​​of objects (511, 521) and backgrounds (512, 522) from the sensor (110).

[0110] According to one embodiment of the present invention, when the device (100) acquires a first distance image (510) in which the difference between the object (511) and the background (512) is less than or equal to a specified value, the device (100) may perform resolution-based sampling on the first point cloud data of the first distance image (510). For example, in the first distance image (510), since the difference between the object (511) and the background (512) is less than or equal to the specified value, the black and white contrast of points represented by the object (511) and the background (512) may appear similar.

[0111] A resolution-based sampling operation according to one embodiment of the present invention may include an operation of selecting points of first point cloud data, an operation of converting the selected points into a spherical coordinate system, an operation of combining the points into bin units, an operation of removing points for each bin, and an operation of converting bins from which the points have been removed into point cloud data.

[0112] According to one embodiment of the present invention, when the device (100) acquires a second distance image (520) in which the difference between the object (521) and the background (522) exceeds a specified value, the device (100) may perform boundary-based sampling on the second point cloud data of the second distance image (520). For example, in the second distance image (520), since the difference between the object (521) and the background (522) exceeds the specified value, the black and white contrast of points expressed as the object (521) and the background (522) is clear, so that the device (100) may detect the boundary between the object (521) and the background (522).

[0113] A boundary-based sampling operation according to one embodiment of the present invention may include an operation of projecting points of second point cloud data into a distance image, an operation of calculating curvature for each pixel of the distance image, an operation of identifying a boundary line by pixels having a curvature value corresponding to the specified value, an operation of identifying an object direction of each pixel of the boundary line, an operation of removing points other than those having values ​​within a range specified as an object direction among the points, and an operation of converting the distance image from which the points have been removed into cloud data.

[0114] FIG. 6 is a diagram illustrating an example of point cloud data according to one embodiment of the present invention.

[0115] According to one embodiment of the present invention, the device (100) can detect point cloud data (a) from a sensor (110), and perform resolution-based sampling or boundary-based sampling on the detected point cloud data to obtain point cloud data (b).

[0116] Referring to FIG. 6, the device (100) performs resolution-based sampling or boundary-based sampling according to the distance between the sensor (110) and the object, thereby obtaining point cloud data (b) from which 80% of the points are removed compared to the original point cloud data (a).

[0117] According to one embodiment of the present invention, the device (100) performs resolution-based sampling or boundary-based sampling according to the distance between the sensor (110) and the object, thereby processing 3D data as point cloud data (b) from which 80% of points are removed from the point cloud data (a), thereby enabling processing of 3D data with a significantly reduced delay time compared to the original point cloud data (a).

[0118] FIG. 7 is a graph showing the results of performing resolution-based sampling and boundary-based sampling on point cloud data in which points are located within a specified distance, according to one embodiment of the present invention.

[0119] Referring to Figure 7, the graph represents the performance of object detection according to the compression ratio (space saving) of sampled point cloud data as an F1 score. For example, the F1 score can be used to measure the performance of object detection for each of edge-based sampling (Edge), resolution-based sampling (Reso), and Pareto optimal sampling.

[0120] According to one embodiment of the present invention, Pareto optimal sampling may perform resolution-based sampling for points of point cloud data located within 20 m from the sensor (110), and boundary-based sampling for points located 40 m or more away from the sensor (110).

[0121] According to one embodiment of the present invention, the F1 score can be calculated using the Waymo dataset to analyze the latency and accuracy of a deep learning model. Furthermore, the F1 score can be calculated using a point-based region-based convolutional neural network (RCNN) model or a PointPillar model.

[0122] According to one embodiment of the present invention, as a result of performing Pareto optimal sampling, 80% of unnecessary points can be removed from most point cloud data without any decrease in accuracy compared to performing resolution-based sampling or boundary-based sampling, respectively.

[0123] FIG. 8 is a graph showing the results of performing resolution-based sampling or boundary-based sampling on point cloud data of an object according to the distance at which points are located, according to one embodiment of the present invention.

[0124] Referring to Figure 8, the graph shows the results of performing resolution-based sampling (Reso) or edge-based sampling (Edge) on point cloud data as an F1 score.

[0125] According to one embodiment of the present invention, the device (100) can perform resolution-based sampling on points located within 20 m from the sensor (110) among points of point cloud data, and perform boundary-based sampling on points located outside 40 m.

[0126] According to one embodiment of the present invention, even if the device (100) compresses point cloud data with a compression ratio (space saving) of 90%, which removes most of the points of the point cloud data by performing resolution-based sampling or boundary-based sampling, the performance of object detection can be calculated as an F1 score of 90% compared to the F1 score when the compression ratio is 0%.

[0127] According to one embodiment of the present invention, the device (100) can reduce the number of points to be processed by removing 80% of unnecessary points of point cloud data by performing resolution-based sampling or boundary-based sampling. Accordingly, a latency reduction of 65.7% can be achieved in an RTX 3090 GPU server environment and 49.3% in a Jetson AGX Xavier environment.

[0128] Although embodiments of the present invention have been described with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

[0129] 100: Device

[0130] 110: Sensor

[0131] 120: Multiplexer

[0132] 130: Resolution-based sampler

[0133] 131: Horizontal Angle Calculator

[0134] 132: Angle Unit Sampler

[0135] 140: Boundary-based sampler

[0136] 141: Street Image Projector

[0137] 142: Boundary detection unit

[0138] 150: Display

Claims

1. In a method for processing 3D data using a point cloud, The act of detecting point cloud data from a sensor; An operation for determining whether points of the above point cloud data are located within a specified distance from the sensor; According to the above judgment result, an operation of sampling the point cloud data; and An operation of outputting the sampled point cloud data; A method comprising:

2. In paragraph 1, As a result of the above determination, if the points are located within a specified distance from the sensor, an operation of performing resolution-based sampling on the point cloud data; A method further comprising:

3. In paragraph 2, An operation of selecting a plurality of first points located within a specified distance value from the sensor among the points of the point cloud data; and An operation of converting the coordinates of the plurality of first points selected above into a spherical coordinate system; A method further comprising:

4. In paragraph 3, An operation of combining the above-mentioned converted points into bin units of specified pixel values; and The action of checking the points corresponding to each bin; A method further comprising:

5. In paragraph 4, An operation of removing all points of each of the above-mentioned bins except the point located closest to the sensor; A method further comprising:

6. In paragraph 5, An operation to convert each bin from which the above points have been removed into point cloud data; A method further comprising:

7. In paragraph 1, As a result of the above judgment, if the points are not located within a specified distance from the sensor, an operation of performing boundary-based sampling on the point cloud data; A method further comprising:

8. In paragraph 7, An operation of projecting points of the above point cloud data into a distance image; and An operation of calculating curvature values ​​for a plurality of pixels of the above distance image; A method further comprising:

9. In paragraph 8, An operation of checking pixels that exhibit a curvature value greater than a specific value among the calculated curvature values; and An action to confirm the above-mentioned pixels as boundaries; A method further comprising:

10. In paragraph 9, An operation of confirming the object direction of the object with respect to the above-mentioned confirmed boundary line; and An operation of removing points of the above distance image, excluding points with values ​​within the range identified in the direction of the object; A method further comprising:

11. In paragraph 10, An operation of converting a distance image from which the above points have been removed into point cloud data; A method further comprising:

12. In a device that processes 3D data using a point cloud, A sensor that detects point cloud data; A resolution-based sampler that performs resolution-based sampling on the above point cloud data; A boundary-based sampler that performs boundary-based sampling on the above point cloud data; A display that outputs point cloud data on which the resolution-based sampling or the boundary-based sampling has been performed; and Determine whether the points of the above-detected point cloud data are located within a specified distance from the sensor, If it is determined that the above points are located within a specified distance from the sensor, the resolution-based sampler is controlled so that the resolution-based sampling is performed, A processor controlling the boundary-based sampler so that boundary-based sampling is performed when it is determined that the points are not located within a specified distance from the sensor; A device comprising:

Images