Method and apparatus for point cloud segmentation

Abstract

The present invention provides a method and apparatus for point cloud segmentation. The method may include and the apparatus may be used to: receive a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight; map the first frame of point cloud into a grid including a plurality of cells; perform weighted averaging on the respective points in each cell of the grid into which the first frame of point cloud is mapped to obtain a first weighted point of the cell; and use the first weighted points in each cell of the grid to perform point cloud segmentation. By adopting the method and apparatus disclosed in the present invention, the processing efficiency of point cloud data can be significantly improved and processing resources can be saved.

Description

Technical Field

[0001] The present invention relates to the processing of point cloud data, and more particularly, to methods and apparatuses for point cloud segmentation. Background Art

[0002] For a long time, autonomous driving has been the subject of research efforts aimed at improving the safety and efficiency of automotive transportation. In recent years, increasingly sophisticated sensors have brought autonomous driving systems closer to reality. For example, 3D scanners (such as lidar, stereo cameras, time-of-flight cameras, etc.) are now widely used in autonomous driving systems. Such 3D scanners measure a large number of points on the surface of an object and often output a point cloud as a data file. A point cloud represents a set of points measured by a 3D scanner. As is known in the art, point clouds can be used for many purposes, including creating 3D maps, object recognition, and object tracking, etc. In order to identify and track objects of interest, it is necessary to segment the acquired point cloud. Generally speaking, the segmentation of point cloud data is a process of dividing the disordered point cloud data into several non-overlapping subsets. After segmentation, points with similar attributes are grouped into one class, so that a series of objects of interest, such as vehicles, streets, trees, etc., can be obtained.

[0003] Generally, 3D scanners such as lidar scan the surrounding objects at a rate of dozens to hundreds of frames per second. When an autonomous vehicle is driving on the road, continuous point cloud frames are obtained by the lidar installed on the vehicle to learn about the vehicle's surrounding environment in real time. Traditionally, segmentation is directly performed on each frame of the point cloud. However, since each frame of the point cloud may contain thousands of points, this approach requires a large amount of processing resources.

[0004] Therefore, there is a need for improved methods and apparatuses for point cloud segmentation. Summary of the Invention

[0005] The Summary of the Invention is provided to introduce in a simplified form some concepts that will be further described in the following Detailed Description. The Summary of the Invention is not intended to identify the key features or essential features of the claimed subject matter, nor is it intended to be used to help determine the scope of the claimed subject matter.

[0006] According to an embodiment of the present invention, there is provided a method for point cloud segmentation, the method may include: receiving a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight; mapping the first frame of point cloud into a grid including a plurality of cells; performing a weighted average on each point mapped into each cell of the grid in the first frame of point cloud to obtain a first weighted point of the cell; and using the first weighted points of each cell of the grid to perform point cloud segmentation.

[0007] According to an embodiment of the present invention, a device for point cloud segmentation is provided. The device may include a receiving unit configured to receive a first frame of point cloud from a sensor, where each point in the first frame of point cloud has an associated weight; a mapping unit configured to map the first frame of point cloud into a grid including a plurality of cells; a calculating unit configured to perform a weighted average on each point in the first frame of point cloud that is mapped into each cell of the grid to obtain a first weighted point of the cell; and a point cloud segmentation unit configured to perform point cloud segmentation using the first weighted points in each cell of the grid.

[0008] According to an embodiment of the present invention, a device for point cloud segmentation is provided. The device may include a memory storing a computer program; and a processor coupled to the memory, where the computer program, when executed by the processor, implements the following steps: receiving a first frame of point cloud from a sensor, where each point in the first frame of point cloud has an associated weight; mapping the first frame of point cloud into a grid including a plurality of cells; performing a weighted average on each point in the first frame of point cloud that is mapped into each cell of the grid to obtain a first weighted point of the cell; and performing point cloud segmentation using the first weighted points in each cell of the grid.

[0009] According to an embodiment of the present invention, a vehicle is provided, which includes: a sensor; and a device for point cloud segmentation according to the present invention.

[0010] According to an embodiment of the present invention, a non-transitory computer-readable medium is provided, which stores a computer program that, when executed by a processor, executes a method for point cloud segmentation according to the present invention.

[0011] By adopting the methods and devices disclosed in the present invention, the processing efficiency of point cloud data can be significantly improved and processing resources can be saved.

[0012] By reading the following detailed description and referring to the associated drawings, these and other features and advantages will become apparent. It should be understood that the foregoing general description and the following detailed description are illustrative only and do not limit the aspects claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to understand in detail the manner in which the above-described features of the present invention are used, reference may be made to the embodiments to describe more specifically the content briefly outlined above, some aspects of which are shown in the drawings. However, it should be noted that the drawings only show some typical aspects of the present invention and should not be considered to limit its scope, as the description may allow other equally effective aspects.

[0014] Figure 1Shows a schematic diagram of an exemplary road environment.

[0015] Figure 2 Shows a schematic diagram of the point cloud of an exemplary road environment obtained by lidar.

[0016] Figure 3 Shows a schematic diagram of an exemplary three-dimensional grid according to an embodiment of the present invention.

[0017] Figure 4 Shows a schematic diagram of using a grid to process the first frame of point cloud according to an embodiment of the present invention.

[0018] Figure 5 Shows a schematic diagram of using a grid to process the second frame of point cloud according to an embodiment of the present invention.

[0019] Figure 6 Shows a flowchart of a method for point cloud segmentation according to an embodiment of the present invention.

[0020] Figure 7 Shows a block diagram of a device for point cloud segmentation according to an embodiment of the present invention.

[0021] Figure 8 Shows a block diagram of an exemplary computing device according to an embodiment of the present invention. Detailed Description of the Invention

[0022] The present invention will be described in detail below with reference to the accompanying drawings, and the features of the present invention will be further manifested in the following detailed description.

[0023] The present invention is mainly based on the following concept: First, divide the space with a grid including multiple cells (for example, divide the two-dimensional space with a two-dimensional grid, or divide the three-dimensional space with a three-dimensional grid), then map the obtained point cloud to each cell of the grid, and then perform weighted averaging on the points mapped to each cell of the grid to obtain the weighted point of the cell (which can also be called the centroid of the cell), thereby representing the points mapped to the cell only with the weighted point.

[0024] Figure 1 Shows a schematic diagram of an exemplary road environment 100. The road environment 100 includes lane lines, lane edges, guardrails, road signs, speed limit signs, street lights, trees, etc. Vehicles (for example, autonomous vehicles) can travel in such a road environment.

[0025] Figure 2A schematic diagram showing a point cloud 200 of an exemplary road environment obtained by lidar. Lidar is an active remote sensing device that uses a laser as the emission light source and adopts optoelectronic detection technology means. Lidar uses laser as the signal source. The pulsed laser emitted by the laser hits objects such as trees, roads, bridges, and buildings on the ground, causing scattering, and a part of the light waves will be reflected back to the receiver of the lidar. According to the laser ranging principle calculation, the distance from the lidar to the target point can be obtained. By continuously scanning the target object with the pulsed laser, the data of all target points on the target object can be obtained. Such data is called point cloud in this field. Each point in the point cloud may include three-dimensional coordinates (x, y, z), color information, and / or reflectance intensity information (e.g., reflectivity), etc. The point cloud 200 obtained by lidar can be stored for further processing.

[0026] Figure 3 A schematic diagram showing an exemplary three-dimensional grid 300 according to an embodiment of the present invention is shown. The size of the grid 300 may correspond to the spatial size covered by a frame of point cloud data (e.g., 200 meters) or larger. The grid 300 may have multiple cells. In Figure 3 the embodiment shown, the grid 300 may have multiple cells of the same size (e.g., cell 301, cell 302), and each cell may have a cubic shape. The side length of each cube can be selected according to actual needs (e.g., 2 - 10 cm). In another embodiment not shown, the grid 300 may have multiple cells of different sizes, and each cell may also have different geometric shapes (e.g., cuboid, cube, etc.). It should be noted that Figure 3 the exemplary grid 300 shown in Figure 3 is shown only for the purpose of facilitating explanation, and the present invention is not limited to

[0027] Figure 4 A schematic diagram showing the processing of the first frame of point cloud using a grid according to an embodiment of the present invention is shown. For the purpose of facilitating explanation, in Figure 4 this, a two-dimensional grid 400 is used to explain the principle of the present invention, which can be easily extended to a three-dimensional grid. The grid 400 may include multiple cells (e.g., cell 401), and each cell may have a square shape.

[0028] At time t1, a first frame of point cloud can be received from a lidar, where each point in the first frame of point cloud can have coordinates, color information, and / or reflectivity. The coordinates are associated with a relative coordinate system with the lidar as the reference point. When the lidar moves as the vehicle moves, the coordinate system adopted by the next frame of point cloud often differs from that adopted by the previous frame of point cloud. In one embodiment, corresponding weights can be assigned to each point based on the color information and / or reflectivity of the point. For example, if red is of interest, a higher weight can be assigned to points with red color. Additionally, a higher weight can be assigned to points with higher reflectivity.

[0029] The first frame of point cloud can be mapped into grid 400. This mapping can include filling each point in the first frame of point cloud into each cell of grid 400. In one embodiment, the coordinate system adopted by grid 400 can be the same as that adopted by the first frame of point cloud, thus eliminating the need for coordinate transformation. In another embodiment, grid 400 can use an absolute coordinate system with a fixed point as the origin, thus requiring the coordinates of each point in the first frame of point cloud to be transformed into this absolute coordinate system. In Figure 4 it, assume there are five points P 11 (x 11 ,y 11 )、P 12 (x 12 ,y 12 )、P 13 (x 13 ,y 13 )、P 14 (x 14 ,y 14 ) and P 15 (x 15 ,y 15 ) falling into cell 401, and they each have weights W 11 、W 12 、W 13 、W 14 and W 15 . Each of the weights W 11 、W 12 、W 13 、W 14 and W 15 can have a value ranging from 0 to 1, or any other suitable value. A weighted average is performed on these five points:

[0030]

[0031] Thereby obtaining the first weighted point P 1w (x 1w ,y 1w ) of cell 401 and this first weighted point P1w (x 1w , y 1w ) has a weight W 1w = W 11 + W 12 + W 13 + W 14 + W 15 . In the same way, similar operations are performed on each of the other cells in the grid 400, so that each cell obtains a first weighted point.

[0032] After obtaining the first weighted points of each cell, these first weighted points can be used for point cloud segmentation. Point cloud segmentation can be performed using conventional point cloud segmentation algorithms, including the K-means algorithm, the K-nearest neighbor algorithm, the GMM algorithm, the region growing algorithm, and / or other well-known algorithms. In one embodiment, point cloud segmentation is performed based on the region growing algorithm, and the growth criterion of the region growing algorithm is defined based on the attributes of each first weighted point. These attributes include at least one of the following: the distance between two adjacent first weighted points, the similarity of the normal directions of two adjacent first weighted points, or the similarity of the reflectivity of the first weighted points. In the present invention, by representing each point in the first frame of point cloud falling into each cell of the grid with the first weighted point, the number of points to be processed when performing point cloud segmentation is greatly reduced.

[0033] Figure 5 FIG. shows a schematic diagram of using a grid to process the second frame of point cloud according to an embodiment of the present invention. The second frame of point cloud can be obtained at time t2 after the first frame of point cloud, and the second frame of point cloud and the first frame of point cloud can be two consecutive frames of point cloud. The grid 500 can correspond to the grid 400 and can include a plurality of cells (e.g., cell 501), and each cell can have a square shape. In addition, each cell in the grid 500 further includes the first weighted point P 1w (x 1w , y 1w ) obtained after processing the first frame of point cloud. Each point in the second frame of point cloud can have coordinates, color information, and / or reflectivity, and a corresponding weight can be assigned to the point based on the color information and / or reflectivity.

[0034] The second frame of point cloud can be mapped into the grid 500. This mapping may include filling each point in the second frame of point cloud into each cell of the grid 500 that already includes the first weighted point. As described above, due to the movement of the lidar, the coordinate system adopted by the second frame of point cloud may be different from the coordinate system adopted by the grid, so coordinate transformation is required. In one embodiment, the coordinates of each point in the second frame of point cloud can be transformed into the coordinate system adopted by the grid 500. For example, if the grid 500 adopts an absolute coordinate system, the coordinates of each point in the second frame of point cloud can be transformed into this absolute coordinate system. Alternatively, if the grid 500 adopts the same coordinate system as the first frame of point cloud, the coordinates of each point in the second frame of point cloud can be transformed into the coordinate system adopted by the first frame of point cloud (for example, this can be performed according to the movement of the lidar between collecting the first frame of point cloud and collecting the second frame of point cloud). In another embodiment, the coordinates of the grid 500 can be transformed into the coordinate system adopted by the second frame of point cloud. In Figure 5 it is assumed that there are also five points P 21 (x 21 ,y 21 ), P 22 (x 22 ,y 22 ), P 13 (x 23 ,y 23 ), P 14 (x 24 ,y 24 ) and P 25 (x 25 ,y 25 ) that fall into the cell 501, and they each have weights W 21 , W 22 , W 23 , W 24 and W 25 . Each of the weights W 21 , W 22 , W 23 , W 24 and W 25 can have a value ranging from 0 to 1, or any other suitable value. Perform a weighted average on these five points and the first weighted point P 1w (x 1w ,y 1w ) located in the cell 501:

[0035]

[0036] Thus, the second weighted point P 2w (x 2w ,y 2w)And the second weighted point P 2w (x 2w , y 2w ) has a weight W 2w = W 1w + W 21 + W 22 + W 23 + W 24 + W 25 . In the same manner, similar operations are performed on each of the other cells in the grid 500, so that each cell obtains a second weighted point.

[0037] It should be noted that since the adjacent two frames of point clouds are obtained within a relatively short time interval, they may contain a large amount of overlapping information. Directly segmenting each frame of point cloud may lose the potential correlation information between the frames of point cloud. To solve this problem, after obtaining the second weighted points of each cell, these second weighted points can be used for point cloud segmentation. As described above, point cloud segmentation can be performed using conventional point cloud segmentation algorithms, including the K-means algorithm, the K-nearest neighbor algorithm, the GMM algorithm, the region growing algorithm, and / or other well-known algorithms. Since the second weighted points in each cell contain both the information of the points in the first frame of point cloud that fall into the cell and the information of the points in the second frame of point cloud that fall into the cell, using the second weighted points for point cloud segmentation can make full use of the correlation between the first frame of point cloud and the second frame of point cloud, thereby more accurately extracting the object structure represented by the point cloud data.

[0038] Similarly, at time t3, a third frame of point cloud can be received from the lidar, and the above process is performed in the same manner, so as to obtain the third weighted points of each cell and use the third weighted points for point cloud segmentation. And so on, after receiving each frame of point cloud, it is mapped into the respective cells of the grid, and the points falling into each cell and the weighted points obtained after processing the previous frame are weighted and averaged, so as to obtain new weighted points and use the new weighted points to perform point cloud segmentation. Performing point cloud fusion and point cloud segmentation in this way can significantly save processing resources and make full use of the correlation between the frames of point cloud.

[0039] Figure 6 shows a flowchart of a method for point cloud fusion according to an embodiment of the present invention. For example, method 600 can be implemented within at least one processor (e.g., Figure 8 processor 804), and this processor can be located in an in-vehicle computer system, a remote server, or a combination thereof. Of course, in various aspects of the present invention, method 600 can also be implemented by any suitable device capable of performing related operations.

[0040] Method 600 begins at step 610. At step 610, method 600 may include receiving a first frame of point cloud from a sensor, where each point in the first frame of point cloud has an associated weight. Here, the sensor may include any sensor capable of scanning an object at a specific frame rate to obtain point cloud data (e.g., lidar, stereo camera, or time-of-flight camera). Each point in the point cloud may include three-dimensional coordinates (x, y, z), color information, and / or reflectance intensity information (e.g., reflectivity), etc. In one embodiment, a corresponding weight may be assigned to a point based on the color information and / or reflectivity of each point. For example, if red is of interest, a higher weight may be assigned to points with red color. Additionally, a higher weight may be assigned to points with higher reflectivity.

[0041] At step 620, method 600 may include mapping the first frame of point cloud into a grid including a plurality of cells. The size of the grid may correspond to the spatial size covered by one frame of point cloud data (e.g., 200 meters). In one embodiment, each cell in the grid may have the same size and the same shape (e.g., a cube with a side length of 2 - 10 cm). In another embodiment, each cell in the grid may have different sizes and different shapes. The mapping may include filling each point in the first frame of point cloud into each cell of the grid. In one embodiment, the coordinate system adopted by the grid may be the same as the coordinate system adopted by the first frame of point cloud, so that no coordinate transformation is required when filling the points in the first frame of point cloud into each cell of the grid. In another embodiment, the grid may use an absolute coordinate system with a fixed point as the origin, so that the coordinates of each point in the first frame of point cloud need to be first transformed to this absolute coordinate system, and then each point is filled into each cell of the grid.

[0042] At step 630, method 600 may include performing weighted averaging on each point in the first frame of point cloud mapped into each cell of the grid to obtain a first weighted point of the cell. An example of performing weighted averaging on each point in each cell to obtain a first weighted point of the cell is shown in the Figure 4 description.

[0043] At step 640, method 600 may include performing point cloud segmentation using the first weighted points in each cell of the grid. Various segmentation algorithms known in the art may be employed. These segmentation algorithms may include the K-means algorithm, the K-nearest neighbor algorithm, the GMM algorithm, the region growing algorithm, and / or other known algorithms. In one embodiment, the point cloud segmentation is performed based on the region growing algorithm, and the growth criterion of the region growing algorithm is defined based on the attributes of each first weighted point. These attributes include at least one of the following: the distance between two adjacent first weighted points, the similarity of the normal directions of two adjacent first weighted points, or the similarity of the reflectivity of the first weighted points.

[0044] In an optional step, method 600 may further include receiving a second frame of point cloud from a sensor, each point in the second frame of point cloud having an associated weight; and mapping the second frame of point cloud into the grid including the first weighted points. In one embodiment, the mapping may include transforming the coordinates of the second frame of point cloud to be associated with the coordinate system used by the grid or transforming the coordinates of the grid to be associated with the coordinate system used by the second frame of point cloud, and filling each point in the second frame of point cloud into each cell of the grid. Additionally, method 600 may further include performing a weighted average of each point in the second frame of point cloud mapped into each cell of the grid and the first weighted point located in that cell to obtain a second weighted point for that cell. In Figure 5 the description shows an example of performing a weighted average of each point in the second frame of point cloud belonging to each cell and the first weighted point located in that cell to obtain a second weighted point for that cell. Additionally, method 600 may further include performing point cloud segmentation using the second weighted points in each cell of the grid. In the present invention, by representing each point in the first frame of point cloud and the second frame of point cloud falling into each cell of the grid with the second weighted points, the number of points to be processed when performing point cloud segmentation is greatly reduced. Moreover, the second weighted points contain the correlation between the first frame of point cloud and the second frame of point cloud, thereby enabling the improvement of the accuracy of point cloud segmentation.

[0045] Figure 7 FIG. shows a block diagram of a device for point cloud fusion according to an embodiment of the present invention. All functional blocks of device 700 (including each unit in device 700) may be implemented by hardware, software, or a combination of hardware and software. Those skilled in the art should understand that Figure 7 the functional blocks described in may be combined into a single functional block or divided into multiple sub-functional blocks.

[0046] The device 700 may include a receiving unit 710 configured to receive a first frame of point cloud from a sensor, where each point in the first frame of point cloud has an associated weight. The device 700 may further include a mapping unit 720 configured to map the first frame of point cloud into a grid including a plurality of cells. The device 700 may further include a computing unit 730 configured to perform a weighted average of the respective points mapped into each cell of the grid in the first frame of point cloud to obtain a first weighted point of the cell. In addition, the device 700 may further include a point cloud segmentation unit 740 configured to perform point cloud segmentation using the first weighted points in each cell of the grid. Further, the receiving unit 710 may be further configured to receive a second frame of point cloud from the sensor, where each point in the second frame of point cloud has an associated weight. The mapping unit 720 may be further configured to map the second frame of point cloud into the grid including the first weighted points. The computing unit 730 may be further configured to perform a weighted average of the respective points mapped into each cell of the grid in the second frame of point cloud and the first weighted points located in the cell to obtain a second weighted point of the cell. The point cloud segmentation unit 740 may be further configured to perform point cloud segmentation using the second weighted points in each cell of the grid.

[0047] Figure 8 A block diagram of an exemplary computing device according to an embodiment of the present invention is shown. The computing device is an example of a hardware device applicable to various aspects of the present invention.

[0048] Reference Figure 8 , a computing device 800 will now be described. The computing device 800 is an example of a hardware device applicable to various aspects of the present invention. The computing device 800 may be any machine configurable to perform processing and / or computing, and may be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a smartphone, an in-vehicle computer, or any combination thereof. The foregoing various methods / devices / servers / client devices may be implemented in whole or at least in part by the computing device 800 or a similar device or system.

[0049] The computing device 800 may include components that can be connected or communicate via one or more interfaces and a bus 802. For example, the computing device 800 may include a bus 802, one or more processors 804, one or more input devices 806, and one or more output devices 808. The one or more processors 804 can be any type of processor and may include, but are not limited to, one or more general-purpose processors and / or one or more dedicated processors (e.g., specialized processing chips). The input device 806 can be any type of device capable of inputting information into the computing device and may include, but are not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 808 can be any type of device capable of presenting information and may include, but are not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The computing device 800 may also include a non-transitory storage device 810 or be connected to the non-transitory storage device, which can be any storage device that is non-transitory and capable of implementing data storage, and the non-transitory storage device may include, but are not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, an optical disk, or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory, and / or any storage chip or cartridge, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transitory storage device 810 can be separated from the interface. The non-transitory storage device 810 may have data / instructions / code for implementing the above methods and steps. The computing device 800 may also include a communication device 812. The communication device 812 can be any type of device or system capable of implementing communication with internal devices and / or communication with a network and may include, but are not limited to, a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset, such as a Bluetooth device, an IEEE 1302.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or a similar device.

[0050] When the computing device 800 is used as an in-vehicle device, it can also be connected to external devices (e.g., a GPS receiver, sensors for sensing different environmental data (such as an acceleration sensor, a wheel speed sensor, a gyroscope, etc.)). In this way, the computing device 800 can receive, for example, positioning data and sensor data indicating the form condition of the vehicle. When the computing device 800 is used as an in-vehicle device, it can also be connected to other devices for controlling the driving and operation of the vehicle (e.g., an engine system, a windshield wiper, an anti-lock braking system, etc.).

[0051] In addition, the non-transitory storage device 810 may have map information and software components, so that the processor 804 can implement route guidance processing. In addition, the output device 806 may include a display for displaying maps, displaying positioning markers of the vehicle, and displaying images indicating the driving status of the vehicle. The output device 806 may also include a speaker or a headphone interface for audio guidance.

[0052] The bus 802 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. In particular, for in-vehicle devices, the bus 802 may also include a Controller Area Network (CAN) bus or other architectures designed for automotive applications.

[0053] The computing device 800 may also include a working memory 814, which can be any type of working memory capable of storing instructions and / or data beneficial to the operation of the processor 804 and may include, but is not limited to, random access memory and / or read-only storage devices.

[0054] The software components may be located in the working memory 814, and these software components include, but are not limited to, an operating system 816, one or more application programs 818, drivers, and / or other data and code. The instructions for implementing the above methods and steps may be included in the one or more application programs 818, and the modules / units / components of the various devices / servers / client devices described above may be implemented by the processor 804 reading and executing the instructions of the one or more application programs 818.

[0055] It should also be recognized that changes can be made according to specific requirements. For example, custom hardware may also be used, and / or specific components may be implemented in hardware, software, firmware, middleware, microcode, a hardware description language, or any combination thereof. In addition, connections with other computing devices, such as network input / output devices, etc., may be adopted. For example, part or all of the disclosed methods and devices may be implemented by programming hardware (such as programmable logic circuits including Field Programmable Gate Arrays (FPGAs) and / or Programmable Logic Arrays (PLAs)) with an assembly language or a hardware programming language (such as VERILOG, VHDL, C++) using the logic and algorithms according to the present invention.

[0056] Although aspects of the present invention have been described so far with reference to the accompanying drawings, the above methods, systems, and devices are merely examples, and the scope of the present invention is not limited to these aspects, but is defined only by the appended claims and their equivalents. Various components may be omitted or may also be replaced by equivalent components. Additionally, the steps may be implemented in an order different from the order described in the present invention. Moreover, the various components may be combined in various ways. It is also important that, as technology develops, many of the components described may be replaced by equivalent components that emerge later.

Claims

1. A method for point cloud segmentation, the method comprising: Receive a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight, where the weight is based on the reflectivity and / or color of each point; Map the first frame of point cloud into a grid including a plurality of cells; Perform a weighted average on each point in the first frame of point cloud mapped into each cell of the grid using the associated weight to obtain a first weighted point for the cell, where the weighted average is performed based only on the respective points in the first frame of point cloud mapped into the cell of the grid; and Perform point cloud segmentation using the first weighted points in each cell of the grid.

2. The method according to claim 1, characterized in that The sensor includes at least one of the following: lidar, stereo camera, or time-of-flight camera.

3. The method according to claim 1, characterized in that Each cell in the grid has a cubic shape and the same size, and the side length of the cube is 2 - 10 cm.

4. The method according to claim 1, characterized in that Each cell in the grid has a different size.

5. The method according to claim 1, characterized in that The point cloud segmentation is performed based on a region growing algorithm, and the growth criterion of the region growing algorithm is defined based on the attributes of each first weighted point.

6. The method according to claim 5, characterized in that The attributes include at least one of the following: the distance between two adjacent first weighted points, the similarity of the normal directions of two adjacent first weighted points, or the similarity of the reflectivity of the first weighted points.

7. The method according to claim 1, characterized in that Further includes: Receive a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; Map the second frame of point cloud into the grid including the first weighted points; Perform a weighted average on each point in the second frame of point cloud mapped into each cell of the grid and the first weighted point located in the cell to obtain a second weighted point for the cell; And Perform point cloud segmentation using the second weighted points in each cell of the grid.

8. The method according to claim 7, characterized in that Mapping the second frame of point cloud into the grid including the first weighted points includes: transforming the coordinates of the second frame of point cloud to be associated with the coordinate system used by the grid.

9. An apparatus for point cloud segmentation, the apparatus comprising: A receiving unit configured to receive a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight, where the weight is based on the reflectivity and / or color of each point; A mapping unit configured to map the first frame of point cloud into a grid including a plurality of cells; A calculation unit configured to perform a weighted average on each point in the first frame of point cloud mapped into each cell of the grid using the associated weight to obtain a first weighted point for the cell, where the weighted average is performed based only on the respective points in the first frame of point cloud mapped into the cell of the grid; and A point cloud segmentation unit configured to perform point cloud segmentation using the first weighted points in each cell of the grid.

10. The apparatus according to claim 9, characterized in that The receiving unit is further configured to receive a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; The mapping unit is further configured to map the second frame of point cloud into the grid including the first weighted points; The computing unit is further configured to perform a weighted average of each point mapped to each cell of the grid in the second frame of point cloud and the first weighted point located in the cell to obtain a second weighted point of the cell; The point cloud segmentation unit is further configured to perform point cloud segmentation using the second weighted points in each cell of the grid.

11. An apparatus for point cloud segmentation, the apparatus comprising: A memory that stores a computer program; And A processor coupled to the memory, the computer program when executed by the processor implements the following steps: Receiving a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight, where the weight is based on the reflectivity and / or color of each point; Mapping the first frame of point cloud into a grid including a plurality of cells; Performing a weighted average of each point mapped to each cell of the grid in the first frame of point cloud using the associated weight to obtain a first weighted point of the cell, where the weighted average is performed based only on the respective points mapped to the cell of the grid in the first frame of point cloud; and Performing point cloud segmentation using the first weighted points in each cell of the grid.

12. The device according to claim 11, wherein The computer program when executed by the processor further implements the following steps: Receiving a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; Mapping the second frame of point cloud into the grid including the first weighted points; Performing a weighted average of each point mapped to each cell of the grid in the second frame of point cloud and the first weighted point located in the cell to obtain a second weighted point of the cell; And Performing point cloud segmentation using the second weighted points in each cell of the grid.

13. A vehicle, comprising: A sensor; The apparatus for point cloud segmentation according to claim 9 or 10.

14. A non-transitory computer-readable medium storing a computer program which, when executed by a processor, performs the method according to any one of claims 1-8.

Images