Method and apparatus for point cloud fusion

Abstract

The present invention provides a method and apparatus for point cloud fusion. The method may include and the apparatus may be used to: receive a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight; map the first frame of point cloud into a grid including a plurality of cells; perform weighted averaging on the respective points in each cell of the grid into which the first frame of point cloud is mapped to obtain a first weighted point of the cell; receive a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; map the second frame of point cloud into the grid including the first weighted point; and perform weighted averaging on the respective points in each cell of the grid into which the second frame of point cloud is mapped and the first weighted point located in the cell to obtain a second weighted point of the cell. By adopting the method and apparatus disclosed in the present invention, the processing efficiency of point cloud data can be significantly improved and processing resources can be saved.

Description

Technical Field

[0001] The present invention relates to the processing of point cloud data, and more particularly, to a method and apparatus for point cloud fusion. Background Art

[0002] For a long time, autonomous driving has been the subject of research efforts aimed at improving the safety and efficiency of automotive transportation. In recent years, increasingly sophisticated sensors have brought autonomous driving systems closer to reality. For example, 3D scanners (such as lidar, stereo cameras, time-of-flight cameras, etc.) are now widely used in autonomous driving systems. Such 3D scanners measure a large number of points on the surface of an object and often output a point cloud as a data file. A point cloud represents a set of points measured by a 3D scanner. As is known in the art, point clouds can be used for many purposes, including creating 3D maps, object recognition, and object tracking, etc.

[0003] Generally, 3D scanners such as lidar scan surrounding objects at a rate of dozens to hundreds of frames per second. When an autonomous vehicle is traveling on a road, continuous point cloud frames are obtained by the lidar mounted on the vehicle to learn about the vehicle's surrounding environment in real time. To this end, it is necessary to segment each frame of the point cloud to extract the structure of the object. Conventionally, segmentation is directly performed on each frame of the point cloud. However, since each frame of the point cloud may contain thousands of points, this approach requires a large amount of processing resources. In addition, since two adjacent frames of the point cloud are obtained within a relatively short time interval, they may contain a large amount of overlapping information. Directly segmenting each frame of the point cloud may also lose potential correlation information between the frames of the point cloud.

[0004] Accordingly, there is a need to fuse each frame of the point cloud for further processing. Summary of the Invention

[0005] The present invention content is provided to introduce some concepts in a simplified form that will be further described in the following detailed description. The present invention content is not intended to identify the key features or essential features of the claimed subject matter, nor is it intended to be used to help determine the scope of the claimed subject matter.

[0006] According to an embodiment of the present invention, a method for point cloud fusion is provided. The method includes: receiving a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight; mapping the first frame of point cloud into a grid including a plurality of cells; performing a weighted average on each point mapped into each cell of the grid in the first frame of point cloud to obtain a first weighted point of the cell; receiving a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; mapping the second frame of point cloud into the grid including the first weighted point; and performing a weighted average on each point mapped into each cell of the grid in the second frame of point cloud and the first weighted point located in the cell to obtain a second weighted point of the cell.

[0007] According to an embodiment of the present invention, a device for point cloud fusion is provided. The device includes: a receiving unit configured to receive a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight; a mapping unit configured to map the first frame of point cloud into a grid including a plurality of cells; and a calculation unit configured to perform a weighted average on each point mapped into each cell of the grid in the first frame of point cloud to obtain a first weighted point of the cell; wherein the receiving unit is further configured to receive a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; wherein the mapping unit is further configured to map the second frame of point cloud into the grid including the first weighted point; and wherein the calculation unit is further configured to perform a weighted average on each point mapped into each cell of the grid in the second frame of point cloud and the first weighted point located in the cell to obtain a second weighted point of the cell.

[0008] According to an embodiment of the present invention, a device for point cloud fusion is provided. The device includes: a memory storing a computer program; and a processor coupled to the memory, the computer program when executed by the processor implementing the following steps: receiving a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight; mapping the first frame of point cloud into a grid including a plurality of cells; performing a weighted average on each point mapped into each cell of the grid in the first frame of point cloud to obtain a first weighted point of the cell; receiving a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; mapping the second frame of point cloud into the grid including the first weighted point; and performing a weighted average on each point mapped into each cell of the grid in the second frame of point cloud and the first weighted point located in the cell to obtain a second weighted point of the cell.

[0009] According to an embodiment of the present invention, a vehicle is provided. The vehicle includes: a sensor; and a device for point cloud fusion according to the present invention.

[0010] According to an embodiment of the present invention, a non-transitory computer-readable medium is provided, which stores a computer program that, when executed by a processor, performs a method for point cloud fusion according to the present invention.

[0011] By adopting the method and device disclosed in the present invention, the processing efficiency of point cloud data can be significantly improved and processing resources can be saved. In addition, the correlation between adjacent point cloud frames can be fully utilized to extract the object structure represented by the point cloud data.

[0012] These and other features and advantages will become apparent by reading the following detailed description and referring to the associated drawings. It should be understood that the foregoing general description and the following detailed description are illustrative only and do not limit the various aspects claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to understand in detail the manner in which the above-described features of the present invention are used, the above briefly summarized content can be described in more detail with reference to the various embodiments, some of which are shown in the drawings. However, it should be noted that the drawings only show some typical aspects of the present invention and should not be considered to limit its scope, as the description may allow other equally effective aspects.

[0014] Figure 1 A schematic diagram showing an exemplary road environment is shown.

[0015] Figure 2 A schematic diagram showing the point cloud of an exemplary road environment obtained by a lidar is shown.

[0016] Figure 3 A schematic diagram showing an exemplary three-dimensional mesh according to an embodiment of the present invention is shown.

[0017] Figures 4A - 4B A schematic diagram showing point cloud fusion of two adjacent frames of point clouds using a mesh according to an embodiment of the present invention is shown.

[0018] Figure 5

[0019] Figure 6 A flowchart showing a method for point cloud fusion according to an embodiment of the present invention is shown.

[0020] Figure 7 A block diagram showing an exemplary computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0021] The present invention will be described in detail below with reference to the accompanying drawings, and the features of the present invention will be further revealed in the following specific description.

[0022] In the present invention, point cloud fusion refers to a technique of combining each frame of point cloud together for further processing. Through point cloud fusion, the correlation between each frame of point cloud can be effectively utilized and the efficiency of point cloud data processing can be improved. The present invention is mainly based on the following concept: First, divide the space with a grid including multiple cells (for example, divide the two-dimensional space with a two-dimensional grid or divide the three-dimensional space with a three-dimensional grid), then map the obtained point cloud to each cell of the grid, and then perform weighted averaging on each point mapped to each cell of the grid to obtain the weighted point of the cell (which can also be called the centroid of the cell), thereby representing each point mapped to the cell only with this weighted point.

[0023] Figure 1 A schematic diagram of an exemplary road environment 100 is shown. The road environment 100 includes lane lines, lane edges, guardrails, road signs, speed limit signs, street lights, trees, etc. A vehicle (for example, an autonomous vehicle) can drive in such a road environment.

[0024] Figure 2 A schematic diagram of a point cloud 200 of an exemplary road environment obtained by a lidar is shown. A lidar is an active remote sensing device that uses a laser as a transmitting light source and adopts optoelectronic detection technology means. The lidar uses laser as the signal source. The pulsed laser emitted by the laser hits trees, roads, bridges, buildings, etc. on the ground, causing scattering, and a part of the light wave will be reflected to the receiver of the lidar. According to the laser ranging principle calculation, the distance from the lidar to the target point can be obtained. The pulsed laser continuously scans the target object, and the data of all target points on the target object can be obtained. Such data is called point cloud in the art. Each point in the point cloud may include three-dimensional coordinates (x, y, z), color information, and / or reflectance intensity information (for example, reflectivity), etc. The point cloud 200 obtained by the lidar can be stored for further processing.

[0025] Figure 3 A schematic diagram of an exemplary three-dimensional grid 300 according to an embodiment of the present invention is shown. The size of the grid 300 can correspond to the size of the space covered by a frame of point cloud data (for example, 200 meters) or larger. The grid 300 may have multiple cells. In Figure 3In the embodiment shown, the grid 300 may have a plurality of cells of the same size (e.g., cell 301, cell 302), and each cell may have the shape of a cube. The side length of each cube can be selected according to actual needs (e.g., 2 - 10 cm). In another embodiment not shown, the grid 300 may have a plurality of cells of different sizes, and each cell may also have different geometric shapes (e.g., cuboid, cube, etc.). It should be noted that, Figure 3 the exemplary grid 300 shown is only shown for the purpose of facilitating explanation, and the present invention is not limited to Figure 3 the exemplary grid 300 shown, but may include any grid suitable for dividing space, and the grid may have any number, any size, and / or any shape of cells.

[0026] Figures 4A - 4B FIG. 400 shows a schematic diagram of point cloud fusion of two adjacent frames of point clouds using a grid according to an embodiment of the present invention. For the purpose of facilitating explanation, in Figures 4A - 4B this, a two-dimensional grid 400 is used to explain the principle of the present invention, which can be easily extended to a three-dimensional grid. The grid 400 may include a plurality of cells (e.g., cell 401), and each cell may have the shape of a square.

[0027] At time t1, a first frame of point cloud may be received from a lidar, where each point in the first frame of point cloud may have coordinates, color information, and / or reflectivity. The coordinates are associated with a relative coordinate system with the lidar as the reference point. When the lidar moves with the movement of the vehicle, the coordinate system used for the next frame of point cloud is often different from the coordinate system used for the previous frame of point cloud. In one embodiment, corresponding weights may be assigned to each point based on the color information and / or reflectivity of the point. For example, if one is interested in red, a higher weight may be assigned to points with red color. In addition, a higher weight may be assigned to points with higher reflectivity.

[0028] The first frame of point cloud may be mapped into the grid 400. This mapping may include filling each point in the first frame of point cloud into each cell of the grid 400. In one embodiment, the coordinate system used by the grid 400 may be the same as the coordinate system used by the first frame of point cloud, so that no coordinate transformation is required. In another embodiment, the grid 400 may use an absolute coordinate system with a fixed point as the origin, so that the coordinates of each point in the first frame of point cloud need to be transformed to this absolute coordinate system. In Figure 4A this, it is assumed that there are five points P 11 (x 11 ,y 11 ), P 12 (x 12 ,y 12 ), P 13 (x13 , y 13 ), P 14 (x 14 , y 14 ) and P 15 (x 15 , y 15 ) fall into unit 401, each having a weight W 11 , W 12 , W 13 , W 14 and W 15 . The weights W 11 , W 12 , W 13 , W 14 and W 15 may each have a value ranging from 0 to 1, or any other suitable value. A weighted average of these five points is performed:

[0029]

[0030] From this, the first weighted point P 1w (x 1w , y 1w ) of unit 401 is obtained, and the weight W 1w (x 1w , y 1w ) of this first weighted point P 1w = W 11 + W 12 + W 13 + W 14 + W 15 . In the same way, similar operations are performed on each of the other units in grid 400, so that each unit obtains a first weighted point, and in the subsequent operations, only these first weighted points are used.

[0031] At time t2, a second frame of point cloud can be received from the lidar. Similarly, each point in this second frame of point cloud can have coordinates, color information, and / or reflectivity, and corresponding weights can be assigned to the point based on this color information and / or reflectivity.

[0032] The second frame of point cloud can be mapped into the grid 400. This mapping may include filling each point in the second frame of point cloud into each cell of the grid 400 that already includes the first weighted point. As described above, due to the movement of the lidar, the coordinate system adopted by the second frame of point cloud may be different from the coordinate system adopted by the grid, so coordinate transformation is required. In one embodiment, the coordinates of each point in the second frame of point cloud can be transformed into the coordinate system adopted by the grid 400. For example, if the grid 400 adopts an absolute coordinate system, the coordinates of each point in the second frame of point cloud can be transformed into this absolute coordinate system. Alternatively, if the grid 400 adopts the same coordinate system as the first frame of point cloud, the coordinates of each point in the second frame of point cloud can be transformed into the coordinate system adopted by the first frame of point cloud (for example, this can be performed according to the movement of the lidar between collecting the first frame of point cloud and collecting the second frame of point cloud). In another embodiment, the coordinates of the grid 400 can be transformed into the coordinate system adopted by the second frame of point cloud. In Figure 4B it is assumed that there are also five points P 21 (x 21 ,y 21 ), P 22 (x 22 ,y 22 ), P 13 (x 23 ,y 23 ), P 14 (x 24 ,y 24 ) and P 25 (x 25 ,y 25 ) that fall into the cell 401, and they each have weights W 21 , W 22 , W 23 , W 24 and W 25 . Each of the weights W 21 , W 22 , W 23 , W 24 and W 25 can have a value ranging from 0 to 1, or any other suitable value. Perform a weighted average on these five points and the first weighted point P 1w (x 1w ,y 1w ) located in the cell 401:

[0033]

[0034] Thus, the second weighted point P 2w (x 2w ,y 2w) And the second weighted point P 2w (x 2w , y 2w ) has a weight W 2w = W 1w + W 21 + W 22 + W 23 + W 24 + W 25 . In the same way, similar operations are performed on each other cell in the grid 400, so that each cell obtains a second weighted point, and in the subsequent operations, only these second weighted points are used.

[0035] At time t3, a third frame of point cloud can be received from the lidar, and the above process is performed in the same way to obtain a third weighted point. And so on. After each frame of point cloud is received, it is mapped into the respective cells of the grid, and the points falling into each cell and the weighted points obtained after processing the previous frame are weighted and averaged to obtain new weighted points. By performing point cloud fusion in this way, processing resources can be significantly saved and the correlation between each frame of point cloud can be fully utilized.

[0036] It should be noted that Figures 4A - 4B the embodiments shown are only for illustrative purposes. In actual operation, the number of points mapped into each cell may be much larger.

[0037] Figure 5 shows a flowchart of a method for point cloud fusion according to an embodiment of the present invention. For example, method 500 can be implemented within at least one processor (e.g., Figure 7 the processor 704), which can be located in an in-vehicle computer system, a remote server, or a combination thereof. Of course, in various aspects of the present invention, method 500 can also be implemented by any suitable device capable of performing related operations.

[0038] Method 500 starts at step 510. At step 510, method 500 may include receiving a first frame of point cloud from a sensor, and each point in the first frame of point cloud has an associated weight. Here, the sensor can include any sensor capable of scanning an object at a specific frame rate to obtain point cloud data (e.g., lidar, stereo camera, or time-of-flight camera). Each point in the point cloud can include three-dimensional coordinates (x, y, z), color information, and / or reflectance intensity information (e.g., reflectivity), etc. In one embodiment, the corresponding weight can be assigned to the point based on the color information and / or reflectivity of each point. For example, if red is of interest, a higher weight can be assigned to the points with red color. In addition, a higher weight can be assigned to the points with higher reflectivity.

[0039] At step 520, method 500 may include mapping a first frame of point cloud into a grid including a plurality of cells. The size of the grid may correspond to the spatial size covered by a frame of point cloud data (e.g., 200 meters). In one embodiment, each cell in the grid may have the same size and the same shape (e.g., a cube with a side length of 2 - 10 cm). In another embodiment, each cell in the grid may have different sizes and different shapes. The mapping may include filling each point in the first frame of point cloud into each cell of the grid. In one embodiment, the coordinate system adopted by the grid may be the same as the coordinate system adopted by the first frame of point cloud, so that no coordinate transformation is required when filling the points in the first frame of point cloud into each cell of the grid. In another embodiment, the grid may use an absolute coordinate system with a fixed point as the origin, so that the coordinates of each point in the first frame of point cloud need to be transformed to this absolute coordinate system first, and then each point is filled into each cell of the grid.

[0040] At step 530, method 500 may include weighted averaging each point mapped to each cell of the grid in the first frame of point cloud to obtain a first weighted point of the cell. An example of weighted averaging each point in each cell to obtain a first weighted point of the cell is shown in the description of Figure 4A .

[0041] At step 540, method 500 may include receiving a second frame of point cloud from the sensor, and each point in the second frame of point cloud has an associated weight. In one embodiment, the first frame of point cloud and the second frame of point cloud may be point cloud frames continuously acquired by the sensor.

[0042] At step 550, method 500 may include mapping the second frame of point cloud into the grid including the first weighted points. In one embodiment, the mapping may include transforming the coordinates of the second frame of point cloud to be associated with the coordinate system used by the grid or transforming the coordinates of the grid to be associated with the coordinate system used by the second frame of point cloud, and filling each point in the second frame of point cloud into each cell of the grid.

[0043] At step 560, method 500 may include weighted averaging each point mapped to each cell of the grid in the second frame of point cloud and the first weighted point located in the cell to obtain a second weighted point of the cell. An example of weighted averaging each point belonging to the second frame of point cloud in each cell and the first weighted point located in the cell to obtain a second weighted point of the cell is shown in the description of Figure 4B .

[0044] In an optional step, method 500 may include performing point cloud segmentation using the second weighted points in each cell of the grid. Various segmentation algorithms known in the art may be employed. These segmentation algorithms may include the K-means algorithm, the K-nearest neighbor algorithm, the GMM algorithm, the region growing algorithm, and / or other known algorithms. In one embodiment, the point cloud segmentation is performed based on the region growing algorithm, and the growth criterion of the region growing algorithm is defined based on the attributes of each second weighted point. These attributes include at least one of the following: the distance between two adjacent second weighted points, the similarity of the normal directions of two adjacent second weighted points, or the similarity of the reflectivity of the second weighted points. In the present invention, by representing each point in the first frame point cloud and the second frame point cloud falling into each cell of the grid with the second weighted points, the number of points to be processed when performing point cloud segmentation is greatly reduced. In addition, the second weighted points contain the correlation between the first frame point cloud and the second frame point cloud, thereby enabling the improvement of the accuracy of point cloud segmentation.

[0045] Figure 6 FIG. shows a block diagram of an apparatus for point cloud fusion according to an embodiment of the present invention. All functional blocks of apparatus 600 (including each unit in apparatus 600) may be implemented by hardware, software, or a combination of hardware and software. Those skilled in the art should understand that Figure 6 the functional blocks described in may be combined into a single functional block or divided into multiple sub-functional blocks.

[0046] Apparatus 600 may include a receiving unit 610 configured to receive a first frame point cloud from a sensor, each point in the first frame point cloud having an associated weight. Apparatus 600 may further include a mapping unit 620 configured to map the first frame point cloud into a grid including a plurality of cells. Apparatus 600 may further include a calculation unit 630 configured to perform weighted averaging on each point in the first frame point cloud mapped into each cell of the grid to obtain a first weighted point of the cell. The receiving unit 610 may be further configured to receive a second frame point cloud from the sensor, each point in the second frame point cloud having an associated weight. The mapping unit 620 may be further configured to map the second frame point cloud into the grid including the first weighted points. In addition, the calculation unit 630 may be further configured to perform weighted averaging on each point in the second frame point cloud mapped into each cell of the grid and the first weighted point located in the cell to obtain a second weighted point of the cell. In another embodiment, apparatus 600 may optionally include a point cloud segmentation unit configured to perform point cloud segmentation using the second weighted points in each cell of the grid.

[0047] Figure 7FIG. shows a block diagram of an exemplary computing device according to an embodiment of the present invention, which is an example of a hardware device applicable to various aspects of the present invention.

[0048] Referring Figure 7 , a computing device 700 will now be described, which is an example of a hardware device applicable to various aspects of the present invention. The computing device 700 can be any machine configured to perform processing and / or computing, and can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a smartphone, an in-vehicle computer, or any combination thereof. The foregoing various methods / devices / servers / client devices can be implemented in whole or at least in part by the computing device 700 or a similar device or system.

[0049] The computing device 700 may include components that can be connected or communicate via one or more interfaces and a bus 702. For example, the computing device 700 may include a bus 702, one or more processors 704, one or more input devices 706, and one or more output devices 708. The one or more processors 704 can be any type of processor and may include, but are not limited to, one or more general-purpose processors and / or one or more dedicated processors (e.g., specialized processing chips). The input device 706 can be any type of device capable of inputting information into the computing device and may include, but are not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 708 can be any type of device capable of presenting information and may include, but are not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The computing device 700 may also include a non-transitory storage device 710 or be connected to the non-transitory storage device, which can be any storage device that is non-transitory and capable of implementing data storage, and the non-transitory storage device may include, but are not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, an optical disk, or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory, and / or any storage chip or cartridge tape, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transitory storage device 710 can be separated from the interface. The non-transitory storage device 710 may have data / instructions / code for implementing the above methods and steps. The computing device 700 may also include a communication device 712. The communication device 712 can be any type of device or system capable of enabling communication with internal devices and / or with a network and may include, but are not limited to, a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset, such as a Bluetooth device, an IEEE 1302.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or a similar device.

[0050] When the computing device 700 is used as an in-vehicle device, it can also be connected to external devices (e.g., a GPS receiver, sensors for sensing different environmental data (such as an acceleration sensor, a wheel speed sensor, a gyroscope, etc.)). In this way, the computing device 700 can receive, for example, positioning data and sensor data indicating the driving condition of the vehicle. When the computing device 700 is used as an in-vehicle device, it can also be connected to other devices for controlling the driving and operation of the vehicle (e.g., an engine system, a windshield wiper, an anti-lock braking system, etc.).

[0051] In addition, the non-transitory storage device 710 may have map information and software components, so that the processor 704 can implement route guidance processing. In addition, the output device 706 may include a display for displaying a map, a positioning marker of the vehicle, and an image indicating the driving condition of the vehicle. The output device 706 may also include a speaker or a headphone jack for audio guidance.

[0052] The bus 702 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. In particular, for in-vehicle devices, the bus 702 may also include a Controller Area Network (CAN) bus or other architectures designed for automotive applications.

[0053] The computing device 700 may also include a working memory 714, which can be any type of working memory capable of storing instructions and / or data beneficial to the operation of the processor 704 and may include, but is not limited to, random access memory and / or read-only storage devices.

[0054] The software components may be located in the working memory 714, and these software components include, but are not limited to, an operating system 716, one or more application programs 718, drivers, and / or other data and code. The instructions for implementing the above methods and steps may be included in the one or more application programs 718, and the modules / units / components of the foregoing various devices / servers / client devices may be implemented by the processor 704 reading and executing the instructions of the one or more application programs 718.

[0055] It should also be recognized that changes can be made according to specific requirements. For example, custom hardware may also be used, and / or specific components may be implemented in hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. In addition, connections with other computing devices, such as network input / output devices, etc., may be adopted. For example, part or all of the disclosed methods and devices may be implemented by programming hardware (e.g., programmable logic circuits including Field Programmable Gate Arrays (FPGAs) and / or Programmable Logic Arrays (PLAs)) with assembly language or hardware programming languages (e.g., VERILOG, VHDL, C++) using the logic and algorithms according to the present invention.

[0056] Although aspects of the present invention have been described so far with reference to the accompanying drawings, the above methods, systems, and devices are merely examples, and the scope of the present invention is not limited to these aspects, but is defined only by the appended claims and their equivalents. Various components may be omitted or may also be replaced by equivalent components. Additionally, the steps may be implemented in an order different from the order described in the present invention. Furthermore, the various components may be combined in various ways. It is also important that, as technology develops, many of the components described may be replaced by equivalent components that emerge later.

Claims

1. A method for point cloud fusion, the method comprising: Receiving a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight based on the reflectivity and / or color of each point; Mapping the first frame of point cloud into a grid including a plurality of cells; Performing a first weighted average on each point in the first frame of point cloud mapped into each cell of the grid to obtain a first weighted point for the cell, the first weighted average being performed based on the weights of the respective points in the first frame of point cloud mapped into the cell of the grid, and the first weighted point having a weight based on the weights of the respective points; Receiving a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; Mapping the second frame of point cloud into the grid including the first weighted points; And Performing a second weighted average on each point in the second frame of point cloud mapped into each cell of the grid and the first weighted point located in the cell to obtain a second weighted point for the cell, the second weighted average being performed based on the weights of the respective points in the second frame of point cloud mapped into the cell of the grid and the weight of the first weighted point.

2. The method according to claim 1, characterized in that, The sensor includes at least one of the following: lidar, stereo camera, or time-of-flight camera.

3. The method according to claim 1, characterized in that, Each cell in the grid has a cubic shape and the same size, and the side length of the cube is 2 - 10 cm.

4. The method according to claim 1, characterized in that, Each cell in the grid has a different size.

5. A device for point cloud fusion, the device comprising: A receiving unit configured to receive a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight based on the reflectivity and / or color of each point; A mapping unit configured to map the first frame of point cloud into a grid including a plurality of cells; And A calculation unit configured to perform a first weighted average on each point in the first frame of point cloud mapped into each cell of the grid to obtain a first weighted point for the cell, the first weighted average being performed based on the weights of the respective points in the first frame of point cloud mapped into the cell of the grid, and the first weighted point having a weight based on the weights of the respective points; Wherein the receiving unit is further configured to receive a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; Wherein the mapping unit is further configured to map the second frame of point cloud into the grid including the first weighted points; and Wherein the calculation unit is further configured to perform a second weighted average on each point in the second frame of point cloud mapped into each cell of the grid and the first weighted point located in the cell to obtain a second weighted point for the cell, the second weighted average being performed based on the weights of the respective points in the second frame of point cloud mapped into the cell of the grid and the weight of the first weighted point.

6. A device for point cloud fusion, the device comprising: A memory that stores a computer program; And A processor coupled to the memory, the computer program when executed by the processor implementing the following steps: Receive a first frame of point cloud from a sensor, each point in the first frame of point cloud having an associated weight based on the reflectivity and / or color of each point; Map the first frame of point cloud into a grid including a plurality of cells; Perform a first weighted average on each point in the first frame of point cloud that is mapped into each cell of the grid to obtain a first weighted point for the cell, the first weighted average being performed based on the weights of the respective points in the first frame of point cloud that are mapped into the cell of the grid, and the first weighted point having a weight based on the weights of the respective points; Receive a second frame of point cloud from the sensor, each point in the second frame of point cloud having an associated weight; Map the second frame of point cloud into the grid including the first weighted points; And Perform a second weighted average on each point in the second frame of point cloud that is mapped into each cell of the grid and the first weighted point located in the cell to obtain a second weighted point for the cell, the second weighted average being performed based on the weights of the respective points in the second frame of point cloud that are mapped into the cell of the grid and the weight of the first weighted point.

7. A vehicle, comprising: Sensor; And An apparatus for point cloud fusion according to any one of claims 5-6.

8. A non-transitory computer-readable medium storing a computer program which, when executed by a processor, performs the method according to any one of claims 1-4.

Images