Point cloud processing method and device

Abstract

The invention discloses a point cloud processing method and device. The method is applied to a field programmable gate array (FPGA) chip, and comprises the following steps: acquiring a plurality of grids corresponding to point cloud data; for each grid, generating amplified feature data of each point in parallel according to the coordinate features corresponding to the plurality of points in the grid; storing the amplified feature data in a first storage resource in the FPGA chip; extracting features of the amplified feature data according to weight parameters stored in a second storage resource in the FPGA chip to obtain extracted feature data; generating a feature image according to the extracted feature data; and storing the feature image in a second storage resource. According to the embodiment of the invention, the overall delay performance can be improved, the power consumption can be reduced, and the storage and calculation integrated acceleration of the high-performance FPGA chip is realized, so that the point cloud processing efficiency is improved.

Description

Technical Field

[0001] This application belongs to the technical field of data processing, and particularly relates to a point cloud processing method and apparatus. Background Art

[0002] Currently, most point cloud processing is performed through traditional computing platforms for inference calculation, such as a CPU (Central Processing Unit) or a GPU (graphics processing unit). For the point cloud processing method based on the CPU platform, due to its serial processing method, the processing speed is slow, and point cloud data usually contains a large number of points and complex calculation operations, making the CPU often unable to meet the requirements of high efficiency during the processing; although the GPU has advantages in parallel computing compared to the CPU, the processing of point cloud data still requires a large amount of computing resources and time.

[0003] In addition, in related technologies, the off-chip memory of an FPGA (Field Programmable Gate Array) is used to store and read / write data, such as DDR (Double Data Rate SDRAM). Since data transmission needs to be performed through an I / O interface (Input / Output), it results in a relatively high time delay and high power consumption, thereby leading to poor point cloud processing effects, especially for point cloud processing in the fields of robotics and autonomous driving. Summary of the Invention

[0004] Embodiments of this application provide a point cloud processing method and apparatus, which can achieve the in-memory computing acceleration of a high-performance FPGA chip and improve the efficiency of point cloud processing.

[0005] In a first aspect, embodiments of this application provide a point cloud processing method, which is applied to a Field Programmable Gate Array (FPGA) chip. The method includes:

[0006] Obtain a plurality of grids corresponding to point cloud data;

[0007] For each of the grids, generate amplified feature data for each point in parallel according to the coordinate features corresponding to the points in the grid;

[0008] Store the amplified feature data in a first storage resource inside the FPGA chip;

[0009] Extract the features of the amplified feature data according to the weight parameters pre-trained and stored in a second storage resource inside the FPGA chip to obtain the extracted feature data;

[0010] Generate a feature image based on the extracted feature data;

[0011] Store the feature image in the second storage resource.

[0012] In a second aspect, an embodiment of the present application provides a point cloud processing device, which is applied to a field programmable gate array (FPGA) chip. The device includes:

[0013] An acquisition module, configured to acquire a plurality of grids corresponding to point cloud data;

[0014] A first generation module, configured to, for each of the grids, generate amplified feature data for each point in parallel according to the coordinate features corresponding to the multiple points within the grid;

[0015] A first storage module, configured to store the amplified feature data in a first storage resource inside the FPGA chip;

[0016] An extraction module, configured to extract features of the amplified feature data according to weight parameters pre-trained and stored in a second storage resource inside the FPGA chip to obtain the extracted feature data;

[0017] A second generation module, configured to generate a feature image based on the extracted feature data;

[0018] A second storage module, configured to store the feature image in the second storage resource.

[0019] The point cloud processing method and device according to the embodiment of the present application are deployed on a high-performance FPGA chip. The coordinate features corresponding to multiple points in each grid corresponding to the point cloud data are processed in parallel to obtain the amplified feature data for each point and the extracted feature data corresponding to the amplified feature data. Finally, a feature image is obtained, and the corresponding data is respectively stored in the first storage resource and the second storage resource inside the FPGA chip for subsequent calculation indexing. Since the internal storage resources of the high-performance FPGA chip are used instead of the peripheral storage resources of the FPGA to store the point cloud feature data after calculation, the overall latency performance is improved, the power consumption is reduced, the memory-computation integration acceleration of the high-performance FPGA chip is realized, and thus the point cloud processing efficiency is improved. Description of the Drawings

[0020] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a schematic flowchart of a point cloud processing method provided by an embodiment of the present application;

[0022] Figure 2 It is a schematic flowchart of a method for obtaining multiple meshes corresponding to point cloud data provided by an embodiment of the present application;

[0023] Figure 3 It is a schematic structural diagram of a point cloud processing device provided by an embodiment of the present application;

[0024] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0025] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than limiting the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present application by showing examples of the present application.

[0026] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0027] Currently, most point cloud processing is performed through traditional computing platforms for inference calculation, such as a CPU (Central Processing Unit) or a GPU (graphics processing unit).

[0028] The point cloud processing method based on the CPU platform has a slow processing speed due to its serial processing method. Point cloud data usually contains a large number of points and complex calculation operations, making it often unable to meet the requirements of real-time and high efficiency during CPU processing. In addition, since the CPU platform is a general computing platform, there is a lack of customized hardware support for point cloud preprocessing tasks. The computational characteristics and requirements of point cloud preprocessing algorithms are different from traditional general computing tasks, so the potential of the hardware cannot be fully utilized, limiting the performance and efficiency of the algorithms.

[0029] Although the GPU has advantages in parallel computing compared to the CPU, its power consumption is usually higher than that of the CPU and FPGA. For mobile devices and embedded systems, energy consumption is an important consideration, so the use of GPUs may be limited in these scenarios.

[0030] The FPGA is more flexible than the GPU because it can be reprogrammed to perform different types of tasks. However, in related technologies, using off-chip memory of the FPGA (Field Programmable Gate Array) to store and read / write data, such as DDR (Double Data Rate SDRAM), results in high time latency and high power consumption due to the need to transfer data through the I / O interface (Input / Output), thus leading to poor point cloud processing effects, especially for point cloud processing in the fields of robotics and autonomous driving.

[0031] To solve the problems of related technologies, the embodiments of the present application provide a point cloud processing method and device, which can support the preprocessing of 4D millimeter-wave radar point cloud data.

[0032] The following combines the accompanying drawings to illustrate in detail the point cloud processing method provided by the embodiments of the present application through specific embodiments and their application scenarios.

[0033] Figure 1 The flowchart of the point cloud processing method according to the embodiments of the present application is shown. As Figure 1 shown, the point cloud processing method may specifically include the following steps:

[0034] S101. Obtain multiple grids corresponding to the point cloud data;

[0035] S102. For each grid, generate augmented feature data for each point in parallel according to the coordinate features corresponding to the multiple points within the grid;

[0036] S103. Store the augmented feature data in the first storage resource inside the FPGA chip;

[0037] S104. Extract the features of the augmented feature data according to the weight parameters pre-trained and stored in the second storage resource inside the FPGA chip to obtain the extracted feature data;

[0038] S105. Generate a feature image according to the extracted feature data;

[0039] S106. Store the feature image in the second storage resource.

[0040] Thus, by being deployed on a high-performance FPGA chip, parallel data processing is performed on the coordinate features corresponding to multiple points in each grid corresponding to the point cloud data to obtain the augmented feature data of each point and the extracted feature data corresponding to the augmented feature data. Finally, a feature image is obtained, and the corresponding data is respectively stored in the first storage resource and the second storage resource inside the FPGA chip for subsequent calculation indexing. Since the internal storage resource of the high-performance FPGA chip is used instead of the peripheral storage resource of the FPGA to store the point cloud feature data after calculation, the overall latency performance is improved, the power consumption is reduced, the memory-computation integrated acceleration of the high-performance FPGA chip is realized, and thus the point cloud processing efficiency is improved.

[0041] The specific implementation manners of the above steps are introduced below.

[0042] In some embodiments, for the specific implementation manner of S101, reference can be made to Figure 2 . As Figure 2 shown, obtaining multiple grids corresponding to the point cloud data may specifically include the following steps: S201 to S203.

[0043] S201. Determine the three-dimensional space range corresponding to the point cloud data according to the coordinate features corresponding to multiple points in the point cloud data.

[0044] In some embodiments, the three-dimensional space range corresponding to the point cloud data at least includes: the three-dimensional space range divided by the Cartesian coordinate parameters pointmax(x, y, z) and pointmin(x, y, z) of the maximum point and the minimum point among multiple points of the point cloud data. Specifically, by traversing the coordinate features corresponding to multiple points in the point cloud data, the maximum point and the minimum point among multiple points can be determined, and the coordinate features of the maximum point and the minimum point are respectively determined as pointmax(x, y, z) and pointmin(x, y, z). Taking the determination of pointmax(x, y, z) as an example, for the coordinate features of the currently traversed point, if it is greater than the coordinate features of the previously traversed point, the coordinate features of the current point are retained while the coordinate features of the previous point are discarded; otherwise, the coordinate features of the previous point are retained, and so on until the last point in the point cloud data is traversed, and finally pointmax(x, y, z) is obtained.

[0045] S202. Calculate the resolution of the grids corresponding to the first direction, the second direction, and the third direction respectively within the three-dimensional space range according to the preset grid size and the three-dimensional space range.

[0046] It should be noted that if the first direction, the second direction, and the third direction are the X, Y, and Z directions in the three-dimensional space respectively, then the resolution sizes of the grids in the X, Y, and Z directions are defined as H×W×Z. Optionally, the grid is a columnar body with the same height as the space of the point cloud data, that is, the grid resolution in the Z direction is 1, so the grid resolution size is H×W×1.

[0047] For the grid resolutions in the X and Y directions, taking the X direction as an example, it can be specifically determined according to the following formula (1):

[0048]

[0049] Among them, pillarsize(x) represents the preset grid size corresponding to the X direction.

[0050] S203. Divide the three-dimensional space range according to the resolutions of the grids corresponding to the first direction, the second direction, and the third direction respectively to obtain a plurality of grids.

[0051] In this way, according to the coordinate characteristics corresponding to the multiple points in the point cloud data and the preset grid size, the three-dimensional space corresponding to the collected point cloud data is divided into a plurality of grids.

[0052] Further, in some embodiments, after S101 and before S102, the point cloud data and the number of point clouds in each grid can be respectively stored in the memory unit matching the grid in the second storage resource, and the address numbers of the memory units correspond one by one to the grids; according to the target address number, read the target memory unit matching the target address number in the second storage resource to obtain the target point cloud data and the target number of point clouds stored in the target memory unit; store the target point cloud data and the target number of point clouds in the first storage resource; read the first storage resource.

[0053] Optionally, for the first storage resource, it can be set that the maximum number of generated grids that can be stored is 4000 at most, and the maximum number of points in the grid is 16 at most. It should be noted that the set parameters can be adjusted and determined according to the specific application scenario, and this embodiment does not limit this.

[0054] Optionally, one of the first storage resource and the second storage resource inside the FPGA chip is a BRAM (Block RAM), and the other is a URAM (Ultra RAM). In this way, by reasonably utilizing the storage and processing capabilities of the high-performance FPGA on-chip resources, the inference acceleration of the efficient point cloud preprocessing algorithm can be realized, the computing performance is improved, and the access latency is low. Moreover, since the high-performance FPGA on-chip storage resources (such as BRAM, URAM, etc.) do not require additional I / O interfaces for data transmission with the outside, the overall system power consumption is low.

[0055] As an alternative embodiment, S102 can be implemented in the following manner: for each grid, according to the coordinate features corresponding to multiple points within the grid, the first point cloud feature difference and the second point cloud feature difference of each point are calculated in parallel, where the first point cloud feature difference is the difference between the coordinate feature of each point and the average value of the coordinates of multiple points within the grid, and the second point cloud feature difference is the difference between the coordinate feature of each point and the coordinates of the center point of the grid; the coordinate feature, the first point cloud feature difference, and the second point cloud feature difference are determined as the augmented feature data.

[0056] During specific implementation, for each grid, according to the coordinate features corresponding to multiple points within the grid, the average value of the coordinates of multiple points within the grid and the coordinates of the center point of the grid are calculated in parallel; according to the coordinate features corresponding to multiple points within the grid, the average value of the coordinates of multiple points within the grid, and the coordinates of the center point of the grid, the first point cloud feature difference and the second point cloud feature difference of each point are calculated in parallel.

[0057] That is to say, it is necessary to calculate the average value of the coordinate features of all points within each grid, and use parallel pipelined recursion to calculate the average value of the coordinate features of all points in the column. Specifically, for the coordinate average value, taking the x coordinate as an example, it can be calculated according to the following formulas (2) and (3):

[0058]

[0059] Among them, when calculating the average value of the coordinates of the first point within a certain grid, n is 1, and the average value of the x coordinate of the first point can be obtained from formula (2). The average value of the x coordinates of subsequent points can be recursively obtained from formula (3). In this way, when all point data is sequentially input and completed, the average value can also be calculated by pipeline. It can be understood that the solution process of the y and z coordinate average values is the same as that of the x coordinate and realizes parallel pipelining.

[0060] Thus, by subtracting the coordinate mean from the coordinate features of each point, the first point cloud feature differences \(x_c\), \(y_c\), and \(z_c\) are calculated. At the same time, the coordinate values of the center points of each grid are calculated in parallel. Specifically, the coordinate feature value of pointmax in formula (1) can be replaced with the coordinate value of the currently calculated point pointcurrent, and the result is rounded down to obtain the corresponding grid coordinate value. Then, the origin coordinate value corresponding to the grid is solved backward from the grid coordinate value, and added to pillarsize / 2, and the coordinate values of the grid center points can be obtained in parallel. Taking the x dimension as an example, the calculation formula is as shown in formula (4).

[0061]

[0062] Then, subtract the coordinate features from the center point coordinates to obtain the second point cloud feature differences \(x_p\), \(y_p\), and \(z_p\). Therefore, through parallel calculation, the feature data of 10 dimensions corresponding to each point are obtained, namely: the original features (coordinate features \(x\), \(y\), \(z\) and velocity doppler), and the first point cloud feature differences \(x_c\), \(y_c\), \(z_c\), and the second point cloud feature differences \(x_p\), \(y_p\), \(z_p\). That is to say, the number of point cloud features is 10, realizing the amplification of point cloud features. Further, in S103, the amplified feature data is stored in the first storage resource. Taking the number of generated grids stored as 4000 and the number of points in the grid as 16 as an example, the size of the point cloud data is 4000×16×10.

[0063] In an embodiment, in S104, linear feature extraction is performed on the feature data based on the weight parameters pre-trained and stored in the second storage resource to obtain the extracted feature data. Among them, the pre-trained weight parameters are matrices of a preset order, and the preset order is determined according to the number of point cloud features and the number of preset expansion dimension channels. For example, the size of the pre-trained weight parameters is 10×64. In this way, after linear feature extraction including linear multiplication, the feature dimension becomes 64 dimensions.

[0064] Further, in some embodiments, batch normalization BN (Batch Normalization) is performed on the extracted feature data, and the activation function (such as relu activation function) is used to remove the negative values in the feature dimension of the extracted feature data, and max pooling is performed.

[0065] In some alternative embodiments, in S105, for the 64-dimensional features of 4000 grids obtained, the 64-dimensional features are mapped back to the corresponding grid positions using the mapping coordinates of the grid columns, and finally a pseudo-image feature map H×W×64 is generated. Thus, in S106, the feature image is stored in the second storage resource.

[0066] It should be noted that some embodiments of the present application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the above embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0067] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present application further provides a point cloud processing device 300, which is applied to a field programmable gate array (FPGA) chip.

[0068] As Figure 3 shown, the point cloud processing device 300 may include:

[0069] An acquisition module 301, configured to acquire a plurality of grids corresponding to point cloud data;

[0070] A first generation module 302, configured to, for each grid, generate amplification feature data of each point in parallel according to the coordinate features corresponding to the multiple points within the grid;

[0071] A first storage module 303, configured to store the amplification feature data in a first storage resource inside the FPGA chip;

[0072] An extraction module 304, configured to extract features of the amplification feature data according to weight parameters pre-trained and stored in a second storage resource inside the FPGA chip to obtain the extracted feature data;

[0073] A second generation module 305, configured to generate a feature image according to the extracted feature data;

[0074] A second storage module 306, configured to store the feature image in the second storage resource.

[0075] In some embodiments, the acquisition module 301 is specifically configured to: determine a three-dimensional space range corresponding to the point cloud data according to the coordinate features corresponding to the multiple points in the point cloud data; calculate the resolution of the grids corresponding to the first direction, the second direction, and the third direction respectively within the three-dimensional space range according to a preset grid size and the three-dimensional space range; divide the three-dimensional space range according to the resolution of the grids corresponding to the first direction, the second direction, and the third direction respectively to obtain a plurality of grids.

[0076] In some embodiments, the point cloud processing device 300 further includes a reading module ( Figure 3(not shown in the figure) for: storing the point cloud data and the number of point clouds in each grid into the memory units matching the grids in the second storage resource respectively, where the address numbers of the memory units correspond one by one to the grids; reading the target memory unit in the second storage resource that matches the target address number according to the target address number, to obtain the target point cloud data and the target number of point clouds stored in the target memory unit; storing the target point cloud data and the target number of point clouds in the first storage resource; and reading the first storage resource.

[0077] In some embodiments, the first generation module 302 is specifically configured to: for each grid, calculate the first point cloud feature difference and the second point cloud feature difference of each point in parallel according to the coordinate features corresponding to the multiple points in the grid, where the first point cloud feature difference is the difference between the coordinate feature of each point and the average coordinate of the multiple points in the grid, and the second point cloud feature difference is the difference between the coordinate feature of each point and the coordinate of the center point of the grid; and determine the coordinate feature, the first point cloud feature difference, and the second point cloud feature difference as the augmented feature data.

[0078] In some embodiments, the first generation module 302 is further specifically configured to: for each grid, calculate the average coordinate of the multiple points in the grid and the coordinate of the center point of the grid in parallel according to the coordinate features corresponding to the multiple points in the grid; and calculate the first point cloud feature difference and the second point cloud feature difference of each point in parallel according to the coordinate features corresponding to the multiple points in the grid, the average coordinate of the multiple points in the grid, and the coordinate of the center point of the grid.

[0079] In some embodiments, the point cloud processing device 300 further includes a normalization module ( Figure 3 (not shown in the figure) for: performing batch normalization on the extracted feature data, removing the negative values in the feature dimensions of the extracted feature data by using an activation function, and performing max pooling.

[0080] In some embodiments, the grid is a columnar body having the same height as the space of the point cloud data.

[0081] In some embodiments, one of the first storage resource and the second storage resource is a BRAM, and the other is a URAM.

[0082] In some embodiments, the pre-trained weight parameter is a matrix of a preset order, and the preset order is determined according to the number of point cloud features and the number of preset extended dimension channels.

[0083] It should be noted that, for the convenience of description, when describing the above device, various modules are described separately according to their functions. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0084] The device of the above embodiment is used to implement the corresponding point cloud processing method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be elaborated here.

[0085] Based on the same technical concept, corresponding to the method of any of the above embodiments, the present application also provides an electronic device.

[0086] Figure 4 Fig. shows a more specific schematic diagram of the hardware structure of the electronic device provided in this embodiment.

[0087] In the electronic device 400, it may include a processor 401 and a memory 402 storing computer program instructions.

[0088] Specifically, the above-mentioned processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0089] The memory 402 may include a mass storage for data or instructions. By way of example and not limitation, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 402 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 402 may be internal or external to the integrated gateway disaster recovery device. In a specific embodiment, the memory 402 is a non-volatile solid state memory.

[0090] In a specific embodiment, the memory may include a read only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described in reference to the method according to one aspect of the present application.

[0091] The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement any one of the point cloud processing methods in the above embodiments.

[0092] In some examples, the electronic device 400 may further include a communication interface 403 and a bus 410. Among them, as Figure 4 shown, the processor 401, the memory 402, and the communication interface 403 are connected through the bus 410 to complete communication with each other.

[0093] The communication interface 403 is mainly used to implement communication between various modules, devices, units, and / or devices in the embodiments of the present application.

[0094] The bus 410 includes hardware, software, or both, and couples the components of the online data flow metering device to each other. By way of example and not limitation, the bus 410 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses or a combination of two or more of these. In suitable cases, the bus 410 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0095] Exemplarily, the electronic device 400 may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc.

[0096] Based on the same technical concept, corresponding to any of the above method embodiments, the present application also provides a non-transitory computer-readable storage medium. Computer program instructions are stored on the computer-readable storage medium; when the computer program instructions are executed by a processor, any of the above point cloud processing methods in the embodiments is implemented. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, etc.

[0097] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present application further provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to execute a point cloud processing method. Corresponding to the execution subject of each step in the respective embodiments of the point cloud processing method, the processor that executes the corresponding step can belong to the corresponding execution subject.

[0098] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, the detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0099] The functional blocks shown in the above structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted via a data signal carried in a carrier wave on a transmission medium or a communication link. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0100] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0101] As described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the present application. It should be understood that each block in the flowchart and / or block diagram, and the combination of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general purpose processor, a special purpose processor, a special application processor or a field programmable logic circuit. It should also be understood that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware performing the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0102] The above is only the specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present application, and these modifications or substitutions should all be covered within the protection scope of the present application.

Claims

1. A point cloud processing method, characterized in that: Applied to a field programmable gate array (FPGA) chip, the method comprises: Get multiple grids corresponding to the point cloud data; For each of the grids, according to the coordinate features corresponding to the multiple points in the grid, the amplified feature data of each point is generated in parallel; Storing the amplified feature data in a first storage resource inside the FPGA chip; Extracting features of the amplified feature data according to pre-trained weight parameters stored in a second storage resource inside the FPGA chip to obtain extracted feature data; Generate a feature image according to the extracted feature data; The feature image is stored in the second storage resource.

2. The method according to claim 1, characterized in that The step of obtaining a plurality of grids corresponding to the point cloud data includes: According to the coordinate features corresponding to the multiple points in the point cloud data, the three-dimensional space range corresponding to the point cloud data is determined; Calculate the resolution of the grids corresponding to the first direction, the second direction and the third direction in the three-dimensional space range according to the preset grid size and the three-dimensional space range; The three-dimensional space range is divided according to the resolutions of the grids corresponding to the first direction, the second direction and the third direction, respectively, to obtain a plurality of grids.

3. The method according to claim 1, characterized in that: Before generating, for each of the grids, augmented feature data of each point in parallel according to coordinate features corresponding to each of the multiple points in the grid, the method further includes: The point cloud data and the number of point clouds in each grid are stored in the memory unit matching the grid in the second storage resource, and the address number of the memory unit corresponds to the grid one by one; According to the target address number, read the target memory unit in the second storage resource that matches the target address number, and obtain the target point cloud data and the number of target point clouds stored in the target memory unit; Storing the target point cloud data and the target point cloud quantity in the first storage resource; The first storage resource is read.

4. The method according to claim 1, characterized in that For each of the grids, according to the coordinate features corresponding to the multiple points in the grid, the augmented feature data of each point is generated in parallel, including: For each of the grids, according to the coordinate features corresponding to the multiple points in the grid, a first point cloud feature difference and a second point cloud feature difference of each point are calculated in parallel, wherein the first point cloud feature difference is a difference between the coordinate feature of each point and the average value of the coordinates of the multiple points in the grid, and the second point cloud feature difference is a difference between the coordinate feature of each point and the coordinates of the center point of the grid; The coordinate feature, the first point cloud feature difference and the second point cloud feature difference are determined as augmented feature data.

5. The method according to claim 4, characterized in that For each of the grids, according to the coordinate features corresponding to the multiple points in the grid, the first point cloud feature difference and the second point cloud feature difference of each point are calculated in parallel, including: For each of the grids, according to the coordinate features corresponding to the multiple points in the grid, the mean values ​​of the coordinates of the multiple points in the grid and the coordinates of the center point of the grid are calculated in parallel; According to the coordinate features corresponding to the multiple points in the grid, the coordinate mean of the multiple points in the grid and the coordinates of the center point of the grid, the first point cloud feature difference and the second point cloud feature difference of each point are calculated in parallel.

6. The method according to claim 1, characterized in that Before generating a feature image according to the extracted feature data, the method further includes: The extracted feature data are batch normalized, and negative values ​​in the feature dimension of the extracted feature data are removed using an activation function, and maximum pooling is performed.

7. The method according to any one of claims 1 to 6, characterized in that: The grid is a columnar body with the same height as the space of the point cloud data.

8. The method according to any one of claims 1 to 6, characterized in that: One of the first storage resource and the second storage resource is a BRAM and the other is a URAM.

9. The method according to claim 1, characterized in that: The weight parameters obtained by the pre-training are matrices of a preset order, and the preset order is determined according to the number of point cloud features and the number of preset expansion dimension channels.

10. A point cloud processing device, characterized in that: Applied to a field programmable gate array (FPGA) chip, the device comprises: An acquisition module, used to acquire multiple grids corresponding to point cloud data; A first generating module is used for generating, for each of the grids, the amplified feature data of each point in parallel according to the coordinate features corresponding to each of the multiple points in the grid; A first storage module, used for storing the amplified feature data in a first storage resource inside the FPGA chip; An extraction module, used to extract the features of the amplified feature data according to the pre-trained weight parameters stored in the second storage resource inside the FPGA chip to obtain extracted feature data; A second generating module is used to generate a feature image according to the extracted feature data; The second storage module is used to store the feature image in the second storage resource.

Images