Square Euclidean distance calculation method for realizing in-memory acceleration point cloud in 2T0C dynamic random access memory array

Abstract

The invention discloses a square Euclidean distance calculation method for realizing in-memory acceleration point cloud by a 2T0C dynamic random access memory array, and belongs to the field of novel in-memory calculation technology and three-dimensional point cloud recognition. According to the method, a point cloud storage array is formed based on a 2T0C DRAM array, when the 2T0C DRAM array stores and inputs point cloud data, a point multiplication result of a vector M and a vector N in a three-dimensional space, a point multiplication result of the vector M and the vector M and a point multiplication result of the vector N and the vector N are obtained, and summation is carried out to obtain a square Euclidean distance result of any two points in the three-dimensional space. Compared with a traditional CMOS accelerator under the Von Noremann architecture, the Von Noremann architecture CMOS accelerator has the advantages that hardware overhead is greatly reduced, data handling is reduced, the processing speed and energy efficiency can be remarkably improved, and universal and efficient hardware acceleration basic configuration can be provided for various point cloud neural network algorithms.

Description

Technical Field

[0001] The present invention belongs to the field of novel in-memory computing technology and three-dimensional point cloud recognition, and specifically relates to a method for calculating the squared Euclidean distance of an in-memory accelerated point cloud using a 2T0C dynamic random access memory array. Background Art

[0002] As an important means of three-dimensional environmental perception, three-dimensional point cloud recognition technology plays an important role in many fields such as autonomous driving, robot navigation, virtual reality, augmented reality, industrial inspection and smart cities, and has irreplaceable application value. Compared with traditional two-dimensional images, three-dimensional point cloud data directly obtains spatial coordinate information through lidar or depth camera, which can accurately characterize the geometric structure and spatial relationship of the target. In particular, the recognition robustness under complex lighting conditions and dynamic scenes is significantly better than that of visual solutions. At present, a variety of neural networks have been developed in the field for efficient three-dimensional point cloud recognition. Among them, point-based point cloud neural networks (such as PointNet, PointNet++, etc.) have become the mainstream in the current research field due to their direct and efficient processing of disordered point clouds and excellent network performance, omitting additional processing steps such as voxelization.

[0003] Existing point cloud accelerators frequently access raw point cloud data stored in external memory when performing downsampling operations. Because point cloud data is inherently a sparse, unstructured set of 3D coordinates, it lacks good spatial or cache locality, resulting in low cache hit rates and high data access bandwidth usage.

[0004] Currently, proposals have been made to accelerate matrix-vector multiplication using analog in-memory computing techniques (e.g., those based on resistive random access memory) to address the high computational overhead of feature extraction. However, this requires converting the original point cloud from its original floating-point data type to the analog domain for computation, introducing additional data conversion overhead. Therefore, utilizing in-memory computing techniques to optimize the design of efficient operators for the key steps of point cloud recognition, such as downsampling and feature calculation, and to achieve matching of data structure types, is of great research significance. Summary of the Invention

[0005] The present invention provides a 2TOC dynamic random access memory (DRAM) array to implement in-memory accelerated squared Euclidean distance calculation of point clouds, which can significantly improve processing speed and energy efficiency, and provide a universal and efficient hardware acceleration basic configuration for various point cloud neural network algorithms.

[0006] To achieve the above objectives, the present invention provides the following technical solutions:

[0007] A method for calculating the squared Euclidean distance of an in-memory accelerated point cloud using a 2TOC dynamic random access memory array comprises the following steps:

[0008] 1) A 2T0C DRAM array is used to store point cloud data. Each 2T0C DRAM cell consists of a write transistor and a read transistor. The drain of the write transistor is connected to the gate of the read transistor to form an information storage node SN. Each node stores 1 bit of information.

[0009] 2) The sign bit, exponent bit, and mantissa bit of the point cloud data are stored separately. The mantissa bit of the same point cloud data is stored separately in multiple columns of the 2T0C DRAM array. All the mantissa bit information constitutes the mantissa bit storage array. The sign bit and exponent bit of the same point cloud data are stored together in the same column of 2T0C DRAM cells in the 2T0C DRAM array.

[0010] 3) Applying a read signal to the column driver, reading the mantissa of the coordinates of point M / point N in a certain direction, and inputting it bit by bit into the RBL of the column where the coordinates of point N / point M in that direction are located, obtaining the point product of the mantissa and mantissa bits through the sense amplifier and shift adder circuit, combining the original sign bit and exponent bit of the floating-point numbers of point M and point N, performing exponent shift alignment and sign adjustment, and obtaining the intermediate product result;

[0011] 4) After completing the input of all the last digits of point M and point N over multiple time periods, all the intermediate product results of vector M and vector N in the three-dimensional space are obtained, and the sum is calculated to obtain the squared Euclidean distance between point M and point N in the three-dimensional space.

[0012] Furthermore, the mantissas of different point cloud data in step 2) are stored column by column.

[0013] Furthermore, in step 3), when the information stored in SN is 0, when a read voltage of 1 or a read voltage of 0 is input through RBL, the output result of RWL is 0; when the information stored in SN is 1, a read voltage of 1 is input through RBL, and RBL output is 1; when a read voltage of 0 is input through RBL, RBL output is 0, thereby realizing a 1-bit data multiplication operation.

[0014] Furthermore, the intermediate product result in step 3) is outputted by the shift adder and then temporarily stored in the register circuit, waiting for the intermediate product results of other bits.

[0015] Furthermore, step 4) adds a cache setting. After obtaining the squared Euclidean distance result between point M and point N, it is output to the cache for storage, waiting for the Euclidean distance result between point M and the remaining points.

[0016] Furthermore, a square Euclidean distance comparator is added to compare the square Euclidean distance results of the M points to achieve farthest point sampling and adjacent sampling.

[0017] The present invention has the following advantages compared with the prior art:

[0018] The present invention constructs a point cloud storage array based on a 2T0C DRAM array. While storing the input point cloud through the 2T0C DRAM array, some square Euclidean distance in-memory calculations can be completed in the storage array. Compared with the CMOS accelerator under the traditional von Neumann architecture, it greatly reduces hardware overhead and data handling, and can significantly improve processing speed and energy efficiency. It can provide a universal and efficient hardware acceleration basic configuration for various point cloud neural network algorithms, and is an effective in-memory acceleration solution for point cloud recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a 1-bit multiplication operation implemented by a 2T0C DRAM cell according to a specific embodiment of the present invention;

[0020] Figure 2 FIG. 2 is a schematic diagram of a 2T0CDRAM array structure and related peripheral circuits according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described below with reference to the accompanying drawings through examples, which however do not limit the scope of the present invention in any way.

[0022] The present invention provides a method for calculating the squared Euclidean distance of an in-memory accelerated point cloud using a 2TOC dynamic random access memory array. The specific steps include the following:

[0023] 1) 2T0C DRAM array is used to store input point cloud, such as Figure 1 As shown, each 2T0C DRAM cell consists of a write transistor and a read transistor. The drain of the write transistor is connected to the gate of the read transistor to form an information storage node SN, and each node stores one bit of information. When writing information, the write bit line (WBL) connected to the write transistor gate is turned on, and a 0 / 1 voltage is applied to the write word line (WWL) connected to the write transistor source. This changes the information at SN, completing the writing process of each bit of data.

[0024] The 2T0C DRAM array of the present invention performs 1-bit multiplication during read operations. When the information stored in SN is 0, a read voltage of 1 or 0 is input through RBL, and the output of RWL is 0. When the information stored in SN is 1, a read voltage of 1 is input through RBL, and the output is 1; when a read voltage of 0 is input through RBL, the output is 0, thus achieving 1-bit data multiplication.

[0025] 2) If Figure 2As shown, the sign bit, exponent bit, and mantissa bit of the point cloud data are stored separately, wherein the mantissa bit of the same point cloud data is stored separately in multiple columns of the 2T0C DRAM array, and the mantissa bits of different point cloud data are stored sequentially column by column. All the mantissa bit information constitutes the mantissa bit storage array, and the sign bit and exponent bit of the same point cloud data are stored together in the same column of the 2T0C DRAM array, constituting the storage array of the remaining information.

[0026] 3) When the 2T0C DRAM array performs squared Euclidean distance calculations, it first expands the distance between M points (vectors) and N points (vectors) in three-dimensional space using the squared Euclidean distance formula:

[0027]

[0028] For any of the X, Y, and Z directions, the distance between two points can be expressed as follows:

[0029]

[0030] Read the mantissa of the coordinate of point M in a certain direction (for example, the X direction) and input it bit by bit into the RBL of the column where the coordinate of point N in that direction (for example, the X direction) is located. When each bit is input, the information of each bit of N points simultaneously completes the above 1-bit multiplication operation to obtain multiple intermediate product results. The specific operation is as follows:

[0031] 3-1) First, a read signal is applied to the column driver to read the mantissa of the coordinates of point M. After passing through the sense amplifier, the output signal is re-input into the column driver. The column driver then inputs the signal bit by bit into the RBL of the column where the coordinates of point N are located. For each bit input, the 1-bit multiplication operation described above is performed simultaneously on each bit of the N points. k represents the number of mantissa bits, which is determined by the floating-point format. For example, in the half-precision floating-point format (FP16), k is 9.

[0032] m0·n0, m0·n1, m0·n2,…, m0·n k

[0033] 3-2) The multiplication result of the 1-bit mantissa of the M-point and the N-point mantissa is obtained through the sense amplifier and shift adder circuit:

[0034]

[0035] The multiplication result is output through the shift adder and temporarily stored in the register circuit, waiting for the calculation results of other bits. After completing the input of all the mantissa bits of point M over multiple time cycles, the multiplication result of each mantissa bit of point M and the mantissa bit of point N is obtained:

[0036]

[0037] 3-3) After the calculation is completed, all the temporarily stored information is read from the register and transferred to the shift adder again. The above results are shifted and added together with the bit information of each point M. The shift adder is used (the shift principle is consistent with the multiplication of the 1-bit mantissa bit of point M and the mantissa bit of point N in the previous process, and the shift adder circuit can be reused) to obtain the multiplication result of the mantissa bit of point M and the mantissa bit of point N, that is:

[0038]

[0039] The dot product result of vector M and vector N in the X direction is input into the register for temporary storage. Similarly, the dot product result of vector M and vector M and the dot product result of vector N and vector N are obtained.

[0040] 3-4) The result of the point multiplication is input into the exponent shifter. The floating-point exponent bits of points M and N are read from the sign and exponent bit 2T0C DRAM array, and the floating-point numbers are aligned according to the exponent bit information. The result is then input into the signed adder / subtractor circuit. The sign bit information of points M and N is combined with the sign in the squared Euclidean distance calculation to obtain the intermediate product of vectors M and N, which is returned to the register for temporary storage:

[0041]

[0042] 4) Similarly, we can obtain the dot product of vector M and vector N in the Y and Z directions. That is, after completing the input of all the digits of points M and N over multiple time periods, we obtain the intermediate product results of vector M and vector N in three-dimensional space, the intermediate product results of vector M and vector M, and the dot intermediate product results of vector N and vector N. Summing them together, we obtain the squared Euclidean distance between points M and N in three-dimensional space:

[0043]

[0044] The specific embodiment of the present invention further adds a cache setting. After the Euclidean distance results between point M and point N are obtained, they can be output to the cache for storage, waiting for the Euclidean distance results between point M and the remaining points. Then, peripheral circuits such as a squared Euclidean distance comparator are used to implement algorithms such as farthest point sampling and adjacent sampling, thereby completing the downsampling link in the point cloud recognition task.

[0045] The above embodiments are only some preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any changes that adopt the design principles of the present invention and are made through non-creative work on this basis should fall within the scope of protection of the present invention.

Claims

1. A method for calculating the squared Euclidean distance of an in-memory accelerated point cloud using a 2TOC dynamic random access memory array, comprising: 1) A 2T0C DRAM array is used to store point cloud data. Each 2T0C DRAM cell consists of a write transistor and a read transistor. The drain of the write transistor is connected to the gate of the read transistor to form an information storage node SN. Each node stores 1 bit of information. 2) The sign bit, exponent bit, and mantissa bit of the point cloud data are stored separately. The mantissa bit of the same point cloud data is stored separately in multiple columns of the 2T0C DRAM array. All the mantissa bit information constitutes the mantissa bit storage array. The sign bit and exponent bit of the same point cloud data are stored together in the same column of 2T0C DRAM cells in the 2T0C DRAM array. 3) Applying a read signal to the column driver, reading the mantissa of the coordinates of point M / point N in a certain direction, and inputting it bit by bit into the RBL of the column where the coordinates of point N / point M in that direction are located, obtaining the point product of the mantissa and mantissa bits through the sense amplifier and shift adder circuit, combining the original sign bit and exponent bit of the floating-point numbers of point M and point N, performing exponent shift alignment and sign adjustment, and obtaining the intermediate product result; 4) After completing the input of all the last digits of point M and point N over multiple time periods, all the intermediate product results of vector M and vector N in the three-dimensional space are obtained, and the sum is calculated to obtain the squared Euclidean distance between point M and point N in the three-dimensional space.

2. The method for calculating squared Euclidean distance of an in-memory accelerated point cloud implemented by a 2TOC dynamic random access memory array according to claim 1, wherein: In step 2), the mantissas of different point cloud data are stored column by column.

3. The method for calculating squared Euclidean distance of an in-memory accelerated point cloud using a 2TOC dynamic random access memory array as claimed in claim 1, wherein: In step 3), when the information stored in SN is 0, when a read voltage of 1 or 0 is input through RBL, the output result of RWL is 0; when the information stored in SN is 1, a read voltage of 1 is input through RBL, and RBL output is 1; when a read voltage of 0 is input through RBL, RBL output is 0, and the intermediate product result is obtained.

4. The method for calculating squared Euclidean distance of an in-memory accelerated point cloud using a 2TOC dynamic random access memory array according to claim 1, wherein: The intermediate product result in step 3) is outputted by the shift adder and then temporarily stored in the register circuit, waiting for the intermediate product results of other bits.

5. The method for calculating squared Euclidean distance of an in-memory accelerated point cloud using a 2TOC dynamic random access memory array as claimed in claim 1, wherein: Step 4) Add a cache setting. After obtaining the squared Euclidean distance result between point M and point N, output it to the cache for storage and wait for the Euclidean distance result between point M and the remaining points.

6. The method for calculating squared Euclidean distance of an in-memory accelerated point cloud using a 2TOC dynamic random access memory array as claimed in claim 5, wherein: Add a square Euclidean distance comparator to compare all square Euclidean distance results of point M to achieve farthest point sampling and adjacent sampling.

Images