A channel type statistical distance calculation accelerator for data fusion application

Abstract

The application discloses a kind of channel type statistical distance calculation accelerator for data fusion application, comprising: data reading unit, for reading the coordinate value of element for statistical distance calculation from the memory where target data set A and maintenance data set B are located;The channel calculation unit is used to calculate the statistical distance value between the coordinate value of each element of target data set A and maintenance data set B according to statistical distance calculation formula, to generate statistical distance matrix, and store statistical distance matrix in data buffer;The data output unit exports statistical distance matrix according to data buffer.The application effectively reduces the time consumption of operation and the time consumption of memory access by parallel computing of multiple channel calculation units, adding comparator, threshold comparison, saving only the value of effective data and its corresponding position information, so it has high operation efficiency, and the number of channels, calculation formula can be customized according to the characteristics of application, and flexibility and configurability are good.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of FPGA parallel computing acceleration, in particular to a channel type statistical distance calculation accelerator for data fusion application. BACKGROUND

[0002] In data fusion application, the algorithm commonly used to determine the correlation between two groups of data includes three steps: 1. Calculate the statistical distance between each element in the target data set A and each element in the maintenance data set B according to the pre-set formula, obtain the statistical distance matrix, compare each element value in the matrix with the threshold value, and directly set 0 for the value greater than the threshold value; 2. Screen the matrix, if the statistical distance between a certain element in the target data set A and an element in the maintenance data set B is less than the threshold value, and the statistical distance between the element in the target data set A and other elements in the maintenance data set B is greater than the threshold value (one-to-one correspondence), then the two points are called matching, the index relationship and the statistical distance value are recorded, and the remaining matrix elements are sent to the next step; 3. Find the optimal corresponding relationship, the commonly used algorithm is auction algorithm, through the auction algorithm, the optimal corresponding relationship between the elements A and B of the maintenance data set is obtained, the index of the optimal corresponding relationship is recorded, and finally the results of the second step and the third step are output together.

[0003] There are several problems in the existing data fusion algorithm when running on a general processor:

[0004] The overall algorithm is implemented by CPU, for the statistical distance calculation of large-scale data set in the first part of the algorithm, the efficiency of CPU serial execution is too low, and the calculation time is too long; in the first part of the algorithm, the CPU needs to save all the calculation results, most of which are invalid results, and the storage occupied is large; in the second part of the algorithm, the CPU needs to compare all the calculation results with the threshold value, which takes a long time. SUMMARY

[0005] The purpose of the present application is to provide a channel type statistical distance calculation accelerator for data fusion application, to solve the problems existing in the existing data fusion algorithm when running on a processor.

[0006] In order to achieve the above task, the present application adopts the following technical scheme:

[0007] A channel type statistical distance calculation accelerator for data fusion application, comprising:

[0008] A data reading unit for reading the coordinate values of the elements used for statistical distance calculation from the storage where the target data set A and the maintenance data set B are located;

[0009] The channel calculation unit is configured to calculate statistical distance values between coordinate values of each element of the target data set A and coordinate values of each element of the maintenance data set B according to a preset statistical distance calculation formula, thereby generating a statistical distance matrix, and storing the statistical distance matrix in the data buffer;

[0010] The data output unit outputs the statistical distance matrix according to the data buffer.

[0011] Further, the data reading unit comprises a control logic unit, a data selector, a ping-pong buffer 1 and a ping-pong buffer 2.

[0012] Further, the specific interaction relationship between the data reading unit and the channel calculation unit is as follows:

[0013] 1) The control logic unit of the data reading unit reads coordinate values of m elements from the memory storing the target data set A every clock cycle and stores the coordinate values in the ping-pong buffer 1;

[0014] 2) The control logic unit sets the data selector to the ping-pong buffer 1, and then the control logic unit sends coordinate values of the first n elements in the m elements to n channel calculators in the channel calculation unit through the data selector; m is an integer multiple of n;

[0015] 3) The control logic unit starts to read coordinate values of elements from the external memory storing the maintenance data set B, reads one element coordinate value every cycle, and sends the element coordinate value to the n channel calculators in the channel calculation unit for statistical distance calculation according to the preset statistical distance calculation formula;

[0016] 4) The control logic unit reads all elements of the maintenance data set B in units of n elements, until all elements of the maintenance data set B complete the statistical distance calculation;

[0017] 5) While the channel calculation unit calculates the statistical distance, the control logic unit reads coordinate values of m elements from the memory storing the target data set A and stores the coordinate values in the ping-pong buffer 2;

[0018] 6) When the coordinate values of the m elements in the ping-pong buffer 1 and all element coordinate values of the maintenance data set B are calculated, the control logic unit sets the data selector to the ping-pong buffer 2, and repeats steps 3) to 4);

[0019] 7) The control logic unit repeats steps 1) to 6) until all elements of the target data set A are taken.

[0020] Further, the channel computing unit is composed of n parallel channel calculators, the channel calculator includes a floating point operation unit and a floating point comparison unit; the floating point operation unit further includes a floating point subtraction unit, a floating point multiplication unit and a floating point addition unit.

[0021] Further, the floating point operation unit is used to complete the calculation of the statistical distance from the elements in the target data set A and the maintenance data set B according to a pre-set statistical distance calculation formula.

[0022] Further, the floating point comparison unit is used to compare the threshold value after each statistical distance calculation is completed.

[0023] If it is less than the pre-set threshold value, the calculation result and the element index of the target data set A and the maintenance data set B participating in this operation are obtained, the calculation result and the element index are stored into the statistical distance matrix, and after all elements of the target data set A and the maintenance data set B are calculated, the statistical distance matrix is sent to the data output unit.

[0024] Further, if it is greater than the threshold value, only the element index of the target data set A and the maintenance data set B participating in this operation is sent to the data output unit.

[0025] Further, the data output unit includes a data buffer and a state register.

[0026] The state register is used to generate a calculation completion flag of the target data set A and the maintenance data set B, after the calculation completion flag is read by an external device, the result in the data buffer is read back, and the new target data set A and the maintenance data set B are written into an external memory for next calculation.

[0027] Compared with the prior art, the application has the following technical features:

[0028] The application proposes a channel type statistical distance calculation accelerator for data fusion application, which effectively reduces the operation time and the memory access time through the parallel calculation of the multi-channel computing unit, the addition of the comparator, the threshold comparison step and the saving of only the effective data value and the corresponding position information in the calculation unit, so that the application has high operation efficiency, and the number of channels, the calculation formula can be customized according to the characteristics of the application, and the flexibility and the configurability are good. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 It is the overall architecture diagram of the accelerator of the application;

[0030] Figure 2 It is the channel computing unit architecture diagram for statistical distance calculation and comparison screening in the application. DETAILED DESCRIPTION

[0031] The application discloses a channel type statistical distance calculation accelerator for data fusion application, and relates to the technical field of FPGA parallel calculation acceleration. Figure 1 As shown in the figure, the accelerator comprises a data reading unit, a channel calculation unit and a data output unit.

[0032] 1. The data reading unit

[0033] The data reading unit is used for reading coordinate values of elements for statistical distance calculation from a memory where a target data set A and a maintenance data set B are located.

[0034] Specifically, the data reading unit comprises a control logic unit, a data selector, ping-pong buffer one and ping-pong buffer two.

[0035] 1) The control logic unit of the data reading unit reads coordinate values of m elements from a memory storing the target data set A every clock cycle and stores the coordinate values in ping-pong buffer one.

[0036] 2) The control logic unit sets the data selector to ping-pong buffer one, and then the control logic unit sends coordinate values of the first n elements in the m elements to n channel calculators in the channel calculation unit through the data selector.

[0037] 3) The control logic unit starts reading coordinate values of elements from an external memory storing the maintenance data set B every clock cycle, and sends the coordinate values to the n channel calculators in the channel calculation unit for statistical distance calculation according to a preset statistical distance calculation formula.

[0038] 4) The control logic unit reads all elements of the maintenance data set B in units of n elements, and the statistical distance calculation of all elements in the maintenance data set B is completed.

[0039] 5) While the channel calculation unit is calculating the statistical distance, the control logic unit reads coordinate values of m elements from the memory storing the target data set A and stores the coordinate values in ping-pong buffer two.

[0040] 6) When the coordinates of the m elements in the ping-pong cache one and the coordinates of all the elements in the maintenance dataset B are calculated, the control logic unit selects the data selector as the ping-pong cache two, and repeats steps 3)-4);

[0041] 7) The control logic unit repeats steps 1)-6) until all the elements in the target dataset A are fetched.

[0042] 2. Channel calculation unit

[0043] The channel calculation unit is used to calculate the statistical distance values between the coordinate values of each element of the target dataset A and the coordinate values of each element of the maintenance dataset B according to a preset statistical distance calculation formula, thereby generating a statistical distance matrix, and storing the statistical distance matrix into a data buffer.

[0044] The channel calculation unit is composed of n parallel channel calculators, and each channel calculator includes a floating point operation unit and a floating point comparison unit; the floating point operation unit further includes a floating point subtraction unit, a floating point multiplication unit and a floating point addition unit.

[0045] The floating point operation unit is used to complete the calculation of the statistical distance of the elements from the target dataset A and the maintenance dataset B according to a preset statistical distance calculation formula; wherein the statistical distance calculation formula is defined by the user according to the actual situation, which can be an existing statistical distance calculation formula or a calculation formula set by the user.

[0046] The floating point comparison unit is used for threshold comparison after each statistical distance calculation is completed.

[0047] If greater than the threshold value, only the element indexes of the target dataset A and the maintenance dataset B participating in this operation are sent to the data output unit; for subsequent processing of the auction algorithm.

[0048] If less than the preset threshold value, the calculation result and the element indexes of the target dataset A and the maintenance dataset B participating in this operation are obtained, and the calculation result and the element indexes are stored into the statistical distance matrix; after all the elements of the target dataset A and the maintenance dataset B are calculated, the statistical distance matrix is sent to the data output unit.

[0049] The present application adds a threshold comparison in the channel calculation unit, only keeps the effective result, reduces the data storage amount and access time, and can customize the number of channels and calculation formula according to the characteristics of the application, and has good flexibility and configurability.

[0050] 3. Data output unit

[0051] The data output unit outputs the statistical distance matrix according to the data buffer.

[0052] The data output unit comprises a data buffer and a state register.

[0053] The state register is used to generate a target data set A and a maintenance data set B, and after the external device reads the calculation completion flag, the result in the data buffer is read back and the new target data set A and the maintenance data set B are written into the external memory for the next calculation. The state register registers the current calculation state of the channel calculation unit, and when the channel calculation unit calculates all the data of the target data set A and the maintenance data set B, the calculation completion flag is generated.

[0054] The application designs a channel type statistical distance calculation accelerator for data fusion application based on FPGA platform. The accelerator adopts parallel computing architecture in design, disperses dense floating point calculation to each channel, and each channel performs calculation at the same time to maximize operation efficiency, so as to realize optimization of low efficiency of CPU serial execution in the prior art. In addition, a comparator is added in the calculation unit in the calculation process, the step of comparing with a threshold in the original algorithm is brought forward, only the value of the effective data and the corresponding position information are saved, so that the operation time and the storage amount are reduced, so as to realize the problems that the storage is occupied and the time is consumed in the prior art when the CPU executes, all the calculation results are saved and the threshold is compared with all the calculation results.

[0055] Embodiment:

[0056] As shown in Figure 1 , Figure 1 The accelerator takes 10 channel calculation units as an example. The first part of the data reading unit of the accelerator is located in the left dashed box, which comprises a control logic unit for controlling the whole data reading and storing, two blocks of cache for alternately using to constitute a ping-pong cache, and a data selector controlled by the control logic unit for sending data in the two blocks of cache to the channel calculation unit.

[0057] In this embodiment, the control flow of the control logic unit is as follows: When the accelerator receives the start signal, the control logic unit reads the coordinate values ​​of 20 elements from the memory storing the target dataset A in each clock cycle and stores them in ping-pong buffer one; in the next cycle, the data selector sends the coordinate values ​​of the first 10 elements to the 10 channel calculators in the channel calculation unit respectively; the control logic unit then reads the element coordinate values ​​from the external memory storing the maintenance dataset B, reading one element coordinate value per cycle and sending it to the 10 channel calculators in the channel calculation unit to calculate according to the preset statistical distance calculation formula (the formula can be customized according to actual needs), and the calculation is completed in two cycles; while calculating the statistical distance, the control logic unit retrieves the coordinates of 20 elements from the target dataset A buffer and stores them in ping-pong buffer two; when the coordinates in buffer one are calculated with the coordinates of all elements in the maintenance dataset B, the control logic unit selects buffer two as the data selector, retrieves the values ​​of all maintenance dataset B again, and each channel continues to calculate. Meanwhile, the control logic unit in the next cycle sequentially retrieves the coordinates of 20 elements from the target dataset A cache and stores them into cache one... and so on, until all the coordinates in the target dataset A cache are retrieved. The entire channel accelerator adopts a pipelined design, and the calculation process can fully utilize the throughput advantage of the FPGA.

[0058] The channel calculation unit architecture diagram used for statistical distance calculation and comparison filtering is as follows: Figure 2 As shown, it includes floating-point subtraction, floating-point multiplication, floating-point addition, and floating-point comparison units. After the coordinates of a pair of elements from the target dataset A and the maintenance dataset B are calculated, they are compared with a threshold in the floating-point comparison unit. If the result is less than the threshold, the output of the floating-point comparison unit is enabled, and the calculation result and the coordinates of the two points are saved to the data buffer of the data output unit. When the entire calculation process is completed, the calculation logic sends a completion signal to the external PC device or the peripheral soft-core processor on the FPGA. The processor reads the valid data of the completed calculation through the data bus so that subsequent algorithms can continue processing.

[0059] The advantages of the accelerator with the above structure are as follows: It fully utilizes the parallelizable parts of the algorithm, distributing dense computations across each channel. Through ping-pong buffering and pipelined channel computation units, continuous computation is achieved, maximizing computational efficiency. Furthermore, by adding a threshold comparison module to the channel computation units, only valid results can be retained, reducing data storage and access time. The second part of the algorithm, which filters only valid results, also significantly reduces filtering time.

[0060] In summary, the application designs a multi-channel, pipeline structure's channel type statistical distance calculation accelerator for data fusion application, improves the efficiency and continuity of the calculation, advances the threshold comparison link, only keeps the effective intermediate result, sparsifies the storage, and reduces the storage and operation time.

[0061] The above examples are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A channelized statistical distance computation accelerator for data fusion applications, characterized by, The application relates to a data processing device for calculating statistical distance between target data set A and maintenance data set B. The data reading unit reads the coordinate values of elements for statistical distance calculation from a memory where the target data set A and the maintenance data set B are stored. The channel calculation unit calculates the statistical distance values between the coordinate values of each element of the target data set A and the coordinate values of each element of the maintenance data set B according to a preset statistical distance calculation formula, thereby generating a statistical distance matrix, and storing the statistical distance matrix in a data buffer. The data output unit outputs the statistical distance matrix according to the data buffer. The specific interaction relationship between the data reading unit and the channel calculation unit is as follows: 1) The control logic unit of the data reading unit reads the coordinate values of m elements from the memory storing the target data set A every clock cycle and stores the coordinate values in ping-pong buffer 1; 2) The control logic unit sets the data selector to ping-pong buffer 1, and then the control logic unit sends the coordinate values of the first n elements in the m elements to the n channel calculators in the channel calculation unit through the data selector; m is an integer multiple of n; 3) The control logic unit starts reading the coordinate values of elements from the external memory storing the maintenance data set B, reads one element coordinate value every cycle, and sends the element coordinate value to the n channel calculators in the channel calculation unit for statistical distance calculation according to the preset statistical distance calculation formula; 4) The control logic unit reads all elements of the maintenance data set B in units of n elements, and the statistical distance calculation of all elements in the maintenance data set B is completed; 5) While the channel calculation unit calculates the statistical distance, the control logic unit reads the coordinate values of m elements from the memory storing the target data set A and stores the coordinate values in ping-pong buffer 2; 6) When the coordinate values of the m elements in ping-pong buffer 1 and all element coordinates in the maintenance data set B are calculated, the control logic unit sets the data selector to ping-pong buffer 2, and repeats steps 3) to 4); 7) The control logic unit repeats steps 1) to 6) until all elements in the target data set A are taken.

2. The lane-wise statistical distance computation accelerator for data fusion applications of claim 1, wherein, The data reading unit comprises a control logic unit, a data selector, ping-pong buffer 1 and ping-pong buffer 2.

3. The lane-wise statistical distance computation accelerator for data fusion applications of claim 1, wherein, The channel calculation unit is composed of n parallel channel calculators, and each channel calculator comprises a floating point operation unit and a floating point comparison unit; the floating point operation unit comprises a floating point subtraction unit, a floating point multiplication unit and a floating point addition unit.

4. The lane-wise statistical distance computation accelerator for data fusion applications of claim 3, wherein, The floating point operation unit is used for calculating the statistical distance between elements from the target data set A and the maintenance data set B according to the preset statistical distance calculation formula.

5. The lane-based statistical distance computation accelerator for data fusion applications of claim 3, wherein, The floating point comparison unit is used for threshold comparison after each statistical distance calculation: If the result is less than the preset threshold value, the calculation result and the element indexes of the target data set A and the maintenance data set B participating in the operation are obtained, and the calculation result and the element indexes are stored in the statistical distance matrix; After all elements of the target data set A and the maintenance data set B are calculated, the statistical distance matrix is sent to the data output unit.

6. The lane-based statistical distance computation accelerator for data fusion applications of claim 5, wherein, If the result is greater than the threshold value, only the element indexes of the target data set A and the maintenance data set B participating in the operation are sent to the data output unit.

7. The lane-based statistical distance computation accelerator oriented towards data fusion applications of claim 1, wherein, The data output unit comprises a data buffer and a state register; The state register is used to generate a target data set A and maintain a data set B calculation completion flag, and after the external device reads the calculation completion flag, the result in the data buffer is read back and a new target data set A and the data set B for the next calculation are written into the external memory.

Images