FPGA-based fast clustering target detection method, device, equipment and medium

Abstract

The application relates to a fast clustering target detection method, device and equipment based on FPGA and a medium, three modules of finding an effective point in a channel, inter-channel clustering and intra-channel clustering are parallelized by utilizing logical resources in the FPGA. The method comprises the following steps: while finding the effective point in the channel, intra-channel clustering marking is performed; while performing the intra-channel clustering, two FIFO buffers are used to cache row point cloud data and corresponding clustering labels output by the intra-channel clustering module until two rows of point cloud data are cached, and after the third row of point cloud data is output by the intra-channel clustering module, label merging is performed on the three rows of point cloud data to realize inter-channel clustering. In the design process of the method, a pipeline processing mode is adopted, and the three submodules in the design can be processed in parallel, thereby effectively improving real-time performance while ensuring clustering processing accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image target detection, in particular to a fast clustering target detection method, device and equipment based on FPGA and a medium. BACKGROUND

[0002] In recent years, with the rapid development of laser radar, it has been widely used in target detection field because of its high precision, strong anti-interference ability and rich detailed information of the obtained point cloud image. Usually, clustering processing is performed during target detection using point cloud image to mark the categories of point cloud image, and the clustering result will directly affect the overall detection result.

[0003] At present, the existing clustering algorithm usually performs multiple repeated traversals on the point cloud image to mark and classify, and if ARM and DSP, which are sequential execution embedded platforms, are used for deployment and implementation, multiple repeated traversals on the point cloud image will also be performed during execution, which will cause the processing process to be time-consuming and the real-time requirement to be difficult to meet.

[0004] Compared with ARM and DSP, FPGA (programmable logic gate array) platform can adopt parallel processing, and if the design is reasonable, the established function can be realized in one traversal of the image during clustering, which can greatly reduce the processing time of the clustering algorithm on the embedded platform. It is feasible to use the parallelization feature of the FPGA platform to realize the fast clustering scheme. SUMMARY

[0005] Therefore, it is necessary to provide a fast clustering target detection method, device, equipment and medium based on FPGA, which can improve the detection efficiency.

[0006] A fast clustering target detection method based on FPGA, the method comprises:

[0007] Obtaining point cloud data, the point cloud data is obtained by laser radar detection, and the point cloud data is transmitted to the effective point recognition unit through the high-speed data interface of the FPGA;

[0008] In the effective point recognition unit, a state machine is constructed by using the logic resources in the FPGA, the point cloud data of each row is determined according to the preset size of the point cloud image, the state machine is used to find the first effective point and the remaining effective points in each row of point cloud data, the first control signal is pulled high when the first effective point is found, the second control signal is pulled high when the remaining effective points are found, and after the effective point recognition of one row of point cloud data is completed, the row of point cloud data, the first control signal and the second control signal are transmitted to the channel clustering unit through the internal wiring resources of the FPGA.

[0009] In the channel clustering unit, based on the parallel processing capability of the FPGA, valid point clustering in a row of point cloud data is performed according to the row of point cloud data, the first control signal and the second control signal, and each valid point is marked to generate a clustering label, and the row of point cloud data and the corresponding clustering label are transmitted to the inter-channel clustering unit;

[0010] In the inter-channel clustering unit, the input row of point cloud data and the corresponding clustering label are buffered by two FIFO buffers until two rows of point cloud data are buffered, and after the third row of point cloud data is input, label merging is performed on the three rows of point cloud data to realize inter-channel clustering, and the row of point cloud data and the merged label are output.

[0011] Target detection is performed according to the merged label and the row of point cloud data.

[0012] In one embodiment, the preset size of the point cloud image includes the number of row point clouds and the number of column point clouds of each frame of point cloud image.

[0013] When the point cloud data is transmitted to the valid point recognition unit, the frame valid signal is pulled high first, representing the transmission of the current frame data, and the row valid signal is pulled high and the row counter is counted, and when the counting result meets the preset row point cloud number, the row valid signal is pulled low, and the column counter is incremented by 1.

[0014] When the row valid signal is pulled high, the point cloud data of a row is determined, and the state machine is used to find the first valid point and the remaining valid points in the determined point cloud data of the row.

[0015] When the counting result meets the preset column point cloud number, the counting is reset to zero, and the counting of the next frame of point cloud data begins.

[0016] In one embodiment, when the state machine is used to find the first valid point and the remaining valid points in each row of point cloud data, it includes:

[0017] The state machine includes an initial idle state, a first valid point waiting state, a first valid point found state and an other valid point waiting state.

[0018] The state machine is initially in an idle state, and when the row valid signal is pulled high, the point cloud data is transmitted, the edge of each point cloud data is detected, and the first valid point waiting state is entered.

[0019] In the first valid point waiting state, when the first point cloud data is greater than 0, the first valid point is found, and the first valid point found state is entered, and if the row valid signal is pulled low, the idle state is returned to wait for the next row of point cloud data.

[0020] In the first valid point finding state, after pulling up the first control signal, jump to other valid point waiting state;

[0021] In the other valid point waiting state, when a certain point cloud data is greater than 0, pull up the second control signal, and return to the idle state to wait for the next line of point cloud data.

[0022] In one embodiment, in the channel clustering unit:

[0023] According to the first control signal, a line of point cloud data is detected, and when the first valid point is detected, the label corresponding to the valid point is assigned;

[0024] After finding the first valid point, the second control signal is used to detect other valid points in a line of point cloud data, and when other valid points are detected, the difference between the current valid point and the previous valid point is calculated, and if the difference is less than a predetermined threshold, the label corresponding to the current valid point is assigned to the label of the previous valid point.

[0025] If the difference is greater than the predetermined threshold, the label corresponding to the current valid point is assigned plus one.

[0026] In one embodiment, in the inter-channel clustering unit, before each line of point cloud data and the corresponding clustering label are cached to the memory, the point cloud data and the corresponding clustering label are spliced, and the spliced data is cached to the FIFO memory.

[0027] In one embodiment, when merging labels in an 8-neighborhood of three lines of point cloud data, the 8-neighborhood is mainly realized by nine REG registers.

[0028] In one embodiment, when merging labels:

[0029] In three lines of point cloud data, a 3X3 sliding window is used to traverse, and it is determined whether the point cloud data at the center position in each 3X3 window is a valid point, if not, the 3X3 window is moved;

[0030] If it is a valid point, it is determined whether there is a valid point in the eight point cloud data adjacent to the valid point, if not, the 3X3 window is moved;

[0031] If there is a valid point, the difference between the valid point at the center position and the valid point in the eight-neighborhood is calculated, and if the difference is less than a threshold, the label of the valid point in the eight-neighborhood is modified to the assignment of the valid point at the center position.

[0032] The application also provides a fast clustering target detection device based on FPGA, which comprises:

[0033] a point cloud data acquisition module configured to acquire point cloud data, the point cloud data being obtained by laser radar air detection and being transmitted to an effective point identification unit through a high-speed data interface of an FPGA;

[0034] an effective point identification module configured to, in the effective point identification unit, construct a state machine by using logic resources in the FPGA, determine point cloud data of each row according to a preset size of a point cloud image, find a first effective point and remaining effective points in the point cloud data of each row by using the state machine, pull up a first control signal when the first effective point is found, pull up a second control signal when the remaining effective points are found, and transmit the point cloud data of each row, the first control signal and the second control signal to an intra-channel clustering unit through wiring resources in the FPGA after effective point identification of the point cloud data of each row is completed;

[0035] an intra-channel clustering module configured to, in the intra-channel clustering unit, perform intra-row effective point clustering according to the point cloud data of each row, the first control signal and the second control signal based on parallel processing capability of the FPGA, and generate a clustering label by performing corresponding clustering marking on each effective point, and transmit the point cloud data of each row and the corresponding clustering label to an inter-channel clustering unit;

[0036] an inter-channel clustering module configured to, in the inter-channel clustering unit, cache the input point cloud data of each row and the corresponding clustering label by using two FIFO buffers until two rows of point cloud data are cached, perform label merging on three rows of point cloud data after inputting a third row of point cloud data, so as to realize inter-channel clustering, and output the point cloud data of each row and the merged label;

[0037] a target detection module configured to perform target detection according to the merged label and the point cloud data of each row.

[0038] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0039] acquire point cloud data, the point cloud data being obtained by laser radar air detection and being transmitted to an effective point identification unit through a high-speed data interface of an FPGA;

[0040] In the effective point recognition unit, a state machine is constructed by using a logic resource in the FPGA, the point cloud data of each row is determined according to a preset size of the point cloud image, the first effective point and the remaining effective points are found in the point cloud data of each row by using the state machine, the first control signal is pulled high when the first effective point is found, the second control signal is pulled high when the remaining effective points are found, and after the effective point recognition of the point cloud data of a row is completed, the point cloud data of this row, the first control signal and the second control signal are transmitted to the intra-channel clustering unit through a wiring resource inside the FPGA.

[0041] In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, the effective point clustering in a row is performed according to the point cloud data of a row, the first control signal and the second control signal, and the corresponding clustering labels are generated for each effective point to generate clustering labels, and the point cloud data of this row and the corresponding clustering labels are transmitted to the inter-channel clustering unit.

[0042] In the inter-channel clustering unit, the input row point cloud data and the corresponding clustering labels are cached by two FIFO buffers until two rows of point cloud data are cached, and after the third row of point cloud data is input, the label merging is performed on the three rows of point cloud data to realize the inter-channel clustering, and the row point cloud data and the merged labels are output.

[0043] Target detection is performed according to the merged labels and the row point cloud data.

[0044] A computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0045] Point cloud data is acquired, the point cloud data is obtained by laser radar detection, and the point cloud data is transmitted to an effective point recognition unit through a high-speed data interface of the FPGA.

[0046] In the effective point recognition unit, a state machine is constructed by using a logic resource in the FPGA, the point cloud data of each row is determined according to a preset size of the point cloud image, the first effective point and the remaining effective points are found in the point cloud data of each row by using the state machine, the first control signal is pulled high when the first effective point is found, the second control signal is pulled high when the remaining effective points are found, and after the effective point recognition of the point cloud data of a row is completed, the point cloud data of this row, the first control signal and the second control signal are transmitted to the intra-channel clustering unit through a wiring resource inside the FPGA.

[0047] In the channel intra-cluster unit, based on the parallel processing capability of the FPGA, the effective points in a row of point cloud data are clustered according to the row of point cloud data, the first control signal and the second control signal, and each effective point is marked to generate a cluster label, and the row of point cloud data and the corresponding cluster label are transmitted to the inter-channel clustering unit;

[0048] In the inter-channel clustering unit, the input row of point cloud data and the corresponding cluster label are buffered by two FIFO buffers until two rows of point cloud data are buffered, and after inputting a third row of point cloud data, the three rows of point cloud data are merged to realize inter-channel clustering, and the row of point cloud data and the merged label are output.

[0049] Target detection is performed according to the merged label and the row of point cloud data.

[0050] The above-mentioned FPGA-based fast clustering target detection method, device, equipment and medium use the logic resources in the FPGA to construct a state machine, use the state machine to find the first effective point and the remaining effective points in each row of point cloud data, pull up the first control signal when the first effective point is found, and pull up the second control signal when the remaining effective points are found, based on the parallel processing capability of the FPGA, cluster the effective points in a row of point cloud data according to the row of point cloud data, the first control signal and the second control signal, and mark each effective point to generate a cluster label, buffer the input row of point cloud data and the corresponding cluster label by two FIFO buffers until two rows of point cloud data are buffered, merge the labels of the three rows of point cloud data after inputting a third row of point cloud data to realize inter-channel clustering, and perform target detection according to the merged label and the row of point cloud data. The method effectively improves the efficiency of clustering in the target detection process. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 It is a flowchart of the FPGA-based fast clustering target detection method in one embodiment;

[0052] Figure 2 It is a schematic diagram of the overall framework of the FPGA in one embodiment;

[0053] Figure 3 It is a state jump schematic diagram in one embodiment;

[0054] Figure 4 It is a timing diagram in the effective point identification unit in one embodiment;

[0055] Figure 5 It is a timing diagram in the channel intra-cluster unit in one embodiment;

[0056] Figure 6 Schematic diagram of a framework of an inter-channel clustering unit in one embodiment;

[0057] Figure 7 1 is a logical diagram of inter-channel clustering in one embodiment;

[0058] Figure 8 This is a structural block diagram of a device for implementing fast clustering target detection based on FPGA in one embodiment;

[0059] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] In view of the problem that in the prior art, when implementing the clustering algorithm, a frame of data needs to be traversed multiple times, resulting in low efficiency, in one embodiment, Figure 1 As shown, a method for fast clustering target detection based on FPGA is provided, which specifically includes the following steps:

[0062] Step S100: acquiring point cloud data. The point cloud data is obtained by detecting the air through a laser radar, and the point cloud data is transmitted to a valid point recognition unit through a high-speed data interface of an FPGA.

[0063] In step S110, in the effective point recognition unit, a state machine is constructed using the logic resources in the FPGA, and each row of point cloud data is determined according to the preset size of the point cloud image. The state machine is used to find the first effective point and the remaining effective points in each row of point cloud data. When the first effective point is found, the first control signal is pulled high, and when the remaining effective points are found, the second control signal is pulled high. After completing the effective point recognition of a row of point cloud data, the row of point cloud data, the first control signal, and the second control signal are transmitted to the clustering unit in the channel through the wiring resources inside the FPGA.

[0064] In step S120, in the intra-channel clustering unit, based on the parallel processing capability of the FPGA, clustering of valid points within a row is performed according to a row of point cloud data, the first control signal, and the second control signal, and corresponding clustering marks are performed on each valid point to generate cluster labels, and this row of point cloud data and the corresponding cluster labels are transmitted to the inter-channel clustering unit.

[0065] Step S130, in the inter-channel clustering unit, the input row point cloud data and the corresponding clustering label are buffered by two FIFO buffers until two rows of point cloud data are buffered, and after the third row of point cloud data is input, the label merging of the construction of 8 fields of the three rows of point cloud data is performed to realize the inter-channel clustering, and the row point cloud data and the merged label are output.

[0066] Step S140, target detection is performed according to the merged label and the row point cloud data.

[0067] In the implementation process of the existing clustering algorithm, image traversal is required to find an effective point at each execution, and a large amount of time is spent in each process. In the present application, the clustering algorithm is implemented by using FPGA in the form of pipeline, which greatly improves the clustering speed. Based on FPGA, firstly, the intra-channel marking is performed at the same time when finding the first effective point of each row, which can save one image traversal time. Secondly, after the intra-channel clustering of three rows is completed, the related upper and lower rows can be generated to perform inter-channel clustering, which can also save one image traversal time. This method is difficult to implement for ARM and DSP sequential embedded devices, therefore, FPGA, a timing and parallel processing embedded device, is adopted to realize fast clustering algorithm acceleration.

[0068] Specifically, FPGA is a parallel acceleration embedded platform with rich programmable logic resources. When performing algorithm processing, each sub-module can be processed in parallel to reduce the number of image traversals, greatly reduce the processing time, and improve the real-time performance.

[0069] As shown in Figure 2 When the clustering algorithm is implemented by using FPGA, it is divided into three units including an effective point identification unit, an intra-channel clustering unit and an inter-channel clustering unit.

[0070] Specifically, when the point cloud data Data is transmitted to the FPGA, it first enters the valid point identification unit for finding the first valid point of each row. In the unit, the frame and row valid signals are calculated according to the size of the point cloud image, and the first valid point of each row is found to generate the control signals First_valid (first control signal) and Data_valid (second control signal) under the timing. The signals are given to the channel clustering unit with the point cloud data for subsequent processing. In the channel clustering unit, the channel clustering is performed according to the control signals First_valid and Data_valid and the point cloud data at this time to generate the label category corresponding to the data. The inter-channel clustering unit reads the data after the intra-channel clustering processing is marked for three rows, and processes according to the merging label condition. Finally, the processed label and data are put into two RAMs for result comparison.

[0071] In step S110, the preset size of the point cloud image includes the row point cloud quantity and the column point cloud quantity of each frame of point cloud image. Further, the data of a frame of point cloud image can be further determined according to the row point cloud quantity and the column point cloud quantity. When the point cloud data is transmitted to the valid point identification unit, the frame valid signal is pulled high to represent the transmission of the current frame data, and the row valid signal is pulled high and the row counter is counted. When the counting result meets the preset row point cloud quantity, it represents that the row data transmission is completed, the row valid signal is pulled low, and the column counter is incremented by 1. When the row valid signal is pulled high, the first valid point and the remaining valid points in the determined row of point cloud data are determined by using the state machine. When the counting result meets the preset column point cloud quantity, the counting is reset to zero, and the counting of the next frame of point cloud data is started.

[0072] Specifically, the frame and row valid signals can be counted according to the input data to generate according to the specified point cloud image size. For example, the point cloud image of 400*125 can pull the row valid signal high when the row counter is counted from 0 to 400, and pull the frame valid signal high when the column counter is counted from 0 to 125.

[0073] As Figure 3As shown, in the state machine for finding the first valid point and the remaining valid points in the point cloud data of each row, the state machine includes an initial idle state (IDLE), a first valid point waiting state (WAIT_FRST_DATA), a first valid point finding state (FIRST_DATA_IDX), and a remaining valid point waiting state (DATA_IDX). The state machine is initially in the idle state (IDLE), waits for the row valid signal to be pulled high, performs edge detection on the point cloud data, and enters the first valid point waiting state (WAIT_FRST_DATA). In the first valid point waiting state, when the first point cloud data is greater than 0, the first valid point is found, and the state machine enters the first valid point finding state. If the row valid signal is pulled low, the state machine returns to the idle state to wait for the point cloud data of the next row. In the first valid point finding state (FIRST_DATA_IDX), after the first control signal is pulled high, the state machine jumps to the remaining valid point waiting state (DATA_IDX). In the remaining valid point waiting state (DATA_IDX), when a point cloud data is greater than 0, the second control signal is pulled high, and the state machine returns to the idle state (IDLE) to wait for the point cloud data of the next row.

[0074] Specifically, in the state IDLE, the row valid signal is pulled high, and the transmission of the first point cloud data begins, where de==1’b1&&de_r==1’b0 is the edge detection state entering the WAIT_FRST_DATA.

[0075] In the state WAIT_FRST_DATA, the first valid point is waited for. When the data data>0, it represents that the point is the first valid point, and the state enters the FIRST_DATA_IDX. If the row valid signal is pulled low, de==1’b0&&de_r==1’b1 represents that there is no valid point in the row, and the state returns to the IDLE state to wait for the transmission of the data of the next row.

[0076] The state FIRST_DATA_IDX is a value assignment state. At this time, the first valid point is detected, and therefore the control signal First_valid is pulled high. Subsequently, the state jumps to the DATA_IDX state.

[0077] In the state DATA_IDX, if there is data data>0 at this time, it represents that the remaining valid points in the row except the first valid point are detected, and therefore the control signal Data_valid is pulled high. If the row valid signal is pulled low, de==1’b0&&de_r==1’b1 represents that the detection of the row is completed, and the state returns to the IDLE state to wait for the row valid signal of the second row to be pulled high for detection of the second row.

[0078] As shown, Figure 4As shown in FIG, a timing diagram of a frame vs, a row valid signal de, point cloud data data, a first control signal First_valid and a second control signal Data_valid in a valid point recognition unit. Figure 4 As can be seen from the figure, Data starts to be detected by the state machine. When the first valid point of Data>0 in each row appears, the First_valid signal will be pulled high. When the valid point of Data>0 appears subsequently, the Data_valid signal will be pulled high.

[0079] In step S120, in the intra-channel clustering unit: first, a line of point cloud data is detected according to the first control signal. When the first valid point is detected, the label corresponding to the valid point is assigned. After finding the first valid point, other valid points are detected on a line of point cloud data according to the second control signal. When other valid points are detected, the current valid point is subtracted from the previous valid point. If the difference is less than the preset threshold, the label corresponding to the current valid point is assigned to the label of the previous valid point, that is, the two valid points are clustered. If the difference is greater than the preset threshold, the label assignment corresponding to the current valid point is increased by one. The valid point may be another target, and the label assignment of the valid point is different from that of the previous valid point.

[0080] like Figure 5 As shown in FIG, in the clustering unit within the channel, the timing diagram of the frame vs, the row valid signal de, the point cloud data data, the first control signal First_vali, the second control signal Data_valid and the cluster label Label. Figure 5 As can be seen from the figure, this operation mode conforms to the logic of the algorithm and uses a pipeline structure. While completing the search for the first valid point in each row, it also completes the marking of the clusters within the channel according to the established marking logic, saving the time of traversing an image compared to the original algorithm.

[0081] In step S130, in the inter-channel clustering unit, the data relationship between the current row and the previous row and the next row is mainly compared, so as to further merge the cluster labels, that is, merge the inter-row data. The overall design is as follows Figure 6 shown.

[0082] In this embodiment, in the inter-channel clustering unit, before caching each row of point cloud data and the corresponding clustering label into the FIFO memory, because both data and labels need to be used in this unit at the same time, the point cloud data and the corresponding clustering label are also spliced ​​to obtain Label+Data, and the spliced ​​data is cached into the FIFO memory.

[0083] Furthermore, it is necessary to build an 8-neighborhood for tag merging. Building an 8-neighborhood requires at least three rows of data, so two FIFOs are needed to cache the data. Compared with the usual construction of 8 neighborhoods, this design is different because it uses the data after tag merging. Therefore, the first row of data cached in FIFO1 is the data after bitwise segmentation, such as Figure 6 As shown by the blue line in the middle, the second row caches the data after tag merging into FIFO1. As shown by the red line, each row after the current frame uses the data after tag merging.

[0084] Finally, after the label merging process, the 8 neighborhoods are mainly composed of 9 REG registers. After completion, the value of register REG window11 is split into data Data and label Label and stored in RAM1 and RAM2 for result comparison. The logic of label merging process is as follows: Figure 7 shown.

[0085] like Figure 7 As shown in the figure, when merging labels: in the three rows of point cloud data, a 3X3 sliding window traversal is used to determine whether the point cloud data at the center point position in each 3X3 window is a valid point. If it is not a valid point, the 3X3 window is moved. If it is a valid point, it is determined whether there is a valid point in the eight point cloud data adjacent to the valid point. If not, the 3X3 window is moved. If there is a valid point, the difference between the valid point at the center position and the valid points in the eight neighborhoods is taken. If the difference is less than the threshold, the label assignment of the valid point in the eight neighborhoods is modified to the assignment of the valid point at the center position.

[0086] Specifically, the 3x3 window corresponds to 9 registers REG Window11-REG Window33, with the center point being REGWindow22. The comparison is performed using 8 independent modules simultaneously, fully utilizing the advantages of FPGA parallel processing to complete label updates for 8 neighborhoods simultaneously.

[0087] In step S140 , the number of targets in a frame of image can be determined based on the merged labels. Subsequently, only clustering result screening and frame marking are required to complete the overall target detection process.

[0088] In the FPGA-based fast clustering target detection method, the point cloud data is processed in a pipeline manner, effectively improving the real-time performance of the algorithm on an embedded platform. Firstly, the channel clustering is completed while finding the first valid point of each row. Secondly, only two rows of data need to be cached for real-time output of the processing result. Combining the two points, about two times of image traversal time is saved. In the design process of the method, a pipeline processing mode is adopted, and the three sub-modules in the design can be processed in parallel, thereby effectively improving the real-time performance while ensuring the accuracy of the algorithm.

[0089] It should be understood that, although Figure 1 The steps in the flowchart of the process are displayed in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the process can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0090] In one embodiment, as shown in Figure 8 A FPGA-based fast clustering target detection device is provided, comprising: a point cloud data acquisition module 200, an effective point identification module 210, an intra-channel clustering module 220, an inter-channel clustering module 230, and a target detection module 240, wherein:

[0091] The point cloud data acquisition module 200 is configured to acquire point cloud data, wherein the point cloud data is obtained by laser radar detection, and the point cloud data is transmitted to the effective point identification unit through the high-speed data interface of the FPGA;

[0092] The effective point identification module 210 is configured to construct a state machine using the logic resources in the FPGA in the effective point identification unit, determine the point cloud data of each row according to the preset size of the point cloud image, find the first valid point and the remaining valid points in each row of point cloud data using the state machine, pull up the first control signal when the first valid point is found, pull up the second control signal when the remaining valid points are found, and transmit the row of point cloud data, the first control signal, and the second control signal to the intra-channel clustering unit through the internal wiring resources of the FPGA after completing the effective point identification of the row of point cloud data;

[0093] The intra-channel clustering module 220 is configured to cluster valid points within a row of point cloud data, the first control signal, and the second control signal in the intra-channel clustering unit based on the parallel processing capability of the FPGA, and to generate a cluster label for each valid point, and transmit the row of point cloud data and the corresponding cluster label to the inter-channel clustering unit.

[0094] An inter-channel clustering module 230 is configured to cache input line point cloud data and corresponding cluster labels in the inter-channel clustering unit using two FIFO buffers until two lines of point cloud data are cached. After a third line of point cloud data is input, the module constructs 8 domains for the three lines of point cloud data and performs label merging to achieve inter-channel clustering, and outputs line point cloud data and merged labels.

[0095] The target detection module 240 is configured to perform target detection based on the merged labels and the line point cloud data.

[0096] Regarding the specific limitations of the FPGA-based fast clustering target detection device, please refer to the limitations of the FPGA-based fast clustering target detection method above, which will not be repeated here. The various modules in the above-mentioned FPGA-based fast clustering target detection device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0097] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for fast clustering target detection based on FPGA is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0098] Those skilled in the art will understand that Figure 9The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0099] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0100] Obtaining point cloud data, the point cloud data being obtained by laser radar air detection, and the point cloud data being transmitted to an effective point recognition unit through a high-speed data interface of the FPGA;

[0101] In the effective point recognition unit, a state machine is constructed using logic resources in the FPGA, the point cloud data of each row is determined according to a preset size of the point cloud image, the first effective point and the remaining effective points are found in the point cloud data of each row using the state machine, the first control signal is pulled high when the first effective point is found, the second control signal is pulled high when the remaining effective points are found, and after the effective point recognition of a row of point cloud data is completed, the row of point cloud data, the first control signal and the second control signal are transmitted to the intra-channel clustering unit through the wiring resources inside the FPGA;

[0102] In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, the effective points in a row are clustered according to the row of point cloud data, the first control signal and the second control signal, and corresponding clustering labels are generated for each effective point to generate clustering labels, and the row of point cloud data and the corresponding clustering labels are transmitted to the inter-channel clustering unit;

[0103] In the inter-channel clustering unit, the input row of point cloud data and the corresponding clustering labels are buffered through two FIFO buffers until two rows of point cloud data are buffered, and after the third row of point cloud data is input, the three rows of point cloud data are constructed to merge the labels to realize inter-channel clustering, and the row of point cloud data and the merged labels are output;

[0104] Performing target detection according to the merged labels and the row of point cloud data.

[0105] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0106] Obtaining point cloud data, the point cloud data being obtained by laser radar air detection, and the point cloud data being transmitted to an effective point recognition unit through a high-speed data interface of the FPGA;

[0107] In the effective point recognition unit, a state machine is constructed by using logic resources in the FPGA, point cloud data of each row is determined according to a preset size of the point cloud image, the state machine is used to find the first effective point and the remaining effective points in the point cloud data of each row, the first control signal is pulled high when the first effective point is found, the second control signal is pulled high when the remaining effective points are found, and after the effective point recognition of the point cloud data of a row is completed, the point cloud data of the row, the first control signal and the second control signal are transmitted to the intra-channel clustering unit through wiring resources inside the FPGA;

[0108] In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, the effective points in a row of point cloud data are clustered according to the point cloud data of the row, the first control signal and the second control signal, and each effective point is marked to generate a clustering label, and the point cloud data of the row and the corresponding clustering label are transmitted to the inter-channel clustering unit;

[0109] In the inter-channel clustering unit, the input row point cloud data and the corresponding clustering label are cached by two FIFO buffers until two rows of point cloud data are cached, and after the third row of point cloud data is input, the three rows of point cloud data are constructed to perform label merging to realize inter-channel clustering, and the row point cloud data and the merged label are output.

[0110] Target detection is performed according to the merged label and the row point cloud data.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0112] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0113] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for fast clustering target detection based on FPGA, characterized in that: The method comprises: Acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of an FPGA; In the effective point recognition unit, a state machine is constructed using logic resources in the FPGA. Each row of point cloud data is determined according to a preset size of the point cloud image. The state machine is used to find the first effective point and the remaining effective points in each row of point cloud data. When the first effective point is found, the first control signal is pulled high. When the remaining effective points are found, the second control signal is pulled high. After the effective point recognition of a row of point cloud data is completed, the row of point cloud data, the first control signal, and the second control signal are transmitted to the intra-channel clustering unit through the wiring resources within the FPGA. In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, valid points in the row are clustered according to the row of point cloud data, the first control signal, and the second control signal, and each valid point is marked with a corresponding cluster to generate a cluster label, and the row of point cloud data and the corresponding cluster label are transmitted to the inter-channel clustering unit; In the inter-channel clustering unit, two FIFO buffers are used to cache the input line point cloud data and the corresponding clustering labels until two lines of point cloud data are cached. After the third line of point cloud data is input, an eight-neighborhood is constructed for the three lines of point cloud data to perform label merging to achieve inter-channel clustering, and the line point cloud data and the merged labels are output; Target detection is performed based on the merged labels and the line point cloud data.

2. The method for fast clustering target detection based on FPGA according to claim 1, characterized in that: The preset size of the point cloud image includes the number of row point clouds and the number of column point clouds in each frame of the point cloud image; When the point cloud data is transmitted to the valid point recognition unit, the frame valid signal is first pulled high, indicating that the current frame data is being transmitted. At the same time, the row valid signal is pulled high and the row counter is counted. When the counting result meets the preset number of row point clouds, it means that the data transmission of the row number is completed, the row valid signal is pulled low, and the column counter is incremented by 1. When the row valid signal is pulled high, the point cloud data of a row is determined, and the state machine is used to find the first valid point and the remaining valid points in the point cloud data of the determined row; When the counting result meets the preset number of column point clouds, the count is reset to zero and the counting of the next frame of point cloud data begins.

3. The method for fast clustering target detection based on FPGA according to claim 2, characterized in that: When using a state machine to find the first valid point and other valid points in each row of point cloud data, it includes: The state machine includes an initial idle state, a first valid point waiting state, a first valid point found state and other valid point waiting states; The state machine is initially in an idle state. When the row valid signal is pulled high and the point cloud data is input, edge detection is performed on each point cloud data, and the state machine enters the first valid point waiting state. In the first valid point waiting state, when the first point cloud data is greater than 0, the first valid point is found, and the state of first valid point found is entered. If the row valid signal is pulled low, the state returns to the idle state and waits for the next row of point cloud data. In the first valid point found state, after the first control signal is pulled high, the state jumps to the other valid point waiting state; In the waiting state for other valid points, when a certain point cloud data is greater than 0, the second control signal is pulled high, and the system returns to the idle state to wait for the next line of point cloud data.

4. The method for fast clustering target detection based on FPGA according to claim 3, characterized in that: In the intra-channel clustering unit: Detecting a line of point cloud data according to the first control signal, and assigning a label corresponding to the first valid point when the first valid point is detected; After finding the first valid point, a line of point cloud data is detected for other valid points according to the second control signal. When other valid points are detected, the current valid point is subtracted from the previous valid point. If the difference is less than a preset threshold, the label corresponding to the current valid point is assigned to the label of the previous valid point. If the difference is greater than a preset threshold, the value of the label corresponding to the current valid point is increased by one.

5. The method for fast clustering target detection based on FPGA according to claim 4, characterized in that: In the inter-channel clustering unit, before caching each row of point cloud data and the corresponding cluster label into the memory, the point cloud data and the corresponding cluster label are spliced, and the spliced ​​data are cached into the FIFO memory.

6. The method for fast clustering target detection based on FPGA according to claim 5, characterized in that: When constructing eight neighborhoods for label merging of three lines of point cloud data, the eight neighborhoods are mainly implemented through 9 REG registers.

7. The method for fast clustering target detection based on FPGA according to claim 6, characterized in that: When merging tags: In the three lines of point cloud data, a 3X3 sliding window is used to traverse and determine whether the point cloud data at the center point position in each 3X3 window is a valid point. If it is not a valid point, the 3X3 window is moved; If it is a valid point, determine whether there is a valid point in the eight point cloud data adjacent to the valid point. If not, move the 3X3 window; If there is a valid point, the difference between the valid point at the center and the valid points in the eight neighborhoods is taken. If the difference is less than the threshold, the label assignment of the valid point in the eight neighborhoods is modified to the value of the valid point at the center.

8. A device for fast clustering target detection based on FPGA, characterized in that: The device comprises: A point cloud data acquisition module is used to acquire point cloud data, the point cloud data being obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of an FPGA; A valid point identification module is configured to construct a state machine using logic resources in the FPGA in the valid point identification unit, determine each row of point cloud data according to a preset size of the point cloud image, use the state machine to find the first valid point and the remaining valid points in each row of point cloud data, and pull high a first control signal when the first valid point is found, and pull high a second control signal when the remaining valid points are found. After valid point identification is completed for a row of point cloud data, the row of point cloud data, the first control signal, and the second control signal are transmitted to the intra-channel clustering unit via wiring resources within the FPGA; an intra-channel clustering module configured to cluster valid points within a row of point cloud data, the first control signal, and the second control signal in the intra-channel clustering unit based on the parallel processing capability of the FPGA, generate a cluster label for each valid point, and transmit the row of point cloud data and the corresponding cluster label to the inter-channel clustering unit; An inter-channel clustering module is configured to cache the input line point cloud data and corresponding clustering labels in the inter-channel clustering unit through two FIFO buffers until two lines of point cloud data are cached. After the third line of point cloud data is input, an eight-neighborhood is constructed for the three lines of point cloud data to perform label merging to achieve inter-channel clustering, and output line point cloud data and merged labels. The target detection module is used to perform target detection based on the merged label and line point cloud data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images