Method, device and equipment for realizing fast clustering target detection based on FPGA (Field Programmable Gate Array) and medium

Abstract

The invention relates to a method, a device and equipment for realizing fast clustering target detection based on an FPGA (Field Programmable Gate Array), and a medium. The design of parallelization of three modules, namely searching for an effective point in a channel, inter-channel clustering and intra-channel clustering, is respectively carried out by utilizing logic resources in the FPGA. The method comprises the following steps of: performing clustering marking in a channel while finding that an effective point exists in the channel; during intra-channel clustering, row point cloud data output by the intra-channel clustering module and corresponding clustering labels are cached through two FIFO buffers until two rows of point cloud data are cached, and after the third row of point cloud data is output by the intra-channel clustering module, the first row of point cloud data is cached; and 8 fields are constructed for the three rows of point cloud data to carry out label merging so as to realize clustering among channels. In the design process of the method, an assembly line processing mode is adopted, the three sub-modules in the design can be processed in parallel, and the real-time performance is effectively improved while the clustering processing accuracy is guaranteed.

Description

Technical Field

[0001] The present application relates to the technical field of image target detection, and in particular to a method, device, equipment and medium for implementing fast clustering target detection based on FPGA. Background Art

[0002] In recent years, with the rapid development of LiDAR, it has been widely used in the field of target detection due to its high precision, strong anti-interference ability and rich details of the point cloud images it obtains. Usually, in the process of using point cloud images for target detection, clustering processing is performed to mark the category of the point cloud images. The quality of the clustering results will directly affect the overall detection results.

[0003] At present, existing clustering algorithms usually perform repeated traversals on point cloud images for labeling and classification. If they are deployed and implemented using sequentially executed embedded platforms such as ARM and DSP, the point cloud images will also be traversed repeatedly during execution, which will make the processing process extremely time-consuming and the real-time requirements will be difficult to meet.

[0004] Compared with ARM and DSP, FPGA (Programmable Gate Array) platform can adopt parallel processing. If the design is reasonable, it can realize the given function in one traversal of the image during clustering, which can greatly reduce the processing time of clustering algorithm on embedded platform. It is feasible to realize fast clustering by using the parallel characteristics of FPGA platform. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, equipment and medium for fast clustering target detection based on FPGA that can improve detection efficiency in response to the above technical problems.

[0006] A method for fast clustering target detection based on FPGA, the method comprising:

[0007] Acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of an FPGA;

[0008] In the effective point recognition unit, a state machine is constructed using logic resources in the FPGA, each row of point cloud data is determined according to a preset size of the point cloud image, the state machine is used to find the first effective point and the remaining effective points in each row of point cloud data, and when the first effective point is found, the first control signal is pulled high, and when the remaining effective points are found, the second control signal is pulled high, and after the effective point recognition of a row of point cloud data is completed, the row of point cloud data, the first control signal and the second control signal are transmitted to the in-channel clustering unit through the wiring resources inside the FPGA;

[0009] In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, clustering of valid points in a row is performed according to a row of point cloud data, the first control signal, and the second control signal, and corresponding clustering marks are performed on each valid point to generate a clustering label, and the row of point cloud data and the corresponding clustering label are transmitted to the inter-channel clustering unit;

[0010] In the inter-channel clustering unit, the input line point cloud data and the corresponding clustering labels are cached by two FIFO buffers until two lines of point cloud data are cached. After the third line of point cloud data is input, 8 fields are constructed for the three lines of point cloud data to merge the labels, so as to realize inter-channel clustering, and the line point cloud data and the merged labels are output;

[0011] Target detection is performed based on the merged labels and the line point cloud data.

[0012] In one embodiment, the preset size of the point cloud image includes the number of row point clouds and the number of column point clouds in each frame of the point cloud image;

[0013] When the point cloud data is transmitted to the valid point recognition unit, the frame valid signal is first pulled high, indicating that the current frame data is transmitted, and at the same time, the row valid signal is pulled high and the row counter is counted. When the counting result meets the preset row point cloud number, it means that the data transmission of the row number is completed, the row valid signal is pulled low, and the column counter is increased by 1;

[0014] When the row valid signal is pulled high, a row of point cloud data is determined, and the state machine is used to find the first valid point and other valid points in the determined row of point cloud data;

[0015] When the counting result meets the preset number of column point clouds, the count is reset to zero and the counting of the next frame of point cloud data begins.

[0016] In one embodiment, when a state machine is used to find the first valid point and other valid points in each row of point cloud data, the process includes:

[0017] The state machine includes an initial idle state, a first valid point waiting state, a first valid point found state, and other valid point waiting states;

[0018] The state machine is initially in an idle state, and when the row valid signal is pulled high and the point cloud data is transmitted, edge detection is performed on each point cloud data, and the state machine enters the first valid point waiting state;

[0019] In the first valid point waiting state, when the first point cloud data is greater than 0, the first valid point is found, and the first valid point is found state is entered. If the row valid signal is pulled low, the state returns to the idle state and waits for the next row of point cloud data.

[0020] In the first valid point found state, after the first control signal is pulled high, jump to other valid point waiting state;

[0021] In the waiting state of other valid points, when a certain point cloud data is greater than 0, the second control signal is pulled high, and the idle state is returned to wait for the next line of point cloud data.

[0022] In one embodiment, in the intra-channel clustering unit:

[0023] Detecting a line of point cloud data according to the first control signal, and assigning a value to a label corresponding to the first valid point when the first valid point is detected;

[0024] After finding the first valid point, other valid points are detected for a row of point cloud data according to the second control signal. When other valid points are detected, the current valid point is subtracted from the previous valid point. If the difference is less than a preset threshold, the label corresponding to the current valid point is assigned to the label of the previous valid point.

[0025] If the difference is greater than a preset threshold, the label value corresponding to the current valid point is assigned a value plus one.

[0026] In one embodiment, before caching each row of point cloud data and the corresponding clustering label into a memory, the inter-channel clustering unit also splices the point cloud data with the corresponding clustering label, and caches the spliced ​​data into a FIFO memory.

[0027] In one embodiment, when constructing 8 neighborhoods for three lines of point cloud data to perform label merging, the 8 neighborhoods are mainly implemented through 9 REG registers.

[0028] In one embodiment, when performing tag merging:

[0029] In the three lines of point cloud data, a 3X3 sliding window traversal is used to determine whether the point cloud data at the center point position in each 3X3 window is a valid point. If it is not a valid point, the 3X3 window is moved;

[0030] If it is a valid point, determine whether there is a valid point in the eight point cloud data adjacent to the valid point. If not, move the 3X3 window;

[0031] If there is a valid point, the difference between the valid point at the center and the valid point in the eight neighborhoods is taken. If the difference is less than the threshold, the label assignment of the valid point in the eight neighborhoods is modified to the assignment of the valid point at the center.

[0032] The present application also provides a device for realizing fast clustering target detection based on FPGA, the device comprising:

[0033] A point cloud data acquisition module is used to acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of the FPGA;

[0034] An effective point recognition module is used to construct a state machine using logic resources in the FPGA in the effective point recognition unit, determine each row of point cloud data according to a preset size of the point cloud image, use the state machine to find the first effective point and the remaining effective points in each row of point cloud data, and pull up a first control signal when the first effective point is found, and pull up a second control signal when the remaining effective points are found, and after completing the effective point recognition of a row of point cloud data, transmit the row of point cloud data, the first control signal and the second control signal to the in-channel clustering unit through the wiring resources inside the FPGA;

[0035] An intra-channel clustering module is used to cluster valid points in a row according to a row of point cloud data, a first control signal and a second control signal in the intra-channel clustering unit based on the parallel processing capability of the FPGA, and to generate cluster labels by correspondingly marking each valid point, and to transmit the row of point cloud data and the corresponding cluster labels to the inter-channel clustering unit;

[0036] An inter-channel clustering module is used to cache the input line point cloud data and the corresponding clustering labels through two FIFO buffers in the inter-channel clustering unit until two lines of point cloud data are cached, and after the third line of point cloud data is input, construct 8 fields for the three lines of point cloud data to merge the labels to achieve inter-channel clustering, and output the line point cloud data and the merged labels;

[0037] The target detection module is used to perform target detection based on the merged label and line point cloud data.

[0038] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0039] Acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of an FPGA;

[0040] In the effective point recognition unit, a state machine is constructed using logic resources in the FPGA, each row of point cloud data is determined according to a preset size of the point cloud image, the state machine is used to find the first effective point and the remaining effective points in each row of point cloud data, and when the first effective point is found, the first control signal is pulled high, and when the remaining effective points are found, the second control signal is pulled high, and after the effective point recognition of a row of point cloud data is completed, the row of point cloud data, the first control signal and the second control signal are transmitted to the in-channel clustering unit through the wiring resources inside the FPGA;

[0041] In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, clustering of valid points in a row is performed according to a row of point cloud data, the first control signal, and the second control signal, and corresponding clustering marks are performed on each valid point to generate a clustering label, and the row of point cloud data and the corresponding clustering label are transmitted to the inter-channel clustering unit;

[0042] In the inter-channel clustering unit, the input line point cloud data and the corresponding clustering labels are cached by two FIFO buffers until two lines of point cloud data are cached. After the third line of point cloud data is input, 8 fields are constructed for the three lines of point cloud data to merge the labels, so as to realize inter-channel clustering, and the line point cloud data and the merged labels are output;

[0043] Target detection is performed based on the merged labels and the line point cloud data.

[0044] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0045] Acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of an FPGA;

[0046] In the effective point recognition unit, a state machine is constructed using logic resources in the FPGA, each row of point cloud data is determined according to a preset size of the point cloud image, the state machine is used to find the first effective point and the remaining effective points in each row of point cloud data, and when the first effective point is found, the first control signal is pulled high, and when the remaining effective points are found, the second control signal is pulled high, and after the effective point recognition of a row of point cloud data is completed, the row of point cloud data, the first control signal and the second control signal are transmitted to the in-channel clustering unit through the wiring resources inside the FPGA;

[0047] In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, clustering of valid points in a row is performed according to a row of point cloud data, the first control signal, and the second control signal, and corresponding clustering marks are performed on each valid point to generate a clustering label, and the row of point cloud data and the corresponding clustering label are transmitted to the inter-channel clustering unit;

[0048] In the inter-channel clustering unit, the input line point cloud data and the corresponding clustering labels are cached by two FIFO buffers until two lines of point cloud data are cached. After the third line of point cloud data is input, 8 fields are constructed for the three lines of point cloud data to merge the labels, so as to realize inter-channel clustering, and the line point cloud data and the merged labels are output;

[0049] Target detection is performed based on the merged labels and the line point cloud data.

[0050] The above-mentioned method, device, equipment and medium for realizing fast clustering target detection based on FPGA, by using the logic resources in FPGA to build a state machine, using the state machine to find the first valid point and the remaining valid points in each row of point cloud data, and pull up the first control signal when the first valid point is found, and pull up the second control signal when the remaining valid points are found, based on the parallel processing capability of FPGA, cluster the valid points in the row according to a row of point cloud data, the first control signal and the second control signal, and generate cluster labels for each valid point by corresponding clustering marks, cache the input row point cloud data and the corresponding cluster labels through two FIFO buffers until two rows of point cloud data are cached, and after the third row of point cloud data is input, the three rows of point cloud data are constructed into 8 fields for label merging to realize clustering between channels, and target detection is performed according to the merged labels and row point cloud data. The method is used to effectively improve the efficiency of clustering in the target detection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a process for implementing a fast clustering target detection method based on FPGA in one embodiment;

[0052] Figure 2 A schematic diagram of an overall framework of FPGA implementation in one embodiment;

[0053] Figure 3 A schematic diagram of state transition in an embodiment;

[0054] Figure 4 is a timing diagram of a valid point identification unit in one embodiment;

[0055] Figure 5 A schematic diagram of the timing in a clustering unit in a channel in one embodiment;

[0056] Figure 6 Schematic diagram of a framework of an inter-channel clustering unit in one embodiment;

[0057] Figure 7 is a logical schematic diagram of inter-channel clustering in one embodiment;

[0058] Figure 8 It is a structural block diagram of a device for realizing fast clustering target detection based on FPGA in one embodiment;

[0059] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] In view of the problem that in the prior art, when implementing the clustering algorithm, a frame of data needs to be traversed multiple times, resulting in low efficiency, in one embodiment, Figure 1 As shown, a method for fast clustering target detection based on FPGA is provided, which specifically includes the following steps:

[0062] Step S100, acquiring point cloud data, the point cloud data is obtained by detecting the air through a laser radar, and the point cloud data is transmitted to a valid point recognition unit through a high-speed data interface of the FPGA.

[0063] Step S110, in the effective point recognition unit, a state machine is constructed using the logic resources in the FPGA, each row of point cloud data is determined according to the preset size of the point cloud image, and the state machine is used to find the first effective point and the remaining effective points in each row of point cloud data, and when the first effective point is found, the first control signal is pulled high, and when the remaining effective points are found, the second control signal is pulled high. After completing the effective point recognition of a row of point cloud data, the row of point cloud data, the first control signal and the second control signal are transmitted to the clustering unit in the channel through the wiring resources inside the FPGA.

[0064] Step S120, in the intra-channel clustering unit, based on the parallel processing capability of the FPGA, cluster the valid points in the row according to a row of point cloud data, the first control signal and the second control signal, and perform corresponding clustering marking on each valid point to generate a clustering label, and transmit this row of point cloud data and the corresponding clustering label to the inter-channel clustering unit.

[0065] Step S130, in the inter-channel clustering unit, the input row point cloud data and the corresponding clustering labels are cached through two FIFO buffers until two rows of point cloud data are cached. After the third row of point cloud data is input, 8 fields are constructed for these three rows of point cloud data for label merging to achieve inter-channel clustering, and the row point cloud data and the merged labels are output.

[0066] Step S140: Detect the target based on the merged label and the line point cloud data.

[0067] In the process of implementing the existing clustering algorithm, each time it is executed, it is necessary to traverse the image once to find the valid points, and a lot of time is spent in each process. In this application, FPGA is used to implement the clustering algorithm, and it is processed in the form of a pipeline, which greatly improves the clustering speed. Based on FPGA, firstly, when searching for the first valid point of each row, marking in the channel can save the time of traversing the image once. Secondly, after the clustering in the channel completes the marking of three rows, the relevant uplink and downlink can be generated for clustering between channels, which can also save the time of traversing the image once. This method will be difficult to implement for embedded devices such as ARM and DSP that execute sequentially. Therefore, FPGA, an embedded device with timing and parallel processing, will be used to accelerate the implementation of the fast clustering algorithm.

[0068] Specifically, FPGA, as an embedded platform capable of parallel acceleration, has abundant programmable logic resources. When performing algorithm processing, each submodule can be processed in parallel, reducing the number of image traversals, greatly reducing processing time, and improving real-time performance.

[0069] like Figure 2 As shown, when the clustering algorithm is implemented using FPGA, it is divided into three units including a valid point recognition unit, an intra-channel clustering unit, and an inter-channel clustering unit.

[0070] Specifically, after the point cloud data Data is transmitted to the FPGA. First, it enters the valid point identification unit that searches for the first valid point of each row. In this unit, the frame and row valid signals must be calculated according to the size of the point cloud image, and the first valid point of each row is found under this timing to generate the control signal First_valid, i.e. the first control signal, and Data_valid, i.e. the second control signal. The signal and the point cloud data are given to the intra-channel clustering unit for subsequent processing. The intra-channel clustering unit performs intra-channel clustering according to the control signals First_valid and Data_valid and the point cloud data at this time to generate the label category of the corresponding data. The inter-channel clustering unit will read out the three rows of data after the intra-channel clustering processing marks them, and process them according to the conditions for merging the labels. Finally, the processed labels and data are placed in two RAMs for result comparison.

[0071] In step S110, the preset size of the point cloud image includes the number of row point clouds and the number of column point clouds of each frame of the point cloud image. Furthermore, the data of a frame of the point cloud image can be further determined according to the number of row point clouds and the number of column point clouds. When the point cloud data is transmitted to the valid point recognition unit, the frame valid signal is first pulled high, indicating that the current frame data is transmitted. At the same time, the row valid signal is pulled high and the row counter is counted. When the counting result meets the preset number of row point clouds, it means that the data transmission of the row number is completed. The row valid signal is pulled low, and the column counter is added by 1. When the row valid signal is pulled high, a row of point cloud data is determined, and the state machine is used to find the first valid point and the remaining valid points in the determined row of point cloud data. When the counting result meets the preset number of column point clouds, the count is reset to zero, and the counting of the next frame of point cloud data begins.

[0072] Specifically, the frame and row valid signals can be counted according to the input data and generated according to the specified point cloud image size. For example, a 400*125 point cloud image can have a row valid signal pulled high when the row counter counts from 0 to 400, and a frame valid signal pulled high when the column counter counts from 0-125.

[0073] like Figure 3As shown, when a state machine is used to find the first valid point and the remaining valid points in each row of point cloud data, it includes: the state machine includes an initial idle state (IDLE), a first valid point waiting state (WAIT_FRST_DATA), a first valid point found state (FIRST_DATA_IDX) and other valid point waiting states (DATA_IDX). The state machine is initially in the idle state (IDLE), and the valid signal of the waiting row is pulled high, the point cloud data is transmitted, and the edge detection is performed on each point cloud data, and the state enters the first valid point waiting state (WAIT_FRST_DATA). In the first valid point waiting state, when the first point cloud data is greater than 0, the first valid point is found, and the state enters the first valid point found state. If the valid signal of the row is pulled low, it returns to the idle state to wait for the next row of point cloud data. In the first valid point found state (FIRST_DATA_IDX), after pulling the first control signal high, it jumps to the other valid point waiting state (DATA_IDX). In the other valid point waiting state (DATA_IDX), when a certain point cloud data is greater than 0, the second control signal is pulled high, and the state returns to the idle state (IDLE) to wait for the next row of point cloud data.

[0074] Specifically, in the IDLE state, the waiting row valid signal is pulled high, and the first point cloud data is started to be transmitted, wherein de==1'b1&&de_r==1'b0 is the edge detection state and enters WAIT_FRST_DATA.

[0075] In the WAIT_FRST_DATA state, wait for the first valid point. When data data>0, it means that this point is the first valid point, and the state enters FIRST_DATA_IDX. If the row valid signal is pulled low, de==1'b0&&de_r==1'b1, it means that there is no valid point in this row, and the state returns to the IDLE state to wait for the next row of data to be transmitted.

[0076] The state FIRST_DATA_IDX is an assignment state. At this time, the first valid point is detected, so the control signal First_valid is pulled high, and then the state jumps to the DATA_IDX state.

[0077] In the DATA_IDX state, if data>0, it means that all valid points except the first valid point of the row are detected, so the control signal Data_valid will be pulled high. If the row valid signal is pulled low, de==1'b0&&de_r==1'b1, it means that the detection of the row has been completed, and it returns to the IDLE state, waiting for the row valid signal of the second row to be pulled high, and then the second row is detected.

[0078] like Figure 4As shown in FIG. 1 , it is a timing diagram of a frame vs, a row valid signal de, point cloud data data, a first control signal First_valid and a second control signal Data_valid in a valid point recognition unit. Figure 4 It can be seen that the Data starts to be detected by the state machine. When the first valid point of Data>0 in each row appears, the First_valid signal will be pulled high. When the valid point of Data>0 appears subsequently, the Data_valid signal will be pulled high.

[0079] In step S120, in the intra-channel clustering unit: first, a line of point cloud data is detected according to the first control signal, and when the first valid point is detected, the label corresponding to the valid point is assigned, and after finding the first valid point, other valid points are detected for a line of point cloud data according to the second control signal, and when other valid points are detected, the current valid point is subtracted from the previous valid point, and if the difference is less than a preset threshold, the label corresponding to the current valid point is assigned to the label of the previous valid point, that is, the two valid points are clustered, and if the difference is greater than the preset threshold, the label assignment corresponding to the current valid point is increased by one, then the valid point may be another target, and the label assignment of the valid point is different from that of the previous valid point.

[0080] like Figure 5 As shown in FIG. 1 , a timing diagram of a frame vs, a row valid signal de, a point cloud data data, a first control signal First_vali, a second control signal Data_valid, and a timing diagram of a clustering label Label in a channel clustering unit. Figure 5 It can be seen that this operation mode conforms to the logic of the algorithm and uses a pipeline structure. While completing the search for the first valid point in each row, the marking of the clusters within the channel is completed according to the established marking logic, which saves the time of traversing an image compared to the implementation of the original algorithm.

[0081] In step S130, in the inter-channel clustering unit, the data relationship between the current row and the previous row and the next row is mainly compared, so as to further merge the clustering labels, that is, merge the inter-row data. Figure 6 shown.

[0082] In this embodiment, in the inter-channel clustering unit, before caching each row of point cloud data and the corresponding clustering label into the FIFO memory, because both the data and the label need to be used in this unit at the same time, the point cloud data and the corresponding clustering label are also spliced ​​to obtain Label+Data, and the spliced ​​data is cached into the FIFO memory.

[0083] Furthermore, it is necessary to build an 8-neighborhood for tag merging. Building an 8-neighborhood requires at least three rows of data, so two FIFOs are needed to cache the data. Compared with the usual construction of 8 neighborhoods, this design is different because it uses the data after tag merging. Therefore, the first row of data cached to FIFO1 is the data after bit concatenation, such as Figure 6 As shown by the blue line in the middle, the second line caches the data after the tag merging process into FIFO1, and as shown by the red line, each line after the current frame will use the data after the tag merging process.

[0084] Finally, after the label merging process, the 8 neighborhoods are mainly composed of 9 REG registers. After completion, the value of register REG window11 is split into data Data and label Label and stored in RAM1 and RAM2 for result comparison. The logic of label merging process is as follows: Figure 7 shown.

[0085] like Figure 7 As shown in the figure, when merging labels: in the three rows of point cloud data, a 3X3 sliding window traversal is used to determine whether the point cloud data at the center point position in each 3X3 window is a valid point. If it is not a valid point, the 3X3 window is moved. If it is a valid point, it is determined whether there is a valid point in the eight point cloud data adjacent to the valid point. If not, the 3X3 window is moved. If there is a valid point, the difference between the valid point at the center position and the valid point in the eight neighborhoods is made. If the difference is less than the threshold, the label assignment of the valid point in the eight neighborhoods is modified to the assignment of the valid point at the center position.

[0086] Specifically, the 3x3 window corresponds to 9 registers REG Window11-REG Window33, with the center point being REGWindow22. The comparison is performed by using 8 independent modules to compare simultaneously, making full use of the advantages of FPGA parallel processing and completing label updates for 8 neighborhoods simultaneously.

[0087] In step S140, the number of targets in a frame of image can be determined based on the merged labels, and subsequently only the clustering result screening and frame marking are required to complete the overall target detection process.

[0088] In the above-mentioned FPGA-based fast clustering target detection method, the point cloud data is processed in a pipeline manner, which effectively improves the real-time performance of the algorithm on the embedded platform. First, the intra-channel clustering is completed while finding the first valid point in each row. Secondly, when clustering within the channel, only two lines of data need to be cached to process the real-time output results. Combining these two points saves about two times of traversing the image. In the design process of this method, the pipeline processing mode is adopted. The three sub-modules in the design can be processed in parallel, which effectively improves the real-time performance while ensuring the accuracy of the algorithm processing.

[0089] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0090] In one embodiment, Figure 8 As shown, a device for realizing fast clustering target detection based on FPGA is provided, comprising: a point cloud data acquisition module 200, a valid point recognition module 210, an intra-channel clustering module 220, an inter-channel clustering module 230 and a target detection module 240, wherein:

[0091] The point cloud data acquisition module 200 is used to acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of the FPGA;

[0092] The effective point recognition module 210 is used to construct a state machine using logic resources in the FPGA in the effective point recognition unit, determine each row of point cloud data according to a preset size of the point cloud image, use the state machine to find the first effective point and the remaining effective points in each row of point cloud data, and when the first effective point is found, pull up the first control signal, and when the remaining effective points are found, pull up the second control signal, and after completing the effective point recognition of a row of point cloud data, transmit the row of point cloud data, the first control signal and the second control signal to the in-channel clustering unit through the wiring resources inside the FPGA;

[0093] The intra-channel clustering module 220 is used to cluster valid points in a row according to a row of point cloud data, the first control signal and the second control signal in the intra-channel clustering unit based on the parallel processing capability of the FPGA, and to generate cluster labels for corresponding cluster marks for each valid point, and transmit the row of point cloud data and the corresponding cluster labels to the inter-channel clustering unit;

[0094] The inter-channel clustering module 230 is used to cache the input line point cloud data and the corresponding clustering labels through two FIFO buffers in the inter-channel clustering unit until two lines of point cloud data are cached, and after the third line of point cloud data is input, construct 8 fields for the three lines of point cloud data to merge the labels to achieve inter-channel clustering, and output the line point cloud data and the merged labels;

[0095] The target detection module 240 is used to perform target detection according to the merged label and the line point cloud data.

[0096] For the specific limitations of the fast clustering target detection device based on FPGA, please refer to the limitations of the fast clustering target detection method based on FPGA above, which will not be repeated here. Each module in the above-mentioned fast clustering target detection device based on FPGA can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0097] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for fast clustering target detection based on FPGA is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0098] Those skilled in the art will understand that Fig. 9The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0099] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0100] Acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of an FPGA;

[0101] In the effective point recognition unit, a state machine is constructed using logic resources in the FPGA, each row of point cloud data is determined according to a preset size of the point cloud image, the state machine is used to find the first effective point and the remaining effective points in each row of point cloud data, and when the first effective point is found, the first control signal is pulled high, and when the remaining effective points are found, the second control signal is pulled high, and after the effective point recognition of a row of point cloud data is completed, the row of point cloud data, the first control signal and the second control signal are transmitted to the in-channel clustering unit through the wiring resources inside the FPGA;

[0102] In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, clustering of valid points in a row is performed according to a row of point cloud data, the first control signal, and the second control signal, and corresponding clustering marks are performed on each valid point to generate a clustering label, and the row of point cloud data and the corresponding clustering label are transmitted to the inter-channel clustering unit;

[0103] In the inter-channel clustering unit, the input line point cloud data and the corresponding clustering labels are cached by two FIFO buffers until two lines of point cloud data are cached. After the third line of point cloud data is input, 8 fields are constructed for the three lines of point cloud data to merge the labels, so as to realize inter-channel clustering, and the line point cloud data and the merged labels are output;

[0104] Target detection is performed based on the merged labels and the line point cloud data.

[0105] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0106] Acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of an FPGA;

[0107] In the effective point recognition unit, a state machine is constructed using logic resources in the FPGA, each row of point cloud data is determined according to a preset size of the point cloud image, the state machine is used to find the first effective point and the remaining effective points in each row of point cloud data, and when the first effective point is found, the first control signal is pulled high, and when the remaining effective points are found, the second control signal is pulled high, and after the effective point recognition of a row of point cloud data is completed, the row of point cloud data, the first control signal and the second control signal are transmitted to the in-channel clustering unit through the wiring resources inside the FPGA;

[0108] In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, clustering of valid points in a row is performed according to a row of point cloud data, the first control signal, and the second control signal, and corresponding clustering marks are performed on each valid point to generate a clustering label, and the row of point cloud data and the corresponding clustering label are transmitted to the inter-channel clustering unit;

[0109] In the inter-channel clustering unit, the input line point cloud data and the corresponding clustering labels are cached by two FIFO buffers until two lines of point cloud data are cached. After the third line of point cloud data is input, 8 fields are constructed for the three lines of point cloud data to merge the labels, so as to realize inter-channel clustering, and the line point cloud data and the merged labels are output;

[0110] Target detection is performed based on the merged labels and the line point cloud data.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0112] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for fast clustering target detection based on FPGA, characterized in that: The method comprises: Acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of an FPGA; In the effective point recognition unit, a state machine is constructed using logic resources in the FPGA, each row of point cloud data is determined according to a preset size of the point cloud image, the state machine is used to find the first effective point and the remaining effective points in each row of point cloud data, and when the first effective point is found, the first control signal is pulled high, and when the remaining effective points are found, the second control signal is pulled high, and after the effective point recognition of a row of point cloud data is completed, the row of point cloud data, the first control signal and the second control signal are transmitted to the in-channel clustering unit through the wiring resources inside the FPGA; In the intra-channel clustering unit, based on the parallel processing capability of the FPGA, clustering of valid points in a row is performed according to a row of point cloud data, the first control signal, and the second control signal, and corresponding clustering marks are performed on each valid point to generate a clustering label, and the row of point cloud data and the corresponding clustering label are transmitted to the inter-channel clustering unit; In the inter-channel clustering unit, the input line point cloud data and the corresponding clustering labels are cached by two FIFO buffers until two lines of point cloud data are cached. After the third line of point cloud data is input, 8 fields are constructed for the three lines of point cloud data to merge the labels, so as to realize inter-channel clustering, and the line point cloud data and the merged labels are output; Target detection is performed based on the merged labels and the line point cloud data.

2. The method for fast clustering target detection based on FPGA according to claim 1, characterized in that: The preset size of the point cloud image includes the number of row point clouds and the number of column point clouds in each frame of the point cloud image; When the point cloud data is transmitted to the valid point recognition unit, the frame valid signal is first pulled high, indicating that the current frame data is transmitted, and at the same time, the row valid signal is pulled high and the row counter is counted. When the counting result meets the preset row point cloud number, it means that the data transmission of the row number is completed, the row valid signal is pulled low, and the column counter is increased by 1; When the row valid signal is pulled high, a row of point cloud data is determined, and the state machine is used to find the first valid point and other valid points in the determined row of point cloud data; When the counting result meets the preset number of column point clouds, the count is reset to zero and the counting of the next frame of point cloud data begins.

3. The method for fast clustering target detection based on FPGA according to claim 2, characterized in that: When using a state machine to find the first valid point and other valid points in each row of point cloud data, it includes: The state machine includes an initial idle state, a first valid point waiting state, a first valid point found state, and other valid point waiting states; The state machine is initially in an idle state, and when the row valid signal is pulled high and the point cloud data is transmitted, edge detection is performed on each point cloud data, and the state machine enters the first valid point waiting state; In the first valid point waiting state, when the first point cloud data is greater than 0, the first valid point is found, and the first valid point is found state is entered. If the row valid signal is pulled low, the state returns to the idle state and waits for the next row of point cloud data. In the first valid point found state, after the first control signal is pulled high, jump to other valid point waiting state; In the waiting state of other valid points, when a certain point cloud data is greater than 0, the second control signal is pulled high, and the idle state is returned to wait for the next line of point cloud data.

4. The method for fast clustering target detection based on FPGA according to claim 3, characterized in that: In the channel clustering unit: Detecting a line of point cloud data according to the first control signal, and assigning a value to a label corresponding to the first valid point when the first valid point is detected; After finding the first valid point, other valid points are detected for a row of point cloud data according to the second control signal. When other valid points are detected, the current valid point is subtracted from the previous valid point. If the difference is less than a preset threshold, the label corresponding to the current valid point is assigned to the label of the previous valid point. If the difference is greater than a preset threshold, the label value corresponding to the current valid point is assigned a value plus one.

5. The method for fast clustering target detection based on FPGA according to claim 4, characterized in that: In the inter-channel clustering unit, before caching each line of point cloud data and the corresponding clustering label into the memory, the point cloud data and the corresponding clustering label are spliced, and the spliced ​​data are cached into the FIFO memory.

6. The method for fast clustering target detection based on FPGA according to claim 5, characterized in that: When constructing 8 neighborhoods for label merging of three-line point cloud data, the 8 neighborhoods are mainly implemented through 9 REG registers.

7. The method for fast clustering target detection based on FPGA according to claim 6, characterized in that: When merging tags: In the three lines of point cloud data, a 3X3 sliding window traversal is used to determine whether the point cloud data at the center point position in each 3X3 window is a valid point. If it is not a valid point, the 3X3 window is moved; If it is a valid point, determine whether there is a valid point in the eight point cloud data adjacent to the valid point. If not, move the 3X3 window; If there is a valid point, the difference between the valid point at the center and the valid point in the eight neighborhoods is taken. If the difference is less than the threshold, the label assignment of the valid point in the eight neighborhoods is modified to the assignment of the valid point at the center.

8. A fast clustering target detection device based on FPGA, characterized in that: The device comprises: A point cloud data acquisition module is used to acquire point cloud data, the point cloud data is obtained by detecting the air with a laser radar, and transmit the point cloud data to a valid point recognition unit through a high-speed data interface of the FPGA; An effective point recognition module is used to construct a state machine using logic resources in the FPGA in the effective point recognition unit, determine each row of point cloud data according to a preset size of the point cloud image, use the state machine to find the first effective point and the remaining effective points in each row of point cloud data, and pull up a first control signal when the first effective point is found, and pull up a second control signal when the remaining effective points are found, and after completing the effective point recognition of a row of point cloud data, transmit the row of point cloud data, the first control signal and the second control signal to the in-channel clustering unit through the wiring resources inside the FPGA; An intra-channel clustering module is used to cluster valid points in a row according to a row of point cloud data, a first control signal and a second control signal in the intra-channel clustering unit based on the parallel processing capability of the FPGA, and to generate cluster labels by correspondingly marking each valid point, and to transmit the row of point cloud data and the corresponding cluster labels to the inter-channel clustering unit; An inter-channel clustering module is used to cache the input line point cloud data and the corresponding clustering labels through two FIFO buffers in the inter-channel clustering unit until two lines of point cloud data are cached, and after the third line of point cloud data is input, construct 8 fields for the three lines of point cloud data to merge the labels to achieve inter-channel clustering, and output the line point cloud data and the merged labels; The target detection module is used to perform target detection based on the merged label and line point cloud data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images