A Real-Time Display Method for Multiple Predicted Boxes Accelerated by FPGA-Based Object Detection Algorithm

Abstract

This invention provides a real-time display method for multiple predicted bounding boxes based on FPGA-accelerated object detection algorithms, relating to the field of deep learning algorithm technology for object detection. The steps are as follows: each feature map segment of the regression branch is post-processed to generate N predicted bounding boxes; the coordinates of the predicted bounding box bbox1 are analyzed [(x1...]. 1 ,y1 1 ),(x2 1 ,y2 1 )], read the original y1 from BRAM 1 The row mask is passed into a one-dimensional register set as the initial value, and the value is set from x1 in the one-dimensional register set. 1 List to x2 1 Column completed y1 1 The row mask is updated and written back to BRAM; the remaining y1 values ​​in the prediction box bbox1 are completed row by row. 1 +1 to y2 1 The masks for all rows are updated and written back to the BRAM; following the same method, the masks for the remaining bbox2 to bboxN are updated one by one and written back to the BRAM; the video stream is bounding based on all updated masks. If the mask is 0, the original image data is used; if the mask is 1, the original image data is replaced with the color of the predicted bounding box. This invention can generate the image mask in real time, eliminating the need to generate all predicted bounding boxes before generating the mask, thus effectively improving inference speed.

Description

Technical Field

[0001] This invention relates to the field of deep learning algorithm technology for object detection, and in particular to a real-time display method for multiple prediction boxes based on FPGA-accelerated object detection algorithm. Background Technology

[0002] In the field of computer vision, deep learning algorithms for object detection have become mainstream, comprising a backbone network, a neck network, and a detection head. Currently, most detection heads employ a decoupled structure without anchor points, separating classification and regression. The detection head generates numerous candidate boxes with varying confidence levels for the same object, often with overlapping areas. Post-processing (e.g., Non-Maximum Suppression, NMS) helps find the optimal predicted bounding box and remove redundant predictions.

[0003] At the edge, accelerating FPGA-based deep learning algorithms for object detection requires real-time performance. This necessitates not only optimizing the deep learning algorithm (e.g., quantization) but also real-time processing of the predicted bounding boxes. For images with high object density and numerous predicted bounding boxes, generating the mask only after all predicted bounding boxes are generated would result in long computation times and poor real-time performance. Summary of the Invention

[0004] Based on the above problems, the purpose of this invention is to propose a real-time display method for multiple prediction boxes based on FPGA-accelerated target detection algorithms. This method can generate the mask of the image in real time, without having to generate all prediction boxes before generating the mask, thus effectively improving the inference speed.

[0005] The technical solution adopted by this invention to achieve its objective is a real-time display method for multiple prediction boxes based on FPGA-accelerated target detection algorithms, comprising the following steps:

[0006] S1. Divide the regression branch feature map of the detection head into several block feature maps;

[0007] S2. Each block feature map undergoes post-processing to generate N predicted bounding boxes bbox1 to bboxN, where the coordinate information of the nth predicted bounding box bboxN is [(x1... n ,y1 n ),(x2 n ,y2 n )], where (x1 n ,y1 n (x2) is the coordinate of the top left corner. n ,y2 n () is the coordinate of the lower right corner, 1≤n≤N, where N is a positive integer;

[0008] S3. Write all prediction boxes into the FIFO;

[0009] S4. Detect the empty signal of the FIFO. When the empty signal is "0", i.e., it is not empty, proceed to step S5.

[0010] S5. Analyze the coordinates of the predicted bounding box bbox1 [(x1... 1 ,y1 1 ),(x2 1 ,y2 1 )];

[0011] S6. Read the original y1 from the 640×640 BRAM. 1 The row mask is passed into the one-dimensional register set DFF640 as y1. 1 The initial value of the row; because of y1 1 The row may contain the mask of the previously predicted bounding box;

[0012] From x1 on the one-dimensional register set DFF640 1 Starting from column x2 1 End of column y1 1 Update the row mask and update y1 in the one-dimensional register set DFF640. 1 The row mask value is written back to y1 of a 640×640 BRAM. 1 OK;

[0013] S7. Following the method in step S6, complete the remaining y1 values ​​in the prediction box bbox1 line by line. 1 +1 to y2 1 Update the mask for all rows and write back the corresponding y1 in the 640×640 BRAM. 1 +1 to y2 1 OK;

[0014] At this point, after steps S6 and S7, the mask update of bbox1 is completed in the 640×640 BRAM;

[0015] S8. Following the method of steps S5 to S7, update the mask of the remaining prediction boxes bbox2 to bboxN one by one, and write back the corresponding position of the 640×640 BRAM.

[0016] S9. The video stream draws bounding boxes based on all the updated masks in step S8. If the mask is 0, the original image data is used; if the mask is 1, the original image data is replaced with the color of the predicted bounding box, such as red.

[0017] Furthermore, the regression branch feature map of the detection head has three sizes: 20×20, 40×40, or 80×80.

[0018] Furthermore, the size of the segmented feature map is 8×8. The sizes of the 20×20, 40×40, and 80×80 feature maps after being divided into 8×8 blocks are 3×3, 5×5, and 10×10, respectively. If there are blocks smaller than 8×8 after the division, "0" is padded to make it 8×8.

[0019] Furthermore, the mask update in steps S6 and S7, for y1 1 line and y2 1 Line, from x1 1 to x2 1 Update the mask column by column for all columns, for y1 1 and y2 1 The rows between them only contain x1 1 and x2 1 Two columns update mask;

[0020] Furthermore, the specific logic for mask updating, i.e., converting the predicted bounding box coordinates into a mask, is as follows:

[0021] In the idle state, the row address addr_y is 0, the column address addr_x is 0, the FIFO read enable fifo_rd_en is 0, and if the FIFO is not empty (empty == 0), it enters the bbox_loop state.

[0022] In the bbox_loop state, if the FIFO is not empty (empty == 0), it enters the fifo_wait state, the FIFO read enable fifo_rd_en is 1, and the mask completion mask_done is 0. If the FIFO is empty (empty == 1), the FIFO read enable fifo_rd_en is 0. At this time, if all bboxes have been generated (bbox_done_flag == 1), the mask completion mask_done is 1, and it enters the idle state.

[0023] In the fifo_wait state, it enters the bram_wait state, the FIFO read enable fifo_rd_en is 0, the row address addr_y is y1, and the column address addr_x is x1;

[0024] In the bram_wait state, enter the y_loop state, and set the original mask reading enable old640_rd_en to 1;

[0025] In the y_loop state, the original mask read enable old640_rd_en is 0. If addr_y == y1 || addr_y == y2, it enters the top_down state. If addr_y < y2 && addr_y > y1, it enters the left state. Otherwise, it enters the mask_update state;

[0026] In the top_down state, the data din_x corresponding to addr_x is 1, the column write enable we_x is 1, and it enters the x_incr state;

[0027] In the x_incr state, the write enable we_x is 0. If addr_x >= x1 && addr_x < x2, the column address addr_x is incremented by 1. Otherwise, it enters the x_loop_end state;

[0028] In the left state, addr_x is x1, the data din_x is 1, the column write enable we_x is 1, and it enters the right state;

[0029] In the right state, addr_x is x2, the data din_x is 1, the column write enable we_x is 1, and it enters the x_loop_end state;

[0030] In the x_loop_end state, addr_x is x1, the data din_x is 0, the column write enable we_x is 0, the row write enable we_y is 1, and it enters the mask_update state;

[0031] In the mask_update state, the row write enable we_y is 0, the row address addr_y is incremented by 1, and it is judged whether addr_y == y2. If true, it enters the y_loop_end state. If false, it enters the bram_wait state to update the mask of the next row;

[0032] In the y_loop_end state, the number of processed bboxes is incremented by 1, and it enters the bbox_loop state;

[0033] Where:

[0034] `idle` represents the idle state, `bbox_loop` represents the bbox traversal state, `fifo_wait` represents the bbox FIFO read state, `bram_wait` represents the original mask read state, `y_loop` represents the loop state of row address `addr_y`, `top_down` represents the state of the top and bottom borders of the bbox, `x_incr` represents the state of the column address `addr_x`, `left` represents the state of the left border of the bbox, `right` represents the state of the right border of the bbox, `x_loop_end` represents the state of the end of column address `addr_x`, `mask_update` represents the mask update state, and `y_loop_end` represents the state of the end of row address `addr_y`.

[0035] The beneficial effects of this invention are as follows:

[0036] When accelerating deep learning algorithms for object detection based on FPGA, FPGA hardware cannot generate 20×20, 40×40, or 80×80 feature maps at once. It requires dividing the feature maps into blocks for computation. For example, if an 8×8 block feature map is generated in one computation, then an N×N feature map is divided into multiple 8×8 block feature maps for computation. In this invention, when each predicted bounding box is generated after post-processing of the 8×8 block feature map of the object detection regression branch, a mask is generated for that predicted bounding box. Mask updates are performed in real-time and efficiently using FIFO, a one-dimensional register set, and a two-dimensional BRAM. This method can pipeline and generate predicted bounding box masks in real-time, eliminating the need to generate masks only after all predicted bounding boxes have been generated. This reduces computational steps, effectively improves inference speed, and enhances real-time performance. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of two branches of the detection head decoupling structure in an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of a 640×640 mask according to an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of a 640×640 two-dimensional mask BRAM according to an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram illustrating the process of drawing frames in a video stream based on an updated mask, according to an embodiment of the present invention.

[0041] Figure 5 This is a schematic diagram of the real-time mask generation method for multiple prediction boxes according to an embodiment of the present invention;

[0042] Figure 6 This is a state transition diagram for mask updating according to an embodiment of the present invention;

[0043] Figure 7A schematic diagram of the process for generating predicted bounding box coordinates and writing them into a FIFO for a segmented feature map according to an embodiment of the present invention;

[0044] Figure 8 This is a schematic diagram illustrating the conversion of predicted bounding box coordinates into a mask according to an embodiment of the present invention. Detailed Implementation

[0045] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Example

[0047] The decoupled structure of the detection head in a deep learning algorithm for object detection comprises two branches: classification and regression. The classification branch outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. For the COCO dataset, each feature map in the classification branch contains 80 channels, representing 80 categories. The regression branch outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. Each feature map in the regression branch contains four channels: top, bottom, left, and right distances.

[0048] Because FPGA hardware cannot generate 20×20, 40×40, or 80×80 feature maps at once, it needs to be processed in blocks. For example, it can generate 8×8 feature maps in one calculation. Therefore, a 20×20 feature map is decomposed into 3×3 8×8 feature maps. For blocks smaller than 8×8, zeros are padded to bring them to 8×8 size. A 40×40 feature map is decomposed into 5×5 8×8 feature maps. An 80×80 feature map is decomposed into 10×10 8×8 feature maps.

[0049] like Figure 1 As shown, the decoupled structure of the detection head has two branches: classification and regression. Taking a 20×20 feature map as an example, it is divided into 3×3 8×8 block feature maps. r8x8 and c8x8 are the row and column coordinates of the 8×8 block, both ranging from 0 to 2. The coordinates of the 3×3 8×8 block feature maps are (0,0), (0,1), (0,2), (1,0), (1,1), (1,2), (2,0), (2,1), and (2,2), respectively. Classification corresponds to 80 output channels (OC), and regression corresponds to 4 output channels (OC).

[0050] The YOLO algorithm takes a 640x640 image as input and ultimately labels the detected objects with predicted bounding boxes on the original image. This requires creating a 640x640 mask. A mask of "0" displays the original image, while a mask of "1" displays the predicted bounding box in its color (e.g., red). A 640x640 mask is shown below. Figure 2 As shown.

[0051] The storage of the 640×640 mask is implemented using a dual-port BRAM, such as... Figure 3 As shown. The data width is 640 bits, corresponding to 640 columns per row. The depth is 640, representing 640 rows. Port A of the BRAM is used to write the mask, and port B of the BRAM is used to read the mask.

[0052] Figure 4 and Figure 5 This invention illustrates a specific implementation of a real-time multi-prediction box display method accelerated by an FPGA-based target detection algorithm, comprising the following steps:

[0053] S1. Divide the regression branch feature map of the detection head into several block feature maps;

[0054] The 20×20 feature map of the regression branch contains 4 output channels OC. The 20×20 feature map is divided into 3×3 8×8 block feature maps, and the calculation order is (0,0), (0,1), (0,2), (1,0), (1,1), (1,2), (2,0), (2,1), (2,2).

[0055] S2. Each 8×8 block feature map is post-processed to generate N predicted bounding boxes bbox1 to bboxN, where the coordinate information of the nth predicted bounding box bboxN is [(x1... n ,y1 n ),(x2 n ,y2 n )], where (x1 n ,y1 n (x2) is the coordinate of the top left corner. n ,y2 n () is the coordinate of the lower right corner, 1≤n≤N, where N is a positive integer;

[0056] In this embodiment, it is assumed that (0,0) corresponds to an 8×8 block feature map, and a prediction box bbox1 is generated after post-processing, with coordinates [(1,1),(7,8)]; it is assumed that (0,1) corresponds to an 8×8 block feature map, and a prediction box bbox2 is generated after post-processing, with coordinates [(5,2),(11,6)]; it is assumed that no corresponding prediction boxes are generated for other 8×8 block feature maps.

[0057] S3. Write the predicted bounding boxes bbox1 and bbox2 into the FIFO; steps S2 and S3 are as follows Figure 7 As shown;

[0058] S4. Detect the empty signal of the FIFO. When the empty signal is "0", i.e., it is not empty, proceed to step S5.

[0059] S5. Analyze the coordinates of the predicted bounding box bbox1 [(1,1),(7,8)];

[0060] S6. Read the original first row mask from the 640×640 BRAM into the one-dimensional register group DFF640 as the initial value of the first row.

[0061] The mask update of the first row is completed from column 1 to column 7 on the one-dimensional register group DFF640, and the updated mask value of the first row of the one-dimensional register group DFF640 is written back to the first row of the 640×640 BRAM.

[0062] S7. Following the method in step S6, continue to update the mask of the remaining rows 2 to 8 in the prediction box bbox1 line by line, and write back the corresponding rows 1 to 8 of the 640×640 BRAM.

[0063] In the mask update in steps S6 and S7 above, for rows 1 and 8, the mask is updated column by column from column 1 to column 7. For rows 2 to 7, only column 1 and column 7 are updated.

[0064] After steps S6 and S7, the mask update of the predicted bounding box bbox1 is completed in the 640×640 BRAM;

[0065] S8. Then, following steps S5 to S7, update the mask of the predicted bounding box bbox2 and write it back to the corresponding position in the 640×640 BRAM; the specific steps for updating the mask of the predicted bounding box bbox2 are as follows:

[0066] Analyze the coordinates of the predicted bounding box bbox2 [(5,2),(11,6)];

[0067] The original second row mask is read from the 640×640 BRAM and stored in the one-dimensional register group DFF640 as the initial value of the second row; it contains the original mask information of bbox2.

[0068] The mask update for the second row is completed from column 5 to column 11 on the one-dimensional register group DFF640, and the updated mask value for the second row of the one-dimensional register group DFF640 is written back to the second row of the 640×640 BRAM.

[0069] Following the above method, continue to update the mask of rows 3 to 6 in the prediction box bbox2 line by line, and write back rows 2 to 6 of the 640×640 BRAM accordingly;

[0070] For rows 2 and 6, the mask is updated column by column from column 5 to column 11. For rows 3 to 5, only column 5 and column 11 are updated. At this point, the mask update of bbox2 is completed in the 640×640 BRAM.

[0071] Figure 8 This illustration shows the process of converting the coordinates of the predicted bounding boxes bbox1 and bbox2 into masks in this embodiment, as well as the mask diagrams of the two final predicted bounding boxes. Figure 8 The coordinates with a mask of "1" indicate the color of the prediction box.

[0072] S9. The video stream draws bounding boxes for the predicted bounding boxes bbox1 and bbox2 according to the updated masks in the above steps. If the mask is 0, the original image data is used; if the mask is 1, the original image data is replaced with the color of the predicted bounding box.

[0073] The specific implementation process of the picture frame is as follows: Figure 4 As shown: The video stream passes through AXI VDMA (Video Direct Memory Access), then through the frame-drawing logic. The frame-drawing logic analyzes the control and data signals of the video stream. The row address generator generates a row address for one row and reads a 640-bit mask from a 640×640 two-dimensional mask BRAM. The column address generator generates a column address for one column and reads one bit of the 640-bit mask. This bit is used as the control signal for the 2-to-1 selector MUX. The video stream and the pre-set prediction frame color data (e.g., red) are used as the two input signals of the 2-to-1 selector MUX. When the mask is 0, the MUX uses the original image data; when the mask is 1, the MUX uses the prediction frame color data, replacing the original image data with the prediction frame color (e.g., red). The video stream after the frame-drawing then passes through AXI4-Stream to Video Out to be displayed via HDMI (High Definition Multimedia Interface).

[0074] Figure 6 In this embodiment, the specific logic for mask updating is as follows:

[0075] In the idle state, the row address addr_y is 0, the column address addr_x is 0, the FIFO read enable fifo_rd_en is 0, and if the FIFO is not empty (empty == 0), it enters the bbox_loop state.

[0076] In the bbox_loop state, if the FIFO is not empty (i.e., empty == 0), it enters the fifo_wait state. The FIFO read enable fifo_rd_en is 1, and the mask completion mask_done is 0. If the FIFO is empty (i.e., empty == 1), the FIFO read enable fifo_rd_en is 0. At this time, if all bboxes are generated (i.e., bbox_done_flag == 1), the mask completion mask_done is 1, and it enters the idle state;

[0077] In the fifo_wait state, it enters the bram_wait state. The FIFO read enable fifo_rd_en is 0, the row address addr_y is y1, and the column address addr_x is x1;

[0078] In the bram_wait state, it enters the y_loop state. The original mask read enable old640_rd_en is 1;

[0079] In the y_loop state, the original mask read enable old640_rd_en is 0. If addr_y == y1 || addr_y == y2, it enters the top_down state. If addr_y < y2 && addr_y > y1, it enters the left state. Otherwise, it enters the mask_update state;

[0080] In the top_down state, the data din_x corresponding to addr_x is 1, the column write enable we_x is 1, and it enters the x_incr state;

[0081] In the x_incr state, the write enable we_x is 0. If addr_x >= x1 && addr_x < x2, the column address addr_x is incremented by 1. Otherwise, it enters the x_loop_end state;

[0082] In the left state, addr_x is x1, the data din_x is 1, the column write enable we_x is 1, and it enters the right state;

[0083] In the right state, addr_x is x2, the data din_x is 1, the column write enable we_x is 1, and it enters the x_loop_end state;

[0084] In the x_loop_end state, addr_x is x1, the data din_x is 0, the column write enable we_x is 0, the row write enable we_y is 1, and it enters the mask_update state;

[0085] In the mask_update state, the row write enable we_y is 0, the row address addr_y is incremented by 1, and it is checked that addr_y == y2. If it is true, it enters the y_loop_end state; if it is false, it enters the bram_wait state and updates the mask of the next row.

[0086] In the y_loop_end state, the number of processed bounding boxes is incremented by 1, and the system enters the bbox_loop state;

[0087] in:

[0088] `idle` represents the idle state, `bbox_loop` represents the bbox traversal state, `fifo_wait` represents the bbox FIFO read state, `bram_wait` represents the original mask read state, `y_loop` represents the loop state of row address `addr_y`, `top_down` represents the state of the top and bottom borders of the bbox, `x_incr` represents the state of the column address `addr_x`, `left` represents the state of the left border of the bbox, `right` represents the state of the right border of the bbox, `x_loop_end` represents the state of the end of column address `addr_x`, `mask_update` represents the mask update state, and `y_loop_end` represents the state of the end of row address `addr_y`.

Claims

1. A real-time display method for multiple prediction bounding boxes accelerated by an FPGA-based target detection algorithm, characterized in that, Includes the following steps: S1. Divide the regression branch feature map of the detection head into several block feature maps; S2. Each block feature map undergoes post-processing to generate N predicted bounding boxes bbox1 to bboxN, where the coordinate information of the nth predicted bounding box bboxN is [(x1... n ,y1 n ),(x2 n ,y2 n )], where (x1 n ,y1 n (x2) is the coordinate of the top left corner. n ,y2 n () is the coordinate of the lower right corner, 1≤n≤N, where N is a positive integer; S3. Write all prediction boxes into the FIFO; S4. Detect the empty signal of the FIFO. When the empty signal is "0", i.e., it is not empty, proceed to step S5. S5. Analyze the coordinates of the predicted bounding box bbox1 [(x1... 1 ,y1 1 ),(x2 1 ,y2 1 )]; S6. Read the original y1 from the 640×640 BRAM. 1 The row mask is passed into the one-dimensional register set DFF640 as y1. 1 The initial value of the row; From x1 on the one-dimensional register set DFF640 1 Starting from column x2 1 End of column y1 1 Update the row mask and update y1 in the one-dimensional register set DFF640. 1 The row mask value is written back to y1 of a 640×640 BRAM. 1 OK; S7. Following the method in step S6, complete the remaining y1 values ​​in the prediction box bbox1 line by line. 1 +1 to y2 1 Update the mask for all rows and write back the corresponding y1 in the 640×640 BRAM. 1 +1 to y2 1 OK; S8. Following the method of steps S5 to S7, update the mask of the remaining prediction boxes bbox2 to bboxN one by one, and write back the corresponding position of the 640×640 BRAM. S9. The video stream draws bounding boxes based on all the updated masks in step S8. If the mask is 0, the original image data is used; if the mask is 1, the original image data is replaced with the color of the predicted bounding box.

2. The real-time display method for multiple prediction boxes accelerated by FPGA-based target detection algorithm according to claim 1, characterized in that: The regression branch feature map of the detection head has three sizes: 20×20, 40×40, or 80×80.

3. The real-time display method for multiple prediction boxes accelerated by the FPGA-based target detection algorithm according to claim 2, characterized in that: The size of the block feature map is 8×8. The sizes of the 20×20, 40×40, and 80×80 feature maps after being divided into 8×8 blocks are 3×3, 5×5, and 10×10, respectively. If there are blocks smaller than 8×8 after the division, "0"s are added to fill the size to 8×8.

4. The real-time display method for multiple prediction boxes accelerated by the FPGA-based target detection algorithm according to claim 1, characterized in that: The mask update in steps S6 and S7, for y1 1 line and y2 1 Line, from x1 1 to x2 1 Update the mask column by column for all columns, for y1 1 and y2 1 The rows between them only contain x1 1 and x2 1 Update the mask in two columns.

5. The real-time display method for multiple prediction boxes accelerated by FPGA-based target detection algorithm according to claim 1, characterized in that, The specific logic for updating the mask is as follows: In the idle state, the row address addr_y is 0, the column address addr_x is 0, the FIFO read enable fifo_rd_en is 0, and if the FIFO is not empty (empty == 0), it enters the bbox_loop state. In the bbox_loop state, if the FIFO is not empty (empty == 0), it enters the fifo_wait state, the FIFO read enable fifo_rd_en is 1, and the mask completion mask_done is 0. If the FIFO is empty (empty == 1), the FIFO read enable fifo_rd_en is 0. At this time, if all bboxes have been generated (bbox_done_flag == 1), the mask completion mask_done is 1, and it enters the idle state. In the fifo_wait state, it enters the bram_wait state, the FIFO read enable fifo_rd_en is 0, the row address addr_y is y1, and the column address addr_x is x1; In the bram_wait state, enter the y_loop state, and set the original mask reading enable old640_rd_en to 1; In the y_loop state, the original mask read enable old640_rd_en is 0. If addr_y == y1 || addr_y == y2, it enters the top_down state.<y2&&addr_y> If y1, enter the left state; otherwise, enter the mask_update state. In the top_down state, the data din_x corresponding to addr_x is 1, the column write enable we_x is 1, and it enters the x_incr state; In the x_incr state, the write enable we_x is 0. If addr_x >= x1 && addr_x < x2, the column address addr_x is incremented by 1; otherwise, it enters the x_loop_end state. In the left state, addr_x is x1, the data din_x is 1, the column write enable we_x is 1, and it enters the right state. In the right state, addr_x is x2, the data din_x is 1, the column write enable we_x is 1, and it enters the x_loop_end state. In the x_loop_end state, addr_x is x1, the data din_x is 0, the column write enable we_x is 0, the row write enable we_y is 1, and it enters the mask_update state. In the mask_update state, the row write enable we_y is 0, the row address addr_y is incremented by 1, and it is judged whether addr_y == y2. If true, it enters the y_loop_end state; if false, it enters the bram_wait state to update the mask of the next row. In the y_loop_end state, the number of processed bboxes is incremented by 1, and it enters the bbox_loop state. Where: idle is the idle state, bbox_loop is the bbox traversal state, fifo_wait is the bbox FIFO read state, bram_wait is the original mask read state, y_loop is the row address addr_y loop state, top_down is the bbox upper and lower border state, x_incr is the column address addr_x increment state, left is the bbox left border state, right is the bbox right border state, x_loop_end is the column address addr_x termination state, mask_update is the mask update state, and y_loop_end is the row address addr_y termination state.

Images