Post-processing optimization technology for single-stage target recognition algorithm

Abstract

The invention discloses a detection head post-processing optimization technology based on a single-stage convolutional neural network target recognition system, and belongs to the field of image processing algorithm target recognition. The input of the post-processing optimization technology is the detection head output of the YOLO series convolutional neural network model, and the output content comprises target feature information and a to-be-decoded target frame. The post-processing optimization technology mainly comprises the following steps: averaging coordinate information, carrying out frame linear regression updating, reconstructing a detection vector, screening target position, category, confidence and other information, and finally realizing position labeling and tracking of a target in a digital image. The optimization technology is suitable for a scene with a single target and low time delay detection requirements, the problem of accelerated reasoning of a YOLO series convolutional neural network post-processing algorithm is solved, and the calculated amount of the whole system is reduced.

Description

Technical Field

[0001] The invention relates to the technical field of image processing algorithm target recognition, and in particular to optimizing and accelerating a post-processing detection head technology of a single-stage target recognition algorithm. Background Art

[0002] Image processing target detection technology mainly uses the framework of deep learning and runs on the hardware carrier of FPGA through a specific detection model. Taking advantage of its high parallel hardware, it is easy to handle a large number of convolution operations. Due to the different setting scales of the convolution kernel, the splicing methods between feature extraction modules are also different, which can help the entire neural network extract different feature information on various dimensions of the target to be identified. This can help the subsequent detection head locate the target to be identified more quickly.

[0003] In pursuit of higher detection performance, the complexity of the target detection network structure has become increasingly higher, and the number of weights involved in the network operation has increased exponentially. Therefore, when we need to port to a hardware platform, we need to optimize the algorithm flow according to specific needs and for specific scenarios. To simplify the processing flow, the post-processing optimization technology of the single-stage target recognition algorithm optimizes the processing flow of the subsequent detection head part under specific scenarios. This meets the requirements of high speed and low latency in engineering. Summary of the invention

[0004] First, the coordinate information averaging module receives the data output from the convolutional neural network detection head, then normalizes part of the data, and outputs the normalized data and the rest of the data output by the detection head to the bounding box linear regression update module.

[0005] Secondly, the border linear regression update module receives all the data output by the coordinate information averaging module, updates the border data therein according to the corresponding linear regression formula, and outputs the updated border data together with the remaining data to the detection vector construction module.

[0006] Again, the detection vector construction module receives the output data from the bounding box linear regression module, filters the category confidence in the data, constructs the detection vector, and outputs it to the filtering module.

[0007] Then, the filtering module receives the detection vectors output from the detection vector construction module, performs bubbling and descending order according to the confidence p0 and filters them, and outputs the first N groups of filtered detection vectors to the coordinate mapping module.

[0008] Finally, the coordinate mapping module receives the first N groups of detection vectors filtered by the filtering module, maps the coordinate information in the detection matrix to the scale (W, H) of the original image, and then merges them into the target information message output.

[0009] 1. The sub-process of the coordinate information averaging module includes:

[0010] First, the data output from the two detection heads of the convolutional neural network is received and downsampled by p times and q times respectively; then, the downsampled data is rearranged in the order of x, y, w, h, p0, class; and normalization operation is performed on x, y, p0, class. The normalization function can be the following two:

[0011]

[0012] Where x, y are the center coordinates of the box, w, h are the width and height of the box, p0 is the confidence of the box, class is the category confidence set, and k i For the elements in it.

[0013] Finally, the p-fold downsampling end outputs A-channel normalized data, and the q-fold downsampling end outputs B-channel normalized data.

[0014] 2. The sub-process of the border linear regression update module includes:

[0015] First, the A and B output normalized data from the coordinate information averaging module are received; then, the A data end pre-generates a grid of size R*S, and the B data end pre-generates a grid of size U*L, where the R, S, U, and L parameters are determined by the original image size and the downsampling multiple; then, according to the YOLO series rules, X groups of initial anchor boxes are generated at the center of the above grids, and the new bounding box (b x , b y , bx, b h ), where b x , b y represents the coordinates of the center point of the newly generated bounding box, b w , b h Represents the width and height of the newly generated bounding box; finally, the bounding box (b x , b y , b w , b h ) and p0, class are combined to output the target positioning information (b x , b y , b w , b h , p0, class).

[0016] 3. The sub-process of the detection vector construction module includes:

[0017] First, receive the target positioning information after linear regression from the bounding box linear regression update module (bx , b y , b w , b h , p0, class); then, for the element k in the category confidence class i Compare the sizes one by one, filter out the K larger targets, and retain their subscript i; then, x , b y , b w , b h , p0,i are concatenated into a new target detection vector; finally, the constructed target detection vector (b x , b y , b w , b h , p0,i).

[0018] 4. The sub-process of the coordinate mapping module includes:

[0019] First, receive the first N groups of target detection vectors (b x , b y , b w , b h , p0, i); then for b x , b y , b w , b h To carry out coordinate mapping, the specific mapping function is shown as follows:

[0020] x t =b x *W t =b w *W

[0021] y t =b y *Hh t =b h *H

[0022] Finally, output the target message x t ,y t , w t ,h t , p0,i, where x t ,y t Represents the target center point mapped to the original image (W, H) scale, w t ,h t represents the width and height of the target box mapped to the original image (W, H) scale, and p0,i represents the confidence and category information of the detected target.

[0023] Compared with the prior art, the present invention has the following significant advantages:

[0024] 1) This invention is different from the traditional deep learning target detection algorithm, and its focus is on real-time target detection. First, the image data is passed into the neural network model for calculation, and then passes through our single-stage target detection post-processing optimization module. The entire data processing process does not need to cache a large amount of intermediate redundant feature data. Therefore, a high-precision and low-latency target detection system can be realized.

[0025] 2) The present invention is based on the data flow model to perform processing operations, and all data are processed in a pipeline operation according to the flow chart. The advantage is that the logic complexity of the program is small, the number of algorithm loops is small, and it is easy to implement in engineering.

[0026] 3) The present invention is a post-processing optimization designed for the YOLO single-stage network model, which has the advantage of improving the recognition speed and accuracy in the single-target scenario, which can effectively improve the resource utilization of hardware and increase the robustness of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a block diagram of the post-processing optimization technology of the single-stage target recognition algorithm of the present invention.

[0028] Figure 2 This is the output architecture design diagram of the convolutional neural network detection head of the present invention

[0029] Figure 3 This is the design diagram of the frame linear regression update module of the present invention

[0030] Figure 4 Design diagram of the detection vector building module of the present invention

[0031] Figure 5 This is a schematic diagram of the filtering and screening module flow of the present invention DETAILED DESCRIPTION

[0032] In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively with reference to the accompanying drawings. The present invention can be implemented in different forms and is not limited to the embodiments described in the text. On the contrary, the embodiments are provided to make the content disclosed in the present invention more thorough and comprehensive. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0033] See also Figure 1 to Figure 5 This example provides a YOLOV4_tiny target detection optimization post-processing method. The feature of the present invention is that half-precision floating-point numbers are used to complete the entire post-processing process, and the processing speed is accelerated by reducing the number of loop levels of the algorithm, thereby improving the computing efficiency.

[0034] The image preprocessing module receives the color image I(W*H*3) transmitted by the camera configured by the FPGA chip, a total of 3 channels, and the valid enable signal of each channel. Then the processing chip will compress the received three-channel information according to the set image width W and height H. The compression algorithm can use the bilinear interpolation algorithm. After compression, it is I t (W t *H t *3), output the image to the YOLOV4_tiny convolutional neural network. The convolutional network algorithm module will receive the digital image data and valid enable signal from the image preprocessing compression, and perform convolution operations in the built YOLOV4_tiny model to obtain the output results of the two detection heads to be decoded and the valid signal, and output them to the averaged coordinate information module.

[0035] The mean coordinate information module first downsamples the two detection head data received by 32 (p) and 16 (q) times respectively; then, it performs σ (z) normalization processing on x, y, p0, class; finally, it outputs the normalized coordinate data and the enable signal to the border linear regression update module.

[0036] The bounding box linear regression update module will receive the normalized coordinate data and enable signal. First, refer to the two detection heads with 32 and 16 times downsampling to generate corresponding grids of 20*15 and 40*30 respectively; then, match the linear regression function in YOLOv4_tiny post-processing and the three groups of anchor boxes generated at the center of the grid; then, the t in the normalized coordinate data is updated. x , t y , t w , t h Perform linear regression to regress the anchor box to the bounding box; finally, the processed result b x , b y , b w , b h , p0, class data is fed into the detection vector construction module, and the specific linear regression function is as follows:

[0037]

[0038] After the detection vector construction module receives the data and valid from the bounding box signal update module, it first compares the box confidence p0 with the confidence k of each class. i Multiply to get p0*k i ; Next, select the maximum item of the result and output its result and category i, as described by the following formula:

[0039]

[0040] Finally, b x , b y , b w , b h , p0,i are reconnected into a new detection vector and output to the filtering module.

[0041] After the filtering module receives the data and enable signal from the detection vector construction module, it first performs bubble sorting on the constructed detection vectors according to the confidence value p0 of each detection grid; then, it outputs the data and enable signal of the first 9 (N) groups of detection construction vectors to the secondary filtering module in descending order; finally, after the secondary filtering, it outputs the detection construction vectors b of the two largest groups. x , b y , b w , b h , p0,i to coordinate mapping module.

[0042] After the coordinate mapping module receives the data of the two sets of detection construction vectors filtered by the filtering module and enables it, first, due to the above image preprocessing, the color image I transmitted by the receiving camera has a size of W*H*3; then, b in the detection construction vector is x , b y , b w , b h Multiply it with the original image I size (W, H), the mapping formula is as follows; finally, the detection box message data x mapped to the original image scale t ,y t , w t ,h t , p0, i are output in sequence.

[0043] x t =b x *W t =b w *W

[0044] y t =b y *Hh t =b h *H

Claims

1. A post-processing optimization technique for a single-stage target recognition algorithm, characterized in that: The post-processing optimization module receives the target feature information and the target frame to be decoded from the output of the convolutional neural network detection head, and inputs them into the coordinate information averaging module, the frame linear regression update module, the detection vector construction module, the filtering module and the coordinate mapping module, and finally outputs the target information message. First, the coordinate information averaging module receives the data output from the convolutional neural network detection head, then normalizes part of the data, and outputs the normalized data and the rest of the data output by the detection head to the bounding box linear regression update module. Secondly, the border linear regression update module receives all the data output by the coordinate information averaging module, updates the border data therein according to the corresponding linear regression formula, and outputs the updated border data together with the remaining data to the detection vector construction module. Again, the detection vector construction module receives the output data from the bounding box linear regression module, filters the category confidence in the data, constructs the detection vector, and outputs it to the filtering module. Then, the filtering module receives the detection vectors output from the detection vector construction module, performs bubbling and descending order according to the confidence p0 and filters them, and outputs the first N groups of filtered detection vectors to the coordinate mapping module. Finally, the coordinate mapping module receives the first N groups of detection vectors filtered by the filtering module, maps the coordinate information in the detection matrix to the scale (W, H) of the original image, and then merges them into the target information message output.

2. The post-processing optimization technique for a single-stage target recognition algorithm according to claim 1 is characterized in that: The processing flow of the coordinate information averaging module includes: First, the data output from the two detection heads of the convolutional neural network is received and downsampled by p times and q times respectively; then, the downsampled data is rearranged in the order of x, y, w, h, p0, class; and normalization operation is performed on x, y, p0, class. The normalization function can be the following two: Where x, y are the center coordinates of the box, w, h are the width and height of the box, p0 is the confidence of the box, class is the category confidence set, and k i For the elements in it. Finally, the p-fold downsampling end outputs A-channel normalized data, and the q-fold downsampling end outputs B-channel normalized data.

3. The post-processing optimization technique for a single-stage target recognition algorithm according to claim 1 is characterized in that: The specific algorithm process of the border linear regression update module includes: First, receive the normalized data of the A and B outputs from the coordinate information averaging module; then, the A data end pre-generates a grid of size R*S, and the B data end pre-generates a grid of size U*L, where the R, S, U, and L parameters are determined by the original image size and the downsampling multiple; then, according to the YOLO series rules, generate X groups of initial anchor boxes at the center of the above grids, and use the initial anchor boxes and regression function to calculate the new bounding box (b x , b y , b w , b h ), where b x , b y represents the coordinates of the center point of the newly generated bounding box, b w , b h Represents the width and height of the newly generated bounding box; finally, the bounding box (b x , b y , b w , b h ) and p0, class are combined to output the target positioning information (b x , b y , b w , b h , p0,class).

4. The post-processing optimization technique for a single-stage target recognition algorithm according to claim 1 is characterized in that: The specific algorithm flow of the detection vector construction module includes: First, receive the target positioning information after linear regression from the bounding box linear regression update module (b x , b y , b w , b h , p0, class); then, for the element k in the category confidence class i Compare the sizes one by one, filter out the K larger targets, and retain their subscript i; then, x , b y , b w , b h , p0,i are concatenated into a new target detection vector; finally, the constructed target detection vector (b x , b y , b w , b h , p0,i).

5. The post-processing optimization technique for a single-stage target recognition algorithm according to claim 1 is characterized in that: The specific algorithm flow of the coordinate mapping module includes: First, receive the first N groups of target detection vectors (b x , b y , b w , b h , p0, i); then for b x , b y , b w , b h To carry out coordinate mapping, the specific mapping function is shown as follows: x t =b x *W w t =b w *W y t =b y *H h t =b h *H Finally, output the target message x t ,y t , w t ,h t , p0,i, where x t ,y t Represents the target center point mapped to the original image (W, H) scale, w t ,h t represents the width and height of the target box mapped to the original image (W, H) scale, and p0,i represents the confidence and category information of the detected target.

Images