An image FHOG feature extraction device

Abstract

The application discloses an image FHOG feature extraction device and belongs to the technical field of image processing. (a,b) The histogram cache module comprises K bank cache units, which are denoted as KBank (m,n) The histogram feature vector of Cell(i,j) is cached; m=i%c; n=j%c; the cache mode is convenient for accumulating the contribution values of the same pixel generated in the same period to each associated Cell into the histogram feature vector of the corresponding associated Cell; all pixels only need to be processed once, data reusability is greatly improved, and the FHOG feature of the image can be extracted at a faster operation speed under the premise of meeting the accuracy requirement.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and more specifically, relates to an image FHOG feature extraction device. Background Technology

[0002] FHOG features are an optimization based on HOG features. On one hand, they distinguish between histogram intervals with opposite gradient directions, performing direction-sensitive and direction-insensitive pixel-level feature mapping on the original image. On the other hand, they employ a "softbinning" method, where each pixel uses bilinear interpolation to contribute to the gradient histograms of multiple cells surrounding that pixel. FHOG features are insensitive to illumination and have a certain degree of deformation resistance, making them widely used in target tracking algorithms such as KCF and DSST.

[0003] In embedded systems, deploying target tracking algorithms such as KCF and DSST places high demands on their real-time performance. However, embedded systems are often constrained by area, power consumption, and resources. When performing FHOG feature extraction, the complexity of FHOG feature calculations, high feature dimensionality, and numerous multiplication and division operations result in significant time overhead, impacting the overall real-time performance of the target tracking algorithm. To address this issue, an existing solution uses fixed-point computation for FHOG feature extraction. While this significantly reduces computational load, hardware resource consumption, and power consumption, it leads to a loss of precision in the feature results compared to floating-point computation. Therefore, designing a high-precision device for frame-by-frame image extraction in resource-constrained, low-power embedded systems is a pressing problem. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the prior art, the present invention provides an image FHOG feature extraction device to solve the technical problem that the prior art cannot extract the FHOG features of the image at a fast computing speed while meeting the accuracy requirements.

[0005] To achieve the above objectives, the present invention provides an image FHOG feature extraction device, comprising: a gradient histogram statistics module, a histogram caching module, and a conversion module;

[0006] The gradient histogram statistics module is used to count the histogram feature vector of each Cell in the image to be processed; a Cell is a pixel block after non-overlapping division of the image to be processed, with a total of C1×C2, denoted as Cell(i,j), i=0,1,2...C1-1, j=0,1,2...C2-1;

[0007] The histogram caching module includes: K bank cache units, denoted as KBank. (a,b) , a=0,1,2...c-1, b=0,1,2...c-1, K=c×c; KBank (m,n) The histogram feature vector is used to cache Cell(i,j); m = i%c; n = j%c; K is the maximum number of associated Cells; associated Cells are the Cells that pixel A can contribute to;

[0008] The conversion module is used to convert the histogram feature vectors of all cells into FHOG features;

[0009] The gradient histogram statistics module includes:

[0010] The gradient calculation unit is used to calculate the gradient vector [dx,dy] at any pixel A in the image to be processed;

[0011] The bin calculation unit is used to calculate the angle range corresponding to the gradient vector [dx,dy] and obtain the corresponding bin number idx; bin is the angle range after dividing the interval [0,360°) into equal angles.

[0012] The associated amplitude calculation unit is used to calculate the contribution value of pixel A to each associated cell in parallel.

[0013] The read / write control unit is used to read the histogram feature vectors of each associated cell from the corresponding bank cache unit in parallel, perform a summation operation in parallel to obtain the new histogram feature vector of the associated cell, and write it back to the corresponding bank cache unit. The summation operation is to add the contribution value of pixel point A to the cell to the idx-th dimension of the histogram feature vector of the associated cell.

[0014] More preferably, the bin calculation unit is used to perform parallel dot product between the vector [|dx|,|dy|] and the unit vector of each bin in the interval [0,90°) to obtain the bin corresponding to the maximum dot product value, and then obtain the angle range corresponding to the gradient vector [dx,dy] based on the direction of the gradient vector [dx,dy], and then obtain the corresponding bin number idx;

[0015] Here, the unit vector of bin is the unit vector in the middle angular direction of the corresponding angular range.

[0016] More preferably, the correlation magnitude calculation unit is used to obtain the correlation cell of pixel A and the corresponding contribution coefficient, and calculate the product of the gradient magnitude of pixel A and different contribution coefficients in parallel to obtain the contribution value of pixel A to each correlation cell.

[0017] More preferably, the gradient histogram statistics module further includes: a pixel caching unit, used to cache the pixel rows of the image to be processed, with the number of pixels cached each time being 2W+1;

[0018] The pixel buffer unit includes registers R1, R2, R3, R4, R5, R6, and R7, as well as a first buffer and a second buffer. Registers R1, R2, R3, R4, R5, R6, and R7 are each used to buffer one pixel. The first buffer is used to buffer W-3 pixels, and the second buffer is used to buffer W-3 pixels. W is the width of the image to be processed. The pixels in the image to be processed are input into the pixel buffer unit sequentially from the top left to the bottom right in a zigzag scanning direction, and are buffered in a first-in, first-out manner. After passing through R1, R2, the first buffer, R3, R4, R5, the second buffer, R6, and R7, the final output is obtained.

[0019] The gradient calculation unit is used to determine the position of a pixel in the image to be processed when there is a pixel cached in register R4. If the pixel is located at the top left, bottom right, bottom left, or top right corner of the image to be processed, its gradient dx in the x-direction is set to 0, and its gradient dy in the y-direction is set to 0. If the pixel is located on the left or right boundary of the image to be processed, excluding the top left, bottom right, bottom left, and top right corners, its gradient dx in the x-direction is set to 0. Otherwise, the pixel values ​​in registers R3 and R5 are read, and the difference between the pixel values ​​in R3 and R5 is calculated to obtain its gradient dx in the x-direction. If the pixel is located on the top or bottom boundary of the image to be processed, excluding the top left, bottom right, bottom left, and top right corners, its gradient dy in the y-direction is set to 0. Otherwise, the pixel values ​​in registers R1 and R7 are read, and the difference between the pixel values ​​in R1 and R7 is calculated to obtain its gradient dy in the y-direction.

[0020] More preferably, the histogram caching module includes port aport and port bport;

[0021] The transformation module is used to normalize, truncate, and perform principal component analysis on the histogram feature vectors of all cells, including: a first executor and a second executor that perform the operations in parallel;

[0022] The first actuator is used to calculate the four normalization coefficients for each cell;

[0023] The second actuator is used to obtain the corresponding four normalization coefficients from the first actuator for each cell, and to perform normalization, truncation and principal component analysis on the cell's histogram feature vector to obtain the corresponding intermediate histogram feature vector; the intermediate histogram feature vectors of all cells together constitute the FHOG feature.

[0024] More preferably, the first executor includes: an aport read request unit, a bin square sum accumulation unit, a bin square sum line cache, and an inverse square root calculation unit;

[0025] The aport read request unit is used to sequentially read the histogram feature vector of each cell from the aport port into the bin square sum accumulation unit; where the dimension of the histogram feature vector of each cell is B-dimensional, and its l-th dimension data is denoted as bin(l), l=0,1,…,B-1; B is the number of angle ranges after dividing the interval [0,360°) into equal angles;

[0026] The bin square sum accumulation unit is used for each received bin(q) and Then, bin(q) and Summing and then squaring yields the final sum of squares.

[0027] The bin sum of squares row buffer includes registers r1, r2, r3, r4, r5, r6, r7, r8, r9, as well as a third buffer and a fourth buffer. Registers r1, r2, r3, r4, r5, r6, r7, r8, and r9 are used to buffer the sum of squares of one cell. The third buffer and the fourth buffer are used to buffer the sum of squares of C2-3 cells. The sum of squares of the cells in the image to be processed is input into the bin sum of squares row buffer in a zigzag scanning direction from the top left to the bottom right. The buffer is buffered in a first-in, first-out manner, passing through r1, r2, r3, the third buffer, r4, r5, r6, the fourth buffer, r7, r8, and r9 in sequence, and finally output.

[0028] The inverse square root calculation unit is used when registers r1, r2, r3, r4, r5, r6, r7, r8, and r9 are all full and the Cell in r5 is not located at the boundary. It reads the values ​​in registers r1, r2, r3, r4, r5, r6, r7, r8, and r9 respectively, and performs inverse square root calculation on the sum of the values ​​in registers r5, r6, r8, and r9, the sum of the values ​​in registers r4, r5, r7, and r8, the sum of the values ​​in registers r2, r3, r5, and r6, and the sum of the values ​​in registers r1, r2, r4, and r5 respectively, to obtain the four normalized coefficients α1, α2, α3, and α4 of the Cell in register r5.

[0029] More preferably, the reading order when the aport read request unit reads the histogram feature vector of the Cell from port aport is as follows:

[0030] More preferably, the second actuator is used to sequentially read the histogram feature vector of each cell from port bport, obtain the corresponding normalized coefficients α1, α2, α3, and α4 from the first actuator, calculate α1bin(l), α2bin(l), α3bin(l), and α4bin(l), and then perform truncation operations to obtain bin1(l), bin2(l), bin3(l), and bin4(l), and then calculate... calculate Then, truncation operations are performed to obtain Bin1(l), Bin2(l), Bin3(l), and Bin4(l), and then the results are calculated. calculate Together they form the intermediate histogram feature vector of the cell.

[0031] More preferably, the reading order when the second actuator reads the histogram feature vector of the Cell from port bport is as follows:

[0032] More preferably, the conversion module further includes a filtering unit for filtering the FHOG features to obtain the final FHOG features.

[0033] More preferably, the above-mentioned image FHOG feature extraction device further includes: a configuration interface module, a data DMA interface module, a dual-port shared buffer, and an image scaling module;

[0034] The configuration interface module is used to receive the configuration information required for FHOG feature extraction from the original image;

[0035] The data DMA interface module is used to load raw image data into the dual-port shared buffer according to the configuration information; it is also used to output the FHOG feature data in the dual-port shared buffer to the address space specified in the configuration information.

[0036] The dual-port shared buffer is used to buffer raw image data loaded from the data DMA interface module when used as an input buffer, and to buffer generated FHOG features when used as an output buffer.

[0037] The image scaling module is used to scale the original image to obtain the image to be processed.

[0038] More preferably, the FHOG feature size generated by the above conversion module is [size missing]. Noted as H*W*C out ;

[0039] The dual-port shared cache includes M bank cache units;

[0040] The above-mentioned image FHOG feature extraction device further includes: a feature output module, which is used to merge the first dimension and the second dimension of the FHOG feature, and the feature data of the x-th row and cout-th column of the merged feature is F(x,cout), and F(x,cout) is cached at the addr address of the m-th bank cache unit of the dual-port shared cache;

[0041] Where x = 0, 1, 2…H*W-1, cout = 0, 1, 2…,C out -1; when 0 ≤ cout < B, m = (x) % M; when When, m = (x+1)%M; when At that time, m = (x + 3) % M;

[0042] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0043] 1. This invention provides an image FHOG feature extraction device, whose histogram caching module includes K bank caching units, denoted as KBank. (a,b) , a=0,1,2...c-1, b=0,1,2...c-1, K=c×c; KBank (m,n) Used to cache the histogram feature vector of Cell(i,j); m = i%c; n = j%c; This caching method facilitates the gradient histogram statistics module to accumulate the contribution values ​​of the same pixel generated in the same period to each associated Cell into the histogram feature vector of the corresponding associated Cell. All pixels only need to be processed once, which greatly improves data reusability and can extract the FHOG features of the image at a faster computing speed while meeting the accuracy requirements.

[0044] 2. Furthermore, in the image FHOG feature extraction device provided by the present invention, when the bin calculation unit obtains the corresponding bin number idx, it only needs to perform parallel dot product of the vector [|dx|,|dy|] with the unit vector of each bin in the interval [0,90°), without having to perform parallel dot product with the unit vector of all bins in 360°. This reduces hardware resource overhead on the one hand, and further reduces the amount of computation and computation delay on the other hand.

[0045] 3. Furthermore, the image FHOG feature extraction device provided by the present invention constructs a cross-shaped window through five registers R1, R3, R4, R5, and R7 in its pixel cache unit. The gradient vector of the pixel in the middle R4 register is obtained based on the four pixel values ​​in the upper register R7, lower register R1, left register R5, and right register R3 of the cross-shaped window. This fully utilizes the locality of the image, eliminates the need to cache the entire image, and eliminates the need to repeatedly load pixel values. On the one hand, this reduces the overhead of storage hardware resources, and on the other hand, it can further improve processing efficiency.

[0046] 4. Furthermore, the image FHOG feature extraction device provided by the present invention includes a conversion module for normalizing, truncating, and performing principal component analysis on the histogram feature vectors of all cells. This module includes: a first executor and a second executor that perform operations in parallel; the first and second executors respectively read the histogram feature vectors of the cells from port aport and port bport in parallel; the normalization coefficients are executed in parallel with normalization, truncating, and principal component analysis, further improving processing efficiency.

[0047] 5. Further, in the image FHOG feature extraction device provided by the present invention, the reading order of the aport read request unit when reading the histogram feature vector of the cell from the port aport is bin(0). bin(1), bin(2), bin(B-1) can further improve the parallelism of feature data output and further improve processing efficiency.

[0048] 6. Further, in the image FHOG feature extraction device provided by the present invention, the second actuator reads the histogram feature vector of the cell from port bport in the order bin(0). bin(1), bin(2), bin(B-1) can further improve the parallelism of feature data output and further improve processing efficiency.

[0049] 7. Further, the image FHOG feature extraction device provided by the present invention further includes: a feature output module, which caches F(x,cout) at the addr address of the m-th bank cache unit of the dual-port shared cache; wherein x = 0, 1, 2…H*W-1, cout = 0, 1, 2…,C out -1; when 0 ≤ cout < B, m = (x) % M; when When, m = (x+1)%M; when At that time, m = (x + 3) % M; The data DMA interface module can read M feature data from the Mbank dual-port shared cache in the same cycle and output them in a channel-by-channel, row-by-row, column-by-column order, which increases the output bandwidth utilization.

[0050] 8. Furthermore, the image FHOG feature extraction device provided by the present invention uses a dual-port shared cache as both an input cache and an output cache. The time-division multiplexing of the dual-port shared cache reduces the overhead of storage hardware resources. Attached Figure Description

[0051] Figure 1 This is an overall block diagram of an image FHOG feature extraction device provided in an embodiment of the present invention;

[0052] Figure 2 This is a diagram illustrating the process of obtaining histogram feature vectors using the gradient histogram statistics module provided in this embodiment of the invention.

[0053] Figure 3 This is a schematic diagram illustrating the order of pixel gradient histograms in the Kbank dual-port histogram cache provided in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the conversion module structure provided in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram illustrating the working process of the feature output module used in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0057] To achieve the above objectives, the present invention provides an image FHOG feature extraction device, comprising: a gradient histogram statistics module, a histogram caching module, and a conversion module;

[0058] The gradient histogram statistics module is used to count the histogram feature vector of each cell in the image to be processed. A cell is a pixel block after the image to be processed is divided into non-overlapping blocks. Each cell is P*P in size, and there are C1×C2 cells, denoted as Cell(i,j), i=0,1,2...C1-1, j=0,1,2...C2-1; C1=Row / P, C2=Col / P, where Row and Col are the number of rows and columns of the image to be processed, respectively.

[0059] The histogram caching module includes: K bank cache units, denoted as KBank. (a,b) , a=0,1,2...c-1, b=0,1,2...c-1, K=c×c; KBank (m,n) The histogram feature vector is used to cache Cell(i,j); m = i%c; n = j%c; K is the maximum number of associated Cells; associated Cells are the Cells that pixel A can contribute to;

[0060] The conversion module is used to convert the histogram feature vectors of all cells into FHOG features;

[0061] The gradient histogram statistics module includes:

[0062] The gradient calculation unit is used to calculate the gradient vector [dx,dy] at any pixel A in the image to be processed;

[0063] The bin calculation unit is used to calculate the angle range corresponding to the gradient vector [dx,dy] and obtain the corresponding bin number idx; bin is the angle range after dividing the interval [0,360°) into equal angles; specifically, the bin numbering method is as follows: the interval [0,360°] is divided into equal angles in advance to obtain B angle ranges, each angle range corresponds to a bin, and a bin number is assigned accordingly;

[0064] The associated magnitude calculation unit is used to calculate the contribution value of pixel A to each associated cell in parallel. Specifically, it obtains the associated cells of pixel A and the corresponding contribution coefficients, and calculates the product of the gradient magnitude of pixel A and different contribution coefficients in parallel to obtain the contribution value of pixel A to each associated cell.

[0065] The read / write control unit is used to read the histogram feature vectors of each associated cell from the corresponding bank cache unit in parallel, perform a summation operation in parallel to obtain the new histogram feature vector of the associated cell, and write it back to the corresponding bank cache unit; wherein, the summation operation is to add the gradient magnitude G to the contribution value of the cell to the idx-th dimension of the histogram feature vector of the associated cell.

[0066] Regarding the determination of the bin number, in one optional implementation, the bin calculation unit performs a parallel dot product between the vector [|dx|,|dy|] and the unit vectors of each bin within the interval [0, 90°), obtaining the bin corresponding to the maximum dot product value. Then, based on the direction of the gradient vector [dx,dy], the angle range corresponding to the gradient vector [dx,dy] is obtained, thus yielding the corresponding bin number idx. Here, the unit vector of the bin is the unit vector in the middle angle direction of the corresponding angle range. It should be noted that the bin calculation unit obtains the bin number idx in more than one way. For example, in another optional implementation, in a second implementation, the gradient vector [dx,dy] can be directly subjected to a parallel dot product with the unit vectors of each bin within the interval [0, 360°), obtaining the bin corresponding to the maximum dot product value, thus obtaining the angle range corresponding to the gradient vector [dx,dy], and finally, the corresponding bin number idx. In another optional implementation, in a third implementation, the angle corresponding to the gradient vector [dx,dy] can be directly calculated to obtain the corresponding angle range. However, compared to the method in Optional Implementation 2 that performs parallel dot product of gradient vector [dx,dy] with unit vectors of each bin in the interval [0,360°) and the method in Optional Implementation 3 that directly calculates the angle, Optional Implementation 1 only requires performing parallel dot product of vector [|dx|,|dy|] with unit vectors of each bin in the interval [0,90°). This reduces hardware resource overhead and further reduces computational latency.

[0067] In one optional implementation, the gradient histogram statistics module further includes a pixel caching unit for caching pixels of the image to be processed. Preferably, the pixel caching unit is used for caching rows of pixels of the image to be processed, with the number of pixels cached each time being 2W+1;

[0068] The pixel buffer unit includes registers R1, R2, R3, R4, R5, R6, and R7, as well as a first buffer and a second buffer. Registers R1, R2, R3, R4, R5, R6, and R7 are each used to buffer one pixel. The first buffer is used to buffer W-3 pixels, and the second buffer is used to buffer W-3 pixels. W is the width of the image to be processed. The pixels in the image to be processed are input into the pixel buffer unit sequentially from the top left to the bottom right in a zigzag scanning direction, and are buffered in a first-in, first-out manner. After passing through R1, R2, the first buffer, R3, R4, R5, the second buffer, R6, and R7, the final output is obtained.

[0069] The gradient calculation unit is used to determine the position of a pixel in the image to be processed when there is a pixel cached in register R4. If the pixel is located at the top left, bottom right, bottom left, or top right corner of the image to be processed, its gradient dx in the x-direction is set to 0, and its gradient dy in the y-direction is set to 0. If the pixel is located on the left or right boundary of the image to be processed, excluding the top left, bottom right, bottom left, and top right corners, its gradient dx in the x-direction is set to 0. Otherwise, the pixel values ​​in registers R3 and R5 are read, and the difference between the pixel values ​​in R3 and R5 is calculated to obtain its gradient dx in the x-direction. If the pixel is located on the top or bottom boundary of the image to be processed, excluding the top left, bottom right, bottom left, and top right corners, its gradient dy in the y-direction is set to 0. Otherwise, the pixel values ​​in registers R1 and R7 are read, and the difference between the pixel values ​​in R1 and R7 is calculated to obtain its gradient dy in the y-direction.

[0070] The design of the pixel cache unit described above allows for caching only 2W+1 pixels at a time, greatly reducing the requirements for the cache unit and minimizing storage hardware resource overhead.

[0071] It should be noted that the above is only one way to calculate the gradient in detail. It mainly implements the gradient calculation method based on the Prewitt operator. The gradient calculation methods based on operators such as Sobel and Laplace are also applicable. These operators only have different coefficients. You only need to multiply the corresponding coefficients at the corresponding positions. They will not be elaborated here.

[0072] In one optional implementation, the histogram caching module is a dual-ended histogram caching module, including port aport and port bport;

[0073] The transformation module is used to normalize, truncate, and perform principal component analysis on the histogram feature vectors of all cells, including: a first executor and a second executor that perform the operations in parallel;

[0074] The first actuator is used to calculate the four normalization coefficients for each cell;

[0075] The second actuator is used to obtain the corresponding four normalization coefficients from the first actuator for each cell, and to perform normalization, truncation and principal component analysis on the cell's histogram feature vector to obtain the corresponding intermediate histogram feature vector; the intermediate histogram feature vectors of all cells together constitute the FHOG feature.

[0076] It should be noted that parallel execution is just one implementation method; sequential execution is also possible, and this is not a limitation. The above design allows for parallel processing of the normalization coefficient calculation and subsequent normalization, truncation, and principal component analysis, further improving computational speed and efficiency.

[0077] In one alternative implementation, the first executor includes: an aport read request unit, a bin square sum accumulation unit, a bin square sum line cache, and an inverse square root calculation unit;

[0078] The aport read request unit is used to sequentially read the histogram feature vector of each cell from the aport port into the bin square sum accumulation unit; where the dimension of the histogram feature vector of each cell is B-dimensional, and its l-th dimension data is denoted as bin(l), l=0,1,…,B-1; B is the number of angle ranges after dividing the interval [0,360°) into equal angles;

[0079] The bin square sum accumulation unit is used for each received bin(q) and Then, bin(q) and Summing and then squaring yields the final sum of squares.

[0080] The bin sum of squares row buffer includes registers r1, r2, r3, r4, r5, r6, r7, r8, r9, as well as a third buffer and a fourth buffer. Registers r1, r2, r3, r4, r5, r6, r7, r8, and r9 are used to buffer the sum of squares of one cell. The third buffer and the fourth buffer are used to buffer the sum of squares of C2-3 cells. The sum of squares of the cells in the image to be processed is input into the bin sum of squares row buffer in a zigzag scanning direction from the top left to the bottom right. The buffer is buffered in a first-in, first-out manner, passing through r1, r2, r3, the third buffer, r4, r5, r6, the fourth buffer, r7, r8, and r9 in sequence, and finally output.

[0081] The inverse square root calculation unit is used when registers r1, r2, r3, r4, r5, r6, r7, r8, and r9 are all full and the Cell in r5 is not located at the boundary. It reads the values ​​in registers r1, r2, r3, r4, r5, r6, r7, r8, and r9 respectively, and performs inverse square root calculation on the sum of the values ​​in registers r5, r6, r8, and r9, the sum of the values ​​in registers r4, r5, r7, and r8, the sum of the values ​​in registers r2, r3, r5, and r6, and the sum of the values ​​in registers r1, r2, r4, and r5 respectively, to obtain the four normalized coefficients α1, α2, α3, and α4 of the Cell in register r5.

[0082] It should be noted that when the aport read request unit reads the histogram feature vector of the Cell from the aport port, it can read sequentially or according to... The reading order is as follows. Preferably, the reading order when the aport read request unit reads the histogram feature vector of the Cell from port aport is as follows: This allows the bin square sum accumulation unit to continuously receive bin(q) and... This can further improve the parallelism of feature data output and further improve processing efficiency.

[0083] In one optional implementation, the second actuator sequentially reads the histogram feature vector of each cell from port bport, obtains the corresponding normalized coefficients α1, α2, α3, and α4 from the first actuator, calculates α1bin(l), α2bin(l), α3bin(l), and α4bin(l), and then performs truncation operations to obtain bin1(l), bin2(l), bin3(l), and bin4(l), and then calculates... calculate Then, truncation operations are performed to obtain Bin1(l), Bin2(l), Bin3(l), and Bin4(l), and then the results are calculated. calculate Together they form the intermediate histogram feature vector of the cell.

[0084] It should be noted that in the above operations, normalization and principal component analysis were directly combined. Principal component analysis mainly involves summation and multiplication by the corresponding coefficients. and By following the steps above, the final result can be calculated in one go, further reducing computational complexity.

[0085] It should be noted that when the bport read request unit reads the histogram feature vector of the Cell from the port bport, it can read sequentially or according to... The reading order is as follows. Preferably, the reading order when the second actuator reads the histogram feature vector of the Cell from port bport is as follows: This allows for the continuous acquisition of bin(q) and... This can further improve the parallelism of feature data output and further improve processing efficiency.

[0086] In one optional implementation, the conversion module further includes a filtering unit for filtering the FHOG features to obtain the final FHOG features. It should be noted that the filtering unit can be a Hanning window filtering unit, a Gaussian window filtering unit, etc. A Hanning window filtering unit is preferred.

[0087] In one optional implementation, the above-mentioned image FHOG feature extraction device further includes: a data DMA interface module, a configuration interface module, a dual-port shared buffer, and an image scaling module;

[0088] The configuration interface module is used to receive the configuration information required for FHOG feature extraction from the original image;

[0089] The data DMA interface module is used to load raw image data into the dual-port shared buffer according to the configuration information; it is also used to output the FHOG feature data in the dual-port shared buffer to the address space specified in the configuration information.

[0090] The dual-port shared buffer is used to buffer raw image data loaded from the data DMA interface module when used as an input buffer, and to buffer generated FHOG features when used as an output buffer.

[0091] The image scaling module is used to scale the original image to obtain the image to be processed.

[0092] In one optional implementation, the FHOG feature size generated by the above conversion module is: Noted as H*W*C out The feature data of its h-th row, w-th column, and cout-th channel is FF(h,w,cout), where h=0,1,2…,H-1, w=0,1,2…,W, and cout=0,1,2…C out The feature data of the x-th row and cout-th column after merging the first and second dimensions of the FHOG feature is denoted as F(x,cout), where x = 0, 1, 2…H*W-1, i.e. F(h*W+w,cout) = FF(h,w,cout).

[0093] The dual-port shared cache includes M bank cache units;

[0094] The above-mentioned image FHOG feature extraction device further includes: a feature output module, which is used to merge the first dimension and the second dimension of the FHOG feature, and the feature data of the x-th row and cout-th column of the merged feature is F(x,cout), and F(x,cout) is cached at the addr address of the m-th bank cache unit of the dual-port shared cache;

[0095] Where x = 0, 1, 2…H*W-1, cout = 0, 1, 2…,C out -1; when 0 ≤ cout < B, m = (x) % M; when When, m = (x+1)%M; when At that time, m = (x + 3) % M;

[0096] This arrangement enables the data DMA interface module to read M feature data from the Mbank dual-port shared cache in the same cycle and output them in a channel-by-channel, row-by-row, column-by-column order, further increasing the output bandwidth utilization.

[0097] To further illustrate the image FHOG feature extraction device provided by the present invention, a detailed description is given below with reference to an embodiment:

[0098] like Figure 1 The diagram shown is an overall block diagram of an image FHOG feature extraction device proposed in this embodiment, comprising:

[0099] The configuration and data DMA interface module receives configuration information, loads image data, and outputs feature data. It includes a configuration interface module and a data DMA interface module. In this embodiment, the configuration interface module is the AXI4-Lite-Slave configuration interface module, and the dual-port shared cache is the Mbank dual-port shared cache.

[0100] First, the configuration interface module of AXI4-Lite-Slave receives the configuration information required for this FHOG feature extraction operation, including the address information of the target image data to be loaded, the parameter information required by the image scaling module, the address space information of the final feature data output, and the start signal. As shown by the dotted line, after the configuration information is received, it is updated to the data DMA interface module and the image scaling module. Then, image data loading is started, and the image data is written to the Mbank dual-port shared buffer. The number of loaded image data rows is updated to the image scaling module in real time. After the entire frame of image data is loaded, a read completion interrupt signal is generated. After the subsequent feature data is generated, feature data output is started. Each clock cycle, M feature data results are read from the Mbank dual-port shared buffer. The data length is concatenated to the output interface bit width and then output to the address space specified in the configuration information. After the entire frame of feature data is output, a write completion interrupt signal is generated.

[0101] The M-bank dual-port shared cache adopts a time-sharing multiplexing mode. After image data loading starts, the dual-port shared cache acts as an input cache, storing the image data loaded from the data interface into M bank caches in a specific order. After FHOG feature data output starts, the dual-port shared cache acts as an output cache, storing the generated FHOG feature data into M bank caches in a specific order. The number of banks M depends on the requirements of the image scaling and interpolation algorithm.

[0102] The image scaling module receives configuration information from the configuration interface module and the number of rows of loaded image data updated in real time from the data DMA interface module. Based on the configuration information, this module calculates the number of image rows required for the current scaling operation. When enough image rows are loaded into the Mbank dual-port shared buffer, the scaling calculation is initiated. Each clock cycle, M image data are read from the Mbank dual-port shared buffer, completing the image scaling result output in parallel with the image data loading. In this embodiment, the final image size is scaled to N*N.

[0103] The gradient histogram statistics module is used to calculate the gradient vector of each pixel and perform block gradient histogram statistics on the gradient vector. The gradient of each pixel contributes to the K cells in the neighborhood. The results are temporarily stored in the Kbank dual-port histogram cache. This module is fully pipelining.

[0104] The Kbank dual-port histogram caching module is used to cache pixel gradient histogram statistics. Based on the contribution of each pixel gradient to the gradients of K different cells in the neighborhood, the gradient histogram results are cached in the corresponding addresses of the K banks.

[0105] The transformation module includes a first executor and a second executor that perform operations in parallel for normalization and principal component analysis (PCA) processing. After all pixels have completed gradient histogram statistics, this module begins operation, processing data in two parallel streams. One stream reads the histogram statistics from the aport of the K-bank dual-port histogram cache, row by row and column by column, to calculate the normalization coefficients. The other stream reads the histogram statistics of the B bins in the corresponding cell (i,j) from the bport of the 4-bank dual-port histogram cache based on the row and column coordinates of the middle cell (i,j) in the normalization window, and performs normalization, truncation, and PCA processing. In addition, the transformation module includes a Hanning window filtering unit to filter the feature data after PCA to obtain the final FHOG features.

[0106] The feature output module writes FHOG features into the Mbank dual-port shared cache in a specific order.

[0107] like Figure 2The diagram illustrates the process of obtaining histogram feature vectors using the gradient histogram statistics module provided in this embodiment. The pixel row caching module performs pixel row caching on the scaled input image data. A cross-shaped window is constructed using five registers: R1, R3, R4, R5, and R7. The gradient vector of the pixel in the middle register R4 is calculated based on the four pixel values ​​from the upper register R7, lower register R1, left register R5, and right register R3 within the cross-shaped window. The gradient vector is simultaneously fed into the bin value calculation unit and the associated amplitude calculation unit. One side performs bin value calculation, and the other performs K-path amplitude calculations in parallel. The bin value calculation unit maps the direction of each pixel gradient vector to the first quadrant. The mapped pixel gradient is then multiplied in parallel with the unit vectors of the L bins in the first quadrant. Based on the bin number with the largest dot product value and the quadrant to which the pixel gradient belonged before mapping, the true histogram bin number to which the pixel gradient vector belongs is obtained. This method significantly reduces the computational workload of determining the bin number. The correlation magnitude calculation unit multiplies the pixel gradient magnitude by K weighted values ​​and calculates the different contributions of the pixel gradient vector to the histograms of K cells in the neighborhood in parallel. The bin number, K magnitudes, and the pixel coordinates are sent to the read / write control unit. The read / write control unit calculates K cache addresses based on these three conditions, reads the corresponding K stored values ​​from the aport port of the Kbank dual-port cache, adds the results of the K magnitude calculations, and then writes them back to the original address from the bport port, completing the accumulation of gradient histogram statistics. If the histogram statistics are accumulated at the same address in two consecutive cycles, bypass processing is required.

[0108] Figure 3 The diagram shows the order of the pixel gradient histograms in the K-bank dual-port histogram cache provided in this embodiment. The K banks are denoted as KBank. (a,b) Where a = 0, 1, 2, ..., c-1, b = 0, 1, 2, ..., c-1, and K = c × c. The number of cells is... Will Let's denote it as C, where C1 = C2 = C; therefore, K banks cache a total of C*C histograms, and each histogram has B bins. The histogram results of Cell(i,j) will be statistically analyzed in KBank. (m,n) Where m = i%c, n = j%c, i = 0, 1, 2...C-1, j = 0, 1, 2...C-1. For KBank (0,0) Addresses 0 to B-1 store the pixel gradient histogram statistics of B bins for Cell (0,0), addresses B to 2B-1 store the statistics of Cell (0,c), and so on. This caching method facilitates the parallel accumulation of K magnitude values ​​of the same pixel gradient generated in the same period into the corresponding K cells in the neighborhood, improving data reusability.

[0109] Figure 4 The diagram shown is a schematic of the conversion module structure provided in this embodiment. The conversion module is fully pipelined. After all pixels have completed gradient histogram statistics, the conversion module starts running, performing parallel processing on two paths. One path starts from aport, and the aport read request unit reads the histogram statistics results from the aport port of the Kbank dual-port histogram cache in the order of cell row by row and column by column. The reading order is bin(0). bin(1), bin(2), bin(B-1), each cell reads B values ​​and sends the results to the cell's internal bin sum of squares accumulation module. The cell's internal bin sum of squares accumulation module will then read bin(0) and bin(B-1). Summing and then squaring, then adding bin(1) and... The square of the sum, bin(2) and The square of the sum of The square of the sum of bin(B-1) is... The sum of squares of the groups. The sum of squares is stored in the bin sum of squares row cache inside the Cell, caching the bin sum of squares of 2 Cells, and a normalization window is constructed using 9 registers: R1, R2, R3, R4, R5, R6, R7, R8, and R9. Once the 9 registers of the normalization window are filled with data, the 9 values ​​in the normalization window are sent to the inverse square root calculation unit to obtain the normalization coefficient of the Cell located at register R5. Starting from bport, based on the row and column coordinates of Cell (i,j) at position R5 of the normalization window, the bport read request module reads the B bin histogram statistics of the corresponding Cell (i,j) from port b of the 4bank dual-port histogram cache, with the reading order being bin(0). bin(1), bin(2), bin(B-1), in conjunction with the normalization coefficients generated by another path, is used for channel... Normalization, truncation, and principal component analysis processing improve the parallelism of feature data output by changing the reading order.

[0110] Figure 5The diagram shown illustrates the working process of the feature output module provided in this embodiment. The feature data generated after principal component analysis first enters the Hanning window filtering unit for filtering. The Hanning window coefficients are symmetrical about the center of the image, and a quarter block of Hanning window coefficients is stored in the ROM. The window coefficients are read from the ROM in real time according to the cell position (i,j) of the input feature data, multiplied, and then output. The final filtered feature map consists of a dimension of [missing information]. Noted as H*W*C out The filtered data enters the feature output module, where one, two, or three feature data points generated at different periods are rearranged and written to the Mbank dual-port shared cache. The width and height dimensions of the feature map are merged row-by-row and column-by-column, so each feature data point can be represented as F(x,cout), where x = 0, 1, 2…H*W-1, and cout = 0, 1, 2…C out -1. Mbank's code is MBank (m) Where m = 0, 1, 2...M-1. For feature data F(x, cout), it is cached in MBank. (m) In (addr), m is the bank number and addr is the address in the bank. When 0 ≤ cout < B, m = (x) % M; when When, m = (x+1)%M; when At that time, m = (x + 3)%M. This arrangement enables the subsequent data DMA interface submodule to read M feature data from the Mbank dual-port shared buffer in the same cycle and output them in a channel-by-channel, row-by-row, column-by-column order, increasing the output bandwidth utilization.

[0111] In summary, this invention enables image FHOG feature extraction with low resource consumption, high accuracy, and low latency.

[0112] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image FHOG feature extraction apparatus characterized by comprising: The application relates to a gradient histogram statistics module for calculating histogram feature vectors of each cell in a to-be-processed image, a conversion module for converting the histogram feature vectors of all the cells into FHOG features, and a read-write control unit for reading the histogram feature vectors of the associated cells from corresponding bank buffer units, performing a sum operation on the histogram feature vectors, obtaining new histogram feature vectors of the associated cells, and writing the new histogram feature vectors back into the corresponding bank buffer units. The gradient histogram statistics module comprises a gradient calculation unit for calculating a gradient vector [dx, dy] of any pixel point A in the to-be-processed image, a bin calculation unit for performing parallel dot products of the vector [|dx|, |dy|] and unit vectors of all bins in the interval [0, 90°) to obtain a bin corresponding to the maximum dot product value, and then obtaining an angle range corresponding to the gradient vector [dx, dy] based on the direction of the gradient vector [dx, dy], and then obtaining a bin number idx corresponding to the angle range, wherein the bins are angle ranges obtained by equally dividing the interval [0, 360°), and the unit vectors of the bins are unit vectors of the middle angle directions of the corresponding angle ranges, and a correlation amplitude calculation unit for calculating contribution values of the pixel point A to the associated cells in parallel, obtaining the associated cells of the pixel point A and corresponding contribution coefficients, and calculating the product of the gradient amplitude of the pixel point A and different contribution coefficients in parallel to obtain the contribution values of the pixel point A to the associated cells. Cell is the pixel block after non-overlapping division of the image to be processed, a total of , denoted as Cell( i , j ), , ; The histogram cache module comprises K bank cache units, denoted as , , , ; The histogram feature vector for caching Cell( i , j ); ; ; K is the maximum number of associated Cells; the associated Cells are Cells to which the pixel point A can contribute. The read-write control unit is used for reading the histogram feature vectors of the associated cells from the corresponding bank buffer units in parallel, performing a sum operation in parallel to obtain new histogram feature vectors of the associated cells, and writing the new histogram feature vectors back into the corresponding bank buffer units; the sum operation is to add the idx-dimensional data of the histogram feature vectors of the associated cells to the contribution values of the pixel point A to the cells. The conversion module is used for normalizing, truncating and principal component analyzing the histogram feature vectors of all the cells, and comprises a first executor and a second executor for performing operations in parallel. The first executor is used for calculating four normalization coefficients of each cell. The second executor is used for obtaining the corresponding four normalization coefficients from the first executor, normalizing, truncating and principal component analyzing the histogram feature vectors of each cell to obtain corresponding intermediate histogram feature vectors; and the intermediate histogram feature vectors of all the cells jointly form the FHOG features. The gradient histogram statistics module further comprises a pixel buffer unit for buffering pixel rows of the to-be-processed image, and the number of pixels buffered at a time is 2W+1. ​ ​ ​ ​ 2. The image FHOG feature extraction apparatus according to claim 1, characterized by, ​ The pixel buffer unit comprises registers R1, R2, R3, R4, R5, R6, R7, and a first buffer and a second buffer; the registers R1, R2, R3, R4, R5, R6, R7 are respectively used for buffering one pixel, the first buffer is used for buffering W-3 pixels, and the second buffer is used for buffering W-3 pixels; W is the width of the image to be processed; the pixels in the image to be processed are sequentially input into the pixel buffer unit from the top left to the bottom right in a zigzag scanning direction and are buffered in a first-in first-out manner, sequentially pass through R1, R2, the first buffer, R3, R4, R5, the second buffer, R6 and R7, and are finally output; The gradient calculation unit is used for judging the position of the pixel in the register R4 in the image to be processed when the pixel is buffered in the register R4, and if the pixel is located at the top left corner, the bottom right corner, the top left corner and the bottom right corner of the image to be processed, the gradient dx of the x direction of the pixel is set to 0, and the gradient dy of the y direction of the pixel is set to 0; if the pixel is located on the left and right boundaries of the image to be processed except the top left corner, the bottom right corner, the top left corner and the bottom right corner, the gradient dx of the x direction of the pixel is set to 0; otherwise, the pixel values in the registers R3 and R5 are read, the difference between the pixel values in R3 and R5 is calculated, and the gradient dx of the x direction of the pixel is obtained; if the pixel is located on the upper and lower boundaries of the image to be processed except the top left corner, the bottom right corner, the top left corner and the bottom right corner, the gradient dy of the y direction of the pixel is set to 0; otherwise, the pixel values in the registers R1 and R7 are read, the difference between the pixel values in R1 and R7 is calculated, and the gradient dy of the y direction of the pixel is obtained.

3. The image FHOG feature extraction apparatus according to claim 1 or 2, characterized by, The histogram buffer module comprises a port aport and a port bport; The first executor comprises an aport read request unit, a bin square sum accumulation unit, a bin square sum row buffer and an inverse square root calculation unit; The aport read request unit is configured to sequentially read the histogram feature vector of each Cell from the port aport into the bin square sum accumulation unit; wherein the dimension of the histogram feature vector of each Cell is B dimensions, and the B dimensions are respectively corresponding to the B angle ranges after the interval [0, 360°) is equally divided. l The B dimensions are respectively corresponding to the B angle ranges after the interval [0, 360°) is equally divided. l ), ; B The B is the number of angle ranges after the interval [0, 360°) is equally divided. The bin square and accumulation unit is used to square and accumulate the bins after each group of bins is received q ) and bin( q + ) are squared after being summed q ) and bin( q + ) are squared after being summed ; The bin square sum line buffer comprises registers r1, r2, r3, r4, r5, r6, r7, r8, r9, and a third buffer and a fourth buffer; the registers r1, r2, r3, r4, r5, r6, r7, r8, r9 are respectively used for buffering the square sum of one Cell; the third buffer and the fourth buffer are both used for buffering the square sum result of one Cell The square sum result of a Cell in a to-be-processed image is input into the bin square sum line buffer in a zigzag scanning direction from top left to bottom right in a first-in first-out manner, sequentially passes through r1, r2, r3, the third buffer, r4, r5, r6, the fourth buffer, r7, r8, r9, and is finally output. The inverse square root calculation unit is configured to read the values in the registers r1, r2, r3, r4, r5, r6, r7, r8, r9 when the registers r1, r2, r3, r4, r5, r6, r7, r8, r9 are all filled and the Cell in r5 is not at the boundary, and perform inverse square root calculation on the sum of the values in the registers r5, r6, r8, r9, the sum of the values in the registers r4, r5, r7, r8, the sum of the values in the registers r2, r3, r5, r6, and the sum of the values in the registers r1, r2, r4, r5, respectively, to obtain four normalization coefficients of the Cell in the register r5 .

4. The image FHOG feature extraction apparatus according to claim 3, characterized by, The reading order of the aport reading request unit when reading the histogram feature vector of the Cell from the port aport is bin(0), bin( ), bin(1), bin( ), bin(2), bin( ), … bin( ), bin( B -1).

5. The image FHOG feature extraction apparatus according to claim 3, characterized by, The second executor is used for sequentially reading the histogram feature vector of each Cell from the port bport, and obtaining the corresponding normalization coefficient from the first executor , calculates 、 、 、 , and then respectively performs a truncation operation to obtain 、 、 、 , and further calculates to obtain ; calculates 、 、 、 , respectively performs a truncation operation to obtain 、 、 、 , and further calculates to obtain ; calculates 、 、 、 , and jointly constitutes the intermediate histogram feature vector of the Cell.

6. The image FHOG feature extraction apparatus according to claim 5, characterized by, The reading order of the second executor when reading the histogram feature vector of the Cell from the port bport is bin(0), bin( ), bin(1), bin( ), bin(2), bin( ), … bin( ), bin( ).

7. The image FHOG feature extraction apparatus according to claim 1, characterized by, Further comprising: a configuration interface module, a data DMA interface module, a dual-port shared buffer and an image scaling module; The configuration interface module is used for receiving configuration information required for FHOG feature extraction of the original image; The data DMA interface module is used for loading original image data to the dual-port shared buffer according to the configuration information, and is further used for outputting FHOG feature data in the dual-port shared buffer to an address space specified by the configuration information; The dual-port shared buffer is used for buffering the original image data loaded from the data DMA interface module when serving as an input buffer, and is used for buffering the generated FHOG feature when serving as an output buffer; The image scaling module is used for scaling the original image to obtain the image to be processed.

8. The image FHOG feature extraction apparatus according to claim 7, characterized by, The conversion module generates FHOG features with a size of , denoted as ; The dual-port shared buffer comprises M bank buffer units; The image FHOG feature extraction device further comprises a feature output module configured to merge the first dimension and the second dimension of the FHOG feature, and the merged feature has a third dimension. x row cout column x feature data F( cout , x , cout ), and the F( m , addr ) is cached to the address of the first bank cache unit of the dual-port shared cache. wherein x = 0, 1, 2… H * W -1, cout = 0, 1, 2… ; when , ; when , ; when , ; .

Images