ORB algorithm acceleration system and method based on FPGA

Abstract

The invention discloses an ORB algorithm acceleration system and method based on an FPGA, and the system comprises an image feature caching module which is used for storing an input original image in a BRAM storage unit according to an image pixel column form; the FAST feature extraction module is used for performing preliminary feature point extraction on the original image based on a dual-channel parallel processing circuit to obtain feature points and angular points; the Gaussian pyramid module is used for performing down-sampling and Gaussian smoothing filtering processing on the original image to obtain a smoothed image; the rotation angle calculation module is used for carrying out gray centroid calculation according to the coordinate values of the feature points and the smoothed image to obtain the rotation angles of the feature points; and the rBRIEF descriptor module is used for calculating and generating an rBRIEF descriptor according to the rotation angle of the feature point and the smoothed image to obtain ORB feature point data. The hardware resource utilization rate and the calculation speed are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hardware image feature extraction algorithms, and in particular to an ORB algorithm acceleration system and method based on FPGA. Background Art

[0002] At present, visual simultaneous localization and mapping (SLAM) technology plays an important role in robot navigation, autonomous driving, augmented reality and other fields. The core of SLAM algorithm is to capture environmental information through visual sensors, realize accurate positioning of the device itself and real-time three-dimensional modeling of the environment.

[0003] Among the many SLAM algorithms, the feature point-based method has attracted widespread attention due to its outstanding performance in positioning accuracy and system stability. The quality of feature point extraction is directly related to the performance of the SLAM system in positioning accuracy and composition quality. Common feature extraction algorithms in computer vision include SIFT (Scale-Invariant Feature Transform), FAST (Features from Accelerated Segment Test) and ORB (Oriented FAST and Rotated BRIEF) algorithms. Among them, the ORB feature detection algorithm combines the detection and extraction of FAST corner points with the generation and matching process of BRIEF descriptors, providing an efficient and robust feature extraction and matching method.

[0004] ORB is suitable for feature point-based sparse SLAM algorithms due to its algorithm robustness and simplicity of calculation process. However, executing the traditional ORB feature extraction algorithm on FPGA (Field-Programmable Gate Array) will result in long idle waiting time and large occupation of memory resources.

[0005] Although some researchers have done a small amount of work to study the above problems, the existing methods still have problems such as low hardware resource utilization, low computing efficiency and computing speed. Summary of the invention

[0006] The technical problem to be solved by the present invention is that the existing methods still have the problems of low hardware resource utilization, low operation efficiency and low calculation speed. The purpose of the present invention is to provide an ORB algorithm acceleration system and method based on FPGA, which includes an image feature cache module, a FAST feature extraction module, a Gaussian pyramid module, a rotation angle calculation module and an rBRIEF descriptor module. Through the coordinated work of the above modules based on FPGA, the ORB algorithm can extract ORB feature points efficiently and in real time on the FPGA, thereby improving the hardware resource utilization, the operation efficiency and the calculation speed.

[0007] The present invention is achieved through the following technical solutions:

[0008] In a first aspect, the present invention provides an ORB algorithm acceleration system based on FPGA, the system comprising:

[0009] The image feature cache module is used to store the input original image in the BRAM storage unit in the form of image pixel columns; during the storage process, data alignment and shift register mode are used to realize data update;

[0010] The FAST feature extraction module is used to perform preliminary feature point extraction on the original image in the image feature cache module based on a dual-channel parallel processing circuit to obtain feature points and corner points;

[0011] The Gaussian pyramid module is used to downsample and perform Gaussian smoothing filtering on the original image in the image feature cache module to obtain a smoothed image, thereby improving the scale invariance of the image;

[0012] The rotation angle calculation module is used to calculate the grayscale centroid according to the coordinate value of the feature point and the smoothed image to obtain the rotation angle of the feature point;

[0013] The rBRIEF descriptor module is used to calculate and generate the rBRIEF descriptor according to the rotation angle of the feature point and the smoothed image to obtain the ORB feature point data. During the calculation process, the two points of a pair of pixels are fixed at the same port of their respective storage units. This constraint is used to reduce the complexity of the interconnection structure, so that the grayscale comparison operation only needs to obtain the operand from each port of each storage unit, thereby avoiding access conflicts between pixel pairs in the same row.

[0014] Furthermore, the image feature cache module uses data alignment and shift register mode to implement data update, including:

[0015] Store data corresponding to the 0th column pixel of the original image on the 0th address line of the BRAM storage unit;

[0016] When each new pixel arrives, the existing data is first read from the BRAM address of the corresponding column, the highest-order pixel is removed from the existing data, and the newly arrived pixel is then appended to the lowest-order pixel to obtain the new pixel data; and the new pixel data is then written back to the same memory address to implement data update;

[0017] When accessing new pixels from multiple rows of input simultaneously, the BRAM is arranged into a shift register mode to construct a multi-row row cache to implement multi-row data update.

[0018] Furthermore, the FAST feature extraction module includes:

[0019] A dual-channel parallel processing unit is used to connect 16 subtractors in parallel in each channel to calculate the grayscale difference between the pixels in the original image, perform a difference operation on the calculated grayscale difference and a preset grayscale threshold T to obtain an operation result; compare the operation result with a zero value, and output a judgment result in a binary manner; if the judgment result is greater than zero, the pixel is marked as a bright spot / dark spot, and a high level 1 is output; otherwise, the pixel is marked as a non-bright spot / non-dark spot, and a low level 0 is output;

[0020] The feature point forming unit is used to form a 16-bit binary output string as a feature point based on the output result of each channel in the dual-channel parallel processing unit and the discrimination results of 16 outputs;

[0021] The corner point judgment unit is used to compare all possible pattern character strings that are pre-defined and stored with the 16-bit binary output string to obtain a comparison result; based on the comparison result, the 16-bit binary output string is processed by a logic AND operation to judge whether it is a bright spot or a dark spot; and the detection results of the bright spot and the dark spot are combined by a logic OR operation to determine whether the pixel point constitutes a corner point.

[0022] Furthermore, the calculation of the grayscale difference includes:

[0023] A Bresenham circle with a radius of 3 is used to select 16 key sampling points. In each clock cycle, a grayscale value comparison operation is performed between the 7×7 central pixel point and the 16 sampling points on the Bresenham circle with a radius of 3 around it to obtain the grayscale difference.

[0024] Furthermore, the Gaussian pyramid module includes:

[0025] The Gaussian filter processing unit is used to perform Gaussian algorithm processing, and first performs head-to-tail symmetrical addition processing and then performs product operation on the column data of the image matrix according to the order of input pixel points to obtain the product result; finally, the filtering result, that is, the smoothed image, is obtained by accumulating the product results.

[0026] Furthermore, the Gaussian pyramid module also includes a dynamic configuration mechanism: dynamically adjusting the depth of the line buffer according to the image size requirements of different levels, specifically including:

[0027] On the basis of determining the number of columns of the original input image, the number of columns of each level of the image in the pyramid structure is calculated by scaling it by 1.2 times;

[0028] The maximum depth of the line buffer corresponds to the scaled image size. Based on this, a step-by-step tapping method is adopted. Through the dynamic configuration mechanism of the multiplexer, the depth of the line buffer required for the current layer image of the pyramid is dynamically adjusted to ensure that it is compatible with the current feature map size.

[0029] Furthermore, during the calculation process, the rBRIEF descriptor module also uses a static scheduling calculation method based on genetic algorithm optimization to schedule access to the pixel pairs in the same row to avoid access conflicts between the pixel pairs in the same row.

[0030] Static scheduling calculation method based on genetic algorithm optimization, including:

[0031] The static mode scheduling problem is represented as a set of pixel pairs P: P = {P1, P2, ..., P N}, where G is a subset of the set P: gsize is a fixed parameter, which indicates the number of concurrent accesses in each group G.

[0032] The maximum number of pixel pairs required; P i represents the i-th element in the set P, that is, the i-th pixel pair; N represents the total number of elements in the set P, that is, the number of all pixel pairs;

[0033] Construct an objective function. The objective function is to minimize the average delay of the computing unit processing pixel pairs for all possible rotation angles under the static model scheduling strategy.

[0034] Based on the constraints, the genetic algorithm is used to solve the objective function and obtain the corresponding solution; the corresponding solution is a tuple<P,G> Represented by, and composed of groups corresponding to each element in the P set.

[0035] Furthermore, the constraints include:

[0036] The first constraint: ensure that any pixel pair does not appear in multiple combinations;

[0037] The second constraint: Each combination must be fully allocated with a fixed size of pixel pairs;

[0038] The third constraint condition: the pixel points need to be bound to the ports that access the bank.

[0039] In a second aspect, the present invention further provides an FPGA-based ORB algorithm acceleration method, which is based on the above-mentioned FPGA-based ORB algorithm acceleration system; the method comprises:

[0040] The original image acquired in real time is stored in the BRAM storage unit in the form of image pixel columns; during the storage process, data alignment and shift register mode are used to realize data update;

[0041] Based on the dual-channel parallel processing circuit, preliminary feature point extraction is performed on the original image in the image feature cache module to obtain feature points and corner points;

[0042] The original image in the image feature cache module is downsampled and subjected to Gaussian smoothing filtering to obtain a smoothed image, thereby improving the scale invariance of the image;

[0043] According to the coordinate value of the feature point and the smoothed image, the grayscale centroid is calculated to obtain the rotation angle of the feature point;

[0044] According to the rotation angle of the feature points and the smoothed image, the N most representative feature points are selected from the M feature points through the non-maximum suppression strategy, and the rBRIEF descriptor is calculated and generated to obtain the ORB feature point data.

[0045] Furthermore, data update is implemented by using data alignment and shift register mode, and the specific steps are as follows:

[0046] Store data corresponding to the 0th column pixel of the original image on the 0th address line of the BRAM storage unit;

[0047] When each new pixel arrives, the existing data is first read from the BRAM address of the corresponding column, the highest-order pixel is removed from the existing data, and the newly arrived pixel is then appended to the lowest-order pixel to obtain the new pixel data; and the new pixel data is then written back to the same memory address to implement data update;

[0048] When accessing new pixels from multiple rows of input simultaneously, the BRAM is arranged into a shift register mode to construct a multi-row row cache to implement multi-row data update.

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] 1. The present invention provides an ORB algorithm acceleration system and method based on FPGA, the system comprising an image feature cache module, a FAST feature extraction module, a Gaussian pyramid module, a rotation angle calculation module and an rBRIEF descriptor module. Through the collaborative work of the above modules based on FPGA, the ORB algorithm can efficiently and real-time extract ORB feature points on the FPGA, thereby improving the utilization rate of hardware resources, the operation efficiency and the calculation speed during calculation;

[0051] 2. The present invention provides an ORB algorithm acceleration system and method based on FPGA. The image feature cache module uses data alignment and shift register mode to implement data update of the input image, which can ensure accurate update and access of pixel data;

[0052] 3. The present invention provides an ORB algorithm acceleration system and method based on FPGA. In the FAST feature extraction module, based on a dual-channel parallel processing circuit, preliminary feature point extraction is performed on the original image in the image feature cache module to achieve a preliminary feature extraction function for the input image.

[0053] 4. The present invention provides an ORB algorithm acceleration system and method based on FPGA. A variable-depth row buffer is proposed in the Gaussian pyramid module to adapt the size of different layers of images in the Gaussian pyramid module, thereby improving the utilization rate of hardware resources.

[0054] 5. The present invention discloses an ORB algorithm acceleration system and method based on FPGA. In order to resolve the access conflict between pairs of pixels in the same row in the rBRIEF descriptor module, the present invention first stipulates that the two points of a pair of pixels are fixed at the same port of their respective storage units, and uses this constraint to reduce the complexity of the interconnection structure, so that the grayscale comparison operation only needs to obtain operands from each port of each storage unit; in order to further reduce the frequency of conflicts, a static scheduling calculation method based on genetic algorithm optimization is proposed to perform static scheduling optimization on the rBRIEF descriptor calculation part in the ORB algorithm, thereby improving the operating efficiency of the hardware calculation of this part, thereby improving the speed of executing the traditional ORB feature extraction algorithm on the FPGA. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0056] Figure 1 This is a schematic diagram of the structure of an ORB algorithm acceleration system based on FPGA of the present invention;

[0057] Figure 2 It is a schematic diagram of the hardware design of the FAST feature point initial extraction of the FAST feature extraction module of the present invention;

[0058] Figure 3 A schematic diagram of the Gaussian filter hardware design structure of the Gaussian pyramid module of the present invention;

[0059] Figure 4 A schematic diagram of a pyramid row buffer mechanism in which the depth of a Gaussian pyramid module of the present invention can be dynamically configured;

[0060] Figure 5 A schematic diagram of the rBRIEF circuit architecture of the present invention that allows parallel access to multiple "point pairs";

[0061] Figure 6 This is a flow chart of an ORB algorithm acceleration method based on FPGA of the present invention;

[0062] Figure 7 This is a detailed flow chart of an ORB algorithm acceleration method based on FPGA of the present invention. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0064] The present invention designs an ORB algorithm acceleration system and method based on FPGA. The system comprises an image feature cache module, a FAST feature extraction module, a Gaussian pyramid module, a rotation angle calculation module and an rBRIEF descriptor module. Through the coordinated work of the above modules based on FPGA, the ORB algorithm can extract ORB feature points efficiently and in real time on the FPGA, thereby improving the utilization rate of hardware resources, the operation efficiency during calculation and the calculation speed.

[0065] Specifically, (1) Image feature cache module: During the storage process, data alignment and shift register mode are used to implement data update of the input image, which can ensure accurate update and access of pixel data;

[0066] (2) FAST feature extraction module: Based on a dual-channel parallel processing circuit, it performs preliminary feature point extraction on the original image in the image feature cache module, thereby realizing the function of preliminary feature extraction on the input image;

[0067] (3) Gaussian pyramid module: A variable-depth line buffer is proposed to adapt the size of different layers of the Gaussian pyramid module, thereby improving the utilization of hardware resources.

[0068] (4) Rotation angle calculation module: estimates the direction of the feature points in the image, that is, calculates the grayscale centroid based on the coordinate values ​​of the feature points and the smoothed image to obtain the rotation angle of the feature points;

[0069] (5) rBRIEF descriptor module: In order to resolve the access conflict between pairs of pixels in the same row, the present invention stipulates that the two points of a pair of pixels are fixed at the same port of their respective storage units. This constraint is used to reduce the complexity of the interconnection structure, so that the grayscale comparison operation only needs to obtain operands from each port of each storage unit. In order to further reduce the frequency of conflicts, a static scheduling calculation method based on genetic algorithm optimization is proposed to perform static scheduling optimization on the rBRIEF descriptor calculation part in the ORB algorithm, thereby improving the operating efficiency of the hardware calculation of this part, thereby improving the speed of executing the traditional ORB feature extraction algorithm on the FPGA.

[0070] Example 1

[0071] like Figure 1 As shown, the present invention provides an ORB algorithm acceleration system based on FPGA, the system comprising:

[0072] The image feature cache module is used to store the input original image in the BRAM storage unit in the form of image pixel columns; during the storage process, data alignment and shift register mode are used to implement data update, which can ensure accurate update and access of pixel data;

[0073] The FAST feature extraction module is used to perform preliminary feature point extraction on the original image in the image feature cache module based on a dual-channel parallel processing circuit to obtain feature points and corner points;

[0074] The Gaussian pyramid module is used to downsample and perform Gaussian smoothing filtering on the original image in the image feature cache module to obtain a smoothed image, thereby improving the scale invariance of the image;

[0075] The rotation angle calculation module is used to calculate the grayscale centroid according to the coordinate value of the feature point and the smoothed image to obtain the rotation angle of the feature point;

[0076] The rBRIEF descriptor module is used to calculate and generate rBRIEF descriptors according to the rotation angle of feature points and the smoothed image to obtain ORB feature point data.

[0077] As a further implementation, the image feature cache module uses data alignment and shift register mode to implement data update, including:

[0078] Store data corresponding to the 0th column pixel of the original image on the 0th address line of the BRAM storage unit;

[0079] When each new pixel arrives, the existing data is first read from the BRAM address of the corresponding column, the highest-order pixel is removed from the existing data, and the newly arrived pixel is then appended to the lowest-order pixel to obtain the new pixel data; and the new pixel data is then written back to the same memory address to implement data update;

[0080] When accessing new pixels from multiple rows of input simultaneously, the BRAM is arranged into a shift register mode to construct a multi-row row cache to implement multi-row data update.

[0081] As a further implementation, Figure 2 As shown, Figure 2 This is a schematic diagram of the hardware design of the FAST feature point initial extraction of the FAST feature extraction module; the FAST feature extraction module includes:

[0082] A dual-channel parallel processing unit is used to identify bright spots and dark spots through dual channels respectively; specifically, 16 subtractors are connected in parallel in each channel to calculate the grayscale difference between pixels in the original image, and the calculated grayscale difference is subjected to a difference operation with a preset grayscale threshold T to obtain an operation result; the operation result is compared with a zero value, and the discrimination result is output in a binary manner; if the discrimination result is greater than zero, the pixel is marked as a bright spot / dark spot, and a high level 1 is output; otherwise, the pixel is marked as a non-bright spot / non-dark spot, and a low level 0 is output; wherein the grayscale difference calculation includes: using a Bresenham circle with a radius of 3 to select 16 key sampling points, and performing a grayscale value comparison operation between a 7×7 central pixel and 16 sampling points on a Bresenham circle with a radius of 3 around it in each clock cycle to obtain a grayscale difference;

[0083] The feature point forming unit is used to form a 16-bit binary output string as a feature point based on the output result of each channel in the dual-channel parallel processing unit and the discrimination results of 16 outputs;

[0084] The corner point judgment unit is used to compare all possible pattern character strings that are pre-defined and stored with the 16-bit binary output string to obtain a comparison result; based on the comparison result, the 16-bit binary output string is processed by a logic AND operation to judge whether it is a bright spot or a dark spot; and the detection results of the bright spot and the dark spot are combined by a logic OR operation to determine whether the pixel point constitutes a corner point.

[0085] As a further implementation, the Gaussian Pyramid module includes:

[0086] The Gaussian filter processing unit is used to convert the multiplication operation into a shift and accumulation operation when executing the Gaussian algorithm to optimize the multiplication operation. Considering that the symmetry characteristics of the Gaussian filter algorithm determine the efficiency of its calculation process, when executing the Gaussian algorithm, the buffer data of the input image should be arranged in a certain order before the operation is performed. Specifically, the column data of the image matrix is ​​first symmetrically added to the end according to the order of the input pixel points, and then the product operation is performed to obtain the product result; finally, the filtering result, that is, the smoothed image, is obtained by accumulating the product results, such as Figure 3 As shown, Figure 3 Schematic diagram of the Gaussian filter hardware design structure;

[0087] In order to reduce the circuit delay of parallel accumulation of multiple data in a single clock cycle and avoid the critical path being too long, the overall 25 groups of operations are decomposed into two groups (groups 1 to 12 and groups 13 to 25), and logical addition operations are performed separately. Using pipeline technology, the output results of the two accumulations are summarized and stored in the register. This design greatly saves DSP resource consumption and reduces the use of multiplier resources by about 50%, while stabilizing the clock frequency. When performing downsampling operations, an image resampling strategy is adopted. In the ORB-SLAM system, The sampling fraction is , and the bilinear interpolation method is used to obtain the pixel value of the resampled point. A deep neural network convolution operation has a variety of feature maps of different sizes. In order to further optimize the utilization of hardware resources, the present invention proposes a pyramid image unit reuse strategy, which makes it more suitable for application in the reasoning calculation of the convolutional neural network. Figure 4 As shown in the figure, for the dynamic configuration mechanism of the line buffer of each layer of the pyramid structure, the depth of the line buffer is dynamically adjusted according to the image size requirements of different layers. On the basis of establishing the number of columns of the original input image, the number of columns of each level of the pyramid structure is calculated by scaling it by 1.2 times; the maximum depth of the line buffer corresponds to the image size after scaling, which is used as a benchmark, and the level-by-level tapping method is adopted. Through the mechanism of dynamic configuration of the multiplexer, the dynamic adjustment of the line buffer depth required for the current layer of the pyramid image is realized to ensure that it is compatible with the current feature map size.

[0088] As a further implementation, the rotation angle calculation module estimates the direction of the feature points in the image, that is, the grayscale centroid is calculated based on the coordinate value of the feature point and the smoothed image to obtain the rotation angle of the feature point; the rotation angle calculation module is subdivided into several processing stages through pipeline technology. The first pipeline stage is responsible for calculating the moment of the pixels in a window, and the subsequent two pipeline stages perform centroid division. The next processing stage involves square sum operation and estimation value calculation, and the next processing stage calculates the fast inverse square root, and finally calculates the sine and cosine values ​​of the corner points to update the sign bit of the final result. By adopting the inverse square root method, a more accurate estimation of the rotation angle can be performed.

[0089] As a further implementation, under a specific rotation angle, the conflict problem is mainly manifested as a conflict of access between pairs of pixels in the same row. The present invention stipulates that the two points of a pair of pixels are fixed at the same port of their respective storage units. This constraint is used to reduce the complexity of the interconnection structure, so that the grayscale comparison operation only needs to obtain operands from each port of each storage unit. For pixel pairs that do not have conflicts, they can be directly routed to their corresponding storage units by row. The output of the storage unit in each row will be routed to a register, which caches the operands for grayscale comparison until all pixels are read out from the image block and the comparison operation is completed.

[0090] In order to further reduce the frequency of conflicts, a static scheduling method based on genetic algorithm optimization is proposed to schedule access to pixel pairs in the same row to avoid access conflicts between pixel pairs in the same row. Figure 5 As shown:

[0091] Static scheduling calculation method based on genetic algorithm optimization, including:

[0092] The static mode scheduling problem is represented as a set of pixel pairs P: P = {P1, P2, ..., P N}, where G is a subset of the set P: Where gsize is a fixed parameter, indicating the maximum number of pixel pairs that can be accessed concurrently in each group G; P i represents the i-th element in the set P, that is, the i-th pixel pair; N represents the total number of elements in the set P, that is, the number of all pixel pairs; for this assignment problem, a tuple is used<P,G> Represented by, and the corresponding solution is constructed by the grouping corresponding to each element in the P set.

[0093] Set constraints, including: (1) First constraint: ensure that any pixel pair does not appear in multiple combinations; (2) Second constraint: each combination must be fully allocated with a fixed size of pixel pairs; (3) Third constraint: additional restrictions derived from the features of the proposed framework, such as the need to bind the ports that allow pixel pairs to access the bank.

[0094] Construct an objective function. The objective function is constructed by minimizing the average delay of the computing unit processing pixel pairs for all possible rotation angles under the static model scheduling strategy. It is worth noting that this objective function does not depend on the content of the image block, and this section assumes that the rotation angle follows a uniform distribution and that the types of known reasonable rotation angles are limited.

[0095] Based on the constraints, the genetic algorithm is used to solve the objective function and obtain the corresponding solution; the corresponding solution is a tuple<P,G> Represented by, and composed of groups corresponding to each element in the P set.

[0096] Specifically, in order to apply the genetic algorithm (GA) as a metaheuristic method, the candidate solutions must be encoded first to construct a chromosome representation. At the same time, it is also necessary to establish relevant fitness evaluation indicators, and to clarify the selection, crossover and mutation of genetic operators and their initialization operations. In this study, the chromosome model used to characterize the static scheduling solution adopts a one-dimensional array form of Ai (0≤i<256), where each array element corresponds to an element in the task processing set P. The configuration of the task group depends on the position of the element in the array Ai, and the specific allocation formula is: This representation was chosen because it automatically follows and satisfies the system constraints.

[0097] To construct the initial population, the present invention adopts a random arrangement of P, and then uses genetic operations to iteratively evolve the population. The specific genetic operations are as follows:

[0098] Fitness evaluation: Fitness in the biological sense is a measure of the efficiency of chromosome reproduction. Because our goal is to minimize the clock cycle, we use the negative value of the objective function -F as the fitness function.

[0099] Selection operation: Select individual solutions based on their fitness evaluation. Make more fit solutions more likely to be selected, with the goal of passing the best chromosomes to the offspring. Among all the alternatives for the selection operator, we choose the tournament selection operator. The tournament selection operator randomly selects a number of individuals from the population, and then repeats the "tournament" screening process. The winner of each competition, defined as the most fit individual, will be selected for crossover recombination.

[0100] Crossover recombination: PMX (Partial Matched Crossover) is selected as the crossover operator. This recombination operator generates two child nodes by selecting some chromosome values ​​and exchanging the values ​​of these indexes within a certain segment of the chromosome sequence of the two parent nodes. This operation has a certain probability of generating an offspring containing the excellent gene sequence of the two parent nodes.

[0101] Mutation operation: The mutation operator is to shuffle each chromosome of an individual with a given probability in each generation. The chromosomes of an individual mutate with a specific probability in each generation, that is, the genetic structure of the individual is randomly rearranged. The role of mutation is to promote chromosome exchange between groups.

[0102] Table 1 Parameters used in conflict optimization based on genetic algorithm

[0103] Parameter name Parameter Value Crossover probability 70％ Mutation probability 20％ Population size 300 Crossover operator Partially matched crossover Crossover operator Partially disrupted crossover Select Operator Competition Selection

[0104] The technical achievements of the present invention are mainly reflected in the following aspects: First, by adopting the circuit structure reuse strategy in the implementation process of the Gaussian pyramid module, the usage of block random access memory (BRAM36K) is saved by 33.33%. Secondly, in the hardware architecture design proposed by the present invention, the optimization criteria of equivalent simplification, approximate substitution and multi-cycle calculation are applied, which effectively reduces the resource consumption of the multiplier (DSP48) by 21.21%. These optimization measures not only improve the resource utilization efficiency, but also ensure good compatibility with the ZYNQ Ultrascale+ chip.

[0105] The hardware resource comparison between the present invention and the prior art is shown in Table 2, and it can be seen that the present invention significantly improves the utilization rate of hardware resources.

[0106] Table 2 Hardware resource comparison

[0107]

[0108] Existing technology: Sun R, Qian J, Jose RH, et al. A flexible and efficient real-time orb-based full-hd image feature extraction accelerator [J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2019, 28 (2): 565-575.

[0109] Example 2

[0110] like Figure 6 and Figure 7As shown, the difference between this embodiment and embodiment 1 is that this embodiment provides an ORB algorithm acceleration method based on FPGA, which is an ORB algorithm acceleration system based on FPGA based on embodiment 1; the method includes:

[0111] According to the image pixel column form, the original image (i.e. Figure 7 The grayscale image in the image is stored in the BRAM storage unit; during the storage process, data alignment and shift register mode are used to implement data update;

[0112] Based on the dual-channel parallel processing circuit, preliminary feature point extraction is performed on the original image in the image feature cache module to obtain feature points and corner points;

[0113] The original image in the image feature cache module is downsampled and subjected to Gaussian smoothing filtering to obtain a smoothed image, thereby improving the scale invariance of the image;

[0114] According to the coordinate value of the feature point and the smoothed image, the grayscale centroid is calculated to obtain the rotation angle of the feature point;

[0115] According to the rotation angle of the feature points and the smoothed image, the N most representative feature points are selected from the M feature points through the non-maximum suppression strategy, and the rBRIEF descriptor is calculated and generated to obtain the ORB feature point data.

[0116] As a further implementation, data alignment and shift register mode are used to implement data update. The specific steps are:

[0117] Store data corresponding to the 0th column pixel of the original image on the 0th address line of the BRAM storage unit;

[0118] When each new pixel arrives, the existing data is first read from the BRAM address of the corresponding column, the highest-order pixel is removed from the existing data, and the newly arrived pixel is then appended to the lowest-order pixel to obtain the new pixel data; and the new pixel data is then written back to the same memory address to implement data update;

[0119] When accessing new pixels from multiple rows of input simultaneously, the BRAM is arranged into a shift register mode to construct a multi-row row cache to implement multi-row data update.

[0120] Among them, each step can be executed according to the execution process of each module of the ORB algorithm acceleration system based on FPGA in Example 1, and will not be described one by one in this embodiment.

[0121] In specific implementation, the ORB feature extraction process is implemented at the front end of the system through the following steps: capturing the original image in real time and storing it in the image feature cache module; the ORB accelerator starts and performs the following operations: the original image first undergoes downsampling and Gaussian smoothing filtering in the Gaussian pyramid module to obtain a smoothed image for subsequent calculations; then, the FAST feature extraction module performs preliminary feature point detection on the original image, detects M feature points, and stores the coordinate information of these feature points in the RAM. Next, the descriptors and scores of these M feature points are calculated in parallel. Finally, through the non-maximum suppression strategy, N most representative feature points (N < M) are selected and the corresponding rBRIEF descriptors are generated, completing the entire process of ORB feature extraction. This process ensures the rapid capture, accuracy, and robustness of feature points, optimizing the performance of the SLAM system.

[0122] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the processes Figure 1 or multiple processes and / or blocks

[0124] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one or more of the processes Figure 1 or multiple processes and / or blocks

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0126] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An ORB algorithm acceleration system based on FPGA, characterized in that: The system includes: The image feature cache module is used to store the input original image in the BRAM storage unit in the form of image pixel columns; during the storage process, data alignment and shift register mode are used to realize data update; A FAST feature extraction module, which is used to perform preliminary feature point extraction on the original image in the image feature cache module based on a dual-channel parallel processing circuit to obtain feature points and corner points; A Gaussian pyramid module is used to perform downsampling and Gaussian smoothing filtering on the original image in the image feature cache module to obtain a smoothed image, thereby improving the scale invariance of the image; A rotation angle calculation module, used to calculate the grayscale centroid according to the coordinate value of the feature point and the smoothed image to obtain the rotation angle of the feature point; The rBRIEF descriptor module is used to calculate and generate the rBRIEF descriptor according to the feature point rotation angle and the smoothed image to obtain ORB feature point data; during the calculation process, the two points of a pair of pixel points are fixed at the same port of their respective storage units.

2. The FPGA-based ORB algorithm acceleration system according to claim 1, characterized in that: The image feature cache module uses data alignment and shift register mode to implement data update, including: Store data corresponding to the 0th column pixel of the original image on the 0th address line of the BRAM storage unit; When each new pixel arrives, the existing data is first read from the BRAM address of the corresponding column, the highest-order pixel is removed from the existing data, and the newly arrived pixel is then appended to the lowest-order pixel to obtain the new pixel data; and the new pixel data is then written back to the same memory address to implement data update; When multiple rows of new pixels are accessed simultaneously, the BRAM is arranged in a shift register mode to construct a multi-row row cache to implement multi-row data update.

3. The FPGA-based ORB algorithm acceleration system according to claim 1, characterized in that: The FAST feature extraction module includes: A dual-channel parallel processing unit is used to connect 16 subtractors in parallel in each channel to calculate the grayscale difference between the pixels in the original image, perform a difference operation on the calculated grayscale difference and the preset grayscale threshold to obtain an operation result; compare the operation result with the zero value, and output the judgment result in binary form; if the judgment result is greater than zero, the pixel is marked as a bright spot / dark spot and a high level 1 is output; otherwise, the pixel is marked as a non-bright spot / non-dark spot and a low level 0 is output; A feature point forming unit, used to form a 16-bit binary output string as a feature point according to the output result of each channel in the dual-channel parallel processing unit and the discrimination results of 16 outputs; The corner point judgment unit is used to compare all possible pattern character strings pre-defined and stored with the 16-bit binary output string to obtain a comparison result; based on the comparison result, a logic AND operation is used to process the 16-bit binary output string to judge whether a bright spot or a dark spot is detected; and the detection results of the bright spot and the dark spot are combined by a logic OR operation to determine whether the pixel point constitutes a corner point.

4. The FPGA-based ORB algorithm acceleration system according to claim 3, characterized in that: The calculation of the grayscale difference includes: A Bresenham circle with a radius of 3 is used to select 16 key sampling points. In each clock cycle, a grayscale value comparison operation is performed between the 7×7 central pixel point and the 16 sampling points on the Bresenham circle with a radius of 3 around it to obtain the grayscale difference.

5. The FPGA-based ORB algorithm acceleration system according to claim 1, characterized in that: The Gaussian pyramid module includes: The Gaussian filter processing unit is used to perform Gaussian algorithm processing, and first performs head-to-tail symmetrical addition processing and then performs product operation on the column data of the image matrix according to the order of input pixel points to obtain the product result; finally, the filtering result, that is, the smoothed image, is obtained by accumulating the product results.

6. The FPGA-based ORB algorithm acceleration system according to claim 1, characterized in that: The Gaussian pyramid module also includes a dynamic configuration mechanism: dynamically adjusting the depth of the line buffer according to the image size requirements of different levels, specifically including: On the basis of determining the number of columns of the original input image, the number of columns of each level of the image in the pyramid structure is calculated by scaling it by 1.2 times; The maximum depth of the line buffer corresponds to the scaled image size. Based on this, a step-by-step tapping method is adopted. Through the dynamic configuration mechanism of the multiplexer, the depth of the line buffer required for the current layer image of the pyramid is dynamically adjusted to ensure that it is compatible with the current feature map size.

7. The FPGA-based ORB algorithm acceleration system according to claim 1, characterized in that: During the calculation process, the rBRIEF descriptor module also uses a static scheduling calculation method based on genetic algorithm optimization to perform access scheduling on the pixel pairs in the same row; The static scheduling calculation method based on genetic algorithm optimization includes: The static mode scheduling problem is represented as a set of pixel pairs P: P = {P1, P2, ..., P N }, where G is a subset of the set P: Where gsize is a fixed parameter, indicating the maximum number of pixel pairs that can be accessed concurrently in each group G; P i represents the i-th element in the set P, that is, the i-th pixel pair; N represents the total number of elements in the set P, that is, the number of all pixel pairs; Constructing an objective function, wherein the objective function is constructed by minimizing the average delay of a computing unit processing pixel pairs for all possible rotation angles under a static model scheduling strategy; Based on the constraint conditions, the objective function is solved by using a genetic algorithm to obtain a corresponding solution; the corresponding solution is a tuple<P,G> Represented by, and composed of groups corresponding to each element in the P set.

8. The FPGA-based ORB algorithm acceleration system according to claim 7, characterized in that: The constraints include: The first constraint: any pixel pair cannot appear in multiple combinations; The second constraint: Each combination must be completely assigned a fixed number of pixel pairs; The third constraint condition is to bind the ports of the pixel pairs accessing the bank.

9. An ORB algorithm acceleration method based on FPGA, characterized in that: The method includes: The original image acquired in real time is stored in the BRAM storage unit in the form of image pixel columns; during the storage process, data alignment and shift register mode are used to realize data update; Based on a dual-channel parallel processing circuit, preliminary feature point extraction is performed on the original image in the image feature cache module to obtain feature points and corner points; Downsampling and Gaussian smoothing filtering are performed on the original image in the image feature cache module to obtain a smoothed image, thereby improving the scale invariance of the image; Calculate the grayscale centroid according to the coordinate value of the feature point and the smoothed image to obtain the rotation angle of the feature point; According to the feature point rotation angle and the smoothed image, rBRIEF descriptor is calculated and generated to obtain ORB feature point data.

10. The FPGA-based ORB algorithm acceleration method according to claim 9, characterized in that: Data update is achieved using data alignment and shift register mode. The specific steps are: Store data corresponding to the 0th column pixel of the original image on the 0th address line of the BRAM storage unit; When each new pixel arrives, the existing data is first read from the BRAM address of the corresponding column, the highest-order pixel is removed from the existing data, and the newly arrived pixel is then appended to the lowest-order pixel to obtain the new pixel data; and the new pixel data is then written back to the same memory address to implement data update; When multiple rows of new pixels are accessed simultaneously, the BRAM is arranged in a shift register mode to construct a multi-row row cache to implement multi-row data update.

Images