Optimization method and implementation device of digital image correlation method based on FPGA

Abstract

This invention provides an optimized method and implementation structure for digital image correlation (DIR) based on FPGA. The optimized method addresses the low efficiency problem of traditional DIR implementations on FPGAs. Its steps include: identifying interest points within an interest region; determining the computation path; generating an initial value list and a result list; inputting the interest points into the interest point computation pipeline; and concurrently acquiring and updating the initial value list and result list. The implementation structure includes an initial value list cache module, a result list cache module, an external memory control module, an interest point scheduling module, and an interest point computation result acquisition module. This invention improves the computational parallelism of the DIR method, thereby increasing computational efficiency while reducing its implementation cost. This invention provides an effective solution to the problems of slow measurement speed and large, complex measurement systems in existing DIR measurements.

Description

Technical Field

[0001] This invention belongs to the field of digital image correlation measurement technology, specifically relating to an optimization method and implementation device for digital image correlation based on FPGA. Background Technology

[0002] Digital image correlation (DIR) is a non-contact measurement method for measuring surface deformation of objects, widely used in fields such as building deformation and material deformation. However, due to its computationally intensive nature, DIR requires a powerful computing platform to meet the application needs in practical engineering.

[0003] Currently, the mainstream digital image correlation measurement computing platforms are heterogeneous combinations of CPUs and GPUs, or both. These platforms suffer from drawbacks such as high cost, bulky system architecture, high power consumption, and difficulty in expansion and upgrades, failing to meet the requirements of some practical measurement engineering projects. In particular, for measurement scenarios with extremely high requirements for speed and stability in digital image correlation measurements, such as dynamic characteristic measurements, real-time measurements, and measurement and feedback control, as well as measurement scenarios with strict limitations on power consumption, these computing platforms cannot meet engineering needs.

[0004] FPGA acceleration methods include pipelining and parallel computing. The biggest feature of pipelining is that after the raw computation data is fed into the computation pipeline, the next batch of raw computation data can be fed in without waiting for the current computation to complete, thereby improving the efficiency of the computation pipeline.

[0005] FPGA's pipelining, parallelism, and low power consumption characteristics can effectively address the aforementioned problems in digital image correlation measurement. However, traditional digital image correlation calculation methods, such as the "seed point diffusion" algorithm for CPU multithreading and the "path independence" algorithm for GPU high parallelism, only consider "parallel computing" and do not take into account the characteristics and problems of pipelining. Therefore, these algorithms have very low execution efficiency on FPGAs. To improve the efficiency of digital image correlation calculations on FPGAs, it is necessary to perform adaptive optimizations of traditional digital image correlation measurement calculation processes for FPGAs to meet the acceleration characteristics of FPGA parallel and pipelining computing, thereby addressing the application needs of digital image correlation measurement in practical engineering. Summary of the Invention

[0006] To address the aforementioned issues, this invention discloses an optimization method and implementation device for digital image correlation based on FPGA. This method offers advantages such as high parallelism, low dependency, low power consumption, low cost, high FPGA compatibility, and easy system upgrades, providing a solution to the problems existing in current digital image correlation measurement systems.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] An optimization method for digital image correlation based on FPGA includes two parts, interest point scheduling and interest point calculation result acquisition, which are carried out in parallel and independently.

[0009] Interest point scheduling includes the following steps:

[0010] (1) Determine the locations of all interest points that need to be calculated within the interest area, determine the calculation path of all interest points, and divide the first row of interest points;

[0011] (2) Based on the calculation paths of all interest points determined in step (1), generate a list containing the initial values ​​of all interest points—the initial value list, and an empty result list;

[0012] (3) Check if the initial value list is empty; if it is empty, stop the calculation.

[0013] (4) Read the initial value of an interest point in the initial value list sequentially, and determine whether the interest point is the first interest point. If it is not the first interest point, jump to step (8).

[0014] (5) Input the point of interest into the point of interest calculation pipeline, and then wait for the calculation result of the point of interest;

[0015] (6) Obtain the calculation result of the point of interest and use the calculation result as the initial value of the point of interest in the next row of the interest region, and write it into the corresponding position of the initial value list;

[0016] (7) Write the calculation result of the point of interest into the result list, and then return to step (3);

[0017] (8) Input the point of interest into the point of interest calculation pipeline and immediately return to step (3) without waiting for the calculation result;

[0018] The process of obtaining the interest point calculation results includes the following steps:

[0019] (9) Wait for the calculation results of the interest points in step (8);

[0020] (10) Use the calculation result of the interest point as the initial value of the interest point in the next row of the interest region, and write it into the corresponding position of the initial value list;

[0021] (11) Write the calculation result of the point of interest into the result list, and then return to step (9).

[0022] Furthermore, the first row of interest points mentioned in step (4) specifically refers to interest points that cannot obtain initial values ​​for calculation through steps (6) and (10) when implementing digital image correlation calculation using the present invention;

[0023] To implement the above optimization method, the present invention provides an implementation device for a digital image correlation optimization method based on FPGA, including an initial value list cache module, a result list cache module, an external memory control module, an interest point scheduling module, an interest point calculation result acquisition module, and an interest point calculation pipeline.

[0024] The initial value list cache module is used to cache the part of the initial value list that needs to be processed soon;

[0025] The initial value list cache module contains a true dual-port BRAM to enable the module to simultaneously read and write data to the external memory control module and the interest point scheduling module, thereby improving computational parallelism.

[0026] The result list cache module is used to cache the calculation results of the completed interest points;

[0027] The result list cache module contains a FIFO for sequential caching of interest point calculation results;

[0028] The external memory control module is used to control the external memory DDR, and on this basis, to provide the required list content to the initial value list cache module, and to store the calculation results in the result list cache module into the DDR.

[0029] The point of interest scheduling module is used to update the initial value list, read the initial value list, and control the point of interest calculation process.

[0030] The interest point calculation result acquisition module is used to acquire interest point calculation results and update the initial value list;

[0031] The interest point calculation pipeline includes an integer-pixel matching pipeline and a sub-pixel matching pipeline;

[0032] The subpixel matching pipeline is used to complete the subpixel matching of interest points through pipelined computation. Its matching algorithm includes, but is not limited to, the inverse synthetic Newton-Gaussian algorithm (IC-GN).

[0033] The integer pixel matching pipeline is used to complete integer pixel matching of points of interest through pipelined computation, and its matching algorithm includes, but is not limited to, template matching algorithm.

[0034] 1. This invention can fully utilize the two computing acceleration features of FPGA: pipelined computing and parallel computing, so that digital image correlation measurement can fully adapt to the acceleration features of FPGA, thereby improving the computing efficiency of the measurement system on FPGA.

[0035] 2. This invention effectively removes the temporal dependency between interest point scheduling and interest point calculation, thereby improving the concurrency between calculation and scheduling;

[0036] 3. Compared with the "seed point diffusion method", this invention reduces the dependency between the interest point results and the calculation of other interest points, and effectively improves the parallelism between the calculation of different interest points;

[0037] 4. This invention can improve the regularity of the raw data required for system calculation, which can improve the utilization rate of raw data by the computing system, thereby reducing the difficulty of system implementation and reducing the performance requirements of the measurement system on the hardware platform.

[0038] 5. Compared with the "independent path" algorithm used on the GPU platform, this invention can reduce the computational complexity of digital image-related methods, thereby reducing the consumption of computing resources.

[0039] 6. The measurement system implemented using this invention can be easily upgraded, replaced, and modified for specific measurement scenarios. Attached Figure Description

[0040] Figure 1 This is a schematic diagram showing the apparatus for implementing the present invention and its associated circuitry.

[0041] Figure 2 This is a flowchart illustrating the calculation process of the optimization method of the FPGA-based digital image correlation method of the present invention.

[0042] List of reference numerals in the attached diagram:

[0043] 1. DDR, 2. External memory control module, 3. Initial value list cache module, 4. Result list cache module, 5. Interest point scheduling module, 6. Interest point calculation result acquisition module, 7. Interest point calculation pipeline. Detailed Implementation

[0044] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0045] like Figure 1As shown, the implementation device of the present invention includes an external memory control module 2, an initial value list cache module 3, a result list cache module 4, an interest point scheduling module 5, and an interest point calculation result acquisition module 6. The external memory control module 2 is connected to DDR1 via a read / write bus and can read and write initial value lists from DDR1; the external memory control module 2 is also responsible for writing a portion of the initial value list read from DDR1 into the initial value list cache module 3. The interest point scheduling module 5 actively reads initial interest point values ​​from the initial value list cache module, puts them into the interest point calculation pipeline 7, and determines whether the interest point is the first row interest point. If it is, it waits for the calculation to complete, reads the calculation result, and updates the initial value list in the initial value list cache module 3; otherwise, it directly ends the current scheduling and rereads new initial interest point values ​​from the initial value list cache module 3. The interest point calculation result acquisition module 6 continuously waits for the calculation results from the interest point calculation pipeline 7. Once an interest point calculation result is output, it is input into the result list cache module 4, and the initial value list in the initial value list cache module 3 is updated. When the initial value list cache module 3 detects that its internal initial value list cache is about to run out, it notifies the external memory control module 2 to write it into DDR1. When the result list cache module 4 detects that its internal result list cache is about to be full, it notifies the external memory control module 2 to write it into DDR1.

[0046] like Figure 2As shown, step 1 is responsible for initialization. In step 1, the positions of all interest points that need to be calculated within the interest area are determined, and the calculation paths of all interest points are determined, and the first row of interest points is divided. In step 2, based on the calculation paths of all interest points determined in step (1), a list containing the initial values ​​of all interest points—the initial value list—and an empty result list are generated and stored in DDR1. In step 3, the initial value list cache module 3 determines whether the initial value list is empty. If it is empty, the calculation ends; otherwise, it jumps to step 4. In step 4, the interest point scheduling module 5 sequentially reads an interest point from the initial value list cache module 3 and determines whether the current interest point is the first row of interest points. If it is, it jumps to step 5; otherwise, it jumps to step 8. In step 5, the interest point is put into the interest point calculation pipeline 7, and then waits to obtain the calculation result of the interest point. In step 8, after the interest point is put into the interest point calculation pipeline 7, it returns to step 3 to start the scheduling of a new interest point. In step 6, the interest point scheduling module 5 obtains the calculation result and updates the initial value list in the initial value list cache module 3. Specifically, the update method is as follows: the calculation result is used as the initial value of the interest point in the next row of the interest region and written to the corresponding position in the initial value list. In step 7, the calculation result of the interest point is written to the result list, and then the process returns to step 3. In step 9, the interest point calculation result acquisition module 6 waits for the calculation result output from the interest point calculation pipeline 7. In step 10, the interest point calculation result acquisition module 6 obtains the calculation result and updates the initial value list. In step 11, the calculation result of the interest point is written to the result list, and then the process returns to step (9) to end the current result acquisition.

[0047] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. An optimization method for digital image correlation based on FPGA, characterized in that: The process includes two parallel and independent parts: interest point scheduling and interest point calculation result acquisition. Interest point scheduling includes the following steps: (1) Determine the locations of all interest points that need to be calculated within the interest area, determine the calculation path of all interest points, and divide the first row of interest points; (2) Based on the calculation paths of all interest points determined in step (1), generate a list containing the initial values ​​of all interest points—the initial value list, and an empty result list; (3) Check if the initial value list is empty; if it is empty, stop the calculation. (4) Read the initial value of an interest point in the initial value list sequentially, and determine whether the interest point is the first interest point. If it is not the first interest point, jump to step (8). (5) Input the point of interest into the point of interest calculation pipeline, and then wait for the calculation result of the point of interest; (6) Obtain the calculation result of the point of interest and use the calculation result as the initial value of the point of interest in the next row of the interest region, and write it into the corresponding position of the initial value list; (7) Write the calculation result of the point of interest into the result list, and then return to step (3); (8) Input the point of interest into the point of interest calculation pipeline and immediately return to step (3) without waiting for the calculation result; The process of obtaining the interest point calculation results includes the following steps: (9) Wait for the calculation results of the interest points in step (8); (10) Use the calculation result of the interest point as the initial value of the interest point in the next row of the interest region, and write it into the corresponding position of the initial value list; (11) Write the calculation result of the point of interest into the result list, and then return to step (9).

2. An implementation apparatus for an optimization method of digital image correlation based on FPGA, characterized in that: It includes an initial value list cache module, a result list cache module, an external memory control module, an interest point scheduling module, an interest point calculation result acquisition module, and an interest point calculation pipeline; The initial value list cache module is used to cache the parts of the initial value list that need to be processed immediately. The result list cache module is used to cache the calculation results of points of interest that have been calculated recently. The external memory control module is used for read and write control of the external memory DDR, and on this basis, provides the required list content to the initial value list cache module, and stores the calculation results in the result list cache module into the DDR. The interest point scheduling module is used to implement all the steps of interest point scheduling in an optimization method of digital image correlation based on FPGA, including... (1) Determine the locations of all interest points that need to be calculated within the interest area, determine the calculation path of all interest points, and divide the first row of interest points; (2) Based on the calculation paths of all interest points determined in step (1), generate a list containing the initial values ​​of all interest points—the initial value list, and an empty result list; (3) Check if the initial value list is empty; if it is empty, stop the calculation. (4) Read the initial value of an interest point in the initial value list sequentially, and determine whether the interest point is the first interest point. If it is not the first interest point, jump to step (8). (5) Input the point of interest into the point of interest calculation pipeline, and then wait for the calculation result of the point of interest; (6) Obtain the calculation result of the point of interest and use the calculation result as the initial value of the point of interest in the next row of the interest region, and write it into the corresponding position of the initial value list; (7) Write the calculation result of the point of interest into the result list, and then return to step (3); (8) Input the point of interest into the point of interest calculation pipeline and immediately return to step (3) without waiting for the calculation result; The interest point calculation result acquisition module is used to implement all the steps of interest point calculation result acquisition in an optimization method of digital image correlation based on FPGA, including... (9) Wait for the calculation results of the interest points in step (8); (10) Use the calculation result of the interest point as the initial value of the interest point in the next row of the interest region, and write it into the corresponding position of the initial value list; (11) Write the calculation result of the point of interest into the result list, and then return to step (9).

3. The implementation apparatus for the optimization method of FPGA-based digital image correlation according to claim 2, characterized in that: The initial value list cache module contains a true dual-port BRAM.

4. The implementation apparatus for the optimization method of FPGA-based digital image correlation according to claim 2, characterized in that: The result list cache module contains a FIFO.

5. The implementation apparatus for the optimization method of digital image correlation based on FPGA according to claim 2, characterized in that: The interest point calculation pipeline includes an integer-pixel matching pipeline and a sub-pixel matching pipeline.

6. The implementation apparatus for the optimization method of digital image correlation based on FPGA according to claim 5, characterized in that: The matching algorithm used in the subpixel matching pipeline includes, but is not limited to, the pipelined inverse synthetic Newton-Gaussian algorithm.

7. The implementation apparatus for the optimization method of digital image correlation based on FPGA according to claim 5, characterized in that: The matching algorithm used in the integer pixel matching pipeline includes, but is not limited to, template matching algorithm.

Images