Image tracking method applicable to implementation of embedded system

Abstract

The invention relates to an image tracking method applicable to implementation of an embedded system. The image tracking method comprises the following steps: continuously storing to-be-matched image pixel data and template image pixel data in a memory respectively according to a small-end storage mode; performing image tracking optimization on to-be-matched image pixel data and template image pixel data by constructing a correlation coefficient; according to the method, the fast tracking algorithm and the NEON parallel optimization method are adopted, the effect of achieving and optimizing the embedded image tracking system is remarkable, reliability and portability are high, and the processing performance of the embedded system in real-time image tracking is greatly improved.

Description

Technical Field

[0001] The present invention is aimed at the design of an embedded system for achieving real-time target tracking, especially an embedded system based on NEON technology, and adopts a fast image tracking algorithm to achieve the design and optimization acceleration method of real-time image tracking. Background Art

[0002] With the development of embedded devices, the application demand of real-time image tracking systems is also developing in the direction of low cost and small size.

[0003] On the one hand, the image tracking algorithm has a huge amount of computation and requires a higher degree of computational parallelism. The embedded processing system based on DSP+FPGA is widely used. DSP has the ability of high-speed multiplication and addition. High-end DSP also has multi-core processors. With the excellent parallel capability, flexible logic control and rich interface resources of FPGA, it can meet the needs of most tracking algorithms.

[0004] On the other hand, embedded systems based on DSP+FPGA are subject to many limitations in applications that require low cost, low power consumption, high volume, and not very high real-time requirements. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an improved image tracking algorithm suitable for embedded system implementation. An ARM processor platform with a NEON core is provided, and an embedded system that uses a fast tracking algorithm to implement image tracking is designed. The algorithm is optimized and accelerated based on the system, which greatly improves the processing performance of real-time image tracking.

[0006] The present invention studies the realization and optimization of the image tracking system based on the ARM processor platform with NEON core. NEON provides an extended SIMD instruction set and a dedicated register system, which greatly improves the processing performance of the embedded system. Due to the high degree of parallel computing, the demand for off-chip memory resources is reduced, reducing the cost and development difficulty.

[0007] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: an image tracking method suitable for implementation in an embedded system, comprising the following steps:

[0008] The pixel data of the image to be matched and the pixel data of the template image are stored continuously in the memory in a little-endian storage manner respectively;

[0009] By constructing a correlation coefficient, the pixel data of the image to be matched and the pixel data of the template image are optimized for image tracking;

[0010] Select the point with the largest correlation coefficient value as the image tracking point to be matched to achieve image tracking.

[0011] The method of optimizing the image tracking of the to-be-matched image pixel data and the template image pixel data by constructing the correlation coefficient comprises the following steps:

[0012] 1) Construct the correlation coefficient:

[0013]

[0014] Where:

[0015]

[0016]

[0017]

[0018] Among them, the width and height of the image to be matched are M and N respectively, ρ(x, y) represents the correlation coefficient between the image to be matched and the template image, x, y represent the pixel coordinates of the image to be matched, (x, y)∈M×N, m, n represent the width and height of the template image respectively; represents the grayscale mean of the template image, represents the mean grayscale value of the image pixels to be matched, δ t represents the variance of the template image; f(i+x,j+y) represents the grayscale value of the pixel point (x+i,y+j) of the image to be matched; g(i,j) represents the grayscale value of the pixel point (i,j) of the template image;

[0019] 2) Build and table:

[0020] S 1 (x,y)=f(x,y)+S 1 (x-1,y)+S 1 (x,y-1)-S 1 (x-1,y-1)

[0021] S 2 (x,y)=f 2 (x,y)+S 2 (x-1,y)+S 2 (x,y-1)-S 2 (x-1,y-1)

[0022] S 3 (x,y)=f(x,y)·g(i,j)+S 3 (x-1,y)+S 3 (x,y-1)-S 3 (x-1,y-1)

[0023] Among them, S 1(x, y) represents the cumulative sum of the grayscale of each pixel in the image to be matched from the starting point to the point (x, y), S 2 (x, y) represents the square sum of the grayscale of each pixel in the image to be matched from the starting point to the point (x, y), S 3 (x,y) represents the cumulative sum of the cross-correlation values ​​of the grayscale of each pixel from the starting point to the point (x,y); f(x,y) represents the grayscale value of the pixel x,y of the image to be matched, and g(i,j) represents the grayscale value of the pixel i,j of the template image;

[0024] 3) Calculate the cumulative sum, square cumulative sum and cross-correlation values ​​respectively:

[0025] Cumulative sum:

[0026]

[0027] Cumulative sum of squares:

[0028]

[0029] Cross-correlation value:

[0030]

[0031] 4) Substitute the cumulative sum, square cumulative sum, and cross-correlation value obtained in step 3) into the corresponding items in the correlation coefficient.

[0032] By constructing a correlation coefficient, the pixel data of the image to be matched and the pixel data of the template image are optimized for image tracking, and after optimization processing by the sum table method, the computational complexity is reduced from o(m×n×M×N) to o(M×N); wherein m and n represent the width and height of the template image, and M and N represent the width and height of the image to be matched.

[0033] By constructing the correlation coefficient, the pixel data of the image to be matched and the pixel data of the template image are subjected to image tracking optimization, and the cumulative sum, the square cumulative sum and the cross-correlation value are calculated for step 3), and the optimization is performed through NEONC inline function optimization and inline assembly language optimization, as follows:

[0034] Add header files to the program code and use C language to directly access the data types and inline functions of the NEON unit to improve the efficiency of algorithm operation.

[0035] An image tracking system suitable for embedded system implementation, comprising:

[0036] A storage module, used for storing pixel data of the image to be matched and pixel data of the template image in a memory in a continuous manner according to a little-endian storage method;

[0037] A tracking optimization module, used for performing image tracking optimization on pixel data of the image to be matched and pixel data of the template image by constructing a correlation coefficient;

[0038] The tracking module is used to select the point with the largest correlation coefficient value as the image tracking point to be matched to achieve image tracking.

[0039] The present invention has the following beneficial effects and advantages:

[0040] The use of fast tracking algorithm and NEON parallel optimization method has significant effects on the implementation and optimization of embedded image tracking system, which is reliable and highly portable, and greatly improves the processing performance of embedded system in real-time image tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the optimization process of the present invention.

[0042] Figure 2 The figure is a practical effect diagram of image tracking using the method of the present invention. DETAILED DESCRIPTION

[0043] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0044] An improved image tracking algorithm suitable for embedded system implementation includes the following steps:

[0045] (1) The pixel data of the image to be matched and the pixel data of the template image are continuously stored in the memory according to the little-endian 8-bit storage method;

[0046] (2) Optimizing the image tracking algorithm to obtain a fast tracking algorithm;

[0047] (3) The parts of the software that require a lot of calculations are parallelized and optimized using NEONC intrinsic functions and NEON assembly language.

[0048] Preferably, in step (2), the correlation coefficient of the normalized cross-correlation algorithm is defined as:

[0049]

[0050] Where f is the real-time image data to be matched, with a size of MxN, g is the template reference image data, with a size of mxn, where M≥m, N≥n, ρ(x,y) is the correlation coefficient between the template and the image to be matched on (x,y), f(i+x,j+y) represents the grayscale value of the pixel in the i+xth row and j+yth column in the image to be matched, g(i,j) is the grayscale value of the pixel in the i-th row and j-th column in the template image, is the grayscale mean of the area to be matched in the real-time image, is the grayscale mean of the template image. Therefore, formula (1) can be transformed into:

[0051]

[0052] Where:

[0053]

[0054]

[0055]

[0056] The template mean in formula (3) and (5) and template variance δ t Only one calculation is needed, and the amount of calculation is reduced by simplifying the formula. However, the covariance calculation between the image to be matched and the template image still needs to be repeated many times. Therefore, the algorithm needs to be further optimized.

[0057] By summing the table, we can reduce The amount of calculation.

[0059] The fast sum table method is to accumulate the pixels starting from the starting point for the image to be matched.

[0060]

[0061] Cumulative sum of squares

[0062]

[0063] and the cross-correlation between the two images

[0064]

[0065] Build and Table:

[0066] S 1 (x,y)=f(x,y)+S 1 (x-1,y)+S 1 (x,y-1)-S 1 (x-1,y-1)

[0067] S 2 (x,y)=f 2 (x,y)+S 2 (x-1,y)+S 2 (x,y-1)-S 2 (x-1,y-1)

[0068] S 3 (x,y)=f(x,y)·g(i,j)+S 3(x-1,y)+S 3 (x,y-1)-S 3 (x-1,y-1)

[0069] Then the cumulative sum, the square cumulative sum, and the cross-correlation value of the two images are:

[0070]

[0071]

[0072]

[0073] After optimization by summing the table method, the computational complexity is reduced from o(m×n×M×N) to o(M×N). The above optimization has effectively compressed the amount of computation.

[0074] Preferably, in step (3), the parallel algorithm optimization based on NEON comprises the following steps:

[0075] NEONC intrinsic function optimization and inline assembly language optimization: Add the "arm_neon.h" header file to the program code, use C language to directly access the data type and intrinsic function of the NEON unit, and improve the algorithm operation efficiency.

[0076] like Figure 1 As shown, an improved image tracking algorithm suitable for embedded system implementation includes the following steps:

[0077] (1) The image data to be processed and the template image data are stored continuously in the memory in the little-endian 8-bit storage mode;

[0078] (2) Optimizing the image tracking algorithm to obtain a fast tracking algorithm;

[0079] (3) The parts of the software that require a lot of calculations use NEONC intrinsic functions and NEON assembly language for parallel optimization;

[0080] First, the image data to be processed and the template image data are continuously stored in the memory according to the little-endian 8-bit storage method, and the image tracking algorithm is optimized to obtain a fast tracking algorithm, and the fast algorithm formula is obtained: Then the amount of calculation is compressed by summing the table method; the part of the algorithm that requires a lot of calculation in the software is parallelized and optimized using NEONC inline functions and NEON assembly language, and NEON instructions are used to 1 (x,y),S 2 (x,y),S 3The calculation of the (x, y) formula mainly involves data loading, extracting vector elements, and multiplication and addition operations. The NEON engine's register with a maximum length of 128 bits can simultaneously process m n-bit integer data (n can be 8, 16, 32, 64; mxn = 128 bits). According to the size of the algorithm's to-be-matched area and the template area, appropriate m and n are selected. The present invention uses the simultaneous loading of 8 8-bit to-be-matched image data and template area image data, namely vld1_u8, and the multiplication of 8 8-bit data in pairs, and the result is directly expanded to 16-bit data, namely vmull_u8; the vgetq_lane_u16 function extracts any 16-bit vector element from a vector with 8 16-bit data for accumulation; the actual operation time after the above optimization is optimized as shown in Table 1;

[0081] Table 1 Comparison of operation time of different optimized algorithms

[0082] Adoption Method Operation time Traditional methods 35.2ms NEONC Inline Function 15.6ms Sum table method + NEONC inline function 6.2ms

[0083] The final image tracking result is as follows: Figure 2 shown.

Claims

1. An image tracking method suitable for embedded system implementation, It is characterized in that The following steps are involved: The pixel data of the image to be matched and the pixel data of the template image are stored continuously in the memory in a little-endian storage manner respectively; By constructing a correlation coefficient, the pixel data of the image to be matched and the pixel data of the template image are optimized for image tracking; Select the point with the largest correlation coefficient value as the image tracking point to be matched to achieve image tracking.

2. The image tracking method according to claim 1, It is characterized in that The method of optimizing the image tracking of the to-be-matched image pixel data and the template image pixel data by constructing the correlation coefficient comprises the following steps: 1) Construct the correlation coefficient: Where: Among them, the width and height of the image to be matched are M and N respectively, ρ(x, y) represents the correlation coefficient between the image to be matched and the template image, x, y represent the pixel coordinates of the image to be matched, (x, y)∈M×N, m, n represent the width and height of the template image respectively; represents the grayscale mean of the template image, represents the mean grayscale value of the image pixels to be matched, δ t represents the variance of the template image; f(i+x,j+y) represents the grayscale value of the pixel point (x+i,y+j) of the image to be matched; g(i,j) represents the grayscale value of the pixel point (i,j) of the template image; 2) Build and table: S 1 (x,y)=f(x,y)+S 1 (x-1,y)+S 1 (x,y-1)-S 1 (x-1,y-1) S 2 (x,y)=f 2 (x,y)+S 2 (x-1,y)+S 2 (x,y-1)-S 2 (x-1,y-1) S 3 (x,y)=f(x,y)·g(i,j)+S 3 (x-1,y)+S 3 (x,y-1)-S 3 (x-1,y-1) Among them, S 1 (x, y) represents the cumulative sum of the grayscale of each pixel in the image to be matched from the starting point to the point (x, y), S 2 (x, y) represents the square sum of the grayscale of each pixel in the image to be matched from the starting point to the point (x, y), S 3 (x,y) represents the cumulative sum of the cross-correlation values ​​of the grayscale of each pixel from the starting point to the point (x,y); f(x,y) represents the grayscale value of the pixel x,y of the image to be matched, and g(i,j) represents the grayscale value of the pixel i,j of the template image; 3) Calculate the cumulative sum, square cumulative sum and cross-correlation values ​​respectively: Cumulative sum: Cumulative sum of squares: Cross-correlation value: 4) Substitute the cumulative sum, square cumulative sum, and cross-correlation value obtained in step 3) into the corresponding items in the correlation coefficient.

3. The image tracking method according to claim 1, It is characterized in that By constructing a correlation coefficient, the pixel data of the image to be matched and the pixel data of the template image are optimized for image tracking, and after optimization processing by the sum table method, the computational complexity is reduced from o(m×n×M×N) to o(M×N); wherein m and n represent the width and height of the template image, and M and N represent the width and height of the image to be matched.

4. The image tracking method according to claim 1, It is characterized in that By constructing the correlation coefficient, the pixel data of the image to be matched and the pixel data of the template image are subjected to image tracking optimization, and the cumulative sum, the square cumulative sum and the cross-correlation value are calculated for step 3), and the optimization is performed through NEONC inline function optimization and inline assembly language optimization, as follows: Add header files to the program code and use C language to directly access the data types and inline functions of the NEON unit to improve the efficiency of algorithm operation.

5. An image tracking system suitable for embedded system implementation, It is characterized in that include: A storage module, used for storing pixel data of the image to be matched and pixel data of the template image in a memory in a continuous manner according to a little-endian storage method; A tracking optimization module, used for performing image tracking optimization on pixel data of the image to be matched and pixel data of the template image by constructing a correlation coefficient; The tracking module is used to select the point with the largest correlation coefficient value as the tracking point of the image to be matched to realize image tracking.

Images