Vision fusion method based on FPGA

Abstract

The invention discloses a vision fusion method based on FPGA. The method comprises the step of establishing a vision system taking the FPGA as an operation processing core, wherein the left view and the right view of a stereo camera are input through a SERDES interface; the system comprises six external SRAMs in 1024*8 bit capacity, wherein each two SRAMs form a group of ping-pong buffer, therebyrealizing seamless transmission and pipelined processing of the left and right views and the parallax image. The requirements on high-speed operation and high-speed image transmission can be satisfied, multiple general interfaces and a surplus hardware resource are provided, thereby further extending and enhancing. And meanwhile, the Census stereo fusion is the non-parametric fusion method and ismore robust to the non-uniform brightness and gain and like conditions, and the system precision is improved. Compared with the traditional general processor, the parallel computing and the reasonablepipeline design are sufficiently utilized, the algorithm is directly mapped to the structure, the system operation speed is greatly improved, and the requirements on high resolution, high precision and high speed are satisfied.

Description

technical field [0001] The invention relates to a visual fusion method, in particular to an FPGA-based visual fusion method. Background technique [0002] The main task of the stereo vision system is to obtain the 3D information of the scene, and it has been widely used in the fields of mobile robots, target tracking, 3D reconstruction and so on. In the stereo vision system, stereo fusion is the key core. Generally speaking, the scenes where mobile robots such as outdoor unmanned vehicles and lunar rovers usually lack regular features such as points and lines, and due to the uncertainty of the scene and the influence of factors such as lighting, feature extraction is often unstable, and only A sparse disparity map can be obtained, and a dense disparity map must be obtained through interpolation to reconstruct the scene. Therefore, the region matching algorithm is more inclined to be used in the real-time stereo vision system. [0003] In addition, the stereo vision applie...

AI Technical Summary

Problems solved by technology

Although redundant calculations can be reduced through certain optimization techniques, the complexity can be reduced to O(N2D), but for general-purpose processors, it still cannot meet the real-time requirements, so it is necessary to use hardware acceleration technology or design dedicated hardware circuits