A Fast Mapping Method between Multi-View Image Patches

Abstract

The present invention discloses a fast mapping method between multi-view image patches, which relates to the technical field of image processing algorithms. The present invention includes the fast calculation of the single mapping matrix H: decompose the H matrix formula in the algorithm process into constant terms and variable terms. For the constant terms, only one calculation and storage are required and they can be applied to all image patches in the image; for the variable terms, approximate all normal vectors by discrete sampling points on the surface of the unit sphere and calculate the values of all variable terms. In this way, only look-up tables are needed to complete the calculation of the H matrix in the subsequent process of calculating the H matrix. Compared with the traditional method, the calculation speed is greatly improved, and the fast mapping of the pixel point coordinates in the image patch; the present invention maps the first pixel point, the middle pixel point, and the last pixel point in the first row, the middle row, and the last row of the image patch to another image according to matrix multiplication, and the remaining points can be quickly mapped according to the cross-ratio invariance, which can greatly improve the calculation speed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing algorithms, and particularly relates to a method for fast mapping between multi-view image blocks. Background Art

[0002] Image processing is a technology that uses a computer to analyze images to achieve the desired results. It is also called image processing. Image processing generally refers to digital image processing. A digital image is a large two-dimensional array obtained by devices such as industrial cameras, video cameras, and scanners. The elements of this array are called pixels, and their values are called gray values. When using a computer to process the images obtained from a scene, various algorithms are needed to assist in the processing;

[0003] A method and device for image processing based on an algorithm library proposed in a patent with the patent publication number CN 104598212 A. The algorithm library includes a parent class, an interface, an interface implementation subclass, and a bottom layer library. By constructing an algorithm library architecture including a parent class, an interface, an interface implementation subclass, and a bottom layer library, parsing the parsing parameters corresponding to the algorithm functions, obtaining the separation parameters for implementing the interface implementation subclass, and calling the interface implementation subclass through the interface, the image processing process and parameter configuration are implemented in the interface implementation subclass, and the corresponding image processing algorithms in the bottom layer library are called based on the image processing process and parameter configuration. Any medical imaging system can use the algorithm library with the same architecture. The image processing algorithms can be reused, and the reuse degree is high and the reuse range is large, which can effectively reduce the system maintenance cost;

[0004] For images obtained in a large-scale scene, when processing them, it is often necessary to register them with each other to calculate the three-dimensional data of the scene or calculate the movement trajectory of the camera. The algorithm operation amount of these processing processes is large, resulting in low calculation efficiency. Summary of the Invention

[0005] The purpose of the present invention is to solve the deficiencies in real life, and therefore a method for fast mapping between multi-view image blocks is proposed to solve the above problems.

[0006] Specifically, the following technical solutions are adopted:

[0007] The algorithm process includes the following:

[0008] Fast calculation of the H matrix: Decompose the H matrix formula in the algorithm process into a constant term and a variable term;

[0009] Classification processing: Perform a single operation on the constant term obtained by decomposing the H matrix formula and then store it. For the obtained variable term, discretize the hemisphere of the unit sphere, and then store all relevant terms;

[0010] Fast mapping of pixel coordinates in an image block: Using the H matrix, map a total of 9 points, namely the first point, the middle point, and the last point of the first row, the middle row, and the last row in the image block, to the adjacent image, and then calculate the mapping of the remaining points according to the principle of cross-ratio invariance.

[0011] Preferably, in the decomposition process of the H matrix formula, the acquisition method of the H matrix formula is as follows:

[0012] Let image I r be the reference image, and let I0, I1, I2...I n be adjacent images of the same scene at different angles to I r Let P r be a square region centered at the coordinate x(a, b) in image I r with side length L, and let P0, P1, P2...P n be the corresponding regions in the adjacent images. Let the three-dimensional point on the surface of the object in the three-dimensional scene corresponding to x(a, b) be X, and its tangent plane be π. Projecting P r onto P0, P1, P2...P n means first projecting P r onto the tangent plane π, and then projecting it onto the corresponding region in the adjacent image through the tangent plane π. Then let k r , k0, k1, k2...k n be the internal parameter matrices of the corresponding images, and R r , R0, R1, R2...R n be the rotation matrices of the corresponding image cameras relative to the scene world coordinate system, and C r , C0, C1, C2...C n be the offsets of the corresponding image camera centers relative to the origin of the world coordinate system. According to the principle of multi-view geometry, the mapping matrix r from P r projected through π onto any adjacent image P

[0013]

[0014] Preferably, the classification processing method of the H matrix formula is to first expand the H matrix formula to obtain formula one:

[0015] In the above calculation process, a space-for-time method is used to accelerate the calculation. For it can be calculated and saved as T1, and for it can be split into two parts, k i R i (C i -C r) is a column vector and is a constant, which can be denoted as T2, and is a row vector, where n is denoted as a variable, and this part can be written as T3(n). Therefore, the above formula can be simplified to Formula 2:

[0016]

[0017] Preferably, for the rapid mapping of the pixel coordinates in the image block, let P r be the square region centered at the coordinate x(a, b) in the image I r with side length L. Project P r onto P i and multiply each pixel coordinate in P r by the matrix Then perform coordinate normalization on the obtained homogeneous coordinates, and according to the properties of projective geometry, transform the square region P r into a straight line, and satisfy the invariance of cross-ratio. Let l r be a certain row in P r . Let X1, X3, and X4 correspond to the first, middle, and last pixel points of this row respectively. X1, X3, and X4 are the corresponding points in the image i, and X1, X2, and X4 are the three-dimensional points on the corresponding straight line L in the three-dimensional space tangent plane. According to the invariance of cross-ratio in projective geometry, Formula 3 can be obtained:

[0018]

[0019] Preferably, in Formula 3, multiply the first pixel point X1, the midpoint X2, and the last point X4 of the first row of pixels in P r by to obtain their corresponding coordinates X1, X3, and X4 in the i-th image. Let X2 be a certain pixel point in the first row of P r , then Formula 4 can be obtained:

[0020]

[0021] Preferably, in Formula 4, for each row in the image block P r , the ratio values of |x1x4|, |x3x4|, |x'1x'4|, and |x'3x'4| can be calculated in the first step of calculating each row and let them be k1. Similarly, for each image block, it only needs to be calculated once. The ratio values of |x1x2| and |x1x4| are k2, then Formula 4 can be converted to Formula 5:

[0022] Furthermore, it can be obtained that:

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. The present algorithm mainly includes the following two steps: First, the fast calculation of the single-view matrix H: The H matrix formula in the algorithm process is decomposed into constant terms and variable terms. For the constant terms, only one calculation and storage are required and they can be applied to all image patches in the image. For the variable terms, the discrete sampling points on the surface of the unit sphere are used to approximate all normal vectors, and the values of all variable terms are calculated. In this way, only a look-up table is needed to complete the calculation of the H matrix during the subsequent calculation of the H matrix. Compared with the traditional method, the calculation speed is greatly improved. The second step is the fast mapping of the pixel coordinates in the image patch: Using the H matrix, the first point, the middle point, and the last point in the first row, the middle row, and the last row of the image patch, a total of 9 points, are mapped to the adjacent image, and then according to the principle of cross-ratio invariance, the mapping of the remaining points is calculated. The present invention maps the first pixel point, the middle pixel point, and the last pixel point in the first row, the middle row, and the last row of the image patch to another image according to matrix multiplication, and the remaining points can be quickly mapped according to the cross-ratio invariance, which can greatly improve the calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic flow chart of a fast mapping method between multi-view image patches in the present invention;

[0026] Figure 2 It is a projection view of a fast mapping method between multi-view image patches in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0028] Embodiment 1: As Figure 1-2 shown, a fast mapping method between multi-view image patches, the algorithm process includes the following:

[0029] Fast calculation of the H matrix: The H matrix formula in the algorithm process is decomposed into constant terms and variable terms;

[0030] Classification processing: Perform one operation on the constant terms obtained by decomposing the H matrix formula and then store them. For the obtained variable terms, discretize the hemisphere of the unit sphere, and then store all relevant terms;

[0031] Fast mapping of the pixel coordinates in the image patch: Using the H matrix, the first point, the middle point, and the last point in the first row, the middle row, and the last row of the image patch, a total of 9 points, are mapped to the adjacent image, and then according to the principle of cross-ratio invariance, the mapping of the remaining points is calculated.

[0032] Specifically, in the H matrix factorization process, the method for obtaining the H matrix is as follows:

[0033] Let the image I r be the reference image, and let I0, I1, I2... I n be neighboring images of the same scene at different angles with I r Let P r be a square region centered at the coordinate x(a, b) in the image I r with side length L, and let P0, P1, P2... P n be the corresponding regions in the neighboring images. Let the three-dimensional point on the surface of the object in the three-dimensional scene corresponding to x(a, b) be X, and its tangent plane be π. Projecting P r onto P0, P1, P2... P n means first projecting P r onto the tangent plane π, and then projecting it onto the corresponding regions in the neighboring images through the tangent plane π. Then let k r , k0, k1, k2... k n be the internal parameter matrices of the corresponding images, R r , R0, R1, R2... R n be the rotation matrices of the corresponding image cameras relative to the scene world coordinate system, C r , C0, C1, C2... C n be the offsets of the corresponding image camera centers relative to the origin of the world coordinate system. According to the principle of multi-view geometry, the mapping matrix r from P r projected onto any neighboring image P

[0034]

[0035] Preferably, the classification processing method of the H matrix is to first expand the H matrix to obtain Formula 1:

[0036] In the above calculation process, the method of trading space for time is used to accelerate the calculation. For it can be calculated and saved as T1, and for it can be split into two parts. k i R i (C i - C r ) is a column vector and is a constant, which can be denoted as T2, and is a row vector, where n is regarded as a variable, and this part can be written as T3(n). Therefore, the above formula can be simplified to Formula 2:

[0037]

[0038] Specifically, for the fast mapping of the pixel coordinates in the image block, let P r be the square region centered at the coordinate x(a, b) in the image I r with side length L. Project P r onto P i and multiply each pixel coordinate in P r by the matrix Then perform coordinate normalization on the obtained homogeneous coordinates. According to the properties of projective geometry, transform the square region P r into a straight line, and satisfy the invariance of cross-ratio. Let l r be a certain row in P r . Let X1, X3, and X4 correspond to the first, middle, and last pixel points of this row respectively. X1, X3, and X4 are the corresponding points in the image i, and X1, X2, and X4 are the three-dimensional points on the corresponding straight line L in the three-dimensional space tangent plane. According to the invariance of cross-ratio in projective geometry, formula three can be obtained:

[0039]

[0040] Specifically, in formula three, multiply the first pixel point X1, the midpoint X2, and the last point X4 of the first row of pixels in P r by to obtain their corresponding coordinates X1, X3, and X4 in the i image. Let X2 be a certain pixel point in the first row of P r , then formula four can be obtained:

[0041]

[0042] Specifically, in formula four, for each row in the image block P r , the ratio values of |x1x4|, |x3x4|, |x'1x'4|, and |x'3x'4| can be calculated in the first step of calculating each row and let it be k1. Similarly, for each image block, it only needs to be calculated once. The ratio values of |x1x2| and |x1x4| are k2, then formula four can be converted into formula five:

[0043] Furthermore, it can be obtained:

[0044] Specifically, when calculating x'2, only 1 multiplication, 2 additions, and 2 divisions are required. Compared with 9 multiplications, 6 additions, and 2 divisions in matrix multiplication, there is a significant speed improvement.

[0045] As can be seen from the above, for the traditional algorithm, the depth and normal vector of each pixel need to be calculated Moreover, for the mapping of each region, each pixel needs to be multiplied by the mapping matrix, which results in huge computational complexity. In this algorithm, the first pixel, the middle pixel, and the last pixel of the first row, the middle row, and the last row of the image patch are mapped to another image according to matrix multiplication, and the remaining points can be quickly mapped according to the cross-ratio invariance. For a typical 31×31 image patch, only 9 points need to be mapped to another image according to matrix multiplication, and the remaining points are all mapped and transformed according to the method of cross-ratio invariance, so that the computational speed can be greatly improved.

[0046] As described above, it is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. The substitution may be the substitution of part of the structure, device, and method steps, or the complete technical solution. Any equivalent substitution or change made according to the technical solution and inventive concept of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A fast mapping method between multi-view image blocks, characterized in that, Including: Fast calculation of the H matrix: Decompose the H matrix formula in the algorithm process into constant terms and variable terms; Classification processing: Perform an operation on the constant terms obtained from the decomposition of the H matrix formula and then store them. For the obtained variable terms, discretize the hemispherical surface of the unit sphere, and then store all relevant terms; Fast mapping of pixel coordinates in the image block: Use the H matrix to map a total of 9 points, namely the first point, the middle point, and the last point of the first row, the middle row, and the last row in the image block to the adjacent image, and then calculate the mapping of the remaining points according to the principle of cross-ratio invariance; In the H matrix formula decomposition process, the acquisition method of the H matrix formula is as follows: Let the image I r be the reference image, and let I0, I1, I2... I n be neighboring images of the same scene at different angles with respect to I r Let P r be a square region centered at the coordinate x(a, b) in the image I r with side length L. Let P0, P1, P2... P n be the corresponding regions in the neighboring images. Let the 3D point on the surface of the object in the 3D scene corresponding to x(a, b) be X, and its tangent plane be π. Projecting P r onto P0, P1, P2... P n means first projecting P r onto the tangent plane π, and then projecting it onto the corresponding regions in the neighboring images through the tangent plane π. Let k r , k0, k1, k2... k n be the internal parameter matrices of the corresponding images, and R r , R0, R1, R2... R n be the rotation matrices of the corresponding image cameras with respect to the scene world coordinate system, and C r , C0, C1, C2... C n be the offsets of the corresponding image camera centers with respect to the origin of the world coordinate system. According to the principles of multi-view geometry, the mapping matrix for projecting P r onto any neighboring image P r through π The classification processing method of the H matrix form is as follows: First, expand the H matrix form to obtain Formula 1: The above calculation process uses the method of trading space for time to accelerate the calculation. For it can be calculated and saved as T1. For it can be split into two parts. k i R i (C i -C r ) is a column vector and is a constant, which can be denoted as T2. And is a row vector, where n is regarded as a variable, and this part can be written as T3(n). Therefore, the above formula can be simplified to Formula 2:

2. The fast mapping method between multi-view image blocks according to claim 1, characterized in that The fast calculation method of the H matrix is to let P r be a square region with the center at the coordinate x(a, b) in the image I r and the side length L. Project P r onto P i . Multiply the pixel coordinates in P r on the left by the matrix . Then perform coordinate normalization on the obtained homogeneous coordinates. According to the properties of projective geometry, transform the P r of the square region into a straight line, and satisfy the invariance of cross-ratio. Let l r be a certain row in P r . X1, X3, and X4 correspond to the first, middle, and last pixel points in this row respectively. X1, X3, and X4 are the corresponding points in the image i. X1, X2, and X4 are the three-dimensional points on the corresponding straight line L on the three-dimensional space tangent plane. According to the invariance of cross-ratio in projective geometry, the formula three can be obtained:

3. A fast mapping method between multi-view image blocks according to claim 2, characterized in that In the third formula, multiply the first pixel point X1, the midpoint X2, and the last point X4 of the first row of pixels in P r by to obtain the corresponding coordinates X1, X3, and X4 in the i-th graph. Let X2 be a certain pixel point in the first row of P r , and then the fourth formula can be obtained:

4. The fast mapping method between multi-view image blocks according to claim 3, characterized in that In the fourth formula, for the image block P r For each row in it, the proportional values of |x1x4|, |x3x4|, |x'1x'4|, and |x'3x'4| can be calculated as k1 in the first step of calculating each row. Similarly, for each image block, it only needs to be calculated once. The proportional values of |x1x2| and |x1x4| are k2. Then, the fourth formula can be converted into the fifth formula: Furthermore, it can be obtained that:

Images