Marking plate center three-dimensional positioning method based on perspective alignment Gabor filter and square lattice

Abstract

The invention relates to the technical field of computer vision, in particular to a marking plate center three-dimensional positioning method based on a perspective alignment Gabor filter and a square dot matrix, which can still realize stable extraction of a pattern quasi-linear phase through the perspective alignment Gabor filter under the condition that a square dot matrix marking plate image has perspective distortion. The method comprises the following steps: firstly, carrying out phase detection on a square dot matrix pattern, then, carrying out zero crossing point detection on the phase to realize center detection with 0.003 pixel precision of the square dot matrix pattern, and finally, improving the precision of three-dimensional positioning by combining with a pose estimation algorithm based on homography transformation, thereby expanding the application of a visual three-dimensional positioning technology in occasions of high-precision calibration, detection, measurement and the like.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and specifically relates to a three-dimensional positioning method for the center of a marker plate based on perspective-aligned Gabor filters and a square dot matrix. Background Art

[0002] With the rapid development of computer vision technology, vision-based 3D positioning methods have become widely used in various research and engineering applications. Compared to traditional 3D positioning methods such as laser trackers, laser scanners, GPS, and inertial measurement units, which extract spatial information from images, the use of vision technology can effectively reduce the reliance on high-precision hardware in traditional positioning systems, lower costs, and offer advantages such as non-contact, high accuracy, and flexible configuration.

[0003] In high-precision applications such as industrial robotics and machine tools, markers are often used to improve the accuracy of visual 3D positioning methods. As a benchmark for visual 3D positioning, their design and distribution play a crucial role in positioning accuracy. Common markers include QR codes, ArUco markers, and checkerboard patterns. By recognizing the features of these patterns, their positions within the image can be determined. Algorithms such as PnP and DLT are then used to map several known points in the world coordinate system to corresponding points in the image plane. This information is then used to estimate the relative pose between the marker and the camera. However, these methods are susceptible to image noise and feature point matching errors, making measurement accuracy difficult to guarantee, especially in the presence of perspective distortion. Improving feature extraction and pose estimation algorithms, as well as corresponding marker design, to further enhance measurement accuracy and stability remains a key challenge in this field.

[0004] Unlike commonly used image brightness information, image phase information can usually be obtained through operations such as Gabor filtering, steerable filtering, and Fourier transform. It can maintain high accuracy in stronger noise environments and is more robust to illumination changes and other interference factors. It has been widely used in motion estimation, image matching and other fields. Among them, phase correlation technology achieves two-dimensional positioning of objects by comparing the phase spectra of two local images, while the phase optical flow method uses tracking phase contours to achieve motion estimation. Especially in the field of motion estimation, the phase optical flow method can even measure 0.003 pixel displacement (the minimum motion resolution of an 8-bit image). However, existing research has very little application in three-dimensional positioning. It is limited by the phase nonlinearity error caused by perspective projection and is difficult to directly apply to high-precision positioning tasks. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a three-dimensional positioning method for the center of a marker plate based on perspective-aligned Gabor filters and square dot arrays. In the case of perspective distortion of the square dot array marker plate image, the perspective-aligned Gabor filters can still be used to stably extract the quasi-linear phase of the pattern. Then, by detecting the zero-crossing point of the phase, the center of the square dot array pattern can be detected with an accuracy of 0.003 pixels (minimum resolution of 8-bit image, 1 / 255 pixel). Finally, combined with a pose estimation algorithm based on homography transformation, the accuracy of three-dimensional positioning is significantly improved.

[0006] To achieve the above object, the present invention provides the following technical solution: a three-dimensional positioning method for the center of a marker plate based on perspective-aligned Gabor filters and a square dot matrix, the steps of which are as follows:

[0007] (1) Designing a dot pattern and a marking plate composed of the dot pattern;

[0008] (2) Calibrate the calibration plate with the camera and take images;

[0009] (3) Two-dimensional coarse positioning of the center of the square dot matrix on the marking plate;

[0010] (4) Refine the center of the square dot matrix on the marking plate;

[0011] (5) Solve the three-dimensional coordinates of the center of the marking plate.

[0012] In some embodiments, the dot pattern in step (1) is composed of an m×m array of evenly spaced squares, the marking plate is formed by orderly combining multiple dot patterns on a single plate, and the dot array arrangement parameters are adjusted according to application requirements.

[0013] In some embodiments, the size of the squares in the dot pattern and the spacing between adjacent squares are set to a, so that the total size of the dot matrix is ​​(2m+1)a. Each is white to form a high-contrast pattern with a black background.

[0014] In some embodiments, according to step (2), the specific method is:

[0015] The camera is calibrated using a checkerboard calibration plate to determine the camera's intrinsic parameters K and distortion parameters (k1, k2, k3, p1, p2). The camera is adjusted and the marker plate is photographed to output an image I.

[0016] Establish the camera coordinate system (X C ,Y C ,Z C ), marker plate coordinate system (X B ,Y B ,Z B) and the image coordinate system (M, N); where the marker plate coordinate system (X B ,Y B ,Z B ) is fixed at the center of the marking plate, Z B Perpendicular to the plate, X B and Y B They are parallel to the edge of the board respectively, and there is a homography transformation relationship (original homography matrix H) between the camera coordinate system and the marker board coordinate system.

[0017] In some embodiments, according to step (3), the specific method is:

[0018] The coordinates of the square dot matrix in the marking board coordinate system {(X Bk ,Y Bk ,0)|k=1…n}, use the template matching algorithm to preliminarily locate the position of the 9 square dot matrix in the image I {(M k ,N k )|k=1…n}, use 9 pairs of points to construct a linear equation system, and use singular value decomposition to preliminarily solve the homography matrix H r .

[0019] In some embodiments, according to step (4), the specific method is:

[0020] The complex Gabor filter is used as the reference filter to generate the phase in the marker coordinate system, which is expressed as:

[0021]

[0022] Where: (u0, v0) represents the center frequency of the complex two-dimensional sine wave; (σ a ,σ r ) are the standard deviations of the two-dimensional Gaussian envelope in the angular and radial directions respectively. The direction and period parameters of the filter are as follows:

[0023] α=tan -1 (v0 / u0)

[0024] A pair of orthogonal filters is set to generate the phase of the marker in two directions (α1 = 0°, α2 = 90°), and the remaining parameters are set according to the actual size of the square lattice: T = 2a; 6σ a =(2m+1)a,

[0025] The complex Gabor filtering and phase extraction of the marker pattern P are performed as follows:

[0026]

[0027] The above formula is the convolution and phase generation of the filter and the marker pattern P in the marker coordinate system. The filter needs to traverse all points and perform weighted sum assignment;

[0028] The captured image I is obtained by passing the original homography matrix H through the continuous pattern P in the marker coordinate system. There is perspective distortion and the original homography matrix H is unknown. In order to convert it into the convolution in the image coordinate system under perspective distortion, the homography matrix H is initially solved. r , adjust the above convolution formula as follows:

[0029]

[0030] Where G H For the proposed perspective-aligned Gabor filter, its formula is as follows:

[0031]

[0032] Convolution under perspective distortion requires weighted summation of the filter G at each point H pass It is obtained by taking the value of Gabor filter G at the corresponding position in the marker coordinate system;

[0033] Through the above-mentioned orthogonal filter (α1 = 0°, α2 = 90°) and its corresponding perspective alignment Gabor filter, two perspective phase images are generated in the image coordinate system, which correspond to the phase images generated by the two Gabor filters in the horizontal and vertical directions in the original marker coordinate system. Based on the position of the 9 square dots in the image I {(X Ik ,Y Ik )|k=1…n}, the center of the lattice is updated by searching for the nearest zero-crossing line to the point on the two perspective phase images and finding the intersection, and then further dedistorting the lattice by combining the distortion parameters (k1, k2, k3, p1, p2) to obtain the center position of the refined 9-square lattice {(X Rk ,Y Rk )|k=1…n}.

[0034] In some embodiments, according to step (5), the specific method is:

[0035] Using the refined lattice center coordinates {(X Rk ,Y Rk )|k=1…n} and its corresponding point {(X Bk ,Y Bk ,0)|k=1…n}, solve the linear equation system containing 9 equations by singular value decomposition, and update the homography matrix to H. Given the known intrinsic parameter matrix K, determine the scaled extrinsic parameter matrix A, which is expressed as:

[0036] A=K -1 H=[a1,a2,a3]

[0037] The translation vector t can be extracted by the following formula:

[0038]

[0039] The translation vector t represents the position of the marker coordinate system in the camera coordinate system, which is the proposed three-dimensional positioning result of the marker center.

[0040] Compared with the existing technology, the beneficial effects of the present invention are: achieving two-dimensional positioning of the center of the square dot array with an accuracy of 0.003 pixels under perspective distortion, which is close to the performance upper limit supported by 8-bit images, and further combining a marker plate composed of multiple square dot arrays and homography transformation, under given conditions, achieving three-dimensional positioning with a total mean absolute error of approximately 0.01 mm, expanding the application of visual three-dimensional positioning technology in high-precision calibration, detection and measurement.

[0041] Achieving 0.003-pixel 2D positioning accuracy under perspective distortion relies primarily on a specially designed square dot pattern and perspective-aligned Gabor filters, enabling quasi-linear phase extraction under perspective distortion, with the dot center precisely located at the phase zero crossing. With the help of precise 2D dot positioning, the 3D pose of the marker using a homography matrix is ​​also more accurate than commonly used algorithms such as PnP and DLT.

[0042] Details of one or more embodiments of the present application are presented in the following drawings and descriptions to make other features, purposes and advantages of the present application more concise and easy to understand, and the present application is fully described and understood through the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a design diagram of the square dot matrix and marking plate of the present invention;

[0044] Figure 2 It is the algorithm flow chart of the present invention;

[0045] Figure 3 A diagram showing the definition of the coordinate system of the present invention;

[0046] Figure 4 Schematic diagram of the perspective-aligned Gabor filter of the present invention;

[0047] Figure 5 This is a diagram showing the positioning accuracy of the experimental verification of the present invention. DETAILED DESCRIPTION

[0048] Existing marker two-dimensional positioning technology was developed for fast and rough positioning of objects. Its two-dimensional positioning accuracy is low, limited to 0.05-0.1 pixel accuracy. Combined with the three-dimensional pose estimation algorithm, its three-dimensional positioning accuracy is still limited. The relevant marker design, feature extraction algorithm and pose estimation algorithm urgently need to be designed and optimized.

[0049] Compared to image brightness, image phase has been shown to provide more stable and reliable feature information, and applications such as motion estimation and image matching have also achieved good measurement performance. However, in applications using markers for 3D positioning, images often suffer from perspective distortion, which exacerbates phase nonlinearity and reduces positioning and measurement accuracy. Therefore, its use still requires optimization and adjustment.

[0050] This application aims to fully utilize the characteristics of image phase to achieve two-dimensional positioning of the center of the block lattice with an accuracy of 0.003 pixels under perspective distortion, and further combine it with homography transformation to expand it to high-precision three-dimensional positioning of the square lattice marker plate. It mainly solves the difficulties in quasi-linear phase extraction under perspective distortion and the problem of block lattice center detection based on non-zero phase points.

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] The present invention provides a three-dimensional positioning method for the center of a marker plate based on perspective alignment Gabor filter and square dot matrix. Figure 1 As shown, the present application first provides a dot matrix pattern and a marking plate composed of the dot matrix. The dot matrix consists of an array of evenly spaced squares (m×m). The size of the squares and the spacing between adjacent squares are set to a, so that the total size of the dot matrix is ​​(2m+1)a. Each is white to form a high-contrast pattern with a black background. The marking plate is formed by an orderly combination of multiple dot matrices on a single plate. The dot matrix arrangement parameters can be adjusted according to application requirements, and its physical dimensions need to be known for three-dimensional positioning. The implementation case adopts the following Figure 1 The parameters shown are as follows: the marking plate contains a total of 9 (n) square dot matrices (m = 6, a = 2.5 mm), the dot matrices are arranged in a nine-square grid, with a spacing of 50 mm and a side length of 150 mm.

[0053] Based on the above dot pattern and marking board, such as Figure 2 As shown, by using the square dot matrix marker center visual three-dimensional positioning method to achieve Figure 1The method accurately locates the center of a dot matrix in two dimensions and subsequently locates a marker plate in three dimensions under moderate perspective distortion. The application is to locate the end position of a parallel robotic arm. The method includes:

[0054] S1. Camera calibration and marker plate image capture

[0055] Use the checkerboard calibration plate to calibrate the camera and determine the camera's intrinsic parameter K and distortion parameters (k1, k2, k3, p1, p2). Figure 1 The square dot matrix marker plate shown in the figure has a dot matrix pattern printing accuracy of 0.01 mm and is fixed to the end of the parallel robotic arm. The lens, camera and lighting settings are optimized to capture high-quality marker plate images I. Figure 3 As shown, establish the camera coordinate system (X C ,Y C ,Z C ), marker plate coordinate system (X B ,Y B ,Z B ) and the image coordinate system (M, N). Among them, the marker coordinate system (X B ,Y B ,Z B ) is fixed at the center of the marking plate, Z B Perpendicular to the plate, X B and Y B They are parallel to the edge of the board respectively, and there is a homography transformation relationship (original homography matrix H) between the camera coordinate system and the marker board coordinate system.

[0056] S2. 2D coarse positioning of the center of the square lattice

[0057] The coordinates of the square dot matrix in the marking board coordinate system {(X Bk ,Y Bk ,0)|k=1…n}, use the template matching algorithm to preliminarily locate the position of the 9 square dot matrix in the image I {(M k ,N k )|k=1…n}, use 9 pairs of points to construct a linear equation system, and use singular value decomposition to preliminarily solve the homography matrix H r .

[0058] S3. Refinement of the center of the square lattice

[0059] This example uses a complex Gabor filter as the reference filter for generating phase in the marker coordinate system, which is expressed as:

[0060]

[0061] Where: (u0, v0) represents the center frequency of the complex two-dimensional sine wave; (σ a ,σr ) are the standard deviations of the two-dimensional Gaussian envelope in the angular and radial directions respectively. The direction and period parameters of the filter are as follows:

[0062] α=tan-(v0 / 0)

[0063] A pair of orthogonal filters is set to generate the phase of the marker in two directions (α1 = 0°, α2 = 90°), and the remaining parameters are set according to the actual size of the square lattice: T = 2a; 6σ a =(2m+1)a. This parameter setting helps to achieve quasi-linear phase extraction, the phase is more stable and the center position of the matrix can be determined by the zero crossing point. The complex Gabor filtering and phase extraction of the marker pattern P are as follows:

[0064]

[0065] like Figure 4 As shown, the above formula is the convolution and phase generation of the filter and the marker pattern P in the marker coordinate system. The filter needs to traverse all points for weighted summation assignment. However, the captured image I is obtained by the continuous pattern P in the marker coordinate system through the original homography matrix H. There is perspective distortion and the original homography matrix H is unknown. In order to convert it into the convolution in the image coordinate system under perspective distortion, combined with the preliminarily solved homography matrix H r , adjust the above convolution formula as follows:

[0066]

[0067] Where G H The perspective-aligned Gabor filter proposed in this patent has the following formula:

[0068]

[0069] Combine Figure 4 , the filter G required for weighted summation of convolution under perspective distortion at each point H Available through It is obtained by taking the value of the Gabor filter G at the corresponding position in the marker coordinate system, which reflects that the convolution under perspective distortion needs to select the Gabor filter with the corresponding distortion degree according to the distortion degree of the convolution position.

[0070] Through the above-mentioned orthogonal filter (α1 = 0°, α2 = 90°) and its corresponding perspective aligned Gabor filter, two perspective phase images can be generated in the image coordinate system, which correspond to the phase images generated by the two Gabor filters in the horizontal and vertical directions in the original marker coordinate system. Based on the position of the 9 square dots in the image I {(X Ik,Y Ik )|k=1…n}, the center of the lattice can be updated by searching for the zero-crossing line closest to the point on the two perspective phase images and finding the intersection, and then further dedistorting the lattice by combining the distortion parameters (k1, k2, k3, p1, p2) to obtain the center position of the refined 9-square lattice.

[0071] {(X Rk ,Y Rk )|k=1…n}.

[0072] S4. Solving the three-dimensional coordinates of the center of the marker plate

[0073] Using the refined lattice center coordinates {(X Rk ,Y Rk )|k=1…n} and its corresponding point {(X Bk ,Y Bk ,0)|k=1…n}, solve the linear equation system containing 9 equations by singular value decomposition, and update the homography matrix to H. Given the known intrinsic parameter matrix K, the scaled extrinsic parameter matrix A can be determined, which is expressed as:

[0074] A=K -1 H=[a1,a2,a3]

[0075] The translation vector t can be extracted by the following formula:

[0076]

[0077] The translation vector t represents the position of the marker plate coordinate system in the camera coordinate system, which is the three-dimensional positioning result of the marker plate center proposed in this patent. Figure 5 As shown, using a laser tracker and camera to simultaneously track the movement of the end of a parallel manipulator, two sets of data, each with 20 points, were obtained. The average absolute error between the two sets was approximately 0.01 mm, which basically meets the requirements for serial manipulator calibration and verifies the accuracy of the 3D positioning technology proposed in this application.

[0078] Through this technical solution, in the implementation scheme of this application, through a marking plate (150×150mm2) containing 9 square dot arrays, this method can use a single camera to achieve a three-dimensional positioning accuracy of a total average absolute error of approximately 0.01 mm at a shooting distance of 1 meter and daily indoor lighting conditions, basically meeting high-precision calibration, detection and measurement tasks.

[0079] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

[0080] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional positioning method for the center of a marker plate based on perspective-aligned Gabor filters and a square dot matrix, characterized by: The steps are: (1) Designing a dot pattern and a marking plate composed of the dot pattern; (2) Calibrate the calibration plate with the camera and take images; (3) Two-dimensional coarse positioning of the center of the square dot matrix on the marking plate; (4) Refine the center of the square dot matrix on the marking plate; (5) Solve the three-dimensional coordinates of the center of the marking plate.

2. The method for three-dimensional positioning of a marker plate center based on perspective-aligned Gabor filters and a square dot matrix according to claim 1, characterized in that: The dot pattern in step (1) is composed of an m×m array of evenly spaced squares. The marking plate is formed by an orderly combination of multiple dot patterns on a single plate. The dot array arrangement parameters are adjusted according to application requirements.

3. The method for three-dimensional positioning of a marker plate center based on perspective-aligned Gabor filters and a square dot matrix according to claim 2, characterized in that: The size of the squares in the dot pattern and the spacing between adjacent squares are both set to a, so that the total size of the dot matrix is ​​(2m+1)a. Each is white to form a high-contrast pattern with a black background.

4. The method for three-dimensional positioning of a marker plate center based on perspective-aligned Gabor filters and a square dot matrix according to claim 1, characterized in that: According to step (2), the specific method is: The camera is calibrated using a checkerboard calibration plate to determine the camera's intrinsic parameters K and distortion parameters (k1, k2, k3, p1, p2). The camera is adjusted and the marker plate is photographed to output an image I. Establish the camera coordinate system (X C ,Y C ,Z C ), marker plate coordinate system (X B ,Y B ,Z B ) and the image coordinate system (M, N); where the marker plate coordinate system (X B ,Y B ,Z B ) is fixed at the center of the marking plate, Z B Perpendicular to the plate surface, X B and Y B They are parallel to the edge of the board respectively, and there is a homography transformation relationship (original homography matrix H) between the camera coordinate system and the marker board coordinate system.

5. The method for three-dimensional positioning of the center of a marker plate based on perspective-aligned Gabor filters and a square dot matrix according to claim 4, characterized in that: According to step (3), the specific method is: The coordinates of the square dot matrix in the marking board coordinate system {(X Bk ,Y Bk ,0)|k=1…n}, use the template matching algorithm to preliminarily locate the position of the 9 square dot matrix in the image I {(M k ,N k )|k=1…n}, use 9 pairs of points to construct a linear equation system, and use singular value decomposition to preliminarily solve the homography matrix H r .

6. The method for three-dimensional positioning of the center of a marker plate based on perspective-aligned Gabor filters and a square dot matrix according to claim 5, characterized in that: According to step (4), the specific method is: The complex Gabor filter is used as the reference filter to generate the phase in the marker coordinate system, which is expressed as: Where: (u0, v0) represents the center frequency of the complex two-dimensional sine wave; (σ a ,σ r ) are the standard deviations of the two-dimensional Gaussian envelope in the angular and radial directions respectively. The direction and period parameters of the filter are as follows: A pair of orthogonal filters is set to generate the phase of the marker in two directions (α1 = 0°, α2 = 90°), and the remaining parameters are set according to the actual size of the square lattice: T = 2a; 6σ a =(2m+1)a, The complex Gabor filtering and phase extraction of the marker pattern P are performed as follows: The above formula is the convolution and phase generation of the filter and the marker pattern P in the marker coordinate system. The filter needs to traverse all points and perform weighted sum assignment; The captured image I is obtained by passing the original homography matrix H through the continuous pattern P in the marker coordinate system. There is perspective distortion and the original homography matrix H is unknown. In order to convert it into the convolution in the image coordinate system under perspective distortion, the homography matrix H is initially solved. r , adjust the above convolution formula as follows: Where G H For the proposed perspective-aligned Gabor filter, its formula is as follows: Convolution under perspective distortion requires weighted summation of the filter G at each point H pass It is obtained by taking the value of Gabor filter G at the corresponding position in the marker coordinate system; Through the above-mentioned orthogonal filter (α1 = 0°, α2 = 90°) and its corresponding perspective alignment Gabor filter, two perspective phase images are generated in the image coordinate system, which correspond to the phase images generated by the two Gabor filters in the horizontal and vertical directions in the original marker coordinate system. Based on the position of the 9 square dots in the image I {(X Ik ,Y Ik )|k=1…n}, the center of the lattice is updated by searching for the nearest zero-crossing line to the point on the two perspective phase images and finding the intersection, and then further dedistorting the lattice by combining the distortion parameters (k1, k2, k3, p1, p2) to obtain the center position of the refined 9-square lattice {(X Rk ,Y Rk )|k=1…n}.

7. The method for three-dimensional positioning of the center of a marker plate based on perspective-aligned Gabor filters and a square dot matrix according to claim 6, characterized in that: According to step (5), the specific method is: Using the refined lattice center coordinates {(X Rk ,Y Rk )|k=1…n} and its corresponding point {(X Bk ,Y Bk ,0)|k=1…n}, solve the linear equation system containing 9 equations by singular value decomposition, and update the homography matrix to H. Given the known intrinsic parameter matrix K, determine the scaled extrinsic parameter matrix A, which is expressed as: <h2 style=";text-align:left;direction:ltr">A=K<h2 style=";text-align:left;direction:ltr"> -1 <h2 style=";text-align:left;direction:ltr"> H=[a1,a2,a3] The translation vector t can be extracted by the following formula: The translation vector t represents the position of the marker coordinate system in the camera coordinate system, which is the proposed three-dimensional positioning result of the marker center.

Images