Planar array shaft hole feature multi-view matching method based on contour detection

Abstract

The invention discloses a plane array shaft hole feature multi-view matching method based on contour detection. The method comprises the following steps: S1, acquiring an image from multiple angles; s2, extracting surface contour features of each image axis and fitting a contour center point; s3, homographic transformation is carried out according to the contour center points to obtain a matching sequence; and S4, adjusting the sequence of the fitting points in the original space to realize multi-view matching. The invention provides a shaft hole feature double-view matching method based on contour detection, which has invariance of rotation, illumination, zooming and the like and provides powerful guarantee for accuracy and stability of matched feature points.

Description

Technical Field

[0001] The present invention relates to the technical field of image extraction, and particularly to a multi-view matching method for planar array shaft hole features based on contour detection. Background Art

[0002] Electronic equipment has extremely wide applications in modern society. Taking radar as an example, the typical planar array shaft hole features of its internal T / R components, frequency conversion units, and digital transceiver modules are characterized by high density, small size, and high precision. At present, the assembly of planar array shaft holes generally relies on manual assembly, and the assembly quality and efficiency are usually low. However, the automated assembly based on machine vision positioning methods can significantly improve the assembly quality and efficiency of planar array shaft hole features.

[0003] With the progress of artificial intelligence, machine vision systems have been applied to the shaft hole assembly tasks in the field of automated product assembly, greatly helping workers and the industry improve the shaft hole assembly efficiency. It has become an inevitable trend to use machine vision to replace humans to complete tasks such as shaft hole assembly. Among them, the two-view matching method is a key method for machine vision systems to achieve three-dimensional information reconstruction. This method can obtain disparity information through the images in the two views and geometrically describe it through the projective disparity relative to the reference plane. This technology can be applied in object recognition, three-dimensional reconstruction, robot navigation, shaft hole assembly, etc.

[0004] Traditional feature point-based methods (such as SIFT, SURF, etc.) often need to extract key feature points in the image. However, due to the possible changes of feature points in the image under the influence of perspective changes, illumination changes, noise, etc., the matching results are unstable. In addition, there may be problems of matching ambiguity for some feature points, affecting the accuracy of the final matching. A relatively new method based on epipolar constraint matching effectively improves the matching accuracy and efficiency of image feature points. However, this method is not applicable to the matching of small-size and high-density feature points because small-size and high-density feature points are more likely to be located on different epipolar lines, resulting in matching errors. Summary of the Invention

[0005] Based on the technical problems existing in the background art, the present invention proposes a multi-view matching method for planar array shaft hole features based on contour detection, providing a shaft hole feature two-view matching method based on contour detection that has invariance to rotation, illumination, scaling, etc., and provides strong guarantees for the accuracy and stability of matching feature points.

[0006] The multi-view matching method for planar array shaft hole features based on contour detection proposed by the present invention has the following method steps: S1: Obtain images from multiple angles; S2: Extract the contour features of the shaft surface of each image and fit the contour center points; S3: Perform a homography transformation based on the contour center point to obtain the matching order; S4: Adjust the order of the fitting points in the original space to achieve multi-view matching.

[0007] Preferably, in S2, the OpenCv algorithm is used to extract the contour features of the shaft surface, and the center point of each contour is fitted according to the OpenCv algorithm.

[0008] Preferably, the method steps for fitting the contour center point are as follows: S21: Perform grayscale processing on the image, and complete Gaussian filtering and global binarization of the image; S22: Perform the superposition processing of opening operation and closing operation; S23: Perform contour detection, and complete the screening of the contour according to the contour area and contour perimeter; S24: Use the OpenCv algorithm again to fit the center point of each contour.

[0009] Preferably, the formula for the homography transformation in S3 is as follows: In the formula, and represent the rotation matrix and the intrinsic parameter matrix of the camera respectively, represents the coordinates of the optical centers of the two cameras in the world coordinate system, and represent a point on the induced plane and its normal vector.

[0010] Preferably, in S4, the one-to-one correspondence between the contour and the fitted center point is used to correct the order of the set of contour fitted center points before transformation by judging whether the transformed point is within the transformed contour.

[0011] The beneficial technical effects of the present invention: The matching method of the present invention first takes pictures through a camera and inputs them into a computer, processes the input image and extracts the contour features and center fitting points of the shaft hole, uses the principle of the relative position invariance of the feature points to perform a homography transformation on the right view to obtain the contour order, combines the transformed contour order to adjust the order of the fitting points in the original space, and finally matches the contour center fitting points of the two views according to the adjusted fitting point order. The matching method avoids the deficiencies of traditional feature point detection schemes that need to directly extract feature points, etc., uses contour detection, can accurately obtain the matching order, and complete the two-view matching; and this matching method can effectively improve the accuracy and accuracy of image matching in the case of plane array shaft hole features with high density, small size and high precision, and can effectively improve the assembly quality and assembly efficiency of plane array shaft hole features. Description of the Drawings

[0012] Figure 1 Flow chart of the multi-view matching method for planar array shaft hole features based on contour detection proposed by the present invention; Figure 2 Schematic diagram of homography transformation proposed by the present invention; where (a) is the schematic diagram of homography transformation induced by a plane, and (b) is the schematic diagram of pure rotation homography transformation; Figure 3 Diagram showing the process of contour feature extraction proposed by the present invention; where (a) is the original image, (b) is the image after filtering and binarization of the original image, (c) is the image after morphological processing, (d) is the image after screening only by contour area, (e) is the image after screening only by contour perimeter, and (f) is the image after screening by both contour area and contour perimeter; Figure 4 Schematic diagram of contour imaging under two-view geometry proposed by the present invention; where (a) is the original image, (b) is the image after magnification containing three shaft contours, and (c) is the schematic diagram of left and right camera shooting; Figure 5 The figure shows the two-view matching result proposed by the present invention. Detailed implementation manners

[0013] The present invention will be further explained below with reference to specific embodiments.

[0014] Refer to Figure 1 , in order to achieve multi-view matching of shaft hole features based on contour detection, in the first step, two industrial cameras are fixed above the shaft of the object bottom plate, ensuring that the two cameras can cover the object within their overlapping fields of view, which requires that the two cameras are not too far apart; at the same time, considering the measurement accuracy of the binocular camera, it is required that the two cameras are not too close, otherwise the z space covered by a single pixel is too large, affecting the three-dimensional reconstruction accuracy. The left and right images are collected by the left and right cameras and then input into the computer.

[0015] In the second step, the input is the left and right two images. Each image uses the OpenCv algorithm to extract the shaft surface contour features. Refer to Figure 3 , Figure 3 . The (a) part of Figure 3 is the original image taken by the camera and input into the computer. It is grayscale processed, and Gaussian filtering and global binarization of the image are completed to obtain Figure 3 . The (b) part of Figure 3 is then obtained by superimposing opening operation and closing operation, aiming to reduce the number of contours in the view and reduce the processing difficulty of subsequent algorithms. Then contour detection is performed, and the contours are screened separately according to the contour area size to obtain Figure 3Part (e) is finally filtered by the contour perimeter and area together to obtain Figure 3 Part (f); then the center points of each contour are fitted again using the OpenCv algorithm, and these center points are saved.

[0016] The third step is as Figure 4 shown in part (c). The matching order of the left view and the right view in the figure is obviously wrong. The correct matches are 1 in the left view with 2 in the right view, and 2 in the left view with 1 in the right view. Therefore, the order in the right view needs to be adjusted to be the same as that in the left view.

[0017] According to the principle of invariant relative position of feature points, a homography transformation that only rotates without changing the relative position of the lowest point of the contour is performed on the right view to ensure that the optical axes of the two cameras are parallel. is the internal parameter matrix after the homography transformation of the right camera. The parameters can be set arbitrarily and do not involve rotation, so they do not affect the contour order. is the rotation matrix between the cameras, obtained through binocular camera calibration, which destroys the relative position of the contours before and after the right pixel transformation; is the internal parameter matrix of the right camera, and the parameters are non-adjustable and are the results of camera calibration. Therefore, adjusting makes the pixels of the right view not lost after transformation, that is, ensures that the journal contour information is not lost. After epipolar rectification, the lowest points of the corresponding contours in the two views are on the same horizontal line, and the contour matching order can be obtained.

[0018] The order of the fitted point sets of the contours after the transformation of the right view is correct, that is, the matching is correct. However, directly reconstructing the fitted results after the transformation, due to the deflection of the right camera, the imaging of the space on the plane is compressed, which affects the accuracy of the 3D reconstruction of the feature points. Therefore, only the detection order of the contours after the transformation of the right view is used as the matching order and does not participate in the reconstruction after matching.

[0019] The principle of invariant relative position of feature points in the third step refers to the principle that the relative positions of any points in the upper, lower, left, and right directions in the spatial plane remain unchanged under certain transformation rules. Taking the homography transformation between two pixel coordinate systems with an induced plane in the binocular camera model as an example, when the two optical axes are parallel and there is no relative rotation, the line connecting the optical centers is perpendicular to the optical axes, and the internal parameters of the two camera models are the same, the projections of the feature point sets on the induced plane in the two pixel coordinate systems maintain their relative positions unchanged.

[0020] The homography transformation in the third step belongs to a linear transformation and has the transitivity of a linear transformation. The homography transformation induced by the plane is shown as follows: where , and , represent the rotation matrix and the intrinsic matrix of the left and right cameras respectively. , represent the coordinates of the optical centers of the two cameras in the world coordinate system. and represent a point on the induced plane and its normal vector. represents the identity matrix. The spatial transformation relationship is as shown in part (a) of Figure 2 .

[0021] The pure rotation homography transformation is the homography transformation generated when the optical centers coincide under the spatial plane induced homography, that is, there is only rotation and no translation between the two planes. The main function of the pure rotation homography transformation is to perform epipolar correction on the plane, so that the two images of the binocular camera are located on the same plane, and the corresponding points have the same ordinate. Taking the left camera as the main camera, the pure rotation homography transformation simplified from the above formula is as follows: where represents the rotation matrix between the two cameras, , represent the intrinsic matrices of the left and right cameras respectively. The spatial transformation relationship is as shown in part (b) of Figure 2 .

[0022] In the fourth step, perform the same pure rotation homography transformation on the fitted points of the right image before transformation. Using the principle that the points are within the contour, correct the order of the fitted points after transformation with the order of the contour after transformation, and adjust the order of the fitted points before transformation to the corrected order of the fitted points. For example, the point with the order of 2 in the fitted points of the right image before transformation falls within the contour with the order of 1 after the homography transformation of the contour after the pure rotation homography transformation, and then adjust the order of the point with the order of 2 in the fitted points of the right image before transformation to 1. In this way, the order of the center points in the right view can be adjusted to be the same as the order of the center points in the left view.

[0023] In the fifth step, match the sets of the fitted points of the contour centers of the two views in the corrected order.

[0024] Figure 5 The figure shows the result of the two-view matching for the bottom plate with 60 axes. All the surface contours of the 60 axes in the left and right views are completely extracted, and the center points are fitted at the same time. The dotted lines indicate the mutual matching of the center points of the left and right views. The matching results of the left and right views in the figure are all accurate and error-free, and they correspond one by one.

[0025] Although embodiments of the present application have been shown and described, those of ordinary skill in the art will understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present application. The scope of the present application is defined by the appended claims and their equivalents, and all of them should be included within the protection scope of the present application.

Claims

1. A multi-view matching method for planar array shaft hole features based on contour detection, characterized in that: The steps are as follows: S1: Acquire images from multiple angles; S2: Extract the surface contour features of each image axis and fit the contour center point; S3: Perform homography transformation according to the contour center point to obtain the matching order; S4: Adjust the order of fitting points in the original space. S5: Match the multi-view contour center fitting point sets according to the corrected sequence numbers.

2. The multi-view matching method for planar array shaft hole features based on contour detection according to claim 1 is characterized in that: In S2, the OpenCv algorithm is used to extract the surface contour features of the shaft, and the center point of each contour is fitted according to the OpenCv algorithm.

3. The multi-view matching method for planar array shaft hole features based on contour detection according to claim 1 is characterized in that: The steps for fitting the center point of the contour are as follows: S21: grayscale processing is performed on the image, and Gaussian filtering and global binarization of the image are completed; S22: performing superposition processing of opening operation and closing operation; S23: Perform contour detection and complete contour screening according to contour area and contour perimeter; S24: Use the OpenCV algorithm again to fit the center point of each contour.

4. The multi-view matching method for planar array shaft hole features based on contour detection according to claim 1 is characterized in that: The formula for the homography transformation induced by the plane in S3 is as follows: In the formula, , and , Represent the rotation matrix and intrinsic parameter matrix of the left and right cameras respectively, , represents the coordinates of the optical centers of the two cameras in the world coordinate system, and represents a point on the induced plane and its normal vector, Represents the identity matrix.

5. The multi-view matching method for planar array shaft hole features based on contour detection according to claim 1 is characterized in that: The formula for pure rotation homography transformation in S3 is as follows: In the formula, represents the rotation matrix between the two cameras, , Represent the intrinsic parameter matrices of the left and right cameras respectively.

6. The multi-view matching method for planar array shaft hole features based on contour detection according to claim 1 is characterized in that: In S4, the one-to-one correspondence between the contour and the fitting center point is used to correct the order of the contour fitting center point set before the transformation by judging whether the transformed point is within the transformed contour.

Images