Pin center location method without edge information

Abstract

The present invention relates to a method for locating the center of a pin lacking edge information. The steps are: firstly, binocular vision calibration is performed, then two cameras are used to synchronously collect images, a Gaussian difference scale space is established for the collected images, and then scale space extreme value region detection is performed on the Gaussian difference scale space, and the center of the scale space extreme value region is matched by means of epipolar constraints to reconstruct the three-dimensional space point of the pin. Experimental results show that this method overcomes the defect that the existing method cannot locate the center of the pin in the absence of edge information, has high accuracy and good robustness, can meet the pin position positioning under various complex working conditions, and guide the robot to dock and assemble.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision, in particular to a method for locating the center of a pin lacking edge information. Background Art

[0002] The quality of installation and docking of large products, such as missiles, satellites, engines, etc., affects their safety and stability to a certain extent. Domestic docking operations basically use manual assembly and docking with V-shaped positioning supports. Its safety and quality mainly rely on the experience and skill level of the installers. The installation accuracy and stability cannot be guaranteed. Traditional methods can no longer meet the needs of the rapidly developing automated production industry.

[0003] The docking process of large products mainly involves the detection of hole positions and pins. Currently, commonly used methods include measuring devices such as total stations and laser trackers. However, the low measurement efficiency of these measurement methods seriously affects the product development cycle. Therefore, many scholars use robot visual servoing to guide assembly and match them by detecting installation holes or pins.

[0004] The commonly used method is to use the Hough circle detection method, but this method has low computational efficiency and low precision. The clustering method is used for ellipse extraction. This method clusters all ellipses according to their center positions and classifies ellipses with a certain center distance between ellipses into one category, thereby realizing screw detection. However, this method uses clustering, which reduces the efficiency of the algorithm and cannot be applied to automated production.

[0005] Wang Zhangfeng and others used the canny edge extraction algorithm to detect the pin edge in "High-precision positioning of pin edges based on machine vision". However, the working conditions in actual production operations are relatively complex. Due to the severe wear of the pins, the worn edges cause great interference to the recognition. Using the existing edge detection method to identify the edge is bound to fail to detect, resulting in failure of automated docking and assembly. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a method for locating the center of a pin lacking edge information. This method can locate the center of a pin when the pin lacks edge information and has strong robustness, high efficiency and high precision.

[0007] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a pin center positioning method lacking edge information comprises the following steps:

[0008] Step 1: Perform binocular positioning on the binocular vision sensor to obtain the intrinsic and extrinsic parameters of the camera;

[0009] Step 2: Synchronously trigger the binocular cameras to collect the original images of the same pin lacking edge information at their respective angles; perform Gaussian convolution on the original image and downsample it using different scale spaces to establish a Gaussian pyramid; subtract adjacent Gaussian pyramid images to obtain a Gaussian difference image; extract the scale-invariant extreme stable region and eliminate the error region to obtain the maximum stable region in the scale space;

[0010] Step 3: Fit the maximum stable region in the scale space to an ellipse to obtain the center of gravity of the maximum stable region in the scale space, i.e., the center of the ellipse;

[0011] Step 4: Using the conversion relationship between the calibrated binocular cameras, perform epipolar constraint matching on the centers of the fitted scale space maximum stable regions in the two images to obtain the point cloud matching point pairs (L 1 ,R 1 ),(L 2 ,R 2 ),…(L n ,R n ), where (L 1 , L 2 , …L n ) is the center point of the pin in the left image, (R 1 , R 2 , … R n ) is the center point of the pin in the right image;

[0012] Step 5: Use the triangulation principle to perform 3D reconstruction of the pin center and obtain the 3D spatial point (P 1 ,P 1 ,…,P n ).

[0013] The intrinsic parameter is the transformation matrix from the industrial camera coordinate system to the image coordinate system, and the extrinsic parameter is the transformation matrix between cameras.

[0014] A two-dimensional flexible target is used to calibrate the intrinsic parameters of each industrial camera.

[0015] The external parameters are realized by the rotation matrix R and the translation matrix T.

[0016] The method of using different scale spaces to perform Gaussian convolution on the original image and downsample the image to establish a Gaussian pyramid includes:

[0017] Step 2.1: Define the scale space image as L(x,y,σ):

[0018]

[0019] Among them, G(x,y,σ) is a Gaussian function with a mean square error of σ, that is, Indicates that the image is convolved in the x and y directions, and I(x,y) is the original image;

[0020] Step 2.2: Using different scales (0, σ, kσ, k 2 σ,k 3 σ......k (L-2) σ) Perform Gaussian convolution on the original image I(x,y) to obtain the first layer image sequence of the Gaussian pyramid;

[0021] Then, the last image in the first layer image sequence is downsampled to obtain the first image in the second layer image sequence;

[0022] Using different scales (0, 2σ, 2kσ, 2k 2 σ,2k 3 σ......2k (L-2) σ) Perform Gaussian convolution on the first image of the second layer image sequence to form the second layer image sequence of the Gaussian pyramid;

[0023] Then, downsampling is performed on the last image in the second layer image sequence to form the first image in the third layer image sequence;

[0024] Using different scales (0, 4σ, 4kσ, 4k 2 σ,4k 3 σ......4k (L-2) σ) performing Gaussian convolution on the first image of the third layer image sequence to form the third layer image sequence of the Gaussian pyramid;

[0025] Repeat the above Gaussian and downsampling operations to build images at each layer of the Gaussian pyramid.

[0026] The method of obtaining a Gaussian difference image by subtracting adjacent Gaussian pyramid images includes:

[0027] Step 2.3: Subtract the adjacent Gaussian pyramid images of the current layer to obtain the Gaussian difference pyramid image. The Gaussian difference image can be expressed as:

[0028] D(x,y,k q-1 σ)=L(x,y,k q σ)-L(x,y,k q-1 σ)

[0029] Where k is the scale space multiplication factor, L(x,y,k q σ) is the mean square error k q The scale space image L(x,y,k q-1 σ) is the mean square error kq-1 The scale space image of the convolution of the Gaussian function of σ and the image, q = 1, 2, ... L-1.

[0030] The step of extracting the scale-invariant extreme stable region and eliminating the error region to obtain the maximum stable region in the scale space includes:

[0031] Extract scale-invariant extreme value stable regions from the image difference Gaussian pyramid. The scale-invariant extreme value stable regions refer to regions with maximum stable extreme values ​​in adjacent images in the Gaussian difference scale space of the image:

[0032] S=D(x,y,k q-1 σ)∩D(x,y,k q σ)∩D(x,y,k q+1 σ)

[0033] Among them, S is the maximum stable extreme value area in the kσ scale space, D(x,y,k q-1 σ), D(x,y,k q σ), D(x,y,k q +1 σ) are the mean square error k q-1 σ, k q σ, k q+1 Gaussian difference pyramid image under σ, q = 1, 2, ... L-2;

[0034] Since there are overlapping similar regions, it is necessary to screen the similar regions and finally obtain the precise maximum stable region in scale space, as shown below:

[0035]

[0036] Among them, R is the area after filtering out overlapping similarities, R i ,R j are the i-th and j-th regions of the scale-invariant extremum stable region respectively.

[0037] The images captured by the camera 1 and the camera 2 are images of the same pin at different viewing angles.

[0038] The present invention has the following beneficial effects and advantages:

[0039] 1. The pin center positioning method lacking edge information described in the present invention can position the pin center when the image lacks the pin edge information.

[0040] 2. The pin center positioning method lacking edge information described in the present invention can be applied to automated production to shorten the product development cycle.

[0041] 3. The pin center positioning method lacking edge information described in the present invention can provide effective guidance information for the docking and assembly of large products with high accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the method of the present invention;

[0043] Figure 2 A Gaussian pyramid diagram of the method of the present invention;

[0044] Figure 3 This is a center positioning diagram of pin images at multiple positions taken by the left and right cameras according to the method of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0046] See attached Figure 1 , Attachment Figure 2 , Attachment Figure 3 The specific steps of the pin center positioning method without edge information of the present invention are as follows:

[0047] Step 1: Perform binocular positioning on the binocular vision sensor device to obtain the camera's intrinsic parameter matrix and extrinsic parameters;

[0048] The intrinsic parameters represent the transformation matrix from the industrial camera coordinate system to the image coordinate system. The degree of distortion of each lens is different, including radial distortion and eccentric distortion. The intrinsic parameters of each industrial camera are calibrated using a two-dimensional flexible target.

[0049] The external parameters are the conversion matrix between cameras, which represents the transformation matrix from the industrial camera coordinate system to the world coordinate system, which is realized by the rotation matrix R and the translation matrix T. The world coordinate system is used for the subsequent output of the actual control point coordinates.

[0050] Step 2: Synchronously trigger the binocular camera to collect images, and process each original image according to the following steps 2.1 to 2.5;

[0051] Step 2.1: Establish the image scale space and define the scale space image as L(x,y,σ):

[0052]

[0053] Among them, G(x,y,σ) is a Gaussian function with a mean square error of σ, that is, It means that the image is convolved in the x and y directions, and I(x,y) is the original image.

[0054] Step 2.2: Using different scales (0, σ, kσ, k2 σ,k 3 σ......k (L-2) σ) performs Gaussian convolution on the image to form the first layer image sequence of the Gaussian pyramid, and then downsamples the last image in the first layer image sequence to form the first image of the second layer image sequence. 2 σ,2k 3 σ......2k (L -2) σ) performs Gaussian convolution on the first image of the second layer image sequence to form the second layer image sequence of the Gaussian pyramid, and then downsamples the last image in the second layer image sequence to form the first image of the third layer image sequence. 2 σ,4k 3 σ......4k (L-2) σ) Perform Gaussian convolution on the first image of the third layer image sequence to form the third layer image sequence of the Gaussian pyramid. Set the iteration cutoff number according to the resolution of the output image (calculated to the 5th layer based on experience in the experiment), and repeat the above Gaussian and downsampling operations to build the images of each layer of the Gaussian pyramid. Where L is the number of images in the current layer.

[0055] Step 2.3: Then, the Gaussian difference pyramid image is obtained by subtracting the adjacent Gaussian pyramid images of the current layer. The Gaussian difference image can be expressed as:

[0056] D(x,y,k q-1 σ)=L(x,y,k q σ)-L(x,y,k q-1 σ)

[0057] Where k is the scale space multiplication factor, L(x,y,k q σ) is the mean square error k q The scale space image L(x,y,k q-1 σ) is the mean square error k q-1 The scale space image of the convolution of the Gaussian function of σ and the image, q = 1, 2, ... L-1.

[0058] Step 2.4: Extract the scale-invariant extreme value stable region from the image difference Gaussian pyramid, where the scale-invariant extreme value stable region refers to the region with the maximum stable extreme value in the adjacent images in the Gaussian difference scale space of the image:

[0059] S=D(x,y,k q-1 σ)∩D(x,y,k qσ)∩D(x,y,k q+1 σ)

[0060] Among them, S is the maximum stable extreme value region in the kσ scale space (in a scale space, many maximum stable extreme value regions of different sizes and positions can be found as long as the conditions of the formula are met), D(x,y,k q-1 σ), D(x,y,k q σ), D(x,y,k q+1 σ) are the mean square error k q-1 σ, k q σ, k q+1 Gaussian difference pyramid image under σ, q = 1, 2, ... L-2. The maximum stable extreme value region is represented by the image where the area change is the smallest within a certain grayscale threshold range.

[0061] Step 2.5: Since there are overlapping similar regions, it is necessary to select similar regions and finally obtain accurate scale-space extremely stable regions. The selection principles are as follows:

[0062]

[0063] Among them, R is the area after filtering out the nested overlapping similarity, R i ,R j are the i-th and j-th regions of the scale-invariant extremum stable region respectively.

[0064] Step 3: Perform an ellipse fitting on the scale-invariant extreme value stable region to obtain the center of gravity of the scale-invariant extreme value stable region, that is, the center of the ellipse.

[0065] Step 4: Using the conversion relationship between the calibrated binocular cameras, perform epipolar constraint matching on the centers of the fitted scale space maximum stable regions in the two images to obtain the point cloud matching point pairs (L 1 ,R 1 ),(L 2 ,R 2 ),…(L n ,R n ), where (L 1 , L 2 , …L n ) is the center point of the pin in the left image, (R 1 , R 2 , … R n ) is the center point of the pin in the right image.

[0066] Step 5: Use the triangulation principle to perform 3D reconstruction of the pin center and obtain the 3D spatial point (P 1 ,P 2 ,…,Pn ). The specific implementation method is as follows:

[0067]

[0068] Among them, (x i ,y i ,z i ) is the reconstructed three-dimensional space coordinate point P i The coordinates of (X 1i ,Y 1i ) is the coordinate of the center point of the scale extreme stable region under the left camera, (X 2i ,Y 2i ) is the right camera down and (X 1i ,Y 1i ) corresponds to the coordinates of the center point of the scale extreme stability region, f m 、f n are the focal ratios of the left and right cameras, and the transformation matrix R between the left and right cameras 12 It can be expressed as

[0069]

[0070] Among them, (r1, r2, r3, r4, r5, r6, r7, r8, r9, t x ,t y ,t z ) is the transformation matrix R 12 Parameter value.

[0071] The following is combined with Figure 3 The effects of the present invention are further described. Figure 3 In the figure, (a), (c), and (e) are the positioning results of the images taken by the left camera, and (b), (d), and (f) are the positioning results of the images taken by the right camera.

[0072] In order to verify the effectiveness and correctness of the method of the present invention, the manta series camera of AVT Company was used, and the camera was equipped with 8mm precision lens of Computar Company. All simulation experiments were implemented in VisualStudio 2013 software under Windows 7 operating system. Figure 3 As shown in the figure, the cross-line marked in the figure is the center position of the pin to be detected. The method of the present invention measures the pins under different conditions, and the measurement accuracy can reach 0.15mm. The measurement accuracy is high, the efficiency is high, and the stability is strong.

[0073] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should be regarded as within the scope of protection of the present invention.

Claims

1. A pin center positioning method lacking edge information, characterized in that: The following steps are involved: Step 1: Perform binocular positioning on the binocular vision sensor to obtain the intrinsic and extrinsic parameters of the camera; Step 2: 2.1) Synchronously trigger the binocular cameras to collect original images of the same pin lacking edge information at respective angles; 2.2) Perform Gaussian convolution on the original image using different scale spaces and downsample it to create a Gaussian pyramid; 2.3) Subtract adjacent Gaussian pyramid images to obtain a Gaussian difference image; 2.4) Extract the scale-invariant extreme stable region and remove the error region to obtain the maximum stable region in scale space; Among them, 2.2) includes: a: Define the scale space image as L(x, y, σ): Among them, G(x, y, σ) is a Gaussian function with a mean square error of σ, that is, Indicates that the image is convolved in the x and y directions, and I(x, y) is the original image; b: Using different scales (0, σ, kσ, k 2 σ, k 3 σ......k (L-2) σ) Perform Gaussian convolution on the original image I(x, y) to obtain the first layer image sequence of the Gaussian pyramid; Then, the last image in the first layer image sequence is downsampled to obtain the first image in the second layer image sequence; Using different scales (0, 2σ, 2kσ, 2k 2 σ, 2k 3 σ......2k (L-2) σ) Perform Gaussian convolution on the first image of the second layer image sequence to form the second layer image sequence of the Gaussian pyramid; Then, downsampling is performed on the last image in the second layer image sequence to form the first image in the third layer image sequence; Using different scales (0, 4σ, 4kσ, 4k 2 σ, 4k 3 σ......4k (L-2) σ) performing Gaussian convolution on the first image of the third layer image sequence to form a third layer image sequence of the Gaussian pyramid; c: Repeat the above Gaussian and downsampling operations to build images at each layer of the Gaussian pyramid; Among them, 2.3) includes: The Gaussian difference pyramid image is obtained by subtracting the adjacent Gaussian pyramid images of the current layer. The Gaussian difference image can be expressed as: D(x,y,k q-1 σ)=L(x,y,k q σ)-L(x,y,k q-1 p) Where k is the scale space multiplication factor, L(x, y, k q σ) is the mean square error k q The scale space image L(x, y, k) is the Gaussian function of σ and the image convolved. q-1 σ) is the mean square error k q-1 The scale space image of the convolution of the Gaussian function of σ and the image, q = 1, 2, ... L-1, in the experiment, Among them, 2.4) includes: Extract scale-invariant extreme value stable regions from the image difference Gaussian pyramid. The scale-invariant extreme value stable regions refer to regions with maximum stable extreme values ​​in adjacent images in the Gaussian difference scale space of the image: S=D(x,y,k q-1 σ)∩D(x,y,k q σ)∩D(x,y,k q+1 p) Among them, s is the maximum stable extreme value region in the kσ scale space, D(x, y, k q-1 σ), D(x, y, k q σ), D(x, y, k q+1 σ) are the mean square error k q-1 σ, k q σ, k q+1 Gaussian difference pyramid image under σ, q = 1, 2, ... L-2; Since there are overlapping similar regions, it is necessary to screen the similar regions and finally obtain the precise maximum stable region in scale space, as shown below: Among them, R is the area after filtering out overlapping similarities, R i , R j are the i-th and j-th regions of the scale-invariant extremum stable region, respectively; Step 3: Fit the maximum stable region in the scale space to an ellipse to obtain the center of gravity of the maximum stable region in the scale space, i.e., the center of the ellipse; Step 4: Using the conversion relationship between the calibrated binocular cameras, perform epipolar constraint matching on the centers of the fitted scale space maximum stable regions in the two images to obtain the point cloud matching point pairs (L 1 , R 1 ), (L 2 , R 2 ),...(L n , R n ), where (L 1 , L 2 , ...L n ) is the center point of the pin in the left image, (R 1 , R 2 ,...R n ) is the center point of the pin in the right image; Step 5: Use the triangulation principle to reconstruct the pin center in three dimensions and obtain the pin's three-dimensional space point (P 1 ,P 1 ,…,P n ).

2. The pin center positioning method without edge information according to claim 1, characterized in that: The intrinsic parameter is the transformation matrix from the industrial camera coordinate system to the image coordinate system, and the extrinsic parameter is the transformation matrix between cameras.

3. The pin center positioning method without edge information according to claim 1 or 2, characterized in that: A two-dimensional flexible target is used to calibrate the intrinsic parameters of each industrial camera.

4. The pin center positioning method lacking edge information according to claim 1 or 2, characterized in that: The external parameters are realized by the rotation matrix R and the translation matrix T.

5. The pin center positioning method lacking edge information according to claim 2, characterized in that: The images taken by camera 1 and camera 2 are images of the same pin from different perspectives.

Images