Object pose measurement method and device based on binocular vision in complex environment

Abstract

The invention provides an object pose measurement method and device based on binocular vision in a complex environment, and the method comprises the steps: obtaining two images, shot by cameras at two angles, of a measured object, and carrying out the camera calibration, stereo correction and filtering of the two images; carrying out closed operation on the two filtered images, and smoothing the edges of the images; for the two smoothed images, identifying external contour features of the measured object in the images through a Sobel algorithm; and carrying out stereo matching and pose calculation according to the external contour features in the two obtained images, and obtaining the object pose of the measured object. Image filtering is carried out in the edge extraction step, noise interference is reduced, and the method can be suitable for object pose measurement in a complex environment; image edge smoothing processing is carried out through closed operation in edge extraction, original features of an object are restored as much as possible, and the measurement accuracy is improved; image contour features are extracted through an edge detection algorithm in edge extraction, and image redundant feature information is reduced; by adding an edge extraction step, the measurement speed is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of optoelectronic detection and countermeasure systems, and particularly relates to a method and device for measuring the object pose based on binocular vision in a complex environment. Background Art

[0002] Pose measurement methods include active and passive methods.

[0003] The active method sends a specific signal to the target object, then receives the feedback of the target object on the transmitted signal, and analyzes the feedback signal to obtain depth information. The common one is the lidar measurement method.

[0004] The passive method collects the target object image through a camera, extracts feature information from the image, obtains depth information to judge the three-dimensional coordinates of the target object. The common ones are monocular and binocular measurement methods.

[0005] The existing pose measurement methods are greatly affected by the environment, and the measurement accuracy is greatly reduced and the credibility is significantly decreased in a complex environment. Summary of the Invention

[0006] The present invention provides a method and device for measuring the object pose based on binocular vision in a complex environment, which can improve the accuracy and speed of pose measurement in a complex environment.

[0007] The first aspect of the present invention provides a method for measuring the object pose based on binocular vision in a complex environment, including:

[0008] Obtain two images of the object to be measured taken by cameras at two angles, and perform camera calibration, stereo rectification and filtering on the two images;

[0009] Perform closing operation on the two filtered images to smooth the image edges;

[0010] Identify the external contour features of the object to be measured in the two smoothed images through the Sobel algorithm;

[0011] Perform stereo matching and pose calculation according to the external contour features obtained from the two images, and obtain the object pose of the object to be measured.

[0012] Optionally, the camera calibration is completed using the Zhang Zhengyou calibration method for binocular cameras.

[0013] Optionally, the stereo rectification adopts the epipolar rectification method, and uses the binocular camera calibration result to rectify the two non-coplanar and non-line-aligned images to eliminate the error caused by the non-parallel camera optical axes.

[0014] Optionally, the filtering adopts the Gaussian filtering method.

[0015] The second aspect of the present invention provides an object pose measurement device based on binocular vision in a complex environment, including:

[0016] A preprocessing module, configured to obtain two images of the object to be measured captured by cameras at two angles, and perform camera calibration, stereo rectification, and filtering on the two images;

[0017] A closing operation module, configured to perform a closing operation on the two filtered images to smooth the image edges;

[0018] An edge extraction module, configured to identify the external contour features of the object to be measured in the images through the Sobel algorithm for the two smoothed images;

[0019] A post-processing module, configured to perform stereo matching and pose calculation based on the external contour features obtained from the two images, and obtain the object pose of the object to be measured.

[0020] Optionally, the preprocessing module is specifically configured to complete binocular camera calibration using the Zhang Zhengyou calibration method.

[0021] Optionally, the preprocessing module is specifically configured to adopt the epipolar rectification method, and use the binocular camera calibration result to rectify the two non-coplanar and non-linearly aligned images to eliminate the error caused by the non-parallel camera optical axes.

[0022] Optionally, the preprocessing module is specifically configured to adopt the Gaussian filtering method.

[0023] The present invention provides an object pose measurement method and device based on binocular vision in a complex environment. The main measure is to add an edge extraction step before the stereo matching algorithm. Edge extraction removes discrete noise points in the image through filtering to reduce the interference of image background noise; then a closing operation is performed on the image to smooth the image edges; finally, the main external contour features of the object to be measured are identified through the Sobel algorithm to reduce redundant information in the image. By performing stereo matching and pose calculation after the edge extraction algorithm in the present invention, the accuracy and speed of pose measurement in a complex environment can be improved. Description of the Drawings

[0024] Figure 1 For reflecting the physical meanings of the various parameters in the pixel three-dimensional coordinate calculation formula;

[0025] Figure 2 It is the calculation flow chart of the binocular pose measurement provided by the present invention. Detailed Embodiments

[0026] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0027] As Figure 1 and Figure 2 shown, the present invention mainly solves the problems of the accuracy and speed of object pose measurement by a binocular camera in a complex environment. The conventional measurement steps are camera calibration, stereo rectification, stereo matching, and pose calculation. The main measure of the present invention is to add an edge extraction step before the stereo matching algorithm. Edge extraction removes discrete noise points in the image through filtering to reduce the interference of image background noise; then performs a closing operation on the image to smooth the image edges; finally, identifies the main external contour features of the object to be measured through the Sobel algorithm to reduce redundant information in the image. After performing stereo matching and pose calculation through the edge extraction algorithm in the present invention, the accuracy and speed of pose measurement in a complex environment can be improved.

[0028] Pose measurement includes five steps: camera calibration, stereo rectification, edge extraction, stereo matching, and pose calculation. Camera calibration obtains the internal parameters and distortion parameters of each camera, as well as the relative position between the left and right cameras, to prepare for image measurement. Stereo rectification corrects cameras with non-parallel optical axes and non-coplanar imaging in reality to complete coplanarity and alignment. Edge detection extracts the edge feature points of the image through filtering, closing operation, and Sobel algorithm. Binocular stereo matching can obtain a disparity map by finding corresponding pixel points on the images of the left and right cameras. Finally, depth information is calculated based on the disparity to complete the calculation of spatial coordinates.

[0029] The calibration uses the Zhang Zhengyou calibration method to complete the binocular camera calibration.

[0030] The rectification adopts the epipolar rectification method. Using the binocular camera calibration result, two non-coplanar and non-aligned images are rectified to eliminate the error caused by the non-parallel optical axes of the cameras.

[0031] For the edge extraction, to reduce the interference of environmental noise and extract the key edge features of the image, first perform filtering, then perform internal filling and edge smoothing operations on the image, and finally extract the image contour features through an edge detection algorithm. It includes 3 steps, namely filtering, closing operation, and edge detection.

[0032] In the edge extraction step, Gaussian filtering is used for filtering to remove high-frequency noise in the image and reduce the interference of background noise. It is calculated according to the following formula, where k = 3, x represents the abscissa of the central pixel, y represents the ordinate of the central pixel, i represents the distance from the abscissa of the central pixel, j represents the distance from the ordinate of the central pixel, I represents the original image, σ is the standard deviation, π is the ratio of the circumference of a circle to its diameter, e is the exponential constant, and I represents the original image, I SMOOTHED is the filtered image.

[0033]

[0034] In the closing operation in the edge extraction step, the discontinuous parts of the image are filled and the image edges are smoothed. It is carried out according to the following formula, where A is the structuring element, and I 闭运算 represents the image after the closing operation.

[0035] I 闭运算 = erosion(dilation(I SMOOTHED , A), A)

[0036] In the edge detection method in the edge extraction step, the Sobel method is adopted. The 3×3 horizontal and vertical edge detection operators are defined to calculate the horizontal gradient G x and the vertical gradient G y of each pixel. Then, the pixel gradient is calculated according to the following formula. G represents the gradient value, which is compared with the set threshold. If it is greater than the threshold, it is an edge point.

[0037]

[0038] In the stereo matching algorithm, first, preprocessing is carried out to normalize the image brightness and strengthen the image texture at the same time. Then, the SAD window is slid along the horizontal epipolar line to search for matching points. Finally, filtering is carried out to delete the mis-matched pixel points. After finding the matching points, the disparity is calculated to obtain the disparity map.

[0039] For the pose calculation, according to the disparity map, the three-dimensional coordinates corresponding to each pixel are obtained according to the following formula. Among them, x L is the x coordinate of the pixel in the left image, y L is the y coordinate of the pixel in the left image, b is the binocular camera spacing, d is the disparity between the left and right images, and f is the camera focal length.

[0040]

[0041] The method provided by the present invention has the following advantages:

[0042] 1. In a complex environment, in order to reduce the interference of background noise on the matching of the target object and improve the accuracy of pose measurement, after stereo rectification of the two images obtained by binocular vision, the images need to be filtered. After stereo rectification, Gaussian filtering is performed on the two images obtained by binocular vision, reducing noise interference and being applicable to the pose measurement of objects in complex environments;

[0043] 2. During the filtering process, in order to weaken the background interference in the image and at the same time retain the edge and detail features of the object to be measured, the Gaussian filtering algorithm is selected to filter the two images respectively.

[0044] 3. In the Gaussian filtering algorithm, after ensuring a certain degree of image blurring to weaken the background interference, the standard deviation should be as small as possible to retain the image contour features.

[0045] 4. To reduce the computational complexity of stereo matching, after filtering, edge detection is performed on the two images respectively.

[0046] 5. After filtering, the edge information and feature information of the image will be "blurred". Therefore, before edge detection, the closing operation method is adopted for the two images respectively to eliminate small holes in the image, fill small breaks, and smooth the boundary of the object to be measured. After Gaussian filtering, image edge smoothing is performed through the closing operation, restoring the original features of the object as much as possible and improving the measurement accuracy;

[0047] 6. Then, the horizontal and vertical gradient operators are defined through the Sobel edge detection algorithm, and the horizontal and vertical gradients of the pixels in the two images are calculated. After calculating the gradients of each pixel, they are compared with the threshold to identify the edge points. After the closing operation, the image contour features are extracted through the Sobel edge detection algorithm, reducing the redundant feature information of the image;

[0048] 7. After extracting the edge features of the two images, stereo matching is performed on the two images. The computational complexity of stereo matching is greatly reduced, and the calculation rate is significantly improved. After extracting the edge features of the two images and then performing stereo matching, the operation speed is greatly increased.

[0049] As described above, only the specific embodiments of the present invention are provided, and the present invention is described in detail. The unelaborated parts are conventional technologies. However, the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for measuring object posture based on binocular vision in a complex environment, characterized in that: include: Obtain two images of the object being measured taken by cameras at two angles, and perform camera calibration, stereo correction and filtering on the two images; Perform a closing operation on the two filtered images to smooth the image edges; The Sobel algorithm is used to identify the external contour features of the object in the two smoothed images; Stereo matching and posture calculation are performed based on the external contour features in the two acquired images to obtain the object posture of the object being measured.

2. The object posture measurement method based on binocular vision in a complex environment according to claim 1 is characterized in that: The camera calibration uses Zhang Zhengyou calibration method to complete the binocular camera calibration.

3. The object posture measurement method based on binocular vision in a complex environment according to claim 1 is characterized in that: The stereo correction adopts an epipolar correction method and utilizes the binocular camera calibration result to correct two non-coplanar and non-row-aligned images, thereby eliminating the error caused by the non-parallel optical axes of the cameras.

4. The object posture measurement method based on binocular vision in a complex environment according to claim 1 is characterized in that: The filtering adopts Gaussian filtering method.

5. An object posture measurement device based on binocular vision in a complex environment, characterized in that: include: A preprocessing module is used to obtain two images of the object being measured taken by cameras at two angles, and perform camera calibration, stereo correction and filtering on the two images; A closing operation module is used to perform a closing operation on the two filtered images to smooth the image edges; The edge extraction module is used to identify the external contour features of the object to be measured in the two smoothed images through the Sobel algorithm; The post-processing module is used to perform stereo matching and pose calculation according to the external contour features in the two acquired images to obtain the object pose of the object under test.

6. The object posture measurement device based on binocular vision in a complex environment according to claim 5, characterized in that: The preprocessing module is specifically used to complete the binocular camera calibration using Zhang Zhengyou calibration method.

7. The object posture measurement device based on binocular vision in a complex environment according to claim 5, characterized in that: The preprocessing module is specifically used to adopt the epipolar correction method and use the binocular camera calibration results to correct the two non-coplanar and non-row aligned images to eliminate the error caused by the non-parallel camera optical axes.

8. The object posture measurement device based on binocular vision in a complex environment according to claim 5, characterized in that: The preprocessing module is specifically used to adopt Gaussian filtering method.

Images