A binocular vision hole position recognition method for an industrial robot assembly system

Abstract

The embodiment of the application discloses a binocular vision hole position recognition method for an industrial robot assembly system, relates to the field of robot high-precision assembly vision detection, and can be applied to the field of robot high-precision assembly vision detection. The method comprises the following steps: calibrating a monocular camera and a binocular camera by using a calibration board; pre-processing images captured by left and right cameras to obtain edge contours of candidate holes; performing elliptical fitting on the edges of the candidate holes to obtain a candidate elliptical set, screening in the candidate elliptical set, and determining a fitting ellipse of a target hole to be selected; performing polar line matching on the target hole to be selected according to a camera calibration result in S1, and performing three-dimensional reconstruction on the determined target hole to be selected; and screening three-dimensional spatial positions of the target hole to be selected to obtain a three-dimensional spatial position of the target hole.

Description

Technical Field

[0001] This invention relates to the field of high-precision assembly visual inspection for robots, and more particularly to a binocular vision hole position recognition method for industrial robot assembly systems. Background Technology

[0002] As a strategic industry, the aerospace industry is a vital force for national economic, technological, and defense development. However, my country lags significantly behind foreign countries in high-end equipment manufacturing. In recent years, major international aerospace companies have been developing automated production equipment to shorten manufacturing cycles. Industrial robots, with their high flexibility, low cost, and high workspace-to-floor area ratio, are widely used in the aerospace field. Some advanced foreign military and civilian aircraft have begun using flexible automated production lines primarily composed of industrial robots in their manufacturing processes, processing large components such as fuselages and wings, significantly improving manufacturing efficiency, processing quality, and reducing costs.

[0003] Many production lines in China still operate under a low-efficiency, low-precision manual operation model, relying on highly skilled technicians to achieve the required precision in the production and assembly of certain components. This is insufficient to meet the ever-increasing demands for both output and quality. Furthermore, industrial robots currently used in China are primarily applied to low-precision applications such as welding, material handling, and painting. However, the machining and assembly of large aircraft components in the aerospace manufacturing industry often requires extremely high precision, and the low overall precision of robots makes it difficult to guarantee assembly accuracy.

[0004] Therefore, improving the assembly precision of industrial robots and narrowing the gap between my country and foreign aerospace giants has become an important research direction. Summary of the Invention

[0005] The embodiments of the present invention provide a binocular vision hole position recognition method for industrial robot assembly systems, which can improve the accuracy and efficiency of hole position recognition in industrial settings and reduce the impact of binocular measurement errors on robot assembly accuracy.

[0006] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0007] The method can be used in a binocular vision hole position recognition system of an industrial robot assembly system. This system includes an industrial robot (1), a binocular vision sensor (2), an auxiliary light source (3), a fixture (4), and a gripper (7). This system is used to assemble a workpiece (5) to be assembled and a workpiece (6) to be assembled off. A gripper (7) is installed at the end of the robotic arm of the industrial robot (1). The gripper (7) is used to clamp the workpiece (6) to be assembled during the assembly process. The fixture (4) is used to fix the workpiece (5) to be assembled off during the assembly process. The binocular vision sensor (2) is mounted on the fixture (4) and faces the workpiece (5) to be assembled off. The depth of field and visual range of the binocular vision sensor (2) cover the candidate holes. The auxiliary light source (3) is connected to the binocular vision sensor (2) via a connector. The connector can be a readily available and widely used steel connector.

[0008] The method includes:

[0009] S1, use calibration board (8) to calibrate monocular and binocular cameras.

[0010] S2, preprocess the images captured by the left and right cameras to obtain the edge contours of the candidate holes.

[0011] S3, perform ellipse fitting on the edges of the candidate holes to obtain a set of candidate ellipses, and then filter from the set of candidate ellipses to determine the fitting ellipse for the target hole to be selected.

[0012] S4. Based on the camera calibration results in S1, perform epipolar matching on the selected target aperture and perform three-dimensional reconstruction on the determined target aperture.

[0013] S5: Filter the three-dimensional spatial positions of the target holes to obtain the three-dimensional spatial positions of the target holes.

[0014] The binocular vision hole position recognition method for industrial robot assembly systems provided in this invention realizes edge detection, contour recognition, multiple ellipse screening, and three-dimensional geometric screening processes for candidate holes. For example, the process of ellipse fitting for the edges of candidate holes has been improved. This effectively improves the accuracy and efficiency of hole position recognition in industrial settings, reduces the impact of binocular measurement errors on robot assembly accuracy, and can meet the hole position detection accuracy requirements during the assembly of large workpieces. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a device architecture in a possible application scenario provided by an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the method flow provided in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of camera calibration provided in an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Embodiments of the present invention will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of the present invention means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0020] This invention provides a binocular vision hole position recognition method for an industrial robot assembly system. The method is used to establish the hardware configuration of an industrial robot assembly system. The system hardware includes an industrial robot (1), a binocular vision sensor (2), an auxiliary light source (3), upper and lower workpieces (5, 6) to be assembled, grippers (7), and matching connectors (4). Specifically, grippers (7) are installed at the end of the robotic arm of the industrial robot (1). The grippers (7) are used to clamp the upper workpiece (6) during assembly. A fixture (4) is used to fix the lower workpiece (5) during assembly. The binocular vision sensor (2) is mounted on the fixture (4) and faces the lower workpiece (5). The depth of field and visual range of the binocular vision sensor (2) cover the candidate holes. The auxiliary light source (3) is connected to the binocular vision sensor (2) via connectors.

[0021] like Figure 2 As shown, the method includes:

[0022] S1, use calibration board (8) to calibrate the binocular vision sensor (2) for camera calibration.

[0023] S2, preprocess the image captured by the monocular camera in the binocular vision sensor (2) to obtain the edge contour of the candidate hole. The monocular camera in the binocular vision sensor (2) includes a left camera and a right camera.

[0024] The candidate holes include candidate holes for both the upper and lower workpieces.

[0025] S3, perform ellipse fitting on the edges of the candidate holes to obtain a set of candidate ellipses, and then filter from the set of candidate ellipses to determine the fitting ellipse for the target hole to be selected.

[0026] S4. Based on the camera calibration results in S1, perform epipolar matching on the selected target aperture and perform three-dimensional reconstruction on the determined target aperture.

[0027] In practical applications, step S4 can employ common 3D reconstruction techniques, such as:

[0028] S41. Based on the camera calibration results in step S1, perform epipolar line matching on the hole position coordinates identified in the left and right images. Draw epipolar lines according to the hole positions in the left image. Draw perpendicular lines from all hole positions in the right image to the epipolar lines in the left image. Select the point in the right image with the shortest perpendicular line distance as the corresponding point in the left image.

[0029] S42 completes the three-dimensional reconstruction of pore sites through steps such as stereo correction, sparse correction, and depth extraction.

[0030] S5: Filter the three-dimensional spatial positions of the target holes to obtain the three-dimensional spatial positions of the target holes.

[0031] In this embodiment, S1 includes:

[0032] S11, the calibration board (8) is captured by the binocular vision sensor (2), wherein, for example Figure 3 As shown, an elliptical calibration plate (8) is placed below a binocular vision sensor (2). By continuously changing the pose of the calibration plate (8), images of the calibration plate (8) in different poses are obtained. Specifically, the elliptical calibration plate (8) is placed below the binocular vision sensor (2), which captures images of the calibration plate (8). Then, the pose of the calibration plate (8) is moved and rotated multiple times to obtain multiple sets of left and right images of the calibration plate (8). When moving the position of the calibration plate (8), it is necessary to ensure that the calibration plate (8) is always within the depth of field and visual range of the binocular vision sensor.

[0033] S12 involves performing single-camera and dual-camera calibration on the left and right cameras to obtain camera parameters, including intrinsic and extrinsic parameter matrices and distortion coefficients, among other inherent camera information. Single-camera calibration refers to calibrating a single camera; this is a specific term in the field of visual calibration. Dual-camera calibration refers to jointly calibrating two cameras, differing from single-camera calibration in content and function. In practical applications, Zhang Zhengyou's camera calibration method and the dual-camera calibration method in OpenCV can be used to calibrate the left and right cameras and the dual cameras respectively, obtaining the intrinsic and extrinsic parameter matrices, distortion coefficients, and other inherent camera information. In practical applications, the intrinsic and extrinsic parameter matrices and distortion coefficients of each camera can typically be acquired in real time to monitor the operating status of each camera. In this embodiment, S2 includes:

[0034] S21, convert the images of the workpiece (5) to be assembled captured by the left and right cameras into grayscale images, and then perform median filtering and Gaussian filtering operations on the grayscale images in sequence to remove noise.

[0035] S22: The Canny edge detection algorithm is used to perform edge detection on the grayscale image processed by S21. The detection parameters of the Canny edge detection need to be adjusted until the edge of the target hole can be correctly detected.

[0036] S23, the edge image processed by S21 is processed by a contour extraction algorithm to obtain a set of contour points of the edge contour in the grayscale image, wherein the topological structure in the contour extraction algorithm is a two-layer structure.

[0037] Since the target hole has a chamfered finish during manufacturing, the topology in the contour extraction algorithm parameters needs to be set to a double-layer structure during extraction to ensure that the inner and outer circles of the hole are correctly separated in subsequent operations.

[0038] In this embodiment, step S3 includes:

[0039] S31: Traverse the set of contour points of the candidate hole's edge contour, and perform ellipse fitting on each contour consisting of multiple points to obtain a set of candidate ellipses, where each contour is fitted with one ellipse. For example: Traverse the set of contour points of the candidate hole's edge contour according to the topology, and perform ellipse fitting on each contour consisting of multiple points, fitting one ellipse for each contour. The ellipse fitting algorithm uses ellipse fitting based on the least squares algorithm.

[0040] S32, For the candidate ellipse set, perform perimeter area screening, defective circle screening, confidence calculation, duplicate ellipse screening, concentric circle screening, ellipse too close screening, quadratic fitting and confidence screening in sequence to obtain the fitted ellipse of the target hole to be selected.

[0041] Specifically, S32 includes:

[0042] S321, for the ellipses in the candidate ellipse set, perform perimeter and area filtering, and delete ellipses whose perimeter or area is not within the threshold range, wherein the area and perimeter of the ellipse are expressed as:

[0043] C = Pi·l + 2·(ls)

[0044] S = Pi·l / 2·s / 2

[0045] C is the circumference of the ellipse, S is the area of ​​the ellipse, Pi is pi, l is the length of the major axis of the ellipse, and s is the length of the minor axis of the ellipse. A fixed threshold constraint of circumference and area is applied to the candidate ellipses; ellipses outside the threshold range are deleted, while ellipses within the threshold range proceed to the next step of selection.

[0046] S322, the set of ellipses processed by S321 is filtered for missing circles, and missing circles are deleted to obtain candidate ellipses. The judgment model for detecting whether an ellipse is a missing circle includes:

[0047]

[0048] std defect >0.5

[0049] d defect The defect degree index, c, is used to measure the extent of elliptical defect. pointsC represents the number of contour points corresponding to the fitted ellipse, and C is the circumference of the ellipse. The formula in S322 above is used to filter all candidate ellipses; those with large defects are deleted, while candidate ellipses with small defects proceed to the next filtering step.

[0050] S323, obtain the confidence scores of all candidate ellipses.

[0051] S324, perform secondary screening on all candidate ellipses, wherein the secondary screening process includes: selecting two ellipses in sequence from the candidate ellipses for ellipse center comparison; if the two-dimensional distance between the centers of the two ellipses is less than a fixed threshold, the two ellipses are determined to overlap and are merged.

[0052] S325, perform concentric circle screening on the candidate ellipses processed in step S324.

[0053] S326. For the candidate ellipses processed in step S325, remove those that are too close to each other.

[0054] S327. Perform a second fitting process on the candidate ellipse processed in step S326.

[0055] S328. For the candidate ellipses processed in step S327, sort all the remaining candidate ellipses in descending order according to their confidence level, and take out the first n ellipses or / and the ellipses with a confidence level of 1 as the fitting ellipses for the target holes to be selected.

[0056] Specifically, S323 includes:

[0057] For the i-th ellipse fitted to the i-th contour, extract the parameters A of the first to fourth ellipses. i B i C i and F i And A i x 2 +B i xy+C i y 2 +F i =0. The length l of the major axis of the i-th ellipse fitted based on the i-th contour. i minor axis length s i and the center c of the ellipse i With the center c of the ellipse i Adjust the length of the major axis l i and minor axis length s i The adjustment range is m times the original size, resulting in an ellipse e. i Concentric extended ellipse e i-large and the reduced ellipse e i-small and satisfy the following relationship:

[0058] A i-large x 2 +B i-large xy+C i-large y 2 +F i-large =0

[0059] A i-small x 2 +B i-small xy+C i-small y 2 +F i-small =0

[0060] Substitute the coordinates of the points in the set of points forming the i-th contour into the above relationship, and count the number of contour points that are inside the annulus of the expanding and shrinking ellipses. Record the count value as n. points The confidence level is then expressed as: Where, d coincidence The confidence index used to measure the quality of an ellipse fit is c. points This represents the number of contour points corresponding to the fitted ellipse. This step calculates the confidence index for each candidate ellipse, but does not filter the candidate ellipses.

[0061] Specifically, based on step S323, duplicate ellipse screening is performed on all candidate ellipses. Two candidate ellipses are selected sequentially for center comparison. If the two-dimensional distance between the centers of the two ellipses is less than a fixed threshold, the two ellipses are determined to overlap. Therefore, in S324, the method for determining whether two ellipses overlap includes:

[0062]

[0063] d long_axis =l1-l2

[0064] d short_axis =s1-s2

[0065] std center <c threshold

[0066] d long_axis <l threshold

[0067] d short_axis <s threshold

[0068] Where, d center Let d be the two-dimensional distance between the centers of the two ellipses. long_axis d is the difference in length between the major axes of the two ellipses. short_axis c is the difference in length between the minor axes of the two ellipses. 1xLet c be the x-coordinate of the center of ellipse 1. 1y Let c be the y-coordinate of the center of ellipse 1. 2x Let c be the x-coordinate of the center of ellipse 2. 2y Let be the y-coordinate of the center of ellipse 2, l1 be the length of the major axis of ellipse 1, l2 be the length of the major axis of ellipse 2, s1 be the length of the minor axis of ellipse 1, s2 be the length of the minor axis of ellipse 2, and c be the y-coordinate of the center of ellipse 2. threshold The threshold value for the distance between the centers of the two ellipses, l threshold s is the threshold value for the difference between the lengths of the major axes of the two ellipses. threshold It is the threshold value for the difference in length between the minor axes of the two ellipses.

[0069] Furthermore, based on step S324, to distinguish between the inner and outer chamfered circles of a hole, concentric circle detection is performed on the candidate elliptical holes. The screening principle is to traverse any two ellipses from the candidate ellipses; if the centers of the two ellipses meet a fixed small threshold, they are determined to be concentric circles. Then, according to the actual task requirements, the inner and outer chamfered circles of the holes are selected and retained, thus completing the concentric circle screening of the holes. Therefore, in S325, the determination method for whether a hole is a concentric circle includes:

[0070]

[0071] std center <c threshold

[0072] Where, d center Let c be the two-dimensional distance between the centers of the two ellipses. threshold c is the threshold value for the distance between the centers of the two ellipses. 1x Let c be the x-coordinate of the center of ellipse 1. 1y Let c be the y-coordinate of the center of ellipse 1. 2x Let c be the x-coordinate of the center of ellipse 2. 2y Let y be the center of ellipse 2.

[0073] Furthermore, based on step S325, the candidate ellipse set is subjected to a "too close" ellipse filtering process. This aims to remove erroneous ellipses caused by image noise or surface impurities such as rust. Therefore, in S326, the methods for determining whether two ellipses are too close include:

[0074]

[0075] std center <c threshold

[0076] For two candidate ellipses with areas smaller than a fixed threshold, the areas of the two candidate ellipses are compared based on whether the outer or inner circle is selected in the actual situation. If the outer circle is retained, the ellipse with the smaller area is deleted, and vice versa.

[0077] In S327, the process of performing a second-order fitting includes:

[0078] The set of contour points corresponding to the i-th ellipse with a confidence level less than 1 is traversed. Points whose positions lie between the expanded and reduced ellipses are saved, and this saved set of contour points is used as the new set of contour points. Then, the new set of contour points is used to perform least-squares-based ellipse fitting again, replacing the original candidate ellipses. Finally, the confidence level of the replaced candidate ellipses is recalculated. For example:

[0079] Step S327 involves performing neighborhood denoising and secondary fitting based on step S326. This process involves analyzing the ellipse parameters of candidate ellipses with a confidence level less than 1 from the entire candidate ellipse set. For example, for the i-th ellipse with a confidence level less than 1, its ellipse parameter A is extracted. i B i C i F i To make it satisfy the general formula for an ellipse, as follows:

[0080] A i x 2 +B i xy+C i y 2 +F i =0

[0081] Based on the major and minor axes l of the i-th ellipse with a confidence level less than 1 i s i With the center c of the ellipse i With the center c of the ellipse i Centered on the ellipse, expand and shrink the ellipse e respectively. i The major and minor axes are expanded and reduced by a factor of m, respectively, to form an ellipse e. i Concentric ellipses e i-large With ellipse e i-small Its ellipse parameters satisfy the following relationship:

[0082] A i-large x 2 +B i-large xy+C i-large y 2 +F i-large =0

[0083] A i-small x 2 +B i-small xy+C i-small y 2 +F i-small =0

[0084] Traverse the set of contour points corresponding to the i-th ellipse with a confidence level less than 1, save the points whose contour point positions are between the expanded ellipse and the reduced ellipse, unify them into a new set of contour points, and use the new set of contour points to perform ellipse fitting based on least squares again to replace the original candidate ellipse, and recalculate its confidence level.

[0085] S328, Based on step S327, according to the confidence level of the ellipse, all candidate ellipses are sorted in descending order, and the first n ellipses or those with a confidence level of 1 are selected as the fitting ellipse for the final target hole.

[0086] In this embodiment, step S5 includes:

[0087] S51, the theoretical distance {d} between the target holes is measured based on the three-dimensional models of the workpiece to be assembled (5) and the workpiece to be assembled (6). theory,1 ,d theory,2 ,d theory,3}, and used as the basis for spatial location selection. Specifically, the three-dimensional spatial coordinates of the hole positions obtained in step S4 include the three-dimensional coordinates of the target hole positions. To accurately extract the three-dimensional coordinates of the target hole positions, the theoretical distance {d} between the target hole positions is first measured based on the three-dimensional model of the upper and lower workpieces to be assembled. theory,1 ,d theory,2 ,d theory,3}, used as a basis for spatial location selection.

[0088] S52, based on the theoretical distance obtained in step S51, search for all three-dimensional points in space. The search process includes: first, for the i-th three-dimensional point p... i Calculate the distance from the remaining three-dimensional points in space to p. i The three-dimensional distance is calculated and recorded as a distance sequence I. i ={d i,1 ,d i,2 ,…,d i,i-1 ,0,d i,i+1 Then repeat the above operation for all other points until the distance sequence of all points {I1,I2,I3,…} is obtained. n Then iterate through the distance sequence of all points to find the one that simultaneously satisfies the theoretical distance d. theory,1 d theory,2 The distance sequence is stored and denoted as I. corner I corner The corresponding points are used as preliminary theoretical points. Based on the distance sequence of the preliminary theoretical points, the system iterates through the points to find those that satisfy d. theory,1 and d theory,2The three-dimensional distance is numbered in the distance sequence, and finally, a specific theoretical point is obtained based on the number. For example, based on the theoretical distance obtained in step S51, a search is performed on all spatial three-dimensional points. The search principle and method are as follows: First, for the i-th spatial three-dimensional point p... i Calculate the distance from the remaining three-dimensional points in space to p. i The three-dimensional distance, i.e., the distance sequence I i ={d i,1 ,d i,2 ,…,d i,i-1 ,0,d i,i+1 Repeat the above operation for all other points to obtain the distance sequence of all points {I1,I2,I3,...,I...}. n}, traverse all distance sequences, and find the one that simultaneously satisfies the theoretical distance d theory,1 d theory,2 The distance sequence is stored and denoted as I. corner The corresponding point is denoted as p. corner That is, the theoretical point p theory,1 According to the theoretical point p theory,1 The distance sequence is traversed to find those that satisfy d. theory,1 d theory,2 The three-dimensional distances are indexed in the distance sequence as index1, index2, etc., which can clearly identify the theoretical points.

[0089] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A binocular vision hole recognition method for an industrial robot assembly system, characterized by, The method is used for a binocular vision hole position recognition system of an industrial robot assembly system, the system comprising: an industrial robot (1), a binocular vision sensor (2), an auxiliary light source (3), a fixed tooling (4) and a gripper (7), the system being used for assembling an upper workpiece to be assembled (5) and a lower workpiece to be assembled (6); The method comprises: S1, camera calibration of the binocular vision sensor (2) is performed by using a calibration board (8); S2, the images taken by the monocular cameras in the binocular vision sensor (2) are preprocessed to obtain the edge contours of candidate holes, the monocular cameras in the binocular vision sensor (2) comprising a left camera and a right camera; S3, the edges of the candidate holes are subjected to elliptical fitting to obtain a candidate elliptical set, and the candidate elliptical set is screened to determine the fitting ellipse of the target hole to be selected; In S3, it comprises: S31, the profile point set of the edge contour of the candidate hole is traversed, and each profile composed of a plurality of points is subjected to elliptical fitting to obtain a candidate elliptical set, wherein each profile is fitted with an ellipse; S32, for the candidate elliptical set, the perimeter area screening, the missing circle screening, the confidence calculation, the repeated ellipse screening, the concentric circle screening, the distance close ellipse screening, the secondary fitting and the confidence screening are sequentially performed to obtain the fitting ellipse of the target hole to be selected; Wherein, the process of secondary fitting processing comprises: traversing the profile point set corresponding to the i-th ellipse with a confidence less than 1, saving the points between the expanded ellipse and the reduced ellipse at the position of the profile point, and taking the saved profile point set as a new profile point set; then the new profile point set is used for least square-based elliptical fitting again, and the original candidate ellipse is replaced; then the confidence of the replaced candidate ellipse is recalculated; S4, according to the camera calibration result in S1, polar line matching is performed on the target hole to be selected, and three-dimensional reconstruction is performed on the determined target hole to be selected; S5, the three-dimensional spatial position of the target hole to be selected is screened to obtain the three-dimensional spatial position of the target hole.

2. The method of claim 1, wherein, The end of the mechanical arm of the industrial robot (1) is provided with a gripper (7), and the gripper (7) is used for clamping the upper workpiece to be assembled (5) during assembly; The fixed tooling (4) is used for fixing the lower workpiece to be assembled (6) during assembly; The binocular vision sensor (2) is installed on the fixed tooling (4), and the binocular vision sensor (2) faces the direction of the lower workpiece to be assembled (6); The depth of field range and the vision range of the binocular vision sensor (2) cover the candidate hole, and the auxiliary light source (3) is connected with the binocular vision sensor (2) through a connecting piece.

3. The method of claim 1, wherein, In S1, it comprises: S11, an image group of the calibration board (8) is taken by the binocular vision sensor (2), wherein the elliptical calibration board (8) is placed below the binocular vision sensor (2), and the images of the calibration board (8) under different poses are obtained by continuously changing the pose of the calibration board (8); S12, single target calibration and double target calibration are performed on the left and right cameras to obtain camera parameters, the camera parameters comprising: internal and external parameter matrices and distortion coefficients.

4. The method of claim 1, wherein, In S2, it comprises: S21, converting the image of the workpiece (6) to be assembled photographed by the left and right cameras into a grayscale image, and then sequentially performing median filtering and Gaussian filtering on the grayscale image; S22, performing edge detection on the grayscale image processed in S21 by using a Canny edge detection algorithm, wherein the detection parameters of the Canny edge detection algorithm need to be adjusted until the hole edge of the target hole position can be correctly detected; S23, processing the edge image processed in S21 by using a contour extraction algorithm to obtain a contour point set of the edge contour in the grayscale image, wherein the topological structure in the contour extraction algorithm is a double-layer structure.

5. The method of claim 1, wherein, In S32, comprising: S321, performing perimeter and area screening on the ellipses in the candidate ellipse set, and deleting ellipses whose perimeter or area is not within a threshold range, wherein the area and perimeter of an ellipse are represented as: C=Pi·l+2·(l-s) S=Pi·l / 2·s / 2 C is the perimeter of the ellipse, S is the area of the ellipse, Pi is the ratio of a circle, l is the length of the major axis of the ellipse, and s is the length of the minor axis of the ellipse; S322, performing defect circle screening on the ellipse set processed in S321 and deleting defect circles to obtain a candidate ellipse set, wherein the judgment model for detecting whether an ellipse is a defect circle includes: s.t.d defect >0.5 d defect As a defect index for measuring the defect degree of the ellipse, c points C is the number of contour points corresponding to the fitted ellipse, and C is the circumference of the ellipse. S323, obtaining the confidence of all candidate ellipses; S324, performing secondary screening on all candidate ellipses, wherein in the process of secondary screening, including: selecting two ellipses in the candidate ellipse set in turn to compare their centers, if the two-dimensional distance between the centers of the two ellipses is less than a fixed threshold, it is determined that the two ellipses coincide, and merging processing is performed; S325, performing concentric circle screening on the candidate ellipses processed in step S324; S326, removing ellipses with too close distances from the candidate ellipses processed in step S325; S327, performing secondary fitting processing on the candidate ellipses processed in step S326; S328, arranging all remaining candidate ellipses in descending order according to the confidence, taking the first n or / and the ellipses with a confidence of 1 as the fitting ellipses of the target hole to be selected.

6. The method of claim 5, wherein, In S323, comprising: extracting first to fourth ellipse parameters A i , B i , C i , and F i for the i-th ellipse fitted to the i-th contour, and A i x 2 +B i xy+C i y 2 +F i = 0; The length l of the major axis of the i-th ellipse fitted based on the i-th contour. i minor axis length s i and the center c of the ellipse i With the center c of the ellipse i Adjust the length of the major axis l i and minor axis length s i The adjustment range is m times the original size, resulting in an ellipse e. i Concentric extended ellipse e i-large and the reduced ellipse e i-small and satisfy the following relationship: A i-large x 2 +B i-large xy+C i-large y 2 +F i-large =0 A i-small x 2 +B i-small xy+C i-small y 2 +F i-small =0 The coordinates of the points in the point set constituting the i-th contour are respectively brought into the above relationship, and the number of points of the contour points inside the expanded ellipse and the reduced ellipse is counted, and the counting value is recorded as n points The confidence is expressed as: where d coincidence is a confidence index for measuring the goodness of the ellipse fitting, c points is the number of contour points corresponding to the fitted ellipse.

7. The method of claim 6, wherein, In S324, the determination method of whether two ellipses coincide includes: d long_axis ＝l1-l2 d short_axis = s1 - s2 s.t.d center <c threshold d long_axis ＜l threshold d short_axis <s threshold Where, d center Let d be the two-dimensional distance between the centers of the two ellipses. long_axis d is the difference in length between the major axes of the two ellipses. short_axis c is the difference in length between the minor axes of the two ellipses. 1x Let c be the x-coordinate of the center of ellipse 1. 1y Let c be the y-coordinate of the center of ellipse 1. 2x Let c be the x-coordinate of the center of ellipse 2. 2y Let be the y-coordinate of the center of ellipse 2, l1 be the length of the major axis of ellipse 1, l2 be the length of the major axis of ellipse 2, s1 be the length of the minor axis of ellipse 1, s2 be the length of the minor axis of ellipse 2, and c be the y-coordinate of the center of ellipse 2. threshold The threshold value for the distance between the centers of the two ellipses, l threshold s is the threshold value for the difference between the lengths of the major axes of the two ellipses. threshold The threshold value is the difference between the lengths of the minor axes of the two ellipses. In S325, the determination method of whether two ellipses are concentric includes: s.t.d center <c threshold where d center is the two-dimensional distance between the centers of the two ellipses, c threshold is a threshold for the distance between the centers of the two ellipses, c 1x is the x-coordinate of the center of ellipse 1, c 1y is the y-coordinate of the center of ellipse 1, c 2x is the x-coordinate of the center of ellipse 2, c 2y is the y-coordinate of the center of ellipse 2.

8. The method of claim 5, wherein, In S326, the method of determining whether the distance between two ellipses is too close includes: s.t.d center <c threshold For two candidate ellipses less than a fixed threshold, the areas of the two candidate ellipses are compared, and the candidate ellipse with a smaller area is deleted.

9. The method of claim 1, wherein, In S5, comprising: S51, according to the three-dimensional model of the upper workpiece (5) and the lower workpiece (6) to be assembled, the theoretical distance {d between the target hole positions is measured theory,1 ,d theory,2 ,d theory,3 ], and as a spatial position screening basis; S52, searching all spatial three-dimensional points according to the theoretical distance obtained in step S51, wherein the searching process includes: First, for the i-th spatial three-dimensional point p i , the three-dimensional distances of the remaining spatial three-dimensional points to p i are calculated and recorded as a distance sequence I i ={d i,1 ,d i,2 ,…,d i,i-1 ,0,d i,i+1 ,…} The above operation is repeated for all the remaining points until the distance sequence {I1, I2, I3,..., In} of all the points is obtained. n ] Then iterate through the distance sequence of all points to find the theoretical distance d. theory,1 d theory,2 The distance sequence is stored and denoted as I. corner I corner The corresponding points serve as preliminary theoretical points; According to the initial distance sequence of the theory point, the three-dimensional distance respectively satisfying d theory,1 and d theory,2 is searched, and the number of the three-dimensional distance in the distance sequence is obtained, and finally the explicit theory point is obtained according to the number.

Images