A hole position off-line instruction generation and online positioning method based on visual detection

Abstract

A kind of off-line instruction generation and online positioning method of hole position based on visual detection, the hole position detection and positioning feature detection of the part that has been drilled are carried out by the visual detection unit installed on robot, then the corresponding hole position is recorded and input to off-line program, finally the same feature of the part to be drilled is detected when drilling online, and then it is positioned automatically according to the positional relationship between hole position and feature in off-line program.The application solves the problem of hole position generation of off-line program and hole position positioning of online control program faced by the application of robot drilling on the part that has been drilled by manual positioning, avoids the process of re-accurately modeling and hole position definition for tens of thousands of various parts, and also avoids the problem of manual teaching programming due to part error, so as to realize a kind of off-line instruction generation and online positioning method of hole position based on visual detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent manufacturing, especially to a method for determining and positioning a hole site of a part in the field of aerospace assembly hole making, in the case of lacking an accurate theoretical mathematical model of a workpiece to be processed, and more particularly to a hole making hole site offline instruction generation and online positioning method based on visual detection, which can be used for online positioning hole making tasks of a robot on a part. First, a visual detection unit is used to determine the hole making position information and the relative reference feature of a part that has been prepared, so as to realize automatic generation of a robot offline processing program, and accordingly, hole site positioning and hole making are performed on the part to be processed. BACKGROUND

[0002] Due to the large number of product structural parts in the aerospace field, the complex coordination relationship, and other reasons, the assembly process thereof is heavily dependent on manual operation. The positioning of manual hole making requires detailed interactive operation of an operator, and the positioning of each part assembly hole and pilot hole is determined by using marking positioning and other methods. However, with the steady increase in the batch quantity of aerospace products, the original hole making positioning method seriously hinders the improvement of efficiency and the control of cost.

[0003] With the rapid development of digital manufacturing technology and robot technology, robot automatic hole making technology has been more and more widely applied due to its many advantages in hole making efficiency, hole making quality, and adaptability. In order to ensure the product assembly quality and shorten the production and manufacturing cycle of products, the development of advanced assembly technology and equipment is of great significance to the development of modern aerospace manufacturing industry. With the rapid development of digital manufacturing technology and robot technology, robot automatic hole making technology has been more and more widely applied.

[0004] Robot hole making requires accurate product numerical models and hole site information, and the determination of a hole making reference. Since a large number of early generated parts on site are not produced and processed by using a model-based definition (MBD) method, the theoretical mathematical model of the part is lacking, and therefore, the workload of realizing robot hole making offline instruction generation and hole site online positioning tasks is huge and prone to errors.

[0005] At present, there are mainly two solutions for robot hole making hole site determination: one is an online teaching-based method, and the other is an offline programming-based method.

[0006] The online teaching-based robot hole-making position determination method involves an operator manually controlling the robot, moving its end effector to a predetermined position, and recording that position to "program" the motion instructions. During online hole making, the robot automatically repeats the hole-making work at these positions according to the instructions. The biggest problem with this method is that for a large number of aerospace sheet metal parts with low precision, the positions determined by the teaching often cannot guarantee requirements such as hole margins. Furthermore, for aerospace parts with complex curved surfaces, online teaching is inefficient and inaccurate, especially in small-batch, multi-variety production models, significantly increasing the robot's operating costs and programming time.

[0007] The offline programming-based robot hole-making position determination method requires pre-designing the positioning datum and the position of the hole to be made on a precise product theoretical mathematical model (MBD model). Then, offline programming software automatically identifies the positions of the datum and the hole to be made, generating offline instructions. During online hole making, the robot detects the corresponding datum and offsets itself to the hole position based on the datum to complete the hole making. The biggest problem with this method is the need for an accurate MBD model. For parts that did not initially conform to MBD design standards, the modeling workload is enormous, thus limiting the applicability of this method.

[0008] Based on the shortcomings and deficiencies of the two methods for determining the hole position in robotic drilling, it is necessary to design a new method that can quickly determine the hole position without re-establishing a digital model, and then complete the positioning and drilling work of the corresponding hole position on the new part to be drilled. Summary of the Invention

[0009] This invention addresses the challenges of offline hole position generation and online hole position positioning in robotic hole making on parts where manual positioning is already used. It proposes a vision-based method for generating offline hole position commands and positioning them online. The method utilizes a vision inspection unit mounted on a robot to detect hole positions and datum features on a pre-prepared workpiece according to process requirements. The corresponding hole positions are then recorded and input into the offline program. Finally, during online hole making, the method detects identical datum features on the part to be drilled and automatically positions the hole according to the positional relationship between the hole and the datum in the offline program.

[0010] The technical solution of the present invention is as follows:

[0011] A method for offline generation and online positioning of hole positions based on visual detection, characterized in that:

[0012] The following steps are used:

[0013] First, the equipment required for this method includes a robotic hole-making machine, a vision inspection unit, a control system, and physical samples. The robotic hole-making machine includes an industrial robotic arm or other robot body and an end effector; the vision inspection unit includes image acquisition devices for detecting hole positions and reference features, including monocular and / or multi-view cameras, lighting devices, etc., and is mounted on the end effector; the control system includes a controller or PLC and other motion control systems and a computer; the physical samples are pre-prepared workpieces according to process requirements. Before using the equipment, the vision inspection unit and the robot should be calibrated to obtain the transformation relationship between the coordinate system of the vision inspection unit and the coordinate system of the robot.

[0014] Second, the manually driven robot hole-making equipment moves the vision inspection unit to the vicinity of the reference feature corresponding to the local hole position on the physical sample, completes the image acquisition of the reference feature, and uses the feature recognition algorithm to identify the feature of the reference, and records the corresponding feature and position to the offline program;

[0015] Third, the manually driven robotic hole-making equipment uses a vision inspection unit to sequentially photograph the hole positions corresponding to the previous area reference on the physical sample, completes the image acquisition of hole position features, and uses a feature recognition algorithm to identify the hole positions to be made, and records the corresponding poses to the offline program; note that the relationship between the reference and the hole position is generally one-to-many, that is, the reference of one area corresponds to multiple hole positions.

[0016] Fourth, after repeating steps two and three to complete the detection of all the references and hole positions of the physical sample, the computer in the control system generates robot processing information based on the reference features and hole position and pose information relative to the hole to be made, automatically completes offline programming, and saves it as the offline processing program of the corresponding part, which can be reused by the robot hole making system.

[0017] Fifth, the robot hole-making system calls the aforementioned corresponding offline processing program to perform hole-making on the workpiece to be processed.

[0018] The beneficial effects of this invention are:

[0019] This invention solves the problems of offline program hole position generation and online control program hole position positioning when applying robotic hole making to parts that already use manual positioning for hole making. It avoids the process of accurately remodeling and defining hole positions for a large number of parts, and also avoids the problem of not being able to manually teach and program due to manufacturing errors in the parts. Thus, it realizes a method for offline generation and online positioning of hole positions based on vision inspection. This solution has the following significant advantages:

[0020] First, traditional aerospace products do not employ Model Based Definition (MBD) technology; their manufacturing and assembly are based on two-dimensional engineering drawings. This means that even with three-dimensional digital models, their consistency with reality cannot be guaranteed, especially for parts that have undergone multiple modifications. The method of this invention directly utilizes selectable, completed physical samples as templates, using visual inspection to identify and locate benchmarks and hole positions, thus completing the offline instruction generation task without the need for modeling.

[0021] Secondly, traditional manual teaching programming with fixed poses cannot meet the requirements of hole margins for sheet metal parts with large precision errors. The method of this invention utilizes the matching information of the datum and the hole position, which can effectively solve precision problems such as short margins caused by part manufacturing errors.

[0022] Third, the visual inspection unit involved in this invention can be used simultaneously during hole making without increasing the cost of the equipment. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the offline command generation and online positioning method for hole making based on visual detection of the present invention.

[0024] Figure 2 This is a schematic diagram of the baseline feature recognition algorithm of the present invention.

[0025] Figure 3 This is the recognition result of the feature recognition algorithm in the embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] Please see Figure 1 The embodiments of the present invention include:

[0028] A method for offline hole position generation and online positioning based on vision inspection is proposed. It utilizes a vision inspection unit mounted on a robot to detect hole positions and datum features on a pre-fabricated workpiece according to process requirements, records the corresponding hole positions, and inputs them into an offline program. During online hole making, after detecting datum features on the workpiece to be drilled that are identical to the actual sample, the method generates offline hole position commands and automatically positions them online based on the positional relationship between the hole positions and datum features in the offline program. The system structure of the equipment used is as follows: Figure 1 As shown, the device includes a physical sample 1, a workpiece to be processed 2, a vision inspection unit 3, a host computer in the control system 4, a PLC or motion controller in the control system 5, and a robot hole-making machine 6.

[0029] The visual inspection unit 3 consists of a monocular camera for detecting hole positions and a 3D scanner for detecting reference features. The 3D scanner can be made using a binocular camera or a structured light scanner, or other types of 3D scanners can be selected according to the actual needs of the application scenario. The 3D scanner and the monocular camera for detecting hole positions are only logically separated; in actual physical implementation, if the 3D scanner uses a binocular camera, it can also be used to detect hole positions simultaneously.

[0030] by Figure 1 Taking the drilling of the upper ear piece of the sheet metal frame as an example, the method of the present invention specifically includes the following steps:

[0031] First, the robotic hole-making machine 6 is manually driven to move the 3D scanner of the vision inspection unit 3 to the vicinity of the reference feature corresponding to the local hole position on the physical sample, completing the image acquisition of the reference feature. The host computer 4 in the control system uses a feature recognition algorithm to identify the reference feature, which is mainly the edge of the reference hole or lug. See [link to relevant documentation]. Figure 2 The corresponding features and locations are recorded in the offline program;

[0032] Second, the manually driven robotic hole-making machine 6 sequentially uses the monocular camera of the vision inspection unit 3 to locate the hole positions corresponding to each ear piece reference (ear piece edge) on the physical sample 1, and takes images of each ear piece on the physical sample 1. The host computer 4 processes and corrects the received ear piece images, and uses a feature recognition algorithm to identify the hole positions, thereby obtaining the relative position information of the round holes on the ear pieces, including the number of round holes and edge distance information. See [link to documentation]. Figure 2 Note that the relationship between the reference and the hole positions is generally one-to-many, meaning that a reference for one area corresponds to multiple hole positions. When the hole layout exceeds the field of view, it may be necessary to manually move the vision inspection unit 3, take multiple ear-shaped images, and perform stitching processing to record the pose of the corresponding hole positions into the offline program;

[0033] Third, after completing the detection of all datum points and hole positions on physical sample 1, the operator can modify the generated offline program according to the actual situation, and edit the hole positions where errors are relatively large due to manual hole making. The computer in the control system generates robot processing information based on the acquired datum features and hole position pose information of the hole to be made, and automatically completes offline programming and datum feature recognition. The datum features here are mainly: the edge of the datum hole or lug, see [link to documentation]. Figure 2 And save it as an offline machining program for the corresponding part for reuse;

[0034] Fourth, the manually driven robotic hole-making machine 6 reaches the first reference feature position in the offline program on the workpiece 2. The 3D scanner of the vision inspection unit 3 scans the reference features on the workpiece 2, and the feature recognition algorithm identifies the reference features and automatically matches them with the features in the offline program. The reference features here are mainly the edges of the reference holes or lugs. In this embodiment, the feature recognition algorithm is based on ellipse detection and edge detection algorithms. Developers can design corresponding feature recognition algorithms based on the selected reference features. See [link to relevant documentation]. Figure 3 This allows us to obtain the transformation relationship between the coordinate system of the offline program reference and hole position and the pose coordinate system of the workpiece 2 to be processed;

[0035] Fifth, the robot hole-making machine 6 automatically transforms the coordinates of the hole position and pose for each local area using the transformation relationship obtained in the previous step, and then drives the robot hole-making system to reach the target pose to make holes at the corresponding positions of the workpiece 2 to be processed.

[0036] Sixth, before drilling, the 3D scanner of the vision inspection unit 3 can be used to scan the reference features of each drilling area again to confirm that the current pose can guarantee the process requirements such as edge distance.

[0037] Finally, after all areas have been drilled, the robotic drilling system 6 returns to its origin, awaiting processing instructions for the next part to be drilled.

[0038] The parts not covered in this invention are the same as or can be implemented using existing technologies.

Claims

1.A method of visual inspection based hole position off-line instruction generation and on-line positioning, The application is characterized in that: firstly, a system required for hole making is configured by parts including a robot hole maker, a visual detection unit, a control system and a physical sample, the robot hole maker further includes a robot body and an end effector, the visual detection unit and the robot body are calibrated to obtain the corresponding relationship between the visual detection unit coordinate system and the robot coordinate system; secondly, the robot hole maker is manually driven to move the visual detection unit to the vicinity of a reference feature corresponding to a local hole position on the physical sample, image acquisition of the reference feature is completed, a feature recognition algorithm is used to recognize the reference feature, and the corresponding feature and position are recorded in an offline program; thirdly, the robot hole maker is manually driven to sequentially take pictures of the hole positions to be made corresponding to the reference of the previous area on the physical sample by the visual detection unit, image acquisition of the hole position features is completed, a feature recognition algorithm is used to recognize the hole positions to be made, and the corresponding positions and poses are recorded in the offline program; the relationship between the reference and the hole position is one-to-many, that is, one reference of an area corresponds to multiple hole making positions; fourthly, the second and third steps are repeatedly executed, after detection of all the references and hole positions of the physical sample is completed, a computer in the control system generates robot machining information according to the obtained reference feature and hole position pose information of the hole to be made, automatically completes offline programming, and saves the offline machining program of the corresponding part for repeated use of the robot hole making system; fifthly, the robot hole maker calls the corresponding offline machining program to make holes on the workpiece to be machined; the visual detection unit includes an image acquisition device for detecting the hole position to be made and the reference feature, and the image acquisition device includes a monocular or / and multi-view camera and an illumination device; the visual detection unit is installed on the end effector. The control system includes a controller or a PLC motion control system and a computer. The physical sample is a workpiece prepared according to process requirements. The robot hole maker is composed of an industrial robot arm and an end effector. ​ 2. The method of claim 1, wherein, ​ 3. The method of claim 1 wherein, ​ 4. The method of claim 1 wherein, ​

Images