In-cabin component assembly method based on laser tracker and binocular vision hybrid guidance

Abstract

The method for assembling cabin components based on mixed guidance of laser tracker and binocular vision belongs to the technical field of automated assembly of cabin components of aerospace products. The present invention aims at the problem that the existing automated assembly method for cabin components cannot achieve high-precision positioning of the robot in the whole process of grasping, releasing and installing. It includes a controller controlling a six-axis industrial robot to be in a preset assembly area, so that its clamping end grasps the component to be assembled and performs image acquisition; performs feature extraction and binocular matching on the image to obtain the posture of the component to be assembled relative to the clamping end; moves the clamping end to the top of the cabin assembly port; calculates the posture deviation; solves the posture deviation as a motion control instruction of the clamping end, controls the clamping end to move relative to the target installation position, and releases the clamping end until the calculated posture deviation is within the set threshold, thereby completing the assembly of the current component to be assembled. The present invention is used for the automated assembly of cabin components.

Description

Technical Field

[0001] The invention relates to an in-cabin component assembly method based on a laser tracker and binocular vision mixed guidance, and belongs to the technical field of automatic assembly of in-cabin components of aerospace products. Background Art

[0002] Aerospace products mainly include launch vehicles, intercontinental missiles, satellites, etc. The internal structure and appearance of aerospace products are complex and varied depending on their application scenarios. Figure 3 As shown, the control cabin of some models of intercontinental missiles has the characteristics of flat appearance structure, poor openness, large number of parts, complex internal instrument component structure, and difficult assembly process.

[0003] The cabin sections to be assembled in aerospace products usually have the following characteristics:

[0004] (1) Flexible movement of cabin sections: Since the cabin sections to be assembled are small in size (approximately 1950 mm × 1730 mm × 600 mm), they can be carried by a trolley and moved flexibly within the factory, and are not limited to a fixed assembly position within the factory.

[0005] (2) Poor openness of the cabin: The opening on the top of the cabin is small, the longitudinal space inside the cabin is deep, and the interior is relatively closed. When the robot is assembling, it is easy for the laser tracker to be cut off or other ground detection equipment to fail to measure.

[0006] (3) The accuracy of the industrial robot's clamping of components cannot be guaranteed: Since the components to be assembled are placed in the positioning grooves on the component rack, the size of the positioning grooves is slightly larger than the components. When placing the components, workers cannot ensure that the components are placed in the exact center of the grooves, which will cause a component placement error of ±0.5mm. In addition, the industrial robot uses offline programming when clamping components. The component placement error will cause the actual position of the industrial robot clamping the components to be inconsistent with the theoretical position of the clamping components, which will have a great impact on the accuracy of the final component installation inside the cabin.

[0007] Current status of cabin component assembly:

[0008] At present, cabin assembly is basically done manually. This traditional manual assembly method has the following problems:

[0009] 1) Low assembly accuracy and reliability: The cabin components to be assembled, such as the laser inertial group, are heavy (30kg). Manual assembly is prone to scratches, bumps, and inaccurate positioning. The accuracy and reliability of assembly mainly rely on the skill level and experience of the operator, which is difficult to meet the requirements of aerospace products with diverse models, short development cycles, and high assembly accuracy.

[0010] 2) Low assembly efficiency: The entire assembly process relies on manual operation, which includes repetitive installation of heavy components, which takes up a lot of time in the assembly process. Since manual labor is usually unable to complete high-intensity repetitive work continuously and stably, and is limited by the operator's own quality and energy level, the consistency of assembly and delivery status is poor.

[0011] In addition, the existing general automated assembly technology cannot meet the high-precision assembly requirements of aerospace products. Automated assembly refers to an assembly technology that uses automated machinery to replace manual labor. Currently, major aerospace product manufacturers mostly use industrial robots for automated assembly. Affected by the robot's own structural properties, its absolute positioning accuracy is generally very poor. In particular, large industrial robots have an error of 3mm even in an unloaded state, which is far from meeting the accuracy requirement of 0.5mm. This seriously affects the application of automated assembly in aerospace product assembly.

[0012] For the assembly of internal components of missile control cabins, attempts are currently being made to automate high-precision assembly. For example, external measurement methods are used to improve the positioning and assembly accuracy of robots offline or online, but none of them have been able to achieve fully automated assembly of molded products. Currently, the main methods include the following:

[0013] (1) Patent publication number CN109895082A discloses a complete equipment assembly control system, which uses binocular vision to measure the characteristic points of the workpiece to be welded. The robot moves to the welding position under the action of the posture adjustment subsystem, and eliminates the influence of stress with the help of the welding stress elimination system, thereby completing the welding work. It can achieve high-precision and high-reliability assembly control requirements.

[0014] (2) The patent with publication number CN112959325B discloses a high-precision control method for collaborative processing of dual mobile robotic arms in a large scene. The C-Track binocular camera is used to measure each coordinate system externally and construct a transformation relationship, and a coordinated control method considering external uncertainty is deduced. It can meet the relative positioning accuracy requirements of the collaborative assembly process of multi-robotic arm systems for large and complex components. However, this method is mainly used for hole making of large workpieces, focusing on the collaborative control strategy of two sets of mobile robots; for two surfaces to be processed at a long distance, there will be a situation of manual station transfer using calibration blocks, and the degree of automation is low. Moreover, C-Track is used for target detection in an open environment, so it cannot meet the needs of automation and high-precision assembly of cabins with poor openness. This method is mainly concentrated in the field of welding, and the positioning accuracy requirements of the robot are relatively low. Moreover, the visual system is on one side of the workpiece, which cannot meet the needs of high-precision assembly of cabins with poor openness.

[0015] (3) The document “Robot automated assembly positioning error compensation method based on calibration model” studies the KUKAKR1000Titan robot, analyzes the various error sources of the robot, and then establishes a corresponding robot kinematic calibration model. It uses a hybrid algorithm of LM and LS to identify the error parameters of the 34 geometric errors analyzed, and brings the identified error parameters into the original robot kinematic model to improve its absolute positioning accuracy. Although this method can effectively identify the robot's kinematic geometric errors, it cannot identify non-geometric errors such as robot overload, so its positioning accuracy is limited. Moreover, this method is an offline calibration method that cannot realize feature recognition of the target. It requires manual teaching to guide the robot to perform assembly operations, and does not achieve automated assembly.

[0016] (4) "Online posture compensation method of robot based on T-MAC" installs T-MAC at the end of the robot, uses a laser tracker to measure the posture of T-MAC in real time, and then obtains the posture of the end of the robot. Then the actual posture of the end of the robot is compared with the theoretical posture it needs to achieve to obtain its posture error, and the motion compensation required by the robot at this time is obtained based on this posture error. This method can directly measure the error amount, without analyzing the error source of the robot, and without establishing a kinematic calibration model, so it has a higher positioning and assembly accuracy. However, this method relies on the T-MAC installed at the end of the robot. For the cabin to be installed with poor openness, the laser tracker may be cut off. In addition, this method relies on the accurate measurement of the cabin posture and the accurate establishment of the cabin CAD model, so it is difficult to apply to the automated assembly of internal components of the missile control cabin.

[0017] (5) “A visual positioning and grasping method based on industrial robots” installs a binocular camera at the end of the robot, and uses the “eye in hand” method to measure the reflective target on the target component, thereby obtaining the relative posture relationship between the end of the robot arm and the component. The control module drives the end to adjust the posture according to the posture feedback, thereby realizing the robot’s grasping of the target component. This method enables the robot to autonomously grasp the target component based on the measurement results of binocular vision, and the target component contains a cooperative target, and the visual detection accuracy is high. However, this method only realizes the grasping of the target component, and does not realize the complete “grasp-release” assembly process, so it does not meet the needs of the entire automated assembly system.

[0018] In summary, the current methods only achieve high-precision positioning of a certain posture feature of the robot in its workspace, or grasping the target in an open space, but there is no complete automated assembly technology for grasping, releasing, and installing. At present, there is no mature automated assembly solution suitable for the automated assembly of cabin components. Summary of the invention

[0019] In view of the problem that the existing automated assembly method for cabin components cannot achieve high-precision positioning of the robot in the entire process of grasping, releasing and installing, the present invention provides an assembly method for cabin components based on hybrid guidance of a laser tracker and binocular vision.

[0020] The present invention discloses a method for assembling components in a cabin based on a laser tracker and binocular vision hybrid guidance, which is implemented based on a controller, a six-axis industrial robot, a heavy-duty automatic cruise vehicle, a component posture measurement visual system, an assembly guidance visual system and a laser tracker; the six-axis industrial robot and the component posture measurement visual system are installed on the heavy-duty automatic cruise vehicle, and the component posture measurement visual system is used to measure the posture of the components to be assembled; the assembly guidance visual system is installed at the clamping end of the six-axis industrial robot, and is used to measure the posture of the target installation position in the cabin; the laser tracker is used to measure the posture of the cabin and the posture of the base coordinate system of the six-axis industrial robot; the method comprises: The components are placed on the component rack of the heavy-duty automatic cruise vehicle in advance; the controller controls the heavy-duty automatic cruise vehicle to approach the cabin according to the cabin posture obtained by the laser tracker and the base coordinate system posture of the six-axis industrial robot, so that the six-axis industrial robot is in the preset assembly area; the controller controls the clamping end of the six-axis industrial robot to grasp the component to be assembled according to the base coordinate system posture of the six-axis industrial robot and the position information of the component to be assembled, and performs image acquisition of the grasped component to be assembled through the component posture measurement visual system; feature extraction and binocular matching are performed according to the acquired image of the component to be assembled and the position information of the component posture measurement visual system to obtain the posture of the component to be assembled relative to the clamping end;

[0021] The controller then controls the clamping end to move to above the cabin assembly port according to the posture of the component to be assembled relative to the clamping end and the cabin posture; then obtains the posture of the target installation position relative to the clamping end through the assembly guidance vision system; calculates the posture deviation according to the posture of the component to be assembled relative to the clamping end and the posture of the target installation position relative to the clamping end; solves the posture deviation into a motion control instruction of the clamping end, and then controls the clamping end to move relative to the target installation position until the calculated posture deviation is within the set threshold, releases the clamping end, and completes the assembly of the current component to be assembled.

[0022] According to the in-cabin component assembly method based on mixed guidance of a laser tracker and binocular vision of the present invention, a coarse positioning groove is provided on the component rack, the component to be assembled is placed in the coarse positioning groove, and has a gap of at least 0.5 mm with the side wall of the coarse positioning groove; the height of the component to be assembled is higher than the depth of the coarse positioning groove.

[0023] According to the in-cabin component assembly method based on laser tracker and binocular vision hybrid guidance of the present invention, both the component posture measurement vision system and the assembly guidance vision system are binocular vision systems;

[0024] In the binocular vision system, the binocular camera is calibrated with internal and external parameters before use, as well as the camera hand-eye calibration; the six-axis industrial robot is calibrated with the base coordinate system before use; and the component rack is calibrated with position and posture before use.

[0025] According to the in-cabin component assembly method based on laser tracker and binocular vision hybrid guidance of the present invention, the six-axis industrial robot base coordinate system is calibrated to calibrate the posture relationship between the robot base coordinate system and the three target ball seat coordinate systems on the heavy-loaded automatic cruise vehicle; the six-axis industrial robot base coordinate system posture is obtained by measuring the positions of the three target ball seats on the heavy-loaded automatic cruise vehicle through a laser tracker.

[0026] According to the in-cabin component assembly method based on laser tracker and binocular vision hybrid guidance of the present invention, after the base coordinate system of the six-axis industrial robot is calibrated, multiple components to be assembled are placed in corresponding coarse positioning grooves according to predetermined positions; then the three reflective target balls of the laser tracker are sequentially installed on the three target ball seats of the cabin frame vehicle; the laser tracker is then used to obtain the cabin posture; and the posture relationship between the six-axis industrial robot and the cabin section is obtained through the base coordinate system posture of the six-axis industrial robot and the cabin posture.

[0027] According to the in-cabin component assembly method based on laser tracker and binocular vision hybrid guidance of the present invention, before assembly, the determined posture conversion relationship includes: completing the hand-eye calibration of the binocular vision system in the component posture measurement vision system according to the posture conversion relationship between the six-axis industrial robot base coordinate system and the component posture measurement vision system; completing the hand-eye calibration of the binocular vision system in the assembly guidance vision system according to the posture conversion relationship between the six-axis industrial robot end flange coordinate system and the assembly guidance vision system.

[0028] According to the in-cabin component assembly method based on laser tracker and binocular vision mixed guidance of the present invention, the posture conversion relationship also includes: using the laser tracker to measure the target ball on the component rack to obtain the posture conversion relationship between the component rack coordinate system and the laser tracker coordinate system.

[0029] According to the in-cabin component assembly method based on the mixed guidance of the laser tracker and binocular vision of the present invention, the coordinate transformation relationship used in the component assembly process includes: obtaining the posture of the laser tracker relative to the target installation position according to the posture relationship of the cabin relative to the target installation position and the posture relationship of the laser tracker relative to the cabin.

[0030] According to the in-cabin component assembly method based on laser tracker and binocular vision mixed guidance of the present invention, the coordinate transformation relationship used in the component assembly process also includes: according to the posture relationship of the laser tracker relative to the target installation position and the posture relationship of the six-axis industrial robot base coordinate system relative to the laser tracker, the posture of the six-axis industrial robot base coordinate system relative to the target installation position is obtained.

[0031] Beneficial effects of the invention: The method of the invention can realize high-precision repetitive assembly of heavy cabin internal components, overcoming the shortcomings of traditional manual assembly methods such as low assembly accuracy and reliability, low intelligence of industrial robots, and poor absolute positioning accuracy. The method of the invention can meet the assembly requirements of flexible cabin section movement, poor cabin section openness, and the inability to ensure the accuracy of industrial robot clamping components. It has the following advantages:

[0032] 1) The present invention combines the "grasping" and "releasing" of the target component. After simulation verification, the assembly positioning accuracy of the target component of 0.5mm is achieved. 2) The laser tracker is used to flexibly construct the six-axis industrial robot base coordinate system and the cabin coordinate system, and then obtain the conversion relationship between them. The functions implemented include: the laser tracker first measures and establishes the cabin coordinate system through the target ball on the cabin frame, and obtains the specific position of the cabin, which can guide the AGV carrying the industrial robot to move to the vicinity of the cabin; the laser tracker then measures the target ball fixed on the AGV, and obtains the base coordinate system of the six-axis industrial robot after conversion, and then calculates the posture relationship between the cabin and the industrial robot. Under the guidance of the posture relationship, the industrial robot clamps the internal components and moves to the top of the cabin, so that the target assembly posture appears in the field of view of the assembly guidance visual system. The use of a laser tracker to measure the posture of the robot and the cabin to be assembled can well adapt to the problem that the cabin can be flexibly moved in the factory, making the assembly system more automated and intelligent. 3) The method of the present invention uses two sets of binocular vision systems for assembly. The two sets of binocular vision are installed in the manner of "eyes on the hand" and "eyes under the hand" respectively, which can improve the accuracy of the final component installed inside the cabin: the component posture measurement vision system is fixed on the AGV in the manner of "eyes under the hand" to measure the posture relationship between the component clamped by the robot and the robot, which can solve the problem that the accuracy of the industrial robot clamping the component cannot be guaranteed; the assembly guidance vision system is fixed at the end of the robot in the manner of "eyes on the hand", and the posture deviation between the component and the cabin installation position is measured as path compensation, which can improve the assembly positioning accuracy and safety. In this way, the problem of poor openness of the cabin to be assembled and the easy interruption of the laser tracker is solved.

[0033] The method of the invention adopts a hybrid guidance mode of binocular vision and laser tracker to guide the robot for high-precision automatic assembly. When assembling the internal components of the missile control cabin, the internal components such as the laser inertial group, servo driver, integrated controller, etc. can be installed on the bracket inside the control cabin with high precision through the opening above the control cabin, and the positioning error is less than 0.5mm. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of the method for assembling cabin components based on hybrid guidance of binocular vision and laser tracker according to the present invention;

[0035] Figure 2 It is a structural block diagram of an automated assembly system used to implement the method of the present invention; Figure 3 It is a schematic diagram of the cabin section to be assembled and its frame. In the figure, A is the opening of the rear cabin, B is the control cabin (rear cabin), C is the control cabin frame, D is the target ball seat of the laser tracker, and E is the mass characteristic measurement platform. Figure 41 is a schematic diagram of the overall implementation of the automated assembly system used to implement the method of the present invention; in the figure, 1 is a six-axis industrial robot, 2 is an assembly guidance visual system, 3 is a binocular camera of the assembly guidance visual system, 4 is a vacuum suction cup fixture, 5 is a rear cabin and its frame vehicle, 6 is a laser tracker, 7 is a heavy-duty automatic cruise vehicle, 8 is a component posture measurement visual system, 9 is a component rack, 10 is a component to be assembled, 11 is an electrical cabinet, and 12 is a KRC4 control cabinet; Figure 5 It is a data flow diagram of the automated assembly system used in the present invention; Figure 6 It is a calibration diagram before the assembly system is used for the first time; Figure 7 1 is a schematic diagram of measuring the position and posture of the cabin by a laser tracker; 16 in the figure is the control cabin section, and 17 is the target sphere; Figure 8 It is a schematic diagram of the conversion relationship of each coordinate system in the method of the present invention; Fig. 9 It is a schematic diagram of coordinate system calibration; Fig.10 It is the flow chart of the target part pose measurement algorithm based on binocular vision in simulation verification; Fig.11 It is the flow chart of the algorithm for extracting the center feature of the cross-marked line in the simulation verification; Fig.12 is the image after binarization; Fig.13 Yes Fig.12 Image after skeleton extraction; Fig.14 It is the cross at the center of the cross-mark after skeleton extraction; Fig.15 This is a schematic diagram of the 9 structural templates used in the hit-miss transformation; Fig.16 This is the effect diagram of the cross-line center extraction; Fig.17 It is a schematic diagram of the errors of various points on the components and the component position errors in previous simulations; Fig.18 It is a schematic diagram of the errors of various points on the simulated cabin sections and the cabin section position errors; Fig.19 It is a schematic diagram of the installation error of each simulation. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention, but are not intended to be a limitation of the present invention.

[0037] Specific implementation method 1. Combination Figures 1 to 9As shown, the present invention provides a method for assembling components in a cabin based on mixed guidance of a laser tracker and binocular vision, which is implemented based on a controller, a six-axis industrial robot, a heavy-duty automatic cruise vehicle, a component posture measurement visual system, an assembly guidance visual system and a laser tracker; the six-axis industrial robot and the component posture measurement visual system are installed on the heavy-duty automatic cruise vehicle, and the component posture measurement visual system is used to measure the posture of the components to be assembled; the assembly guidance visual system is installed at the clamping end of the six-axis industrial robot to measure the posture of the target installation position in the cabin; the laser tracker is used to realize the measurement of the cabin posture and the measurement of the posture of the base coordinate system of the six-axis industrial robot; including placing the components to be assembled on the component rack of the heavy-duty automatic cruise vehicle in advance; the controller controls the heavy-duty automatic cruise vehicle to approach the cabin according to the cabin posture obtained by the laser tracker and the posture of the base coordinate system of the six-axis industrial robot, so that the six-axis industrial robot is in a preset assembly area; the controller controls the heavy-duty automatic cruise vehicle to approach the cabin according to the cabin posture obtained by the laser tracker and the posture of the base coordinate system of the six-axis industrial robot, so that the six-axis industrial robot is in a preset assembly area The invention relates to a six-axis industrial robot which is provided with a plurality of control components, and a plurality of control components are controlled to grasp the component to be assembled according to the position information of the component to be assembled and the position of the component to be assembled by the component posture measurement visual system, and the image of the grasped component to be assembled is collected; feature extraction and binocular matching are performed according to the collected image of the component to be assembled and the position information of the component posture measurement visual system, so as to obtain the position and posture of the component to be assembled relative to the clamping end; the controller controls the clamping end to move to above the cabin assembly port according to the position and posture of the component to be assembled relative to the clamping end and the cabin posture; the position and posture of the target installation position relative to the clamping end is obtained by the assembly guidance visual system; the position and posture deviation is calculated according to the position and posture of the component to be assembled relative to the clamping end and the position and posture of the target installation position relative to the clamping end; the position and posture deviation is solved as the motion control instruction of the clamping end, and the clamping end is controlled to move relative to the target installation position until the calculated position and posture deviation is within the set threshold, and the clamping end is released to complete the assembly of the current component to be assembled.

[0038] In order to meet the assembly requirements of high precision, high efficiency and free movement of industrial robots, the method of the present invention is based on the automated assembly system such as Figure 2 and Figure 4 As shown. Figure 2 As shown in the figure, the automated assembly system includes four subsystems: a controller, a six-axis industrial robot, a heavy-duty automatic cruise vehicle, and a 6D measurement system. The 6D measurement system is further divided into a component posture measurement vision system, an assembly guidance vision system, and a laser tracker. The controller monitors the operation status of each subsystem and the various measurement data fed back in real time, and uniformly controls and deploys the system based on the operation status and measurement data. Figure 4 As shown, in a specific implementation, the assembly system may include a six-axis industrial robot, a heavy-duty automated cruise vehicle (AGV), a component posture measurement vision system, an assembly guidance vision system, a laser tracker, a control cabinet, an electrical cabinet, a component rack, a fixture, etc.

[0039] The six-axis industrial robot 1 can be taught offline under the control of the robot control cabinet (controller), or it can make corresponding high-precision assembly movements through the robot control cabinet after receiving the motion instructions from the host computer. The AGV carries the industrial robot to the specified position under the instructions of the host computer, or it can move autonomously by remote control, thereby realizing the free movement of the robot's operating position.

[0040] Furthermore, a rough positioning groove is provided on the component rack, and the component to be assembled is placed in the rough positioning groove and has a gap of at least 0.5 mm with the side wall of the rough positioning groove; the height of the component to be assembled is higher than the depth of the rough positioning groove. Figure 4 As shown, the clamping end can use a vacuum suction cup as a clamp to complete the grasping and releasing of the component to be assembled.

[0041] In order to facilitate the robot to quickly grab the components to be assembled, the component rack is installed on the AGV. The AGV carries the components to be assembled and the robot moves continuously, which can realize the assembly of multiple compartments and improve the assembly efficiency. In order to reduce the positioning error of component placement and ensure that the robot can quickly grab the components, the component rack is designed with specific coarse positioning grooves. Figure 4 As shown, the assembly to be assembled is placed in the rough positioning groove; there is at least 0.5mm clearance between the assembly to be assembled and the side wall of the rough positioning groove, and the height is higher than the rough positioning groove, which can ensure smooth assembly placement. The worker places the assembly on the assembly rack according to the rough positioning groove, and then the robot grabs the assembly offline according to the pre-taught program.

[0042] The data flow of the entire assembly system is as follows Figure 5 As shown in the figure, the SDK development component of Daheng Camera is used as the posture measurement vision system and assembly guidance vision system. The data is uploaded to the host computer through the USB3.0 interface, so that the host computer can have the functions of setting the camera parameters and controlling the camera to collect and save photos. The host computer uses pycharmIDE and combines openCV for image processing software development to extract features of target corner points, perform binocular matching and 3D reconstruction, and calculate the posture information of the target. The host computer processes the posture information and calculates the corresponding control instructions.

[0043] The laser tracker feeds back the target ball coordinates in the laser tracker coordinate system to the host computer through TCP / IP communication. The host computer processes the data uniformly and sends control instructions (correction angles of the six-axis industrial robot and motion instructions of the AGV) to the PLC through the RS485 communication interface. The PLC uses the Profibus bus to communicate with the six-axis industrial robot and AGV. The six-axis industrial robot and AGV act as slaves and only need to receive instructions and execute them. The PLC sends angle correction parameters to the robot controller through Profibus. The controller controls each joint angle and vacuum gripper to perform corresponding actions based on these parameters. Assembly process: The components to be assembled can be placed on the component rack in order manually, and the laser tracker target ball can be moved manually to calibrate the basic posture relationship between the robot and the cabin. The assembly system then automatically completes the assembly work, such as Figure 1 shown.

[0044] Furthermore, both the component posture measurement vision system and the assembly guidance vision system are binocular vision systems; the binocular camera in the binocular vision system is calibrated for internal and external parameters before use, as well as the camera hand-eye calibration; the six-axis industrial robot is calibrated for its base coordinate system before use; and the component rack is calibrated for posture before use.

[0045] Combination Figure 6 As shown in the figure, the assembly system needs to be calibrated before the first use. In most cases, this step will not appear in the formal assembly process. The calibration required before the first use includes the calibration of the internal and external parameters of the binocular camera, the camera hand-eye calibration of the two visual systems "eyes on the hand" and "eyes under the hand", the calibration of the robot base coordinate system, and the position and posture calibration of the component rack.

[0046] It should be emphasized here that the robot base coordinate system calibration here is to calibrate the position and posture relationship between the robot base coordinate system and the three target ball seat coordinate systems installed on the AGV. Therefore, during the formal assembly, it is only necessary to measure the positions of the three target ball seats installed on the AGV through the laser tracker to obtain the position and posture of the robot base coordinate system.

[0047] Combination Figure 7 As shown in the figure, after the first calibration is completed, the worker places the assembled components on the component rack in a certain order according to the coarse positioning slots on the component rack, and then configures the components to be installed. After the component configuration is completed, the laser tracker reflective target ball is manually installed on the three target ball seats of the control cabin rack vehicle in sequence, and the laser tracker is used to obtain the position and posture of the control cabin. The host computer controls the AGV to move to the assembly position based on the acquired control cabin posture. Similarly, the target balls are manually placed on the target ball seats on the AGV in sequence, and the laser tracker is used to obtain the robot base coordinate system posture. Then the posture relationship between the robot and the control cabin section is obtained.

[0048] Furthermore, the six-axis industrial robot base coordinate system is calibrated to calibrate the posture relationship between the robot base coordinate system and the three target ball seat coordinate systems on the heavy-loaded automatic cruise vehicle; the positions of the three target ball seats on the heavy-loaded automatic cruise vehicle are measured by a laser tracker to obtain the posture of the six-axis industrial robot base coordinate system.

[0049] After the calibration of the base coordinate system of the six-axis industrial robot is completed, multiple components to be assembled are placed in the corresponding rough positioning grooves according to the predetermined positions; then the three reflective target balls of the laser tracker are installed on the three target ball seats of the cabin frame in sequence; the laser tracker is used to obtain the cabin posture; the posture relationship between the six-axis industrial robot and the cabin section is obtained through the six-axis industrial robot base coordinate system posture and the cabin posture. The robot can grab the workpiece according to the motion trajectory programmed offline in advance. The robot moves the grabbed workpiece to the component posture measurement visual system, uses the component posture measurement visual system to collect images of the assembled components, pre-processes the obtained image information, extracts features, performs binocular matching, and performs detection calculations, and finally obtains the posture relationship between the installed component and the robot's end effector.

[0050] After identifying the component posture, the robot moves to the top of the control cabin, and the assembly guidance vision system detects the target assembly posture, compares the target assembly posture with the current component posture to obtain the posture deviation, and solves the posture deviation into the robot's motion data, which is transmitted to the motion control module to drive the robot end to move, and repeat the above steps until the deviation value between the current posture and the target posture is within the set threshold. The fixture is released, and the robot grabs and loads the next workpiece, and repeats this process. After the workpiece is loaded, the robot grabs the hatch and places it in the opening, thereby completing the assembly work.

[0051] System working principle and conversion relationship of each coordinate system: According to the composition and working process of the system, the working principle of the system and conversion relationship of each coordinate system are introduced below.

[0052] Combination Figure 8 As shown, before assembly, the determined posture conversion relationship includes: according to the posture conversion relationship between the six-axis industrial robot base coordinate system and the component posture measurement vision system, the eye-under-hand hand-eye calibration of the binocular vision system in the component posture measurement vision system is completed; according to the posture conversion relationship between the six-axis industrial robot end flange coordinate system and the assembly guidance vision system, the eye-under-hand hand-eye calibration of the binocular vision system in the assembly guidance vision system is completed.

[0053] The posture conversion relationship also includes: using a laser tracker to measure the target ball on the component rack to obtain a posture conversion relationship between the component rack coordinate system and the laser tracker coordinate system.

[0054] Furthermore, the coordinate transformation relationship used in the component assembly process includes: obtaining the posture of the laser tracker relative to the target installation position based on the posture relationship of the cabin relative to the target installation position and the posture relationship of the laser tracker relative to the cabin.

[0055] The coordinate transformation relationship used in the component assembly process also includes: obtaining the posture of the six-axis industrial robot base coordinate system relative to the target installation position based on the posture relationship of the laser tracker relative to the target installation position and the posture relationship of the six-axis industrial robot base coordinate system relative to the laser tracker; obtaining the posture of the six-axis industrial robot terminal flange relative to the component to be assembled based on the posture relationship of the component posture measurement vision system relative to the component to be assembled, the posture relationship of the six-axis industrial robot base coordinate system relative to the component posture measurement vision system, and the posture relationship of the six-axis industrial robot terminal flange relative to the six-axis industrial robot base coordinate system; and then obtaining the posture of the component to be assembled relative to the target installation position based on the posture relationship of the assembly guidance vision system relative to the target installation position, the posture relationship of the six-axis industrial robot terminal flange relative to the assembly guidance vision system, and the posture relationship of the component to be assembled relative to the six-axis industrial robot terminal flange.

[0056] Many coordinate systems are needed during assembly, which involves the establishment, conversion and unification of system coordinate systems. The conversion relationship between each coordinate system is as follows: Figure 8 shown. Figure 8 The names and descriptions of the coordinate systems are shown in the following table:

[0057] Table of coordinate system names and descriptions

[0058]

[0059] Figure 8 In the figure, the dotted line between the coordinate system {I} and the coordinate system {Goal} is the deviation between the current pose of the component to be assembled and the pose of the target installation position; the solid line connecting the other coordinate systems represents the pose relationship calibrated before the first assembly, and the dotted line represents the pose relationship that needs to be measured during the actual assembly process.

[0060] Before assembly begins, the following posture relationships need to be determined:

[0061] (1) Fig. 9 As shown in the figure, the robot base coordinate system must be calibrated first: fix the laser tracker in one position, and install a laser tracker target ball at the end of the robot through an "L"-shaped tooling. Manually operate the robot to move 6 positions in the workspace. Record the readings of the robot teaching pendant and the laser tracker for each position, and use the base coordinate system calibration algorithm based on the invariant spatial position to solve the position relationship between the robot base coordinate system and the laser tracker. Install three laser tracker target spherical mounts on the AGV in advance. 1 、m 2 、m 3 , the laser tracker remains stationary, and the target balls are placed on it one by one manually. The corresponding coordinates are measured by the laser tracker, and then the robot target ball seat coordinate system {G} is constructed, thereby establishing the position and posture transformation relationship between the coordinate system {G} and the laser tracker coordinate system {Laser} Then the transformation relationship between coordinate systems {G} and {B} is constructed: In this way, even if the laser tracker is transferred, the pose of the robot's base coordinate system can be quickly obtained through the {G} coordinate system, and this pose is used as the unified reference coordinate system of the assembly system. This calibration only needs to be performed before the first assembly. (2) The pose conversion relationship between the robot's base coordinate system and the component pose measurement vision system That is, the eye-to-hand calibration in binocular vision. This calibration only needs to be performed before the first assembly. (3) The position and posture transformation relationship between the robot end flange coordinate system and the assembly guidance vision system That is, the eye-in-hand calibration in binocular vision. This calibration only needs to be performed before the first assembly. (4) The position conversion relationship between the robot base coordinate system and the robot end flange coordinate system It can be directly obtained through the robot teaching device. (5) The posture transformation relationship between the target assembly posture and the control cabin (carriage) coordinate system The posture relationship remains unchanged and can be directly obtained through the CAD drawing of the control cabin. (6) The posture transformation relationship between the workpiece frame coordinate system and the laser tracker coordinate system The target ball mounted on the workpiece holder is measured using a laser tracker. This calibration only needs to be performed before the first assembly. (7) In addition, the Zhang Zhengyou calibration method is also required to calibrate the internal and external parameter matrices of the binocular vision system, and then the binocular vision system can be used to identify the three-dimensional coordinates of the target. This calibration only needs to be performed before the first assembly. After the system calibration is completed, assembly is performed according to the assembly method of the present invention:

[0062] First, the position of the cabin can be measured using a laser tracker. The specific implementation method is as follows: The cabin section is placed on the cabin section trolley, and three laser tracker target ball seats P are installed at fixed positions on the trolley. 1 , P 2 , P 3 Therefore, the three-dimensional coordinates of the three target spheres in the {W} coordinate system (cabin coordinate system) are known. Let the coordinates of the three target spheres in the {W} coordinate system be:

[0063] W P 1 ,W P 2 , W P 3 (one)

[0064] The three-dimensional coordinates of the three target balls in the {Laser} coordinate system (laser tracker coordinate system) are measured by the laser tracker as follows:

[0065] Laser P 1 , Laser P 2 , Laser P 3 (two)

[0066] Then we can construct the following mathematical model to solve

[0067]

[0068] Where X W , Y W , Z W Represents the three-dimensional coordinates of a point in the coordinate system {W}. Laser , Y Laser , Z Laser Represents the three-dimensional coordinates of a point in the coordinate system {Laser}. R is the rotation matrix, and λ is the scale factor, which is 1 here.

[0069] The above formula can be written as:

[0070] w P = λ(t+R Laser P) (IV)

[0071] Where t is the translation vector, then, It can be expressed as:

[0072]

[0073] Construct the rotation matrix R through the antisymmetric matrix S:

[0074]

[0075] Where a, b, and c are antisymmetric matrix constants.

[0076] Then, the rotation matrix R can be expressed as:

[0077] R=(I+S)(IS) -1 (seven)

[0078] Among them, I represents the third-order unit matrix.

[0079] Then, formula (VII) can be expanded as:

[0080]

[0081] Formula (8) is the Rodriguez matrix. By solving the Rodriguez matrix, the antisymmetric matrix S is solved, and then the rotation matrix R is indirectly solved.

[0082] P 1 , P 2 Substitute into formula (4) and subtract:

[0083]

[0084] In formula (9), W P 1x is P in the {W} coordinate system 1 The value of the point in the X direction, and so on.

[0085] For the convenience of expression, let:

[0086] X W12 = W P 1x - W P 2x (ten)

[0087] That is X W12 Indicates that P in the {W} coordinate system 1 With P 2 The difference between two points in the X direction.

[0088] Formula (9) can be written as:

[0089]

[0090] Substitute equation (VII) into equation (XI):

[0091]

[0092] Further simplification yields:

[0093]

[0094] The coefficient matrix of the equation group of formula (13) is a singular matrix. Only two of the three equations are independent. Therefore, it is necessary to introduce other points to construct two or more such equation groups, and then solve the values ​​of a, b, and c.

[0095] Therefore, using the same method as above, according to P 1 With P 3 Deducing from these two points, we can get the equation group (14):

[0096]

[0097] Combining equations (13) and (14), we get the overdetermined equation:

[0098]

[0099] make:

[0100]

[0101] Then formula (15) can be written as:

[0102] b=AM (sixteen)

[0103] Solve for M using the least squares method:

[0104] M=[abc] T =(A T A) -1 A T b (Seventeen)

[0105] Substituting the solved a, b, and c into formula (8) can obtain the rotation matrix R. On this basis, substituting any point into formula (4) can obtain the translation vector, and finally the posture transformation matrix can be obtained according to formula (5)

[0106] Then the position relationship between the target position and the laser tracker coordinate system is obtained

[0107]

[0108] When the AGV is driven to move near the assembly position, the laser tracker is used to construct the coordinate system {G}, and the Here’s how:

[0109] Three target balls 1 、m 2 、m 3 Installed on AGV, use laser tracker to measure the position of three points under {Laser}: m 1 =(x 1 ,y 1 , z 1 ) T 、m 2 =(x 2 ,y 2 , z 2 ) T 、m 3 =(x 3 ,y 3 , z 3 ) T . In m 2is the origin of the coordinate system {G}, is the x direction of {G}, then the unit vector of the x direction of {G} under {Laser} is expressed as:

[0110]

[0111] n x 、n y 、n z They represent the projections of the unit vector in the X direction of the coordinate system {G} in the X, Y, and Z directions of the coordinate system {Laser}.

[0112] Likewise, It can be expressed as:

[0113]

[0114] Cross product As the positive direction of the Z axis of the coordinate system {G}, the unit vector in the Z direction of {G} is expressed as:

[0115]

[0116] Where α x , α y , α z They respectively represent the projections of the unit vector in the Z direction of the coordinate system {G} in the X, Y, and Z directions of the coordinate system {Laser}.

[0117] Cross product As the positive direction of the Y axis of the coordinate system {G}, the unit vector in the Y direction of {G} is expressed as:

[0118]

[0119] α x , α y , α z They respectively represent the projections of the unit vector in the Y direction of the coordinate system {G} in the X, Y, and Z directions of the coordinate system {Laser}.

[0120] Then, the homogeneous pose transformation matrix of the {G} coordinate system relative to the {Laser} coordinate system can be expressed as:

[0121]

[0122] According to the {G} coordinate system, the pose relationship between the {Laser} coordinate system and the {B} coordinate system can be obtained:

[0123]

[0124] The position relationship between the target position and the robot base coordinate system can be calculated

[0125]

[0126] The robot grips the component and moves it to the component posture measurement visual system, and measures the posture relationship of the assembled component relative to the visual system at this time Then the position relationship between the component and the robot end flange can be calculated

[0127]

[0128] According to the pose relationship calculated by formula (XXIV), the robot gripper moves to the top of the control cabin, and the assembly guidance vision system is used to measure the pose relationship between the target pose and the vision system. When performing actual assembly, the goal is to assemble the component {I} to the position of the target {Goal}, that is, the coordinate system {I} coincides with {Goal}. Theoretically, when the assembly is complete, Therefore, according to the above relationship, the posture deviation between the component and the target assembly posture can be calculated:

[0129]

[0130] The posture deviation The correction angle of each robot joint is calculated by the joint angle compensation model, which is then solved into the motion data of the robot arm and transmitted to the motion control module to drive the robot end to move. The above steps are repeated until the deviation between the current posture and the target posture is within the set threshold, thereby completing the assembly.

[0131] Simulation verification of assembly accuracy: CoppeliaSim is used for simulation to verify that the assembly positioning accuracy of the method of the present invention is higher than 0.5 mm:

[0132] CoppeliaSim is a dynamic robot simulator with an integrated development environment and a physical engine. It can support the simulation of visual sensors embedded in image processing, and has a complete kinematic solver that can solve the forward and inverse kinematics of any mechanism. In addition, CoppeliaSim not only has Lua embedded scripts that run independently, but also can support multiple mainstream programming languages ​​such as Python, Java, C++ through the remoteAPI interface, and the simulation scene can be customized by the user. Therefore, the present invention uses CoppeliaSim to simulate and verify the assembly accuracy of the components.

[0133] 1. Construction of simulation scene: According to the working principle of the method of the present invention, that is, the conversion relationship of each coordinate system, the objects to be simulated are constructed. In the assembly process, the laser tracker is used to determine the basic position relationship between the robot and the cabin, which belongs to rough positioning and has no effect on the assembly accuracy of the components. Therefore, the actual laser tracker is not reflected when using CoppeliaSim for component assembly simulation. In the simulation scene, it mainly includes five parts: assembly component simulation parts, cabin simulation parts, large industrial robots, component posture measurement visual systems, assembly guidance visual systems and fixtures. Each module in the scene is a collidable module driven by a physical engine. In addition, the posture information of each model can be directly obtained in the simulation software, which greatly facilitates the verification of installation accuracy and visual measurement accuracy.

[0134] When assembling internal components of aerospace vehicles, they are usually positioned by machining cross-marks. For precision machine tools, the machining accuracy of the cross-marks can be controlled within 0.01mm, which fully meets the positioning requirement of 0.5mm. Adding reflective material to the cross-marks makes it easier for the visual system to extract the center features of the marks.

[0135] The assembly simulation parts and cabin simulation parts have their own assembly coordinate systems and cabin coordinate systems, each with four cross-mark feature points for positioning. The theoretical three-dimensional coordinates of the center points of the four cross-marks relative to their respective coordinate systems can be extracted through SolidWorks 3D design software. When modeling, a "Dummy" in CoppeliaSim is placed at the center of each cross mark as position feedback, which is then used to determine the three-dimensional deviation of the center position of the cross mark measured by the visual system. The cabin simulation parts were simplified during modeling, and useless outer shells were removed, leaving only the cabin bottom plate part that can be observed by the visual system. The simplification of the model can speed up the simulation, but it does not affect the authenticity and final accuracy of the simulation.

[0136] 2. Simulation operation architecture:

[0137] As a simulator that can be customized by customers, CoppeliaSim has two main control modes: one is to directly control by the embedded Lua script inside CoppeliaSim, which is part of the simulation scene and loaded together with the model. The other control mode is to communicate with external applications through the remoteAPI interface, which supports C++, Python, Matlab, Java, and Lua.

[0138] This implementation adopts a hybrid control mode for simulation, using the romoteAPI interface to extend Python as the client, and the Lua script in CoppeliaSim as the server. The Python client processes the image and various posture data, and sends the posture of the robot end to the server. After receiving the client's instructions, the Lua server uses the kinematic solver to calculate the rotation angle of each joint of the robot, and finally controls the robot to move.

[0139] The server side mainly implements the following three functions by setting relevant solvers and writing sysCall_thread scripts:

[0140] 1. Establish communication with romoteAPI and Python client. 2. Use kinematic solver to solve the inverse kinematics of the robot. 3. Use sim.rmlMoveToPosition and other methods according to the server's instructions to realize the robot's movement, component gripping and release.

[0141] The client mainly implements the following functions by writing Python code:

[0142] 1. Establish romoteAPI to communicate with the Lua server; 2. Write the robot_move method to send the robot end position command to the server; 3. Write the vision_get_on and vision_get_ground methods to obtain the left and right cameras of the assembly guidance vision system and the left and right cameras of the component pose measurement vision system; 4. Write the image_process_on and image_process_ground methods to preprocess, extract features, perform stereo matching, and calculate the target pose of the images obtained by the two vision systems; 5. Write the grab and release methods to control the simulation of grabbing and releasing components; 6. Write the error_process method to obtain and process the pose deviation of component installation.

[0143] The overall operation architecture of the simulation is described as follows:

[0144] 1) First, the Python client initiates the robot motion command through the robot_move method, transmits the command to the Lua server through the API interface, and the sysCall_thread responds to the command to control the robot to move above the component to be assembled. 2) After the robot movement is completed, the Python client sends a grab command through the grab method, making the component to be assembled a sub-object of the last joint at the end of the robot, thereby completing the Lua server-side control of the robot to grab. 3) Similarly, after the grab is completed, the Python client initiates the robot motion command through the robot_move method, and the Lua server thread responds to the motion command to control the robot to move in front of the posture measurement vision system, so that the grasped component is within the field of view of the posture measurement vision system. 4) At this time, the Python client sends a command to obtain the image of the posture measurement vision system through the vision_get_ground method, and the main script of the Lua server responds to the command to read the images of the left and right cameras back to the Python client through the API interface. The client receives and saves the image data. 5) On the Python client, the image_process_ground method reads the image data just saved from the posture measurement vision system. The images of the left and right cameras are respectively processed by image preprocessing, feature extraction, and stereo matching algorithms. The Rodriguez matrix and least squares method are used to measure the component pose, and the pose of the component relative to the end of the robot is calculated according to formula (25). 6) After the image processing of the posture measurement vision system is completed, the Python client initiates a motion command through the robot_move method, and the Lua server thread responds to control the robot to move above the cabin. Make the four cross marks of the cabin within the field of view of the assembly guidance vision system. 7) Similarly, the Python client sends a command to obtain the image of the assembly guidance vision system through the vision_get_on method, and the main script of the Lua server responds, reading the images of the left and right cameras back to the Python client and saving them. 8) After reading the image of the assembly guidance vision system, the Python client calculates the image obtained in step 5) Bring it into the image_process_on method to process the images of the left and right cameras and calculate the target pose of the robot end 9) The Python client uses the robot_move method to move the target position It is sent to the Lua server through the API interface. The Lua server calculates the compensation angle of each joint angle through the kinematic solver according to the target posture, and controls the robot to move to the target posture. 10) After the robot reaches the target posture, the Python client sends a release command, and the Lua server releases the sub-object restriction of the component to complete the release action. 11) After the component is released, the Python client sends a command through the error_process function to read the component posture, compares the current component posture with the theoretical posture, and calculates the installation posture error of the component.

[0145] 3. Image processing algorithm: In simulation, the image_process_on and image_process_ground methods need to process the left and right camera images to calculate the target's position and posture. The core algorithms are the same, such as Fig.10 As shown in the figure, the three-dimensional coordinates of the center of the cross marks on each target part are measured, and the position and posture of the target part are solved using the Rodriguez matrix and the least squares method.

[0146] Image preprocessing: By preprocessing the simulated image, useless noise in the image can be removed in advance, making it easier to detect and segment the target image, thereby improving the accuracy and robustness of feature extraction. In the method of the present invention, since the light source in the field of view is stable and the brightness difference between the target crosshairs and the background is large, the following preprocessing process is adopted:

[0147] First, grayscale the image and reduce the RGB three-channel image to one channel to reduce the amount of data and improve the computing efficiency. Then perform median filtering to remove useless scattered points similar to salt and pepper noise. Since the brightness difference between the target crosshairs and the background is large, the threshold can be found well by using the adaptive threshold binarization method. As shown in formula (XXVII), the pixels in the image that are less than the threshold are set to 0, and the pixels that are greater than the threshold are set to 255 (8-bit depth image).

[0148]

[0149] Cross-mark center feature extraction: After obtaining the binary image, the following can be used Fig.11 The processing flow shown in the figure extracts the center point of each cross mark: 1) Image skeleton extraction: Fig.12 As shown in the figure, in the binarized image, each engraved line is composed of a segment of thin and long pixels with a gray value of 255. Then the central axis of each engraved line can be well obtained through skeleton extraction, and the image contour with a certain width can be transformed into a skeleton with a width of only one pixel by successive thinning (successive erosion to remove the edge), as shown in the figure. Fig.13 shown.

[0150] 2) Hit-miss transformation: The formula of hit-miss transformation is shown in formula (XXVIII), where A is the object to be searched and B is the image to be searched. B is the structural element template, and the target with the same pixel arrangement features as B is searched from the image, such as a single pixel, cross or vertical features in particles, right-angle edges or other user-defined features. During calculation, the value of the center pixel will be set to 1 only when the structural element is exactly the same as the image area it covers, otherwise it is set to 0.

[0151]

[0152] Where W is a small window template slightly larger than B, and D is a local background subset of W.

[0153] Since each cross-marked line after skeleton extraction is a pixel strip with a width of one pixel, the center of the cross-marked line is generally a cross composed of four pixels, such as Fig.14 shown.

[0154] At different observation angles, the pixel lines extracted from the skeleton will have certain deviations, and not necessarily be a cross every time. Therefore, the 9 structural element templates shown in 15 can be used to perform a hit-miss transformation on the image after skeleton extraction, and the point sets obtained by hitting each template are combined into a total point set. However, at some relatively bad observation angles (where the observation angle is greatly offset), the pixel lines extracted from the skeleton have large deviations, and using Fig.15 When matching the structural template shown in the figure, multiple pixels will be extracted from the center of the same cross-marked line. These pixels are very close, and their distance is generally no more than two pixels. These pixels can be considered as the center of the cross-marked line. Therefore, it is necessary to delete the adjacent points within 2 pixels and only keep one. The effect after the cross-marked line center feature extraction is as follows Fig.16 As shown, the point at the intersection of the cross is the center point extracted from the original image.

[0155] Calculation of three-dimensional coordinates of feature points: The camera model is established using the usual pinhole imaging method. In order to simulate the selection of real cameras and lenses as much as possible, in the CoppeliaSim simulation environment, the resolution of the visual sensor is set to 3750×3750, the working distance is 0.533m, the field of view angle is 42.48°, the depth information and Packet1 are ignored, and the focus blur option is turned on.

[0156] According to the settings, the camera's intrinsic parameters can be calculated:

[0157]

[0158] Where f x is the ratio of the camera focal length to the horizontal size of the pixel, fy is the ratio of the camera focal length to the vertical size of the pixel, u 0 is the horizontal center of the photosensor, υ 0 It is the vertical center of the photosensor.

[0159] According to the camera selection, the size of each pixel of the camera is dx=2.5μm. Among them:

[0160]

[0161] The calculated focal length of the lens is f=25mm, which is consistent with the actual focal length of the fixed-focus lens.

[0162] Using the above data, we can calculate that the field of view size is 200mm×200mm. For a resolution of 3750×3750, the accuracy of each pixel is 0.053mm.

[0163] Then, the three-dimensional coordinates (X, Y, Z) of each feature point in the camera coordinate system can be calculated by the following formula:

[0164]

[0165] Where x l ,y l 、x r and r are the horizontal and vertical coordinates of the feature points of the left and right images in the pixel coordinate system. l is the focal length of the left camera, f r is the focal length of the right camera, T x is the value of the right camera relative to the left camera in the X direction, T z is the value of the right camera relative to the left camera in the Z direction, f r is the value of the right camera relative to the left camera in the Y direction. Among them, R is the relative posture relationship between the left and right cameras, which can be expressed by the following formula:

[0166]

[0167] Fitting the target part pose: Now that the three-dimensional coordinates of the four feature points in the camera coordinate system {C} and the three-dimensional coordinates in the target coordinate system {T} are known, the Rodriguez matrix can be constructed, and equation (15) can be upgraded to equation (16). The pose transformation matrix of the target coordinate system {T} relative to the camera coordinate system {C} can be obtained by solving it using the least squares method.

[0168]

[0169] Analysis of simulation experiment results: According to the above simulation process and algorithm, simulation experiments were conducted jointly on the CoppeliaSim simulation platform and Pycharm IDE. Starting from different initial positions, 17 simulation experiments were conducted. The component position error measured by the component posture measurement visual system and the error curves of the four points on the component are shown in the figure. Fig.17 The error curves of the cabin position error and the four points on the cabin measured by the assembly guidance vision system are shown in Fig.18 shown.

[0170] In 17 simulations, the mean component position error measured by the component posture measurement visual system was 0.161mm, the maximum was 0.437mm, the minimum was 0.034mm, and the variance was 0.014mm. The mean cabin position error measured by the assembly guidance visual system was 0.133mm, the maximum was 0.237mm, the minimum was 0.059mm, and the variance was 0.003mm.

[0171] Whether from the perspective of graphics or variance, it can be seen that the assembly guidance vision system is more stable and has a smaller error fluctuation range than the component posture measurement vision system. The reason is that the cabin is fixed on the ground, so the only variable is that the shooting position of the assembly guidance vision system is slightly different each time; while the ground camera not only has the variable of different shooting positions, but also has the variable of different grasping positions each time the robot grasps the component, so the error fluctuation of the component posture measurement vision system in feature point detection is larger.

[0172] The component installation error data obtained from 17 simulations are shown in the table below.

[0173] Table 5-2 Component installation error simulation data

[0174]

[0175] Fig.19 It is the installation error of 17 simulation components. The mean of the installation error is 0.359mm, the maximum is 0.486mm, the minimum is 0.142mm, and the variance is 0.011mm. The error is less than 0.5mm, which meets the requirements of installation accuracy and stability.

[0176] Although the invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention.

Claims

1. A method for assembling components in a cabin based on hybrid guidance of a laser tracker and binocular vision, which is implemented based on a controller, a six-axis industrial robot, a heavy-duty automatic cruise vehicle, a component posture measurement visual system, an assembly guidance visual system and a laser tracker; the six-axis industrial robot and the component posture measurement visual system are installed on the heavy-duty automatic cruise vehicle, and the component posture measurement visual system is used to measure the posture of the components to be assembled; the assembly guidance visual system is installed at the clamping end of the six-axis industrial robot, and is used to measure the posture of the target installation position in the cabin; the laser tracker is used to realize the measurement of the cabin posture and the measurement of the six-axis industrial robot's base coordinate system posture; it is characterized in that include, Place the components to be assembled on the component rack of the heavy-loaded automatic cruise vehicle in advance; The controller controls the heavy-loaded automatic cruise vehicle to approach the cabin according to the cabin posture obtained by the laser tracker and the six-axis industrial robot's base coordinate system posture, so that the six-axis industrial robot is in the preset assembly area; The controller controls the gripping end of the six-axis industrial robot to grasp the component to be assembled according to the position information of the six-axis industrial robot base coordinate system and the component to be assembled, and collects images of the grasped component to be assembled through the component position measurement visual system; Performing feature extraction and binocular matching based on the collected image of the component to be assembled and the position information of the component posture measurement visual system to obtain the posture of the component to be assembled relative to the clamping end; The controller then controls the clamping end to move above the cabin assembly opening according to the posture of the assembly to be assembled relative to the clamping end and the cabin posture; Then, the position and posture of the target installation position relative to the clamping end is obtained through the assembly guidance vision system; the position and posture deviation is calculated according to the position and posture of the assembly to be assembled relative to the clamping end and the position and posture of the target installation position relative to the clamping end; The posture deviation is solved as the motion control instruction of the clamping end, and then the clamping end is controlled to move relative to the target installation position until the calculated posture deviation is within the set threshold, and the clamping end is released to complete the assembly of the current component to be assembled.

2. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 1 is characterized in that: The component rack is provided with a rough positioning groove, and the component to be assembled is placed in the rough positioning groove and has a gap of at least 0.5 mm with the side wall of the rough positioning groove; the height of the component to be assembled is higher than the depth of the rough positioning groove.

3. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 1 or 2, characterized in that: Both the component posture measurement vision system and the assembly guidance vision system are binocular vision systems; In the binocular vision system, the binocular camera is calibrated with internal and external parameters before use, as well as the camera hand-eye calibration; The six-axis industrial robot base coordinate system is calibrated before use; The component rack is calibrated before use.

4. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 3 is characterized in that: The six-axis industrial robot base coordinate system calibration is to calibrate the posture relationship between the robot base coordinate system and the three target ball seat coordinate systems on the heavy-load automatic cruise vehicle; the positions of the three target ball seats on the heavy-load automatic cruise vehicle are measured by a laser tracker to obtain the six-axis industrial robot base coordinate system posture.

5. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 4 is characterized in that: After the six-axis industrial robot base coordinate system is calibrated, multiple components to be assembled are placed in the corresponding rough positioning grooves according to the predetermined positions; Then, the three reflective target balls of the laser tracker are sequentially mounted on the three target ball seats of the cabin frame; and the position and posture of the cabin are obtained by using the laser tracker; The posture relationship between the six-axis industrial robot and the cabin section is obtained through the posture of the six-axis industrial robot base coordinate system and the cabin posture.

6. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 5 is characterized in that: Before assembly, the determined posture transformation relationships include: According to the posture conversion relationship between the six-axis industrial robot base coordinate system and the component posture measurement visual system, the eye-hand calibration of the binocular vision system in the component posture measurement visual system is completed; According to the posture transformation relationship between the six-axis industrial robot end flange coordinate system and the assembly guidance vision system, the eye-on-hand hand-eye calibration of the binocular vision system in the assembly guidance vision system is completed.

7. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 6 is characterized in that: The posture transformation relationship also includes: The target ball on the component rack is measured using a laser tracker to obtain the position and posture transformation relationship between the component rack coordinate system and the laser tracker coordinate system.

8. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 7 is characterized in that: The coordinate transformation relationships used in the component assembly process include: The position and posture of the laser tracker relative to the target installation position is obtained according to the position and posture relationship of the cabin relative to the target installation position and the position and posture relationship of the laser tracker relative to the cabin.

9. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 8, characterized in that: The coordinate transformation relationships used in the component assembly process also include: According to the posture relationship of the laser tracker relative to the target installation position and the posture relationship of the six-axis industrial robot base coordinate system relative to the laser tracker, the posture of the six-axis industrial robot base coordinate system relative to the target installation position is obtained.

10. The method for assembling cabin components based on hybrid guidance of laser tracker and binocular vision according to claim 9, characterized in that: The coordinate transformation relationships used in the component assembly process also include: The posture of the end flange of the six-axis industrial robot relative to the component to be assembled is obtained according to the posture relationship of the component posture measurement visual system relative to the component to be assembled, the posture relationship of the six-axis industrial robot base coordinate system relative to the component posture measurement visual system, and the posture relationship of the end flange of the six-axis industrial robot relative to the six-axis industrial robot base coordinate system; Then, based on the posture relationship of the assembly guidance vision system relative to the target installation position, the posture relationship of the end flange of the six-axis industrial robot relative to the assembly guidance vision system, and the posture relationship of the component to be assembled relative to the end flange of the six-axis industrial robot, the posture of the component to be assembled relative to the target installation position is obtained.

Images