A door assembly guidance method and system based on high-precision positioning 3D camera

Abstract

The present invention relates to the field related to machine vision technology, and discloses a door assembly guidance method and system based on a high-precision positioning 3D camera. By adopting the steps of point cloud conversion, point cloud fusion, high-precision point cloud matching, secondary photo adjustment and the like, higher-precision positioning guidance is achieved, which helps to realize the automatic accompanying assembly of doors on the automobile assembly production line, effectively solving the problem in the prior art that the automobile assembly production line is insufficiently accurate and difficult to assemble high-precision doors, and at the same time efficiently replaces the traditional technical solution of manual assembly.

Description

Technical Field

[0001] The present invention relates to the field related to machine vision technology, and in particular to a vehicle door assembly guidance method and system based on a high-precision positioning 3D camera. Background Art

[0002] In the automobile assembly line, the installation of car doors needs to be carried out as the car body moves. The assembly of car doors on the assembly line is currently mainly done manually. Workers can flexibly adjust the operation according to actual conditions, which is particularly useful for complex or customized tasks. The flexibility of the human body itself can achieve high-precision adaptation to environmental changes and can handle irregularly shaped parts or unconventional working environments.

[0003] As for automated mechanical equipment, in order to realize automatic accompanying assembly of car doors, that is, the industrial robot grabs the car door and moves simultaneously with the car body to install the car door during the movement, it is necessary to first realize high-precision positioning and guide the robotic arm, and achieve a positioning accuracy of at least 0.5mm. Due to the limitations of the robotic arm accuracy, hand-eye calibration accuracy, and the measurement range and accuracy of the 3D camera, the traditional method of guiding the robotic arm with a 3D camera is difficult to achieve the required accuracy. Summary of the invention

[0004] The object of the present invention is to provide a door assembly guidance method and system based on a high-precision positioning 3D camera to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A door assembly guidance method based on a high-precision positioning 3D camera, comprising:

[0007] Point cloud data is collected based on the preset dual 3D cameras, and point cloud fusion modeling is performed based on the collected point cloud data to obtain the conversion coordinate data of the hinge and body parts;

[0008] The spatial changes are obtained by point cloud registration to fit the nearest point iterative fine registration objective function. The point cloud registration method is based on a coarse-to-fine registration strategy. The objective function is used to improve the registration accuracy and convergence speed.

[0009] Performing real-time moving object monitoring to obtain a position and posture change of a set of hinged doors relative to a vehicle body portion, and generating an assembly guidance scheme based on the position and posture change;

[0010] The installation is guided according to the assembly guidance plan, and the position and posture data before assembly are verified through secondary image data acquisition to correct the assembly posture, update the coordinate data and guide the completion of the assembly.

[0011] As a further solution of the present invention: the step of collecting point cloud data based on the preset dual 3D cameras and performing point cloud fusion modeling based on the collected point cloud data to obtain the transformation coordinate data of the hinge and the body part specifically includes:

[0012] Data collection of the upper and lower hinged body and door parts is performed using dual 3D cameras, and hand-eye calibration is performed on a pair of the 3D cameras;

[0013] Guide the robot to grab the door to the assembly point and record it, collect data through dual 3D cameras, obtain the point cloud data of the upper and lower hinged body and door parts at the current assembly point and record it;

[0014] Based on the hand-eye calibration results, the point cloud data of the body and door at the assembly point in the camera coordinate system are converted to the robot tool coordinate system to obtain the converted coordinate data of the body and door parts of the upper and lower hinges in the robot tool coordinate system.

[0015] As a further solution of the present invention: when the step of obtaining spatial changes by point cloud registration to fit the nearest point iterative precise registration objective function is executed;

[0016] The closest point iteration uses the distance from point to point or from point to plane as the objective function, and the distance can only be zero when the transformed source point coincides with the target point or falls on the plane determined by the target point and its normal vector, so the objective function is:

[0017] E(T)=∑ (p，q)∈K ((p-Tq)·(n p +n q )), where: T represents spatial transformation, E represents objective function, p represents target point, q represents source point, np represents target point normal vector, nq represents source point normal vector, and K represents the set of corresponding points in source point cloud and target point cloud.

[0018] As a further solution of the present invention, the steps of obtaining the position and posture change of a set of hinged doors relative to the vehicle body specifically include:

[0019] Based on the fused registration model, point cloud registration is performed separately to obtain the position and posture data of the car body and door parts in the robot tool coordinate system;

[0020] The mobile robot installs the door to the car body, records the assembly completion position and repeatedly obtains position and posture data through dual 3D cameras.

[0021] As a further solution of the present invention: the steps of guiding the installation according to the assembly guidance scheme, verifying the position and posture data before assembly by secondary image data acquisition to correct the assembly posture, updating the coordinate data and guiding the completion of the assembly include:

[0022] generating a guidance control signal to guide the vehicle door to the assembly position through the preparation installation position and performing image data acquisition before the vehicle door moves;

[0023] Perform secondary image data acquisition at the location to be assembled to obtain the current position and posture data for high-precision positioning correction, and calculate and update the guidance control data based on the correction results;

[0024] Based on the guidance control data, the vehicle door is guided from the to-be-assembled position to the installation completion position to complete the assembly process.

[0025] The embodiment of the present invention aims to provide a door assembly guidance system based on a high-precision positioning 3D camera, comprising:

[0026] A data acquisition module is used to collect point cloud data based on a preset dual 3D camera, and to perform point cloud fusion modeling based on the collected point cloud data to obtain the conversion coordinate data of the hinge and the body part;

[0027] A registration processing module is used to obtain spatial changes through point cloud registration to fit the nearest point iterative fine registration objective function. The point cloud registration method is based on a coarse-to-fine registration strategy. The objective function is used to improve the registration accuracy and convergence speed.

[0028] A motion monitoring module, used for real-time monitoring of moving objects to obtain the position and posture changes of a set of hinged doors relative to the vehicle body, and generating an assembly guidance scheme based on the position and posture changes;

[0029] The posture correction module is used to guide the installation according to the assembly guidance plan, verify the position and posture data before assembly through secondary image data acquisition, correct the assembly posture, update the coordinate data and guide the completion of the assembly.

[0030] As a further solution of the present invention: the data acquisition module includes:

[0031] A calibration unit, used to collect data of the upper and lower hinged body and door parts through dual 3D cameras, and perform hand-eye calibration on the pair of said 3D cameras;

[0032] The recording unit is used to guide the robot to grab the door to the assembly point and record it. It collects data through dual 3D cameras, obtains the point cloud data of the upper and lower hinged body and door parts at the current assembly point and records it.

[0033] The conversion unit is used to convert the point cloud data of the vehicle body and the door at the assembly point in the camera coordinate system to the robot tool coordinate system based on the hand-eye calibration result, so as to obtain the conversion coordinate data of the vehicle body and the door part of the upper and lower hinges in the robot tool coordinate system.

[0034] As a further solution of the present invention: when the registration processing module is executed, it includes:

[0035] The objective function selection unit is used for the nearest point iteration. The distance between point and point or point and plane is used as the objective function. The distance between the transformed source point and the target point can be zero only when the source point coincides with the target point or falls on the plane determined by the target point and its normal vector. Therefore, the objective function is:

[0036] E(T)=∑ (p，q)∈K ((p-Tq)·(n p +n q )), where: T represents spatial transformation, E represents objective function, p represents target point, q represents source point, np represents target point normal vector, nq represents source point normal vector, and K represents the set of corresponding points in source point cloud and target point cloud.

[0037] As a further solution of the present invention: the motion monitoring module includes:

[0038] A point cloud registration unit is used to perform point cloud registration based on the fused registration model to obtain the position and posture data of the vehicle body and door parts in the robot tool coordinate system;

[0039] The posture recording unit is used for the mobile robot to install the door to the car body, record the assembly completion position and repeatedly obtain the position and posture data through dual 3D cameras.

[0040] As a further solution of the present invention: the posture correction module includes:

[0041] a positioning verification unit, for generating a guidance control signal to guide the door to the assembly position through the preparation installation position and to collect image data before the door moves;

[0042] A secondary correction unit is used to collect secondary image data at the position to be assembled to obtain current position and posture data for high-precision positioning correction, and to calculate and update guidance control data based on the correction results;

[0043] The assembly guide unit is used to guide the vehicle door through the to-be-assembled position to the installation completion position based on the guidance control data to complete the assembly process.

[0044] Compared with the prior art, the beneficial effects of the present invention are: by adopting point cloud conversion, point cloud fusion, high-precision point cloud matching, secondary photo adjustment and other steps, higher-precision positioning guidance is achieved, which helps to realize the automatic assembly of car doors on the automobile assembly production line, and effectively solves the problem of insufficient precision of automobile assembly production lines in the prior art that it is difficult to assemble high-precision car doors, and at the same time efficiently replaces the traditional technical solutions of manual assembly. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The flowchart is a method for guiding the assembly of vehicle doors based on a high-precision positioning 3D camera.

[0046] Figure 2 The present invention is a flowchart of the steps of obtaining conversion coordinate data in a door assembly guidance method based on a high-precision positioning 3D camera.

[0047] Figure 3 The block diagram of a door assembly guidance system based on a high-precision positioning 3D camera. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0050] like Figure 1 The present invention provides a door assembly guidance method based on a high-precision positioning 3D camera, which includes the following steps:

[0051] S10, collecting point cloud data based on the preset dual 3D cameras, and performing point cloud fusion modeling based on the collected point cloud data to obtain conversion coordinate data of the hinge and the body part;

[0052] S20, obtaining spatial changes through point cloud registration to fit the nearest point iterative fine registration objective function, wherein the point cloud registration method is based on a coarse-to-fine registration strategy, and the objective function is used to improve the registration accuracy and convergence speed;

[0053] S30, monitoring the moving object in real time to obtain a position and posture change of a set of hinged doors relative to the vehicle body, and generating an assembly guidance scheme based on the position and posture change;

[0054] S40, guiding the installation according to the assembly guidance plan, verifying the position and posture data before assembly through secondary image data acquisition, correcting the assembly posture, updating the coordinate data and guiding the completion of the assembly.

[0055] In this embodiment, a door assembly guidance method based on a high-precision positioning 3D camera is provided. By adopting the steps of point cloud conversion, point cloud fusion, high-precision point cloud matching, secondary photo adjustment, etc., higher-precision positioning guidance is achieved, which helps to realize the automatic accompanying assembly of doors on the automobile assembly production line, and effectively solves the problem of insufficient precision of automobile assembly production lines in the prior art that it is difficult to assemble doors with high precision, and at the same time efficiently replaces the traditional technical solutions of manual assembly; specifically, in the automobile assembly production line, the installation of doors needs to be carried out as the car body moves, and the assembly of doors on the assembly production line is currently mainly carried out manually. To realize the automatic accompanying assembly of doors, that is, the industrial robot grabs the door and moves with the car body at the same time, and installs the door during the movement, first To achieve high-precision positioning and guiding the robotic arm, a positioning accuracy of at least 0.5mm must be achieved. Due to the limitations of the robotic arm accuracy, hand-eye calibration accuracy, and the measurement range and accuracy of the 3D camera, the traditional method of guiding the robotic arm with a 3D camera is difficult to achieve the required accuracy. The purpose of the present invention is to provide a method for high-precision positioning and guiding door assembly with a 3D camera. During the door assembly process on the final assembly line, the body and the door are in motion, and the door and the body are connected by upper and lower hinges. The 3D camera needs to simultaneously capture the body and door parts of the upper and lower hinges to obtain the position and posture of the body and the door, and guide the robotic arm to install the door on the body. It is necessary to ensure that the upper and lower hinges can be accurately combined at the same time, and the gap after the hinges are combined is less than 0.1mm.

[0056] like Figure 2 As shown, as another preferred implementation of the present invention, the step of collecting point cloud data based on the preset dual 3D cameras and performing point cloud fusion modeling based on the collected point cloud data to obtain the transformation coordinate data of the hinge and the body part specifically includes:

[0057] S11, collecting data of the vehicle body and door parts of the upper and lower hinges through dual 3D cameras, and performing hand-eye calibration on the pair of 3D cameras;

[0058] S12, guiding the robot to grab the door to the point to be assembled and record it, collecting data through dual 3D cameras, obtaining point cloud data of the upper and lower hinged body and door parts at the current assembly point and recording it;

[0059] S13, based on the hand-eye calibration result, the point cloud data of the vehicle body and the door at the assembly point in the camera coordinate system is converted to the robot tool coordinate system to obtain the converted coordinate data of the vehicle body and the door part of the upper and lower hinges in the robot tool coordinate system.

[0060] In this embodiment, since the two hinges on the car door are far apart, if only a single 3D camera is used to shoot the hinge at one end for guidance, the slight angle error will be magnified due to the lever effect, and the accuracy cannot meet the requirements. In this embodiment, two 3D cameras are used to shoot the positions of the upper and lower hinges respectively, and the camera poses obtained by hand-eye calibration are used to fuse the point cloud data of the two cameras, and then a 3D point cloud model is established; specifically, three steps are included. First, two 3D structured light cameras are installed at the end of the industrial robot arm, which are used to shoot the body part and door part of the upper and lower hinges respectively, and the hand-eye calibration is performed on the two cameras respectively. The calibration results are recorded as tool H cam1 , tool H cam2 ; Where H represents the pose transformation matrix, tool represents the robot tool coordinate system, cam1 and cam2 represent the coordinate systems of camera 1 and camera 2 respectively; Second, the robot grabs the car door and moves it to the assembly position, which is recorded as position A. At this time, the upper and lower hinged door parts are located about 2 cm above the corresponding hinged body parts. The car door moves downward to ensure that the hinged door part and the body part are combined. 3D cameras 1 and 2 take pictures, and the point clouds of the upper and lower hinged door parts and the body part are obtained at the same time, which are recorded as:

[0061] {( cam1 P obj1上 ) i},{( cam2 P obj1下 ) i},{( cam1 P obj2上 ) i},{( cam2 P obj2下 ) i}, {} represents a point cloud set, where obj1 top and obj1 bottom represent the upper and lower hinged door parts, and obj2 top and ojb2 top represent the upper and lower hinged body parts; thirdly, the point cloud data of the hinged door part and the body part in the camera coordinate system are converted to the robot tool coordinate system using the hand-eye calibration result, that is:

[0062] ( tool P obj1上 ) i = tool H cam1 *( cam1 P obj1上 ) i

[0063] ( tool P obj1下 ) i = toolH cam2 *( cam2 P obj1下 ) i

[0064] ( tool P obj2上 ) i = tool H cam1 *( cam1 P obj2上 ) i

[0065] ( tool P obj2下 ) i = tool H cam2 *( cam2 P obj2下 ) i ,

[0066] Where P represents any point in the point cloud, and i represents the number of the point; then:

[0067] {( tool P obj1 ) i}={( tool P obj1上 ) i}∪{( tool P obj1下 ) i}

[0068] {( tool P obj2 ) i}={( tool P obj2上 ) i}∪{( tool P obj2下 ) i},

[0069] Thus, we obtain the point cloud of the door part and the body part of the upper and lower hinges in the robot tool coordinate system;

[0070] use{( tool P obj1 ) i}and{( tool P obj2 ) i} as the fused registration model, respectively denoted as M obj1 and M obj2 ;

[0071] use{( tool P obj1上 ) i}and{( tool P obj2上 ) i} as the registration model for rough positioning, respectively denoted as M obj1上 and M obj2上 ;

[0072] Or use {( tool P obj1下 ) i}and{( tool P obj2下 ) i} as the registration model for rough positioning, respectively denoted as M obj1下 and M obj2下 ;

[0073] Similarly, the hand-eye calibration results can be used to convert all point clouds captured by 3D cameras 1 and 2 into point cloud scenes in the robot tool coordinate system. Markers 1 and 2 are used to correspond to the scenes where obj1 and obj2 are located, and the point cloud scenes are recorded as: S 1上 , S 1下 , S 2上 , S 2下 , the fused scene is recorded as S 1 and S 2 .

[0074] As another preferred implementation of the present invention, the step of obtaining spatial changes by point cloud registration to fit the nearest point iterative precise registration objective function is performed;

[0075] The closest point iteration uses the distance from point to point or from point to plane as the objective function, and the distance can only be zero when the transformed source point coincides with the target point or falls on the plane determined by the target point and its normal vector, so the objective function is:

[0076] E(T)=∑ (p，q)∈K ((p-Tq)·(n p +n q )), where: T represents spatial transformation, E represents objective function, p represents target point, q represents source point, np represents target point normal vector, nq represents source point normal vector, and K represents the set of corresponding points in source point cloud and target point cloud.

[0077] In this embodiment, by registering the point cloud S in the coordinate system of the robot tool 1 To M obj1 or S 2 To M obj2 You can get space transformation tool H obj1 and tool H obj2 , but there are some challenges: First, the point cloud converted to the robot tool coordinate system, S1上 and S 1下 , S 2上 and S 2下 are far apart, this shape distribution leads to direct registration S 1 To M obj1 or S 2 To M obj2 It is very difficult. Using traditional global matching will lead to unreliable registration results. Therefore, we first use S 1上 Align to M obj1上 (or S 1下 Align to M obj1下 ), S 2上 Align to M obj2上 (or S 2下 Align to M obj2下 ), obtain a rough spatial transformation, and then transform the entire point cloud S 1 To M obj1 , S 2 To M obj2 Use iterative closest point (ICP) for precise registration to obtain a higher-precision spatial transformation; secondly, when performing iterative closest point (ICP) precise registration, how to obtain the highest possible registration accuracy in the shortest possible processing time. In the process of accompanying installation, processing speed and accuracy are equally important, and the ICP algorithm needs to converge as quickly as possible. In addition, because the transformed source point needs to coincide with the target point or fall on the plane determined by the target point and its normal vector to be 0, and because the hinge part is generally a columnar structure, most of the transformed source points can be 0 as long as they fall within the cylindrical surface determined by the source point and the target point and their normal vectors. Therefore, the objective function is set here based on the requirements. In the objective function, when point p and point Tq are on the same cylinder, the vector (p-Tq) and the vector (n p +n q ) is exactly vertical, then E(T) is 0. By using this objective function, the registration accuracy and convergence speed can be improved.

[0078] As another preferred embodiment of the present invention, the step of real-time monitoring of moving objects to obtain position and posture changes of a set of hinged doors relative to the vehicle body specifically includes:

[0079] Based on the fused registration model, point cloud registration is performed separately to obtain the position and posture data of the car body and door parts in the robot tool coordinate system;

[0080] The mobile robot installs the door to the car body, records the assembly completion position and repeatedly obtains position and posture data through dual 3D cameras.

[0081] In this embodiment, during the installation process, the vehicle body and the door are in motion, and the assembly can be guided by recording the position transformation of the two hinged door parts relative to their vehicle body parts. The specific process is as follows: Use the fused registration model M obj1 and M obj2 , and M obj1上 and M obj2上 (or M obj1下 and M obj2下 ) According to step S20, point cloud registration is performed respectively, and the spatial transformation of the door and the body in the robot tool coordinate system, that is, the position and posture, can be obtained respectively, which is recorded as tool H obj1 and tool H obj2 Then we have:

[0082] ( obj2 H obj1 ) A =( tool H obj2 ) -1 * tool H obj1 , which represents the position transformation of the upper and lower hinged door parts relative to the body part at position A; the mobile robot installs the door on the body, which is recorded as position B, i.e., the assembly completion position, and 3D cameras 1 and 2 take pictures. Repeating step S20 can obtain ( obj2 H obj1 ) B , which represents the position transformation of the upper and lower hinged door parts relative to the body part at position B.

[0083] As another preferred implementation of the present invention, the steps of guiding the installation according to the assembly guidance scheme, verifying the position and posture data before assembly by secondary image data acquisition to correct the assembly posture, updating the coordinate data and guiding the completion of the assembly include:

[0084] generating a guidance control signal to guide the vehicle door to the assembly position through the preparation installation position and performing image data acquisition before the vehicle door moves;

[0085] Perform secondary image data acquisition at the location to be assembled to obtain the current position and posture data for high-precision positioning correction, and calculate and update the guidance control data based on the correction results;

[0086] Based on the guidance control data, the vehicle door is guided from the to-be-assembled position to the installation completion position to complete the assembly process.

[0087] In this embodiment, the mobile robot moves the door to a position slightly away from the vehicle body (about 5 cm) for assembly, and 3D cameras 1 and 2 take pictures. Step S20 is repeated to perform point cloud matching to obtain the toolH obj1 and tool H obj2 , the robot tool coordinates at this time are marked as base H tool , in order to reach position A, even if the upper and lower hinged door parts are in the position relative to the body part ( obj2 H obj1 ) A , then base H tool_new = base H tool * tool H obj1 *( obj2 H obj1 ) A -1 ( tool H obj2 ) -1 ,in base H tool_new Indicates the new coordinates to which the robot tool needs to move, and guides the robot to move to the new coordinates according to the calculation results; after being guided to the right position, trigger 3D cameras 1 and 2 to take pictures again, and repeat step S20. At this time, the relative posture of the camera and the hinge is closer to the posture when modeling in step S20, and a point cloud that is nearly aligned with the modeling can be obtained. Compared with the first photo taken in step S40, the distance to the hinge is closer, and better sampling can be performed. The actual test results show that a higher-precision positioning result can be obtained. According to step S40, a new base H tool_new , guide the robot to move to the new coordinates according to the calculation results to complete the adjustment; trigger 3D cameras 1 and 2 again at position A to take pictures again, and repeat step 2 to get the tool H obj1 and tool H obj2 , in order to reach position B, we have base H tool_new = base H tool * tool H obj1 *( obj2 H obj1 ) B -1 ( tool H obj2 ) -1 , guide the robot to move to the new coordinates according to the calculation results to complete the installation.

[0088] like Figure 3 As shown, the present invention also provides a door assembly guidance system based on a high-precision positioning 3D camera, which comprises:

[0089] The data acquisition module 100 is used to acquire point cloud data based on the preset dual 3D cameras, and to perform point cloud fusion modeling based on the acquired point cloud data to obtain the conversion coordinate data of the hinge and the body part;

[0090] A registration processing module 200 is used to obtain spatial changes through point cloud registration to fit the nearest point iterative fine registration objective function. The point cloud registration method is based on a coarse-to-fine registration strategy. The objective function is used to improve the registration accuracy and convergence speed.

[0091] A motion monitoring module 300, for real-time monitoring of moving objects to obtain a position and posture change of a set of hinged doors relative to a vehicle body portion, and generating an assembly guidance scheme based on the position and posture change;

[0092] The posture correction module 400 is used to guide the installation according to the assembly guidance plan, verify the position and posture data before assembly through secondary image data acquisition, correct the assembly posture, update the coordinate data and guide the completion of the assembly.

[0093] As another preferred embodiment of the present invention, the data acquisition module includes:

[0094] A calibration unit, used to collect data of the upper and lower hinged body and door parts through dual 3D cameras, and perform hand-eye calibration on the pair of said 3D cameras;

[0095] The recording unit is used to guide the robot to grab the door to the assembly point and record it. It collects data through dual 3D cameras, obtains the point cloud data of the upper and lower hinged body and door parts at the current assembly point and records it.

[0096] The conversion unit is used to convert the point cloud data of the vehicle body and the door at the assembly point in the camera coordinate system to the robot tool coordinate system based on the hand-eye calibration result, so as to obtain the conversion coordinate data of the vehicle body and the door part of the upper and lower hinges in the robot tool coordinate system.

[0097] As another preferred embodiment of the present invention, when the registration processing module is executed, it includes:

[0098] The objective function selection unit is used for the nearest point iteration. The distance between point and point or point and plane is used as the objective function. The distance between the transformed source point and the target point can be zero only when the source point coincides with the target point or falls on the plane determined by the target point and its normal vector. Therefore, the objective function is:

[0099] E(T)=∑ (p，q)∈K ((p-Tq)·(n p +n q)), where: T represents spatial transformation, E represents objective function, p represents target point, q represents source point, np represents target point normal vector, nq represents source point normal vector, and K represents the set of corresponding points in source point cloud and target point cloud.

[0100] As another preferred embodiment of the present invention, the motion monitoring module includes:

[0101] A point cloud registration unit is used to perform point cloud registration based on the fused registration model to obtain the position and posture data of the vehicle body and door parts in the robot tool coordinate system;

[0102] The posture recording unit is used for the mobile robot to install the door to the car body, record the assembly completion position and repeatedly obtain the position and posture data through dual 3D cameras.

[0103] As another preferred embodiment of the present invention, the posture correction module includes:

[0104] a positioning verification unit, for generating a guidance control signal to guide the door to the assembly position through the preparation installation position and to collect image data before the door moves;

[0105] A secondary correction unit is used to collect secondary image data at the position to be assembled to obtain current position and posture data for high-precision positioning correction, and to calculate and update guidance control data based on the correction results;

[0106] The assembly guide unit is used to guide the vehicle door through the to-be-assembled position to the installation completion position based on the guidance control data to complete the assembly process.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0108] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0109] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A door assembly guidance method based on a high-precision positioning 3D camera, characterized in that: Include: Point cloud data is collected based on the preset dual 3D cameras, and point cloud fusion modeling is performed based on the collected point cloud data to obtain the conversion coordinate data of the hinge and body parts; The spatial changes are obtained by point cloud registration to fit the nearest point iterative fine registration objective function. The point cloud registration method is based on a coarse-to-fine registration strategy. The objective function is used to improve the registration accuracy and convergence speed. Performing real-time moving object monitoring to obtain a position and posture change of a set of hinged doors relative to a vehicle body portion, and generating an assembly guidance scheme based on the position and posture change; Guide the installation according to the assembly guidance plan, verify the position and posture data before assembly through secondary image data acquisition, correct the assembly posture, update the coordinate data and guide the assembly to complete; The step of obtaining spatial changes by point cloud registration to fit the nearest point iterative precise registration objective function is performed; The closest point iteration uses the distance from point to point or from point to plane as the objective function, and the distance can only be zero when the transformed source point coincides with the target point or falls on the plane determined by the target point and its normal vector, so the objective function is: , where: T represents the spatial transformation, E represents the objective function, p represents the target point, q represents the source point, represents the target point normal vector, represents the source point normal vector, and K represents the set of corresponding points in the source point cloud and the target point cloud.

2. A door assembly guidance method based on a high-precision positioning 3D camera according to claim 1, characterized in that: The step of collecting point cloud data based on the preset dual 3D cameras and performing point cloud fusion modeling based on the collected point cloud data to obtain the transformation coordinate data of the hinge and the body part specifically includes: Data collection of the upper and lower hinged body and door parts is performed using dual 3D cameras, and hand-eye calibration is performed on a pair of the 3D cameras; Guide the robot to grab the door to the assembly point and record it, collect data through dual 3D cameras, obtain the point cloud data of the upper and lower hinged body and door parts at the current assembly point and record it; Based on the hand-eye calibration results, the point cloud data of the body and door at the assembly point in the camera coordinate system are converted to the robot tool coordinate system to obtain the converted coordinate data of the body and door parts of the upper and lower hinges in the robot tool coordinate system.

3. A door assembly guidance method based on a high-precision positioning 3D camera according to claim 2, characterized in that: The steps of obtaining a set of position and posture changes of hinged doors relative to the vehicle body specifically include: Based on the fused registration model, point cloud registration is performed separately to obtain the position and posture data of the car body and door parts in the robot tool coordinate system; The mobile robot installs the door to the car body, records the assembly completion position and repeatedly obtains position and posture data through dual 3D cameras.

4. The door assembly guidance method based on a high-precision positioning 3D camera according to claim 3, characterized in that: The steps of guiding the installation according to the assembly guidance scheme, verifying the position and posture data before assembly by collecting secondary image data, correcting the assembly posture, updating the coordinate data and guiding the assembly to be completed include: generating a guidance control signal to guide the vehicle door to the assembly position through the preparation installation position and performing image data acquisition before the vehicle door moves; Perform secondary image data acquisition at the location to be assembled to obtain the current position and posture data for high-precision positioning correction, and calculate and update the guidance control data based on the correction results; Based on the guidance control data, the vehicle door is guided from the to-be-assembled position to the installation completion position to complete the assembly process.

5. A door assembly guidance system based on a high-precision positioning 3D camera, characterized in that: Include: A data acquisition module is used to collect point cloud data based on a preset dual 3D camera, and to perform point cloud fusion modeling based on the collected point cloud data to obtain the conversion coordinate data of the hinge and the body part; A registration processing module is used to obtain spatial changes through point cloud registration to fit the nearest point iterative fine registration objective function. The point cloud registration method is based on a coarse-to-fine registration strategy. The objective function is used to improve the registration accuracy and convergence speed. A motion monitoring module, used for real-time monitoring of moving objects to obtain the position and posture changes of a set of hinged doors relative to the vehicle body, and generating an assembly guidance scheme based on the position and posture changes; The posture correction module is used to guide the installation according to the assembly guidance plan, verify the position and posture data before assembly through secondary image data acquisition, correct the assembly posture, update the coordinate data and guide the completion of the assembly; When the registration processing module is executed, it includes: The objective function selection unit is used for the nearest point iteration. The distance between point and point or point and plane is used as the objective function. The distance between the transformed source point and the target point can be zero only when the source point coincides with the target point or falls on the plane determined by the target point and its normal vector. Therefore, the objective function is: , where: T represents the spatial transformation, E represents the objective function, p represents the target point, q represents the source point, represents the target point normal vector, represents the source point normal vector, and K represents the set of corresponding points in the source point cloud and the target point cloud.

6. A door assembly guidance system based on a high-precision positioning 3D camera according to claim 5, characterized in that: The data acquisition module comprises: A calibration unit, used to collect data of the upper and lower hinged body and door parts through dual 3D cameras, and perform hand-eye calibration on the pair of said 3D cameras; The recording unit is used to guide the robot to grab the door to the assembly point and record it. It collects data through dual 3D cameras, obtains the point cloud data of the upper and lower hinged body and door parts at the current assembly point and records it. The conversion unit is used to convert the point cloud data of the vehicle body and the door at the assembly point in the camera coordinate system to the robot tool coordinate system based on the hand-eye calibration result, so as to obtain the conversion coordinate data of the vehicle body and the door part of the upper and lower hinges in the robot tool coordinate system.

7. A door assembly guidance system based on a high-precision positioning 3D camera according to claim 6, characterized in that: The motion monitoring module comprises: A point cloud registration unit is used to perform point cloud registration based on the fused registration model to obtain the position and posture data of the vehicle body and door parts in the robot tool coordinate system; The posture recording unit is used for the mobile robot to install the door to the car body, record the assembly completion position and repeatedly obtain the position and posture data through dual 3D cameras.

8. The door assembly guidance system based on high-precision positioning 3D camera according to claim 7, characterized in that: The posture correction module comprises: a positioning verification unit, for generating a guidance control signal to guide the door to the assembly position through the preparation installation position and to collect image data before the door moves; A secondary correction unit is used to collect secondary image data at the position to be assembled to obtain current position and posture data for high-precision positioning correction, and to calculate and update guidance control data based on the correction results; The assembly guide unit is used to guide the vehicle door through the to-be-assembled position to the installation completion position based on the guidance control data to complete the assembly process.

Images