Multi-machine cooperative automobile door cover tightening and assembling method, device and equipment and storage medium

Abstract

The invention relates to a multi-machine cooperative automobile door cover tightening and assembling method, device and equipment and a storage medium. The method comprises the steps that after an adjusting robot grabs a door cover of an automobile to reach a preset position, the first tail end flange pose of the adjusting robot is recorded, and a reference normal vector of a bolt on the door cover at the preset position is determined; obtaining a second tail end flange pose after the pose of the door cover is adjusted by the adjusting robot; the relative position between the adjusted door cover and the vehicle body meets a preset condition; according to the pose change information between the second tail end flange pose and the first tail end flange pose, the normal vector variable quantity of the bolt before and after adjustment is determined; and according to the normal vector variable quantity and the reference normal vector of the bolt, a target normal vector of the adjusted bolt is determined, and according to the target normal vector, a tightening robot is controlled to tighten the bolt so as to assemble the door cover to the vehicle body. The method can improve the success rate of screwing assembly of the automobile door cover.

Description

Technical Field

[0001] The present application relates to the field of automobile manufacturing technology, and in particular to a method, device, equipment and storage medium for tightening and assembling a multi-machine coordinated automobile door cover. Background Art

[0002] With the rapid development of automotive manufacturing technologies (such as new energy vehicle technologies), automated assembly has become the mainstream solution for many OEMs. Manual assembly solutions typically rely on mechanical fixtures and manual adjustments, which are not adaptable to dynamic assembly scenarios and have high hardware costs. Automated assembly typically uses vision technology to guide robots during assembly.

[0003] Automated assembly of car door bonnets typically requires two robots (an adjustment robot and a tightening robot) to perform the process in stages. Traditionally, the adjustment robot grasps the door (i.e., the door bonnet) to be assembled and places it at a preset position near the vehicle body. An adjustment algorithm then guides the gripper to adjust the door's position until the clearance and surface difference between the door and the vehicle body meet process requirements. The tightening robot then uses a 3D camera to take real-time photos of the two bolts on the door. Using a visually aware 3D algorithm, it calculates the bolt's normal vector in real time, guiding the tightening robot to complete the tightening task.

[0004] However, because bolts are typically small (e.g., typically ≤10mm in diameter) and have a concave structure caused by stamping, resulting in an uneven end surface (e.g., often tapered), 3D algorithms cannot accurately calculate and locate the bolt's real-time normal vector, significantly reducing the success rate of tightening the car door bonnet. Therefore, a new method is urgently needed to overcome the technical problem of low success rate in tightening the car door bonnet. Summary of the Invention

[0005] Based on this, it is necessary to provide a multi-machine collaborative automobile door cover tightening assembly method, device, computer equipment, computer-readable storage medium and computer program product to address the above technical problems, which can improve the success rate of automobile door cover tightening.

[0006] In a first aspect, the present application provides a multi-machine collaborative automobile door cover tightening and assembly method, the method comprising:

[0007] After the adjustment robot grasps the door cover of the car and reaches a preset position, recording the position of the first end flange of the adjustment robot and determining the reference normal vector of the bolt on the door cover at the preset position;

[0008] Obtaining the posture of the second end flange after the adjustment robot adjusts the posture of the door cover; the relative position between the adjusted door cover and the vehicle body meets a preset condition;

[0009] Determining a change in the normal vector of the bolt before and after adjustment based on posture change information between the posture of the second end flange and the posture of the first end flange;

[0010] determining a target normal vector of the bolt after adjustment according to the normal vector variation and a reference normal vector of the bolt;

[0011] According to the target normal vector, the tightening robot is controlled to tighten the bolt to assemble the door cover to the vehicle body.

[0012] In one embodiment, controlling a tightening robot to tighten the bolt according to the target normal vector to assemble the door cover to the vehicle body includes:

[0013] Obtaining the target end face center coordinates of each bolt after adjustment, and determining the current real-time position and posture of the door cover according to the target end face center coordinates of each bolt and the target normal vector;

[0014] Determining a reference posture of the door cover according to the reference end face center coordinates of each bolt before adjustment and the reference normal vector;

[0015] Obtaining a reference end tool pose when the tightening robot is in a tightening position in a teaching state;

[0016] According to the posture transformation between the real-time posture of the door cover and the reference posture of the door cover, the reference end tool posture is corrected, and the tightening robot is controlled to tighten the bolt based on the corrected target end tool posture.

[0017] In one embodiment, the tightening robot is equipped with a 3D camera; the door cover is provided with a plurality of bolts;

[0018] The obtaining of the target end face center coordinates of each bolt after adjustment includes:

[0019] Obtaining bolt point cloud data of the plurality of bolts after adjustment; the bolt point cloud data is collected by using the 3D camera at a preset photographing point for the plurality of bolts after the door cover is adjusted;

[0020] Projecting the bolt point cloud data onto a preset plane; the preset plane is obtained by fitting the local point cloud data of the vehicle body plane around the plurality of bolts;

[0021] The target end face center coordinates of each of the bolts are detected from the projected preset plane.

[0022] In one embodiment, the target end face center coordinates, the reference end face center coordinates, the real-time position of the door cover, and the reference position of the door cover are all located in the same user coordinate system; wherein, the user coordinate system is established by a laser tracker at the center position of the bolts on both sides of the door cover.

[0023] In one embodiment, detecting the target end face center coordinates of each bolt from the projected preset plane includes:

[0024] Detecting the initial end face center coordinates of each bolt in the camera coordinate system from the projected preset plane;

[0025] Converting the initial end face center coordinates into target end face center coordinates in the user coordinate system through a first coordinate transformation relationship, a second coordinate transformation relationship, and a third coordinate transformation relationship in sequence;

[0026] Among them, the first coordinate transformation relationship refers to the transformation relationship between the camera coordinate system and the tool coordinate system of the tightening robot, the second coordinate transformation relationship is the transformation relationship between the tool coordinate system of the tightening robot and the base coordinate system of the tightening robot, and the third coordinate transformation relationship is the transformation relationship between the base coordinate system of the tightening robot and the user coordinate system.

[0027] In one embodiment, the first coordinate transformation relationship is obtained by hand-eye calibration; the second coordinate transformation relationship is determined based on the end tool posture of the tightening robot at the photographing point; and the third coordinate transformation relationship is obtained by coordinate system calibration using a laser tracker.

[0028] In one embodiment, the number of the bolts is four; the four bolts include a first bolt, a second bolt, a third bolt, and a fourth bolt; and determining the current real-time position of the door cover according to the target end face center coordinates and the target normal vector of each of the bolts includes:

[0029] According to the target end face center coordinates of each bolt, the center positions of the four bolts are determined as the origin of the door cover coordinate system;

[0030] determining an X-axis vector of the door cover coordinate system based on an average modulus vector of a first direction vector, a second direction vector, and a third direction vector; wherein the first direction vector is a direction vector from the first bolt to the second bolt, the second direction vector is a direction vector from the first bolt to the third bolt, and the third direction vector is a direction vector from the first bolt to the fourth bolt;

[0031] Determining the Z-axis vector of the door cover coordinate system according to the average modulus vector of the target normal vectors of the four bolts;

[0032] Determining the Y-axis vector of the door cover coordinate system according to the cross product of the X-axis vector and the Z-axis vector;

[0033] The real-time position and posture of the door cover is determined according to the origin of the door cover coordinate system, the X-axis vector, the Y-axis vector, and the Z-axis vector.

[0034] In a second aspect, the present application further provides a multi-machine coordinated automobile door cover tightening and assembly device, the device comprising:

[0035] The door cover adjustment module is used to record the position of the first end flange of the adjustment robot after the adjustment robot grasps the door cover of the car and reaches a preset position, and determine the reference normal vector of the bolt on the door cover at the preset position; obtain the position of the second end flange after the adjustment robot adjusts the position of the door cover; the relative position between the adjusted door cover and the car body meets the preset conditions;

[0036] a change determination module, configured to determine a change in a normal vector of the bolt before and after adjustment based on posture change information between the posture of the second end flange and the posture of the first end flange;

[0037] The tightening assembly module is used to determine the target normal vector of the bolt after adjustment based on the normal vector change and the reference normal vector of the bolt; and control the tightening robot to tighten the bolt based on the target normal vector to assemble the door cover to the vehicle body.

[0038] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of each embodiment of the present application when executing the computer program.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of each embodiment of the present application when executed by a processor.

[0040] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that implements the steps in each embodiment of the present application when executed by a processor.

[0041] In the aforementioned multi-machine collaborative automobile door bonnet tightening assembly method, apparatus, computer device, storage medium, and computer program product, the change in the bolt's normal vector during the adjustment process is determined based on information about the change in the end flange pose of the adjustment robot before and after the door bonnet is adjusted. Furthermore, the computer device can determine the target normal vector of the bolt after adjustment based on the change in the bolt's normal vector and the reference normal vector of the bolt before the door bonnet is adjusted. This is equivalent to using a bolt normal vector template (i.e., the reference normal vector of the bolt before adjustment) combined with rigid body transformation invariance (the end flange of the adjustment robot is rigidly related to the door bonnet component) to directly map the change in the end flange pose before and after the door bonnet is adjusted to the change in the bolt's normal vector, thereby calculating the target normal vector of the bolt after the door bonnet is adjusted. Traditional 3D vision recognition algorithms have difficulty identifying the normal vector of bolts with small dimensions and uneven end faces. Furthermore, traditional 3D vision recognition algorithms do not consider the impact of adjustment factors on the change in the bolt's pose, and therefore cannot accurately identify the bolt's normal vector. Calculating the target normal vector of the bolt after the door bonnet is adjusted using rigid body transformation invariance is more accurate and convenient. Furthermore, traditional 3D vision recognition algorithms require the collection of large amounts of training data for model training, which is very costly. The method in this application can more easily identify the target normal vector of the adjusted bolt, reducing both the difficulty and the cost. Furthermore, the computer can control the tightening robot to tighten the bolt based on the adjusted target normal vector, enabling more accurate assembly of the door cover to the vehicle body, improving the success rate of tightening, and reducing the cost of the tightening operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a diagram illustrating an application environment of a multi-machine collaborative automobile door cover tightening and assembly method according to one embodiment;

[0043] Figure 2 The figure is a flow chart of a method for tightening and assembling a car door cover by coordinating multiple machines in one embodiment;

[0044] Figure 3 A schematic flow chart of a tightening assembly step in one embodiment;

[0045] Figure 4 Schematic diagram of a flow chart of the steps for determining the real-time position and posture of a door cover in one embodiment;

[0046] Figure 5 This is a structural block diagram of a multi-machine coordinated automobile door cover tightening and assembly device in one embodiment;

[0047] Figure 6 A structural block diagram of a tightening assembly module in one embodiment;

[0048] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0050] For ease of understanding, before formally introducing the solutions in the embodiments of this application, Figure 1 This article introduces the application scenario of the multi-machine collaborative automobile door cover tightening assembly method.

[0051] Figure 1 The various numbers are shown as follows:

[0052] 10: Tightening robot, used to carry 3D camera and tightening gun

[0053] 11 / 12: 3D camera for bolt photo positioning

[0054] 13 / 14: Tightening gun, used for bolt tightening operation (tightening gun is end tool or end tightening tool)

[0055] 20: Adjust the robot to grab the door cover and adjust its posture

[0056] 30: The car door cover is a workpiece to be assembled

[0057] 31 / 32: Bolts on the door cover

[0058] It should be understood that Figure 1 Only some components are shown for simplified illustration. For example, only some tightening guns and some bolts are shown, which should not be construed as limiting. For example, a car door cover may have four bolts, two by two, on either side. Similarly, the tightening robot 10 may be equipped with four tightening guns for tightening the four bolts respectively.

[0059] like Figure 1 As shown, after the control and adjustment robot 20 has grasped the car's door cover 30 to a preset position, a computer device (not shown in the figure, i.e., a control device) can determine the normal vector (i.e., a reference normal vector) of the bolts 31 / 32 before the door cover 30 is adjusted based on the images captured by the 3D cameras 11 / 12 of the bolts 31 / 32. The control and adjustment robot 20 then adjusts the position of the door cover 30 at the preset position until the relative position between the door cover and the vehicle body meets preset conditions, such as the clearance and flushness around the door and vehicle body meet preset process requirements. At this point, the position and posture adjustment of the door cover 30 can be stopped.

[0060] The computer device can determine the change in the normal vector of the bolt 31 / 32 during the installation process based on the change information of the end flange posture of the adjustment robot 20 before and after the door cover 30 is adjusted, that is, determine the change in the normal vector of the bolt 31 / 32. Furthermore, the computer device can determine the normal vector of the bolt 31 / 32 after adjustment (that is, the target normal vector) based on the change in the normal vector of the bolt 31 / 32 and the normal vector of the bolt 31 / 32 before the door cover 30 is adjusted (that is, the reference normal vector). Then, the computer device can control the tightening robot 10 to tighten the bolt 31 / 32 based on the normal vector of the bolt 31 / 32 after adjustment, thereby assembling the door cover 30 to the vehicle body (not shown in the figure).

[0061] The computer devices in the embodiments of the present application may include, but are not limited to, various personal computers, laptops, smartphones, tablet computers, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices may include smart watches, smart bracelets, head-mounted devices, etc.

[0062] For ease of understanding, the following description will provide a more detailed description of the multi-machine collaborative automobile door cover tightening and assembly method in the embodiment of the present application.

[0063] like Figure 2 As shown, in some embodiments of the present application, a multi-machine collaborative automobile door cover tightening assembly method is proposed, which can be applied to computer equipment and specifically includes the following steps:

[0064] S21, after the adjustment robot grasps the door cover of the car and reaches a preset position, the position of the first end flange of the adjustment robot is recorded, and a reference normal vector of the bolt on the door cover at the preset position is determined.

[0065] Among them, the adjustment robot is used to adjust the position of the car's door cover.

[0066] Specifically, the robot is adjusted to grasp a car door bonnet to a preset position. After reaching the preset position, the computer device can record the adjusted position of the robot's end flange at this time, recording it as the first end flange position. It should be understood that the first end flange position is the position of the robot's end flange before adjusting the position of the door bonnet at the preset position.

[0067] There are multiple bolts on the door cover of a car, for example, there may be 4 bolts. The computer device can calculate the normal vectors of the 4 bolts at a preset position and record them as reference normal vectors of the bolts.

[0068] In some embodiments, before adjusting the pose of the door cover at the preset position, the computer device can control a 3D camera mounted on the tightening robot to capture image data of the door cover at the preset position, thereby obtaining point cloud data of the door cover at the preset position. Based on the point cloud data, the door cover plane is fitted, and the normal vector of the fitted door cover plane is used as the reference normal vector of the bolt. It should be understood that the normal vector of the bolt is approximately the same as the normal vector of the door cover plane, so the normal vector of the fitted door cover plane before pose adjustment can be used as the reference normal vector of the bolt. After the pose of the door cover is adjusted, the reference normal vector of the bolt can be used in conjunction with steps S22 to S24 to more accurately determine the target normal vector of the bolt after pose adjustment.

[0069] S22, obtaining the posture of the second end flange after the adjustment robot adjusts the posture of the door cover; the relative position between the adjusted door cover and the vehicle body meets a preset condition.

[0070] Specifically, the adjustment robot can adjust the door cover until the relative position between the door cover and the vehicle body satisfies a preset adjustment, and then stop adjusting. The computer device can obtain the terminal flange posture after the adjustment robot adjusts the door cover, and record it as the second terminal flange posture.

[0071] S23: Determine a change in the normal vector of the bolt before and after adjustment based on the posture change information between the posture of the second end flange and the posture of the first end flange.

[0072] It should be understood that the end flange (gripper) of the adjustment robot is rigidly connected to the door cover component. Therefore, the change in the bolt's normal vector can be determined based on the change in the robot's end flange's posture before and after the door cover's posture is adjusted. Specifically, the computer device can determine the change in the bolt's normal vector before and after adjustment based on the posture change between the second end flange's posture and the first end flange's posture.

[0073] S24 , determining a target normal vector of the bolt after adjustment according to the normal vector variation and the reference normal vector of the bolt.

[0074] Specifically, the computer device can add the normal vector variation determined in step S23 to the reference normal vector of the bolt to obtain the target normal vector of the bolt after adjustment (for the door cover posture).

[0075] S25 , controlling a tightening robot to tighten the bolt according to the target normal vector, so as to assemble the door cover to the vehicle body.

[0076] Specifically, after determining the target normal vector for the bolt, the computer device can control the tightening robot to tighten the bolt based on the target normal vector, thereby assembling the door cover to the vehicle body. It should be understood that each bolt has a corresponding normal vector, and the computer device controls the tightening robot to tighten multiple bolts based on the target normal vector of each bolt.

[0077] For example, the computer device can control the tightening robot to tighten the bolts based on the target normal vectors and target end face center coordinates of each bolt to assemble the door cover to the vehicle body. The target end face center coordinates of each bolt refer to the center coordinates of the bolt end face after the door cover posture is adjusted.

[0078] The above-mentioned multi-machine collaborative automobile door bonnet tightening and assembly method adjusts the reference normal vector of the front bolt and combines it with the invariance of rigid body transformation. The change in the position of the front and rear end flanges of the door bonnet is directly mapped to the change in the normal vector of the bolt, thereby solving the target normal vector of the bolt after the door bonnet is adjusted. Traditional 3D visual recognition algorithms have great difficulty in identifying the normal vector of bolts with small size and uneven end faces. Moreover, traditional 3D visual recognition algorithms do not consider the impact of adjustment factors on the change in the bolt posture, and therefore cannot accurately identify the bolt normal vector. The target normal vector of the bolt after the door bonnet is adjusted, calculated by rigid transformation invariance, is more accurate and convenient. In addition, traditional 3D visual recognition algorithms require the collection of a large amount of training data for model training in the early stage, which is very costly. The method of the present application can more conveniently identify the target normal vector of the bolt after adjustment, reducing the difficulty of identification and reducing the cost. Furthermore, the computer equipment can control the tightening robot to tighten the bolt according to the target normal vector of the bolt after adjustment, so that the door bonnet can be assembled to the vehicle body more accurately, improving the tightening success rate and reducing the cost of the tightening operation.

[0079] In some embodiments, as Figure 3 As shown, step S25 controls the tightening robot to tighten the bolt according to the target normal vector to assemble the door cover to the vehicle body (referred to as the tightening assembly step), including:

[0080] S251, obtaining the target end face center coordinates of each bolt after adjustment, and determining the current real-time position and posture of the door cover according to the target end face center coordinates of each bolt and the target normal vector.

[0081] In some embodiments, the tightening robot is equipped with a 3D camera; the door cover is provided with a plurality of bolts. Obtaining the target end face center coordinates of each bolt after adjustment includes: obtaining bolt point cloud data of the plurality of bolts after adjustment; the bolt point cloud data is acquired by using the 3D camera at preset photographic points for the plurality of bolts after the door cover is adjusted; projecting the bolt point cloud data onto a preset plane; and detecting the target end face center coordinates of each bolt from the projected preset plane.

[0082] Since the bolt end faces are not flat, they cannot be processed directly using a circle detection algorithm. Therefore, in some embodiments, the computer device can project the adjusted (i.e., after the door cover is adjusted) bolt point cloud data onto a preset plane (which can be recorded as a first preset plane). The preset plane is fitted based on the local point cloud data of the vehicle body plane around the multiple bolts after adjustment in the teaching state. The computer device can project the adjusted bolt point cloud data onto the above-mentioned preset plane, and relatively flat and regular data can be obtained on the preset plane. Therefore, the computer device can extract edges from the projected preset plane and identify the center coordinates of each bolt end face based on the 3D circle detection algorithm to obtain the target end face center coordinates of each bolt.

[0083] It should be understood that the target end face center coordinates of the bolts are equivalent to the position information of the bolts. In step S24, the target normal vectors of each bolt have been calculated. Therefore, the computer device can determine the real-time position and posture of the door cover based on the target end face center coordinates and target normal vectors of each bolt.

[0084] In some embodiments, the number of the bolts is 4. Figure 4 As shown, the step of determining the current real-time pose of the door cover (hereinafter referred to as the step of determining the real-time pose of the door cover) based on the target end face center coordinates of each bolt and the target normal vector includes:

[0085] S251a, according to the target end face center coordinates of each bolt, determine the center position of the four bolts as the origin of the current door cover coordinate system.

[0086] S251b: Determine the X-axis vector of the door cover coordinate system according to the average modulus vector of the first direction vector, the second direction vector, and the third direction vector.

[0087] Specifically, the four bolts include a first bolt, a second bolt, a third bolt, and a fourth bolt. The first direction vector is the direction vector from the adjusted first bolt to the adjusted second bolt, the second direction vector is the direction vector from the adjusted first bolt to the adjusted third bolt, and the third direction vector is the direction vector from the adjusted first bolt to the adjusted fourth bolt.

[0088] S251c, determining the Z-axis vector of the door cover coordinate system according to the average modulus vector of the target normal vectors of the four bolts.

[0089] S251d: Determine the Y-axis vector of the door cover coordinate system according to the cross product of the X-axis vector and the Z-axis vector.

[0090] S251e: Determine the real-time position and posture of the door cover according to the origin of the door cover coordinate system, the X-axis vector, the Y-axis vector, and the Z-axis vector.

[0091] Assume that the four bolts are A, B, C, and D, and the first direction vector is The second direction vector is The third direction vector is The normal vectors of the four bolts are F A 、F B 、F C 、F D The origin of the door cover coordinate system after adjustment is the center position of the four bolts, which can be the average value of the target end face center coordinates of the four bolts. The X-axis vector of the door cover coordinate system after adjustment is and The average modulus vector of the door cover coordinate system after adjustment is the normal vector F of the four bolts. A 、F B 、F C 、F D The average modulus vector of the door cover. The Y-axis vector of the adjusted door cover coordinate system is the cross product of the X-axis and Z-axis vectors. The computer device can determine the real-time position and posture of the door cover based on the origin, X-axis vector, Y-axis vector, and Z-axis vector of the adjusted door cover coordinate system.

[0092] For example, the computer device can construct a rotation matrix based on the X-axis vector, Y-axis vector and Z-axis vector of the adjusted door cover coordinate system, and combine the center positions of the adjusted four bolts (that is, the mean of the reference end face center coordinates of the four bolts) to construct a 4*4 pose matrix of the adjusted door cover, that is, the real-time pose of the door cover.

[0093] S252: Determine the reference posture of the door cover according to the reference end face center coordinates of each bolt before adjustment and the reference normal vector.

[0094] The reference end face center coordinates of each bolt refer to the center coordinates of the bolt end face before the door cover at the preset position is adjusted. The reference pose of the door cover refers to the pose of the door cover at the preset position before the pose is adjusted.

[0095] In some embodiments, a computer device can collect bolt point cloud data of multiple bolts before adjustment using a 3D camera mounted on a tightening robot, and project the bolt point cloud data before adjustment (i.e., before the door cover is adjusted) onto a preset plane (which may be recorded as a second preset plane). The preset plane is fitted based on the local point cloud data of the vehicle body plane surrounding the multiple bolts in the teaching state before adjustment. The bolt point cloud data before adjustment is projected onto the preset plane, and the computer device extracts edges from the projected preset plane and identifies the center coordinates of each bolt end face based on a 3D circle detection algorithm to obtain the reference end face center coordinates of each bolt.

[0096] Furthermore, the computer device can determine the reference pose of the door cover based on the reference end face center coordinates of each bolt before adjustment and the reference normal vector. For example, the door cover has four bolts. Based on the reference end face center coordinates of each bolt, the computer device can determine the center positions of the four bolts before adjustment as the origin of the door cover coordinate system before adjustment; and determine the X-axis vector of the door cover coordinate system before adjustment based on the average modulus vector of the fourth, fifth, and sixth direction vectors. The fourth direction vector is the direction vector from the first bolt before adjustment to the second bolt before adjustment, the fifth direction vector is the direction vector from the first bolt before adjustment to the third bolt before adjustment, and the sixth direction vector is the direction vector from the first bolt before adjustment to the fourth bolt before adjustment. The computer device can determine the Z-axis vector of the door cover coordinate system before adjustment based on the average modulus vector of the reference normal vectors of the four bolts; determine the Y-axis vector of the door cover coordinate system before adjustment based on the cross product of the X-axis vector and the Z-axis vector of the door cover coordinate system before adjustment; and determine the reference pose of the door cover based on the origin, X-axis vector, Y-axis vector, and Z-axis vector of the door cover coordinate system before adjustment.

[0097] Exemplarily, the computer device can construct a rotation matrix based on the X-axis vector, Y-axis vector and Z-axis vector of the door cover coordinate system before adjustment, and combine the origin of the door cover coordinate system before adjustment (that is, the mean of the reference end face center coordinates of the 4 bolts) to construct a 4*4 pose matrix of the door cover before adjustment, that is, the reference pose of the door cover.

[0098] S253, obtaining a reference end tool pose when the tightening robot is in a tightening position in a teaching state.

[0099] Specifically, when a tightening robot carries a tightening gun to perform bolt teaching and tightening, the computer device can record the end tool posture of the tightening robot in the tightening position. This end tool posture can be used as a reference during the tightening operation, so it can be recorded as a reference end tool posture, and can also be called the reference end tool posture of the tightening robot for teaching tightening.

[0100] S254, correcting the reference end tool posture according to the posture transformation between the real-time posture of the door cover and the reference posture of the door cover, and controlling the tightening robot to tighten the bolt based on the corrected target end tool posture.

[0101] In some embodiments, the computer device can determine the ratio of the real-time pose of the door cover to the reference pose of the door cover, and multiply the ratio by the reference end tool pose to obtain the target end tool pose of the tightening robot. For example, the target end tool pose of the tightening robot can be expressed by the following formula: (M*Mt -1 )*Mr; where M represents the real-time position of the door cover, Mt represents the reference position of the door cover, and Mr represents the reference end-tool position for the tightening robot during teaching.

[0102] In the above solution, the computer device can perform positioning correction based on the positional transformation between the door cover's real-time position and its reference position, i.e., correct the reference end-tool position. The corrected target end-tool position is more accurate, and the computer device can then control the tightening robot to tighten the bolt more accurately based on the target end-tool position, thereby improving the tightening success rate.

[0103] In some embodiments, the target end face center coordinates, the reference end face center coordinates, the real-time position of the door cover, and the reference position of the door cover are all located in the same user coordinate system; wherein, the user coordinate system is established by a laser tracker at the center position of the bolts on both sides of the door cover.

[0104] It should be understood that the base coordinates of the bolts and the robot (adjustment robot and tightening robot) bodies are far apart, so if the bolts are positioned based on the robot's base coordinates, it is difficult to accurately guide the tightening action of the bolts. Therefore, in this embodiment, the adjustment robot and the tightening robot are calibrated by a laser tracker to establish the same user coordinate system at the center position of the bolts on both sides of the door cover to be assembled. Furthermore, the target end face center coordinates, the reference end face center coordinates, the real-time posture of the door cover, and the reference posture of the door cover can all be converted to the same user coordinate system for calculation. In this way, the scale of visual perception information is standardized and the error is reduced to a controllable range, thereby greatly improving the positioning accuracy of small-sized objects such as bolts.

[0105] In some embodiments, detecting the target end face center coordinates of each of the bolts from the preset plane after projection includes: detecting the initial end face center coordinates of each of the bolts in the camera coordinate system from the preset plane after projection; and converting the initial end face center coordinates into the target end face center coordinates in the user coordinate system through a first coordinate transformation relationship, a second coordinate transformation relationship, and a third coordinate transformation relationship in sequence.

[0106] The first coordinate transformation relationship refers to the transformation relationship between the camera coordinate system and the tool coordinate system of the tightening robot, and the first coordinate transformation relationship is obtained by hand-eye calibration.

[0107] The second coordinate transformation relationship is a transformation relationship between the tool coordinate system of the tightening robot and the base coordinate system of the tightening robot. The second coordinate transformation relationship is determined according to the end tool posture of the tightening robot at the shooting point.

[0108] The third coordinate transformation relationship is a transformation relationship between the base coordinate system of the tightening robot and the user coordinate system. The third coordinate transformation relationship is obtained by calibrating the coordinate system using a laser tracker.

[0109] Specifically, the tightening robot collects point cloud data for multiple bolts on the adjusted door cover through the 3D camera it carries, obtains bolt point cloud data, and projects the bolt point cloud data onto a preset plane. In this case, the initial end face center coordinates of each bolt in the camera coordinate system are detected from the projected preset plane. Then, based on the first coordinate transformation relationship, the initial end face center coordinates are transformed from the camera coordinate system to the tool coordinate system of the tightening robot. Then, based on the second coordinate transformation relationship, the end face center coordinates in the tool coordinate system are transformed to the base coordinate system of the tightening robot. Finally, based on the third coordinate transformation relationship, the end face center coordinates in the base coordinate system of the tightening robot are transformed to the user coordinate system to obtain the target end face center coordinates in the user coordinate system.

[0110] It should be understood that other parameters can also be converted to the user coordinate system based on the first coordinate transformation relationship, the second coordinate transformation relationship and the third coordinate transformation relationship. The principle is the same as above and will not be repeated here.

[0111] The above solution converts the parameters of the adjustment robot and the tightening robot into the same user coordinate system through multiple coordinate transformation relationships, which can reduce the error and greatly improve the positioning accuracy of small-sized objects such as bolts, thereby increasing the success rate of subsequent tightening.

[0112] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0113] Based on the same inventive concept, embodiments of the present application also provide a multi-machine coordinated automobile door bonnet tightening and assembly device for implementing the aforementioned method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the multi-machine coordinated automobile door bonnet tightening and assembly device provided below can be found in the above-mentioned limitations of the multi-machine coordinated automobile door bonnet tightening and assembly method, and will not be further elaborated here.

[0114] like Figure 5 As shown, in some embodiments, the present application also provides a multi-machine coordinated automobile door cover tightening and assembly device, the device comprising:

[0115] The door cover adjustment module 501 is configured to record the position of the first end flange of the adjustment robot after the adjustment robot grasps the door cover of the vehicle and reaches a preset position, and determine the reference normal vector of the bolts on the door cover at the preset position; obtain the position of the second end flange after the adjustment robot adjusts the position of the door cover; and ensure that the relative position between the adjusted door cover and the vehicle body meets a preset condition;

[0116] a change determination module 502, configured to determine a change in a normal vector of the bolt before and after adjustment based on information about a change in posture between the second end flange posture and the first end flange posture;

[0117] The tightening assembly module 503 is used to determine the target normal vector of the bolt after adjustment based on the normal vector change and the reference normal vector of the bolt; and control the tightening robot to tighten the bolt based on the target normal vector to assemble the door cover to the vehicle body.

[0118] In some embodiments, as Figure 6 As shown, the tightening assembly module 503 further includes:

[0119] The door cover posture determination module 5031 is used to obtain the target end face center coordinates of each bolt after adjustment, and determine the current real-time posture of the door cover based on the target end face center coordinates and the target normal vector of each bolt; and determine the reference posture of the door cover based on the reference end face center coordinates and the reference normal vector of each bolt before adjustment;

[0120] The tool posture correction module 5032 is used to obtain a reference end-tool posture when the tightening robot is in the tightening position in the teaching state; and correct the reference end-tool posture according to the posture transformation between the real-time posture of the door cover and the reference posture of the door cover;

[0121] The tightening module 5033 is used to control the tightening robot to tighten the bolt based on the corrected target end tool posture.

[0122] In some embodiments, the tightening robot is equipped with a 3D camera; a plurality of bolts are provided on the door cover; the door cover posture determination module 5031 is also used to obtain bolt point cloud data of the plurality of bolts after adjustment; the bolt point cloud data is collected for the plurality of bolts at a preset shooting point using the 3D camera after the door cover is adjusted; the bolt point cloud data is projected onto a preset plane; the preset plane is fitted based on the local point cloud data of the vehicle body plane around the plurality of bolts; and the target end face center coordinates of each of the bolts are detected from the projected preset plane.

[0123] In some embodiments, the target end face center coordinates, the reference end face center coordinates, the real-time position of the door cover, and the reference position of the door cover are all located in the same user coordinate system; wherein, the user coordinate system is established by a laser tracker at the center position of the bolts on both sides of the door cover.

[0124] In some embodiments, the door cover posture determination module 5031 is also used to detect the initial end face center coordinates of each bolt in the camera coordinate system from the preset plane after projection; the initial end face center coordinates are converted into the target end face center coordinates in the user coordinate system through the first coordinate transformation relationship, the second coordinate transformation relationship and the third coordinate transformation relationship in sequence; wherein, the first coordinate transformation relationship refers to the transformation relationship between the camera coordinate system and the tool coordinate system of the tightening robot, the second coordinate transformation relationship is the transformation relationship between the tool coordinate system of the tightening robot and the base coordinate system of the tightening robot, and the third coordinate transformation relationship is the transformation relationship between the base coordinate system of the tightening robot and the user coordinate system.

[0125] In some embodiments, the first coordinate transformation relationship is obtained by hand-eye calibration; the second coordinate transformation relationship is determined based on the end tool posture of the tightening robot at the photographing point; and the third coordinate transformation relationship is obtained by coordinate system calibration using a laser tracker.

[0126] In some embodiments, the number of the bolts is 4; the 4 bolts include a first bolt, a second bolt, a third bolt, and a fourth bolt. The door cover posture determination module 5031 is further used to determine the center positions of the 4 bolts as the origin of the door cover coordinate system based on the target end face center coordinates of each bolt; determine the X-axis vector of the door cover coordinate system based on the average modulus vector of the first direction vector, the second direction vector, and the third direction vector; the first direction vector is the direction vector from the first bolt to the second bolt, the second direction vector is the direction vector from the first bolt to the third bolt, and the third direction vector is the direction vector from the first bolt to the fourth bolt; determine the Z-axis vector of the door cover coordinate system based on the average modulus vector of the target normal vectors of the 4 bolts; determine the Y-axis vector of the door cover coordinate system based on the cross product of the X-axis vector and the Z-axis vector; and determine the real-time posture of the door cover based on the origin of the door cover coordinate system, the X-axis vector, the Y-axis vector, and the Z-axis vector.

[0127] Each module in the aforementioned multi-machine coordinated automobile door bonnet tightening and assembly apparatus can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor within a computer device in hardware form, or stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0128] In one embodiment, a computer device is provided, whose internal structure diagram can be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multi-machine collaborative automobile door cover tightening assembly method is realized.

[0129] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0130] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in each embodiment of the present application are implemented.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in each embodiment of the present application are implemented.

[0132] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in each embodiment of the present application when executed by a processor.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0134] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0135] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A multi-machine coordinated automobile door cover tightening and assembly method, characterized in that: The method comprises: After the adjustment robot grasps the door cover of the car and reaches a preset position, recording the position of the first end flange of the adjustment robot and determining the reference normal vector of the bolt on the door cover at the preset position; Obtaining the posture of the second end flange after the adjustment robot adjusts the posture of the door cover; the relative position between the adjusted door cover and the vehicle body meets a preset condition; Determining a change in the normal vector of the bolt before and after adjustment based on posture change information between the posture of the second end flange and the posture of the first end flange; determining a target normal vector of the bolt after adjustment according to the normal vector variation and a reference normal vector of the bolt; According to the target normal vector, the tightening robot is controlled to tighten the bolt to assemble the door cover to the vehicle body.

2. The method according to claim 1, characterized in that The step of controlling a tightening robot to tighten the bolt according to the target normal vector to assemble the door cover to the vehicle body includes: Obtaining the target end face center coordinates of each bolt after adjustment, and determining the current real-time position and posture of the door cover according to the target end face center coordinates of each bolt and the target normal vector; Determining a reference posture of the door cover according to the reference end face center coordinates of each bolt before adjustment and the reference normal vector; Obtaining a reference end tool pose when the tightening robot is in a tightening position in a teaching state; According to the posture transformation between the real-time posture of the door cover and the reference posture of the door cover, the reference end tool posture is corrected, and the tightening robot is controlled to tighten the bolt based on the corrected target end tool posture.

3. The method according to claim 2, characterized in that The tightening robot is equipped with a 3D camera; the door cover is provided with a plurality of bolts; The obtaining of the target end face center coordinates of each bolt after adjustment includes: Obtaining bolt point cloud data of the plurality of bolts after adjustment; the bolt point cloud data is collected by using the 3D camera at a preset photographing point for the plurality of bolts after the door cover is adjusted; Projecting the bolt point cloud data onto a preset plane; the preset plane is obtained by fitting the local point cloud data of the vehicle body plane around the plurality of bolts; The target end face center coordinates of each of the bolts are detected from the projected preset plane.

4. The method according to claim 3, characterized in that The target end face center coordinates, the reference end face center coordinates, the real-time posture of the door cover and the reference posture of the door cover are all located in the same user coordinate system; wherein, the user coordinate system is established at the center position of the bolts on both sides of the door cover by a laser tracker.

5. The method according to claim 4, characterized in that Detecting the target end face center coordinates of each bolt from the projected preset plane includes: Detecting the initial end face center coordinates of each bolt in the camera coordinate system from the projected preset plane; Converting the initial end face center coordinates into target end face center coordinates in the user coordinate system through a first coordinate transformation relationship, a second coordinate transformation relationship, and a third coordinate transformation relationship in sequence; Among them, the first coordinate transformation relationship refers to the transformation relationship between the camera coordinate system and the tool coordinate system of the tightening robot, the second coordinate transformation relationship is the transformation relationship between the tool coordinate system of the tightening robot and the base coordinate system of the tightening robot, and the third coordinate transformation relationship is the transformation relationship between the base coordinate system of the tightening robot and the user coordinate system.

6. The method according to claim 5, characterized in that The first coordinate transformation relationship is obtained by hand-eye calibration; the second coordinate transformation relationship is determined based on the end tool posture of the tightening robot at the photographing point; and the third coordinate transformation relationship is obtained by coordinate system calibration using a laser tracker.

7. The method according to any one of claims 2 to 6, characterized in that The number of the bolts is 4; the 4 bolts include a first bolt, a second bolt, a third bolt and a fourth bolt; Determining the current real-time position of the door cover according to the target end face center coordinates and the target normal vector of each bolt includes: According to the target end face center coordinates of each bolt, the center positions of the four bolts are determined as the origin of the door cover coordinate system; determining an X-axis vector of the door cover coordinate system based on an average modulus vector of a first direction vector, a second direction vector, and a third direction vector; wherein the first direction vector is a direction vector from the first bolt to the second bolt, the second direction vector is a direction vector from the first bolt to the third bolt, and the third direction vector is a direction vector from the first bolt to the fourth bolt; Determining the Z-axis vector of the door cover coordinate system according to the average modulus vector of the target normal vectors of the four bolts; Determining the Y-axis vector of the door cover coordinate system according to the cross product of the X-axis vector and the Z-axis vector; The real-time position and posture of the door cover is determined according to the origin of the door cover coordinate system, the X-axis vector, the Y-axis vector, and the Z-axis vector.

8. A multi-machine coordinated automobile door cover tightening and assembly device, characterized in that: The device comprises: The door cover adjustment module is used to record the position of the first end flange of the adjustment robot after the adjustment robot grasps the door cover of the car and reaches a preset position, and determine the reference normal vector of the bolt on the door cover at the preset position; obtain the position of the second end flange after the adjustment robot adjusts the position of the door cover; the relative position between the adjusted door cover and the car body meets the preset conditions; a change determination module, configured to determine a change in a normal vector of the bolt before and after adjustment based on posture change information between the posture of the second end flange and the posture of the first end flange; The tightening assembly module is used to determine the target normal vector of the bolt after adjustment based on the normal vector change and the reference normal vector of the bolt; and control the tightening robot to tighten the bolt based on the target normal vector to assemble the door cover to the vehicle body.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images