Vehicle body defect coordinate mapping correction method, system, device and medium

Abstract

This application relates to the field of automotive manufacturing technology and provides a method, system, device, and medium for correcting vehicle body defect coordinate mapping. The method includes: registering the real-time point cloud of the vehicle under test acquired by the target 3D camera with the point cloud of the template to obtain the relative change in vehicle body pose; determining a fourth relative transformation matrix from the main manipulator base coordinate system to the vehicle body coordinate system of the vehicle under test based on the first relative transformation matrix from the target 3D camera coordinate system to the target manipulator base coordinate system, the relative change in vehicle body pose, the second relative transformation matrix from the target manipulator base coordinate system to the main manipulator base coordinate system, and the third relative transformation matrix from the main manipulator base coordinate system to the vehicle body coordinate system of the template vehicle; and transforming the pixel coordinates of the target vehicle body defect to the vehicle body coordinate system of the vehicle under test based on the second and fourth relative transformation matrices to obtain the corrected coordinates of the target vehicle body defect, thereby improving the efficiency and accuracy of vehicle body defect localization.

Description

Technical Field

[0001] This application belongs to the field of automotive manufacturing technology, and in particular relates to a method, system, device and medium for correcting vehicle body defect coordinate mapping. Background Technology

[0002] In the automotive manufacturing industry, the painting process not only affects the visual appeal of the vehicle body but also directly determines the product's weather resistance, corrosion resistance, and long-term performance. Traditional paint quality inspection methods, which rely on human visual inspection, are often limited by visual fatigue, subjective fluctuations, environmental interference, and efficiency bottlenecks, making them unsuitable for the stringent requirements of modern production lines for high precision, high consistency, and high efficiency. Therefore, introducing intelligent and systematic inspection methods has become an inevitable trend to improve the overall manufacturing quality and process reliability of vehicles.

[0003] Existing vision inspection systems based on multi-robotic arms equipped with structured light cameras park vehicles at the inspection station using methods such as skids, and calculate the 3D coordinates of vehicle body defects using multi-view stereo vision reconstruction or structured light triangulation algorithms. However, this approach has the following technical problems: (1) Long computation time: The computational complexity of traditional triangulation algorithms increases exponentially when dealing with complex surfaces. When the number of defects increases, the computation time is significantly extended, which seriously restricts production efficiency. (2) Poor positioning accuracy: When the vehicle under test is moved to the inspection station by means of skids, most of them are mechanically fixed by physical equipment, which restricts their degree of freedom. However, the existing system performs coordinate mapping based on the preset calibration position. When the vehicle body has position deviation or attitude change during the skid positioning process, the defect positioning accuracy drops sharply, and the error range can reach more than ±20 mm. This cannot meet the requirements of high-precision grinding process. Furthermore, at the edge of the vehicle body, the defect cannot be visualized on the digital model of the vehicle body due to the large deviation, resulting in missed defects and reducing the defect detection performance of the system.

[0004] Therefore, how to quickly and accurately locate the defects in the vehicle body when the position of the vehicle under test changes relative to the preset calibration position has become an urgent problem to be solved. Summary of the Invention

[0005] To address at least one of the aforementioned technical problems, and to quickly and accurately locate vehicle body defects when the position of the vehicle under test changes relative to a preset calibration position, this application proposes a method, system, device, and medium for correcting vehicle body defect coordinate mapping.

[0006] In a first aspect, this application provides a method for correcting vehicle body defect coordinate mapping, the method comprising: S1, obtain the first relative transformation matrix from the camera coordinate system of the target 3D camera to the base coordinate system of the target robotic arm, wherein the target 3D camera is any 3D camera on the detection station, and the target robotic arm is the robotic arm on the detection station that is within a preset range and closest to the target 3D camera; S2, register the real-time point cloud of the vehicle under test acquired by the target 3D camera with the template point cloud of the template vehicle to obtain the relative change of the vehicle body pose in the camera coordinate system of the target 3D camera, which is used to indicate the relative change of the pose from the vehicle under test to the template vehicle. S3. Based on the first relative transformation matrix, the relative change in vehicle posture, the second relative transformation matrix from the base coordinate system of the target robotic arm to the base coordinate system of the main robotic arm, and the third relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle coordinate system of the template vehicle, determine the fourth relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle coordinate system of the vehicle under test. S4, obtain the pixel coordinates of the target body defect of the vehicle under test collected by the target 2D camera, and transform the pixel coordinates into the body coordinate system of the vehicle under test based on the second relative transformation matrix and the fourth relative transformation matrix to obtain the corrected coordinates of the target body defect. The target 2D camera is any 2D camera set at the end of the target robotic arm.

[0007] In one possible implementation, step S3 includes: S31, based on the first relative transformation matrix, the second relative transformation matrix, the third relative transformation matrix and the relative change in vehicle body pose, determine the fifth relative transformation matrix from the vehicle body coordinate system of the test vehicle to the vehicle body coordinate system of the template vehicle; S32, Based on the third relative transformation matrix and the fifth relative transformation matrix, determine the fourth relative transformation matrix.

[0008] In one possible implementation, step S4 includes: S41, based on the sixth relative transformation matrix from the pixel coordinate system of the target 2D camera to the camera coordinate system of the target 2D camera, the seventh relative transformation matrix from the camera coordinate system of the target 2D camera to the camera coordinate system of the target main camera, the eighth relative transformation matrix from the camera coordinate system of the target main camera to the flange end coordinate system of the target robotic arm, the ninth relative transformation matrix from the flange end coordinate system of the target robotic arm to the base coordinate system of the target robotic arm, the second relative transformation matrix, and the fourth relative transformation matrix, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect. The target main camera is the main camera set at the end of the target robotic arm.

[0009] In one possible implementation, when the target 2D camera is the target main camera, step S41 includes: Based on the sixth relative transformation matrix, the eighth relative transformation matrix, the ninth relative transformation matrix, the second relative transformation matrix, and the fourth relative transformation matrix, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect.

[0010] In one possible implementation, when the target robotic arm is the main robotic arm, step S3 includes: S31', based on the first relative transformation matrix, the third relative transformation matrix and the relative change in vehicle body pose, determine the fifth relative transformation matrix from the vehicle body coordinate system of the test vehicle to the vehicle body coordinate system of the template vehicle; S32', Based on the third relative transformation matrix and the fifth relative transformation matrix, determine the fourth relative transformation matrix.

[0011] In one possible implementation, when the target robotic arm is the main robotic arm, step S4 includes: S41', based on the sixth relative transformation matrix from the pixel coordinate system of the target 2D camera to the camera coordinate system of the target 2D camera, the seventh relative transformation matrix from the camera coordinate system of the target 2D camera to the camera coordinate system of the target main camera, the eighth relative transformation matrix from the camera coordinate system of the target main camera to the flange end coordinate system of the target robotic arm, the ninth relative transformation matrix from the flange end coordinate system of the target robotic arm to the base coordinate system of the target robotic arm, and the fourth relative transformation matrix, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect. The target main camera is the main camera set at the end of the target robotic arm.

[0012] In one possible implementation, when the target 2D camera is the target main camera, step S41' includes: Based on the sixth relative transformation matrix, the eighth relative transformation matrix, the ninth relative transformation matrix, and the fourth relative transformation matrix, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect.

[0013] Secondly, this application provides a vehicle body defect coordinate mapping correction system, the system comprising: a controller, one or more 3D cameras disposed on a detection station, a plurality of robotic arms, and a plurality of 2D cameras disposed at the ends of each of the robotic arms; The target 3D camera is any one of the 3D cameras at the detection station, used to acquire real-time point clouds of the vehicle under test and send them to the controller; The target 2D camera is any 2D camera installed at the end of the target robotic arm, used to acquire the pixel coordinates of the target body defects of the vehicle under test and send them to the controller; the target robotic arm is the robotic arm at the detection station that is within a preset range and closest to the target 3D camera; The controller is configured to: acquire a first relative transformation matrix from the camera coordinate system of the target 3D camera to the base coordinate system of the target robotic arm; register the real-time point cloud of the vehicle under test acquired by the target 3D camera with the template point cloud of the template vehicle to obtain the relative change in vehicle body pose in the camera coordinate system of the target 3D camera, which is used to indicate the relative change in pose between the vehicle under test and the template vehicle; determine a fourth relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the vehicle under test based on the first relative transformation matrix, the relative change in vehicle body pose, a second relative transformation matrix from the base coordinate system of the target robotic arm to the base coordinate system of the main robotic arm, and a third relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the template vehicle; acquire the pixel coordinates of the target vehicle body defect acquired by the target 2D camera, and transform the pixel coordinates to the vehicle body coordinate system of the vehicle under test based on the second relative transformation matrix and the fourth relative transformation matrix to obtain the corrected coordinates of the target vehicle body defect.

[0014] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in the first aspect or any of the implementations thereof.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in the first aspect or any of the implementations thereof.

[0016] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method as described in the first aspect or any of the implementations thereof.

[0017] The advantages of this application compared to the prior art are as follows: First, a first relative transformation matrix is ​​obtained from the camera coordinate system of the target 3D camera at the inspection station to the base coordinate system of the target robotic arm; second, the real-time point cloud of the vehicle under test acquired by the target 3D camera is registered with the template point cloud of the template vehicle to obtain the relative change in vehicle body pose in the camera coordinate system of the target 3D camera, which is used to indicate the relative change in pose from the vehicle under test to the template vehicle; third, based on the first relative transformation matrix, the relative change in vehicle body pose, the second relative transformation matrix from the base coordinate system of the target robotic arm to the base coordinate system of the main robotic arm, and the third relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the template vehicle, the main robotic arm is determined. The fourth relative transformation matrix is ​​used to transform the base coordinate system of the test vehicle to the body coordinate system of the test vehicle. The second and fourth relative transformation matrices are used to correct the coordinate mapping of the body defect. The pixel coordinates of the target body defect of the test vehicle acquired by the target 2D camera are transformed into the body coordinate system of the test vehicle to obtain the corrected coordinates of the target body defect. Compared with directly mapping the coordinates of the body defect position through the body coordinate system of the preset template vehicle, this method avoids problems such as inaccurate positioning of body defect coordinates and missed defects when the position of the test vehicle changes relative to the preset template vehicle calibration position. This improves the positioning accuracy of body defects and increases the polishing efficiency of workers or automated equipment.

[0018] It is understood that the vehicle body defect coordinate mapping correction system, electronic device, computer-readable storage medium and computer program product provided in this application have the same beneficial effects as the above-described vehicle body defect coordinate mapping correction method, and will not be repeated here. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the architecture of a vehicle body defect coordinate mapping correction system provided in an embodiment of this application; Figure 2 A flowchart illustrating a method for correcting vehicle body defect coordinate mapping according to an embodiment of this application; Figure 3 This is a comparison diagram of the vehicle body defect mapping results before and after correction, provided as an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of this application. However, those skilled in the art will understand that this application may be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0023] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0025] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0027] This application proposes a method, system, device, and medium for correcting vehicle body defect coordinate mapping. It breaks through the limitations of traditional static calibration systems by utilizing a 3D camera to acquire point cloud data of the vehicle body. The relative transformation matrix of the vehicle body relative to the main robotic arm is corrected through point cloud registration, thereby correcting the offset of the defect coordinates and achieving precise positioning of the vehicle body defect. Furthermore, the two-dimensional image coordinates of the vehicle body defect are mapped using ray tracing to quickly obtain its three-dimensional coordinates on the vehicle body, improving the efficiency of vehicle body defect positioning. For ease of understanding, the technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0028] This application provides a vehicle body defect coordinate mapping correction system, including: a controller, one or more 3D cameras set on the inspection station, several robotic arms, and several 2D cameras set at the end of each robotic arm; The target 3D camera is any 3D camera on the inspection station, used to acquire the real-time point cloud of the vehicle under test and send it to the controller; The target 2D camera is any 2D camera set at the end of the target robotic arm, used to acquire the pixel coordinates of the target body defects of the vehicle under test and send them to the controller; the target robotic arm is the robotic arm at the inspection station that is within a preset range and closest to the target 3D camera. The controller is used to acquire the first relative transformation matrix from the camera coordinate system of the target 3D camera to the base coordinate system of the target robotic arm; to register the real-time point cloud of the vehicle under test acquired by the target 3D camera with the template point cloud of the template vehicle to obtain the relative change of the vehicle body pose in the camera coordinate system of the target 3D camera, which is used to indicate the relative change of the vehicle body pose from the vehicle under test to the template vehicle; based on the first relative transformation matrix, the relative change of the vehicle body pose, the second relative transformation matrix from the base coordinate system of the target robotic arm to the base coordinate system of the main robotic arm, and the third relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the template vehicle, to determine the fourth relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the vehicle under test; to acquire the pixel coordinates of the target vehicle body defect acquired by the target 2D camera, and to transform the pixel coordinates to the vehicle body coordinate system of the vehicle under test based on the second and fourth relative transformation matrices to obtain the corrected coordinates of the target vehicle body defect.

[0029] Furthermore, the system also includes a light source and a data storage system. Several 2D cameras and light sources form a deflection imaging system, which is installed at the end of each robotic arm to acquire images of the vehicle body surface and obtain the two-dimensional pixel coordinates of vehicle body defects. The controller connects to each 2D and 3D camera and is equipped with a target detection model and a vehicle body defect localization algorithm. The data storage system communicates with the controller to receive and display the vehicle body defect localization results.

[0030] Optionally, the inspection station can be equipped with different numbers of robotic arms and 3D cameras according to actual conditions. When one 3D camera is set at the inspection station, it is preferred to place the 3D camera at the front longitudinal beam of the vehicle body, and the robotic arm that is closest to the 3D camera within a preset range is designated as the main robotic arm. When two 3D cameras are set at the inspection station, they are preferably placed at the front and rear longitudinal beams of the vehicle body, respectively, and any one of the two robotic arms that is closest to the two 3D cameras within a preset range is designated as the main robotic arm. When four 3D cameras are set at the inspection station, they are preferably placed at the four corners of the vehicle body, where the imaging satisfies the diffuse reflection condition, and any one of the four robotic arms that is closest to the four 3D cameras within a preset range is designated as the main robotic arm. Each robotic arm is equipped with a deflection imaging system at its end, and any one of the 2D cameras in each deflection imaging system is selected as the main camera.

[0031] Preferred, such as Figure 1 As shown, the vehicle body defect coordinate mapping correction system 10 includes four 3D cameras (referred to as the first 3D camera, the second 3D camera, the third 3D camera, and the fourth 3D camera), four robotic arms (referred to as the first robotic arm, the second robotic arm, the third robotic arm, and the fourth robotic arm), a controller, and a data storage system. The four 3D cameras are respectively set at the four corners of the vehicle body. The robotic arm that is closest to the first 3D camera within a preset range is the first robotic arm, the robotic arm that is closest to the second 3D camera within a preset range is the second robotic arm, the robotic arm that is closest to the third 3D camera within a preset range is the third robotic arm, and the robotic arm that is closest to the fourth 3D camera within a preset range is the fourth robotic arm. The trajectories of each 3D camera and the robotic arm closest to it within the preset range do not interfere with each other. Any one of the four robotic arms is selected as the main robotic arm.

[0032] The first robotic arm has a first deflection imaging system at its end effector, the second robotic arm has a second deflection imaging system at its end effector, the third robotic arm has a third deflection imaging system at its end effector, and the fourth robotic arm has a fourth deflection imaging system at its end effector. Each deflection imaging system includes several 2D cameras and a light source (not shown in the figure). The data storage system is communicatively connected to the controller, and the controller is communicatively connected to each 3D camera and each 2D camera.

[0033] Understandable Figure 1 The schematic diagram shown is merely an example of the vehicle body defect coordinate mapping correction system provided in this application. In other embodiments of this application, the vehicle body defect coordinate mapping correction system 10 may include more or fewer components than those shown, or combine some components, or split some components, or have different component arrangements. The components shown may be implemented in hardware, software, or a combination of software and hardware, and this application does not limit them in this regard.

[0034] In response to the aforementioned vehicle body defect coordinate mapping correction system, this application proposes a vehicle body defect coordinate mapping correction method. The working principle involves point cloud registration, relative transformation between multiple coordinate systems, and three-dimensional coordinate mapping of defects.

[0035] First, refer to Figure 1 The system architecture diagram shown first briefly explains the existing technology of using a robotic arm carrying a 2D camera to map defects in the vehicle body paint surface: By performing hand-eye calibration on the four robotic arms, the relative transformation matrices from the camera coordinate system of each 2D camera to the coordinate system of the robotic arm end effector, the relative transformation matrices from the base coordinate system of each robotic arm to the base coordinate system of the main robotic arm, and the relative transformation matrices from the vehicle body coordinate system of the template vehicle to the camera coordinate system of each 2D camera can be obtained; by performing multi-target calibration on the deflection imaging system carried by the robotic arm, the relative transformation matrices from the camera coordinate system of each 2D camera to the main camera coordinate system can be obtained; using the self-developed Kunwu platform, combined with vehicle model and virtual simulation construction technology, the relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the template vehicle can be obtained. Then, coordinate transformation can be used to convert the image coordinates of the vehicle body defect from pixel coordinates to the corresponding 2D camera coordinates, then to the main camera coordinates, then to the corresponding robotic arm end effector coordinates, then to the robotic arm base coordinates, then to the main robotic arm base coordinates, and finally to the vehicle body coordinates of the template car. Combined with ray tracing, the 3D defect coordinates in the vehicle body coordinates can be obtained, achieving 3D mapping of the vehicle body defect. It should be noted that the calibration process for the above relative transformation matrices can refer to existing calibration schemes, and will not be elaborated upon here. In the following description of the technical solution of this application, the above relative transformation matrices will be treated as known quantities.

[0036] However, when there is a deviation between the actual body position of the vehicle under test and the body position of the template vehicle during calibration, the relative transformation matrix from the base coordinate system of the main robotic arm to the body coordinate system of the template vehicle in the data obtained by the above scheme will not reflect the true situation of the vehicle under test, resulting in a deviation in the defects finally mapped onto the vehicle body.

[0037] Based on the aforementioned technical problems, this application proposes a method for correcting the coordinate mapping of vehicle body defects. The method involves using the point cloud of the vehicle body to be tested, acquired by a 3D camera, to calculate the relative pose change between the vehicle body and the template vehicle in the camera coordinate system of the 3D camera. Then, through coordinate transformation, this is converted to the vehicle body coordinate system of the vehicle body to obtain the relative transformation matrix from the vehicle body to the template vehicle. Finally, using the relative transformation matrix from the vehicle body to the template vehicle, the relative transformation matrix from the base coordinate system of the calibrated main robotic arm to the vehicle body coordinate system of the template vehicle is updated online in real time. The three-dimensional mapping coordinates of the vehicle body defects are corrected by correcting the relative transformation matrix. The specific implementation process is as follows: Figure 2 As shown.

[0038] Figure 2 This is a flowchart illustrating a method for correcting vehicle body defect coordinate mapping according to an embodiment of this application. For ease of explanation, only the parts relevant to this embodiment are shown. The method provided in this embodiment includes the following steps: S1, obtain the first relative transformation matrix from the camera coordinate system of the target 3D camera to the base coordinate system of the target robotic arm. The target 3D camera is any 3D camera on the detection station, and the target robotic arm is the robotic arm on the detection station that is within a preset range and closest to the target 3D camera.

[0039] Optional, with Figure 1 Taking the vehicle body defect coordinate mapping correction system shown as an example, if the second 3D camera is selected as the target 3D camera, then the second robotic arm that is closest to the second 3D camera within a preset range is the target robotic arm. By performing eye-to-hand calibration on the second 3D camera and the second robotic arm, the first relative transformation matrix from the camera coordinate system of the target 3D camera to the base coordinate system of the target robotic arm is obtained. .

[0040] S2, register the real-time point cloud of the vehicle under test acquired by the target 3D camera with the template point cloud of the template vehicle to obtain the relative change of the vehicle body pose in the camera coordinate system of the target 3D camera, which is used to indicate the relative change of the pose from the vehicle under test to the template vehicle.

[0041] Optionally, before performing step S2, the original point cloud of the template vehicle is acquired by the target 3D camera. If the shooting position is diffuse reflection, the region of interest (ROI) is set according to the generation quality of the point cloud, and the original point cloud is cropped to create a template point cloud corresponding to the target 3D camera.

[0042] As an example, step S2 may optionally include: Acquire the real-time point cloud of the vehicle under test captured by the target 3D camera; Perform preprocessing operations such as point cloud segmentation, filtering, and voxel downsampling on real-time point clouds; Using a point cloud registration algorithm, the preprocessed real-time point cloud is registered with the template point cloud corresponding to the target 3D camera, thereby obtaining the relative change in vehicle body pose in the camera coordinate system of the target 3D camera. .

[0043] S3. Based on the first relative transformation matrix, the relative change in vehicle body pose, the second relative transformation matrix from the base coordinate system of the target robotic arm to the base coordinate system of the main robotic arm, and the third relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the template vehicle, determine the fourth relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the vehicle under test.

[0044] In one possible implementation, step S3 includes: S31. Based on the first relative transformation matrix, the second relative transformation matrix, the third relative transformation matrix and the relative change in vehicle body pose, determine the fifth relative transformation matrix from the vehicle body coordinate system of the test vehicle to the vehicle body coordinate system of the template vehicle. S32, based on the third and fifth relative transformation matrices, determine the fourth relative transformation matrix.

[0045] For example, see Figure 1 The vehicle body defect coordinate mapping correction system shown selects the second 3D camera as the target 3D camera. The second robotic arm, which is within a preset range and closest to the second 3D camera, is then selected as the target robotic arm. The first robotic arm is selected as the main robotic arm. In the case where the target robotic arm is not the main robotic arm, the system is based on the first relative transformation matrix. Second relative transformation matrix The third relative transformation matrix and the relative change in vehicle body position Determine the fifth relative transformation matrix from the vehicle body coordinate system of the test vehicle to the vehicle body coordinate system of the template vehicle. Then based on the third relative transformation matrix and the fifth relative transformation matrix Determine the fourth relative transformation matrix .

[0046] As an example, when the target robotic arm is the master robotic arm, step S3 includes: S31', based on the first relative transformation matrix, the third relative transformation matrix and the relative change in vehicle body pose, determine the fifth relative transformation matrix from the vehicle body coordinate system of the test vehicle to the vehicle body coordinate system of the template vehicle. S32', based on the third and fifth relative transformation matrices, determine the fourth relative transformation matrix.

[0047] For example, see Figure 1 The vehicle body defect coordinate mapping correction system shown selects a first 3D camera as the target 3D camera and a first robotic arm as the main robotic arm. The first robotic arm, which is within a preset range and closest to the first 3D camera, is then considered the target robotic arm. In this case, both the target robotic arm and the main robotic arm are the first robotic arm. Based on the first relative transformation matrix... The third relative transformation matrix and the relative change in vehicle body position Determine the fifth relative transformation matrix from the vehicle body coordinate system of the test vehicle to the vehicle body coordinate system of the template vehicle. Based on the third relative transformation matrix and the fifth relative transformation matrix Determine the fourth relative transformation matrix .

[0048] S4. Obtain the pixel coordinates of the target body defect of the vehicle under test acquired by the target 2D camera, and transform the pixel coordinates to the vehicle body coordinate system of the vehicle under test based on the second relative transformation matrix and the fourth relative transformation matrix to obtain the corrected coordinates of the target body defect. The target 2D camera is any 2D camera set at the end of the target robotic arm.

[0049] Optionally, the target vehicle body defect can be any vehicle body defect captured by the target 2D camera.

[0050] In one possible implementation, step S4 includes: S41, based on the sixth relative transformation matrix from the pixel coordinate system of the target 2D camera to the camera coordinate system of the target 2D camera, the seventh relative transformation matrix from the camera coordinate system of the target 2D camera to the camera coordinate system of the target main camera, the eighth relative transformation matrix from the camera coordinate system of the target main camera to the flange end coordinate system of the target robotic arm, the ninth relative transformation matrix from the flange end coordinate system of the target robotic arm to the base coordinate system of the target robotic arm, the second relative transformation matrix, and the fourth relative transformation matrix, the pixel coordinates are transformed to the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect. The target main camera is the main camera set at the end of the target robotic arm.

[0051] For example, see Figure 1 The vehicle body defect coordinate mapping correction system shown uses a second 3D camera as the target 3D camera and a first robotic arm as the main robotic arm. The second robotic arm, being within a preset range and closest to the second 3D camera, is the target robotic arm. The second robotic arm is equipped with several 2D cameras. Any one of these cameras is selected as the main camera, and any 2D camera other than the main camera is selected as the target 2D camera. That is, when the target robotic arm is not the main robotic arm and the target 2D camera is not the target main camera, the sixth relative transformation matrix from the pixel coordinate system of the target 2D camera to the camera coordinate system of the target 2D camera is used. The seventh relative transformation matrix from the camera coordinate system of the target 2D camera to the camera coordinate system of the target main camera. The eighth relative transformation matrix from the camera coordinate system of the target main camera to the flange end coordinate system of the target robotic arm. The ninth relative transformation matrix from the target robot arm's flange end coordinate system to the target robot arm's base coordinate system. Second relative transformation matrix and the fourth relative transformation matrix , pixel coordinates Transform to the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target body defect. .

[0052] As an example, when the target 2D camera is the target main camera, step S41 includes: Based on the sixth, eighth, ninth, second, and fourth relative transformation matrices, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect.

[0053] For example, when the target robotic arm is not the main robotic arm, and the target 2D camera is the target main camera, then based on the sixth relative transformation matrix... The eighth relative transformation matrix Ninth relative transformation matrix Second relative transformation matrix and the fourth relative transformation matrix , pixel coordinates Transform to the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target body defect. .

[0054] In another possible implementation, when the target robotic arm is the master robotic arm, step S4 includes: S41', based on the sixth relative transformation matrix from the pixel coordinate system of the target 2D camera to the camera coordinate system of the target 2D camera, the seventh relative transformation matrix from the camera coordinate system of the target 2D camera to the camera coordinate system of the target main camera, the eighth relative transformation matrix from the camera coordinate system of the target main camera to the flange end coordinate system of the target robotic arm, the ninth relative transformation matrix from the flange end coordinate system of the target robotic arm to the base coordinate system of the target robotic arm, and the fourth relative transformation matrix, the pixel coordinates are transformed to the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect. The target main camera is the main camera set at the end of the target robotic arm.

[0055] For example, see Figure 1 The vehicle body defect coordinate mapping correction system shown selects a first 3D camera as the target 3D camera and a first robotic arm as the main robotic arm. The first robotic arm, which is within a preset range and closest to the first 3D camera, is the target robotic arm. In this case, both the target robotic arm and the main robotic arm are the first robotic arm. When the selected target 2D camera is not the target main camera, the sixth relative transformation matrix from the pixel coordinate system of the target 2D camera to the camera coordinate system of the target 2D camera is used. The seventh relative transformation matrix from the camera coordinate system of the target 2D camera to the camera coordinate system of the target main camera. The eighth relative transformation matrix from the camera coordinate system of the target main camera to the flange end coordinate system of the target robotic arm. The ninth relative transformation matrix from the target robot arm's flange end coordinate system to the target robot arm's base coordinate system. and the fourth relative transformation matrix , pixel coordinates Transform to the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target body defect. .

[0056] As an example, when the target 2D camera is the target main camera, step S41' includes: Based on the sixth, eighth, ninth, and fourth relative transformation matrices, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect.

[0057] For example, when the target robotic arm is the main robotic arm and the target 2D camera is the target main camera, then based on the sixth relative transformation matrix... The eighth relative transformation matrix Ninth relative transformation matrix and the fourth relative transformation matrix , the pixel coordinates of the target vehicle body defect Transform to the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target body defect. .

[0058] As an example, the technical solution of this application is verified, such as... Figure 3 As shown, (a) is the vehicle body defect detected by the vehicle body surface defect detection model, (b) is the result of vehicle body defect coordinate mapping correction without using the technical solution of this application, and (c) is the result of vehicle body defect coordinate mapping correction using the technical solution of this application. It can be seen that the technical solution provided by this application can accurately map vehicle body defects into three-dimensional coordinates in the vehicle body coordinate system. Compared with the vehicle body defect coordinate mapping without using the technical solution of this application, it improves the accuracy of vehicle body defect positioning and avoids missed defect detection.

[0059] In summary, to address the problems of low accuracy and long mapping time in existing automotive body defect detection technologies due to changes in the vehicle's pose between the inspection station and calibration time, this application provides a method for correcting vehicle body defect coordinate mapping. It introduces real-time point cloud registration technology, continuously monitors vehicle pose changes using a 3D camera, and develops an online relative transformation matrix update algorithm to achieve dynamic correction of coordinate mapping relationships. Simultaneously, it overcomes the performance limitations of traditional trimesh algorithms, achieving an accelerated ray tracing mapping method. Specifically, maintaining the previous results in the mapping stage, the 3D coordinates of the defect in the vehicle coordinate system are calculated using ray tracing mapping. It is worth noting that most research teams currently develop ray tracing mapping algorithms based on trimesh. However, when the number of defects is large, trimesh-based ray tracing mapping algorithms are time-consuming. This application uses the latest version of Open3d (0.19.0) to support ray tracing and provides related acceleration interfaces; therefore, this application develops a ray tracing mapping algorithm based on Open3d. Experiments show that the ray tracing mapping algorithm based on Open3d significantly improves mapping speed while maintaining accuracy. Taking 5000 rays as an example, ray tracing mapping based on trimesh takes approximately 200 seconds, while the newly encapsulated ray tracing mapping method in this application takes less than 0.5 seconds. Furthermore, the technical solution provided in this application accurately maps vehicle body defects to three-dimensional coordinates in the vehicle body coordinate system. It corrects vehicle body offset using a 3D camera and performs coordinate transformation using calibrated data, updating the relative transformation matrix from the robotic arm coordinate system to the vehicle body coordinate system. This corrects the three-dimensional coordinates of the mapped vehicle body defects, improving defect positioning accuracy from ±20mm to ±5mm. This achieves adaptive accuracy maintenance under vehicle body pose changes, breaking the dependence of traditional systems on fixed fixtures.

[0060] The technical solution provided in this application obtains a first relative transformation matrix from the camera coordinate system of the target 3D camera to the base coordinate system of the target robotic arm at the inspection station; registers the real-time point cloud of the vehicle under test acquired by the target 3D camera with the template point cloud of the template vehicle to obtain the relative change in vehicle body pose in the camera coordinate system of the target 3D camera, which is used to indicate the relative change in pose from the vehicle under test to the template vehicle; and determines the base coordinates of the main robotic arm based on the first relative transformation matrix, the relative change in vehicle body pose, the second relative transformation matrix from the base coordinate system of the target robotic arm to the base coordinate system of the main robotic arm, and the third relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the template vehicle. The fourth relative transformation matrix is ​​used to map the vehicle body coordinate system of the vehicle under test. The second and fourth relative transformation matrices are used to correct the coordinate mapping of the vehicle body defects. The pixel coordinates of the target vehicle body defects captured by the target 2D camera are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defects. Compared with directly mapping the coordinates of the vehicle body defects through the preset template vehicle's coordinate system, this method avoids problems such as inaccurate positioning of vehicle body defects and missed defects when the position of the vehicle under test changes relative to the preset template vehicle's calibration position. This improves the positioning accuracy of vehicle body defects and increases the polishing efficiency of workers or automated equipment.

[0061] On the other hand, this application also provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-mentioned vehicle body defect coordinate mapping correction methods.

[0062] On the other hand, this application also provides an electronic device, including a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute any of the above-mentioned vehicle body defect coordinate mapping correction methods.

[0063] For example, the program code may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the program code in an electronic device.

[0064] The electronic device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The electronic device may include, but is not limited to, processors and memory. Those skilled in the art will understand that the electronic device may also include input / output devices, network access devices, buses, etc.

[0065] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0066] The memory can be an internal storage unit of the electronic device, such as a hard drive or RAM. It can also be an external storage device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units. The memory is used to store the program code and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output.

[0067] The computer storage medium and electronic device described above are created based on the above method. Their technical functions and beneficial effects will not be elaborated here. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for correcting vehicle body defect coordinate mapping, characterized in that, The method includes: S1, obtain the first relative transformation matrix from the camera coordinate system of the target 3D camera to the base coordinate system of the target robotic arm, wherein the target 3D camera is any 3D camera on the detection station, and the target robotic arm is the robotic arm on the detection station that is within a preset range and closest to the target 3D camera; S2, register the real-time point cloud of the vehicle under test acquired by the target 3D camera with the template point cloud of the template vehicle to obtain the relative change of the vehicle body pose in the camera coordinate system of the target 3D camera, which is used to indicate the relative change of the pose from the vehicle under test to the template vehicle. S3. Based on the first relative transformation matrix, the relative change in vehicle posture, the second relative transformation matrix from the base coordinate system of the target robotic arm to the base coordinate system of the main robotic arm, and the third relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle coordinate system of the template vehicle, determine the fourth relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle coordinate system of the vehicle under test. S4, obtain the pixel coordinates of the target body defect of the vehicle under test collected by the target 2D camera, and transform the pixel coordinates into the body coordinate system of the vehicle under test based on the second relative transformation matrix and the fourth relative transformation matrix to obtain the corrected coordinates of the target body defect. The target 2D camera is any 2D camera set at the end of the target robotic arm.

2. The method according to claim 1, characterized in that, Step S3 includes: S31, based on the first relative transformation matrix, the second relative transformation matrix, the third relative transformation matrix and the relative change in vehicle body pose, determine the fifth relative transformation matrix from the vehicle body coordinate system of the test vehicle to the vehicle body coordinate system of the template vehicle; S32, Based on the third relative transformation matrix and the fifth relative transformation matrix, determine the fourth relative transformation matrix.

3. The method according to claim 1, characterized in that, Step S4 includes: S41, based on the sixth relative transformation matrix from the pixel coordinate system of the target 2D camera to the camera coordinate system of the target 2D camera, the seventh relative transformation matrix from the camera coordinate system of the target 2D camera to the camera coordinate system of the target main camera, the eighth relative transformation matrix from the camera coordinate system of the target main camera to the flange end coordinate system of the target robotic arm, the ninth relative transformation matrix from the flange end coordinate system of the target robotic arm to the base coordinate system of the target robotic arm, the second relative transformation matrix, and the fourth relative transformation matrix, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect. The target main camera is the main camera set at the end of the target robotic arm.

4. The method according to claim 3, characterized in that, When the target 2D camera is the target main camera, step S41 includes: Based on the sixth relative transformation matrix, the eighth relative transformation matrix, the ninth relative transformation matrix, the second relative transformation matrix, and the fourth relative transformation matrix, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect.

5. The method according to claim 1, characterized in that, When the target robotic arm is the main robotic arm, step S3 includes: S31', based on the first relative transformation matrix, the third relative transformation matrix and the relative change in vehicle body pose, determine the fifth relative transformation matrix from the vehicle body coordinate system of the test vehicle to the vehicle body coordinate system of the template vehicle; S32', Based on the third relative transformation matrix and the fifth relative transformation matrix, determine the fourth relative transformation matrix.

6. The method according to claim 1, characterized in that, When the target robotic arm is the main robotic arm, step S4 includes: S41', based on the sixth relative transformation matrix from the pixel coordinate system of the target 2D camera to the camera coordinate system of the target 2D camera, the seventh relative transformation matrix from the camera coordinate system of the target 2D camera to the camera coordinate system of the target main camera, the eighth relative transformation matrix from the camera coordinate system of the target main camera to the flange end coordinate system of the target robotic arm, the ninth relative transformation matrix from the flange end coordinate system of the target robotic arm to the base coordinate system of the target robotic arm, and the fourth relative transformation matrix, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect. The target main camera is the main camera set at the end of the target robotic arm.

7. The method according to claim 6, characterized in that, When the target 2D camera is the target main camera, step S41' includes: Based on the sixth relative transformation matrix, the eighth relative transformation matrix, the ninth relative transformation matrix, and the fourth relative transformation matrix, the pixel coordinates are transformed into the vehicle body coordinate system of the vehicle under test to obtain the corrected coordinates of the target vehicle body defect.

8. A vehicle body defect coordinate mapping correction system, characterized in that, The system includes: a controller, one or more 3D cameras set on the inspection station, several robotic arms, and several 2D cameras set at the ends of each robotic arm; The target 3D camera is any one of the 3D cameras at the detection station, used to acquire real-time point clouds of the vehicle under test and send them to the controller; The target 2D camera is any 2D camera installed at the end of the target robotic arm, used to acquire the pixel coordinates of the target body defects of the vehicle under test and send them to the controller; the target robotic arm is the robotic arm at the detection station that is within a preset range and closest to the target 3D camera; The controller is configured to: acquire a first relative transformation matrix from the camera coordinate system of the target 3D camera to the base coordinate system of the target robotic arm; register the real-time point cloud of the vehicle under test acquired by the target 3D camera with the template point cloud of the template vehicle to obtain the relative change in vehicle body pose in the camera coordinate system of the target 3D camera, which is used to indicate the relative change in pose between the vehicle under test and the template vehicle; determine a fourth relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the vehicle under test based on the first relative transformation matrix, the relative change in vehicle body pose, a second relative transformation matrix from the base coordinate system of the target robotic arm to the base coordinate system of the main robotic arm, and a third relative transformation matrix from the base coordinate system of the main robotic arm to the vehicle body coordinate system of the template vehicle; acquire the pixel coordinates of the target vehicle body defect acquired by the target 2D camera, and transform the pixel coordinates to the vehicle body coordinate system of the vehicle under test based on the second relative transformation matrix and the fourth relative transformation matrix to obtain the corrected coordinates of the target vehicle body defect.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images