A method and system for 2D defect back-projection to 3D digital model

Abstract

The application relates to a 2D defect back-projection-to-3D digital model method and system, and belongs to the technical field of vehicle body detection. An image of non-collinear feature points on a vehicle body is collected in real time by a positioning camera, sub-pixel level coordinates are extracted, a real-time 6-degree-of-freedom pose of the vehicle body is solved and smoothed in combination with a PnP algorithm and an extended Kalman filter (EKF), meanwhile, a fixed target is used to dynamically calibrate and compensate for drift of a defect detection camera; subsequently, a convolutional neural network (CNN) is used to identify defect points in an image, and according to camera internal and external parameters, real-time vehicle body pose and a unified coordinate system established by a laser tracker, pixel coordinates of the defect points are high-precision back-projected to a three-dimensional vehicle body CAD digital model surface through a multi-step coordinate transformation chain, and finally mapped through k-d tree nearest neighbor search. The application realizes full-automatic, high-precision and strong-adaptive defect positioning and visualization, and significantly improves detection efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of vehicle body inspection technology, and in particular to a method and system for back-projecting 2D defects onto a 3D digital model. Background Technology

[0002] Currently, in the automotive manufacturing and inspection field, especially in the defect detection of body-in-white, 2D image inspection systems are commonly used to inspect the surface quality of the vehicle body. Traditional methods rely on static tooling or single-point vision systems to detect defects, followed by manual comparison or simplified geometric mapping to mark the approximate location of the defects on a 3D vehicle body model. This method mainly includes: capturing defect images using a fixed-mounted 2D camera; performing planar projection and coordinate transformation using calibration plates or simple geometric relationships; and relying on manual assistance to complete the positioning and alignment between the 2D inspection and the 3D model. This results in the following problems: 1. Low positioning accuracy: Existing solutions lack a complete attitude information acquisition process, making it impossible to accurately achieve the back projection of defect detection points onto the 3D vehicle body model. 2. Inconsistent coordinate system: A unified coordinate system is not established between the defect detection camera, the vehicle body positioning device, and the 3D digital model, affecting projection accuracy. 3. Lack of automation: The defect mapping process relies heavily on manual intervention, which is detrimental to industrial automation integration and production cycle control. 4. Ignoring attitude deviations: Ignoring the 6-DOF attitude deviations of the vehicle body when it is actually installed at the inspection station leads to large back projection errors of defect points. Summary of the Invention

[0003] Therefore, the purpose of this invention is to provide a method and system for back-projecting 2D defects to a 3D digital model, which solves the detection error caused by vehicle body position offset by combining vehicle body pose with the back-projection mechanism.

[0004] To achieve the above objectives, the present invention provides a method for back-projecting a 2D defect to a 3D digital model, comprising the following steps:

[0005] S1. Obtain the real-time 6-DOF pose information of the current body-in-white at the inspection station;

[0006] S2. Dynamically calibrate the defect detection camera. Based on the calibration reference fixed at the detection station, determine the external parameter matrix of the defect detection camera relative to the coordinate system of the detection station in real time.

[0007] S3. Based on the real-time 6-DOF pose information, the vehicle body surface image is acquired through the defect detection camera, and the pixel coordinates of the defect points are identified.

[0008] S4. Based on the real-time 6-DOF pose information, the extrinsic and intrinsic parameter matrices of the defect detection camera, the pixel coordinates of the defect point are back-projected onto the coordinate system of the three-dimensional mathematical model of the white body to obtain the precise spatial position of the defect point in the 3D body mathematical model.

[0009] Further preferred, in S1, the body-in-white is positioned, including:

[0010] S101. Select at least three feature holes in the key area of ​​the vehicle body as marker points; wherein the three feature holes are not collinear;

[0011] S102. Use a positioning camera to acquire images of the marker points and extract the sub-pixel-level image coordinates of the marker points;

[0012] S103. Based on the positioning camera intrinsic parameter matrix K and the three-dimensional template coordinates of the marker points, calculate the rotation matrix of the vehicle body relative to the coordinate system of the inspection station. With translation vector ; Obtain the real-time homogeneous transformation matrix of the vehicle body in the coordinate system of the inspection station;

[0013] S104. Filter the real-time homogeneous transformation matrix to obtain real-time 6-DOF pose information.

[0014] In a further preferred embodiment, in S103, based on the intrinsic parameter matrix K of the positioning camera and the three-dimensional template coordinates of the marker points, the PnP algorithm is used to solve for the rotation matrix of the vehicle body relative to the coordinate system of the detection station. With translation vector ;

[0015]

[0016] in, The three-dimensional template coordinates of the marker point This indicates the coordinates of the marker point in the pixel coordinate system.

[0017] In a further preferred embodiment, in S104, the real-time homogeneous transformation matrix is ​​filtered, and a stable vehicle attitude estimate is obtained as the real-time 6-DOF pose information according to the following formula:

[0018]

[0019] in, This is the estimated value of the vehicle body attitude. This refers to the vehicle's position at the previous moment. The vehicle's position at time t. This represents the extended Kalman filter function.

[0020] More preferably, in S2, the extrinsic parameter matrix of the defect detection camera relative to the coordinate system of the detection station is determined in real time based on a calibration reference fixed to the detection station, including:

[0021] During the detection cycle, the calibration reference object fixed at the detection station is identified, and the extrinsic parameter matrix of the defect detection camera is calculated in real time. ;

[0022] in, The three-dimensional position of the reference calibration point in the main coordinate system of the inspection station. This represents the pixel coordinates sampled at time t from the reference point; K is the camera intrinsic parameter.

[0023] Hardware trigger time synchronization is performed for all defect detection cameras;

[0024] When drift is detected in the extrinsic parameters of multiple defect detection cameras, the minimum reprojection error criterion is used to jointly optimize the extrinsic parameters of multiple cameras.

[0025] More preferably, when using the minimum reprojection error criterion to jointly optimize the extrinsic parameters of multiple cameras, the optimization function is as follows:

[0026]

[0027] in Represents the projection function. Indicates camera intrinsic parameters. This represents the extrinsic transformation matrix of the camera coordinate system relative to the detection station coordinate system during the i-th observation. This indicates the three-dimensional reference position of the marker point in the coordinate system of the inspection station. This represents the pixel coordinates of the marker point observed for the i-th time in the image.

[0028] More preferably, in S3, the step of acquiring an image of the vehicle body surface using a defect detection camera and identifying the pixel coordinates of the defect points includes:

[0029] Using a convolutional neural network to detect the region to be tested in the image coordinate system, and outputting a set of defect pixel center points:

[0030]

[0031] in, Indicates the center of the defective pixel;

[0032] Based on the intrinsic parameters and depth information of the defect detection camera, the three-dimensional coordinates of the defect point in the defect detection coordinate system are obtained:

[0033]

[0034] in, This represents the depth information corresponding to the center of the defective pixel, where K is a camera intrinsic parameter.

[0035] The three-dimensional coordinates of the defect points are back-projected onto the three-dimensional mathematical model coordinate system of the body-in-white:

[0036]

[0037] in, Represents the three-dimensional coordinates of the defect point in the camera coordinate system.

[0038] A further preferred embodiment includes, when back-projecting the three-dimensional coordinates of the defect points onto the three-dimensional mathematical model coordinate system of the body-in-white, the following:

[0039] After coarsely registering the 3D coordinates of the defect points with the surface of the 3D mathematical model, the nearest neighbor search is used to find a one-to-one correspondence between the defect points and the surface of the 3D mathematical model, thus forming the 3D spatial annotation result of the defect points.

[0040] More preferably, when performing coarse registration of the three-dimensional coordinates of the defect point with the surface of the three-dimensional mathematical model, the rotation matrix R and translation vector t for rigid body registration are calculated using the following optimization function: R Translation vector t The optimal solution:

[0041] ;

[0042] Where R is the rotation matrix and t is the translation vector. To calibrate any point in the feature point set of the 3D vehicle body mathematical model, It is any point in the feature point set on the actual vehicle body.

[0043] Furthermore, the step of using nearest neighbor points to search for a one-to-one correspondence between defect points and the numerical model surfaces of the three-dimensional mathematical model includes:

[0044] Construct a kd-tree from the CAD digital model point cloud Q_CAD;

[0045] For each defect point p, search for the nearest neighbor in the kd-tree:

[0046] a. Starting from the root node, recursively search downwards, deciding whether to search the left or right subtree based on the partition dimension and partition value of the current node, until a leaf node is reached;

[0047] b. During the backtracking process, check if there are any closer points, including points that might appear in another subtree;

[0048] Return the nearest neighbor point q, which is the corresponding point of the defect point p on the CAD model surface;

[0049] The final 3D spatial annotation result of the defect points:

[0050] .

[0051] The present invention also provides a system for back-projecting 2D defects to a 3D digital model, comprising the steps of the above-described method for back-projecting 2D defects to a 3D digital model, including:

[0052] Multi-camera module: including at least one vehicle body positioning camera and at least one defect detection camera;

[0053] Dynamic calibration module: configured to perform vehicle dynamic pose estimation and real-time calibration of camera extrinsic parameters;

[0054] Filtering module: Uses a Kalman filter to smooth the vehicle body pose;

[0055] Defect detection module: Identifies defect regions in images based on a deep learning model;

[0056] Back projection engine: configured to map defect points from the image coordinate system to the 3D digital model after multi-coordinate system transformation;

[0057] Visualization module: Used to display the location of defects on a 3D digital model.

[0058] The method and system for back-projecting 2D defects to a 3D digital model disclosed in this application have at least the following advantages compared to the prior art:

[0059] This application eliminates projection errors caused by changes in vehicle position / angle by calculating and compensating for the 6-DOF pose deviation of the vehicle body caused by hoisting and transportation in real time.

[0060] This application establishes a unified "metric system" for the entire site using a laser tracker, ensuring that the positioning camera, defect camera, and CAD model are on the same spatial reference, thus eliminating the cumulative error of coordinate system transformation between devices.

[0061] This application utilizes a positioning camera to quickly capture the vehicle's posture and a defect camera to finely scan the surface. All data are fused and processed through a unified coordinate system. This constructs a multimodal perception system that achieves a synergistic effect, improving detection efficiency while maintaining accuracy. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of a method for back-projecting a 2D defect to a 3D digital model proposed in this invention;

[0063] Figure 2 This is a structural block diagram of a 2D defect back-projection to a 3D digital model system proposed in this invention. Detailed Implementation

[0064] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] like Figure 1 As shown, one embodiment of the present invention provides a method for back-projecting a 2D defect onto a 3D digital model, which includes the following steps:

[0066] S1. Obtain the real-time 6-DOF pose information of the current body-in-white at the inspection station;

[0067] The process of locating the body-in-white includes:

[0068] S101. Select at least three feature holes of different lines in the key areas of the vehicle body as marker points; select at least three spatially non-collinear feature holes in the key positioning areas of the vehicle body (such as the front crossbeam, rear longitudinal beam, and door frame).

[0069] S102. Use a positioning camera to acquire images of the marker points and extract the sub-pixel-level image coordinates of the marker points;

[0070] Since digital images are composed of discrete pixel arrays, if a feature point is determined to be located at the pixel in the 10th row and 20th column, its coordinates are integers (20, 10). Therefore, in this application, subpixel-level image coordinates refer to the fractional part of the pixel with precision; that is, the precise location of the feature point is at (20.**, 10.**), which is the basis for the high-precision pose estimation of this application, which is based on millimeter-level back projection. Only by positioning the image coordinates at the subpixel level can subsequent pose calculation algorithms such as PnP be used. In this application, corner points based on gray-level centroids can be used, such as calculating the gray-level weighted center within the feature region for subpixel localization; or subpixel localization based on template matching can be used.

[0071] For example, an ideal template of marker points is created in advance, the best matching position is found in the image through normalized cross-correlation (NCC) or similarity measurement, and sub-pixel offset is obtained through interpolation.

[0072] S103. Based on the positioning camera intrinsic parameter matrix K and the three-dimensional template coordinates of the marker points, calculate the rotation matrix of the vehicle body relative to the coordinate system of the inspection station. With translation vector ; Obtain the real-time homogeneous transformation matrix of the vehicle body in the coordinate system of the inspection station;

[0073] In S103, based on the positioning camera intrinsic parameter matrix K and the 3D template coordinates of the marker points, the rotation matrix of the vehicle body relative to the coordinate system of the detection station is solved using the PnP algorithm. With translation vector 3D template coordinates refer to the vertex coordinates of an object in its own defined, local, and standardized 3D spatial coordinate system.

[0074]

[0075] in, The three-dimensional template coordinates of the marker point This indicates the coordinates of the marker point in the pixel coordinate system.

[0076] The PnP algorithm establishes a geometric constraint relationship between "feature points with known 3D coordinates" and "the 2D pixel coordinates of their projection in the image." It takes as input a set of 3D world coordinate points, their corresponding 2D pixel coordinates in the image, and the camera's intrinsic parameter matrix; and outputs the camera's rotation matrix and translation vector. The camera's rotation matrix and translation vector together constitute the transformation matrix of the camera coordinate system relative to the world coordinate system. In this application, PnP directly outputs a complete 6-DOF pose (3 rotational degrees of freedom + 3 translational degrees of freedom), which is the complete information necessary to compensate for any position and angle deviation of the vehicle body at the inspection station (i.e., the "ignoring attitude deviation" problem pointed out in the background art). Traditional methods may only perform simple 2D image matching or 1D measurement, failing to obtain the vehicle body's pitch, roll, and other rotational information, leading to systematic errors during projection. Since this application obtains sub-pixel-level image coordinates in S102 and uses 3D template coordinates from a high-precision CAD model for projection calculation, achieving an accuracy of 0.1mm, this "high-precision input +..." The combination of "precise geometric models" gives the pose calculated by PnP itself a very high absolute accuracy, which is an important foundation for all subsequent coordinate transformations and back projections to achieve millimeter-level accuracy.

[0077] The real-time homogeneous transformation matrix of the body-in-white in the coordinate system of the inspection station is obtained as follows:

[0078]

[0079] S104. Filter the real-time homogeneous transformation matrix to obtain real-time 6-DOF pose information. It should be noted that since this application is a dynamic calibration, the entire timeline matrix is ​​recorded and analyzed when performing dynamic smoothing and prediction on the pose data.

[0080] In S104, the real-time homogeneous transformation matrix is ​​filtered, and a stable vehicle attitude estimate is obtained as the real-time 6-DOF pose information according to the following formula: To eliminate the influence of camera shake and noise, an extended Kalman filter (EKF) is used to dynamically smooth and predict the pose data, forming a stable vehicle attitude estimate.

[0081]

[0082] in, This is the estimated value of the vehicle body attitude. This refers to the vehicle's position at the previous moment. The vehicle's position at time t. This represents the extended Kalman filter function.

[0083] S2. Dynamically calibrate the defect detection camera. Based on the calibration reference fixed at the detection station, determine the extrinsic parameter matrix of the defect detection camera relative to the coordinate system of the detection station in real time. The calibration reference can be a visual marker, a calibration plate, etc.

[0084] Based on a calibration reference fixed at the inspection station, the extrinsic parameter matrix of the defect detection camera relative to the coordinate system of the inspection station is determined in real time, including:

[0085] During the detection cycle, the calibration reference object fixed at the detection station is identified, and the extrinsic parameter matrix of the defect detection camera is calculated in real time. ;

[0086] in, The three-dimensional position of the reference calibration point in the main coordinate system of the inspection station. This represents the pixel coordinates sampled at time t from the reference point; This refers to the camera's internal parameters.

[0087] Hardware trigger time synchronization is performed for all defect detection cameras;

[0088] When drift is detected in the extrinsic parameters of multiple defect detection cameras, the minimum reprojection error criterion is used to jointly optimize the extrinsic parameters of multiple cameras.

[0089] When using the minimum reprojection error criterion to jointly optimize the extrinsic parameters of multiple cameras, the optimization function is as follows:

[0090]

[0091] in Represents the projection function. Indicates camera intrinsic parameters. This represents the extrinsic transformation matrix of the camera coordinate system relative to the detection station coordinate system during the i-th observation. This indicates the three-dimensional reference position of the marker point in the coordinate system of the inspection station. This represents the pixel coordinates of the marker point observed for the i-th time in the image.

[0092] S3. Based on the real-time 6-DOF pose information, the vehicle body surface image is acquired through the defect detection camera, and the pixel coordinates of the defect points are identified.

[0093] In S3, the step of acquiring an image of the vehicle body surface using a defect detection camera and identifying the pixel coordinates of defect points includes:

[0094] Using a convolutional neural network to detect the region to be tested in the image coordinate system, and outputting a set of defect pixel center points:

[0095]

[0096] in, Indicates the center of the defective pixel;

[0097] In this application, the convolutional neural network can specifically employ the conventional YOLOv5 algorithm to identify defect points, including:

[0098] Image preprocessing includes grayscale conversion and filtering.

[0099] Defect region segmentation: The image is binarized using a Gaussian adaptive threshold to highlight the defect region.

[0100] Morphological processing involves performing morphological closing operations (dilation followed by erosion) on the binary image to connect adjacent defect regions.

[0101] Connected component analysis uses a connected component labeling algorithm (such as the two-pass scanning method) to find all connected regions. Based on the characteristics of the connected regions, such as area, perimeter, and circularity, regions that do not meet the defect characteristics are filtered out.

[0102] Outputting the center point coordinates of each filtered connected region yields the set of pixel coordinates for the defective point.

[0103] Based on the intrinsic parameters and depth information of the defect detection camera, the three-dimensional coordinates of the defect point in the defect detection coordinate system are obtained:

[0104]

[0105] in, This represents the depth information corresponding to the center of the defect pixel, and the 3D coordinates of the defect point are back-projected onto the 3D mathematical model coordinate system of the body-in-white:

[0106]

[0107] in, Represents the three-dimensional coordinates of the defect point in the camera coordinate system.

[0108] S4. Based on the real-time 6-DOF pose information and the extrinsic and intrinsic parameter matrices of the defect detection camera, the pixel coordinates of the defect point are back-projected onto the coordinate system of the three-dimensional mathematical model of the white body to obtain the precise spatial position of the defect point in the 3D body mathematical model.

[0109] When the three-dimensional coordinates of the defect points are back-projected onto the coordinate system of the three-dimensional mathematical model of the body-in-white, including

[0110] After coarsely registering the 3D coordinates of the defect points with the surface of the 3D mathematical model, the nearest neighbor search is used to find a one-to-one correspondence between the defect points and the surface of the 3D mathematical model, thus forming the 3D spatial annotation result of the defect points.

[0111] Specifically, when performing coarse registration of the three-dimensional coordinates of the defect point with the surface of the three-dimensional mathematical model, the rotation matrix R and translation vector t for rigid body registration are calculated using SVD. The following optimization function is employed to calculate the rotation matrix. R Translation vector t The optimal solution:

[0112] ;

[0113] Where R is the rotation matrix and t is the translation vector. To calibrate any point in the feature point set of the 3D vehicle body mathematical model, It is any point in the feature point set on the actual vehicle body.

[0114] Furthermore, the step of using nearest neighbor points to search for a one-to-one correspondence between defect points and the numerical model surfaces of the three-dimensional mathematical model includes:

[0115] Construct a kd-tree from the CAD digital model point cloud Q_CAD;

[0116] For each defect point p, search for the nearest neighbor in the kd-tree:

[0117] a. Starting from the root node, recursively search downwards, deciding whether to search the left or right subtree based on the partition dimension and partition value of the current node, until a leaf node is reached.

[0118] b. During the backtracking process, check for closer points, including points that may appear in another subtree.

[0119] Returning the nearest neighbor point q, which is the corresponding point of defect point p on the CAD model surface, ultimately forms the 3D spatial annotation result of the defect point:

[0120] .

[0121] like Figure 2 The present invention also provides a vehicle body defect localization system, which implements the steps of the above-mentioned method of back-projecting 2D defects to a 3D digital model, including:

[0122] Multi-camera module: including at least one vehicle body positioning camera and at least one defect detection camera;

[0123] Dynamic calibration module: configured to perform vehicle dynamic pose estimation and real-time calibration of camera extrinsic parameters;

[0124] Filtering module: Uses a Kalman filter to smooth the vehicle body pose;

[0125] Defect detection module: Identifies defect regions in images based on a deep learning model;

[0126] Back projection engine: configured to map defect points from the image coordinate system to the 3D digital model after multi-coordinate system transformation;

[0127] Visualization module: Used to display the location of defects on a 3D digital model.

[0128] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for back-projecting a 2D defect onto a 3D digital model, characterized in that, Includes the following steps: S1. Obtain the real-time 6-DOF pose information of the current body-in-white at the inspection station; S2. Dynamically calibrate the defect detection camera, and based on a calibration reference fixed at the inspection station, determine the extrinsic parameter matrix of the defect detection camera relative to the coordinate system of the inspection station in real time; including: During the detection cycle, the calibration reference object fixed at the detection station is identified, and the extrinsic parameter matrix of the defect detection camera is calculated in real time. ; in, The three-dimensional position of the reference calibration point in the main coordinate system of the inspection station. This represents the pixel coordinates sampled at time t from the reference point; K is the camera intrinsic parameter. Hardware trigger time synchronization is performed for all defect detection cameras; When multiple defect detection camera extrinsic parameters are detected to be drifting, the minimum reprojection error criterion is used to jointly optimize the extrinsic parameters of multiple cameras. The optimization function for jointly optimizing the extrinsic parameters of multiple cameras using the minimum reprojection error criterion is as follows: in Represents the projection function. Indicates camera intrinsic parameters. This represents the extrinsic transformation matrix of the camera coordinate system relative to the detection station coordinate system during the i-th observation. This indicates the three-dimensional reference position of the marker point in the coordinate system of the inspection station. This represents the pixel coordinates of the i-th observed marker point in the image; S3. Based on the real-time 6-DOF pose information, the vehicle body surface image is acquired through the defect detection camera, and the pixel coordinates of the defect points are identified. S4. Based on the real-time 6-DOF pose information, the extrinsic and intrinsic parameter matrices of the defect detection camera, the pixel coordinates of the defect point are back-projected onto the coordinate system of the three-dimensional mathematical model of the white body to obtain the precise spatial position of the defect point in the 3D body mathematical model.

2. The method for back-projecting a 2D defect to a 3D digital model according to claim 1, characterized in that, In S1, the body-in-white is positioned, including: S101. Select at least three feature holes in the key area of ​​the vehicle body as marker points; wherein, the three feature holes are not collinear; S102. Use a positioning camera to acquire images of the marker points and extract the sub-pixel-level image coordinates of the marker points; S103. Based on the positioning camera intrinsic parameter matrix K and the three-dimensional template coordinates of the marker points, calculate the rotation matrix of the vehicle body relative to the coordinate system of the inspection station. With translation vector ; Obtain the real-time homogeneous transformation matrix of the vehicle body in the coordinate system of the inspection station; S104. Filter the real-time homogeneous transformation matrix to obtain real-time 6-DOF pose information.

3. The method for back-projecting a 2D defect to a 3D digital model according to claim 2, characterized in that, In S103, based on the positioning camera intrinsic parameter matrix K and the 3D template coordinates of the marker points, the rotation matrix of the vehicle body relative to the coordinate system of the detection station is solved using the PnP algorithm. With translation vector ; in, The three-dimensional template coordinates of the marker point This represents the coordinates of the marker point in the pixel coordinate system, where K is the camera intrinsic parameter.

4. The method for back-projecting a 2D defect to a 3D digital model according to claim 2, characterized in that, In S104, the real-time homogeneous transformation matrix is ​​filtered, and a stable vehicle attitude estimate is obtained as the real-time 6-DOF pose information according to the following formula: in, This is the estimated value of the vehicle body attitude. This refers to the vehicle's position at the previous moment. The vehicle's position at time t. This represents the extended Kalman filter function.

5. The method for back-projecting a 2D defect to a 3D digital model according to claim 4, characterized in that, In S3, the defect detection camera acquires images of the vehicle body surface and identifies the pixel coordinates of the defect points; include: Using a convolutional neural network to detect the region to be tested in the image coordinate system, and outputting a set of defect pixel center points: in, Indicates the center coordinates of the defective pixel; Based on the intrinsic parameters and depth information of the defect detection camera, the three-dimensional coordinates of the defect point in the defect detection coordinate system are obtained: in, This represents the depth information corresponding to the center of the defect pixel, and the 3D coordinates of the defect point are back-projected onto the 3D mathematical model coordinate system of the body-in-white: in, This represents the three-dimensional coordinates of the defect point in the camera coordinate system. This indicates the camera's internal parameters.

6. The method for back-projecting a 2D defect to a 3D digital model according to claim 5, characterized in that, This also includes the process of back-projecting the three-dimensional coordinates of the defect points onto the coordinate system of the three-dimensional mathematical model of the body-in-white, including... After coarsely registering the three-dimensional coordinates of the defect points with the surface of the three-dimensional mathematical model, the nearest neighbor point search is used to find a one-to-one correspondence between the defect points and the surface of the three-dimensional mathematical model, and finally the three-dimensional spatial annotation result of the defect points is formed. When performing coarse registration of the three-dimensional coordinates of the defect point with the surface of the three-dimensional mathematical model, the rotation matrix R and translation vector t for rigid body registration are calculated using SVD. The following optimization function is employed to calculate the rotation matrix. R Translation vector t The optimal solution: ； Where R is the rotation matrix and t is the translation vector. To calibrate any point in the feature point set of the 3D vehicle body mathematical model, It is any point in the feature point set on the actual vehicle body.

7. The method for back-projecting a 2D defect to a 3D digital model according to claim 6, characterized in that, The method of using nearest neighbor point search to find a one-to-one correspondence between defect points and the numerical model surface of the three-dimensional mathematical model includes: Construct a kd-tree from the CAD digital model point cloud Q_CAD; For each defect point p, search for the nearest neighbor in the kd-tree: a. Starting from the root node, recursively search downwards, deciding whether to search the left or right subtree based on the partition dimension and partition value of the current node, until a leaf node is reached; b. During the backtracking process, check if there are any closer points, including points that might appear in another subtree; Return the nearest neighbor point q, which is the corresponding point of the defect point p on the CAD model surface; The final 3D spatial annotation result of the defect points: 。 8. A system for back-projecting a 2D defect to a 3D digital model, used to implement the steps of the method for back-projecting a 2D defect to a 3D digital model according to any one of claims 1-7, characterized in that, include: Multi-camera module: including at least one vehicle body positioning camera and at least one defect detection camera; Dynamic calibration module: configured to perform vehicle dynamic pose estimation and real-time calibration of camera extrinsic parameters; Filtering module: Uses a Kalman filter to smooth the vehicle body pose; Defect detection module: Identifies defect regions in images based on a deep learning model; Back projection engine: configured to map defect points from the image coordinate system to the 3D digital model after multi-coordinate system transformation; Visualization module: Used to display the location of defects on a 3D digital model.

Images