A method for detecting defects in automobile parts

Abstract

This invention relates to the field of image processing technology and discloses a method for detecting defects in automotive parts. The method includes the following steps: acquiring two-dimensional image data and three-dimensional point cloud data of the automotive parts; generating a depth map from the three-dimensional point cloud data and concatenating it with the color channels of the two-dimensional image data to form a multi-channel input tensor; feeding the multi-channel input tensor into a feature extraction network to generate a feature saliency map; using the feature saliency map to guide a deformable convolution module to adaptively sample and convolve the feature map; performing multi-task prediction on candidate defect regions using an anchorless detection head; constructing a topological association graph of the candidate defect regions; calculating the connection weights between nodes; pruning the graph using a preset weight threshold; and merging the candidate defect regions into the final defect detection result. This invention overcomes the detection limitations of a single data source under complex lighting and diverse defect morphologies, improving the comprehensive detection capability and stability for various defects.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method for detecting defects in automotive parts. Background Technology

[0002] In the automotive manufacturing industry, the surface quality of parts directly affects the performance, safety, and aesthetics of the entire vehicle. Therefore, automated and high-precision defect detection is a core component of the quality control system. Currently, machine vision-based inspection technology is the mainstream means of achieving automation. Traditional two-dimensional image inspection methods acquire surface images of parts using industrial cameras and then analyze them using image processing algorithms or classic machine learning models. However, these methods have significant bottlenecks: on the one hand, the inspection results are easily affected by environmental factors such as ambient lighting conditions, surface reflections, and shadows, resulting in poor stability and repeatability; on the other hand, for defects with three-dimensional spatial geometric features, such as dents, protrusions, and scratches, two-dimensional images cannot provide depth information, making accurate identification and quantitative evaluation difficult, and often leading to confusion between surface stains and geometric deformation defects.

[0003] To overcome the limitations of two-dimensional inspection, some solutions have introduced three-dimensional measurement techniques, such as laser scanning or structured light imaging, to detect geometric deformation defects by acquiring three-dimensional point cloud data of the part's surface. While three-dimensional data can accurately describe the contour and shape of a part, it lacks crucial surface information such as color and texture, limiting its ability to detect non-geometric defects such as rust, color difference, printing errors, and microcracks. Furthermore, even with the application of deep learning algorithms to defect detection, existing models still face challenges. For example, standard convolutional neural networks (CNNs) employ fixed receptive fields and sampling methods, resulting in insufficient adaptability and specificity in feature extraction for irregular defects of varying shapes and sizes found in industrial settings. Meanwhile, in terms of outputting and integrating test results, most methods only stop at locating and classifying defects, failing to provide a quantitative assessment of severity. Furthermore, when dealing with dense or banded defect groups, simple post-processing logic (such as non-maximum suppression) can easily lead to accidental deletion or incomplete merging, failing to truly reflect the overall shape and spatial correlation of defects. This limits the depth of application of test results in subsequent intelligent production processes such as quality grading and maintenance decisions. Summary of the Invention

[0004] This invention provides a method for detecting defects in automotive parts to solve the problems in the prior art, such as the detection results being easily affected by environmental factors such as ambient lighting conditions, surface reflection and shadows of the parts, and the difficulty in accurately identifying defects with three-dimensional spatial geometric features, resulting in poor accuracy of the detection results.

[0005] The method for detecting defects in automotive parts according to the present invention includes the following steps:

[0006] Two-dimensional image data and three-dimensional point cloud data of the automotive part to be inspected are acquired. The three-dimensional point cloud data is projected onto a two-dimensional plane to generate a depth map, and then stitched with the color channels of the two-dimensional image data to form a multi-channel input tensor.

[0007] The multi-channel input tensor is fed into a preset feature extraction network. The feature extraction network generates a feature saliency map through a spatial attention module, and uses the feature saliency map to guide a deformable convolution module to adaptively sample and convolve the highly saliency regions in the feature map in order to extract the deep contextual features of the part.

[0008] Based on the deep context features, the defect candidate region is predicted by a multi-task method using an anchorless detection head to obtain the center point coordinates, size, category probability, and a corresponding continuous defect severity level score of the defect candidate region.

[0009] A topological association graph of the defect candidate regions is constructed, wherein each defect candidate region is a graph node. The connection weight between nodes is calculated based on the normalized center distance and intersection-union ratio between the defect candidate regions. The graph is pruned using a preset weight threshold, and the defect candidate regions in each connected component are merged into the final defect detection result.

[0010] The automotive part defect detection method of this invention fuses the color and texture information of two-dimensional images with the geometric depth information of three-dimensional point clouds to construct a multi-channel input. This effectively overcomes the detection limitations of a single data source under complex lighting and diverse defect morphologies, significantly improving the comprehensive detection capability and stability for various defects. At the feature extraction level, a spatial attention mechanism guides deformable convolution, enabling the network to focus on potential defect regions and sample features based on the actual contours of the defects, thereby achieving accurate capture of complex defect features with irregular shapes and varying sizes. Furthermore, this invention not only locates and classifies defects but also outputs a continuous defect severity score, providing a refined quantitative basis for quality assessment. The post-processing strategy based on topological correlation graphs can reasonably merge dense or strip-shaped defect clusters, avoiding the accidental deletion or incomplete merging of traditional methods, ensuring the completeness and accuracy of the final detection results.

[0011] Preferably, the step of projecting the 3D point cloud data onto a 2D plane to generate a depth map and concatenating it with the color channels of the 2D image data to form a multi-channel input tensor includes: using the Z-buffer algorithm to orthogonally project the 3D point cloud data along the Z-axis onto the XY plane, obtaining the original depth value of each pixel and normalizing it to the range [0, 255] to generate a depth map; and concatenating the depth map with the R, G, and B color channels of the 2D image data to form a 4-channel input tensor.

[0012] The Z-buffer algorithm can solve the visual occlusion problem of "near point occlusion of far point" in three-dimensional space. Ultimately, at each pixel position of the two-dimensional depth map, only the depth value of the three-dimensional point that is actually visible from that viewpoint is retained.

[0013] Preferably, the feature extraction network generates a feature saliency map through a spatial attention module, including: applying max pooling and average pooling operations along the channel dimension of the input feature map to generate two two-dimensional feature maps; concatenating the two two-dimensional feature maps to form a dual-channel feature description; and passing the dual-channel feature description through a convolutional kernel with a size of 7. The convolutional layer of layer 7 performs convolution, and the output value is mapped to the range of 0 to 1 using the Sigmoid activation function to generate the feature saliency map.

[0014] By performing max pooling and average pooling on the input feature map along the channel dimension, key spatial information can be captured from different angles, thereby enhancing effective features and suppressing redundant information.

[0015] Preferably, the step of using the feature saliency map to guide the deformable convolution module to adaptively sample and convolve highly saliency regions in the feature map includes: performing element-wise multiplication of the feature saliency map and the input feature map to obtain a weighted feature map; feeding the weighted feature map into a 3D model. The bypass convolutional layer 3 specifically learns the two-dimensional offset of the sampling grid; the learned two-dimensional offset is applied to the regular grid sampling points of the deformable convolutional module, so that the regular grid sampling points gather towards the region with stronger feature response, thus completing adaptive sampling and convolution.

[0016] Preferably, the multi-task prediction of the defect candidate region using the anchorless detection head includes: the anchorless detection head contains four parallel prediction branches, each prediction branch consisting of two 3×3 convolutional layers and one 1×1 convolutional layer; the four prediction branches are a first branch, a second branch, a third branch, and a fourth branch, the first branch is used to predict the heatmap of the defect center point; the second branch is used to predict the width and height of the defect region; the third branch is used to predict the category probability of the defect; and the fourth branch is used to regress a continuous value between 0 and 1 as the defect severity level score.

[0017] The four parallel prediction branches predict defects from different angles, which can capture defect features more comprehensively. By predicting the defect center point heatmap and width and height, the location and range of defects can be located more accurately. Predicting the defect category probability can accurately determine the defect type. Predicting the severity level score can further provide a quantitative basis for defect assessment, thereby improving the overall detection accuracy.

[0018] Preferably, the step of calculating the connection weights between nodes based on the normalized center distance and intersection-union ratio between candidate defect regions includes: for any two nodes i and j, their connection weights... Calculated using the following formula: ;

[0019] in, Let i be the intersection-union ratio of the candidate defect regions corresponding to nodes i and j. Let be the Euclidean distance between the center points of the two defect candidate regions, σ be a preset hyperparameter used to control the decay rate of the Gaussian function, and exp() be an exponential function.

[0020] Preferably, the step of pruning the graph using a preset weight threshold and merging the defect candidate regions within each connected component into the final defect detection result includes: removing all edges in the topological association graph whose connection weight is less than the preset weight threshold to complete the graph pruning; in the pruned graph, merging all defect candidate regions belonging to the same connected component into a final defect, wherein the bounding box of the final defect is determined by the minimum bounding rectangle of all defect candidate regions within the connected component, and its defect severity level score is the weighted average of the corresponding scores of all defect candidate regions within the connected component, with the weight being the category probability score of each defect candidate region.

[0021] Preferably, the original depth value is normalized to the range [0, 255] using the maximum-minimum normalization method.

[0022] The min-max normalization method can linearly scale the original depth values ​​without changing the relative size relationship and overall distribution trend of the data, ensuring that the normalized depth map can still reflect the true spatial geometric relationship.

[0023] Preferably, the acquisition of two-dimensional image data and three-dimensional point cloud data of the automotive parts to be inspected includes: acquiring an RGB three-channel two-dimensional image with a resolution of 2048×2048 pixels using an industrial line scan camera, and acquiring three-dimensional point cloud data of the surface of the automotive parts using a structured light three-dimensional scanner.

[0024] Preferably, feeding the multi-channel input tensor into a preset feature extraction network includes: sending the multi-channel input tensor into a feature extraction network with ResNet-50 as the backbone.

[0025] The beneficial effects of this invention are as follows: By fusing the color and texture information of two-dimensional images with the geometric depth information of three-dimensional point clouds, a multi-channel input is constructed, effectively overcoming the detection limitations of a single data source under complex lighting and diverse defect morphologies, and significantly improving the comprehensive detection capability and stability for various defects. At the feature extraction level, a spatial attention mechanism guides deformable convolution, enabling the network to focus on potential defect regions and perform feature sampling based on the actual contours of the defects, thereby achieving accurate capture of complex defect features with irregular shapes and varying sizes. Furthermore, this invention not only locates and classifies defects but also outputs a continuous defect severity level score, providing a refined quantitative basis for quality assessment. The post-processing strategy based on topological correlation graphs can reasonably merge dense or strip-shaped defect clusters, avoiding the erroneous deletion or incomplete merging of traditional methods, ensuring the completeness and accuracy of the final detection results. By using four parallel prediction branches to predict defects from different angles, defect features can be captured more comprehensively. By predicting the defect center point heatmap and width and height, the location and extent of defects can be located more accurately. Predicting the defect category probability can accurately determine the defect type. Predicting the severity level score can further provide a quantitative basis for defect assessment, thereby improving the overall detection accuracy. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the defect detection method for automotive parts provided in an embodiment of the present invention. Detailed Implementation

[0027] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0028] like Figure 1 As shown, the automotive parts defect detection method provided in this embodiment of the invention specifically includes the following steps:

[0029] S1. Acquire two-dimensional image data and three-dimensional point cloud data of the automotive part to be inspected, project the three-dimensional point cloud data onto a two-dimensional plane to generate a depth map, and stitch it with the color channel of the two-dimensional image data to form a multi-channel input tensor.

[0030] Specifically, an industrial linear scan camera acquires a 2048×2048 pixel resolution RGB three-channel 2D image, while a structured light 3D scanner acquires 3D point cloud data of the automotive part surface. Using orthogonal projection, the Z-coordinate value of each point in the 3D point cloud is normalized to between 0 and 255, generating a single-channel depth map. Finally, this single-channel depth map is used as a fourth channel and concatenated with the R, G, and B color channels of the 2D image to form a 4×2048×2048 four-channel input tensor.

[0031] S2. The multi-channel input tensor is fed into a preset feature extraction network. The feature extraction network generates a feature saliency map through a spatial attention module, and uses the feature saliency map to guide the deformable convolution module to adaptively sample and convolve the highly saliency regions in the feature map in order to extract the deep contextual features of the part.

[0032] Specifically, the four-channel input tensor is fed into a feature extraction network with ResNet-50 as its backbone. After the third and fourth residual stages of the feature extraction network, a spatial attention module within a convolutional block attention module is inserted. The spatial attention module performs global average pooling and max pooling operations on the input feature map, and the results are concatenated and passed through a convolutional layer to generate a two-dimensional feature saliency map. Each pixel value in the feature saliency map reflects the importance of the corresponding location. Subsequently, the feature saliency map and the original feature map are jointly input into a deformable convolutional module DCNv2. The values ​​of the feature saliency map are used to guide the deformable convolutional module DCNv2 to perform non-mesh sampling on the feature map, so that the convolution operation focuses on the defect-related regions identified by the feature saliency map, thereby extracting more discriminative deep features.

[0033] S3. Based on the deep context features, perform multi-task prediction on the defect candidate region using an anchorless detection head to obtain the center point coordinates, size, category probability, and a corresponding continuous defect severity level score of the defect candidate region.

[0034] Specifically, an anchorless detection head similar to FCOS is employed. This head comprises three parallel prediction branches, each operating on a multi-scale feature map output by the feature extraction network. The first branch is a classification branch, using a convolutional layer with a sigmoid activation function to predict the probability of each pixel belonging to a particular defect category. The second branch is a regression branch, predicting the four distances (up, down, left, right) from the current pixel to the bounding box of its corresponding defect candidate region, thereby determining the center coordinates and size of the candidate region. The third branch is a severity level prediction branch, which maps the features to a continuous numerical value through a fully connected layer, then normalizes it using the sigmoid function and multiplies it by a preset maximum severity value, such as 10, to obtain a continuous defect severity level score ranging from 0 to 10.

[0035] S4. Construct a topological association graph of the defect candidate regions, where each defect candidate region is a graph node. Calculate the connection weights between nodes based on the normalized center distance and intersection-union ratio between the defect candidate regions. Prune the graph using a preset weight threshold and merge the defect candidate regions within each connected component into the final defect detection result.

[0036] Specifically, all candidate defect regions with confidence scores higher than a preset threshold, output from the previous step, are used as initial nodes in the graph. For any two nodes, the Euclidean distance between their bounding box centers is calculated and normalized by dividing by the image diagonal length to obtain the normalized center distance. Simultaneously, the intersection-union ratio (IUU) of these two nodes is calculated. The connection weights are a weighted combination of these two metrics; for example, the weight equals 0.5 multiplied by the reciprocal of the normalized center distance plus 0.5 multiplied by the intersection-union ratio. A weight threshold, such as 0.7, is set, and all connections with weights below this threshold are removed from the graph, completing graph pruning. Finally, a breadth-first search algorithm is used to traverse the pruned graph to find all connected components. For each connected component, the bounding boxes of all defect candidate regions contained within it are merged to form the smallest bounding rectangle that completely encloses all defects within the defect candidate region, serving as the final defect detection result.

[0037] In an optional embodiment, the 3D point cloud data is projected onto a 2D plane to generate a depth map, and then concatenated with the color channels of the 2D image data to form a multi-channel input tensor. This includes: using the Z-buffer algorithm to orthogonally project the 3D point cloud data along the Z-axis onto the XY plane, obtaining the original depth value of each pixel and normalizing it to the range [0, 255] to generate a depth map; and concatenating the depth map with the R, G, and B color channels of the 2D image data to form a 4-channel input tensor. The aforementioned steps aim to fuse 3D geometric information and 2D texture information. First, the acquired 3D point cloud data is processed using the Z-buffer algorithm. This process can be imagined as looking down at the entire point cloud from directly above the Z-axis, projecting each 3D point (X, Y, Z) onto the XY 2D plane. When multiple points are projected onto the same pixel coordinates, the Z-buffer algorithm retains the point with the smallest Z value, i.e., the point closest to the projection plane, and its Z value is recorded as the original depth value of the pixel.

[0038] Specifically, after obtaining the raw depth values ​​of all pixels, normalization is required to facilitate integration with image data. For example, assuming the Z-coordinate range of all points in the scene is 1000 mm to 1500 mm, then the normalized depth value of a point with a Z-value of 1250 mm is... This is approximately equal to 128. Thus, the entire point cloud is converted into a grayscale image, or depth map, where the grayscale value of each pixel ranges from 0 to 255, representing the relative distance of objects. This depth map is then stacked with the original R, G, and B three-channel color image to form a 1920×1080×4 four-channel tensor, which serves as the input to the subsequent neural network, enabling the network to simultaneously perceive color texture and spatial depth.

[0039] In an optional embodiment, the feature extraction network generates a feature saliency map through a spatial attention module, including: applying max pooling and average pooling operations to the input feature map along the channel dimension to generate two two-dimensional feature maps; concatenating the two two-dimensional feature maps to form a dual-channel feature description; and convolving the dual-channel feature description through a convolutional layer with a kernel size of 7×7, and using a sigmoid activation function to map its output value to the range of 0 to 1 to generate the feature saliency map.

[0040] The purpose of the spatial attention module is to focus the feature extraction network on the most informative regions in the feature map. Assume the input is a feature map with dimensions H (height), W (width), and C (number of channels), for example, 64×64×256. First, the spatial attention module performs two pooling operations along the channel dimension. For each spatial location (h, w) in the feature map, max pooling selects the maximum value from its 256 channel values, while average pooling calculates the average of these 256 values. After these operations, two two-dimensional feature maps of size H×W are generated, capturing the most prominent feature response at each location and the overall feature information, respectively.

[0041] Two 2D feature maps of size H x W are stacked together to form a two-channel feature descriptor of size H x W x 2. The descriptor is fed into a convolutional layer using a large 7x7 kernel, which helps integrate contextual information over a wider spatial range to determine the importance of a region. The output of the convolution operation is passed through a sigmoid activation function, which compresses the output value at each location to between 0 and 1. The resulting H x W single-channel map is the feature saliency map; pixels with values ​​close to 1 correspond to regions in the original feature map that are more likely to contain critical information such as defects.

[0042] In an optional embodiment, the step of using the feature saliency map to guide the deformable convolution module to adaptively sample and convolve highly saliency regions in the feature map includes: performing element-wise multiplication of the feature saliency map and the input feature map to obtain a weighted feature map; feeding the weighted feature map into a 3×3 bypass convolutional layer, which is specifically designed to learn the two-dimensional offset of the sampling grid; and applying the learned two-dimensional offset to the regular grid sampling points of the deformable convolution module, causing the regular grid sampling points to cluster towards regions with stronger feature responses, thereby completing the adaptive sampling and convolution.

[0043] Specifically, the generated feature saliency map is used to guide feature extraction. This feature saliency map, with values ​​between 0 and 1, is multiplied element-wise with the input feature map. This operation is equivalent to assigning a weight to each spatial location of the input feature map. Regions with high saliency have their feature values ​​basically unchanged or enhanced, while background regions with low saliency have their feature values ​​suppressed and approach zero, thus obtaining a weighted feature map.

[0044] The weighted feature map is fed into a separate side-channel convolutional layer, such as a 3×3 convolutional layer. The task of this 3×3 convolutional layer is not to extract features for classification or localization, but rather to learn offsets for the sampling points in the deformable convolution. For a standard 3×3 convolutional kernel, which has 9 sampling points, the side-channel convolutional layer predicts a two-dimensional offset vector (Δx, Δy) for each sampling point of the center pixel. Since the input is saliency-weighted, the learned offsets naturally point in the direction of stronger feature responses. Finally, in the main deformable convolutional module, the positions of the sampling points are no longer fixed grids, but rather regular grid points plus the learned offsets. For example, a feature that should be in… The location sampling point will now be... By sampling at specific locations, the receptive field of the convolution operation can dynamically fit the actual shape of the defect, enabling accurate feature extraction of irregular targets.

[0045] In an optional embodiment, the multi-task prediction of the defect candidate region using the anchorless detection head includes: the anchorless detection head contains four parallel prediction branches, each prediction branch consisting of two 3×3 convolutional layers and one 1×1 convolutional layer; the four prediction branches are a first branch, a second branch, a third branch, and a fourth branch, the first branch is used to predict the heatmap of the defect center point; the second branch is used to predict the width and height of the defect region; the third branch is used to predict the category probability of the defect; and the fourth branch is used to regress a continuous value between 0 and 1 as the defect severity level score.

[0046] Specifically, after the feature map is processed by the feature extraction network and the spatial attention module, it is fed into the anchorless detection head for final prediction. The anchorless detection head does not rely on preset anchor boxes, but directly predicts at each location of the feature map. It consists of four parallel prediction branches with different functions. Each prediction branch is a small network structure, for example, deepening the features through two 3×3 convolutional layers, and then adjusting the number of feature channels to the required output dimension through a 1×1 convolutional layer.

[0047] The first branch outputs a single-channel heatmap of defect centers. The value of each pixel represents the probability that its location is a defect center; for example, pixels with a value greater than 0.7 are considered potential defect centers. The second branch outputs two values ​​for each pixel, predicting the width and height of the defect bounding box centered on that pixel, in pixels. The third branch handles classification. If three types of defects need to be identified, it outputs three probability values ​​for each pixel, corresponding to the likelihood of being a Class I, Class II, or Class III defect. The fourth branch outputs a single continuous value, such as 0.85, which directly quantifies the severity of the defect at that location. 0 represents harmless, and 1 represents the most severe, achieving end-to-end regression of defect levels.

[0048] In an optional embodiment, calculating the connection weights between nodes based on the normalized center distance and intersection-union ratio between defect candidate regions includes: for any two nodes i and j, their connection weights... Calculated using the following formula: ;

[0049] in, Let i be the intersection-union ratio of the candidate defect regions corresponding to nodes i and j. Let be the Euclidean distance between the center points of the two defect candidate regions, σ be a preset hyperparameter used to control the decay rate of the Gaussian function, and exp() be an exponential function.

[0050] Specifically, the anchorless detection head generates a large number of dense defect candidate regions, which need to be organized and filtered. Each defect candidate region is considered as a node in a graph. To determine whether any two defect candidate regions describe the same real defect, the connection strength between them, i.e., the edge connection weight, needs to be calculated. Connection weights It is obtained by multiplying two parts, taking into account both shape overlap and spatial distance. The first part is the intersection-union ratio. It calculates the ratio of the intersection area to the union area of ​​two candidate defect regions, with a value between 0 and 1. The larger the value, the higher the degree of overlap between the two defect candidate regions. The second part is a Gaussian penalty term related to the distance between the center points. Calculate the Euclidean distance between the center points of two defect candidate regions, for example, 20 pixels. The square of this Euclidean distance is divided by the square of a hyperparameter sigma, which can be preset to, for example, 50. The larger the distance, the closer the exponent term approaches 0, meaning that even if two distant defect candidate regions overlap, their connection weight will be significantly weakened. Only when two defect candidate regions have significant overlap and are very close will their connection weight be relatively high.

[0051] In an optional embodiment, the step of pruning the graph using a preset weight threshold and merging the defect candidate regions within each connected component into the final defect detection result includes: removing all edges in the topological association graph whose connection weight is less than the preset weight threshold to complete the graph pruning; in the pruned graph, merging all defect candidate regions belonging to the same connected component into a final defect, wherein the bounding box of the final defect is determined by the minimum bounding rectangle of all defect candidate regions within the connected component, and its defect severity level score is the weighted average of the corresponding scores of all defect candidate regions within the connected component, with the weight being the category probability score of each defect candidate region.

[0052] Specifically, after constructing a fully connected graph containing all defect candidate region nodes and their connection weights, simplification is needed to distinguish different defect instances. A weight threshold is set, for example, 0.4. Then, all edges in the graph are traversed, and edges with weights less than 0.4 are removed. This process is called graph pruning, which breaks the connections between weakly related defect candidate regions, decomposing the originally complex graph into several independent subgraphs, each called a connected component. After pruning, all nodes within the same connected component are considered to collectively describe a single real defect, therefore they need to be merged. The merging process first determines the location and size of the final defect by calculating the minimum bounding rectangle of all defect candidate regions within the connected component, i.e., finding the smallest rectangle that can exactly enclose all these defect candidate regions. The final defect severity level is determined using a weighted average. For example, if a connected component contains defect candidate regions A and B with defect severity level scores of 0.9 and 0.7 respectively, and the model's confidence scores for their class predictions are 0.95 and 0.80 respectively, then the final defect severity level score is... Thus, candidate defect regions with higher confidence levels have greater influence in determining the final defect severity level score.

[0053] The implementation principle of the automotive part defect detection method in this invention is as follows: By fusing the color and texture information of a two-dimensional image with the geometric depth information of a three-dimensional point cloud, a multi-channel input is constructed. This overcomes the limitations of a single data source under complex lighting and diverse defect morphologies, significantly improving the comprehensive detection capability and stability for various defects. At the feature extraction level, a spatial attention module guides deformable convolution, enabling the feature extraction network to focus on potential defect regions and sample features based on the actual contours of the defects, thereby achieving accurate capture of complex defect features with irregular shapes and varying sizes. Furthermore, this invention not only locates and classifies defects but also outputs a continuous defect severity score, providing a refined quantitative basis for quality assessment. Moreover, the post-processing strategy based on a topological correlation graph can reasonably merge dense or strip-shaped defect clusters, preventing accidental deletion or incomplete merging, and ensuring the completeness and accuracy of the final detection results.

[0054] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting defects in automotive parts, characterized in that, Includes the following steps: Two-dimensional image data and three-dimensional point cloud data of the automotive part to be inspected are acquired. The three-dimensional point cloud data is projected onto a two-dimensional plane to generate a depth map, and then stitched with the color channels of the two-dimensional image data to form a multi-channel input tensor. The multi-channel input tensor is fed into a preset feature extraction network. The feature extraction network generates a feature saliency map through a spatial attention module, and uses the feature saliency map to guide a deformable convolution module to adaptively sample and convolve the highly saliency regions in the feature map in order to extract the deep contextual features of the part. Based on the deep context features, the defect candidate region is predicted by a multi-task method using an anchorless detection head to obtain the center point coordinates, size, category probability, and a corresponding continuous defect severity level score of the defect candidate region. A topological association graph of the defect candidate regions is constructed, wherein each defect candidate region is a graph node. The connection weight between nodes is calculated based on the normalized center distance and intersection-union ratio between the defect candidate regions. The graph is pruned using a preset weight threshold, and the defect candidate regions in each connected component are merged into the final defect detection result.

2. The method for detecting defects in automotive parts according to claim 1, characterized in that, The step of projecting the 3D point cloud data onto a 2D plane to generate a depth map and concatenating it with the color channels of the 2D image data to form a multi-channel input tensor includes: using the Z-buffer algorithm to orthogonally project the 3D point cloud data onto the XY plane along the Z-axis, obtaining the original depth value of each pixel and normalizing it to the range [0, 255] to generate a depth map; and concatenating the depth map with the R, G, and B color channels of the 2D image data to form a 4-channel input tensor.

3. The method for detecting defects in automotive parts according to claim 1, characterized in that, The feature extraction network generates a feature saliency map through a spatial attention module, including: applying max pooling and average pooling operations along the channel dimension of the input feature map to generate two two-dimensional feature maps; concatenating the two two-dimensional feature maps to form a dual-channel feature description; and convolving the dual-channel feature description through a convolutional layer with a kernel size of 7×7, and using the Sigmoid activation function to map its output value to the range of 0 to 1 to generate the feature saliency map.

4. The method for detecting defects in automotive parts according to claim 1, characterized in that, The step of using the feature saliency map to guide the deformable convolution module to adaptively sample and convolve highly saliency regions in the feature map includes: performing element-wise multiplication of the feature saliency map and the input feature map to obtain a weighted feature map; feeding the weighted feature map into a 3×3 bypass convolutional layer, which specifically learns the two-dimensional offset of the sampling grid; and applying the learned two-dimensional offset to the regular grid sampling points of the deformable convolution module, causing the regular grid sampling points to cluster towards regions with stronger feature responses, thereby completing adaptive sampling and convolution.

5. The method for detecting defects in automotive parts according to claim 1, characterized in that, The multi-task prediction of the defect candidate region using the anchorless detection head includes: the anchorless detection head contains four parallel prediction branches, each prediction branch consisting of two 3×3 convolutional layers and one 1×1 convolutional layer; the four prediction branches are the first branch, the second branch, the third branch, and the fourth branch, respectively. The first branch is used to predict the heatmap of the defect center point; the second branch is used to predict the width and height of the defect region; the third branch is used to predict the category probability of the defect; and the fourth branch is used to regress a continuous value between 0 and 1 as the defect severity level score.

6. The method for detecting defects in automotive parts according to claim 1, characterized in that, The calculation of connection weights between nodes based on the normalized center distance and intersection-union ratio between candidate defect regions includes: For any two nodes i and j, their connection weights Calculated using the following formula: ； in, Let i be the intersection-union ratio of the candidate defect regions corresponding to nodes i and j. Let be the Euclidean distance between the center points of the two defect candidate regions, σ be a preset hyperparameter used to control the decay rate of the Gaussian function, and exp() be an exponential function.

7. The method for detecting defects in automotive parts according to claim 1, characterized in that, The step of pruning the graph using a preset weight threshold and merging the defect candidate regions within each connected component into the final defect detection result includes: removing all edges in the topological association graph whose connection weight is less than the preset weight threshold to complete the graph pruning; in the pruned graph, merging all defect candidate regions belonging to the same connected component into a final defect, wherein the bounding box of the final defect is determined by the minimum bounding rectangle of all defect candidate regions within the connected component, and its defect severity level score is the weighted average of the corresponding scores of all defect candidate regions within the connected component, with the weight being the category probability score of each defect candidate region.

8. The method for detecting defects in automotive parts according to claim 2, characterized in that, The original depth value is normalized to the range [0, 255] using the maximum-minimum normalization method.

9. The method for detecting defects in automotive parts according to claim 1, characterized in that, The acquisition of two-dimensional image data and three-dimensional point cloud data of the automotive part to be inspected includes: acquiring image data with a resolution of [missing information - likely a resolution value] using an industrial linear scan camera. The RGB three-channel two-dimensional image of the pixels is used to obtain three-dimensional point cloud data of the surface of the car parts using a structured light three-dimensional scanner.

10. The method for detecting defects in automotive parts according to claim 1, characterized in that, The step of feeding the multi-channel input tensor into a preset feature extraction network includes: sending the multi-channel input tensor into a feature extraction network with ResNet-50 as the backbone.

Images