A method for detecting appearance defects of an automobile plastic part

Abstract

The application provides a kind of appearance defect detection method of automobile plastic parts, method includes: the two-dimensional image data and three-dimensional point cloud data of corresponding model automobile plastic parts are collected, two-dimensional image data and three-dimensional point cloud data are mapped into unified coordinate system after processing;In the unified coordinate system, two-dimensional image data and three-dimensional point cloud data are executed pose correction operation;Based on the corrected two-dimensional image data, generate abnormal heat map, determine the suspected defect candidate area of automobile plastic parts based on abnormal heat map;To suspected defect candidate area executes multi-feature complex operation to obtain final defect area;To final defect area executes type identification operation to obtain defect type, and calculates the defect quantization parameter of final defect area.The application scheme improves the stability and accuracy of appearance defect detection of automobile plastic parts under the condition of low texture, easy reflection, random posture and multi-model mixed line.

Description

Technical Field

[0001] This application relates to the field of industrial inspection technology, and in particular to a method for detecting appearance defects in automotive plastic parts. Background Technology

[0002] With the development of the automotive industry, automotive plastic parts are widely used in products such as smart key housings, interior panels, function buttons, controller housings, and decorative parts. These automotive plastic parts are mostly exposed components that users can directly observe and frequently touch; their appearance quality affects the overall vehicle quality perception, assembly reliability, and brand quality evaluation. Common defects on the exposed surfaces of automotive plastic parts include stains, embossing, scratches, flash, shrinkage, silver streaks, watermarks, and white spots.

[0003] In the past, visual inspection during the production of automotive plastic parts mainly relied on manual visual sampling. Inspectors typically observed the surface of the workpiece under a fixed light source, and then manually marked the location of defects after finding them. This method suffers from technical problems such as low inspection efficiency, high labor intensity, strong subjectivity, and poor result consistency, making it difficult to meet the needs of stable judgment and quantitative inspection under mass production conditions.

[0004] Existing technologies have gradually adopted methods such as 2D machine vision, 3D vision, image processing, deep learning, and unsupervised anomaly detection for appearance defect detection. These methods have improved the automation level of inspection to some extent. However, due to the characteristics of automotive plastic parts, such as low texture, easy reflection, lack of feature points, random posture, similar surface color, and micro-deformation during injection molding, existing inspection technologies still have the following shortcomings in practical production line applications.

[0005] The limitations of 2D machine vision inspection technology lie in the fact that automotive plastic parts have relatively smooth surfaces and sparse texture information, and may experience localized micro-deformation, slight warping, and edge shrinkage during injection molding. Furthermore, the randomness of workpiece loading, conveying, and placement on the production line leads to inconsistencies in the position, angle, and orientation of the workpiece under inspection. 2D machine vision inspection technology is prone to unstable matching when there are insufficient effective feature points, and it still has limitations in handling perspective distortion, localized micro-deformation, and orientation flipping.

[0006] The limitations of 3D vision inspection technology lie in the fact that automotive plastic parts have characteristics such as weak geometric features, low-texture surfaces, and local injection molding deformation. When processing such workpieces, 3D vision inspection is easily affected by insufficient effective geometric features, point cloud noise, and local deformation, resulting in insufficient point cloud registration stability. At the same time, some appearance defects mainly manifest as color, brightness, or texture abnormalities, which are difficult to fully identify by relying solely on 3D height differences and point cloud morphology information.

[0007] The limitations of image processing inspection technology lie in the fact that different models of plastic parts may have differences in outer contour, button layout, marking position, and decorative structure. If a single standard template or a single normal sample library is used to compare all models in the same way, these model differences are easily misjudged as appearance defects during template differentiation and abnormal heat map generation, thus affecting the stability of the inspection results.

[0008] The limitations of deep learning detection technology lie in the fact that the number of defect samples in industrial production sites is limited, the types are uneven, and the frequency of occurrence is low. It is also difficult to obtain a sufficient number of anomaly samples with complete types in the early stages of new product introduction. Deep learning usually relies on a large number of defect samples and manual annotation. When the number of defect samples is insufficient or the defect types change, the model training and deployment become more difficult, and the generalization ability of the detection results is easily affected.

[0009] The shortcomings of unsupervised anomaly detection technology are that when it is directly applied to the inspection of automotive plastic parts, it is easily affected by posture deviations, reflection interference, model differences, and complex backgrounds. The inherent structure of the model, such as brand logos, buttons, and metal inserts on the surface of automotive plastic parts, is easily identified as a suspected defect area; small defects such as tiny white spots, minor scratches, and shallow imprints have little difference from the background, making it difficult to reliably distinguish them using a single anomaly scoring or single threshold segmentation method.

[0010] In summary, how to provide a method for accurately identifying and locating appearance defects in automotive plastic parts is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0011] To overcome the shortcomings of the prior art, this application provides a method for detecting appearance defects in automotive plastic parts, which improves the stability and accuracy of appearance defect detection in automotive plastic parts under conditions of low texture, high reflectivity, random posture, and mixed lines of multiple models.

[0012] The technical solution adopted in this application is as follows: In a first aspect, a method for detecting appearance defects in automotive plastic parts is provided, the method comprising: Two-dimensional image data and three-dimensional point cloud data of automotive plastic parts of the corresponding model are collected, and the two-dimensional image data and the three-dimensional point cloud data are processed and mapped to a unified coordinate system. In the unified coordinate system, pose correction operations are performed on the two-dimensional image data and the three-dimensional point cloud data; An anomaly heatmap is generated based on the corrected two-dimensional image data, and a candidate region for suspected defects in the automotive plastic part is determined based on the anomaly heatmap. Perform a multi-feature re-judgment operation on the suspected defect candidate regions to obtain the final defect regions; Perform a type identification operation on the final defect region to obtain the defect type, and calculate the defect quantization parameters of the final defect region.

[0013] Preferably, the step of processing the two-dimensional image data and the three-dimensional point cloud data and mapping them to a unified coordinate system includes: The two-dimensional image data is subjected to image enhancement processing, and the three-dimensional point cloud data is subjected to point cloud data cleaning processing. Based on the calibration parameters of the image acquisition device, a mapping relationship is established between the pixel coordinates of the automotive plastic part in the two-dimensional image data and the spatial coordinates in the three-dimensional point cloud data, so that the data representing the same physical location in the two-dimensional image data and the three-dimensional point cloud data correspond to the same position in a unified coordinate system.

[0014] Preferably, performing pose correction operation on the two-dimensional image data and the three-dimensional point cloud data includes: Extract local feature descriptors from the two-dimensional image data and extract geometric feature descriptors from the three-dimensional point cloud data; The local feature descriptor and the geometric feature descriptor are respectively matched to obtain two-dimensional matching point pairs and three-dimensional matching point pairs; Determine whether the number of the two-dimensional matching point pairs meets the preset conditions; If the preset conditions are met, the transformation relationship of the two-dimensional image data relative to the standard template is determined based on the two-dimensional matching point pairs. An attitude registration correction operation is performed on the two-dimensional image data based on the transformation relationship, and the attitude registration correction operation is simultaneously applied to the three-dimensional point cloud data. Then, a fine registration operation is performed on the attitude-registered and corrected three-dimensional point cloud data based on the three-dimensional matching point pairs.

[0015] Preferably, the step of performing pose correction operation on the two-dimensional image data and the three-dimensional point cloud data further includes: If the preset conditions are not met, the outer contour of the automotive plastic part is extracted based on the two-dimensional image data, the rotation angle and size of the two-dimensional image data relative to the standard template are determined based on the outer contour, the contour bottoming correction operation is performed on the two-dimensional image data based on the rotation angle and size, and the contour bottoming correction operation is simultaneously applied to the three-dimensional point cloud data.

[0016] Preferably, the step of generating an anomaly heatmap based on the corrected two-dimensional image data, and determining the candidate regions for suspected defects in the automotive plastic parts based on the anomaly heatmap, includes: The corrected two-dimensional image data is input into a pre-trained feature extraction network to extract local features at multiple scales and generate local feature vectors. Each of the local feature vectors is compared with the normal feature vectors in the normal feature memory of the corresponding model, and the anomaly score of each local feature vector is calculated. The anomaly scores of each of the local feature vectors are mapped to the image space to generate an anomaly heatmap and then normalized. Based on a preset anomaly scoring threshold, the normalized anomaly heatmap is segmented to form a candidate region for suspected defects in the automotive plastic parts.

[0017] Preferably, the step of performing a multi-feature re-judgment operation on the suspected defect candidate region to obtain the final defect region includes: Two-dimensional image difference calculation and three-dimensional shape difference calculation are performed on the suspected defect candidate regions to obtain two-dimensional texture deviation information and three-dimensional shape deviation information; Based on preset color feature conditions, the color feature of the suspected defect candidate region is re-judged to obtain color deviation information; Extract the geometric morphological features of the suspected defect candidate region and perform a geometric morphological verification operation on the suspected defect candidate region; Based on the two-dimensional texture deviation information, the three-dimensional shape deviation information, the color deviation information, and the results of the geometric shape verification operation, the final defect region is determined from the suspected defect candidate region.

[0018] Preferably, determining the final defect region from the suspected defect candidate region based on the two-dimensional texture deviation information, the three-dimensional shape deviation information, the color deviation information, and the result of the geometric shape verification operation includes: When the suspected defect candidate region passes the geometric shape verification operation, the two-dimensional texture deviation information, the three-dimensional shape deviation information, the color deviation information, and the geometric shape features are weighted and summed to obtain a re-judgment score; When the re-judgment score reaches the preset re-judgment threshold for the corresponding model, the corresponding suspected defect candidate area is determined as the final defect area.

[0019] Preferably, after generating an anomaly heatmap based on the corrected two-dimensional image data and determining the suspected defect candidate region of the automotive plastic part based on the anomaly heatmap, and before performing a multi-feature re-judgment operation on the suspected defect candidate region to obtain the final defect region, the method further includes: The main body mask of the automotive plastic part is generated based on the corrected two-dimensional image data, and the whitelist area of ​​the corresponding model is obtained; wherein, the main body mask is used to identify the main area of ​​the automotive plastic part, and the whitelist area contains the position information of the inherent normal structure of the corresponding model; Based on the main body mask, suspected defect candidate areas located outside the main body area of ​​the automotive plastic part are eliminated, and based on the whitelist area, suspected defect candidate areas corresponding to the inherent normal structure are blocked.

[0020] Preferably, the step of performing a type identification operation on the final defect region to obtain the defect type includes: Based on the texture and color features of the final defect region, two-dimensional texture defects are identified; Based on the morphological features of the final defect region, three-dimensional morphological defects are identified; If the final defect region simultaneously satisfies both the two-dimensional texture defect condition and the three-dimensional morphology defect condition, it is identified as a composite defect. The calculation of the defect quantification parameters for the final defect region includes: The pixel coordinates of the final defect area are converted into spatial coordinates to obtain the defect location of the final defect area on the automotive plastic part. Based on the pixel area and spatial resolution of the final defect region, the defect area is calculated, and based on the height difference of the final defect region, the defect depth and defect volume are calculated. The defect location, defect area, defect depth, and defect volume constitute the defect quantification parameters.

[0021] Preferably, the method further includes: Based on the appearance judgment rules corresponding to the model of the automotive plastic part, and combined with the defect type of the final defect area, the defect quantification parameters, and the detection area to which the final defect area belongs, the defect level of the final defect area is generated. Based on the defect level, a detection conclusion and handling recommendation are generated, and a detection report is generated; The inspection report includes the model of the automotive plastic part, the defect quantification parameters, the defect level, the inspection conclusion, and the handling recommendations.

[0022] In a second aspect, a system for detecting appearance defects in automotive plastic parts is provided, the system comprising: The acquisition unit is used to acquire two-dimensional image data and three-dimensional point cloud data of the corresponding model of automotive plastic parts; A coordinate unification unit is used to process the two-dimensional image data and the three-dimensional point cloud data and map them to a unified coordinate system; The correction unit is used to perform pose correction operations on the two-dimensional image data and the three-dimensional point cloud data in the unified coordinate system. An abnormal heatmap generation unit is used to generate an abnormal heatmap based on the corrected two-dimensional image data. The initial screening unit is used to determine the candidate areas of suspected defects in the automotive plastic parts based on the abnormal heat map. The re-judgment unit is used to perform multi-feature re-judgment operation on the suspected defect candidate region to obtain the final defect region; A type identification unit is used to perform a type identification operation on the final defect region to obtain the defect type; A quantization calculation unit is used to calculate the defect quantization parameters of the final defect region.

[0023] In a third aspect, a computer device is provided, including a memory and a processor; the memory stores a computer program that can run on the processor, and when the computer program is executed by the processor, it performs the method provided in the first aspect.

[0024] In the fourth aspect, a computer-readable storage medium is provided that stores a computer program, which, when executed, implements the method provided in the first aspect.

[0025] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method for detecting appearance defects in automotive plastic parts. The method includes: acquiring two-dimensional image data and three-dimensional point cloud data of automotive plastic parts of a corresponding model; processing the two-dimensional image data and three-dimensional point cloud data and mapping them to a unified coordinate system; performing pose correction operations on the two-dimensional image data and three-dimensional point cloud data in the unified coordinate system; generating an anomaly heatmap based on the corrected two-dimensional image data; determining suspected defect candidate regions of the automotive plastic parts based on the anomaly heatmap; performing multi-feature re-judgment operations on the suspected defect candidate regions to obtain the final defect region; performing type recognition operations on the final defect region to obtain the defect type, and calculating the defect quantification parameters of the final defect region. This application first maps two-dimensional image data and three-dimensional point cloud data to a unified coordinate system to achieve multimodal data fusion. On this basis, pose correction is performed to eliminate pose deviations of automotive plastic parts. Then, the corrected two-dimensional image data is used to generate an abnormal heat map for unsupervised initial screening to locate suspected defect areas. Next, multi-feature re-judgment is performed on the area to suppress misjudgments caused by differences in normal structure and model. Finally, the defect type and quantitative parameters are output. This improves the stability and accuracy of appearance defect detection under conditions of low texture, easy reflection, random pose, and mixed models. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a general flowchart of the appearance defect detection method for automotive plastic parts provided in the embodiments of this application; Figure 2 This is a flowchart of the pose correction operation performed on two-dimensional image data and three-dimensional point cloud data, provided in an embodiment of this application. Figure 3 This is a flowchart illustrating the process of determining potential defect candidate regions for automotive plastic parts, as provided in an embodiment of this application. Figure 4 This is a flowchart provided in an embodiment of the present application for performing a multi-feature re-judgment operation on a suspected defect candidate region to obtain the final defect region; Figure 5 This is a flowchart of obtaining the final defect area based on the re-judgment score provided in an embodiment of this application; Figure 6 This is a schematic diagram of the appearance defect detection system for automotive plastic parts provided in the embodiments of this application; Figure 7 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0028] The technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art are within the scope of protection of this application.

[0029] As described in the background section, existing technologies have gradually adopted methods such as 2D machine vision, 3D vision, image processing, deep learning, and unsupervised anomaly detection for appearance defect detection. These methods have improved the automation level of inspection to some extent. However, due to the characteristics of automotive plastic parts, such as low texture, high reflectivity, scarcity of feature points, random poses, similar surface colors, and micro-deformation during injection molding, existing detection technologies still have some shortcomings in practical production line applications. Specifically, 2D machine vision methods have poor registration stability when there are insufficient feature points on low-texture surfaces; 3D vision methods struggle to fully identify color and texture defects; deep learning methods rely on a large number of defect samples, which are usually scarce on production lines; and unsupervised anomaly detection methods are prone to misjudging normal structures such as logos and buttons as defects, and model differences are easily misreported as appearance defects when dealing with mixed-model production lines.

[0030] Based on this, this application provides a method for detecting appearance defects in automotive plastic parts, aiming to solve the problems in the prior art, such as unstable posture correction of low-texture surfaces, difficulty in co-detecting two-dimensional texture defects and three-dimensional morphological defects, easy misjudgment of normal structures, and difficulty in quantifying and outputting detection results.

[0031] The embodiments of this application will be analyzed in detail below with reference to the accompanying drawings.

[0032] Example 1 refer to Figure 1 The method for detecting appearance defects in automotive plastic parts provided in this embodiment includes: S1: Collect two-dimensional image data and three-dimensional point cloud data of the corresponding model of automotive plastic parts, process the two-dimensional image data and three-dimensional point cloud data and map them to a unified coordinate system; S2: Perform pose correction operations on two-dimensional image data and three-dimensional point cloud data in a unified coordinate system; S3: Generate an anomaly heatmap based on the corrected two-dimensional image data, and determine the candidate areas of suspected defects in automotive plastic parts based on the anomaly heatmap; S4: Perform multi-feature re-judgment operation on the suspected defect candidate region to obtain the final defect region; S5: Perform a type identification operation on the final defect region to obtain the defect type, and calculate the defect quantification parameters of the final defect region.

[0033] In step S1, for the same automotive plastic part under test, its two-dimensional image data and three-dimensional point cloud data are acquired simultaneously. The two-dimensional image data carries surface information such as texture, color, and brightness, while the three-dimensional point cloud data provides spatial information such as depth, height, and surface morphology. Since the resolutions of the two-dimensional image data acquisition devices and the three-dimensional point cloud data acquisition devices are different, and their imaging coordinate systems are independent, directly mapping them by pixel number would introduce positional deviation. Therefore, this step establishes a mapping relationship between the two-dimensional image data and the three-dimensional point cloud data after processing them, so that the data representing the same physical location in the two-dimensional image data and the three-dimensional point cloud data correspond to the same location in a unified coordinate system, providing a unified spatial reference basis for subsequent steps.

[0034] In step S2, due to the randomness of the loading, conveying, and placement processes of automotive plastic parts in the production line environment, the position, angle, and orientation of the part under test are not completely consistent with the standard template. Directly performing subsequent analysis would generate a large number of false anomalies. Therefore, this step performs pose correction on the two-dimensional image data and three-dimensional point cloud data in a unified coordinate system, transforming the data under test to an orientation consistent with the standard template, eliminating deviations such as rotation, translation, and perspective distortion, so that the subsequent generation and analysis of anomaly heatmaps can be carried out under a unified orientation reference.

[0035] In step S3, the anomaly heatmap is a distribution map reflecting the degree of anomaly in various local areas of an image. It can mark areas where the surface of the tested automotive plastic part differs from normal appearance features. This step does not rely on defect samples or manual annotation; it can locate the anomaly area using only the features of normal samples. At the same time, it limits the analysis scope of subsequent multi-feature re-judgment to the candidate area indicated by the anomaly heatmap, avoiding the computational overhead of performing full-dimensional re-judgment pixel by pixel on the entire image.

[0036] In step S4, since the candidate regions formed by relying on a single anomaly score in the initial screening stage may contain some false anomalies, such as the inherent structural features of the automotive plastic parts surface or local responses caused by slight residual attitude deviations, this step S4 further verifies and analyzes the candidate regions from multiple feature dimensions. It comprehensively judges the probability of a real defect in each candidate region, selects the final defect region from the suspected defect candidate regions, and suppresses false alarms caused by the inherent structure of the model and detection noise.

[0037] In step S5, the defect type is identified based on the final defect area's performance characteristics. Simultaneously, the pixel coordinates of the defect are converted into actual spatial coordinates on the automotive plastic part, and quantitative parameters such as defect location, defect area, defect depth, and defect volume are calculated. This transforms the appearance defect from a qualitative judgment into a quantitative indicator, providing a basis for subsequent defect level determination.

[0038] It should be noted that before step S1, the method also includes a preparation stage and a detection task initialization stage. In the preparation stage, a model-level detection configuration file and a model-level normal feature memory are pre-established for each product model. The configuration file includes the corresponding 2D image standard template, 3D point cloud standard template, ROI region, detection parameters, model-level whitelist region, and handling strategy. The whitelist region is used to mark the positions of the model's inherent normal structures such as logos, text, buttons, and metal inserts.

[0039] The construction process of the model-level normal feature memory is as follows: First, defect-free normal samples are collected for each product model and archived according to product model, surface grade, and detection area. Following the same acquisition conditions and processing procedures as online detection, normal samples undergo data acquisition, preprocessing, and pose standardization. When pose registration does not meet preset requirements, contour correction is performed to obtain pose-standardized 2D image data and 3D point cloud data. Next, a WideResNet50 network pre-trained on a large natural image dataset is used as the backbone network. Normal samples are input into the pre-trained feature extraction network to extract multi-scale feature maps of layers 2 and 3, and then processed through 3... A sliding window is used to locally partition the feature map, encoding each local patch into a 512-dimensional local feature vector. Layer 2 and Layer 3 refer to the mid-level and high-level feature maps output by the residual modules of the second and third stages in the WideResNet50 backbone network, respectively. Layer 0 and Layer 1 mainly correspond to the input convolutional layer and shallow texture feature extraction layer, and are not used as features for building the model-level normal feature memory. The feature maps of Layer 2 and Layer 3 are upsampled and aligned to the same patch grid, then fused using channel concatenation and dimensionality reduction, and finally encoded into a 512-dimensional local feature vector. Finally, the local feature vectors of all normal samples under the same product model are aggregated to form a model-level normal feature memory. Different product models correspond to different normal feature memories, retrieval indexes, and version information to avoid false detections caused by mixing features across different models.

[0040] The model-level normal characteristic memory is denoted as: ;in, This represents the normal feature memory at the model level. Indicates the first There are n normal local feature vectors, where n represents the total number of feature vectors in the model-level normal feature memory. The model-level normal feature memory is used to characterize the distribution of local features of normal appearance of the same model and to provide normal feature references for subsequent anomaly heatmap generation.

[0041] When the number of normal samples is small, a full feature retention strategy is adopted; when the number of normal samples is large, a core set sampling method is used to compress the memory size, and representative feature vectors are selected according to the feature space coverage distance, with the core set sampling rate set to 0.1. An approximate nearest neighbor retrieval index is built using the Faiss high-efficiency vector retrieval library for quickly calculating the distance between the test sample and the model-level normal feature memory during the online inference stage.

[0042] During the initialization phase of the inspection task, after receiving the inspection task start signal, the system controller obtains the model, inspection station, production batch, and current production line status of the automotive plastic part to be tested based on the work order information. Simultaneously, it calls the model-level inspection configuration file, loading the standard template, inspection parameters, normal feature memory, retrieval index, whitelist blocking information, and handling strategies corresponding to the current model. It initializes the image acquisition parameters, light source parameters, and calibration parameters of the image acquisition device, including acquisition resolution, exposure time, light source illuminance, triggering method, and the coordinate mapping relationship between 2D image data and 3D point cloud data. A single-part inspection buffer is established, generating a unique inspection identifier (IDpart) for the automotive plastic part to be tested. Information such as model, production batch, mold cavity number, and production line number are bound to the IDpart for inspection record storage, defect location, and necessary manual verification.

[0043] After the above preparation and initialization work is completed, the data acquisition process in step S1 begins. As an example, acquiring 2D image data and 3D point cloud data of the corresponding model of automotive plastic parts includes the following steps: Once the automotive plastic part under test reaches the inspection area, a trigger signal is sent to the image acquisition device. Two-dimensional image data of the automotive plastic part is acquired, and grayscale, brightness, color space, and texture edge information are extracted. The two-dimensional image data is acquired by a global shutter industrial camera with a resolution of 768×960. Simultaneously, three-dimensional point cloud data of the automotive plastic part is acquired, obtaining three-dimensional topographic information such as depth map, height map, normal vector, and curvature. The three-dimensional point cloud data is acquired by a 3D structured light camera with an acquisition accuracy of ±0.02mm and a resolution of 1024×2048.

[0044] During the data acquisition process, a ring-shaped shadowless light source and a linearly polarized light source were used for illumination to enhance the visibility of minute defects and reduce reflective interference. Two-dimensional image data, three-dimensional point cloud data, acquisition time, and the vehicle plastic part number and model were bound together to form a single-part multimodal inspection data package. The synchronization error between the acquisition time of the two-dimensional image data and the three-dimensional point cloud data was no greater than 1ms.

[0045] As a preferred implementation, S1: After processing the two-dimensional image data and the three-dimensional point cloud data, map them to the same position in a unified coordinate system, including: S10: Perform image enhancement processing on 2D image data and point cloud data cleaning processing on 3D point cloud data; S11: Based on the calibration parameters of the image acquisition device, establish the mapping relationship between the pixel coordinates of automotive plastic parts in two-dimensional image data and their spatial coordinates in three-dimensional point cloud data, so that the data representing the same physical location in two-dimensional image data and three-dimensional point cloud data correspond to the same location in a unified coordinate system.

[0046] In step S10, as one example, image enhancement processing includes converting the 2D image data from the RGB color space to the LAB color space, extracting the L luminance channel, and performing CLAHE processing, where clipLimit is set to 2.0 and the block size is set to 8×8. Subsequently, a 5×5 Gaussian kernel smoothing filter is applied, and the ROI region is cropped based on the prior dimensions of the automotive plastic part to remove pure background areas and reduce subsequent computation. As another example, point cloud data cleaning processing includes noise point removal, point cloud smoothing, downsampling, and local geometric feature extraction of the 3D point cloud data; where the number of neighborhood points k in the statistical filter is set to 50, the standard deviation factor is set to 1.0, and the voxel downsampling size is set to 0.05mm. The ROI is extracted based on the main area of ​​the automotive plastic part, and non-automotive plastic part areas such as fixtures and background surfaces are removed.

[0047] In step S11, the image acquisition device is specifically a camera. Specifically, based on the camera's intrinsic and extrinsic parameters and calibration results, the system establishes a mapping relationship between two-dimensional pixel coordinates, three-dimensional point cloud coordinates, and standard coordinates of automotive plastic parts, so that defects at the same physical location correspond to the same location in the two-dimensional image data and the three-dimensional point cloud data, and the mapping error between the two-dimensional pixel coordinates and the three-dimensional spatial coordinates is no greater than 0.03 mm.

[0048] Furthermore, due to the different original resolutions of 2D image data and 3D point cloud data, a one-to-one correspondence based on pixel numbers is not possible. Therefore, instead of directly matching 2D image pixel numbers with 3D point cloud point numbers, the standard coordinate system of automotive plastic parts is used as a unified coordinate system to establish a correspondence between 2D image data, 3D point cloud data, and the actual positions of automotive plastic parts. First, based on the camera calibration results, the intrinsic and extrinsic parameters and coordinate transformation relationships between the 2D camera and the 3D structured light camera are obtained. Each 3D point in the 3D point cloud data is transformed to the standard coordinate system of the automotive plastic part according to the calibration extrinsic parameters, obtaining the actual spatial position of the 3D point on the automotive plastic part. Simultaneously, based on the 2D camera imaging model, the spatial position in the standard coordinate system of the automotive plastic part is projected onto the 2D image coordinate plane to obtain the corresponding 2D pixel position. Through the above coordinate transformation and projection relationships, the 3D height value and 3D spatial coordinates corresponding to each pixel position in the 2D image data can be determined.

[0049] When multiple 3D points are projected onto the same 2D pixel location, the nearest valid 3D point to the 2D camera is selected as the corresponding 3D point for that pixel location. If a 2D pixel location does not have a directly corresponding 3D point, it is filled in by resampling based on its neighboring valid 3D points. During resampling, for continuous 3D data such as height and depth values, bilinear interpolation is used for resolution matching; for discrete data such as subject masks, whitelist masks, and valid point markers, nearest neighbor interpolation is used for resolution matching. After this processing, defective areas in the 2D image data can correspond to the same location as height variation areas in the 3D point cloud data, thus enabling joint analysis of 2D texture information and 3D shape information.

[0050] As a preferred embodiment, refer to Figure 2 S2: Perform pose correction operations on 2D image data and 3D point cloud data, including: S20: Extract local feature descriptors from 2D image data and extract geometric feature descriptors from 3D point cloud data; S21: Perform feature matching on the local feature descriptor and the geometric feature descriptor respectively to obtain two-dimensional matching point pairs and three-dimensional matching point pairs; S22: Determine whether the number of two-dimensional matching point pairs meets the preset conditions; S23: If the preset conditions are met, the transformation relationship of the two-dimensional image data relative to the standard template is determined based on the two-dimensional matching point pairs. The attitude registration and correction operation is performed on the two-dimensional image data based on the transformation relationship, and the attitude registration and correction operation is applied synchronously to the three-dimensional point cloud data. Then, the fine registration operation is performed on the three-dimensional point cloud data after attitude registration and correction based on the three-dimensional matching point pairs.

[0051] This implementation first extracts local feature descriptors from the texture information of 2D image data, and simultaneously extracts geometric feature descriptors from the spatial information of 3D point cloud data. These two methods describe the automotive plastic part under test from the texture and geometric dimensions, respectively. Feature matching is then performed to obtain 2D and 3D matching point pairs. Based on this, it is determined whether the number of 2D matching point pairs meets a preset condition. If so, the transformation relationship between the test data and the standard template is estimated primarily using the 2D matching point pairs. Accordingly, pose registration and correction are performed on the 2D image data, and this correction is simultaneously applied to the 3D point cloud data to ensure consistent pose. Finally, fine registration is performed on the corrected 3D point cloud data based on the 3D matching point pairs to further improve alignment accuracy.

[0052] As an example, in step S20, when extracting local feature descriptors from the two-dimensional image data, ORB local image feature points are extracted from the preprocessed two-dimensional image data, and the corresponding feature descriptors are calculated. The maximum number of ORB feature points is set to 30,000, the image pyramid scaling factor is set to 1.1, and the number of pyramid layers is set to 12. The ORB feature descriptors are used to represent the grayscale, texture, and orientation variation features around key points in the two-dimensional image. When extracting geometric feature descriptors from the three-dimensional point cloud data, FPFH local geometric feature descriptors are extracted from the three-dimensional point cloud data to describe the local spatial geometric distribution features of the automotive plastic part surface.

[0053] In step S21, during feature matching, the FLANN fast approximate nearest neighbor algorithm is used to complete 2D image feature matching and 3D point cloud geometric feature matching. A distance ratio test is used to filter high-quality matching point pairs, with a distance ratio threshold set to 0.7. The ORB feature descriptors and FPFH feature descriptors are input into the FLANN retrieval module to obtain 2D and 3D candidate matching point pairs. After distance ratio testing, ORB matching point pairs are used for subsequent transformation relationship estimation, while FPFH matching point pairs are used to establish a coarse 3D registration relationship and provide an initial correspondence for subsequent fine registration.

[0054] In step S22, it is determined whether the number of two-dimensional matching point pairs meets a preset condition. Specifically, the preset condition is that the number of valid matching points N ≥ 15. When N ≥ 15 is met, proceed to step S23.

[0055] In step S23, the RANSAC random sampling consensus algorithm is used to estimate the homography transformation matrix between the two-dimensional image data to be corrected and the two-dimensional standard template. The homography transformation matrix is ​​a 3×3 matrix containing 8 degrees of freedom, represented as: The transformation relationship is as follows: in, The pixel coordinates of the two-dimensional image data to be corrected. These are the homogeneous coordinates obtained through homography transformation. The transformed two-dimensional pixel coordinates are... The system performs perspective transformation on the two-dimensional image data based on the homography transformation matrix, and synchronously updates the corresponding coordinates of the three-dimensional point cloud data according to the two-dimensional / three-dimensional calibration mapping relationship. The two-dimensional pose transformation is synchronously applied to the three-dimensional point cloud data, so that the two-dimensional texture information and the three-dimensional shape information are aligned in the same standard coordinate system, thereby realizing the pose alignment of automotive plastic parts with standard templates.

[0056] After the 2D pose transformation is completed, the ICP algorithm is used to perform fine registration of the 3D point cloud data to optimize the rotation and translation relationship of the automotive plastic parts relative to the 3D standard template and improve the alignment accuracy between the 3D point cloud data and the standard template.

[0057] As mentioned earlier, the aforementioned standard templates refer to the benchmark data pre-established for each product model during the preparation stage before testing, used for pose correction. This includes the corresponding 2D image standard template and 3D point cloud standard template. The 2D image standard template consists of 2D image data collected from defect-free automotive plastic parts of that model in a standard pose, used for pose registration correction and transformation relationship estimation of the 2D image data. The 3D point cloud standard template consists of 3D point cloud data collected from defect-free automotive plastic parts of that model in a standard pose, used for fine registration and alignment accuracy optimization of the 3D point cloud data. Each standard template corresponds one-to-one with a model-level testing configuration file and is loaded and called according to the model of the automotive plastic part to be tested during the initialization of the testing task.

[0058] As a preferred embodiment, refer to Figure 2 S2: Perform pose correction operations on 2D image data and 3D point cloud data, including: S24: If the preset conditions are not met, the outer contour of the automotive plastic part is extracted based on the two-dimensional image data, the rotation angle and size of the two-dimensional image data relative to the standard template are determined based on the outer contour, the contour bottoming correction operation is performed on the two-dimensional image data based on the rotation angle and size, and the contour bottoming correction operation is simultaneously applied to the three-dimensional point cloud data.

[0059] In this embodiment, when the surface texture information of the automotive plastic part is extremely sparse or the number of feature points is insufficient, making it impossible to reliably estimate the transformation relationship through feature matching, the system switches to the contour bottom-up correction mode based on the outer contour geometry information. The outer contour shape of the automotive plastic part is used as the basis for attitude estimation, ensuring that pose correction can be completed under various working conditions and avoiding interruption of the detection process due to feature matching failure.

[0060] As an example, if any of the following conditions are met: the number of ORB feature points is less than 10, the number of valid matching points is less than 15, or RANSAC solution fails, the preset conditions are deemed not met, and the contour correction mode is entered. In contour correction mode, the maximum outer contour of the main body of the automotive plastic part is extracted using adaptive binarization, and the minimum bounding rectangle of this maximum outer contour is calculated to obtain the principal orientation angles of the image under test and the standard template image. and And calculate the angle difference: Ambiguity correction is applied to the angle difference to bring it within the range of -45° to 45°: Subsequently, with the image center Construct an affine transformation matrix for the center of rotation : in, Indicates the coordinates of the image rotation center. This represents the principal direction angle difference between the image under test and the standard template. This represents the affine transformation matrix used for image rotation correction. Based on this matrix, the 2D image data is rotated and corrected, and the corresponding coordinates of the 3D point cloud data are synchronously updated according to the 2D / 3D calibration mapping relationship, ensuring that the automotive plastic part maintains the same posture as the standard template. Simultaneously, the corrected image is scaled to the same size as the standard template.

[0061] Furthermore, for automotive plastic parts with approximately symmetrical outer contours where contour correction alone is insufficient to determine orientation, orientation locking is achieved based on the color and location features of a preset orientation recognition area, determined by the detection configuration file for the current model. Taking a model with white buttons as an example, the white button area is used as the orientation recognition area. The corrected 2D image data is converted to the HSV color space, and a button area mask is generated based on the color features of the white buttons. in, Represents pixel coordinates. The mask value indicating whether the pixel belongs to the white button area. , and These represent the hue, saturation, and lightness components of the pixel in the HSV color space, respectively. Extract the maximum connected component of the button area and calculate its center ordinate. .Will The center ordinate of the corresponding button area in the standard template If the two are located in the upper and lower halves of the image respectively, it is determined that the car plastic part has been flipped 180°, and a 180° rotation correction is performed simultaneously on the two-dimensional image data and the three-dimensional point cloud data.

[0062] As a preferred embodiment, refer to Figure 3 S3: Generate an anomaly heatmap based on the corrected 2D image data, and determine candidate regions for suspected defects in automotive plastic parts based on the anomaly heatmap, including: S30: Input the corrected 2D image data into the pre-trained feature extraction network to extract local features at multiple scales and generate local feature vectors; S31: Compare each local feature vector with the normal feature vector in the normal feature memory of the corresponding model, and calculate the abnormal score of each local feature vector. S32: Map the anomaly scores of each local feature vector to the image space, generate an anomaly heatmap, and perform normalization processing; S33: Based on the preset anomaly scoring threshold, the normalized anomaly heatmap is segmented to form a candidate region for suspected defects in automotive plastic parts.

[0063] In this implementation, a pre-trained feature extraction network is first used to extract multi-scale local feature vectors from the corrected two-dimensional image data. These vectors are then compared with a normal feature memory for the corresponding model. An anomaly score is obtained by calculating the degree of deviation between each local region and normal features. The anomaly score is then mapped back to the image space to generate an anomaly heatmap. After normalization and threshold segmentation, suspected defect candidate regions are formed. The construction of this anomaly screening model does not rely on defect samples or manual annotation; it can quickly locate anomaly regions using only the local feature distribution of normal samples. Simultaneously, the analysis scope for subsequent multi-feature re-judgment is limited to the finite candidate regions indicated by the heatmap.

[0064] As an example, in step S30, the pose-corrected 2D image data of the automotive plastic part is input into a pre-trained feature extraction network. This pre-trained feature extraction network is consistent with the backbone network used to construct the model-level normal feature memory. It employs a WideResNet50 network pre-trained on a large natural image dataset, extracts multi-scale feature maps from layers 2 and 3, and performs local partitioning of the feature maps using a 3×3 sliding window. The local feature vector to be tested is generated according to the same patch partitioning method as the model-level normal feature memory, denoted as: in, Represents the set of local features to be tested. Indicates the first A local feature vector to be tested, This represents the total number of local feature vectors of the sample to be tested.

[0065] In step S31, for each local feature vector to be tested... In the normal feature memory of the corresponding model The system uses an approximate nearest neighbor retrieval index to find the nearest neighbor normal features and calculates the anomaly score: in, Represents the local feature vector to be measured Abnormal scores, This represents the Euclidean distance. A higher anomaly score indicates a greater deviation between the local area and normal appearance features, and a higher likelihood of a defect.

[0066] It should be noted that the preset anomaly scoring thresholds for different product models, surface grades, and inspection areas are calibrated independently, rather than using a uniform threshold across all models. During calibration, the anomaly scoring distribution of defect-free normal samples for the current model is used as a benchmark, and a percentage value between 85% and 95% is selected. This percentage value is then used as the initial anomaly scoring threshold for the current inspection area. For key inspection areas with defects such as tiny white spots, shallow scratches, and minor indentations, an anomaly score close to 85% is used; for areas with complex normal structures such as logos, text, and button edges that are prone to normal fluctuations, an anomaly score close to 95% is used. Subsequently, this initial anomaly scoring threshold is calibrated based on manual review results. After calibration, the anomaly scoring thresholds for each product model, surface grade, and inspection area are written into the model-level inspection configuration file for use during online inspection.

[0067] In step S32, the anomaly scores of all local regions are mapped back to the image space to generate an anomaly heatmap. Normalize the abnormal heatmap: in, Represents coordinates in an abnormal heat map The original anomaly score at the location, and These represent the minimum and maximum anomaly scores in the entire anomaly heatmap, respectively. To prevent extremely small constants with a denominator of zero, This represents the outlier score after normalization.

[0068] In step S33, the normalized anomaly heatmap is smoothed, thresholded, and subjected to morphological opening and closing operations to form candidate regions of suspected defects in automotive plastic parts. At the same time, the coordinates, area, and preliminary anomaly score of each candidate region of suspected defects are output to provide input for the multi-feature re-judgment in the subsequent step S4.

[0069] As a preferred embodiment, refer to Figure 4 S4: Perform multi-feature re-judgment on the suspected defect candidate regions to obtain the final defect regions, including: S40: Perform two-dimensional image difference calculation and three-dimensional morphology difference calculation on the suspected defect candidate regions to obtain two-dimensional texture deviation information and three-dimensional morphology deviation information; S41: Based on preset color feature conditions, perform color feature re-judgment on suspected defect candidate areas to obtain color deviation information; S42: Extract the geometric features of the suspected defect candidate region and perform geometric verification operation on the suspected defect candidate region; S43: Based on the results of two-dimensional texture deviation information, three-dimensional shape deviation information, color deviation information and geometric shape verification operation, the final defect area is determined from the suspected defect candidate area.

[0070] In this implementation, the candidate regions formed by relying on a single anomaly score in the initial screening stage may contain some false anomalies, such as inherent model structure features on the surface of automotive plastic parts or local responses caused by slight residual posture deviations. These are difficult to effectively distinguish from real defects based solely on the anomaly score. Therefore, this step analyzes the candidate regions from four dimensions: two-dimensional texture deviation, three-dimensional shape deviation, color deviation, and geometric morphological features. Through the complementarity and verification of multi-dimensional information, real defects are distinguished from false anomalies, and the final defect region is selected from the suspected defect candidate regions.

[0071] As an example, in step S40, when performing two-dimensional image difference calculation, the pose-corrected two-dimensional image data to be tested is compared with the corresponding model's defect-free standard template image for brightness difference to highlight two-dimensional texture anomalies such as stains, scratches, and embossing. Specifically, brightness difference is used: in, This indicates the position of the image under test after pose correction at the pixel level. The brightness value at that location, This represents the brightness value at the corresponding pixel in a defect-free standard template image. This represents the brightness difference result. Subsequently, the difference result is normalized: in, and These represent the minimum and maximum values ​​in the difference plot, respectively. To prevent extremely small constants with a denominator of zero, This represents the normalized two-dimensional difference map, i.e., the two-dimensional texture deviation information.

[0072] When performing 3D topographic difference calculations, the pose-corrected 3D point cloud data to be measured is registered with the corresponding standard point cloud template. Then, the height difference and normal vector difference at the same coordinate position are calculated. The formula for calculating the height difference is: The formula for calculating the difference in normal vectors is: in, Indicates the coordinates of the point cloud to be measured. The height value at that location, This represents the height value of the standard point cloud template at the corresponding location. and These represent the normal vectors of the point cloud to be measured and the standard point cloud template at this location, respectively. Indicates the height difference. This indicates the difference in the direction of the normal vector. When... or When the preset tolerance threshold is exceeded, the corresponding area is marked as a candidate area for three-dimensional topographic anomaly. The height difference and the difference in normal vectors together constitute the three-dimensional topographic deviation information.

[0073] In step S41, during color feature re-judgment, a color level threshold for white spots and white fuzz is set in the HSV color space to generate a white pixel mask: in, Represents pixels The mask value indicating whether it belongs to the white candidate region. , and These represent the hue, saturation, and brightness values ​​of the pixel in the HSV color space. This mask is used to extract candidate areas for light-colored defects such as white spots, whitish fuzz, and white patches, thus forming color deviation information.

[0074] In step S42, during geometric shape verification, the connected components of the candidate pixels after re-judgment are extracted, and the area, bounding rectangle width, height, aspect ratio, and center ordinate of the candidate region are calculated. For any candidate region, it is determined to pass the geometric shape verification when it meets the following conditions: in, The area of ​​the candidate region. and These are the width and height of the bounding rectangle, respectively. Aspect ratio, The vertical coordinate of the candidate region center is... Image height, This represents the coordinates of the four corner points and the center point of the bounding rectangle of the candidate region. The area range, aspect ratio range, and position range are determined by the detection parameters corresponding to the current product model.

[0075] In step S43, the candidate regions are weighted and scored based on the combined results of the two-dimensional texture deviation information, three-dimensional shape deviation information, color deviation information, and geometric shape verification operation, and the final defect region is determined from the suspected defect candidate regions. The specific implementation method of the weighted scoring judgment will be explained in detail in subsequent steps.

[0076] As a preferred embodiment, refer to Figure 5 S43: Based on the results of two-dimensional texture deviation information, three-dimensional topography deviation information, color deviation information, and geometric shape verification operations, the final defect region is determined from the suspected defect candidate region, including: S430: When a suspected defect candidate region passes the geometric shape verification operation, the two-dimensional texture deviation information, three-dimensional shape deviation information, color deviation information and geometric shape features are weighted and summed to obtain the re-judgment score. S431: When the re-judgment score reaches the preset re-judgment threshold for the corresponding model, the corresponding suspected defect candidate area is determined as the final defect area.

[0077] In this process, geometric morphology verification serves as a preliminary screening condition, applying weighted scoring only to candidate regions that pass the verification, while candidate regions that fail the verification are directly excluded. The weighted summation integrates information from four dimensions—two-dimensional texture deviation, three-dimensional shape deviation, color deviation, and geometric morphology features—into a single re-judgment score. A final judgment is then made based on a preset re-judgment threshold, achieving comprehensive decision-making based on multi-dimensional information.

[0078] As an example, in step S430, the system generates a differential candidate mask based on the two-dimensional template difference map segmented by a preset threshold. And combined with the main mask of automotive plastic parts Mask of normal structure region Construct basic candidate regions: Based on this, combine with light-colored pixel masks Constructing light-colored feature channels: in, This represents the basic candidate region after being constrained by the main body mask and shielded by the normal structure region. This represents candidate regions that meet the light-colored feature condition. The light-colored pixel mask only participates in the light-colored defect re-judgment and is not a necessary condition for all defects.

[0079] right The system then re-evaluates and scores candidate regions that have passed geometric morphology verification. For each candidate region, a weighted summation is used to calculate the re-evaluation score. Before calculating the re-evaluation score, the system normalizes the two-dimensional template difference response, three-dimensional height difference response, normal vector difference response, light color feature response, and geometric morphology feature response of the candidate regions, transforming each feature response to the range of 0 to 1, and converting each feature response into a candidate region-level feature value. The re-evaluation score of the candidate region is expressed as follows: in, Represents the normalized two-dimensional template difference response. Indicates the three-dimensional height difference response. Represents the normal vector difference response. Indicates light-colored feature response, It represents the response of geometric shape features. , , , , These are the weights of the corresponding features, and they satisfy: The weights of each feature are calibrated based on the manual verification samples of the current product model, with the goal of minimizing the combined false detection rate and false negative rate, and are stored in the model-level detection configuration file.

[0080] In step S431, when When the preset re-judgment threshold corresponding to the current product model is reached, and the candidate region is not blocked by the model-level whitelist region, the corresponding suspected defect candidate region is determined as the final defect region, and the confidence level, defect type, two-dimensional feature parameters and three-dimensional morphology parameters of each final defect region are output.

[0081] As a preferred embodiment, after S3: generating an anomaly heatmap based on the corrected two-dimensional image data, and determining the suspected defect candidate regions of the automotive plastic parts based on the anomaly heatmap, and before S4: performing a multi-feature re-judgment operation on the suspected defect candidate regions to obtain the final defect regions, the method further includes: S34: Generate a main body mask for automotive plastic parts based on the corrected two-dimensional image data, and obtain the whitelist area for the corresponding model; wherein, the main body mask is used to identify the main body area of ​​automotive plastic parts, and the whitelist area contains the position information of the inherent normal structure of the corresponding model; S35: Eliminate suspected defect candidate regions located outside the main area of ​​automotive plastic parts based on the main body mask, and mask suspected defect candidate regions corresponding to the inherent normal structure based on the whitelist region.

[0082] Because the abnormal heatmap generated in the initial screening stage, after threshold segmentation, forms a candidate region for suspected defects that may contain not only actual defects but also background noise outside the main body area of ​​the automotive plastic parts, as well as false trigger responses within the inherent normal structural areas of the model, such as logos, text, buttons, and metal inserts. Directly sending these regions into the multi-feature re-evaluation would not only increase unnecessary computational overhead but could also lead to the inherent structure of the model being misjudged as a defect. Therefore, steps S34 to S35 generate a main body mask to remove background areas and a whitelist area to mask inherent normal structures, thus pre-filtering the candidate regions for suspected defects and ensuring that subsequent multi-feature re-evaluation only targets valid candidate regions.

[0083] As an example, in step S34, when generating the subject mask, the pose-corrected 2D image data is converted into a grayscale image, Gaussian blurred, and then adaptive binarization is performed. Morphological closing operations are used to fill the internal holes of the automotive plastic part, and then opening operations are used to eliminate background noise, resulting in the subject mask. This subject mask is used to identify the main area of ​​the automotive plastic part in the 2D image data, distinguishing the automotive plastic part from non-detection areas such as the background surface and fixtures.

[0084] When obtaining the whitelist region, the location information of inherent normal structures such as logos, text, buttons, and metal inserts is extracted based on the preset model-level whitelist region coordinates in the current model's detection configuration file to form a fixed whitelist region. Further, for normal structures such as model identifiers and printed patterns whose positions may be slightly offset, the edges of candidate regions are extracted and their edge density is calculated. When the edge density exceeds a preset threshold and is located near the corresponding identifier region, the region is added to the dynamic whitelist mask, forming a complete whitelist region together with the fixed whitelist region.

[0085] In step S35, the suspected defect candidate regions obtained in step S3 are compared with the main body mask, and candidate regions located outside the main body mask area and belonging to the background or fixture area are eliminated. Simultaneously, the remaining candidate regions are compared with the whitelist region, and candidate regions corresponding to inherent normal structural positions such as logos, text, buttons, and metal inserts are masked. After the main body mask elimination and whitelist region masking, the remaining suspected defect candidate regions enter the multi-feature re-judgment process in step S4.

[0086] As a preferred implementation, S5A: Perform a type identification operation on the final defect region to obtain the defect type, including: S50: Identify two-dimensional texture defects based on the texture and color features of the final defect area; S51: Identify three-dimensional morphological defects based on the morphological features of the final defect region; S52: If the final defect area simultaneously satisfies the two-dimensional texture defect condition and the three-dimensional morphology defect condition, it is identified as a composite defect.

[0087] Since different types of defects exhibit different characteristic patterns in terms of texture, color, and shape, such as stains and scratches mainly manifest as two-dimensional texture and color abnormalities, shrinkage and dents mainly manifest as three-dimensional shape changes, while defects such as embossing may simultaneously exhibit two-dimensional and three-dimensional features, this step S50 to S52 determines the defect type based on the performance characteristics of the final defect area in each dimension, providing a classification basis for subsequent defect level assessment.

[0088] As an example, when identifying two-dimensional texture defects, a comprehensive judgment is made based on the color difference, texture change, and edge morphology of the final defect area. Two-dimensional texture defects include defect types such as stains, scratches, white spots, and watermarks, which are mainly characterized by abnormal surface texture or color. When identifying three-dimensional shape defects, a comprehensive judgment is made based on the height difference, curvature change, and normal vector change of the final defect area. Three-dimensional shape defects include defect types such as shrinkage, depressions, warping, and embossing, which are mainly characterized by changes in surface shape. For a final defect area that simultaneously meets the conditions for both two-dimensional texture anomalies and three-dimensional shape anomalies, it is determined to be a composite defect, and the corresponding composite defect type is output.

[0089] As a preferred implementation, S5B: Calculate the defect quantification parameters of the final defect region, including: S53: Convert the pixel coordinates of the final defect area into spatial coordinates to obtain the defect location of the final defect area on the automotive plastic part. S54: Calculate the defect area based on the pixel area and spatial resolution of the final defect region, and calculate the defect depth and defect volume based on the height difference of the final defect region; Among them, the defect location, defect area, defect depth, and defect volume constitute the defect quantification parameters.

[0090] After determining the final defect area and completing type identification through multi-feature re-judgment, it is necessary to convert the defect from a pixel description in the image space into a quantitative indicator of the actual physical space of the automotive plastic part, so as to provide an objective basis for subsequent defect level determination.

[0091] As an example, in step S53, the two-dimensional pixel coordinates of the defect are transformed to three-dimensional spatial coordinates using a calibration matrix to obtain the position coordinates of the defect in the standard coordinate system of automotive plastic parts: in, For the defect pixel coordinates, For the camera intrinsic parameter matrix, This represents the depth value corresponding to the pixel location. This represents the three-dimensional coordinates of the defect point in the camera coordinate system. Let R represent the three-dimensional coordinates of the defect point in the standard coordinate system of the automotive plastic part, R represent the rotation matrix from the standard coordinate system of the automotive plastic part to the camera coordinate system, and t represent the translation vector from the standard coordinate system of the automotive plastic part to the camera coordinate system. Let R be the transpose of the rotation matrix R.

[0092] In step S54, the quantization parameters of each defect are calculated based on the calibration ratio and the 3D height difference. The defect area is obtained by converting the pixel area. in, This represents the number of pixels contained in the defective region. and They are respectively direction and The actual spatial resolution of the orientation, that is, the actual size of each pixel in physical space.

[0093] For three-dimensional defects, the defect depth and volume are calculated based on the height differences within the defect region. The defect depth is taken as the maximum absolute value of the height differences within the defect region. in, This is a defective area. Coordinates within this region The height difference at the location, D_max represents the maximum defect depth.

[0094] The defect volume is obtained by summing the products of the height differences at each point within the defect region and the spatial resolution: in, The defect volume is represented. The defect location, defect area, defect depth, and defect volume together constitute the defect quantification parameters. Further, the detection area to which the defect belongs is determined based on the defect coordinates. The detection area includes surface A, surface B, surface C, and a preset detection area, and the detection area information is also included in the defect structured parameter output.

[0095] As a preferred embodiment, the method further includes: S6: Based on the appearance judgment rules corresponding to the model of the automotive plastic parts, combined with the defect type, defect quantification parameters and the detection area to which the final defect area belongs, the defect level of the final defect area is generated. S7: Generate detection conclusions and handling suggestions based on defect levels, and generate a detection report; Wherein, the inspection report includes the model of the automotive plastic part, defect quantization parameters, defect level, inspection conclusion and disposal suggestion.

[0096] As an example, the appearance determination rule in step S6 sets respective determination criteria for different inspection areas of the automotive plastic part, and the inspection areas include A-surface, B-surface, C-surface and a preset inspection area, wherein the A-surface is an exposed surface that is directly visible and frequently contacted by users, with the strictest appearance requirements; the appearance requirements for the B-surface and the C-surface decrease sequentially. The defect level is comprehensively evaluated according to the defect type, defect area, defect depth and the area where the defect is located, and specifically includes four levels: critical defect, serious defect, general defect and minor defect.

[0097] In step S7, an automotive plastic part with no defect detected or a defect level lower than the minimum determination threshold is determined to be qualified, and a release suggestion is output. For an automotive plastic part with only minor defects, a re-inspection suggestion is output, and it is decided whether to release after manual review. For an automotive plastic part with serious defects or critical defects, suggestions for rework, sorting or manual confirmation are output to ensure that unqualified products will not enter the next process. The inspection conclusion and disposal suggestion can be output to the production line control end or the manual review end, and linked with the production line automatic sorting equipment or the manual review station.

[0098] While generating the inspection conclusion, a structured inspection report is generated and relevant image data is saved. The inspection report includes the serial number, model, defect quantization parameters, defect level, inspection conclusion and disposal suggestion of the automotive plastic part, which is used for manual review and result traceability. At the same time, the abnormal heat map, two-dimensional difference map, three-dimensional difference map and result annotation map generated during the inspection process are saved to facilitate subsequent analysis and process improvement.

[0099] Second Embodiment The second embodiment provides an appearance defect inspection system for automotive plastic parts, with reference to Figure 6 , the system comprises: an acquisition unit, configured to acquire two-dimensional image data and three-dimensional point cloud data of an automotive plastic part of a corresponding model; a coordinate unification unit, configured to process the two-dimensional image data and the three-dimensional point cloud data and then map them into a unified coordinate system; a correction unit, configured to perform pose correction operations on the two-dimensional image data and the three-dimensional point cloud data in the unified coordinate system; an abnormal heat map generation unit, configured to generate an abnormal heat map based on the corrected two-dimensional image data; a primary screening unit, configured to determine a suspected defect candidate area of the automotive plastic part based on the abnormal heat map; a re-determination unit, configured to perform a multi-feature re-determination operation on the suspected defect candidate area to obtain a final defect area; A type identification unit is used to perform a type identification operation on the final defect area to obtain the defect type; The quantization calculation unit is used to calculate the defect quantization parameters of the final defect area.

[0100] The technical effects of the system provided in this second embodiment are the same as those of the method provided in the first embodiment, and will not be repeated here.

[0101] Example 3 This embodiment three provides a computer device, including a memory and a processor; the memory stores a computer program that can run on the processor, and when the computer program is executed by the processor, it executes the appearance defect detection method for automotive plastic parts provided in embodiment one above.

[0102] Among them, such as Figure 7 As shown, an exemplary computer device of this embodiment is illustrated, which may specifically include a processor 1510, a video display adapter 1511, a disk drive 1512, an input / output interface 1513, a network interface 1514, and a memory 1520. The processor 1510, video display adapter 1511, disk drive 1512, input / output interface 1513, network interface 1514, and memory 1520 can be communicatively connected via a communication bus 1530.

[0103] The processor 1510 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0104] The memory 1520 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1520 can store the operating system 1521 for controlling the operation of the computer device, and the basic input / output system 1522 for controlling the low-level operations of the computer device. Additionally, it can store a web browser 1523, a data storage management system 1524, and a device identification information processing system 1525, etc. The aforementioned device identification information processing system 1525 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 1520 and is called and executed by the processor 1510.

[0105] Input / output interface 1513 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0106] Network interface 1514 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0107] The communication bus 1530 includes a pathway for transmitting information between various components of the device, such as processor 1510, video display adapter 1511, disk drive 1512, input / output interface 1513, network interface 1514, and memory 1520.

[0108] In addition, the device can also obtain information on specific claim conditions from the virtual resource object claim condition information database for condition judgment, and so on.

[0109] It should be noted that although the above-described device only shows the processor 1510, video display adapter 1511, disk drive 1512, input / output interface 1513, network interface 1514, memory 1520, communication bus 1530, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0110] Example 4 This fourth embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed, it implements the appearance defect detection method for automotive plastic parts provided in the first embodiment above.

[0111] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0112] The above provides a detailed description of the method for detecting appearance defects in automotive plastic parts provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting appearance defects in automotive plastic parts, characterized in that, The method includes: Two-dimensional image data and three-dimensional point cloud data of automotive plastic parts of the corresponding model are collected, and the two-dimensional image data and the three-dimensional point cloud data are processed and mapped to a unified coordinate system. In the unified coordinate system, pose correction operations are performed on the two-dimensional image data and the three-dimensional point cloud data; An anomaly heatmap is generated based on the corrected two-dimensional image data, and a candidate region for suspected defects in the automotive plastic part is determined based on the anomaly heatmap. Perform a multi-feature re-judgment operation on the suspected defect candidate regions to obtain the final defect regions; Perform a type identification operation on the final defect region to obtain the defect type, and calculate the defect quantization parameters of the final defect region.

2. The method for detecting appearance defects in automotive plastic parts according to claim 1, characterized in that, The process of mapping the two-dimensional image data and the three-dimensional point cloud data to a unified coordinate system includes: The two-dimensional image data is subjected to image enhancement processing, and the three-dimensional point cloud data is subjected to point cloud data cleaning processing. Based on the calibration parameters of the image acquisition device, a mapping relationship is established between the pixel coordinates of the automotive plastic part in the two-dimensional image data and the spatial coordinates in the three-dimensional point cloud data, so that the data representing the same physical location in the two-dimensional image data and the three-dimensional point cloud data correspond to the same position in a unified coordinate system.

3. The method for detecting appearance defects in automotive plastic parts according to claim 1, characterized in that, The pose correction operation performed on the two-dimensional image data and the three-dimensional point cloud data includes: Extract local feature descriptors from the two-dimensional image data and extract geometric feature descriptors from the three-dimensional point cloud data; The local feature descriptor and the geometric feature descriptor are respectively matched to obtain two-dimensional matching point pairs and three-dimensional matching point pairs; Determine whether the number of the two-dimensional matching point pairs meets the preset conditions; If the preset conditions are met, the transformation relationship of the two-dimensional image data relative to the standard template is determined based on the two-dimensional matching point pairs. An attitude registration correction operation is performed on the two-dimensional image data based on the transformation relationship, and the attitude registration correction operation is simultaneously applied to the three-dimensional point cloud data. Then, a fine registration operation is performed on the attitude-registered and corrected three-dimensional point cloud data based on the three-dimensional matching point pairs.

4. The method for detecting appearance defects in automotive plastic parts according to claim 3, characterized in that, The step of performing pose correction on the two-dimensional image data and the three-dimensional point cloud data further includes: If the preset conditions are not met, the outer contour of the automotive plastic part is extracted based on the two-dimensional image data, the rotation angle and size of the two-dimensional image data relative to the standard template are determined based on the outer contour, the contour bottoming correction operation is performed on the two-dimensional image data based on the rotation angle and size, and the contour bottoming correction operation is simultaneously applied to the three-dimensional point cloud data.

5. The method for detecting appearance defects in automotive plastic parts according to claim 1, characterized in that, The process of generating an anomaly heatmap based on the corrected two-dimensional image data, and determining potential defect candidate regions for the automotive plastic parts based on the anomaly heatmap, includes: The corrected two-dimensional image data is input into a pre-trained feature extraction network to extract local features at multiple scales and generate local feature vectors. Each of the local feature vectors is compared with the normal feature vectors in the normal feature memory of the corresponding model, and the anomaly score of each local feature vector is calculated. The anomaly scores of each of the local feature vectors are mapped to the image space to generate an anomaly heatmap and then normalized. Based on a preset anomaly scoring threshold, the normalized anomaly heatmap is segmented to form a candidate region for suspected defects in the automotive plastic parts.

6. The method for detecting appearance defects in automotive plastic parts according to claim 1, characterized in that, The step of performing a multi-feature re-judgment operation on the suspected defect candidate region to obtain the final defect region includes: Two-dimensional image difference calculation and three-dimensional shape difference calculation are performed on the suspected defect candidate regions to obtain two-dimensional texture deviation information and three-dimensional shape deviation information; Based on preset color feature conditions, the color feature of the suspected defect candidate region is re-judged to obtain color deviation information; Extract the geometric morphological features of the suspected defect candidate region and perform a geometric morphological verification operation on the suspected defect candidate region; Based on the two-dimensional texture deviation information, the three-dimensional shape deviation information, the color deviation information, and the results of the geometric shape verification operation, the final defect region is determined from the suspected defect candidate region.

7. The method for detecting appearance defects in automotive plastic parts according to claim 6, characterized in that, The step of determining the final defect region from the suspected defect candidate regions based on the two-dimensional texture deviation information, the three-dimensional shape deviation information, the color deviation information, and the result of the geometric shape verification operation includes: When the suspected defect candidate region passes the geometric shape verification operation, the two-dimensional texture deviation information, the three-dimensional shape deviation information, the color deviation information, and the geometric shape features are weighted and summed to obtain a re-judgment score; When the re-judgment score reaches the preset re-judgment threshold for the corresponding model, the corresponding suspected defect candidate area is determined as the final defect area.

8. The method for detecting appearance defects in automotive plastic parts according to claim 1, characterized in that, After generating an anomaly heatmap based on the corrected two-dimensional image data, and determining the suspected defect candidate region of the automotive plastic part based on the anomaly heatmap, before performing a multi-feature re-judgment operation on the suspected defect candidate region to obtain the final defect region, the method further includes: The main body mask of the automotive plastic part is generated based on the corrected two-dimensional image data, and the whitelist area of ​​the corresponding model is obtained; wherein, the main body mask is used to identify the main area of ​​the automotive plastic part, and the whitelist area contains the position information of the inherent normal structure of the corresponding model; Based on the main body mask, suspected defect candidate areas located outside the main body area of ​​the automotive plastic part are eliminated, and based on the whitelist area, suspected defect candidate areas corresponding to the inherent normal structure are blocked.

9. The method for detecting appearance defects in automotive plastic parts according to claim 1, characterized in that, The step of performing a type identification operation on the final defect region to obtain the defect type includes: Based on the texture and color features of the final defect region, two-dimensional texture defects are identified; Based on the morphological features of the final defect region, three-dimensional morphological defects are identified; If the final defect region simultaneously satisfies both the two-dimensional texture defect condition and the three-dimensional morphology defect condition, it is identified as a composite defect. And / or, The calculation of the defect quantification parameters for the final defect region includes: The pixel coordinates of the final defect area are converted into spatial coordinates to obtain the defect location of the final defect area on the automotive plastic part. Based on the pixel area and spatial resolution of the final defect region, the defect area is calculated, and based on the height difference of the final defect region, the defect depth and defect volume are calculated. The defect location, defect area, defect depth, and defect volume constitute the defect quantification parameters.

10. The method for detecting appearance defects in automotive plastic parts according to claim 1, characterized in that, The method further includes: Based on the appearance judgment rules corresponding to the model of the automotive plastic part, and combined with the defect type of the final defect area, the defect quantification parameters, and the detection area to which the final defect area belongs, the defect level of the final defect area is generated. Based on the defect level, a detection conclusion and handling recommendation are generated, and a detection report is generated; The inspection report includes the model of the automotive plastic part, the defect quantification parameters, the defect level, the inspection conclusion, and the handling recommendations.

Images