Defect detection method and system for automobile interior door panel

Abstract

An automobile interior door panel defect detection method belongs to the field of machine vision detection, and can effectively solve the problem of inaccurate buckle detection caused by wiring harness shielding by collecting images through a main view angle camera and a compensation view angle camera and combining with a light source system. And meanwhile, the scratch contrast is improved through specific angle illumination of the blue light LED and a self-adaptive image enhancement technology, a deep learning segmentation model is improved in combination, fine scratches are stably recognized, and the detection accuracy of the automotive interior door panel is improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method and system for detecting defects in automotive interior door panels. Background Technology

[0002] Interior door panels are a critical component of a vehicle, and their manufacturing quality directly affects the vehicle's aesthetics, safety, and user experience. The assembly of interior door panels requires the precise installation of multiple clips and ensuring a flawless surface. However, due to the complexity of the process, material properties, and production cycle limitations, misinstallation or omission of clips, as well as scratches on the door panel surface, have become two common quality issues.

[0003] For example, CN202411609038.9 describes a visual inspection system for automotive door panel components. This system uses a single front-view camera to acquire images and determines the presence of clips through image matching technology. However, this method makes it difficult to accurately identify clips obscured by wiring harnesses, which can easily lead to misjudgments. Similarly, CN202411123647.3 describes a machine vision-based automotive door panel performance inspection system and method. This technical solution uses a visual sensing unit to inspect the performance of the door panel, but it is not specific enough for detecting the installation status of clips and surface scratches on automotive interior door panels. Summary of the Invention

[0004] This invention provides a method for detecting defects in automotive interior door panels. The technical solution is as follows:

[0005] On the one hand, a method for detecting defects in automotive interior door panels is provided. This method is implemented by electronic devices and includes: deploying at least one main-view camera and at least two compensated-view cameras. The optical axis of the main-view camera has an angle ≤5° with the normal of the buckle mounting surface. The compensated-view cameras are symmetrically distributed and their optical axes have an angle θ∈[45°, 75°] with the main-view camera. Both the main-view camera and the compensated-view camera are equipped with adjustable light sources.

[0006] The main view image is subjected to buckle detection to obtain the prediction confidence and occlusion area ratio; when the prediction confidence is lower than the first preset threshold or the occlusion area ratio is higher than the second preset threshold, the compensated view image is called to reconstruct the three-dimensional boundary of the buckle occlusion area, the buckle information is compared with the standard template, and the defect type is output.

[0007] Scratch detection is performed on the surface area. The blue LED is adjusted to the incident angle α. The image is enhanced by CLAHE and then input into the U-Net model. Scratch defects are detected through image enhancement and segmentation model.

[0008] Preferably, the angle between the optical axis of the main-view camera and the normal of the buckle mounting surface is 0°.

[0009] Preferably, when calling the compensated viewpoint image for 3D boundary reconstruction, a priority scheduling mechanism is adopted:

[0010] If the occlusion area ratio is higher than the second preset threshold but lower than the third preset threshold, the image captured by the compensated viewing angle camera with a smaller angle to the main viewing angle will be used first.

[0011] If the occlusion area ratio is higher than or equal to the third preset threshold, then all images acquired by available compensated time-view cameras are synchronously invoked and multi-view 3D point cloud fusion processing is performed.

[0012] Preferably, when the image undergoes CLAHE enhancement processing, its parameters are adaptively adjusted according to the door panel color:

[0013] For dark-colored door panels, a higher contrast limit threshold ClipLimit = 3.0-4.0 and a larger tile size TileGridSize = 12*12 are used.

[0014] For light-colored door panels, a higher contrast limit threshold ClipLimit = 1.5-2.0 and a larger tile size TileGridSize = 6*6 are used.

[0015] Preferably, the incident angle α of the blue LED light source and the camera viewing angle satisfy the following constraint relationship:

[0016]

[0017] Where θ is the angle between the optical axis of the compensated view camera and the main view, n 门板 The refractive index of the door panel material is used to maximize the intensity of scattered light in the scratched area by dynamically adjusting the incident angle.

[0018] Preferably, when calling the compensated viewpoint image for boundary completion, the following steps are included:

[0019] The correspondence between the main viewpoint and the compensated viewpoint is established by SIFT feature matching; the depth information of the occluded area is calculated based on epipolar geometry; and the three-dimensional boundary is reconstructed by fitting B-spline curves.

[0020] Preferably, when calling the compensated viewpoint image for boundary completion, a priority scheduling mechanism is adopted: if the occlusion area is >15% but <30%, the compensated viewpoint camera image with a smaller angle to the main viewpoint is called first; if the occlusion area is ≥30%, all available compensated viewpoint camera images are called simultaneously, and multi-view 3D point cloud fusion is performed.

[0021] Preferably, in the improved U-Net model, the compression ratio of the SE attention module is set to 16, and an ASPP module is introduced in the decoder stage, with a hole ratio combination of [1,3,6,9].

[0022] Preferably, the loss function for the instance segmentation model's mask prediction is the Dice coefficient loss.

[0023]

[0024] Improved channel weight calculation for the SE attention module in the U-Net model:

[0025] w = σ(W2δ(W1z)), z: global average pooling feature. (Compression ratio r = 16);

[0026] Multi-scale fusion of ASPP modules:

[0027] y = Concat[DConV(x,1),DConV(x,3),DConV(x,6),DConV(x,9)]DConV(d): dilated convolution with dilation rate d.

[0028] On the other hand, a defect detection system for automotive interior door panels is provided. This device is applied to a method for detecting defects in automotive interior door panels, and the device includes:

[0029] A multi-view imaging module includes at least one main-view industrial camera and at least two compensated-view industrial cameras. The angle between the optical axis of the main-view industrial camera and the normal of the snap-fit ​​mounting surface is ≤5°. The compensated-view industrial cameras are symmetrically distributed and the angle between their optical axes and the main-view camera is θ∈[45°,75°].

[0030] An adjustable light source module equips each of the industrial cameras with an independent bar light source and a blue LED light source;

[0031] The image acquisition and marking module is configured to synchronously control the multi-view imaging module and the adjustable light source module to acquire images and mark them with timestamps and unique traceability codes for the door panels.

[0032] The snap-in detection module is configured to process the main view image based on the instance segmentation model, extract the snap-in region mask and boundary contour, and calculate the prediction confidence and occlusion area ratio.

[0033] The 3D reconstruction module is configured to call the compensated viewpoint image and perform SIFT feature matching, epipolar geometry calculation and B-spline curve fitting when the prediction confidence or occlusion area ratio exceeds a threshold, so as to complete the 3D boundary reconstruction of the occluded area.

[0034] The scratch detection module is configured to control the adjustable light source module to adjust the incident angle α of the blue LED, and after performing CLAHE enhancement processing on the acquired surface image, input it into the U-Net model for scratch segmentation;

[0035] The defect determination module is configured to compare the detection results with a standard template and output the defect type and classification results.

[0036] Preferably, the adjustable light source module includes:

[0037] The polarized strip light source has a length that matches the width of the door panel, and the polarization direction of the light source forms an angle of 80°-100° with the cross-polarizing filter at the camera end; the servo mechanism that dynamically adjusts the incident angle of the blue LED ensures that the intensity of scattered light in the scratched area is maximized.

[0038] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described method for detecting defects in automotive interior door panels.

[0039] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement the above-described method for detecting defects in automotive interior door panels.

[0040] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0041] 1. This invention can automatically call up the compensation view image and perform high-precision three-dimensional boundary reconstruction when the buckle feature recognition is incomplete due to wire harness occlusion in the main view. This solves the problem of missed detection and misjudgment caused by occlusion in traditional vision methods, and realizes complete and accurate detection of buckle installation status.

[0042] 2. This invention combines blue LED specific angle illumination with adaptive image enhancement, which significantly improves the contrast of scratch imaging. It also improves the detection of minor scratches by improving the deep learning segmentation model. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a logic block diagram of a method for detecting defects in automotive interior door panels provided by the present invention;

[0045] Figure 2 This is a camera deployment diagram of a method for detecting defects in automotive interior door panels provided by the present invention;

[0046] Figure 3 This invention provides a UML timing diagram for buckle detection in automotive interior door panel defect detection.

[0047] Figure 4 This invention provides a UML timing diagram for scratch detection in automotive interior door panel defect detection.

[0048] Figure 5 This invention provides a logic block diagram for scratch detection in automotive interior door panel defect detection. Detailed Implementation

[0049] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0050] like Figure 1 As shown, the overall logical framework of this invention is as follows:

[0051] A method for detecting defects in automotive interior door panels, such as Figure 2 As shown, a multi-view imaging system is deployed to acquire images of the door panel. This system includes at least one main-view camera and at least two compensated-view cameras. The angle between the optical axis of the main-view camera and the normal to the mounting surface of the latch is ≤5°. The compensated-view cameras are symmetrically distributed, and their optical axes form an angle θ∈[45°, 75°] with the main-view camera. This angle is an optimal value in stereo vision that balances baseline and viewing angle, providing sufficient parallax information for 3D reconstruction. Notably, the angle between the optical axis of the main-view camera and the normal to the mounting surface of the latch is 0°.

[0052] It is worth mentioning that the main view camera can be a high-resolution, low-distortion telecentric lens camera, and the compensation camera can be a high frame rate camera.

[0053] Furthermore, each camera is equipped with an independent adjustable light source, including a strip light source and a blue LED light source. The strip light source can be a high-density white LED strip light source, with a length of 1 / 2 the width of the interior door panel, evenly distributed on both sides of the door panel to be inspected at an angle of 45°-60°, providing low-angle illumination to highlight the broad edges of protruding or recessed structures such as buckles and buttons. The blue LED light source uses a high-power, highly uniform blue LED linear array light source with a wavelength of 450nm. Its adjustment mechanism requires a highly integrated, high-precision stepper motor, allowing the program to precisely control its incident angle α. The above light sources are driven by an independent programmable controller. The above optical design improves image quality by enhancing the quality of the acquired image through illumination angle and wavelength, reducing the difficulty of subsequent image processing algorithms.

[0054] Furthermore, images are captured and marked with timestamps and unique traceability codes for the door panels.

[0055] Specifically, image data is named in the format {traceability code}-{timestamp}-{camera ID}.raw and stored in the database. All processing structures are bound to this unique identifier to achieve full lifecycle quality traceability.

[0056] Furthermore, such as Figure 3 and Figure 5 As shown, a snap-fit ​​detection is performed on the main view image to obtain the prediction confidence and occlusion area ratio. When the prediction confidence is lower than a first preset threshold or the occlusion area ratio is higher than a second preset threshold, a compensation view image is invoked to reconstruct the three-dimensional boundary of the snap-fit ​​occluded area. The snap-fit ​​information is compared with a standard template to output the defect type. When invoking the compensation view image for three-dimensional boundary reconstruction, a priority scheduling mechanism is adopted.

[0057] If the occlusion area ratio is higher than the second preset threshold but lower than the third preset threshold, the image acquired by the compensated view camera with a smaller angle to the main view is preferentially used; if the occlusion area ratio is higher than or equal to the third preset threshold, the images acquired by all available compensated view cameras are used simultaneously, and multi-view 3D point cloud fusion processing is performed.

[0058] When calling the compensated viewpoint image for boundary completion, the following steps are included:

[0059] The correspondence between the main viewpoint and the compensated viewpoint is established by SIFT feature matching; the depth information of the occluded area is calculated based on epipolar geometry; and the three-dimensional boundary is reconstructed by fitting B-spline curves.

[0060] When calling the compensated viewpoint image for boundary completion, a priority scheduling mechanism is adopted: if the occlusion area is >15% but <30%, the compensated viewpoint camera image with a smaller angle to the main viewpoint is called first; if the occlusion area is ≥30%, all available compensated viewpoint camera images are called simultaneously, and multi-view 3D point cloud fusion is performed. This priority scheduling mechanism achieves the optimal balance between detection efficiency and detection accuracy through intelligent resource allocation strategy.

[0061] Specifically, the instance segmentation model adopts a Mask R-CNN architecture based on ResNet-50-FPN, where the first preset threshold for prediction confidence is set to 0.92, and the second preset threshold for occlusion area ratio is set to 15%. During 3D reconstruction, the RANSAC algorithm is first used to remove SIFT mismatched point pairs, ensuring a matching accuracy greater than 98%. Then, a sparse 3D point cloud of the occlusion edges is calculated using triangulation under epipolar geometric constraints. Finally, a cubic uniform B-sample curve is used for fitting, and the control point coordinates are optimized using the least squares method to ensure that the error between the reconstructed boundary and the real physical contour is no greater than ±0.2mm. This error is verified by sampling within the measurement range using a laser tracker, and its repeatability must meet ±0.05mm.

[0062] Furthermore, such as Figure 4 As shown, scratch detection is performed on the surface area. The blue LED is adjusted to the incident angle α, and the image is enhanced by CLAHE before being input into the U-Net model. Scratch defects are detected through image enhancement and segmentation models. The parameters of the image are adaptively adjusted according to the door panel color during CLAHE enhancement.

[0063] For dark-colored door panels, a higher contrast limit threshold ClipLimit = 3.0-4.0 and a larger tile size TileGridSize = 12*12 are used.

[0064] For light-colored door panels, a higher contrast limit threshold ClipLimit = 1.5-2.0 and a larger tile size TileGridSize = 6*6 are used.

[0065] The incident angle α of the blue LED light source and the camera viewing angle satisfy the following constraint relationship:

[0066]

[0067] Where θ is the angle between the optical axis of the compensated view camera and the main view, n 门板 The refractive index of the door panel material is used to maximize the intensity of scattered light in the scratched area by dynamically adjusting the incident angle.

[0068] In the improved U-Net model, the compression ratio of the SE attention module is set to 16, and the ASPP module is introduced in the decoder stage, with a hole ratio combination of [1,3,6,9].

[0069] The loss function for the instance segmentation model's mask prediction uses the Dice coefficient loss.

[0070]

[0071] Improved channel weight calculation for the SE attention module in the U-Net model:

[0072] w = σ(W2δ(W1z)), z: global average pooling feature. (Compression ratio r = 16);

[0073] Multi-scale fusion of ASPP modules:

[0074] y = Concat[DConV(x,1),DConV(x,3),DConV(x,6),DConV(x,9)]DConV(d): dilated convolution with dilation rate d.

[0075] On the other hand, a defect detection system for automotive interior door panels is provided. This device is applied to a method for detecting defects in automotive interior door panels, and the device includes:

[0076] A multi-view imaging module includes at least one main-view industrial camera and at least two compensated-view industrial cameras. The angle between the optical axis of the main-view industrial camera and the normal of the snap-fit ​​mounting surface is ≤5°. The compensated-view industrial cameras are symmetrically distributed and the angle between their optical axes and the main-view camera is θ∈[45°,75°].

[0077] An adjustable light source module equips each of the industrial cameras with an independent bar light source and a blue LED light source;

[0078] The image acquisition and marking module is configured to synchronously control the multi-view imaging module and the adjustable light source module to acquire images and mark them with timestamps and unique traceability codes for the door panels.

[0079] The snap-in detection module is configured to process the main view image based on the instance segmentation model, extract the snap-in region mask and boundary contour, and calculate the prediction confidence and occlusion area ratio.

[0080] The 3D reconstruction module is configured to call the compensated viewpoint image and perform SIFT feature matching, epipolar geometry calculation and B-spline curve fitting when the prediction confidence or occlusion area ratio exceeds a threshold, so as to complete the 3D boundary reconstruction of the occluded area.

[0081] The scratch detection module is configured to control the adjustable light source module to adjust the incident angle α of the blue LED, and after performing CLAHE enhancement processing on the acquired surface image, input it into the U-Net model for scratch segmentation;

[0082] The defect determination module is configured to compare the detection results with a standard template and output the defect type and classification results.

[0083] Furthermore, the adjustable light source module includes:

[0084] The polarized strip light source has a length that matches the width of the door panel, and the polarization direction of the light source forms an angle of 80°-100° with the cross-polarizing filter at the camera end; the servo mechanism that dynamically adjusts the incident angle of the blue LED ensures that the intensity of scattered light in the scratched area is maximized.

[0085] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described method for detecting defects in automotive interior door panels.

[0086] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement the above-described method for detecting defects in automotive interior door panels.

[0087] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting defects in an automotive interior door panel, the method comprising: obtaining a digital image of the door panel; and processing the digital image to detect defects in the door panel. The method comprises the following steps: deploy at least one main view camera and at least two compensation view cameras, the optical axis of the main view camera is ≤5° with the normal of the buckle installation surface, the compensation view cameras are symmetrically distributed and the optical axis is θ∈[45°, 75°] with the main view, and the main view camera and the auxiliary view camera are both equipped with adjustable light sources; the main view image is detected to obtain a prediction confidence and a shielding area ratio; when the prediction confidence is lower than a first preset threshold or the shielding area ratio is higher than a second preset threshold, the compensation view image is called to perform three-dimensional boundary reconstruction on the buckle shielding area, the buckle information is compared with a standard template, and a defect type is output; the surface area is detected for scratches, the blue light LED is adjusted to an incident angle α, the image is input into a U-Net model after CLAHE enhancement, and the scratch defect is detected through image enhancement and segmentation model.

2. The method of claim 1, wherein, The optical axis of the main view camera is 0° with the normal of the buckle installation surface.

3. The method of claim 1, wherein, When the three-dimensional boundary reconstruction is performed on the compensation view image, a priority scheduling mechanism is adopted: if the shielding area ratio is higher than the second preset threshold but lower than a third preset threshold, the image collected by the compensation view camera with a smaller angle with the main view is preferentially called; if the shielding area ratio is higher than or equal to the third preset threshold, the images collected by all available compensation view cameras are synchronously called, and multi-view three-dimensional point cloud fusion processing is performed.

4. The method of claim 1, wherein, When the image is subjected to CLAHE enhancement processing, the parameters are adaptively adjusted according to the color of the door panel: for a dark door panel, a higher contrast limit threshold ClipLimit=3.0-4.0 and a larger block size TileGridSize=12*12 are adopted; for a light door panel, a higher contrast limit threshold ClipLimit=1.5-2.0 and a larger block size TileGridSize=6*6 are adopted.

5. The method of claim 1, wherein, The incident angle α of the blue light LED light source and the camera view angle satisfy the following constraint relationship: where θ is the angle between the camera optical axis and the main viewing angle, n 门板 is the refractive index of the door panel material, and the light intensity in the scratch region is maximized by dynamically adjusting the incident angle.

6. The method of claim 1, wherein, When the boundary is completed by calling the compensation view image, the following steps are included: the corresponding relationship between the main view and the compensation view is established through SIFT feature matching; the depth information of the buckle shielding area is calculated based on epipolar geometry; three-dimensional boundary reconstruction is completed by using B-spline curve fitting.

7. The method according to any one of claims 1 to 6, wherein the method constitutes a system for detecting defects in an automotive interior door panel, wherein the system comprises: It comprises: a multi-view imaging module for deploying at least one main view camera and at least two compensation view cameras and determining the spatial position relationship thereof; an adjustable light source module for providing adjustable light sources for the main view camera and the compensation view camera and controlling the light source switch, intensity and angle; an image acquisition module for controlling the main view camera and the compensation view camera to collect automobile interior door panel images; a marking module for marking the time stamp and the unique traceable code identification of the collected image; a buckle detection module for detecting the main view image to obtain a prediction confidence and a shielding area ratio; a three-dimensional reconstruction module for performing three-dimensional reconstruction on the buckle shielding area; a scratch detection module for detecting the surface area for scratches, adjusting the incident angle of the blue light LED light source, performing CLAHE enhancement on the image and U-Net model segmentation to realize scratch detection. The defect judging module is used for comparing the buckle information with a standard template, combining with the scratch detection result, and outputting a final defect type.

8. The system of claim 7, wherein, The adjustable light source module further comprises a polarization bar-shaped illumination for providing an adaptive door plate width, and a servo mechanism for dynamically adjusting an LED incident angle.

9. An electronic device comprising a memory, a processor, and a motion computer program stored on the memory and in motion on the processor, characterized in that, The processor implements the method of any one of claims 1-6 when executing the program.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-6.

Images