Automobile covering part surface abnormal defect synthesis and detection method based on illumination condition constraint

Abstract

The invention discloses an automobile covering part surface abnormal defect synthesis and detection method based on illumination condition constraint. The method comprises the following steps: acquiring an automobile covering part defect data set; an illumination condition continuous mapping table is constructed, bilinear interpolation sampling is carried out on standardized sampling coordinates containing illumination conditions and defect types to obtain high-dimensional vectors, and encoding and mapping of defect sensing illumination conditions are achieved; a UNet adaptive modulation network integrated with an illumination perception residual block is constructed, abnormal texture enhancement is realized through the adaptive modulation network, and finally, the mask, the normal image and the abnormal texture enhanced image are fused to perform synthesis of an abnormal sample; a pre-trained backbone Wide ResNet50 is used as a feature extractor to carry out multi-scale feature aggregation on an illumination perception anomaly synthesis sample, a dichotomy discriminator is trained to discriminate the aggregated multi-scale features, and a trained model is used to carry out anomaly detection and positioning on a test chart. According to the method, the illumination condition is used as an optimizable control variable to be deeply fused into an abnormal synthesis and detection framework, illumination controllable abnormal synthesis, illumination robust feature learning and high-precision defect positioning are realized, and an effective solution is provided for industrial appearance detection under complex illumination.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and to an industrial appearance inspection method under complex lighting conditions, particularly a method for synthesizing and detecting abnormal defects on the surface of automotive body panels based on lighting condition constraints. Background Technology

[0002] In recent years, with the continuous development of artificial intelligence (AI) technology, the integration of AI with detection technology has enabled automated detection technology, capable of identifying and detecting various defects from diverse images, to become an important means of improving production efficiency and product quality. Over the past few years, numerous research efforts have made progress in core areas such as defect image generation and the construction of detection models.

[0003] Meanwhile, with increasing global attention to environmental protection and sustainable development, the market for new energy vehicles...

[0004] Demand continues to grow. Body panels for new energy vehicles not only need to meet structural strength requirements but also require good aesthetic quality. Various surface defects, such as scratches, dents, and holes, may occur during the production process. These defects not only affect the vehicle's appearance but also directly impact its service life and safety. Therefore, how to efficiently and accurately detect surface defects in body panels, especially minute defects, has become a pressing issue in the manufacturing process of new energy vehicles.

[0005] Traditional manual inspection methods rely on the experience of skilled workers, resulting in high subjectivity, low efficiency, and significant influence of human factors on the inspection results. While traditional machine vision methods, such as optical inspection, have improved inspection efficiency to some extent, they are limited by fixed rules and feature engineering design, making it difficult to effectively handle complex backgrounds, lighting variations, and diverse defect morphologies. In recent years, the development of deep learning technology has provided a new solution for surface defect detection in automotive body panels. Models such as Faster R-CNN, YOLO, and Transformer can accurately identify and classify defects. However, the effectiveness of these supervised models requires a large amount of labeled data. In situations where data is scarce, methods that synthesize anomalous samples using techniques such as Generative Adversarial Networks (GANs) or autoencoders to train detection models have also opened up new avenues for defect detection. However, these data synthesis methods still face some inherent limitations. They typically require a large number of normal and anomalous data samples to learn the distribution characteristics of a specific dataset, which is often difficult to achieve in scenarios with limited data. Furthermore, the diversity and realism of the images generated by these methods are often limited, making it difficult to fully cover the complex and varied defect patterns in real-world scenes.

[0006] The Generative Latent Anomaly Synthesis System (GLASS) achieves image-level anomaly synthesis by superimposing textures in the normal image space and feature-level anomaly synthesis by manipulating features in the latent space. It achieves broader anomaly synthesis coverage through manifold and hypersphere distribution constraints at both the feature-level global anomaly synthesis and the image-level local anomaly synthesis. However, this approach still suffers from the following limitations at the image-level synthesis level: 1. Synthetic images lack lighting and shadows and have a lack of three-dimensionality: Existing methods usually do not consider the influence of ambient lighting on the appearance of defects, resulting in a lack of diversity and realism in the generated abnormal samples. This reduces the generalization ability and robustness of the detection model in dealing with complex lighting environments in real industrial scenarios.

[0007] 2. Limited ability to detect minor defects: For minor defects (such as fine scratches or slight discoloration) that are visually similar to normal areas, existing synthesis methods have failed to effectively simulate their subtle changes under different lighting conditions, resulting in low sensitivity and recognition accuracy of the trained detection model for minor defects.

[0008] In addition, the existing practice of adding lighting conditions to the synthesis of abnormal samples using global or local lighting also has the following drawbacks: 1. Global illumination changes cause distribution shifts: The overall brightness statistics of real samples and synthetic anomalous samples are significantly different, which leads to the discriminator trained on these samples often learning the "brightness difference" rather than the "defect difference", resulting in weak generalization ability.

[0009] 2. Local illumination lacks lighting condition constraints and feature-level modulation: Simple pixel-level adjustments to local illumination cannot calculate gradients and cannot participate in end-to-end training, thus preventing the lighting conditions from being optimized according to the detection task.

[0010] 3. Separation of illumination conditions from the anomaly detection framework: Existing methods treat illumination adjustment as an independent data augmentation step and do not deeply integrate it with the anomaly detection framework. As a result, illumination conditions cannot affect subsequent discriminator training, and the discriminator training process does not consider the influence of illumination conditions, making it impossible to learn the anomaly feature representation of illumination perception. Summary of the Invention

[0011] This invention aims to address the problems of existing generative latent anomaly synthesis frameworks lacking realistic illumination changes during image-level anomaly synthesis, directly adding illumination leading to decreased detection performance, and existing local illumination methods lacking illumination condition constraints and feature-level modulation. It provides a method for synthesizing and detecting surface anomalies and defects in automotive body panels based on illumination condition constraints.

[0012] This invention constructs an illumination-aware anomaly synthesis and detection framework, using normal images, abnormal textures, and abnormal masks as conditional inputs, and illumination condition parameters as control inputs. Through illumination condition encoding and mapping, illumination enhancement only in defect areas, and illumination-aware feature modulation mechanisms, it achieves continuous sampleable representation of illumination conditions and end-to-end training that is deeply integrated with the detection framework, thereby realizing high-precision anomaly detection in industrial scenarios.

[0013] To achieve the above objectives, the present invention adopts the following technical solution: A method for detecting abnormal defects on the surface of automotive body panels based on illumination constraints includes the following steps: S1. Obtain the dataset of defects in automotive body panels; S2. Construct a continuous mapping table of illumination conditions, and perform bilinear interpolation sampling on the normalized sampling coordinates containing illumination conditions and defect types to obtain a high-dimensional vector, thereby realizing the encoding and mapping of illumination conditions for defect perception. S3. Construct a UNet adaptive modulation network that incorporates illumination-aware residual blocks, and use the adaptive modulation network to enhance abnormal textures. Finally, fuse the mask, normal image, and abnormal texture-enhanced image to synthesize abnormal samples. S4. Use a pre-trained backbone network (Wide ResNet50) as a feature extractor to perform multi-scale feature aggregation on the synthetic samples of light perception anomalies, train a binary classification discriminator to discriminate the aggregated multi-scale features, and use the trained model to detect and locate anomalies in the test image. Preferably, the automotive body panel defect dataset in step S1 is collected from the automotive body panel stamping production line, and includes data on three types of defects: spots, dents, and scratches. The data acquisition system is triggered by an NPN diffuse reflection photoelectric sensor. When the automotive body panel is detected to have reached the designated shooting area, the image acquisition unit is immediately triggered to take a picture. The image acquisition unit consists of 16 industrial cameras, which can simultaneously capture images of the body panel. Automotive body panels are characterized by large macroscopic dimensions and high surface quality requirements. Therefore, the acquired images need to be divided into blocks, and each small block is individually inspected using an algorithm to detect the presence of dents, scratches, and spots within that small area. Finally, the images are stitched together to form a panoramic view of the entire component's surface quality, forming part of our training dataset.

[0014] Preferably, the specific process of step S2 is as follows: S21. The Continuous Illumination Condition Map (L-CME) is an orthogonal embedding matrix of size d×d, obtained by QR decomposition. The specific steps are as follows: Generate a random matrix:

[0015] QR decomposition of a random matrix decomposes any full-rank square matrix into the product of an orthogonal matrix Q and an upper triangular matrix R:

[0016] S22. Normalized sampling coordinates are obtained by combining the one-dimensional illumination condition parameter L (preferably ranging from -5 to 5) and three different defect category information (ranging from 0 to 1). Standardization is required. The specific steps are: Conditional normalization is performed on the parameters and categories respectively. To improve numerical stability and match the boundaries of the subsequent embedding space, L and C are linearly normalized to [0,1], resulting in the normalized sampling coordinates P=[ , ]:

[0017] S23. Using a continuous mapping table of illumination conditions, bilinear interpolation is performed on the normalized sampled coordinates to obtain a high-dimensional vector:

[0018] Bilinear interpolation ensures that gradients can be backpropagated to the mapping link during end-to-end training. Nearby sampling ensures that the generated lighting code is strongly correlated with the visual characteristics of specific defects (such as diffuse reflection of dents and specular highlights of scratches). And... By sampling nearby areas, continuous and smooth control of light intensity is achieved. This yields... This is a high-dimensional representation of the illumination conditions, which can be used as a conditional input for subsequent modulation networks.

[0019] Preferably, the specific process of step S3 is as follows: S31. The illumination-aware adaptive modulation network uses UNet as the generative backbone. UNet is the generator in the encoder-decoder architecture. The encoder performs multi-layer downsampling to extract multi-scale semantics, and the decoder performs upsampling at each level and fused with aligned encoded features via skip connections. An illumination-aware residual block is added after the backbone convolution at each scale. Therefore, the encoder contains an initial convolutional layer, an illumination-aware residual block, and downsampling, while the decoder contains upsampling, an illumination-aware residual block, and finally, the output is normalized.

[0020] The illumination-aware residual block receives the illumination condition vector and scales and biases the channel features, which only takes effect at the defect mask, thereby achieving illumination-consistent abnormal texture reconstruction and enhancement. The specific steps are as follows: Extracted Generate a set of scaling and offset Typically, a perceptron (MLP) containing multiple linear layers and activation functions (SiLu) is used for generation.

[0021] The illumination-aware residual block integrates a mask-gated channel modulation mechanism (MG-FILM). For intermediate features F in the feature extraction network, the generated [feature] is used... and Modulate its illumination:

[0022] Among them, defect mask ( Strictly limited to the characteristic locations within the defect area. As modulated features, this mechanism allows the network to generate different scale and shift feature responses based on lighting conditions, adaptively learning how to adjust feature extraction and anomaly generation methods under different lighting scenarios.

[0023] S32. The UNet network, employing a light-sensing mechanism, processes the abnormal texture image. The output is a local abnormal lighting map learned by the network that highly adapts to the defect region, denoted as... : It is a picture with Single-channel images of the same size, The texture images are derived from the DTD dataset, and their values ​​represent the light intensity that the network has learned should be added to each pixel. To guide the network in generating physically accurate lighting effects, we use a pre-defined physical lighting model. As supplementary supervision, constraints will be applied during training. Within the defect area and Get as close as possible. A local illumination field is a location-dependent intensity map, often used to simulate the shape of "center attenuation" in local illumination. It is a scalar value determined by the user's desired brightness level L, and uses gamma mapping to control the base brightness of the entire light spot. The function controls the light spot to darken from the center outwards. These are the centers of different defect types. By controlling the attenuation scale, an intensity map can be obtained that is bright in the center and gradually darkens around the edges.

[0024] The lighting map generated by the network Applied to raw anomalous textures:

[0025] in This represents an abnormal texture image after lighting processing. The value representing the light injection intensity scalar determines "how much light is added" and can be obtained by mapping from the brightness parameter L.

[0026] S33. Finally, the abnormal sample is synthesized by fusing the mask, normal image, and abnormal texture image. The formula is as follows:

[0027] Preferably, the specific process of step S4 is as follows: S41. Based on a pre-trained backbone network (Wide ResNet50) as the feature extractor B, its parameters are usually frozen during the training and inference phases to ensure feature stability and prevent overfitting, and it is used only as a feature extractor. Multi-scale features are extracted from the image to obtain the feature map F:

[0028] Divide the feature map at each scale into multiple patches:

[0029] Feature maps at different levels have different spatial dimensions and need to be aligned to a uniform size. Bilinear interpolation is used to align multi-scale features to a reference size, with the default being alignment to the first level (e.g., ...). ):

[0030] Preprocess the features at each level separately, and unify the dimensions:

[0031] Finally, the features from multiple levels are aggregated into a unified representation:

[0032] S42. Train a binary classifier D, including light-sensing abnormal samples and normal samples. The input to the discriminator is the unified patch feature vector that has been preprocessed and aggregated in the previous steps. The output is the abnormal probability that distinguishes normal and abnormal features. S43. Use the trained model to score the test map patch and output the anomaly probability of each patch (the closer to 1, the more likely it is to be an anomaly):

[0033] The anomaly probabilities of the patch are reconstructed into a spatial heatmap, followed by anomaly detection and localization.

[0034] This invention offers the following advantages: It constructs a lighting-aware anomaly synthesis and detection framework. This framework, through a continuous lighting condition mapping table and an adaptive modulation network, achieves continuous and precise control over the lighting conditions of synthesized anomaly samples, effectively simulating the complex and variable lighting environment in industrial scenarios. Furthermore, through end-to-end training deeply integrated with the detection framework, it achieves high-precision anomaly detection in industrial settings. Unlike existing methods that ignore lighting or employ global enhancement, the mask-gated feature modulation mechanism and local lighting field model proposed in this invention can perform lighting-consistent feature enhancement and texture reconstruction only within the defect area, thereby avoiding model misjudgments caused by global distribution shifts. This invention achieves significant improvements over existing generative latent anomaly synthesis frameworks on automotive body panel defect datasets across multiple metrics. Taking spot defects as an example, in anomaly recognition, image-level detection metrics I_AUC and I_AP reach 99.48% and 99.5%, respectively, representing improvements of 3.9% and 0.68%. In defect localization, pixel-level detection metrics P_AUC and P_PRO reach 99.33% and 94.98%, respectively, representing improvements of 3.9% and 3.13%. Extensive experimental results demonstrate that this invention successfully integrates illumination conditions as an optimizable control variable into the anomaly synthesis and detection framework, achieving illumination-controlled anomaly synthesis, illumination-robust feature learning, and high-precision defect localization, providing an effective solution for industrial appearance inspection under complex lighting conditions. Attached Figure Description

[0035] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0037] like Figure 1 As shown, a method for synthesizing and detecting abnormal defects on the surface of automotive body panels based on illumination constraints includes the following steps: S1. Obtain the dataset of defects in automotive body panels; In step S1, the automotive body panel defect dataset was collected from the automotive body panel stamping production line, containing data on three types of defects: spots, dents, and scratches. The data acquisition system is triggered by an NPN diffuse reflection photoelectric sensor. When an automotive body panel is detected to be within the designated shooting area, the image acquisition unit is immediately triggered to take a picture. The image acquisition unit consists of 16 industrial cameras, capable of simultaneously capturing images of the body panel. Automotive body panels are characterized by large macroscopic dimensions and high surface quality requirements. Therefore, the acquired images need to be divided into blocks, and each block is individually inspected using an algorithm to detect the presence of dents, scratches, and spots within that block. Finally, these images are stitched together to form a panoramic image of the entire component's surface quality, forming part of our training dataset.

[0038] S2. Construct a continuous mapping table of illumination conditions, and perform bilinear interpolation sampling on the normalized sampling coordinates containing illumination conditions and defect types to obtain a high-dimensional vector, thereby realizing the encoding and mapping of illumination conditions for defect perception. The specific process of step S2 is as follows: S21. The Continuous Illumination Condition Map (L-CME) is an orthogonal embedding matrix of size d×d, obtained by QR decomposition. The specific steps are as follows: Generate a random matrix:

[0039] QR decomposition of a random matrix decomposes any full-rank square matrix into the product of an orthogonal matrix Q and an upper triangular matrix R: S22. Normalized sampling coordinates are obtained by combining the one-dimensional illumination condition parameter L (preferably ranging from -5 to 5) and three different defect category information (ranging from 0 to 1). Standardization is required. The specific steps are: Conditional normalization is performed on the parameters and categories respectively. To improve numerical stability and match the boundaries of the subsequent embedding space, L and C are linearly normalized to [0,1], resulting in the normalized sampling coordinates P=[ , ]:

[0040] S23. Using a continuous mapping table of illumination conditions, bilinear interpolation is performed on the normalized sampled coordinates to obtain a high-dimensional vector:

[0041] Bilinear interpolation ensures that gradients can be backpropagated to the mapping link during end-to-end training. Nearby sampling ensures that the generated lighting code is strongly correlated with the visual characteristics of specific defects (such as diffuse reflection of dents and specular highlights of scratches). And... By sampling nearby areas, continuous and smooth control of light intensity is achieved. This yields... This is a high-dimensional representation of the illumination conditions, which can be used as a conditional input for subsequent modulation networks.

[0042] S3. Construct a UNet adaptive modulation network that incorporates illumination-aware residual blocks, and use the adaptive modulation network to enhance abnormal textures. Finally, fuse the mask, normal image, and abnormal texture-enhanced image to synthesize abnormal samples. The specific process of step S3 is as follows: S31. The illumination-aware adaptive modulation network uses UNet as the generative backbone. UNet is the generator in the encoder-decoder architecture. The encoder performs multi-layer downsampling to extract multi-scale semantics, and the decoder performs upsampling at each level and fused with aligned encoded features via skip connections. An illumination-aware residual block is added after the backbone convolution at each scale. Therefore, the encoder contains an initial convolutional layer, an illumination-aware residual block, and downsampling, while the decoder contains upsampling, an illumination-aware residual block, and finally, the output is normalized.

[0043] The illumination-aware residual block receives the illumination condition vector and scales and biases the channel features, which only takes effect at the defect mask, thereby achieving illumination-consistent abnormal texture reconstruction and enhancement. The specific steps are as follows: Extracted Generate a set of scaling and offset Typically, a perceptron (MLP) containing multiple linear layers and activation functions (SiLu) is used for generation.

[0044] The illumination-aware residual block integrates a mask-gated channel modulation mechanism (MG-FILM). For intermediate features F in the feature extraction network, the generated [feature] is used... and Modulate its illumination:

[0045] Among them, defect mask ( Strictly limited to the characteristic locations within the defect area. As modulated features, this mechanism allows the network to generate different scale and shift feature responses based on lighting conditions, adaptively learning how to adjust feature extraction and anomaly generation methods under different lighting scenarios.

[0046] S32. The UNet network, employing a light-sensing mechanism, processes the abnormal texture image. The output is a local abnormal lighting map learned by the network that highly adapts to the defect region, denoted as... : It is a picture with Single-channel images of the same size, The texture images are derived from the DTD dataset, and their values ​​represent the light intensity that the network has learned should be added to each pixel. To guide the network in generating physically accurate lighting effects, we use a pre-defined physical lighting model. As supplementary supervision, constraints will be applied during training. Within the defect area and Get as close as possible. A local illumination field is a location-dependent intensity map, often used to simulate the shape of "center attenuation" in local illumination. It is a scalar value determined by the user's desired brightness level L, and uses gamma mapping to control the base brightness of the entire light spot. The function controls the light spot to darken from the center outwards. These are the centers of different defect types. By controlling the attenuation scale, an intensity map can be obtained that is bright in the center and gradually darkens around the edges.

[0047] The lighting map generated by the network Applied to raw anomalous textures: in This represents an abnormal texture image after lighting processing. The value representing the light injection intensity scalar determines "how much light is added" and can be obtained by mapping from the brightness parameter L.

[0048] S33. Finally, the abnormal sample is synthesized by fusing the mask, normal image, and abnormal texture image. The formula is as follows: S4. Use the pre-trained backbone (Wide ResNet50) as a feature extractor to perform multi-scale feature aggregation on the synthetic samples of light perception anomalies, train a binary classification discriminator to discriminate the aggregated multi-scale features, and use the trained model to detect and locate anomalies in the test image. The specific process of step S4 is as follows: S41. Based on a pre-trained backbone (Wide ResNet50) as the feature extractor B, its parameters are typically frozen during the training and inference phases to ensure feature stability and prevent overfitting, and it is used only as a feature extractor. Multi-scale features are extracted from the image to obtain the feature map F: Divide the feature map at each scale into multiple patches:

[0049] Feature maps at different levels have different spatial dimensions and need to be aligned to a uniform size. Bilinear interpolation is used to align multi-scale features to a reference size, with the default being alignment to the first level (e.g., ...). ):

[0050] Preprocess the features at each level separately, and unify the dimensions:

[0051] Finally, the features from multiple levels are aggregated into a unified representation:

[0052] S42. Train a binary classifier D, including light-sensing abnormal samples and normal samples. The input to the discriminator is the unified patch feature vector that has been preprocessed and aggregated in the previous steps. The output is the abnormal probability that distinguishes normal and abnormal features. S43. Use the trained model to score the test map patch and output the anomaly probability of each patch (the closer to 1, the more likely it is to be an anomaly):

[0053] The patch scores are reconstructed into a spatial heatmap, followed by anomaly detection and localization.

[0054] Performance testing: The effects of this invention are further illustrated by the following simulation experiments.

[0055] 1) Simulation conditions

[0056] This invention was trained on a server with an Nvidia GeForce RTX 5090 GPU and 32GB of VRAM, and the developed deep learning framework is based on PyTorch. The primary language used in this invention is Python.

[0057] 2) Simulation content

[0058] The experimental setup for this embodiment is as follows: a. The Adam optimizer is used for network training, with an initial learning rate of 0.0001 and illumination parameters limited to [-5, 5]; b. The size of the input image is uniformly adjusted to 288×288 pixels; c. The training period is 100 epochs.

[0059] This invention uses an automotive body panel defect dataset, which contains data on three types of defects: spots, dents, and scratches. The dataset contains 229 normal images and 298, 547, and 333 abnormal images, respectively. For anomaly identification, image-level detection metrics I_AUC and I_AP are used; for defect localization, pixel-level detection metrics P_AUC and P_PRO are used.

[0060] The percentage of correctly identified anomalies in all real anomaly maps is considered the true positive rate. The percentage of true anomaly maps that are misclassified as anomalies is considered the false positive rate. A Relative Occurrence Recognition (ROC) curve is plotted with FPR on the horizontal axis and TPR on the vertical axis. I_AUC is the area under this ROC curve, used to measure the model's overall ability to distinguish between anomalous and normal images at different judgment thresholds. The percentage of images predicted as anomalous that are actually anomalous is considered the accuracy. The recall rate is defined as the percentage of images that are truly anomalous that are identified. A PR curve is plotted with P as the horizontal axis and R as the vertical axis. I_AP is the area under this PR curve, measuring the average accuracy of the model at different recall levels. P_AUC is similar to I_AUC, but the basic unit of judgment becomes "a single pixel," measuring the model's overall ability to distinguish normal pixels from defective pixels at the pixel level. The Cross-Union Ratio (CUI) is calculated for each independent connected defect region and the true defect region. The average of the CUI scores for all regions is then considered as P_PRO, measuring the model's ability to detect weak, small-sized defects. Table 1 shows the comparison results after adding illumination constraints to the generative latent anomaly synthesis framework of this invention.

[0061] Table 1

[0062] As shown in Table 1, the experimental results of this invention demonstrate that introducing illumination constraints into the generative latent anomaly synthesis (GLASS) framework significantly improves all performance metrics of the model on the automotive body panel defect dataset, especially in image-level anomaly recognition. Both I-AUC and I-AP metrics are near perfect, indicating that the model's anomaly detection is highly reliable. In the more challenging pixel-level localization task, this method also achieves systematic progress in P-AUC and P-PRO metrics, with the most significant improvement in the localization accuracy of dents, increasing P-PRO from 92.73% to 97.53%, proving the effectiveness of this method in achieving accurate defect segmentation. Extensive experimental results verify that deeply integrating illumination conditions as a controllable variable into the anomaly synthesis process can significantly improve the robustness and generalization ability of the detection model under real industrial lighting scenarios.

[0063] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for synthesizing and detecting abnormal defects on the surface of automotive body panels based on illumination constraints, characterized in that, Includes the following steps: S1. Obtain the dataset of defects in automotive body panels; S2. Construct a continuous mapping table of illumination conditions, and perform bilinear interpolation sampling on the normalized sampling coordinates containing illumination conditions and defect types to obtain a high-dimensional vector, thereby realizing the encoding and mapping of illumination conditions for defect perception. S3. Construct a UNet adaptive modulation network that incorporates illumination-aware residual blocks, and use the adaptive modulation network to enhance abnormal textures. Finally, fuse the mask, normal image, and abnormal texture-enhanced image to synthesize abnormal samples. S4. Using the pre-trained backbone network Wide ResNet50 as a feature extractor, multi-scale feature aggregation is performed on the synthetic samples of light-sensing anomalies. A binary classification discriminator is trained to discriminate the aggregated multi-scale features. The trained model is then used to detect and locate anomalies in the test image.

2. The method for synthesizing and detecting abnormal defects on the surface of automotive body panels based on illumination constraints as described in claim 1, characterized in that, In step S1, the automotive body panel defect dataset is collected from the automotive body panel stamping production line and includes data on three types of defects: stains, dents, and scratches. The data acquisition system is triggered by an NPN diffuse reflection photoelectric sensor. When the automotive body panel is detected to have reached the designated shooting area, the image acquisition unit is immediately triggered to take a picture. The image acquisition unit consists of an industrial camera that simultaneously captures images of the cover. The acquired images are divided into blocks, and each block is individually tested using an algorithm to detect whether there are dents, scratches, or spots in the block area. Finally, the images are stitched together to form a panoramic view of the surface quality of the entire component, which is part of our training dataset.

3. The method for synthesizing and detecting abnormal defects on the surface of automotive body panels based on illumination constraints as described in claim 1, characterized in that, The specific process of step S2 is as follows: S21. The continuous illumination condition mapping table L-CME is an orthogonal embedding matrix of size d×d, obtained by QR decomposition. The specific steps are: Generate a random matrix: QR decomposition of a random matrix decomposes any full-rank square matrix into the product of an orthogonal matrix Q and an upper triangular matrix R: S22. Normalized sampling coordinates normalize the one-dimensional illumination condition parameter L and the information of three different defect categories; the specific steps are: Conditional normalization is performed on the parameters and categories respectively. To improve numerical stability and match the boundaries of the subsequent embedding space, L and C are linearly normalized to [0,1], resulting in the normalized sampling coordinates P=[ , ]: S23. Using a continuous mapping table of illumination conditions, bilinear interpolation is performed on the normalized sampled coordinates to obtain a high-dimensional vector: Bilinear interpolation ensures that gradients can be backpropagated to the mapping link during end-to-end training; Nearby sampling ensures that the generated illumination code is strongly correlated with the visual characteristics of the specific defect; while... By sampling nearby areas, continuous and smooth control of light intensity is achieved; here we obtain This is a high-dimensional representation of the illumination conditions, which can be used as a conditional input for subsequent modulation networks.

4. The method for synthesizing and detecting abnormal defects on the surface of automotive body panels based on illumination constraints as described in claim 1, characterized in that, The illumination condition parameter L mentioned in step S22 has a value range of [-5, 5], and the value ranges of the three different defect category information are as follows: .

5. The method for synthesizing and detecting abnormal defects on the surface of automotive body panels based on illumination constraints as described in claim 1, characterized in that, The specific process of step S3 is as follows: S31. The illumination-aware adaptive modulation network uses UNet as the generation backbone. UNet is the generator in the encoder-decoder architecture. The encoder performs multi-layer downsampling to extract multi-scale semantics, and the decoder performs upsampling step by step and fused with the aligned encoded features. After the convolution of the backbone at each scale, a new illumination-aware residual block is added. Therefore, the encoder contains an initial convolutional layer, an illumination-aware residual block, and downsampling. The decoder contains upsampling and an illumination-aware residual block, and finally the output is normalized. The illumination-aware residual block receives the illumination condition vector and scales and biases the channel features, which only takes effect at the defect mask, thereby achieving illumination-consistent abnormal texture reconstruction and enhancement. The specific steps are as follows: Extracted Generate a set of scaling and offset Typically, a perceptron MLP containing multiple linear layers (Linea) and the activation function SiLu is used for generation. The illumination-aware residual block integrates a mask-gated channel modulation mechanism (MG-FILM). For intermediate features F in the feature extraction network, the generated [feature] is used... and Modulate its illumination: Among them, defect mask ( Strictly limited to the characteristic locations within the defect area. As modulated features, this mechanism allows the network to generate different scale and shift feature responses based on lighting conditions, and adaptively learn how to adjust feature extraction and anomaly generation methods under different lighting scenarios. S32. The UNet network, employing a light-sensing mechanism, processes the abnormal texture image. The output is a local abnormal lighting map learned by the network that highly adapts to the defect region, denoted as... : It is a picture with Single-channel images of the same size, The texture images are derived from the DTD dataset, and their values ​​represent the light intensity that the network has learned should be added to each pixel. To guide the network in generating physically accurate lighting effects, we use a pre-defined physical lighting model. As supplementary supervision; during training, constraints will be applied. Within the defect area and As close as possible; A local illumination field is a location-dependent intensity map, often used to simulate local illumination with a "center attenuation" shape. It is a scalar value determined by the user's desired brightness level L, and uses gamma mapping to control the base brightness of the entire light spot; The function controls the light spot to darken from the center outwards. These are the centers of different defect types. By controlling the attenuation scale, an intensity map can be obtained that is bright in the center and gradually darkens around the edges. The lighting map generated by the network Applied to raw anomalous textures: in This represents an abnormal texture image after lighting processing. The light injection intensity scalar represents "how much light is added" and can be obtained by mapping from the brightness parameter L; S33. Finally, the abnormal sample is synthesized by fusing the mask, normal image, and abnormal texture image. The formula is as follows: 。 6. The method for synthesizing and detecting abnormal defects on the surface of automotive body panels based on illumination constraints as described in claim 1, characterized in that, The specific process of step S4 is as follows: S41. Based on the pre-trained backbone network Wide ResNet50 as the feature extractor B, to ensure feature stability and prevent overfitting, its parameters are usually frozen during the training and inference phases, and it is used only as a feature extractor; Multi-scale features are extracted from the image to obtain the feature map F: Divide the feature map at each scale into multiple patches: Feature maps at different levels have different spatial dimensions and need to be aligned to a uniform size. Bilinear interpolation is used to align multi-scale features to a reference size, with the default alignment being the first level. Preprocess the features at each level separately, and unify the dimensions: Finally, the features from multiple levels are aggregated into a unified representation: S42. Train a binary classifier D, including light-sensing abnormal samples and normal samples. The input to the discriminator is the unified patch feature vector that has been preprocessed and aggregated in the previous steps. The output is the abnormal probability that distinguishes normal and abnormal features. S43. Use the trained model to score the test map patch and output the anomaly probability of each patch: The anomaly probabilities of the patch are reconstructed into a spatial heatmap, followed by anomaly detection and localization. 。

Images