Deep learning-based empennage plate surface defect identification method and system

Abstract

The invention discloses an empennage plate surface defect identification method and system based on deep learning, and relates to the technical field of deep learning, and the method comprises the following steps: generating synthetic training data based on a physical rendering engine simulation imaging condition; constructing an artifact removal encoder and a defect identification decoder in a deep learning network based on the synthetic training data; performing adversarial training on the artifact removal encoder and the defect recognition decoder based on an adversarial generative network to obtain the artifact removal encoder and the defect recognition decoder after adversarial training; inputting an image acquired by the industrial camera into an artifact removal encoder after adversarial training; and outputting a defect identification result based on the defect identification decoder after adversarial training. According to the method, the omission ratio and the false alarm rate can be effectively reduced, so that the production efficiency and the reliability of product quality control are improved, and a more accurate and stable solution is provided for surface quality detection of the automobile spoiler plate.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and more specifically, to a method and system for identifying surface defects on a tail fin based on deep learning. Background Technology

[0002] As a critical structural component, the surface quality of a car's rear spoiler directly affects driving safety and performance. Therefore, the industry widely employs automated inspection systems based on deep learning. These systems use industrial cameras to acquire images of the spoiler's surface and utilize pre-trained deep learning methods to identify various defects such as scratches, dents, bubbles, and foreign objects. The accuracy and stability of these deep learning methods heavily rely on high-quality training sample sets, which are typically manually calibrated by experienced engineers over considerable time to ensure accurate selection and classification of defective areas.

[0003] When imaging these high-curvature, complex-geometric tail fin surfaces using existing fixed industrial cameras and lighting systems on production lines, severe imaging artifacts inevitably occur in areas with dramatic curvature changes. Specifically, because the illumination angle of a fixed light source is relatively constant, the reflection angles of light vary greatly across different areas of the high-curvature surface, potentially creating glaring highlights or shadows where light cannot reach, obscuring or obscuring the true surface texture and potential defects, or causing a loss of detail. Simultaneously, the fixed viewing angle of the industrial camera lens causes objects at the image edges to appear stretched or compressed due to perspective effects, distorting defect morphology—a phenomenon known as viewing angle distortion. These artifacts, including uneven lighting, highlight reflections, and viewing angle distortion, severely degrade the quality and interpretability of the original image.

[0004] Faced with images containing complex artifacts, even experienced engineers struggle to accurately and consistently identify and label them manually. For example, minute scratches located in highlight areas may be missed (under-labeling), while normal highlight areas that resemble defects in visual characteristics may be mislabeled (mis-labeling). Similarly, blurred details in shadow areas and changes in defect morphology caused by viewpoint distortion also make it difficult for labelers to accurately identify defects. This uncertainty is not random error but is strongly correlated with specific types and locations of imaging artifacts, forming a systematic data noise that directly contaminates the quality of the training sample set.

[0005] When a deep learning method is trained on a new sample set containing system noise, it not only learns the true features of defects during the learning process but also incorrectly associates imaging artifacts (such as highlights, shadow boundaries, or distorted shapes at specific angles) with defect features. This erroneous association causes the method's intrinsic weights to be "contaminated" by imaging artifacts, significantly reducing its generalization ability and ability to distinguish real defects, making it unable to reliably differentiate between real physical defects and visual artifacts introduced by imaging conditions. In actual production line inspection, this method exhibits high false negative and high false positive rates, severely impacting production efficiency and the reliability of product quality control. Summary of the Invention

[0006] This application discloses a deep learning-based method and system for identifying surface defects in tail fins, aiming to solve the problem of imaging artifacts caused by the complex geometric configuration of tail fins in the prior art, as well as the resulting technical difficulties of reduced training sample quality, generalization ability and recognition accuracy of deep learning models.

[0007] The technical solution of this application is as follows:

[0008] In a first aspect, this application discloses a deep learning-based method for identifying surface defects in a tail fin, specifically including the following steps:

[0009] A physical rendering engine is established based on the computer-aided design geometric model and material optical parameters of the tail fin. Based on the physical rendering engine, imaging conditions are simulated to generate synthetic training data, which includes: synthetic images of imaging artifacts, ideal defect images without artifacts, and artifact masks.

[0010] Based on synthetic training data, an artifact removal encoder and a defect recognition decoder are constructed in a deep learning network. The artifact removal encoder is used to extract artifact-independent feature representations from synthetic images, and the defect recognition decoder is used to identify defects based on feature representations.

[0011] Adversarial training is performed on the artifact removal encoder and defect identification decoder based on adversarial generative network to obtain the adversarially trained artifact removal encoder and defect identification decoder.

[0012] The image captured by the industrial camera is input into the adversarially trained artifact removal encoder, and the artifact removal encoder outputs an artifact-independent feature image based on the adversarially trained artifact removal encoder.

[0013] The feature image, which is independent of artifacts, is input into the defect recognition decoder after adversarial training, and the defect recognition result is output based on the defect recognition decoder after adversarial training.

[0014] Furthermore, the process of establishing a physically based rendering engine using the computer-aided design geometric model and material optical parameters based on the tail fin, and generating synthetic training data by simulating imaging conditions using the physically based rendering engine, specifically includes:

[0015] The computer-aided design geometric model of the tail fin is imported into the rendering software platform and converted into a virtual product structure under the rendering engine.

[0016] The material optical parameters corresponding to the virtual product structure are set according to the actual material and surface coating of the tail fin to form a virtual tail fin model;

[0017] In the rendering engine, the simulation imaging condition parameters of the virtual tail fin model are set to form a virtual environment scene. These simulation imaging condition parameters include: simulation lighting environment parameters and virtual camera parameters.

[0018] Generate various types of defect images on the surface of a virtual tail fin model based on a virtual environment scene;

[0019] Synthetic images of imaging artifacts, artifact-free ideal defect images, and artifact masks are generated based on various types of defect images.

[0020] Furthermore, in generating various types of defect images on the surface of the virtual tail fin model based on the virtual environment scene, specifically including:

[0021] In each preset virtual scene, a scratch effect is generated on the surface of the virtual tail fin model using a geometric deformation method;

[0022] The physical structure morphology for adjusting the scratch effect includes: scratch location, scratch size, scratch shape, and scratch depth;

[0023] The adjusted scratch effect is assigned corresponding optical properties, including the roughness inside the scratch and the degree of diffuse reflection of the scratch effect.

[0024] The rendering engine outputs the corresponding defect image based on optical properties.

[0025] Preferably, the artifact removal encoder and defect recognition decoder in the deep learning network constructed based on synthetic training data specifically include:

[0026] An artifact removal encoder and a defect identification decoder are constructed. The artifact removal encoder adopts the downsampling structure of U-Net, and the defect identification decoder adopts the upsampling structure of U-Net.

[0027] The synthetic training data is input into the artifact removal encoder for training and learning, and the artifact removal encoder obtains the artifact-independent feature representation of the artifact removal encoder.

[0028] The feature representations unrelated to artifacts are input into the defect recognition decoder for training and learning, thereby obtaining the defect recognition decoder's ability to identify and locate each real defect on the tail fin.

[0029] Furthermore, the adversarial training of the artifact removal encoder and defect identification decoder based on the above-mentioned adversarial generative network to obtain the adversarially trained artifact removal encoder and defect identification decoder specifically includes:

[0030] The PatchGAN discriminator receives the feature representation output by the artifact removal encoder and identifies the source of the feature representation based on the feature representation. The source is either an artifact-free ideal defect image generated by the rendering engine or a feature of the input image containing artifacts after processing by the artifact removal encoder.

[0031] Based on the results of feature representation recognition, the artifact removal encoder and defect recognition decoder are trained with adversarial loss, reconstruction loss and defect recognition loss to obtain the adversarially trained artifact removal encoder and defect recognition decoder.

[0032] Furthermore, the training of the artifact removal encoder and defect recognition decoder based on the feature representation recognition results includes the following:

[0033] The purity score of pixel regions in an ideal defect image is evaluated based on the results of feature representation recognition.

[0034] The purity score is used as a weight to weight the loss of the PatchGAN discriminator;

[0035] The PatchGAN discriminator, after weighting, is used to train the artifact removal encoder and the defect identification decoder with adversarial loss, reconstruction loss, and defect identification loss.

[0036] Furthermore, the evaluation of the purity score of pixel regions in an ideal defective image based on the results of feature representation recognition specifically includes:

[0037] Identify different types of subtle imaging artifacts in overlapping regions within the feature space output by the artifact removal encoder;

[0038] Based on the defect prediction results of overlapping areas by combining different types of minor imaging artifacts with the defect recognition decoder and the label information of overlapping areas in the ideal defect image, it is determined whether the minor imaging artifacts match the characteristics of real minor defects.

[0039] Based on the judgment results, the local feature deviations corresponding to each subtle imaging artifact are weighted and combined to form a purity score.

[0040] As a technical improvement, the identification of different types of subtle imaging artifacts in the overlapping region within the feature space output by the artifact removal encoder specifically includes:

[0041] A dynamic feature reference library is established in the feature space of the artifact removal encoder output. This dynamic feature reference library stores feature prototypes of known subtle imaging artifact types.

[0042] The sample image is input into the artifact removal encoder to obtain the feature representation of the sample image in the feature space. The sample image is generated by an industrial camera.

[0043] The feature representations of the sample images are matched with the feature prototypes in the dynamic feature reference library based on similarity.

[0044] Determine whether the similarity between the feature representation of the sample image and the existing feature prototype is lower than a preset threshold;

[0045] If the similarity is determined to be lower than the preset threshold, then the overlapping area is determined to contain subtle imaging artifacts.

[0046] Cluster analysis is performed on the feature representation of the variable fine imaging artifacts, and different types of fine imaging artifacts are obtained based on the cluster analysis.

[0047] Furthermore, the process of inputting the image captured by the industrial camera into the adversarially trained artifact removal encoder and outputting an artifact-independent feature image based on the adversarially trained artifact removal encoder specifically includes:

[0048] The images captured by the industrial camera are normalized to obtain normalized image data;

[0049] The artifact removal encoder trained by adversarial methods is used to remove artifact information from the normalized image data, resulting in a feature image that contains only surface physical features and defect information.

[0050] Secondly, this application also discloses a deep learning-based system for identifying surface defects in tail fins, specifically including:

[0051] The rendering engine module is used to establish a physical rendering engine based on the computer-aided design geometric model and material optical parameters of the tail fin, and to generate synthetic training data based on the simulation imaging conditions of the physical rendering engine. The synthetic training data includes: synthetic images of imaging artifacts, ideal defect images without artifacts, and artifact masks.

[0052] The deep learning module is used to construct an artifact removal encoder and a defect recognition decoder in the deep learning network based on synthetic training data. The artifact removal encoder is used to extract artifact-independent feature representations from synthetic images, and the defect recognition decoder is used to identify defects based on feature representations.

[0053] The adversarial training module is used to perform adversarial training on the artifact removal encoder and the defect identification decoder based on the adversarial generative network, so as to obtain the adversarially trained artifact removal encoder and defect identification decoder.

[0054] The feature image module is used to input the image captured by the industrial camera into the adversarially trained artifact removal encoder, and output an artifact-independent feature image based on the adversarially trained artifact removal encoder.

[0055] The defect identification module is used to input artifact-independent feature images into the adversarially trained defect identification decoder and output defect identification results based on the adversarially trained defect identification decoder.

[0056] This application discloses a deep learning-based method for identifying surface defects in tail fins. By establishing a physically based rendering engine, it can simulate complex imaging conditions based on the computer-aided design geometric model and material optical parameters of the tail fin, thereby generating synthetic images containing imaging artifacts, ideal defect images without artifacts, and synthetic training data with artifact masks. This method effectively solves the problems in existing technologies, such as severe uneven illumination, high light reflection, and viewing angle distortion artifacts generated when imaging tail fin surfaces with high curvature and complex geometry, which lead to decreased training sample quality, difficulties in manual calibration, and reduced generalization ability and recognition accuracy of deep learning models.

[0057] Specifically, this application utilizes synthetic training data to construct an artifact removal encoder and a defect recognition decoder in a deep learning network. The artifact removal encoder is specifically designed to extract artifact-independent feature representations from synthetic images, while the defect recognition decoder identifies defects based on these clean feature representations. Furthermore, by adversarially training the encoder and decoder using a generative adversarial network, the model learns how to effectively distinguish between real defect features and imaging artifacts. Thus, in practical applications, even when dealing with images captured by industrial cameras containing complex artifacts, the artifact removal encoder can output artifact-independent feature images, and the defect recognition decoder can ultimately output accurate defect recognition results.

[0058] Through the above technical solution, this application overcomes the limitation of existing deep learning models being susceptible to imaging artifacts, significantly improving the model's ability to distinguish and generalize real defects. In actual production line inspection, this method can effectively reduce the false negative rate and false alarm rate, thereby improving production efficiency and the reliability of product quality control, providing a more accurate and stable solution for the surface quality inspection of automotive rear spoilers. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of a deep learning-based tail fin surface defect identification method in an embodiment of the present invention.

[0061] Figure 2 This is a flowchart of the method for constructing an artifact removal encoder and a defect recognition decoder in a deep learning network based on synthetic training data in an embodiment of the present invention.

[0062] Figure 3 This is a flowchart of a method in this invention for performing adversarial training on an adversarial generative network to obtain an adversarially trained artifact removal encoder and a defect identification decoder.

[0063] Figure 4 This is a schematic diagram of the structure of the deep learning-based tail fin surface defect recognition system in an embodiment of the present invention. Detailed Implementation

[0064] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0065] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0066] Specifically, Figure 1 The flowchart of the deep learning-based tail fin surface defect identification method in an embodiment of the present invention is shown, including the following steps:

[0067] S101. Based on the computer-aided design geometric model and material optical parameters of the tail fin, a physical rendering engine is established, and synthetic training data is generated by simulating imaging conditions based on the physical rendering engine.

[0068] It should be noted that the synthetic training data includes: synthetic images of imaging artifacts, ideal defect images without artifacts, and artifact masks. Here, the physically based rendering engine refers to software or a system that renders images based on physical optics principles. It can simulate the reflection, refraction, and scattering of light on different material surfaces, thereby generating highly realistic images. In this application, the physically based rendering engine is used to simulate images of the tail fin under various imaging conditions, including defects and artifacts. The synthetic training data here refers to artificial image data generated by the physically based rendering engine, which includes synthetic images of imaging artifacts, ideal defect images without artifacts, and artifact masks. This data is used to train a deep learning network to distinguish between real defects and imaging artifacts.

[0069] First, regarding the generation of synthetic training data, this application establishes a physically based rendering engine based on the computer-aided design geometric model and material optical parameters of the tail fin, and generates synthetic training data based on the simulation of imaging conditions using this physically based rendering engine. This synthetic training data includes synthetic images of imaging artifacts, artifact-free ideal defect images, and artifact masks. Specifically, this can be achieved in the following ways:

[0070] One approach involves importing the computer-aided design geometric model of the tail fin into a professional rendering software platform, such as Blender, Maya, or 3ds Max, and converting it into a virtual product structure within a rendering engine. Subsequently, based on the actual material (e.g., aluminum alloy, composite material) and surface coating (e.g., paint, anti-corrosion coating) of the tail fin, the corresponding material optical parameters of the virtual product structure are set, such as reflectivity, refractive index, and roughness, thus forming a highly realistic virtual tail fin model. Next, the simulation imaging condition parameters of the virtual tail fin model are set in the rendering engine, including simulation lighting environment parameters (e.g., light source type, position, intensity, color) and virtual camera parameters (e.g., camera position, focal length, aperture, exposure time), forming a virtual environmental scene. Finally, based on the virtual environmental scene, various types of defect images are generated on the surface of the virtual tail fin model. For example, scratches, dents, and bubbles are simulated through geometric deformation and texture overlay. Based on these defect images, a composite image of imaging artifacts, an ideal defect image without artifacts, and an artifact mask are output.

[0071] Another approach is to create a geometric model of the tail fin using parametric modeling tools (such as SolidWorks and CATIA) and export it to a format recognizable by the rendering engine (such as OBJ or FBX). Then, the model is loaded into the rendering engine via a programming interface (API), and material optical parameters are set manually or via script to match the visual characteristics of a real tail fin. When simulating imaging conditions, multiple lighting schemes and camera angles can be preset, such as simulating LED light source illumination at different angles and simulating camera shooting at different distances and tilt angles, to cover all imaging artifacts that may occur in actual industrial inspection. When generating defect images on the surface of the virtual tail fin model, procedural texture generation technology can be used to automatically generate defect textures based on preset defect types and parameters (such as scratch length, width, depth, pit diameter, and depth), and map them onto the virtual model surface. Finally, the rendering engine will generate a synthetic image containing artifacts, an ideal defect image without artifacts, and an accurate artifact mask based on these settings for subsequent deep learning training.

[0072] S102. Construct an artifact removal encoder and a defect recognition decoder in a deep learning network based on the synthetic training data;

[0073] It should be noted that the artifact removal encoder is used to extract artifact-independent feature representations from the synthesized image, and the defect identification decoder is used to identify defects based on these feature representations. The artifact removal encoder is part of a deep learning network, and its main function is to extract artifact-independent feature representations from images containing imaging artifacts, i.e., stripping away artifact information while preserving the essential features of the defects. The defect identification decoder is another part of the deep learning network; it receives the artifact-independent feature representations output by the artifact removal encoder and identifies and locates defects on the tail fin surface based on these features.

[0074] Secondly, regarding the construction of the deep learning network, this application constructs an artifact removal encoder and a defect recognition decoder in the deep learning network based on the synthesized training data. The artifact removal encoder is used to extract artifact-independent feature representations from the synthesized image, and the defect recognition decoder is used to identify defects based on these feature representations. Specifically, this can be implemented in the following way:

[0075] One implementation involves constructing a deep learning network where the artifact removal encoder uses a U-Net downsampling structure, and the defect recognition decoder uses a U-Net upsampling structure. The artifact removal encoder downsamples the input image through a series of convolutional and pooling layers, progressively extracting abstract features while filtering out artifact information. The defect recognition decoder upsamples the feature representation output by the encoder through a series of deconvolutional and upsampling layers, progressively recovering the spatial information of the image and ultimately outputting the defect recognition result. The synthetic training data is input into the artifact removal encoder for training, enabling it to accurately extract artifact-independent feature representations from synthetic images containing artifacts. Subsequently, these artifact-independent feature representations are input into the defect recognition decoder for training, enabling it to accurately identify and locate real defects in the tail fin based on these clean feature representations.

[0076] Another approach is to use pre-trained backbone networks such as ResNet and DenseNet as the downsampling structure for the artifact removal encoder, leveraging their powerful feature extraction capabilities. The defect recognition decoder can employ structures like FPN (Feature Pyramid Network) or PANet (Path Aggregation Network) to better integrate multi-scale features and improve the recognition accuracy for defects of different sizes. During training, a multi-task learning approach can be used to simultaneously optimize both artifact removal and defect recognition tasks. For example, the artifact removal encoder can be trained as an autoencoder aiming to reconstruct an artifact-free, ideal defect image, while the defect recognition decoder can be trained as a segmentation network aiming to output a pixel-level mask of the defect. In this way, the artifact removal encoder can better learn artifact-independent feature representations, providing a cleaner input for the defect recognition decoder.

[0077] S103. Perform adversarial training on the artifact removal encoder and defect identification decoder based on the adversarial generative network to obtain the adversarially trained artifact removal encoder and defect identification decoder.

[0078] Generative Adversarial Networks (GANs) are deep learning models that typically consist of a generator and a discriminator. In this application, they are used to adversarially train an artifact removal encoder and a defect detection decoder to improve the model's ability to detect artifacts in complex environments.

[0079] Furthermore, regarding adversarial training, this application uses an adversarial generative network to perform adversarial training on the artifact removal encoder and defect identification decoder, resulting in adversarially trained artifact removal encoder and defect identification decoder. Specifically, this can be achieved in the following way:

[0080] One implementation involves introducing a PatchGAN discriminator. This discriminator receives the feature representation output by the artifact removal encoder and identifies the source of the feature representation based on it. The source can be features from an artifact-free, ideal defect image generated by the rendering engine, or features from an input image containing artifacts processed by the artifact removal encoder. The discriminator aims to distinguish between these two sources, while the artifact removal encoder aims to generate features that can "fool" the discriminator, making it unable to distinguish the source of the features. Based on the result of the feature representation identification, the artifact removal encoder and the defect identification decoder are trained with adversarial loss, reconstruction loss, and defect identification loss. The adversarial loss prompts the artifact removal encoder to generate more realistic artifact-independent features, the reconstruction loss ensures that the artifact removal encoder can effectively remove artifacts and preserve defect information, and the defect identification loss guides the defect identification decoder to accurately identify defects. Through this adversarial training, the artifact removal encoder and the defect identification decoder can mutually promote each other and improve performance.

[0081] Another approach is to use other types of discriminators besides PatchGAN, such as Spectral Normalization GAN or WGAN-GP (Wasserstein GAN with Gradient Penalty), to improve training stability and the quality of generated features. In the design of the loss function, perceptual loss can be introduced. By comparing the features output by the artifact removal encoder with the features of the ideal defect image at a high-level semantic level, the artifact removal effect can be further improved. Furthermore, an adaptive weight adjustment strategy can be used to dynamically adjust the weights of adversarial loss, reconstruction loss, and defect recognition loss based on changes in different loss terms during training, thereby optimizing the overall training process. For example, in the early stages of training, the weight of reconstruction loss can be increased to ensure that the artifact removal encoder can effectively remove artifacts; in the later stages of training, the weights of adversarial loss and defect recognition loss can be increased to improve the model's generalization ability and recognition accuracy.

[0082] S104. Input the image captured by the industrial camera into the adversarially trained artifact removal encoder, and output an artifact-independent feature image based on the adversarially trained artifact removal encoder.

[0083] S105. Input the artifact-independent feature image into the adversarially trained defect recognition decoder, and output the defect recognition result based on the adversarially trained defect recognition decoder.

[0084] Finally, regarding the output of the defect identification result, this application inputs the image captured by the industrial camera into the adversarially trained artifact removal encoder, and outputs an artifact-independent feature image based on the adversarially trained artifact removal encoder. Subsequently, the artifact-independent feature image is input into the adversarially trained defect identification decoder, and the defect identification result is output based on the adversarially trained defect identification decoder. Specifically, this can be implemented in the following way:

[0085] One implementation involves normalizing the images captured by the industrial camera, for example, scaling the pixel values ​​to between 0 and 1 to obtain normalized image data. This processing helps eliminate differences in brightness, contrast, etc., between different images, making the model more adaptable to the input images. Then, the artifact removal encoder trained adversarially removes artifact information from the normalized image data, obtaining a feature image containing only surface physical features and defect information. This feature image is the output of the artifact removal encoder after processing the original image, and it has removed the interference of imaging artifacts to the greatest extent. Finally, the artifact-independent feature image is input into the defect recognition decoder trained adversarially, and the defect recognition result, such as the type, location, and size of the defect, is output based on the adversarially trained defect recognition decoder.

[0086] Another approach is to perform preprocessing operations, such as image denoising, sharpening, or color correction, before inputting the images acquired by the industrial camera into the artifact removal encoder to further improve image quality. The feature image output by the artifact removal encoder can be a multi-channel feature map, where each channel represents different semantic information. The defect recognition decoder can employ post-processing modules, such as Conditional Random Fields (CRF) or morphological operations, to optimize the defect recognition results, obtaining more accurate defect boundaries and smoother segmentation results. Furthermore, the defect recognition results can be presented in various forms, such as overlaying defect masks on the original image, generating defect reports, or displaying the defect location and type in real time through a visual interface, to facilitate subsequent processing by operators.

[0087] This application introduces a physical rendering engine to generate high-quality synthetic training data and utilizes a generative adversarial network to co-train the artifact removal encoder and the defect recognition decoder, thereby effectively solving the interference of complex imaging artifacts on the accuracy of defect recognition and significantly improving the robustness and accuracy of defect recognition on the tail fin surface.

[0088] The deep learning-based method for identifying surface defects on tail fins proposed in this application effectively solves the problem of complex imaging artifacts interfering with the accuracy of defect identification by introducing a physical rendering engine to generate high-quality synthetic training data and using a generative adversarial network to co-train the artifact removal encoder and the defect identification decoder.

[0089] First, this application innovatively utilizes a physically based rendering engine to generate large-scale, high-quality synthetic training data. By establishing a physically based rendering engine based on a computer-aided design geometric model of the tail fin and material optical parameters, and simulating various imaging conditions, it can accurately generate synthetic images containing imaging artifacts, artifact-free ideal defect images, and artifact masks. This method overcomes the shortcomings of traditional manual data annotation, such as high cost, low efficiency, and susceptibility to artifact interference, providing a clean and diverse data source for training deep learning models. Compared to traditional methods that rely on limited and artifact-contaminated real images for manual annotation, this application can generate an unlimited amount of training data with precise control over artifact types and defect features, fundamentally solving the data quality problem.

[0090] Secondly, this application constructs an artifact removal encoder and a defect recognition decoder, and employs an adversarial generative network for co-training. The artifact removal encoder is specifically designed to extract artifact-independent feature representations from synthetic images containing artifacts, while the defect recognition decoder identifies defects based on these clean feature representations. Through adversarial training, the artifact removal encoder is trained to generate artifact-independent features sufficient to "fool" the discriminator, while the discriminator prompts the encoder to continuously improve its artifact removal capabilities. This adversarial mechanism enables the artifact removal encoder to more effectively remove imaging artifacts, thereby providing the defect recognition decoder with cleaner and more discriminative features. Compared to traditional methods that directly train the defect recognition model on images containing artifacts, this application significantly improves the robustness and accuracy of defect recognition by pre-purifying features through the artifact removal encoder, avoiding the problem of the model misclassifying artifacts as defects or masking real defects.

[0091] Therefore, the method proposed in this application can transform images containing artifacts captured by industrial cameras into artifact-independent feature images through an adversarially trained artifact removal encoder, and then output accurate defect identification results by a defect identification decoder. This end-to-end solution not only improves the accuracy and stability of surface defect identification for rear wing panels, but also reduces reliance on manual annotation, enhances the efficiency and reliability of automated inspection, and provides strong technical support for quality control in the automotive manufacturing industry.

[0092] The steps in this application, which involve establishing a physically based rendering engine using a computer-aided design geometric model and material optical parameters of a tail fin, and generating synthetic training data based on simulated imaging conditions using the physically based rendering engine, include: importing the computer-aided design geometric model of the tail fin into a rendering software platform and converting it into a virtual product structure under the rendering engine; setting the material optical parameters corresponding to the virtual product structure according to the actual material and surface coating of the tail fin to form a virtual tail fin model; setting the simulated imaging condition parameters of the virtual tail fin model in the rendering engine and forming a virtual environmental scene, wherein the simulated imaging condition parameters include: simulated lighting environment parameters and virtual camera parameters; generating various types of defect images on the surface of the virtual tail fin model based on the virtual environmental scene; and outputting a synthetic image of imaging artifacts, an ideal defect image without artifacts, and an artifact mask based on the various types of defect images.

[0093] Specifically, importing the computer-aided design geometric model of the tail fin into the rendering software platform refers to loading the 3D geometric model file of the tail fin (in formats such as STEP, IGES, and STL) created by computer-aided design (CAD) software (such as SolidWorks, CATIA, Pro / E, etc.) into professional rendering software (such as Blender, Unity 3D, Unreal Engine, V-Ray, etc.). The purpose of this step is to provide an accurate geometric foundation for subsequent virtual scene construction. Subsequently, the geometric model is converted into a virtual product structure that can be recognized and processed within the rendering engine for ray tracing and rendering calculations in the virtual environment.

[0094] Specifically, setting the material optical parameters corresponding to the virtual product structure based on the actual material and surface coating of the tail fin refers to assigning corresponding optical properties to its virtual representation in the rendering engine based on the actual physical properties of the tail fin, such as metal, composite materials, and paint coatings. These optical parameters may include, but are not limited to, diffuse reflection coefficient, specular reflection coefficient, refractive index, roughness, and texture mapping. The purpose is to ensure that the virtual tail fin model can realistically simulate the interaction between light and actual materials during rendering, thereby forming a highly realistic virtual tail fin model.

[0095] In practical applications, setting the simulated imaging condition parameters of the virtual tail fin model in the rendering engine to form a virtual environmental scene refers to configuring various parameters in the rendering software to simulate the shooting environment of a real industrial camera. These simulated imaging condition parameters specifically include simulated lighting environment parameters and virtual camera parameters. Simulated lighting environment parameters can include the type of light source (e.g., point light, parallel light, ambient light), position, intensity, color, and environment textures (e.g., HDRI) to simulate the lighting conditions in the actual inspection environment. Virtual camera parameters can include the camera's position, orientation, focal length, aperture, exposure time, sensor size, noise model, etc., to simulate the imaging characteristics of an industrial camera. The purpose is to construct a virtual environment highly similar to the actual inspection scene, laying the foundation for generating realistic synthetic images.

[0096] Furthermore, generating various types of defect images on the surface of the virtual tail fin model based on the aforementioned virtual environment refers to simulating various common surface defects on the virtual tail fin model in the constructed virtual environment, either programmatically or manually, such as scratches, dents, cracks, corrosion spots, and coating peeling. The geometric morphology and optical properties of these defects can be precisely controlled and adjusted to cover the types and variations of defects that may be encountered in actual production.

[0097] Therefore, outputting synthetic images with imaging artifacts, ideal defect images without artifacts, and artifact masks based on various types of defect images refers to generating synthetic images containing imaging artifacts during the rendering process, according to set simulated imaging conditions. Simultaneously, by controlling rendering parameters, ideal defect images without any artifacts can also be generated, displaying only the true shape of the defect. Furthermore, artifact masks are generated to accurately mark the regions with artifacts in the synthetic image. These different types of images collectively constitute the synthetic training data used for training deep learning networks.

[0098] This application's solution ensures the authenticity and diversity of the generated data by breaking down the complex process of generating synthetic training data into a series of controllable and precise sub-steps. First, by importing precise computer-aided design geometric models and converting them into virtual product structures under the rendering engine, a high-fidelity geometric foundation is provided for subsequent virtual scene construction. Second, by setting optical parameters consistent with actual materials, the virtual tail fin model can realistically simulate the interaction between light and materials, thus ensuring the physical accuracy of the rendered image. Third, by finely setting simulated imaging condition parameters, including simulated lighting environment parameters and virtual camera parameters, the imaging characteristics of actual industrial cameras under different lighting and shooting conditions can be highly reproduced, effectively simulating imaging artifacts in the real world. Finally, various types of defect images are generated in the virtual scene, and synthetic images containing imaging artifacts, ideal defect images without artifacts, and artifact masks are output. This provides rich and precisely labeled training samples for the deep learning network, enabling the network to learn to distinguish between real defects and imaging artifacts.

[0099] Specifically, the above-mentioned generation of various types of defect images on the surface of the virtual tail fin model based on the virtual environment scene includes: generating scratch effects on the surface of the virtual tail fin model through geometric deformation in each preset virtual scene; adjusting the physical structure of the scratch effect, the physical structure including: scratch location, scratch size, scratch shape, and scratch depth; assigning corresponding optical properties to the adjusted scratch effect, the optical properties including: roughness inside the scratch and the degree of diffuse reflection of the scratch effect; and outputting the corresponding defect image based on the rendering engine according to the optical properties.

[0100] The geometric deformation method refers to using computer graphics technology to locally modify the geometry of a virtual tail fin model to simulate the physical form of realistic scratches. This can include vertex displacement, normal perturbation, or bump mapping of the model mesh. The aim is to create a realistic three-dimensional scratch structure in a virtual environment.

[0101] Furthermore, the physical structure can be adjusted; for example, the scratch location can be precisely set in a specific area of ​​the virtual tail fin model, the scratch size can be adjusted from the micrometer to the millimeter level, the scratch shape can simulate straight lines, curves, or irregular shapes, and the scratch depth affects its shadow and reflection effects under illumination. This fine control of parameters aims to simulate various scratch types and variations that may occur in actual production.

[0102] Furthermore, the imparting of these optical properties specifically involves the roughness within the scratch simulating the microstructure of the scratch surface material, thus affecting the light scattering characteristics; the degree of diffuse reflection from the scratch effect determines the intensity of light scattering by the scratched area. By precisely setting these optical parameters, it is possible to ensure that the rendered defect images possess a high degree of realism and diversity under different lighting conditions. Therefore, based on the set geometric deformations and optical properties, the rendering engine can accurately simulate the interaction between light and the scratches on the surface of the virtual tail fin model, thereby generating realistic defect images.

[0103] This application's solution generates scratch effects with controllable physical structure and optical properties on the surface of a virtual tail fin model, systematically simulating the diversity of surface defects on real-world tail fins. Specifically, geometric deformation methods provide the foundation for creating various scratch shapes and sizes; adjustments to the scratch location, size, shape, and depth allow the generated defect images to cover a wide range of variations of actual defects; and assigning optical properties such as internal roughness and diffuse reflection to the scratches ensures that the visual appearance of the defects under different lighting and viewing angles is highly consistent with reality. This refined defect generation mechanism enables synthetic training data to more comprehensively and accurately reflect the characteristics of real defects, thus providing high-quality input for subsequent deep learning network training.

[0104] Specifically, Figure 2 The flowchart illustrates a method for constructing an artifact removal encoder and a defect recognition decoder in a deep learning network based on synthetic training data, as shown in this embodiment of the invention. The method specifically includes:

[0105] S201. Construct an artifact removal encoder and a defect recognition decoder;

[0106] It should be noted that the artifact removal encoder adopts the U-Net downsampling structure; the defect recognition decoder adopts the U-Net upsampling structure.

[0107] Specifically, the artifact removal encoder is designed with a U-Net downsampling structure. Its main function is to progressively extract high-level semantic features from the input synthetic image while effectively filtering out the influence of imaging artifacts, thereby obtaining artifact-independent feature representations. The downsampling structure typically includes convolutional and pooling layers, achieving feature abstraction and spatial dimensionality reduction through multi-layer stacking. The defect detection decoder uses a U-Net upsampling structure. Its role is to receive the artifact-independent feature representations output by the artifact removal encoder and progressively restore them to the spatial resolution of the original image for accurate defect identification and localization. The upsampling structure typically includes deconvolutional or upsampling layers, combined with skip connections to fuse features extracted by the encoder at different levels, thus preserving rich spatial detail information.

[0108] S202. Input the synthetic training data into the artifact removal encoder for training and learning, and obtain the artifact removal encoder’s feature representation that is independent of artifacts.

[0109] During training, synthetic training data is first input into the artifact removal encoder. Through supervised learning, the artifact removal encoder is trained to learn how to extract clean feature representations from synthetic images containing imaging artifacts, containing only the physical features and defect information of the tail fin surface. The goal of this stage is to enable the encoder to effectively remove artifact information from image features. Subsequently, the artifact-independent feature representations obtained by the artifact removal encoder are input into the defect recognition decoder. The defect recognition decoder is trained on this basis, aiming to accurately identify and locate various real defects on the tail fin surface based on these clean feature representations. This phased training strategy ensures that the encoder focuses on artifact removal, while the decoder focuses on defect recognition, thereby optimizing the overall network performance.

[0110] S203. Input the artifact-independent feature representation into the defect recognition decoder for training and learning, and obtain the defect recognition decoder's function of recognizing and locating each real defect of the tail fin.

[0111] The proposed solution employs a U-Net downsampling structure for the artifact removal encoder, enabling it to effectively capture image features at multiple scales and progressively compress spatial information, thereby effectively suppressing imaging artifacts during feature extraction. Simultaneously, the defect recognition decoder utilizes a U-Net upsampling structure and leverages skip connections to fuse feature information from different levels of the encoder. This allows the decoder to fully utilize the rich contextual information and fine spatial details extracted by the encoder when restoring spatial resolution for defect recognition, thus improving the accuracy and localization precision of defect recognition. By inputting synthetic training data separately into both the artifact removal encoder and the defect recognition decoder for training, the encoder learns robust artifact-independent feature representations, while the decoder performs efficient defect recognition based on these clean features, thus synergistically improving the overall system performance.

[0112] Specifically, Figure 3 This invention illustrates a method for adversarial training of an artifact removal encoder and a defect identification decoder based on an adversarial generative network, resulting in the adversarially trained artifact removal encoder and defect identification decoder. The method specifically includes:

[0113] S301. The PatchGAN discriminator receives the feature representation output by the artifact removal encoder and identifies the source of the feature representation based on the feature representation.

[0114] It should be noted that the source is an ideal defect image without artifacts generated by the rendering engine or a feature of an input image containing artifacts after processing by an artifact removal encoder.

[0115] S302. Based on the result of the feature representation recognition, the artifact removal encoder and the defect recognition decoder are trained with adversarial loss, reconstruction loss and defect recognition loss to obtain the adversarially trained artifact removal encoder and defect recognition decoder.

[0116] Specifically, the PatchGAN discriminator is a discriminator specifically designed for local region discrimination. It can determine the authenticity of local regions in an image or feature map, rather than judging the entire image. Here, the PatchGAN discriminator is configured to receive feature representations output by the artifact removal encoder. These feature representations are features extracted by the artifact removal encoder from a synthetic image containing imaging artifacts, and are theoretically artifact-free. The core function of the PatchGAN discriminator is to identify the source of these feature representations. Specifically, it needs to determine whether these feature representations originate from an artifact-free, ideally flawed image directly generated by the rendering engine (i.e., the true data distribution) or from features of an input image containing artifacts processed by the artifact removal encoder (i.e., the generated data distribution). In this way, the PatchGAN discriminator can effectively supervise the artifact removal encoder, prompting it to generate feature representations that are closer to the true artifact-free feature distribution.

[0117] Based on the PatchGAN discriminator's identification of feature representation sources, the artifact removal encoder and defect identification decoder are trained using multi-objective losses. These losses include adversarial loss, reconstruction loss, and defect identification loss. The adversarial loss updates the artifact removal encoder and defect identification decoder based on the PatchGAN discriminator's judgment, prompting the artifact removal encoder to generate more realistic artifact-independent features and enabling the defect identification decoder to better utilize these features. The reconstruction loss aims to ensure that the artifact removal encoder retains key defect-related information in the original image while removing artifacts, for example, by comparing the feature representation output by the artifact removal encoder with the feature representation of the ideal defect image. The defect identification loss directly affects the output of the defect identification decoder, optimizing the decoder's ability to locate and classify defects by comparing it with the true defect labels. By combining these three losses, the model's performance can be comprehensively improved, achieving a high level in both artifact removal and defect identification.

[0118] This application's scheme introduces a PatchGAN discriminator, enabling the artifact removal encoder to not only learn to remove artifacts during training but also to generate feature representations highly consistent with the distribution of artifact-free features. The PatchGAN discriminator's ability to distinguish local regions allows it to capture the artifact removal encoder's shortcomings in handling subtle artifacts, thus forcing the encoder to generate cleaner, more realistic features. Simultaneously, by combining adversarial loss, reconstruction loss, and defect recognition loss during training, a multi-dimensional optimization objective is formed. The adversarial loss ensures the realism of the feature representation, the reconstruction loss guarantees the complete preservation of the original defect information, and the defect recognition loss directly optimizes the final performance of the defect recognition decoder. This comprehensive training strategy allows the artifact removal encoder to more thoroughly remove artifact information and provides the defect recognition decoder with high-quality, easily identifiable features, effectively solving the problems of impure feature representations and limited recognition accuracy that may exist in traditional adversarial training.

[0119] In some preferred embodiments, it is assumed that during the identification of defects on the tail fin surface, images captured by industrial cameras often contain subtle imaging artifacts caused by uneven lighting, reflections, or dust. These artifacts may be visually similar to real minute defects, making them difficult for traditional deep learning models to distinguish. The solution in this application is implemented in the following way:

[0120] First, the artifact removal encoder receives these images containing artifacts and attempts to extract artifact-independent feature representations. Simultaneously, the rendering engine generates a batch of ideal, artifact-free defective images, whose feature representations are treated as "real" features.

[0121] Next, the PatchGAN discriminator is trained to distinguish between the feature representation output by the artifact removal encoder and the feature representation of the ideal defective image generated by the rendering engine. If the feature representation output by the artifact removal encoder still contains artifact information, or differs significantly from the ideal features, the PatchGAN discriminator can identify its "fake" nature and give a lower discrimination score.

[0122] Based on the discrimination results of the PatchGAN discriminator, the artifact removal encoder and defect recognition decoder undergo iterative training. The adversarial loss prompts the artifact removal encoder to generate features that are more difficult for the discriminator to distinguish, i.e., features closer to the ideal artifact-free state. The reconstruction loss ensures that the artifact removal encoder removes artifacts without losing crucial information such as the shape and size of defects in the original image. The defect recognition loss directly optimizes the defect recognition decoder, enabling it to accurately locate and classify defects based on these clean features.

[0123] For example, in the early stages of training, the feature representation output by the artifact removal encoder may still contain traces of light spot artifacts. The PatchGAN discriminator identifies these traces and feeds them back to the encoder. The encoder then adjusts its internal parameters to further learn how to remove these light spot artifacts. After multiple rounds of adversarial training, the artifact removal encoder can eventually output highly pure feature representations that contain only the physical texture and defect information of the tail fin surface, eliminating interference from various imaging artifacts. The defect recognition decoder can then accurately identify various defects such as scratches and dents based on these high-quality features, even if these defects are obscured by faint artifacts.

[0124] In response, this application further proposes the following steps for training the artifact removal encoder and the defect identification decoder based on the feature representation recognition results: evaluating the purity score of pixel regions in an ideal defect image based on the feature representation recognition results; weighting the loss of the PatchGAN discriminator with the purity score as a weight; and training the artifact removal encoder and the defect identification decoder with adversarial loss, reconstruction loss, and defect identification loss based on the weighted PatchGAN discriminator.

[0125] Specifically, the purity score can be understood as the reliability or quality of a specific pixel region in an ideal defect image in terms of its absence of artifacts. This score aims to quantify the presence of subtle imaging artifacts or uncertainties in an image region that may interfere with defect recognition. For example, a region with a high purity score indicates that its feature representation is closer to the true defect or defect-free state and is minimally affected by artifacts; while a region with a low purity score may suggest the presence of subtle artifacts that are difficult to completely remove, or that its feature representation has low distinguishability from the true defect. Using the purity score as a weight to weight the loss of the PatchGAN discriminator means that when calculating the loss of the PatchGAN discriminator, instead of simply applying the same loss contribution to all pixel regions or feature blocks, the proportion of each region in the total loss is adjusted according to its purity score. For example, regions with high purity scores can be given greater weight, allowing them to have a greater impact on the discriminator and generator (i.e., the artifact removal encoder and the defect recognition decoder) during training, prompting the network to learn the features of these high-quality regions more accurately. Conversely, regions with lower purity scores can be assigned smaller weights to reduce the negative impact of potential noise or uncertainty on training. In practical applications, the artifact removal encoder and defect detection decoder are trained using adversarial loss, reconstruction loss, and defect detection loss based on the weighted PatchGAN discriminator. This means that throughout the adversarial training loop, the discriminator's feedback signal (i.e., the loss gradient) is adjusted according to the purity of the pixel region. This weighting mechanism allows the training process to more intelligently focus on high-quality regions in the image that are crucial for defect detection, thereby guiding the artifact removal encoder to better remove artifacts and enabling the defect detection decoder to more accurately identify and locate defects.

[0126] This application's solution effectively addresses the problem of treating all image regions equally in traditional adversarial training by introducing a purity score for pixel regions and applying it to the loss weighting of the PatchGAN discriminator. Specifically, when the discriminator receives the feature representations output by the artifact removal encoder, it not only determines the source of these feature representations (whether they are features from an ideal defect image or features from an image processed by the encoder) but also adjusts its loss calculation based on the pre-evaluated purity score. The loss for high-purity regions is amplified, prompting the artifact removal encoder to work harder to generate artifact-free feature representations highly consistent with the ideal defect image, enabling the defect recognition decoder to learn more accurate defect patterns from these high-quality features. Simultaneously, the loss weight for low-purity regions is reduced, preventing the network from over-optimizing in potentially blurry or uncertain regions, thereby improving training stability and generalization ability. Thus, the entire training process can more finely guide the network's learning, enabling it to more effectively extract and utilize features related to real defects while removing artifacts.

[0127] In some preferred embodiments, a specific example is given below. Assume that during training, the rendering engine generates a batch of ideal defect images. For one of these ideal defect images, the purity score of different pixel regions in the image is evaluated by analyzing its matching degree with the real defect labels, or by using preset rules (e.g., deviation analysis based on the geometric model and the rendering result). For example, a clear scratch defect in the center region of the image might have a purity score of 0.95 (close to 1, indicating very pure), while the purity score of image edges or background regions, due to possible rendering errors or textures unrelated to the defect, might be evaluated as 0.70. When the feature representation processed by the artifact removal encoder is input into the PatchGAN discriminator, the discriminator calculates the loss based on these purity scores. Specifically, for a scratch region with a purity score of 0.95, the weights for the adversarial loss, reconstruction loss, and defect recognition loss will be 0.95; while for a background region with a purity score of 0.70, the loss weight will be 0.70. This weighting mechanism causes the network to prioritize feature learning in high-purity regions during training. This prompts the artifact removal encoder to remove artifacts more accurately while preserving defect features, and guides the defect recognition decoder to more accurately identify defects in these high-purity regions. In this way, the network can more effectively learn the essential features of real defects and reduce overfitting to potentially noisy or uncertain regions, ultimately improving the overall defect recognition performance.

[0128] In response, this application further proposes that the above-mentioned evaluation of the purity score of pixel regions in an ideal defect image based on the feature representation recognition results includes: identifying different types of subtle imaging artifacts in the feature space output by the artifact removal encoder; determining whether the subtle imaging artifacts match the features of real micro-defects based on the defect prediction results of the defect recognition decoder and the label information of the overlapping regions in the ideal defect image, according to the different types of subtle imaging artifacts; and weighting and combining the local feature deviations corresponding to each subtle imaging artifact according to the determination results to form a purity score.

[0129] Specifically, identifying different types of subtle imaging artifacts within the overlapping region of the feature space output by the artifact removal encoder refers to distinguishing and locating minute interferences caused by imaging conditions that may overlap with the defect region by analyzing local feature patterns in the image feature representation processed by the artifact removal encoder. These subtle imaging artifacts may include, but are not limited to, uneven illumination, shadows, reflections, and texture noise, and they exhibit specific patterns in the feature space. Identifying these artifacts can be achieved using methods such as cluster analysis, anomaly detection, or pre-trained classifiers.

[0130] In this process, based on different types of subtle imaging artifacts, the defect prediction results of the defect identification decoder for the overlapping region, and the label information of the overlapping region in the ideal defect image, it is determined whether the subtle imaging artifacts match the characteristics of real micro-defects. This can be understood as a multi-dimensional verification of the identified subtle imaging artifacts. The defect identification decoder makes a preliminary defect prediction for the overlapping region based on its learned defect features, providing a preliminary judgment on whether the region is a defect. Simultaneously, the label information of the overlapping region in the ideal defect image provides the true defect state of the region (i.e., the ground truth). By comparing the artifact features, the decoder prediction results, and the true labels, it is possible to more accurately determine whether a region identified as an artifact is indeed a pure artifact, or whether it contains real micro-defect components, thereby avoiding misclassifying artifacts as defects or real defects as artifacts.

[0131] In practical applications, a purity score is formed by weighted combining the local feature deviations corresponding to each type of subtle imaging artifact based on the judgment results. This means that after determining the true nature of the subtle imaging artifacts, the degree of deviation of each type of artifact from the ideal artifact-free state in the feature space is calculated. For example, if an artifact is judged as a pure artifact and does not contain real defects, its deviation value may be high; if an artifact is judged to match the characteristics of a real defect, its deviation value may be low. These deviation values ​​can be weighted according to their interference with defect identification or their salience in the image, and finally combined into a comprehensive purity score. This purity score can more precisely quantify the "purity" of an image region, that is, the degree to which it is free from artifact interference, providing a more accurate basis for subsequent loss weighting.

[0132] This application's scheme precisely identifies and distinguishes different types of subtle imaging artifacts in the feature space output by the artifact removal encoder, and performs multiple verifications by combining the prediction results of the defect recognition decoder with the real label information of the ideal defect image, thereby more accurately determining the nature of these subtle imaging artifacts. Specifically, by cross-comparing artifact features with decoder predictions and real labels, it effectively avoids misclassifying artifacts similar to real defects as defects, or misclassifying real minor defects as artifacts. Therefore, for each subtle imaging artifact, its local feature deviation from the ideal state can be calculated and weighted to form a more refined and accurate purity score. This purity score more realistically reflects the purity of the image region, providing a more reliable basis for the loss weighting of the PatchGAN discriminator, thus guiding the artifact removal encoder and defect recognition decoder to more effectively learn to distinguish between artifacts and real defects during adversarial training, improving the model's robustness.

[0133] In some preferred embodiments, it is assumed that a subtle imaging artifact resembling a small scratch is identified in the feature space output by the artifact removal encoder within an overlapping region of the tail fin surface image. In this case, the defect identification decoder's prediction for this overlapping region indicates a low-confidence defect. Simultaneously, the label information of the ideal defect image indicates that the region is actually a very small real scratch. In this situation, the proposed solution determines that the subtle imaging artifact matches the features of a real small defect. Further, if another artifact caused by uneven illumination is identified, and the defect identification decoder predicts no defect, and the ideal defect image label also indicates no defect, then this artifact is determined not to match a real defect. Based on these determinations, a lower weight is assigned to the local feature deviation corresponding to the scratch-like artifact (actually a defect), while a higher weight is assigned to the local feature deviation corresponding to the uneven illumination artifact (pure artifact). This weighted combination forms a more accurate purity score. This score will be used to guide the loss calculation of the PatchGAN discriminator, prompting the model to focus more on the identification of real defects during training and effectively suppressing the interference of artifacts.

[0134] To address this, this application further proposes the following steps for identifying different types of subtle imaging artifacts within the feature space output by the artifact removal encoder: establishing a dynamic feature reference library in the feature space output by the artifact removal encoder, wherein the dynamic feature reference library stores feature prototypes of known subtle imaging artifact types; inputting a sample image into the artifact removal encoder to obtain the feature representation of the sample image in the feature space, wherein the sample image is generated by an industrial camera; performing similarity matching between the feature representation of the sample image and the feature prototypes in the dynamic feature reference library; determining whether the similarity between the feature representation of the sample image and the existing feature prototypes is lower than a preset threshold; if the similarity is lower than the preset threshold, determining that there are variant subtle imaging artifacts in the overlapping region; performing cluster analysis on the feature representation of the variant subtle imaging artifacts, and obtaining different types of subtle imaging artifacts based on the cluster analysis.

[0135] Specifically, the dynamic feature reference library can be understood as a continuously updated knowledge base, whose purpose is to store and manage known and verified feature patterns of subtle imaging artifacts. These feature prototypes can be pre-generated through expert annotation, simulation, or learned from historical data. In practical applications, this reference library can be configured to be updated periodically to incorporate new artifact types or refine existing artifact features. Inputting sample images into the artifact removal encoder and obtaining the feature representation of the sample images in the feature space refers to processing the raw images captured by industrial cameras using a deep learning network, transforming them from pixel space to a more abstract and discriminative feature space. This feature representation can effectively remove artifact information from the image, highlighting surface physical features and potential defect information. In practical applications, the feature representation of the sample image is matched with the feature prototypes in the dynamic feature reference library for similarity, for example, using cosine similarity, Euclidean distance, or other distance metrics commonly used in machine learning to quantify the degree of similarity between the two. The purpose is to quickly determine whether there are features in the current sample image that match known artifact types. Furthermore, it is determined whether the similarity between the feature representation of the sample image and the existing feature prototype is lower than a preset threshold. This preset threshold is an adjustable parameter, and its setting should comprehensively consider the sensitivity and specificity of recognition. If the similarity is lower than this threshold, it indicates that the artifact features in the current sample image do not match any known artifact type in the reference library, and are therefore identified as variant subtle imaging artifacts. Thus, if the similarity is determined to be lower than the preset threshold, variant subtle imaging artifacts are identified in the overlapping region. This means that the system can proactively discover and label new artifact types that do not belong to known categories. Finally, the feature representations of the variant subtle imaging artifacts are subjected to cluster analysis, and different types of subtle imaging artifacts are obtained based on the cluster analysis. Cluster analysis is an unsupervised learning method, such as K-means, DBSCAN, etc., which aims to automatically group similar variant artifact features to discover and define new artifact categories. In this way, the system can continuously learn and adapt to new artifact patterns, enhancing its ability to recognize unknown artifacts.

[0136] This application's solution effectively addresses the problem of unknown artifacts encountered when identifying subtle imaging artifacts by introducing a dynamic feature reference library and a clustering analysis mechanism. Specifically, after sample images acquired by an industrial camera are input into the artifact removal encoder and their feature representations are obtained, these representations are first matched for similarity with known artifact feature prototypes stored in the dynamic feature reference library. This process quickly identifies known subtle imaging artifacts. However, when the similarity is below a preset threshold, the system does not simply ignore or misjudge them, but identifies them as variant subtle imaging artifacts. Subsequently, the feature representations of these variant artifacts are further subjected to clustering analysis. Clustering analysis automatically classifies these unknown artifacts into different categories based on the inherent similarity of features, enabling the system to discover and define new artifact types and incorporate them into the subsequent identification and evaluation system. It is precisely because of this dynamic learning and adaptation mechanism that this application's solution can continuously update its understanding of artifacts, ensuring effective identification and classification even when faced with previously unseen artifact types.

[0137] In some preferred embodiments, it is assumed that during the production of the tail fin, the introduction of new coating materials or processing techniques generates a previously unrecorded, subtle imaging artifact with specific optical properties. When an industrial camera captures an image containing this novel artifact, the image is input into an artifact removal encoder, generating its feature representation in the feature space. Subsequently, this feature representation is matched for similarity with existing artifact prototypes such as scratches, bubbles, and stains in a dynamic feature reference library. Since the features of this novel artifact do not match any known prototypes in the reference library, its similarity will be below a preset threshold, and the system therefore classifies it as a variant subtle imaging artifact. Next, the system performs cluster analysis on the feature representations of multiple such variant artifacts, for example using the DBSCAN algorithm, automatically classifying them into a new artifact type based on feature density and distance, and assigning them a unique identifier. In this way, even unknown artifacts appearing for the first time can be effectively identified, classified, and learned by the system, thereby updating its artifact recognition capabilities and ensuring the accuracy of subsequent purity score evaluation and defect identification.

[0138] Specifically, the steps described above—inputting the image captured by the industrial camera into the adversarially trained artifact removal encoder and outputting an artifact-independent feature image based on the adversarially trained artifact removal encoder—can be further refined into the following operations: normalizing the image captured by the industrial camera to obtain normalized image data; using the adversarially trained artifact removal encoder to remove artifact information from the normalized image data to obtain a feature image containing only surface physical features and defect information.

[0139] Normalizing images captured by industrial cameras involves scaling the pixel values ​​of the original images to fall within a preset numerical range, such as [0, 1] or [-1, 1]. This aims to eliminate differences in brightness or contrast caused by different image acquisition conditions (such as light intensity and camera gain), ensuring that the data input into the deep learning network has consistent distribution characteristics, thereby improving the training efficiency and generalization ability of the model.

[0140] Furthermore, the artifact removal encoder trained adversarially removes artifact information from the normalized image data. This means that after adversarial training, the encoder possesses the ability to identify and remove artifacts from images containing imaging artifacts. Artifact information, such as non-defect features caused by uneven lighting, reflection, shadows, and sensor noise, is effectively filtered out by the encoder. After artifact removal, the resulting feature image will primarily focus on the physical features of the tail fin surface and genuine defect information, providing a cleaner and more discriminative input for subsequent defect identification.

[0141] The proposed solution first normalizes the images captured by the industrial camera, ensuring the consistency and stability of the input data and avoiding recognition bias caused by changes in the external environment. Subsequently, an adversarially trained artifact removal encoder effectively identifies and removes various complex imaging artifacts from the normalized image data. During training, this encoder learns to distinguish between genuine defect features and artifact features, thus accurately extracting feature images containing only the physical characteristics and defect information of the tail fin surface. This process ensures the accuracy of subsequent defect recognition and avoids interference from artifacts in the recognition results.

[0142] Figure 4 A schematic diagram of a deep learning-based tail fin surface defect identification system according to an embodiment of the present invention is shown, including:

[0143] The rendering engine module is used to establish a physical rendering engine based on the computer-aided design geometric model and material optical parameters of the tail fin, and to generate synthetic training data based on the simulated imaging conditions of the physical rendering engine. The synthetic training data includes: synthetic images of imaging artifacts, ideal defect images without artifacts, and artifact masks.

[0144] The deep learning module is used to construct an artifact removal encoder and a defect recognition decoder in the deep learning network based on the synthetic training data. The artifact removal encoder is used to extract artifact-independent feature representations from the synthetic image, and the defect recognition decoder is used to identify defects based on the feature representations.

[0145] The adversarial training module is used to perform adversarial training on the artifact removal encoder and the defect identification decoder based on the adversarial generative network, so as to obtain the adversarially trained artifact removal encoder and defect identification decoder.

[0146] The feature image module is used to input the image captured by the industrial camera into the adversarially trained artifact removal encoder, and output an artifact-independent feature image based on the adversarially trained artifact removal encoder.

[0147] The defect identification module is used to input the artifact-independent feature image into the adversarially trained defect identification decoder and output the defect identification result based on the adversarially trained defect identification decoder.

[0148] The proposed system organically integrates the generation of synthetic training data, the construction and training of deep learning networks, and the final defect identification process. The rendering engine module generates high-quality, diverse synthetic training data, effectively simulating artifacts and defects under real imaging conditions, providing a solid foundation for subsequent deep learning model training. The deep learning module and adversarial training module work together to build and optimize the artifact removal encoder and defect identification decoder, ensuring the model can accurately extract defect features from complex artifacts. Finally, the feature image module and defect identification module process the actual acquired images through artifact removal and feature extraction to achieve accurate identification of defects on the tail fin surface. This effectively solves the problem of decreased recognition accuracy caused by imaging artifacts when dealing with tail fins with high curvature and complex geometry, significantly improving the robustness and reliability of the detection.

[0149] Therefore, the system proposed in this application can transform images containing artifacts captured by industrial cameras into artifact-independent feature images through a feature image module and an adversarially trained artifact removal encoder. The defect recognition module and decoder then output accurate defect recognition results. This end-to-end system solution not only improves the accuracy and stability of surface defect recognition for rear wing panels but also reduces reliance on manual annotation, enhancing the efficiency and reliability of automated inspection and providing strong technical support for quality control in the automotive manufacturing industry.

[0150] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for identifying surface defects in a tail fin based on deep learning, characterized in that, Includes the following steps: A physical rendering engine is established based on the computer-aided design geometric model and material optical parameters of the tail fin. Based on the physical rendering engine, imaging conditions are simulated to generate synthetic training data, which includes: synthetic images of imaging artifacts, ideal defect images without artifacts, and artifact masks. Based on the synthetic training data, an artifact removal encoder and a defect recognition decoder are constructed in a deep learning network. The artifact removal encoder is used to extract artifact-independent feature representations from the synthetic image, and the defect recognition decoder is used to identify defects based on the feature representations. The artifact removal encoder and defect identification decoder are adversarially trained based on the generative adversarial network to obtain the adversarially trained artifact removal encoder and defect identification decoder. The image captured by the industrial camera is input into the adversarially trained artifact removal encoder, and the artifact-independent feature image is output based on the adversarially trained artifact removal encoder. The artifact-independent feature image is input into the adversarially trained defect identification decoder, and the defect identification result is output based on the adversarially trained defect identification decoder.

2. The deep learning-based tail fin surface defect identification method according to claim 1, characterized in that, The computer-aided design geometric model based on the tail fin and the material optical parameters are used to establish a physical rendering engine, and the synthetic training data is generated based on the simulation imaging conditions of the physical rendering engine, including: The computer-aided design geometric model of the tail fin is imported into the rendering software platform and converted into a virtual product structure under the rendering engine. The material optical parameters corresponding to the virtual product structure are set according to the actual material and surface coating of the tail fin to form a virtual tail fin model; In the rendering engine, the simulation imaging condition parameters of the virtual tail fin model are set to form a virtual environmental scene. The simulation imaging condition parameters include: simulation lighting environment parameters and virtual camera parameters. Based on the virtual environment scene, various types of defect images are generated on the surface of the virtual tail fin model; Synthetic images of imaging artifacts, artifact-free ideal defect images, and artifact masks are generated based on various types of defect images.

3. The deep learning-based tail fin surface defect identification method according to claim 2, characterized in that, The generation of various types of defect images on the surface of the virtual tail fin model based on the virtual environment scene includes: In each preset virtual scene, a scratch effect is generated on the surface of the virtual tail fin model using a geometric deformation method; The physical structure morphology for adjusting the scratch effect includes: scratch location, scratch size, scratch shape, and scratch depth; The adjusted scratch effect is assigned corresponding optical properties, including: the roughness inside the scratch and the degree of diffuse reflection of the scratch effect; The rendering engine outputs the corresponding defect image based on the optical properties.

4. The deep learning-based tail fin surface defect identification method according to claim 1, characterized in that, The artifact removal encoder and defect recognition decoder constructed based on the synthetic training data in the deep learning network include: An artifact removal encoder and a defect identification decoder are constructed. The artifact removal encoder adopts the downsampling structure of U-Net, and the defect identification decoder adopts the upsampling structure of U-Net. The synthetic training data is input into the artifact removal encoder for training and learning, thereby obtaining the artifact removal encoder's feature representation that is independent of artifacts; The artifact-independent feature representation is input into the defect identification decoder for training and learning, thereby obtaining the defect identification decoder's ability to identify and locate each real defect in the tail fin.

5. The deep learning-based tail fin surface defect identification method according to claim 1, characterized in that, The adversarial training of the artifact removal encoder and defect identification decoder based on the adversarial generative network, resulting in the adversarially trained artifact removal encoder and defect identification decoder, includes: The PatchGAN discriminator receives the feature representation output by the artifact removal encoder and identifies the source of the feature representation based on the feature representation. The source is either an ideal defect image without artifacts generated by the rendering engine or a feature of an input image containing artifacts after processing by the artifact removal encoder. Based on the results of the feature representation recognition, the artifact removal encoder and the defect recognition decoder are trained with adversarial loss, reconstruction loss and defect recognition loss to obtain the adversarially trained artifact removal encoder and defect recognition decoder.

6. The deep learning-based tail fin surface defect identification method according to claim 5, characterized in that, The step of training the artifact removal encoder and the defect identification decoder with adversarial loss, reconstruction loss, and defect identification loss based on the result of the feature representation recognition includes: The purity score of pixel regions in an ideal defect image is evaluated based on the results of the feature representation recognition. The purity score is used as a weight to weight the loss of the PatchGAN discriminator. The PatchGAN discriminator, after weighting, is used to train the artifact removal encoder and the defect identification decoder with adversarial loss, reconstruction loss, and defect identification loss.

7. The deep learning-based tail fin surface defect identification method according to claim 6, characterized in that, The step of evaluating the purity score of pixel regions in an ideal defect image based on the result of the feature representation recognition includes: Identify different types of subtle imaging artifacts in the feature space of the artifact removal encoder output; Based on the defect prediction results of the overlapping region in combination with the defect recognition decoder and the label information of the overlapping region in the ideal defect image, it is determined whether the fine imaging artifacts match the characteristics of real small defects. Based on the judgment results, the local feature deviations corresponding to each subtle imaging artifact are weighted and combined to form a purity score.

8. The deep learning-based tail fin surface defect identification method according to claim 7, characterized in that, The process of identifying different types of subtle imaging artifacts in the feature space of the artifact removal encoder output includes: A dynamic feature reference library is established in the feature space output by the artifact removal encoder. The dynamic feature reference library stores feature prototypes of known subtle imaging artifact types. The sample image is input into the artifact removal encoder to obtain the feature representation of the sample image in the feature space. The sample image is generated by an industrial camera. The feature representation of the sample image is matched with the feature prototype in the dynamic feature reference library for similarity matching; Determine whether the similarity between the feature representation of the sample image and the existing feature prototype is lower than a preset threshold; If the similarity is determined to be lower than a preset threshold, then the overlapping region is determined to contain subtle imaging artifacts. Cluster analysis is performed on the feature representation of the variable fine imaging artifacts, and different types of fine imaging artifacts are obtained based on the cluster analysis.

9. The deep learning-based tail fin surface defect identification method according to claim 1, characterized in that, The step of inputting the image captured by the industrial camera into the adversarially trained artifact removal encoder and outputting an artifact-independent feature image based on the adversarially trained artifact removal encoder includes: The images captured by the industrial camera are normalized to obtain normalized image data; The artifact removal encoder trained by the adversarial method is used to remove artifact information from the normalized image data, resulting in a feature image that contains only surface physical features and defect information.

10. A deep learning-based system for identifying surface defects in a tail fin, characterized in that, include: The rendering engine module is used to establish a physical rendering engine based on the computer-aided design geometric model and material optical parameters of the tail fin, and to generate synthetic training data based on the simulated imaging conditions of the physical rendering engine. The synthetic training data includes: synthetic images of imaging artifacts, ideal defect images without artifacts, and artifact masks. The deep learning module is used to construct an artifact removal encoder and a defect recognition decoder in the deep learning network based on the synthetic training data. The artifact removal encoder is used to extract artifact-independent feature representations from the synthetic image, and the defect recognition decoder is used to identify defects based on the feature representations. The adversarial training module is used to perform adversarial training on the artifact removal encoder and the defect identification decoder based on the adversarial generative network, so as to obtain the adversarially trained artifact removal encoder and defect identification decoder. The feature image module is used to input the image captured by the industrial camera into the adversarially trained artifact removal encoder, and output an artifact-independent feature image based on the adversarially trained artifact removal encoder. The defect identification module is used to input the artifact-independent feature image into the adversarially trained defect identification decoder and output the defect identification result based on the adversarially trained defect identification decoder.

Images