Mirror surface shell defect detection method based on image recognition

Abstract

This invention belongs to the field of machine vision inspection technology, specifically disclosing a method for detecting defects in mirror-like outer shells based on image recognition. The method includes: constructing a three-dimensional digital twin model containing geometric and optical attributes; generating a high-fidelity mirror appearance image using differentiable rendering; mapping real defect textures to the virtual image using a generative adversarial network to form labeled virtual defect samples; optimizing illumination, roughness, and viewpoint parameters through backpropagation to improve sample realism; training a deep convolutional neural network based on the optimized samples; and acquiring multi-view images under a controllable multi-light source environment and inputting them into the model for defect discrimination and localization. Through the above technical solution, this invention achieves high-precision and robust detection of minute defects such as scratches and pinholes without relying on a large number of real defect samples, improving the system's generalization ability and industrial applicability.

Description

Technical Field

[0001] This invention belongs to the field of machine vision inspection technology, specifically relating to a method for detecting defects in mirror shells based on image recognition. Background Technology

[0002] With the deep integration of industrial automation and intelligent manufacturing technologies, surface defect detection based on machine vision has become a crucial step in ensuring product quality. In the manufacturing of precision electronic components and high-end consumer goods, the sensory quality of the casing directly determines the product's market competitiveness. Due to the high reflectivity and complex optical reflection mechanisms of mirror materials, the accurate capture and identification of defects on such surfaces is a core research topic in the field of industrial vision.

[0003] Deep learning-based image recognition technology offers a new approach to detecting minute defects on highly reflective surfaces. This approach typically involves acquiring high-resolution images of the surface to be inspected using industrial cameras, and then employing convolutional neural networks to extract and classify image features to automatically filter out defects such as pinholes and scratches. This process is highly dependent on the completeness of training samples, the accuracy of annotation, and the algorithm's adaptability to different lighting conditions.

[0004] Current technologies still face significant challenges in handling defects in mirror-like surfaces. The scarcity of high-quality defect samples in actual production processes leads to an imbalanced distribution in training datasets, making it difficult to cover all potential physical defect morphologies. Furthermore, pixel-level annotation of highly reflective surfaces is extremely complex and costly, and existing models exhibit vulnerability to lighting fluctuations, lacking a deep understanding of complex surface geometries and light reflection paths. Traditional static image enhancement methods cannot simulate dynamic reflection changes in real-world physical environments, resulting in detection models failing to meet the generalization performance requirements of industrial production in highly variable lighting scenarios.

[0005] Therefore, there is a need for image recognition-based defect detection methods for mirror shells. Summary of the Invention

[0006] The purpose of this invention is to provide a method for detecting defects in mirror shells based on image recognition, which can solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is: a method for detecting defects in mirror-like outer shells based on image recognition, comprising the following specific steps: Step 1: Construct a three-dimensional digital twin model of the mirror shell. Based on the geometric parameters and surface material characteristics of the real product, establish a virtual model that includes complete geometric structure and optical properties. Step 2: Based on the principles of physical optics, simulate the reflection path of light on the surface of the three-dimensional digital twin model, and generate a high-fidelity mirror appearance image by combining a differentiable rendering mechanism. Step 3: Using a generative adversarial network, the minute defect textures collected in the real production environment are mapped onto the high-fidelity mirror appearance image to form a virtual defect sample with pixel-level annotation information; Step 4: Optimize the virtual defect sample through backpropagation using a neural network, dynamically adjusting the lighting angle, surface roughness, and viewpoint parameters during the rendering process, so that the generated virtual defect sample visually approximates the real defect. Step 5: Train a deep convolutional neural network model based on the optimized virtual defect sample set, enabling it to identify minute scratches and pinhole-like defects on the surface of the mirror shell; Step 6: Place the mirror shell to be tested in a multi-light source controllable imaging environment, collect multi-view images of it under different lighting conditions, and input them into the trained deep convolutional neural network model for defect discrimination and localization.

[0008] Preferably, the three-dimensional digital twin model constructed in step 1 not only includes the overall outline of the mirror shell, but also accurately models the local curvature change region and edge transition structure, and assigns the model bidirectional reflection distribution function parameters that conform to the actual material properties, so as to accurately characterize its reflection behavior under different incident light conditions.

[0009] Preferably, the differentiable rendering mechanism used in step 2 allows the gradient information of the rendered output image to be back-transmitted to each parameter node inside the rendering pipeline, making the entire image generation process a continuously differentiable module that can be optimized end-to-end by the neural network, ensuring that the subsequent generative adversarial network can effectively participate in joint training.

[0010] Preferably, the generative adversarial network used in step 3 consists of a generator and a discriminator. The generator is responsible for embedding low-dimensional defect textures into a highly reflective surface image, while the discriminator is used to distinguish subtle differences between the synthesized image and the real defect-free image. The two continuously improve the realism of the generated image and the rationality of the defect embedding through adversarial game.

[0011] Preferably, during the backpropagation optimization process in step 4, the neural network synchronously updates the generator weights and rendering parameters based on the loss signal fed back by the discriminator, so that the final generated virtual defect sample can maintain the authenticity of the defect shape and also naturally blend into the mirror reflection background, avoiding artificial traces or uncoordinated light and shadow abrupt changes.

[0012] Preferably, in step 5, when training the deep convolutional neural network model, a multi-scale feature fusion strategy is adopted to extract local texture details and global illumination response patterns in the image respectively, and the attention mechanism is used to enhance the attention to potential defect areas, thereby improving the robustness of the model under complex illumination interference.

[0013] Preferably, the multi-light source controllable imaging environment set in step 6 includes multiple independently adjustable directional light source units. Each light source unit can flexibly adjust the illumination direction within a preset angle range and synchronously acquire image data with a high dynamic range industrial camera to cover the appearance of the mirror shell under various typical lighting configurations.

[0014] Preferably, after the deep convolutional neural network model completes training, an online adaptive fine-tuning mechanism is further introduced to continuously receive a small number of labeled real samples for incremental learning during the actual deployment stage, in order to cope with equipment aging or process drift that may occur during long-term operation of the production line environment.

[0015] Preferably, the method further includes confidence assessment and spatial consistency verification of the detection results. By cross-validating the detection results obtained from the same mirror shell under multiple viewing angles, false alarms caused by single-view occlusion or strong reflection interference are eliminated, thereby improving the accuracy of the final defect determination.

[0016] Preferably, the virtual defect sample set covers a variety of surface anomalies, including but not limited to micron-level linear scratches, sub-millimeter-level circular pinholes, and local depressions. Different size, depth, and orientation distribution patterns are set for each defect type to enhance the diversity and representativeness of the training data.

[0017] Compared with the prior art, the present invention has the following beneficial effects: By integrating differentiable rendering and physically driven generative adversarial networks, a technical approach was developed to efficiently generate high-quality virtual training data without relying on a large number of real defect samples, thus solving the problems of scarce defect samples and high annotation costs for mirror shells. Since the virtual defect samples are synthesized under the premise of strictly following the laws of physical optics, the trained model has a deep understanding of the relationship between light and surface geometry, reducing the tendency to overfit to specific lighting environments.

[0018] This invention utilizes a combination of multi-view, multi-source imaging and deep neural networks to achieve high-precision identification and stable positioning of minute defects on highly reflective surfaces, thereby improving the generalization performance and industrial applicability of the detection system. Attached Figure Description

[0019] Figure 1 The flowchart is based on the present invention; Figure 2 This is a schematic diagram of data flow according to the present invention; Figure 3 This is a flowchart illustrating the virtual defect sample generation and backpropagation optimization based on the coupling of a differentiable rendering mechanism and a generative adversarial network according to the present invention. Figure 4 This is a flowchart illustrating the training of a deep convolutional neural network model based on multi-scale feature fusion and attention mechanisms according to the present invention. Figure 5 This is a flowchart illustrating defect identification and localization based on multi-source controllable imaging environment acquisition and online adaptive fine-tuning according to the present invention. Detailed Implementation

[0020] Example 1: Please refer to the appendix Figure 1 To be continued Figure 5 To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0021] In the image recognition-based defect detection method for mirror shells, step 1 involves constructing a three-dimensional digital twin model of the mirror shell. Based on the geometric parameters and surface material characteristics of the real product, a virtual model containing complete geometric structure and optical properties is established. Specifically, the construction of the three-dimensional digital twin model first involves analyzing the original design data of the mirror shell. By reading standardized computer-aided design files, precise three-dimensional coordinate data, surface equations, and topological connections of the mirror shell are obtained. The three-dimensional digital twin model not only includes the overall outline of the mirror shell but also accurately models local curvature variation regions and edge transition structures. For the microscopic geometric features of the mirror shell surface, a high-order continuous non-uniform rational bicubic spline surface is used for fitting, ensuring that the model maintains a smooth normal vector distribution at any scaling factor, which is crucial for subsequent optical reflection simulation.

[0022] Based on the established geometric structure, the model is further endowed with bidirectional reflectance distribution function parameters that conform to the actual material properties, in order to accurately characterize its reflection behavior under different incident light conditions. The bidirectional reflectance distribution function parameters are obtained by performing multi-angle spectrophotometric measurements on a real specular material, extracting reflectivity distribution data under different incident and observation angles. The bidirectional reflectance distribution function model is divided into diffuse reflection components, specular reflection components, and micro-surface roughness coefficients. The diffuse reflection component describes the uniform outgoing light generated by internal scattering of the material, while the specular reflection component follows Fresnel's law of reflection, describing the coherent reflection of light at the interface. The micro-surface roughness coefficient is based on a statistical distribution model, describing the broadening effect of microscopic irregularities on the reflected beam. All these parameters are integrated into a set of multidimensional tensors, serving as optical attribute tags for the three-dimensional digital twin model, and stored in the system's material database.

[0023] In the above method, step 2, based on the principles of physical optics, simulates the reflection path of light on the surface of the 3D digital twin model and generates a high-fidelity mirror appearance image using a differentiable rendering mechanism. The physical optics simulation process employs a path tracing algorithm to track millions of sampled rays emitted from the photosensitive plane of the virtual camera in the 3D virtual scene. When each ray intersects the surface of the 3D digital twin model, its reflection direction and energy attenuation are calculated according to the bidirectional reflection distribution function defined in step 1. To simulate the imaging process of a real industrial camera, a virtual lens model is also added to the rendering pipeline, considering physical factors such as aperture, focal length, and sensor noise.

[0024] The differentiable rendering mechanism employed in step 2 allows the gradient information of the rendered output image to be fed back to each parameter node within the rendering pipeline, making the entire image generation process a continuously differentiable module that can be optimized end-to-end by the neural network. During rendering, the final color value of each pixel is defined as a complex function of scene geometry parameters, light source position, material properties, and camera parameters. Partial derivatives of the pixel color value with respect to each of the aforementioned input parameters are calculated using analytical differentiation or automatic differentiation techniques. When the generated rendered image differs from the target image, the resulting loss signal can be passed back layer by layer to the lowest-level physical parameters using a chain rule, enabling dynamic correction of the scene configuration. This mechanism ensures that the subsequent generative adversarial network can effectively participate in joint training, transforming the rendering process from a closed black box into a transparent, learnable physical layer.

[0025] In the above method, step 3 utilizes a generative adversarial network (GAN) to map the minute defect textures collected from the real production environment onto the high-fidelity mirror appearance image, forming virtual defect samples with pixel-level annotation information. The GAN used in step 3 consists of a generator and a discriminator. The generator is responsible for embedding the low-dimensional defect textures into the highly reflective surface image, while the discriminator distinguishes subtle differences between the synthesized image and the real, defect-free image. The generator's input includes two parts: the defect-free high-fidelity image generated in step 2 and a low-dimensional vector representing the defect features, which encodes the defect type, length, width, and distribution density. The generator uses multi-layer transposed convolution operations to superimpose the defect textures onto the original mirror image using residual learning, while considering the local perturbation of the surrounding light and shadow distribution by the defect region, ensuring the embedded defects have geometric topological coherence.

[0026] The discriminator employs a deep residual network structure, extracting high-frequency detail features through multi-scale downsampling of the image. The discriminator's training objective is to identify artificial traces in the synthesized image to the greatest extent possible, while the generator's objective is to continuously adjust parameters to make the discriminator unable to distinguish between real and fake images. Through adversarial competition, the two continuously improve the realism of the generated images and the rationality of the defect embedding. Since the generation process of virtual defects is completely controlled by the system, each generated image is automatically associated with a binary mask image, which precisely records the coordinate position of each defect pixel, achieving perfect pixel-level automatic annotation. The virtual defect sample set covers various types of surface anomalies, including but not limited to micron-level linear scratches, sub-millimeter-level circular pinholes, and localized depressions, with different size, depth, and orientation distribution patterns set for each defect type.

[0027] In the above method, step 4 involves backpropagation optimization of the virtual defect sample using a neural network. This dynamically adjusts the lighting angle, surface roughness, and viewpoint parameters during the rendering process, making the generated virtual defect sample visually approximate the real defect. During the backpropagation optimization process, the neural network synchronously updates the generator weights and rendering parameters based on the loss signal fed back from the discriminator. This loss signal includes not only traditional adversarial loss but also physical consistency loss and perceptual loss. Physical consistency loss constrains the generated defects to conform to the laws of optical reflection; for example, defects in bright light areas should exhibit higher contrast, while defects in shadow areas should be relatively blurred. Perceptual loss measures the distribution distance between the synthesized image and the real sample in the feature space through a pre-trained feature extraction network.

[0028] Based on these combined loss signals, the neural network automatically searches for the optimal combination of rendering parameters using stochastic gradient descent or adaptive momentum estimation algorithms. For example, if the discriminator finds the lighting effects of the synthesized image too harsh, the gradient information guides the system to reduce the coherence of the virtual light source or fine-tune the surface roughness parameters of the 3D digital twin model. This dynamic adjustment process allows the final generated virtual defect samples to maintain the realism of the defect morphology while naturally blending into the specular reflection background, avoiding artificial traces or uncoordinated lighting abrupt changes. Through thousands of iterative optimizations, the virtual generation system can spontaneously learn the complex physical laws governing the interaction between light and specular defects in the real environment, generating training data with extremely high generalization value.

[0029] In the above method, step 5 involves training a deep convolutional neural network model based on an optimized set of virtual defect samples, enabling it to identify minute scratches and pinhole-like defects on the surface of the mirror shell. The training process is performed on a computing platform containing a high-performance graphics processor. During the training of the deep convolutional neural network model in step 5, a multi-scale feature fusion strategy is employed to extract local texture details and global illumination response patterns from the image. The lower layers of the network are responsible for capturing microscopic features such as edges and color spots, while the higher layers are responsible for understanding the global geometric structure of the object and the distribution patterns of ambient light.

[0030] By employing an attention mechanism to enhance focus on potential defect areas, the system automatically calculates the weight coefficient of each region in the feature map, assigning higher response values ​​to suspected defective regions. In the loss function design, a weighted cross-entropy loss is used to improve the model's focus on rare defect types. Since the training data includes simulations under various extreme lighting conditions, the model learns to identify defects and possesses a deep understanding of the relationship between light and surface geometry, reducing the tendency to overfit to specific lighting environments. After each training iteration, the model is evaluated using an independent validation set, calculating the mean precision, recall, and false positive rate. A learning rate decay strategy guides the model to converge to the global optimum.

[0031] In the above method, step 6 involves placing the mirror housing to be inspected in a multi-light source controllable imaging environment, acquiring multi-view images under different lighting conditions, and inputting these images into the trained deep convolutional neural network model for defect identification and localization. The multi-light source controllable imaging environment set up in step 6 includes multiple independently adjustable directional light source units, each of which can flexibly adjust its illumination direction within a preset angle range. These light source units are arranged on a hemispherical support, and the brightness and triggering sequence of each LED unit are controlled by a digital controller. Image data is simultaneously acquired using a high dynamic range industrial camera to cover the appearance of the mirror housing under various typical lighting configurations. High dynamic range imaging technology, by synthesizing a sequence of images with different exposure times, ensures that there is no overflow in the strongly reflective areas of the mirror and that details are preserved in the dark areas, providing the richest information input for the model.

[0032] After training, the deep convolutional neural network model further incorporates an online adaptive fine-tuning mechanism. During actual deployment, it continuously receives a small number of labeled real samples for incremental learning to address potential equipment aging or process drift during long-term operation in the production line environment. For example, when industrial camera sensors produce dead pixels or light source brightness decreases over time, the online learning module automatically extracts these changing features and, without disrupting the original weights, fine-tunes the classification layer parameters to adapt to the new data distribution. Furthermore, the method includes confidence assessment and spatial consistency verification of the detection results. For each candidate defect point output by the model, the system calculates a confidence value between 0 and 1. By cross-validating the detection results obtained from the same mirror shell under multiple viewpoints, if the defect points to the same geometric location in the 3D reconstruction space under multiple viewpoints, it is determined to be a real defect; conversely, if it only appears under a single viewpoint and its location is not fixed, it is determined to be a false alarm point caused by strong reflection interference.

[0033] Further refining step 1, the technical details of constructing the 3D digital twin model are described. After obtaining the geometric parameters of the mirror shell, mesh subdivision processing is required to transform the complex freeform surface into a mesh composed of millions of tiny triangles. Each triangle vertex carries not only spatial position coordinates but also tangent vectors, binormal vectors, and UV texture coordinates. When describing the surface material properties, in addition to the basic bidirectional reflection distribution function, the multi-layer structure of the material must also be considered. For example, some mirror shell surfaces are covered with a transparent protective coating, and their optical behavior involves two refractions and reflections of light at the air-coating interface and the coating-substrate interface. This embodiment calculates the energy transmittance and reflectance of light at different medium interfaces by establishing a two-layer Fresnel model. The transmittance equals 1 minus the reflectance, while the reflectance is calculated by half the sum of the squares of the vertical and horizontal polarization components. This refined optical modeling ensures that the virtual model has extremely high fidelity when simulating the light spill effect and Fresnel edge brightening effect of a highly reflective shell.

[0034] Further refining step 2, the internal implementation of the differentiable rendering mechanism is described. The rendering pipeline is abstracted as a computation graph, where each node represents a physical operation, such as coordinate transformation, vector dot product, exponential operation, etc. When a ray collides with the model, the color of the collision point is obtained by importance sampling of the ambient octagon map. The calculation of gradient information involves differentiating the collision point position relative to the coordinates of the triangle vertices. If the collision point is inside the triangle, its position can be represented as the weighted sum of the centroid coordinates of the three vertices, with the weights summing to 1. This relationship is linear and differentiable everywhere. For the non-differentiability problem caused by boundary regions, an edge sampling compensation algorithm is used. By adding random sampling points near the geometric edges and applying a smooth step function, the gradient of the rendering output relative to the geometric deformation becomes smooth. In this way, when the generative adversarial network feedback requires adjustment of the defect position or shape, the rendering engine can provide a clear direction for parameter updates.

[0035] Step 3 is further refined to generate the defect texture mapping logic in the adversarial network. The realistically acquired minute defect textures are first preprocessed to extract their local contrast and frequency domain features. After receiving the high-fidelity mirror image, the generator does not simply overlay the defect image; instead, it calculates the projection of the defect onto the mirror surface using a spatial transformation network. Because the mirror shell has curvature, the projection of the defect in the image will be stretched or compressed, and this deformation must conform to the principles of perspective projection. The generator contains an affine transformation module, whose parameters are calculated in real-time by the curvature-aware unit. Simultaneously, the normal vector of the defect region is fine-tuned, causing a small offset angle relative to the normal vector of the original smooth surface. This offset angle directly participates in the re-rendering process in step 2, enabling the defect region to produce realistic, view-dependent light and shadow flickering effects.

[0036] Further refining step 4, the parameter adjustment strategy for backpropagation optimization is described. The system maintains a parameter space library, including illumination intensity vectors, light source position coordinates, camera exposure parameters, and model surface roughness gradients. In each optimization step, the Cramer-Lao bound between the current synthetic sample and the target real sample distribution is calculated. If the divergence between the two distributions exceeds a preset threshold, the optimizer will increase the search step size. To prevent the optimization process from getting stuck in local optima, a simulated annealing mechanism is introduced, allowing for larger random fluctuations in parameter adjustments in the early stages of optimization, gradually reducing the fluctuation amplitude as the number of iterations increases. Specifically, for illumination angle optimization, the system simulates a spherical light source array, adjusting the weighting coefficients of each light source to find the lighting configuration that best exposes the visual characteristics of defects. The results of this optimization are not only used to generate data but can also be fed back to the multi-light source environment design in step 6, guiding the physical arrangement of light sources in the actual production line.

[0037] Further refining step 5, the feature fusion details of the deep convolutional neural network are described. Multi-scale feature fusion is achieved through a feature pyramid network, which upsamples and element-wise adds convolutional feature maps of different depths. Lower-level feature maps retain rich sub-pixel-level texture information, aiding in the localization of pinhole-like defects; higher-level feature maps possess stronger semantic information, helping to identify long-distance scratches. In the attention mechanism, a dual strategy of parallel spatial attention and channel attention is employed. Spatial attention generates a two-dimensional weight map by calculating average pooling and max pooling of the feature maps in the spatial dimension, highlighting the spatial regions where defects may exist; channel attention extracts the correlation between channels through global average pooling, enhancing the gain of defect-sensitive color channels. This combined approach enables the model to effectively suppress irrelevant backgrounds and pinpoint weak defect signals when facing flare interference from highly reflective surfaces.

[0038] Step 6 is further refined into a multi-source acquisition and evaluation process. In the actual inspection phase, the mirror shell is placed on a precision rotating stage with a repeatability accuracy of 0.01 degrees. The light source controller cycles through 24 preset illumination modes, each corresponding to a specific defect detection scenario; for example, mode A is specifically for detecting shallow scratches, and mode B is specifically for detecting deep pinholes. A high dynamic range camera continuously captures three images with different gains in each illumination mode and merges them into a single high-resolution image with 32-bit depth using a lookup table technique. The model processes these images in parallel, outputting a predicted heatmap for each viewpoint. The spatial consistency verification module projects these two-dimensional heatmaps back into the coordinate system of the three-dimensional digital twin model. If a location is marked as high-risk in more than three independent viewpoints, and the three-dimensional curvature characteristics of that point conform to the geometric statistical laws of defects, it is ultimately confirmed as a defect. This multi-dimensional cross-validation reduces false alarms caused by static dust or water vapor condensation during a single image capture.

[0039] Example 2: To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0040] In the image recognition-based defect detection method for mirror shells, this embodiment focuses on describing adaptive adjustments for different material properties and robustness enhancement schemes under complex environments. In step 1, when constructing the 3D digital twin model, different physical layer models are established for mirror shells with a metallic texture and mirror shells with a glass texture. For the metallic texture shell, the complex refractive index parameter is described, which includes a real part and an imaginary part; the real part corresponds to refraction, and the imaginary part corresponds to absorption. By modifying the value of the complex refractive index, the reflection spectral characteristics of different metal surfaces, from stainless steel to aluminum alloy, can be simulated. For the glass texture shell, a subsurface scattering model is added to describe the process of light entering the material's interior, being scattered multiple times among tiny impurities, and then re-emitting.

[0041] In the above method, step 2, when generating images using a differentiable rendering mechanism, incorporates environment mapping technology. By loading a 360-degree high dynamic range panoramic image captured from a real production line as the environmental background into the virtual scene, the rendered mirrored shell can reflect the robotic arms, conveyor belts, and factory lights on the production line. This realism of the background is crucial for training the generative adversarial network. When calculating gradients, the differentiable rendering engine considers the brightness gradient of the environmental background simultaneously. If the ambient light is too cluttered, causing defect features to be obscured, the gradient descent algorithm automatically optimizes the aperture parameters of the virtual camera, highlighting the subject features by reducing the depth of field or adjusting the contrast.

[0042] In the above method, step 3, the defect texture mapping in the generative adversarial network, employs a more complex physical driving mechanism. It goes beyond simple color overlay; it involves modeling the microscopic geometry of the defect. For example, a scratch is modeled as a groove with a triangular cross-section. When light shines on this groove, it undergoes two reflections: the first on one slope of the groove, and the second on the other. This causes the scratch area to appear extremely bright on one side and extremely dark on the other from certain viewing angles. By learning this physical law of double reflection, the generator can generate scratch samples that are more optically accurate. The discriminator is then endowed with frequency domain analysis capabilities, transforming the image to the frequency domain using a Fast Fourier Transform to detect whether the energy distribution of the synthesized image at different frequency bandwidths is consistent with the real sample.

[0043] In the above method, step 4, the backpropagation optimization process, introduces a meta-learning strategy. The neural network not only optimizes the generation parameters of a single image but also learns how to quickly adapt to new defect types. When a completely new, never-before-seen defect morphology appears on the production line (e.g., local color difference caused by a new process), the meta-learning algorithm can quickly adjust the initial weights of the rendering pipeline using a small number of samples, enabling the system to generate a large number of virtual samples targeting this new defect in a very short time. This self-evolutionary capability ensures that the detection system can keep pace with changes in production processes without requiring the redevelopment of a new detection algorithm.

[0044] In the above method, step 5, the training of the deep convolutional neural network, employs a combination of data augmentation and adversarial training. Random noise, blurring, rotation, and affine transformations are added to the virtual defect samples to simulate vibrations or defocusing that may occur during actual data acquisition. Simultaneously, the model continuously faces "adversarial samples" generated by the generator, specifically tailored to the model's weaknesses. These samples lie in the ambiguous zone of the classification boundary; through continuous learning from these samples, the model's decision boundaries become clearer and more robust.

[0045] In the above method, step 6, confidence assessment and spatial consistency verification, employs a Bayesian inference framework. The system outputs not only a binary result of "defect presence or absence," but also a probability distribution of defect existence. By fusing the probability distributions from multiple perspectives, a global posterior probability is calculated. If the global posterior probability is greater than a preset 0.95, an alarm is triggered, and the rejection device on the production line is activated. Simultaneously, spatial consistency verification incorporates kinematic information. By tracking the trajectory of the mirror shell on the conveyor belt, the system can predict the defect's position in the next frame image, further eliminating transient optical artifacts through continuous observation over time.

[0046] Example 3: To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0047] This embodiment provides a detailed description of lightweight deployment and efficient operation in an embedded edge computing environment. In step 1, when constructing the 3D digital twin model, the mesh data is simplified. An edge-folding algorithm compresses the complex model with millions of faces into tens of thousands of faces, while normal mapping technology preserves high-frequency microscopic surface details. This approach significantly reduces the amount of geometric computation during rendering while maintaining the accuracy of optical reflection simulation.

[0048] In the above method, step 2, the differentiable rendering mechanism, employs a lighting pre-computation technique based on spherical harmonic functions. It decomposes complex ambient lighting into a linear combination of a set of spherical harmonic basis functions, transforming integral operations into simple vector dot product operations. This optimization enables real-time, high-fidelity rendering output even on embedded devices with limited computing power, providing a hardware foundation for online adaptive fine-tuning.

[0049] In the above method, step 3 of the generative adversarial network employs a lightweight depthwise separable convolutional structure. Standard convolution is split into channel-wise convolution and pointwise convolution, reducing the computational cost to 1 / 10 of the original without significant loss of accuracy. The number of parameters for the generator and discriminator is strictly controlled to within a few megabytes, allowing them to be directly loaded into the on-chip cache of the embedded processor, avoiding latency caused by frequent memory read / write operations.

[0050] In the above method, step 4, backpropagation optimization, employs a quantization-aware training strategy. During parameter optimization, low bit width (e.g., 8-bit integer) computational errors are simulated, enabling the final generated virtual samples and trained model to seamlessly migrate to fixed-point computing platforms. This strategy ensures that the detection accuracy remains stable when the model is migrated from a high-performance graphics card to an application-specific integrated circuit (ASIC) chip, without performance degradation due to floating-point truncation.

[0051] In the above method, step 5 involves pruning the deep convolutional neural network model. By evaluating the importance of neurons, connections that contribute less to defect identification are removed, further compressing the model size. Simultaneously, a knowledge distillation mechanism is introduced, utilizing a large-scale, well-trained teacher network to guide the learning of a lightweight student network. The teacher network transfers its profound understanding of light relationships to the student network in the form of soft labels, enabling the student network to possess strong generalization capabilities even with a relatively small parameter scale.

[0052] In the above method, step 6 involves the multi-source acquisition environment and the edge-side inference engine working collaboratively. The inference engine directly controls the rapid flashing of the light source through a general-purpose input / output interface, achieving millisecond-level synchronous acquisition and recognition. After local preprocessing, the acquired image data is directly fed into the inference engine to draw conclusions. Only when the system encounters a suspected defect that cannot be accurately judged (confidence level in the middle range) will the original image be uploaded to the cloud server for secondary in-depth analysis. This cloud-edge collaborative architecture ensures the real-time performance of production line inspection while leveraging the powerful computing capabilities of the cloud to solve the judgment challenges of a very small number of complex cases.

[0053] In further implementation, to address non-defect interference such as grease and fingerprints that may exist on the surface of the mirror casing, this embodiment specifically generates a batch of "interference samples" containing such interference in step 3. By simulating the diffuse reflection enhancement effect of grease on light and the attenuation effect on mirror reflection, the training model can accurately distinguish between real defects and surface stains. In step 6, auxiliary irradiation is performed using a specific ultraviolet light source. Since grease has a specific fluorescence reaction under ultraviolet light, multispectral analysis further improves the rejection rate of non-defect interference.

[0054] In summary, this invention achieves the automatic generation of high-fidelity mirror appearance images by constructing a three-dimensional digital twin model of a mirror shell and combining physical optics principles with a differentiable rendering mechanism. A generative adversarial network is used to fuse real defect textures with the rendered image, forming a virtual defect sample set with pixel-level annotations. Backpropagation is used to dynamically adjust rendering parameters, making the generated virtual samples visually approximate real defects. A deep convolutional neural network model trained on this high-quality data, combined with a multi-source controllable imaging environment, achieves high-precision and robust detection of minute defects on the surface of the mirror shell. This method solves the problems of sample scarcity and high annotation costs, improving the universality and reliability of the detection system in complex industrial environments.

[0055] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting defects in mirror-like outer shells based on image recognition, characterized in that, Includes the following steps: Step 1: Construct a three-dimensional digital twin model of the mirror shell. Based on the geometric parameters and surface material characteristics of the real product, establish a virtual model that includes complete geometric structure and optical properties. Step 2: Based on the principles of physical optics, simulate the reflection path of light on the surface of the three-dimensional digital twin model, and generate a high-fidelity mirror appearance image by combining a differentiable rendering mechanism. Step 3: Using a generative adversarial network, the minute defect textures collected in the real production environment are mapped onto the high-fidelity mirror appearance image to form a virtual defect sample with pixel-level annotation information. Step 4: Optimize the virtual defect sample through backpropagation using a neural network, dynamically adjusting the lighting angle, surface roughness, and viewpoint parameters during the rendering process, so that the generated virtual defect sample visually approximates the real defect. Step 5: Train a deep convolutional neural network model based on the optimized virtual defect sample set, so that it has the ability to identify minute scratches and pinhole-like defects on the surface of the mirror shell. Step 6: Place the mirror shell to be tested in a multi-light source controllable imaging environment, collect multi-view images of it under different lighting conditions, and input them into the trained deep convolutional neural network model for defect identification and localization.

2. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, The process of constructing the three-dimensional digital twin model in step 1 includes: Analyze the computer-aided design files of the mirror shell to obtain the precise three-dimensional coordinate data, surface equations, and topological connections of the mirror shell; A high-order continuous non-uniform rational bicubic spline surface is used for surface fitting to ensure that the model maintains a smooth normal vector distribution under different scaling factors. The three-dimensional digital twin model is given bidirectional reflectance distribution function parameters, which are obtained by multi-angle spectrophotometric measurement of real mirror material, and the bidirectional reflectance distribution function parameters are divided into diffuse reflection component, specular reflection component and micro-surface roughness coefficient. The diffuse reflection component is used to describe the uniform outgoing light generated by internal scattering of the material, the specular reflection component describes the coherent reflection of light at the interface based on Fresnel's law of reflection, and the micro-surface roughness coefficient describes the broadening effect of micro-irregularities on the reflected beam based on a statistical distribution model. The geometric parameters and optical properties are integrated into a multidimensional tensor and stored as attribute tags for the three-dimensional digital twin model.

3. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, The process of generating a high-fidelity mirror appearance image in step 2 includes: A path tracing algorithm is used to track sampling rays emitted from the photosensitive plane of a virtual camera in a 3D virtual scene, and the reflection direction and energy attenuation of each ray when it intersects with the surface of the 3D digital twin model are calculated based on the bidirectional reflection distribution function parameters. A virtual lens model is introduced into the rendering pipeline, which takes into account aperture, focal length, and sensor noise parameters. The differentiable rendering mechanism defines the color value of each pixel in the rendered output image as a continuously differentiable function with respect to scene geometric parameters, light source position, material properties, and camera parameters. By using automatic differentiation technology to calculate the partial derivative of the pixel color value with respect to each of the above parameters, the loss signal at the pixel level is transmitted back to the underlying physical parameters through a chain rule, thereby achieving dynamic correction of the scene configuration.

4. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, The process of forming virtual defect samples in step 3 includes: A generative adversarial network consisting of a generator and a discriminator is constructed. The generator receives the high-fidelity mirror appearance image and a low-dimensional vector representing the defect features. The low-dimensional vector encodes the type, length, width, and distribution density of the defect. The generator superimposes the defect texture onto the original image using residual learning through multi-layer transposed convolution operations, and simulates the local perturbation of the surrounding light and shadow distribution by the defect region. The discriminator employs a deep residual network structure, which extracts high-frequency detail features by performing multi-scale downsampling on the image to distinguish between synthetic images and real, defect-free images. The generator and the discriminator improve the rationality of defect embedding through adversarial game theory; While generating each virtual defect sample image, the system automatically generates a corresponding binarized mask image. The binarized mask image records the coordinate position of each defect pixel to achieve automatic pixel-level annotation. The virtual defect sample set covers micron-level linear scratches, sub-millimeter-level circular pinholes, and localized depressions, and sets different size, depth, and orientation distribution patterns for each defect type.

5. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, The specific process of step 4 includes: During the backpropagation optimization process, the deep convolutional neural network synchronously updates the weights of the generator and the physical parameters in the rendering pipeline based on the loss signal fed back by the discriminator. The loss signals include adversarial loss, physical consistency loss, and perceptual loss; The physical consistency loss is used to constrain the generated defects to conform to the optical reflection law, so that the defects in the bright light area appear high contrast and the defects in the shadow area appear blurred. The perceptual loss measures the distribution distance between the synthetic image and the real sample in the feature space through a pre-trained feature extraction network; The deep convolutional neural network uses an adaptive momentum estimation algorithm to search for the optimal combination of rendering parameters in the parameter space; A simulated annealing mechanism is introduced during the optimization process. In the early stage of optimization, a large parameter is set to adjust the random fluctuation space, and the fluctuation amplitude is gradually reduced as the number of iterations increases.

6. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, The process of training the deep convolutional neural network model in step 5 includes: adopting a multi-scale feature fusion strategy, using a feature pyramid network to upsample and add convolutional feature maps of different depths element by element to extract local texture details and global illumination response patterns in the image. A dual attention mechanism is introduced, which includes spatial attention and channel attention. A two-dimensional weight map is generated by calculating the average pooling and max pooling of the feature map in the spatial dimension to highlight the suspected defect area, and the correlation between channels is extracted by global average pooling to enhance the gain of the defect-sensitive color channel. We employ weighted cross-entropy loss in the loss function to improve the model's attention to rare defect types; An online adaptive fine-tuning mechanism is introduced during training, enabling the model to receive labeled real samples for incremental learning during the actual deployment phase. By fine-tuning the classification layer parameters, the model can cope with changes in data distribution caused by equipment aging or process drift.

7. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, The process of acquiring multi-view images in step 6 includes: The mirror housing to be tested is placed on a precision rotating stage, and multiple independent directional light source units arranged on a hemispherical support are controlled by a digital controller to cycle through various preset lighting modes. High dynamic range (HDR) industrial cameras are used to synchronously acquire image data in each lighting mode, and HDR images are generated by synthesizing a sequence of images with different exposure times. Each light source unit in the multi-light source controllable imaging environment adjusts its illumination direction within a preset angle range to cover the appearance of the mirror shell under various lighting configurations. The trained deep convolutional neural network model processes the acquired images in parallel and outputs a predicted heatmap for each viewpoint.

8. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, The method also includes a process of confidence assessment and spatial consistency verification of the detection results: The system calculates the confidence score for each candidate defect point output by the deep convolutional neural network model; The probability distributions from multiple perspectives are fused using a Bayesian inference framework, and the global posterior probability is calculated. Two-dimensional heatmaps from different perspectives are projected back into the coordinate system of the three-dimensional digital twin model to verify the consistency of the three-dimensional space. When a location is marked as a defect risk point in more than three independent viewpoints, and the three-dimensional curvature characteristics of the location conform to the geometric statistical law of defects, it is determined to be a real defect. If a candidate defect point appears only from a single viewpoint and its location is not fixed, it is determined to be a false alarm point caused by strong reflection interference. Simultaneously, by combining kinematic information and tracking the motion trajectory of the mirror shell during transportation, the location of defects in time-series images is predicted to eliminate transient optical ghosting.

9. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, Different physical layer models are established for mirror shells with metallic and glass textures respectively; for metallic shells, complex refractive index parameters are set in the three-dimensional digital twin model, the complex refractive index parameters include real and imaginary parts, where the real part corresponds to the refraction process and the imaginary part corresponds to the absorption process; For glass-like shells, a subsurface scattering model is added to the model to describe the outgoing behavior of light after multiple scatterings inside the material; When generating the image in step 2, an environment mapping technique is introduced. By loading a 360-degree high dynamic range panoramic image collected from the real production line into the virtual scene as the environmental background, the rendered mirror shell reflects the characteristics of the production line environment. When calculating the gradient, the differentiable rendering engine simultaneously considers the brightness gradient of the environmental background and adjusts the aperture parameters of the virtual camera to highlight the defect features of the main body.

10. The image recognition-based defect detection method for mirror shells according to claim 1, characterized in that, The method also includes a lightweight deployment and interference removal process: when constructing a three-dimensional digital twin model, an edge folding algorithm is used to simplify the mesh data, and normal mapping technology is used to preserve microscopic surface details; In step 2, a lighting pre-computation technique based on spherical harmonics is used to decompose ambient lighting into a linear combination of spherical harmonic basis functions and to transform integral operations into vector dot product operations. The deep convolutional neural network model adopts a depthwise separable convolutional structure and splits standard convolution into channel-wise convolution and point-wise convolution. We use model pruning techniques to remove low-contribution connections and introduce a knowledge distillation mechanism, where a pre-trained teacher network guides a lightweight student network in learning. Interference samples simulating grease and fingerprints are added to the virtual defect sample set, and training is performed by simulating the diffuse reflection enhancement effect of grease on light and the attenuation effect on specular reflection. During the detection phase, an ultraviolet light source is used for auxiliary irradiation. By analyzing the fluorescence reaction of the oil under ultraviolet light, non-defect interference can be eliminated.

Images