A machine vision-based online detection method and system for picking up a defective plating layer

Abstract

The present application relates to the technical field of image processing, in particular to a kind of online detection method and system of drawing spindle plating layer defect based on machine vision, by obtaining multi-view image, feature extraction is carried out to image data stream after pre-processing, and match or cluster with preset defect dictionary, preliminary identification potential defect area and trigger fine multi-angle image acquisition, apply multi-task CNN model of fusion polarization perception convolution kernel, joint optimization pixel-level segmentation loss and boundary prediction loss, output the pixel-level semantic segmentation and accurate boundary of defect area, calculate reconstruction error to carry out defect classification.Finally, the system also has defect root cause analysis and process flow adjustment function, realizes the traceability of defect and production optimization.The system integrates advanced visual technology, deep learning and multi-modal information fusion, realizes the online, accurate, automated detection of drawing spindle plating layer defect.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an online detection method and system for ingot plating defects based on machine vision. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing, the demand for online product quality inspection is increasing. Especially in complex manufacturing processes such as ingot removal and coating, traditional manual inspection methods are inefficient, susceptible to subjective factors, and difficult to guarantee consistency.

[0003] Due to the inherent specular or mixed reflective properties of the coating surface, captured images often contain highlights, glare, or multiple reflections, which can easily affect the accuracy of defect identification and make it difficult to reflect the true physical structure of the ingot coating. For example, a tiny crack may be obscured or blurred due to reflection interference, or even its shape may be distorted, severely impacting the accuracy of machine defect identification. The complex layered structure within the ingot coating means that defects may be located within multiple layers of material. These layers can not only interfere with the detection process but also introduce false information, leading to false alarms or missed detections, which is a key challenge currently facing machine vision inspection.

[0004] To address this, a machine vision-based online detection method and system for ingot plating defects is proposed. Summary of the Invention

[0005] This invention provides a machine vision-based online detection method and system for plating defects on ingots. By combining machine vision, deep learning, and multimodal image processing technologies, it achieves automatic identification, classification, and traceability of plating surface defects, improving product quality and optimizing production processes. The method includes the following steps: Using a camera array equipped with polarizing filters, multi-view continuous images of the ingot plating layer under different lighting conditions and polarization states are acquired; the multi-view continuous images are preprocessed to obtain an image data stream; Feature extraction is performed on the image data stream, and the extracted features are matched and clustered with a preset dictionary to identify potential defect areas in the plating layer. For the potential defect areas, refined multi-angle image acquisition is triggered, and the surface normal and depth information of the plating layer are reconstructed using a self-calibrated photometric stereo algorithm to obtain the feature vector to be detected. A CNN model with fused polarization-aware convolutional kernels is used for defect detection. The feature vector to be detected is input into the CNN model for reconstruction, and the reconstruction error is calculated. Conventional visual features and polarization optical features are extracted simultaneously from multimodal input data, and a dual-branch output structure is designed to jointly optimize the segmentation loss and boundary prediction loss. The segmentation result and boundary localization map of the defect region are output, and the defect classification is obtained by combining the reconstruction error. Trace the root cause of the defects and adjust the process flow in a timely manner.

[0006] Preferably, the step of preprocessing the multi-view continuous image specifically includes: dynamically fusing and enhancing the multi-view continuous image, identifying local characteristics of the image, the local characteristics including gradient magnitude, local contrast and texture complexity, and dynamically adjusting the fusion weight of the multi-view continuous image according to the local characteristics to generate an image data stream.

[0007] Preferably, the step of feature extraction from the image data stream specifically includes: constructing a feature dictionary for a preset defect pattern; performing feature extraction on the image data stream, the feature extraction process including scale space extremum detection, key point localization, dominant direction assignment, and generation of high-dimensional descriptors; matching and clustering the extracted features with the preset dictionary to identify potential defect regions in the ingot coating; reconstructing the surface normal using the Lambert reflection model, and then obtaining depth information through integration to generate a feature vector to be detected.

[0008] Preferably, the step of performing defect detection using a CNN model with fused polarization-aware convolutional kernels specifically includes: A CNN structure is adopted, and a polarization-sensitive processing layer is embedded in the convolution kernel of the CNN architecture to process polarization information, resulting in a CNN model with fused polarization-aware convolution kernel. Anomalies are identified by learning the normal patterns and features of a defect-free surface. When performing defect detection, the reconstruction error of the feature vector to be detected is calculated, and reconstruction errors higher than a preset threshold are marked as abnormal regions. For the abnormal regions, a lightweight convolutional neural network is used for defect classification. The defect classification includes uneven coating thickness, missing parts, surface scratches, pits, peeling, blistering, color difference abnormalities, and oxide spots. A binary neural network is trained separately for each defect classification and works in conjunction with a CNN model that incorporates polarization-aware convolutional kernels.

[0009] Preferably, the defect detection specifically employs a multi-task deep learning architecture, including: a shared feature extraction backbone network configured to extract common feature representations from the feature vector to be detected; and multiple task heads for bounding box localization and pixel-level semantic segmentation, each performing its own recognition task based on the shared features, synchronously generating the bounding box geometric coordinates and geometric contours of the defect region in a single computation process, determining the precise geometric shape, size, and spatial distribution of the defect by analyzing the pixel-level semantic segmentation results, and classifying the defect accordingly.

[0010] Preferably, the multimodal input data includes: intensity image, polarization degree map, polarization angle map, surface depth map, surface normal map, and penetration imaging data. The penetration imaging data further includes near-infrared, terahertz, and ultrasonic data, which are integrated through a customized input layer and feature fusion module.

[0011] Preferably, the steps for segmenting the defect region and generating the boundary localization map are as follows: the CNN model with fused polarization-aware convolutional kernels is equipped with an independent feature encoder to simultaneously extract conventional visual features and polarization optical features from multimodal input data; it includes dual output branches: the main branch generates a pixel-level semantic segmentation probability map of the defect region, and the auxiliary branch predicts the probability distribution of the defect boundary; by jointly optimizing the segmentation loss and the boundary prediction loss, it captures the fine-grained contour features of the defect and simultaneously outputs the defect segmentation result and the boundary localization map.

[0012] A machine vision-based online detection system for plating defects in ingots includes: Data acquisition and preprocessing module: Using a camera array equipped with polarizing filters, acquire multi-view continuous images of the ingot plating layer under different lighting conditions and polarization states; preprocess the multi-view continuous images to obtain image data stream; Potential defect area identification module: extracts features from the image data stream, matches and clusters the extracted features with a preset dictionary, and identifies potential defect areas in the ingot plating layer; Refined feature vector generation module: For the potential defect area, it triggers refined multi-angle image acquisition and uses a self-calibrated photometric stereo algorithm to reconstruct the surface normal and depth information of the ingot coating to obtain the feature vector to be detected; Defect detection and analysis module: A CNN model with fused polarization-aware convolutional kernels is used for defect detection. The feature vector to be detected is input into the CNN model for reconstruction, and the reconstruction error is calculated. Conventional visual features and polarization optical features are extracted simultaneously from multimodal input data, and a dual-branch output structure is designed to jointly optimize the segmentation loss and boundary prediction loss. The segmentation result and boundary localization map of the defect region are output, and the defect classification is obtained by combining the reconstruction error. Defect Root Cause Analysis and Optimization Module: Traces the root cause of defects and adjusts the process flow in a timely manner.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By acquiring multi-view, multi-polarization images using a camera array equipped with polarization filters, the polarization information can effectively distinguish surface defects from normal areas, exhibiting higher sensitivity to subtle defects and different types of defects, such as scratches, dents, and color differences. This greatly enriches the image information. Furthermore, the deep learning anti-reflection network can effectively separate reflection interference, ensuring the purity of the input image data and further improving the accuracy of subsequent feature extraction and defect identification.

[0014] 2. The application of a CNN architecture that integrates polarization-sensing convolutional kernels and a self-calibrated photometric stereo algorithm enables the system to acquire defect features from multiple dimensions such as geometric and optical properties, thereby achieving accurate identification and classification of various complex defects and maintaining high detection accuracy even in online high-speed detection scenarios.

[0015] 3. It can simultaneously generate the bounding box geometric coordinates and pixel-level geometric contours of the defect region, with each pixel labeled with a confidence probability, which are then used to determine whether it belongs to the detected defect region. Furthermore, through a multi-task deep learning architecture and a multimodal input data segmentation network, it can more accurately determine the defect location, shape, size, and spatial distribution. This refined defect localization capability provides reliable data support for subsequent process optimization and quality control. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of an online detection method for plating defects based on machine vision proposed in this invention. Figure 2 This is a schematic diagram of the structure of an online detection system for ingot plating defects based on machine vision proposed in this invention; Figure 3 This is a flowchart of the multi-task deep learning architecture in Embodiment 1 of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figures 1 to 3 This invention provides a machine vision-based online detection method and system for ingot plating defects. The technical solution is as follows, please refer to... Figure 1 This is a flowchart illustrating the steps of an online detection method for plating defects based on machine vision proposed in this invention; it includes the following steps: Using a camera array equipped with polarizing filters, multi-view continuous images of the ingot plating layer under different lighting conditions and polarization states are acquired; the multi-view continuous images are preprocessed to obtain an image data stream; Feature extraction is performed on the image data stream, including scale space extremum detection, key point localization, dominant direction allocation, and generation of high-dimensional descriptors; the extracted features are matched and clustered with a preset dictionary to identify potential defect areas in the plating layer; for the potential defect areas, refined multi-angle image acquisition is triggered, and the surface normal and depth information of the plating layer are reconstructed using a self-calibrated photometric stereo algorithm to obtain the feature vector to be detected; A CNN architecture with fused polarization-aware convolutional kernels is applied, and a dedicated layer for polarization feature extraction is embedded in the network to perform defect detection. The defect detection involves inputting the feature vector to be detected into a pre-trained model for reconstruction, calculating the error between the original vector and the reconstructed vector, and classifying defects based on the error. This approach traces the root cause of defects and provides support for process optimization.

[0019] Reference Figure 2 This is a schematic diagram of the structure of an online detection system for plating defects based on machine vision proposed in this invention; it includes modules for data acquisition and preprocessing, potential defect area identification, refined feature vector generation, defect detection and analysis, and defect root cause analysis and optimization. Example

[0020] This embodiment aims to elaborate on a machine vision online inspection system that combines polarization imaging, deep learning, and multi-task learning to achieve high-precision, real-time identification, classification, and source tracing of defects in ingot plating.

[0021] First, deploy an array of at least 4-6 industrial-grade high-resolution cameras. These cameras are arranged in a ring or fan shape around the top of the ingot plating conveyor belt, ensuring that the plating surface is captured from different angles, for example, one camera every 60 degrees or 45 degrees. Each camera lens is equipped with a switchable polarizing filter, such as a linear polarizer, which can be rapidly rotated to four angles: 0°, 45°, 90°, and 135°, and can be quickly switched via a precision mechanical device or a liquid crystal adjustable polarizing filter.

[0022] The system employs a programmable LED light source array arranged in an orderly manner to provide at least three different lighting conditions, such as uniform diffused light, directional light at a specific angle, and high-contrast oblique light, to simulate various lighting scenarios that may be encountered in actual production. These lighting scenarios have a specific impact on the coating acquisition results. The brightness of the LED light source is adjustable, and it will quickly respond to the brightness adjustment command after issuing the command.

[0023] Furthermore, all cameras and light sources are precisely synchronized via an industrial controller. When the plating layer passes through the detection area, the photoelectric sensor triggers image acquisition, ensuring clear, blur-free continuous images are obtained even when the ingot is moving at high speed. The image acquisition frequency is set at 100-200 frames per second, forming a continuous set of processable images.

[0024] Assuming each spindle is 50cm long and the detection area is 10cm wide, six cameras are deployed at 100 frames per second, capturing images corresponding to polarization and illumination changes every second.

[0025] A dataset was constructed containing reflection-interference images and their corresponding reflection-free real-world scene images. This data was collected from actual production lines, specifically through comparisons of reflective and reflection-free images, achieved via special shading or polarization settings, or by simulating different reflection conditions using rendering software. During training, the model learns to convert reflective input images into "clean" reflection-free images. Considering the real-time requirements of online detection, the network model needs to be pruned and quantized to reduce computational overhead, and deployed on a high-performance GPU server to ensure that the dereflection processing time for each image is within 10-20 milliseconds.

[0026] The preprocessing steps include dynamic fusion and enhancement. The intensity of the multi-view images is normalized, and the weights are dynamically adjusted according to characteristics such as local gradient, contrast, and texture complexity for weighted fusion. Finally, adaptive histogram equalization is performed to optimize image quality.

[0027] Furthermore, the local gradient magnitude, local contrast, and texture complexity of each frame are precisely calculated and quantized, and the fusion weights of images under different lighting conditions are dynamically adjusted to form high-quality images. Then, based on these local characteristics, the fusion weights of images from different viewpoints are dynamically adjusted to obtain higher-quality synthetic images, thereby generating an optimized image data stream that can be used as input to the model.

[0028] By acquiring images from multiple perspectives, polarizations, and lighting conditions, the system comprehensively captures information about the plating layer on the ingot, effectively avoiding the missed detection problems of traditional methods. Simultaneously, a deep learning-based anti-reflection network intelligently eliminates interference from highly reflective surfaces, ensuring image clarity. Combined with dynamic fusion and enhancement techniques, the system effectively highlights defect features and suppresses background noise, providing high-quality data for subsequent accurate identification.

[0029] By intelligently identifying and prioritizing samples with low model prediction confidence, novel defects that differ significantly from the pre-defined defect dictionary, and regions with blurred pixel-level segmentation boundaries, these samples are submitted to the expert feedback loop. The annotation interface precisely annotates these high-value samples and confirms their defect types, while simultaneously updating the defect dictionary. This newly annotated data is then used for incremental training of the deep learning model, employing incremental learning to avoid "catastrophic forgetting." This achieves adaptive, continuous iteration, and optimization of model performance, enabling efficient handling of emerging, unknown, or marginal defect types.

[0030] Furthermore, image features are extracted through scale-space extremum detection, keypoint localization, dominant orientation assignment, and generation of high-dimensional descriptors. Subsequently, these features are matched and clustered with a pre-defined defect dictionary to identify potential defect regions.

[0031] The system compares the high-dimensional descriptor of each key point extracted from the current image with a pre-built defect feature dictionary. This dictionary stores typical feature descriptors of various known defect types. The descriptors are given in text form. By calculating the similarity between the descriptors, the system can find regions that highly match the defect features described in the dictionary.

[0032] For key points that successfully match defect features, the system performs cluster analysis. This cluster analysis identifies the overall outline and extent of the defect from discrete key point information. The system groups key points that are spatially close and have similar features into the same group, thus outlining a complete and continuous potential defect area, roughly determining the location of the defect. Then, it triggers refined multi-angle image acquisition of the current defect area, followed by refined recognition using a CNN model.

[0033] Images are acquired using at least three light sources. The direction and brightness of the light sources are optimized by photographing a reference object of known shape, and the camera response is calibrated. Based on different illumination intensities and light source parameters, the surface normals are reconstructed using the Lambert reflection model, and depth information is obtained through integration to generate the feature vector to be detected.

[0034] This step aims to generate a general defect area and enrich the multi-angle image data within the area. After determining the approximate range, the contour boundary is then finely identified to provide the best input for subsequent feature extraction. Dynamic fusion can improve the recall rate of subsequent defect identification, especially for the detection of fine scratches and dents.

[0035] Furthermore, referring to Figure 3This is a flowchart of the multi-task deep learning architecture in Embodiment 1 of the present invention. In the early layers of the CNN, a custom layer specifically designed for processing and fusing polarization features is embedded. This layer is responsible for converting the raw polarization measurements into more physically meaningful features, such as polarization degree maps and polarization angle maps, and fusing them with intensity image features. Specifically, a "polarization fusion convolutional block" is designed to receive feature maps from different modalities and perform weighted fusion.

[0036] High-quality, defect-free ingot coating images were collected as a training dataset. These images were then used to train a model based on either an autoencoder or streaming generation. The training objective was to enable the model to accurately reconstruct defect-free input images or feature vectors. The model learned the intrinsic representation, texture regularity, and polarization response characteristics of normal coating surfaces by optimizing the reconstruction loss.

[0037] Furthermore, during the detection phase, after data acquisition and preprocessing, the "feature vector to be detected" of the ingot plating image is input into a pre-trained model. The system calculates the pixel-level difference between the original input vector and the model's reconstructed vector, i.e., the reconstruction error. When the reconstruction error of a pixel or region exceeds a preset threshold, it is marked as an abnormal region. The threshold is set to the average value of the normal reconstruction error plus 3-5 times the standard deviation. For example, through testing on 1000 defect-free samples, the average value of the normal reconstruction error was found to be 0.02, and the standard deviation was 0.005. Therefore, the threshold can be set to 0.035. Any pixel with a reconstruction error exceeding 0.035 is judged as a potential anomaly.

[0038] Furthermore, for the aforementioned marked abnormal regions, multiple lightweight convolutional neural networks are used for refined defect classification. A strategy of training a separate binary neural network for each preset defect type is employed. Defect types include: uneven coating thickness, missing parts, surface scratches, pits, peeling, blistering, color difference anomalies, and oxide spots. For each defect type, thousands of images containing that type of defect are collected and input into the corresponding binary classification network for training. The training uses a combined loss function, which includes bounding box loss, targetability loss, and segmentation loss. Furthermore, the combined loss function also includes a unified IoU loss function, which optimizes localization by dynamically adjusting the bounding box scaling mechanism and a two-way weight allocation strategy, measuring the overlap and shape similarity between the predicted box and the ground truth box. This enables the system to output extremely accurate defect geometry information, including shape, size, and spatial distribution, thereby providing more scientific and reliable data support for subsequent process optimization and quality control.

[0039] This modular design avoids class imbalance problems. Each classifier can focus on identifying micro-features of a specific type. When a new defect type appears, only a new binary classifier needs to be trained, without retraining the entire large network.

[0040] The core of the multi-task deep learning architecture is a shared feature extraction backbone network. This network is configured to extract common and semantically rich feature representations from the "feature vector to be detected". The backbone network was chosen for its lightweight and efficient characteristics, allowing it to be pre-trained on massive image datasets to learn general visual features. It can then be fine-tuned using ingot plating defect data to adapt it to specific domain features.

[0041] Furthermore, multiple task-specific heads for bounding box localization and pixel-level semantic segmentation are connected above the output of the shared backbone network. These heads, based on shared feature representations, independently but collaboratively perform their respective recognition tasks. The bounding box localization head is responsible for outputting the geometric coordinates of the bounding box for each detected defect. It predicts the position and size of the bounding box using a regression loss, while simultaneously using a classification loss to determine whether the region actually contains a defect. The pixel-level semantic segmentation head outputs the pixel-level geometric contour of the defect region, i.e., a binary mask with the same resolution as the input image, and is trained using a pixel-level classification loss.

[0042] The key advantage of this multi-task architecture lies in its ability to simultaneously generate the bounding box geometry coordinates of detected defects and the pixel-level geometric contours of the defect region in a single, unified computation process. A single forward propagation provides a coarse location and precise shape of the defect. By describing the location and appearance of the defect and then analyzing the pixel-level semantic segmentation results, the geometry, size, and spatial distribution of the defect can be accurately determined, and the defect type can be identified accordingly.

[0043] Furthermore, the multimodal input data includes intensity images (traditional grayscale images reflecting surface brightness information), polarization maps reflecting surface roughness and material scattering characteristics, polarization angle maps reflecting surface normal direction or texture anisotropy, surface depth maps reconstructing three-dimensional height information from photometric stereo algorithms, directly reflecting the surface's undulations, surface normal maps reflecting the surface normal direction of each pixel, exhibiting extremely strong ability to identify microscopic geometric features, and penetrating imaging data such as near-infrared, terahertz, or ultrasonic data, used to detect defects inside the coating or on the subsurface.

[0044] The customized input layer extracts features from different modal data such as intensity, polarization, depth, and normal, and designs dedicated branches for penetrating imaging data, such as near-infrared and terahertz. The feature fusion module adopts a multi-level attention fusion mechanism, dynamically learning and fusing high-level semantic features from various modalities through splicing and cross-modal attention.

[0045] For each modality of data, an independent input branch is designed. For example, intensity images, polarization maps, and polarization angle maps can be directly used as independent channel inputs to the CNN, while depth maps, normal maps, and penetration imaging data undergo preliminary feature extraction through their respective convolutional layers.

[0046] These heterogeneous multimodal data are integrated through a customized input layer and feature fusion module to ensure that information from different modalities can be effectively utilized and complement each other. The feature fusion module enables the network to adaptively learn the dependencies between features from different modalities and dynamically allocate weights.

[0047] Features are learned and extracted from multimodal input data through an independent feature encoder. Specifically, the decoder consists of two independent heads: the first head is responsible for outputting a probability map of defect regions, which is a grayscale image of the same size as the input image. The value of each pixel represents the probability that the pixel belongs to a defect region, with a probability of 1 indicating that it belongs to a defect region; the second head specifically outputs a probability map of defect boundaries. This map emphasizes the edges of defect regions, with a probability greater than 0.8 indicating that it is suspected to belong to a defect region. In other words, the border of the defect region is marked according to the probability, and the boundary between the defect and the normal region is highlighted.

[0048] Furthermore, to ensure the network can perform both tasks with high quality simultaneously, the model is trained by jointly optimizing the pixel-level segmentation loss and the boundary prediction loss. The pixel-level segmentation loss measures the difference between the predicted segmentation mask and the ground truth mask, ensuring correct classification at the pixel level. The boundary prediction loss measures the similarity between the predicted boundary and the ground truth boundary, encouraging the predicted boundary to be closer to the ground truth boundary. The total loss function is a weighted sum of these two loss terms.

[0049] Through this joint optimization, the model is encouraged during training to not only distinguish between defective and normal regions, but also to capture and output the detailed contours of defects, thereby providing extremely accurate geometric information.

[0050] The integrated system can correlate detailed information about each defect, including its type, location, size, and occurrence time, with real-time process parameters and raw material batch data from the production line. When a defect is detected, potential problems in the production process can be quickly identified. By introducing an active learning mechanism, images that are uncertain or prone to errors in the model can be intelligently selected for manual annotation, thereby achieving rapid iteration and improvement of model performance with less manpower.

[0051] The integrated system automates data transmission and processing between modules. When a thin, linear anomaly is detected, with abrupt changes in the normal direction and an aspect ratio > 5:1, it is identified as a surface scratch; output: Scratch A, 1.2mm × 0.05mm, coordinates (X1, Y1) - (X2, Y2). When a circular, low-contrast area with radially changing normals and an area < 0.5mm² is detected, it is identified as a pit; output: Pit B, diameter 0.2mm / depth 0.03mm, coordinates (X3, Y3). When an abnormality in polarization / thickness or irregular patches is detected, it is identified as uneven coating; output: Area 5mm², thickness fluctuation 0.01mm, coordinates (X4, Y4). Other defect examples: Peeling D (2mm², irregular, X5, Y5); Bubble E (0.1mm, circular, X6, Y6). The integrated system uploads defect data to the central data platform in real time via MQTT, aligning it with real-time process parameters, raw material batches, and other data using timestamps. The system constructs a data-driven root cause analysis model, utilizing Granger causality tests to infer the causal relationship between process parameter changes and defects, and automatically generates process optimization suggestions.

[0052] This highly automated process reduces the detection and analysis process, which may have taken several minutes or even hours, to seconds or even sub-seconds, greatly improving the overall efficiency and response speed of the production line. It eliminates the delays caused by manual intervention and data conversion, and enables high-speed operation of the entire process from image acquisition to defect identification, classification and reporting, ensuring that the plating coating can be monitored in real time and accurately even in high-speed production.

[0053] This embodiment achieves comprehensive automated detection of defects in ingot-removed coatings. From data acquisition and preprocessing, from potential defect identification to refined feature extraction, and finally to intelligent defect detection, classification, and root cause analysis, this solution provides a comprehensive and forward-looking approach. The solution implemented in this embodiment improves the quality control level of ingot-removed coating products, reduces labor costs, and provides strong technical support for the construction of smart factories, ultimately promoting continuous optimization of production processes and enhancing industry competitiveness. Example

[0054] The challenge of this embodiment lies in the fact that the surface of the plating layer has non-planar and complex curved geometric features; and the defect types are more refined, including subtle or subsurface defects such as microcracks, internal oxidation, crystal anomalies, and interlayer peeling, which are difficult to identify by a single mode. The specific steps of the embodiment are as follows: The system deploys a high-resolution, 12-megapixel, 500fps global shutter industrial camera, coupled with a polarizing filter to capture images of intensity, degree of polarization, and polarization angle. It integrates a high-precision laser triangulation sensor with a Z-axis repeatability of ±2 micrometers, capable of identifying micro-dimples deeper than 10 micrometers. Some cameras are equipped with multispectral imaging units for detecting internal oxidation or subsurface delamination defects. The system employs a programmable, multi-angle, multi-band, variable polarization active light source array, dynamically adjusting illumination within 10 milliseconds to ensure optimal imaging.

[0055] Furthermore, after preprocessing, the raw data from each modality are precisely registered in a unified 3D coordinate system, with cross-modal registration errors controlled within 2 pixels. The core employs a 3D convolutional neural network for feature extraction and fusion, integrating a cross-modal attention mechanism to improve the signal-to-noise ratio by up to 25%. The system incorporates temporal analysis capabilities, connecting multi-view images captured at different time points into a single image to capture dynamic defects in the coating, allowing for a clearer identification of the root causes of these defects. An advanced deep learning artifact removal algorithm reduces the false detection rate caused by reflection artifacts to below 0.1%.

[0056] Based on fused 3D multimodal features, the system utilizes an advanced 3D semantic segmentation network to achieve accurate defect segmentation. The overall defect detection recall rate exceeds 98.5%, and the detection accuracy for micron-level cracks reaches 0.05 mm. The system can identify and classify various complex defects, with a key defect type classification accuracy exceeding 97%. High-precision 3D quantization is performed on the identified defects; for example, the depth measurement error for tiny pits can be controlled within ±5%. The system employs a multi-task learning paradigm, simultaneously completing 3D bounding box localization, pixel / voxel-level segmentation, and fine classification. The system deeply correlates defect data with real-time process parameters, successfully associating over 80% of typical defects with at least one key process parameter, and then inferring or determining the cause of the defects using historical data.

[0057] Once the defect is identified and located, the system associates its data, including type, location, size, and timestamp of occurrence, with real-time production parameters of the production line, such as plating solution temperature, current density, deposition time, raw material batch, and equipment operating status, thus linking the defect with the process flow and accurately controlling the source of the problem.

[0058] Furthermore, time-series data is analyzed, and a large amount of defect data and its associated production parameters are summarized. For example, if a specific type of pit is found to appear in large numbers within a certain period of time, and the associated data shows that the plating solution temperature rises abnormally during that period, it can be preliminarily inferred that the abnormal temperature is the root cause of the pit formation.

[0059] The system generates detailed defect reports and trend charts, visually displaying how defect types, frequency, and severity change over time and with production parameters. It also provides real-time feedback of the analysis results to production line operators and engineers, offering specific process optimization suggestions. For example, the system might display: "When the plating bath temperature exceeds the set threshold, the scratch rate on the plating surface increases significantly," and provide a clear maintenance reminder: "It is recommended to check the temperature control system." This embodiment expands the application of three-dimensional convolutional neural networks, enabling rapid localization of tiny and easily overlooked defects and outlining the defect area. The inclusion of this closed-loop feedback mechanism is key to achieving intelligent manufacturing and continuous quality improvement. The system achieves rapid detection and decision-making, achieving a 99% automatic rejection rate for serious defects. Furthermore, it can continuously improve its recognition capability by 5-10% through incremental learning, increasing the product qualification rate by 2-3 percentage points. The online defect identification and process optimization measures reduce the scrap rate, fully validating the system's effectiveness in improving quality control and optimizing production.

[0060] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A machine vision-based online detection method for plating defects in ingots, characterized in that, Includes the following steps: Using a camera array equipped with polarizing filters, multi-view continuous images of the ingot plating layer under different lighting conditions and polarization states are obtained; The multi-view continuous images are preprocessed to obtain an image data stream; Feature extraction is performed on the image data stream, and the extracted features are matched and clustered with a preset dictionary to identify potential defect areas in the plating layer. For the potential defect areas, refined multi-angle image acquisition is triggered, and the surface normal and depth information of the plating layer are reconstructed using a self-calibrated photometric stereo algorithm to obtain the feature vector to be detected. A CNN model with fused polarization-aware convolutional kernels is used for defect detection. The feature vector to be detected is input into the CNN model for reconstruction. The reconstruction error is calculated. Conventional visual features and polarization optical features are extracted simultaneously from multimodal input data. A dual-branch output structure is designed to jointly optimize the segmentation loss and boundary prediction loss. The segmentation result and boundary localization map of the defect region are output. The defect classification is obtained by combining the reconstruction error. The steps for performing defect detection using a CNN model with fused polarization-aware convolutional kernels specifically include: A CNN architecture is adopted, and a polarization-sensitive processing layer is embedded in the convolutional kernel of the CNN architecture to process polarization information, resulting in a CNN model with fused polarization-aware convolutional kernels. Anomalies are identified by learning the normal patterns and features of defect-free surfaces. When performing defect detection, the reconstruction error after the feature vector to be detected is reconstructed is calculated, and reconstruction errors higher than a preset threshold are marked as abnormal regions. For the abnormal regions, a lightweight convolutional neural network is used for defect classification. The defect classification includes uneven coating thickness, missing parts, surface scratches, pits, peeling, blistering, color difference anomalies, and oxide spots. A binary neural network is trained separately for each defect classification and works in conjunction with the CNN model with fused polarization-aware convolutional kernels. Trace the root cause of the defects and adjust the process flow in a timely manner.

2. The online detection method for ingot plating defects based on machine vision according to claim 1, characterized in that, The preprocessing steps for the multi-view continuous images specifically include: The multi-view continuous images are dynamically fused and enhanced to identify local characteristics of the images, including gradient magnitude, local contrast and texture complexity. The fusion weights of the multi-view continuous images are dynamically adjusted according to the local characteristics to generate an image data stream.

3. The online detection method for ingot plating defects based on machine vision according to claim 1, characterized in that, The specific steps for feature extraction from the image data stream include: A feature dictionary is constructed for a preset defect pattern; features are extracted from the image data stream, including scale space extremum detection, key point localization, dominant direction assignment, and generation of high-dimensional descriptors; the extracted features are matched and clustered with the preset dictionary to identify potential defect areas in the ingot coating; the surface normal is reconstructed using the Lambert reflection model, and depth information is obtained through integration to generate the feature vector to be detected.

4. The online detection method for ingot plating defects based on machine vision according to claim 1, characterized in that, The defect detection specifically employs a multi-task deep learning architecture, including: A shared feature extraction backbone network is configured to extract common feature representations from the feature vector to be detected; and multiple task heads for bounding box localization and pixel-level semantic segmentation, which perform their respective recognition tasks based on the shared features. In a single computation process, the bounding box geometric coordinates and geometric contours of the defect region are generated synchronously. The precise geometric shape, size and spatial distribution of the defect are determined by analyzing the pixel-level semantic segmentation results, and the defect is classified accordingly.

5. The online detection method for ingot plating defects based on machine vision according to claim 1, characterized in that, The multimodal input data includes: Intensity images, polarization maps, polarization angle maps, surface depth maps, surface normal maps, and penetration imaging data, including near-infrared, terahertz, and ultrasonic data, are integrated through a customized input layer and feature fusion module.

6. The online detection method for ingot plating defects based on machine vision according to claim 1, characterized in that, The steps for segmenting the defective region and creating its boundary location map are as follows: The CNN model with fused polarization-aware convolutional kernels is equipped with an independent feature encoder that simultaneously extracts conventional visual features and polarization optical features from multimodal input data. It includes dual output branches: the main branch generates a pixel-level semantic segmentation probability map of the defect region, and the auxiliary branch predicts the probability distribution of the defect boundary. By jointly optimizing the segmentation loss and the boundary prediction loss, it captures the fine-grained contour features of the defect and simultaneously outputs the defect segmentation result and the boundary localization map.

7. A machine vision-based online detection system for plating defects in ingots, characterized in that, include: Data acquisition and preprocessing module: Using a camera array equipped with polarizing filters, acquire multi-view continuous images of the ingot coating under different lighting conditions and polarization states; The multi-view continuous images are preprocessed to obtain an image data stream; Potential defect area identification module: extracts features from the image data stream, matches and clusters the extracted features with a preset dictionary, and identifies potential defect areas in the ingot plating layer; Refined feature vector generation module: For the potential defect area, it triggers refined multi-angle image acquisition and uses a self-calibrated photometric stereo algorithm to reconstruct the surface normal and depth information of the ingot coating to obtain the feature vector to be detected; Defect Detection and Analysis Module: A CNN model with fused polarization-aware convolutional kernels is used for defect detection. The feature vector to be detected is input into the CNN model for reconstruction. The reconstruction error is calculated. Conventional visual features and polarization optical features are extracted simultaneously from multimodal input data. A dual-branch output structure is designed to jointly optimize the segmentation loss and boundary prediction loss. The segmentation result and boundary localization map of the defect region are output. The defect classification is obtained by combining the reconstruction error. The steps for performing defect detection using a CNN model with fused polarization-aware convolutional kernels specifically include: A CNN architecture is adopted, and a polarization-sensitive processing layer is embedded in the convolutional kernel of the CNN architecture to process polarization information, resulting in a CNN model with fused polarization-aware convolutional kernels. Anomalies are identified by learning the normal patterns and features of defect-free surfaces. When performing defect detection, the reconstruction error after the feature vector to be detected is reconstructed is calculated, and reconstruction errors higher than a preset threshold are marked as abnormal regions. For the abnormal regions, a lightweight convolutional neural network is used for defect classification. The defect classification includes uneven coating thickness, missing parts, surface scratches, pits, peeling, blistering, color difference anomalies, and oxide spots. A binary neural network is trained separately for each defect classification and works in conjunction with the CNN model with fused polarization-aware convolutional kernels. Defect Root Cause Analysis and Optimization Module: Traces the root cause of defects and adjusts the process flow in a timely manner.

Images