Laminated iron surface defect detection method and device based on polarization-hyperspectral combined imaging

Abstract

The invention discloses a laminated iron surface defect detection method and device based on polarization-hyperspectral combined imaging, and the method comprises the steps: firstly, designing an illumination module employing a 30-degree incident angle, and avoiding overexposure and energy consumption of conventional diffuse reflection illumination; secondly, according to the characteristics of surface imaging of the laminated steel, a 0-degree mirror surface + 60-degree scattering double-channel parallel imaging framework is designed, and spectral information of the surface of the laminated steel can be fully obtained. And meanwhile, a neural network algorithm fusing a dynamic wave band selection module, an inverted residual three-dimensional convolutional neural network and an SE weighted fusion module is designed, so that high-speed and low-energy-consumption accurate detection of the surface defects of the laminated steel is realized, and the method has wide application market space and economic value.

Description

Technical Field

[0001] This invention relates to a method and apparatus for detecting defects on coated iron surfaces based on polarization-hyperspectral imaging. Background Technology

[0002] Coated steel is manufactured by hot-pressing a layer of PET or PP film onto a chrome-plated steel sheet, forming a "steel-plastic" structure that combines the strength of metal with the corrosion resistance of plastic. This makes it an ideal substrate for food and beverage cans. However, surface defects can cause damage, leaks, corrosion, and pressure loss, making surface inspection crucial. In actual production and inspection, the relatively smooth surface of coated steel makes it prone to overexposure due to mirror reflections. Furthermore, internal, deep defects are difficult to detect using traditional methods, leading to potential hazards in the coated steel process. Currently, the industry commonly uses chemical development, mechanical peeling, and high-speed spectroscopy to inspect coated steel surfaces. However, these methods suffer from low accuracy, high cost, and slow speed, failing to meet current needs for coated steel surface inspection. Summary of the Invention

[0003] This invention provides a method and apparatus for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging. First, an illumination module with a 30° incident angle is designed to avoid overexposure and energy consumption associated with traditional diffuse reflection illumination. Then, considering the characteristics of coated iron surface imaging, a 0° specular plane + 60° scattering dual-channel parallel imaging architecture is designed to fully acquire the spectral information of the coated iron surface. Simultaneously, a neural network algorithm integrating a dynamic band selection module, an inverted residual 3D convolutional neural network, and an SE-weighted fusion module is designed, achieving high-speed, low-energy, and accurate detection of defects on coated iron surfaces, possessing broad application market potential and economic value.

[0004] The technical solutions adopted in this invention are as follows: A method for detecting defects on coated iron surfaces based on polarization-hyperspectral imaging includes the following steps: A linearly polarized beam of 420 nm–1000 nm is emitted onto the coated iron surface at a fixed incident angle of 30°. The imaging module simultaneously receives 0° specular reflected light and 60° scattered light. After polarization beam splitting and tunable filtering, hyperspectral-polarized cubic data of the coated iron surface are obtained in the same exposure cycle. Dynamic band selection is performed on the cube data to retain defect-sensitive bands, resulting in simplified data; The simplified data is input into a three-dimensional convolutional neural network to extract spectral-polarization joint features. The joint features are then separated into spatial and spectral branches. The SE module performs weighted fusion of the two branches to obtain the fused features. Based on the fusion features, the defect probability and the depth index used to distinguish between the upper, middle and lower parts of the film are output pixel by pixel to complete the detection of defects on the coated iron surface.

[0005] Furthermore, when performing dynamic band selection on the hyperspectral-polarization cube data, band weights are generated through 1×1×32 convolution and Softmax operation, and the 16 bands with the highest weights are retained to form the simplified data.

[0006] Furthermore, the three-dimensional convolutional neural network is an inverted residual structure, which sequentially includes 1×1×1 dimensionality increase, 3×3×3 grouped depthwise convolution, and 1×1×1 dimensionality reduction, used to extract the spectral-polarization joint features.

[0007] Furthermore, the SE module first performs global average pooling on the spatial branch and the spectral branch respectively, then generates channel weights through fully connected compression-excitation, and finally adds the weighted features of the two branches pixel by pixel to obtain the fused features.

[0008] Furthermore, the defect probability and depth index are output in parallel by the same decoding head, wherein the depth index includes three categories: above the membrane, in the membrane, and below the membrane.

[0009] Furthermore, the 420 nm–1000 nm linearly polarized beam is formed by a mixed array of white LEDs and near-infrared LEDs, followed by a collimating rod lens, a linear polarizer, and a cylindrical lens.

[0010] This invention discloses a device for detecting defects on coated iron surfaces based on polarization-hyperspectral imaging, comprising, along the optical path: An illumination module for emitting a linearly polarized beam of 420 nm–1000 nm onto the coated iron surface at a fixed incident angle of 30°; The imaging module is used to simultaneously receive 0° specular reflected light and 60° scattered light. After polarization beam splitting and tunable filtering, hyperspectral-polarization cubic data of the coated iron surface are obtained within the same exposure cycle. A dynamic band selection module, which is connected to the imaging module, is used to dynamically select bands for the cube data and output simplified data that retains defect-sensitive bands. A three-dimensional convolutional neural network module is data-connected to the dynamic band selection module, used to extract spectral-polarization joint features from the simplified data, and separate the joint features into spatial branches and spectral branches; The SE module is data-connected to the three-dimensional convolutional neural network module and is used to perform weighted fusion of the spatial branch and the spectral branch and output the fusion feature. The decoding head module is connected to the SE module for data transmission and is used to output the defect probability pixel by pixel based on the fusion features and the depth index used to distinguish between the upper, middle and lower parts of the film, thereby completing the detection of defects on the coated iron surface.

[0011] Furthermore, the lighting module includes a mixed array of white LEDs and near-infrared LEDs arranged sequentially along the light emission direction, a collimating rod lens, a linear polarizer, and a cylindrical lens. The linear polarizer has an extinction ratio ≥1000:1 and can rotate ±90°.

[0012] Furthermore, the imaging module includes a 50:50 beam splitter and two parallel channels, which correspond to the reception of 0° mirror reflection light and 60° scattered light, respectively. Each channel is equipped with a lens, a polarizing beam splitter, a tunable filter, and an imaging surface.

[0013] Furthermore, it also includes an encoder synchronization module, which is used to trigger the illumination module and the imaging module to keep the emission of the linearly polarized beam and the acquisition of cube data in line frequency synchronization.

[0014] The present invention has the following beneficial effects: (1) Using 30° oblique incident ray polarized illumination effectively suppresses overexposure caused by specular reflection on the coated iron surface, ensuring sufficient dynamic range of imaging without additional energy consumption.

[0015] (2) The 0° mirror reflection channel captures the film surface morphology, and the 60° scattering channel simultaneously acquires the information under the film. A single shooting can simultaneously expose surface scratches and underlying layer defects, reducing the number of work stations and the number of shooting cycles.

[0016] (3) By using dynamic band selection, only the bands most sensitive to defects are retained, which preserves the spectral fingerprint and significantly reduces the computational load of subsequent networks, making it suitable for high-speed real-time processing on production lines.

[0017] (4) The inverted residual three-dimensional convolution jointly extracts spectral-polarization features in the "height×width×band" space. The SE module adaptively weights spatial and spectral information, avoiding manual parameter adjustment and improving detection stability.

[0018] (5) The network outputs the defect probability per pixel and gives the categories of on-film, in-film and under-film. No additional physical film peeling or secondary scanning is required, which can guide subsequent sorting and process backtracking.

[0019] (6) The entire detection process is based on optical imaging and algorithms, without the need for chemical development, stripping or tracers, saving operating costs and meeting the requirements of green manufacturing. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a visual inspection structure for coated iron surfaces.

[0021] Figure 2 This is a schematic diagram of the lighting module structure.

[0022] Figure 3 This is a schematic diagram of the imaging module structure.

[0023] Figure 4 A schematic diagram of the module structure for dynamic band selection.

[0024] Figure 5 This is a schematic diagram of the structure of a three-dimensional convolutional neural network module.

[0025] Figure 6 This is a schematic diagram of the SE module structure. Detailed Implementation

[0026] The invention will now be further described with reference to the accompanying drawings.

[0027] The coated steel process involves laminating a polyester film onto a chrome-plated steel substrate under high temperature and pressure, combining the strength of metal with the corrosion resistance of plastic. Existing visual inspection solutions are mostly based on RGB or monochrome line scanning imaging, utilizing only intensity information. They are severely affected by mirror reflections from the metal surface, resulting in insufficient detection rates for defects such as inclusions under the film, micro-scratches, and delamination. Furthermore, imaging on the metal surface is prone to spectral distortion due to oversaturation, and the film-substrate interface reflections and defect signals are mixed, all of which negatively impact the effectiveness of visual inspection of coated steel. This invention addresses the problems of "mirror reflection interference, missed detection of deep defects, and low detection efficiency" in the surface defect detection of coated steel. The specific solution of this invention is as follows: I. Construction and operation of the "polarization-hyperspectral" imaging device.

[0028] 1) Overall device architecture setup; The structure of the dual-channel imaging scheme is as follows: Figure 1 As shown, the core of the device consists of three parts: an illumination module, an imaging module, and an encoder synchronization module. These modules are connected collaboratively according to the optical path and data flow direction. Optical path connection: The light output direction of the illumination module is directly facing the coated iron surface, and the incident angle is fixed at 30°; the lens of the imaging module is aligned with the reflected light path of the coated iron surface, and the reflected light is split into two paths by a 50:50 beam splitter, corresponding to the 0° mirror reflection light and the 60° scattered light reception respectively. Signal synchronization: The encoder synchronization module is electrically connected to the lighting module and the imaging module respectively, triggering the light source of the lighting module to flicker and the imaging module to expose, ensuring that the two maintain line frequency synchronization according to the movement speed of the coated iron roll, and avoiding imaging misalignment; Functional division of labor: The illumination module provides a stable polarization-hyperspectral light source, the imaging module acquires hyperspectral-polarization data, and the encoder synchronization module ensures the stability of the device operation.

[0029] 2) Design of the lighting module; The lighting module is one of the decisive factors in whether the entire system can "see accurately and run stably." The structure of the lighting module is as follows: Figure 2 As shown, it consists of an LED array 21, a collimating rod lens 22, a linear polarizer 23, a cylindrical lens 24, and a temperature control module 25.

[0030] (1) Component selection and assembly; LED array 21: It adopts a mixed structure of white LEDs and near-infrared LEDs, which are evenly distributed alternately to output a line beam in the 420-1000 nm band. The beam has high uniformity, controllable polarization and strobe synchronization, and the incident direction is fixed at 30°. Collimating rod lens 22: mounted on the light-emitting side of the LED array, matching the divergence angle of the LED array, converting "large-angle divergent light" into "narrow-angle parallel light" to ensure beam directionality; Linear polarizer 23: Installed behind the collimating rod lens, with an extinction ratio ≥1000:1, it can be rotated ±90° by a mechanical structure. It is used to find the optimal cross-polarization angle during the adjustment phase and to reduce the interference of mirror reflection. Cylindrical lens 24: mounted behind the linear polarizer, with a focal length matching the width of the coated iron roll, instantly focusing the parallel beam into a thin line in the width direction of the roll, adapting to the requirements of line scanning imaging; Temperature control module 25: Temperature control module 25 monitors the ambient temperature of the lighting module in real time through a temperature sensor, and keeps the ambient temperature of the lighting module within a suitable range to avoid the LED heating up and affecting the stability of the light source.

[0031] (2) Light source output control The encoder synchronization module triggers a signal to control the LED array to operate in "strobe synchronization" mode. The strobe frequency is consistent with the exposure frequency of the imaging module, ensuring that a stable line beam with polarization of 420-1000 nm is output during each exposure.

[0032] 3) Design of the imaging module.

[0033] The imaging module adopts a "polarization-hyperspectral" design concept, including a 50:50 beam splitter and two parallel channels. The two parallel channels respectively receive 0° specular reflection and 60° scattered light. Each channel includes a lens 31, a polarization beam splitter 32, a tunable filter 33, and an imaging surface 34. The polarization beam splitter acquires the polarized light reflected from the coated iron surface, and the tunable filter is used to acquire the unique continuous characteristic spectrum within a specific spectral range of the target, ultimately obtaining the hyperspectral characteristics of the coated iron. Its structure is as follows: Figure 3 As shown.

[0034] When imaging a coated iron surface, the target light first passes through a 50:50 beam splitter, splitting the light into two parts. Since the incident angle of the illumination system in this invention is set to 30°, the peak direction of light scattering should be 60°. Therefore, the two optical channels in the imaging system receive light at 0° and 60° respectively. After entering each channel, the light is further processed by a metasurface polarization beam splitter to generate four sub-images at 0°, 45°, 90°, and 135°. The core feature of hyperspectral imaging is that it takes pictures sequentially in multiple very narrow bands and then stitches them together to form a "spectral curve." The light generated by the polarization beam splitter passes through a liquid crystal tunable filter, allowing the system to continuously switch between multiple narrow bands within tens of microseconds. In this invention, a liquid crystal tunable filter is used to image a specific color light of approximately 2nm width each time, ultimately forming 32 samples of the same pixel. Therefore, with the help of a dual-channel, four-way polarization beam splitter and a liquid crystal tunable filter, 2×4×32=256 hyperspectral images of the coated iron surface can be obtained with each exposure.

[0035] II. High-speed defect detection network for coated iron.

[0036] To accomplish the task of detecting defects on coated iron surfaces, this invention designs a high-speed defect detection network tailored to the characteristics of coated iron surfaces. This network employs a fusion network structure consisting of a dynamic band selection module, a 3D convolutional neural network module, and an SE-weighted fusion module, enabling efficient and accurate defect detection on coated iron surfaces. The data flow is as follows: Imaging module output → Dynamic band selection module (data simplification) → 3D convolutional neural network module (feature separation) → SE-weighted fusion module (feature fusion) → Decoding head (detection result).

[0037] (1) Dynamic band selection module; First, the 256 hyperspectral images obtained from the imaging module are stacked in the order of "band-polarization-light incident angle". Since each exposure generates data from two channels in a specific dimension... R ×1×8×32, where R It is the imaging resolution, and therefore ultimately we get a dimension of R A tensor of size ×1×256, denoted as M, is input into the dynamic band selector module. The module structure is as follows: Figure 4 As shown, the data is first processed using a 1×1×32 convolution, transforming it into... R A tensor of dimension 1×128 is generated, and then further processed using a 1×1×128 convolution to produce 32-dimensional band weights. These weights are then activated using the Softmax activation function to obtain 16 maximum values, denoted as K. Finally, for tensor M, the 16 most sensitive bands are selected based on K, meaning only the 16 most sensitive band data are retained, resulting in a tensor of dimension 1×128. RUsing a tensor of ×1×128 greatly reduces the computational resource consumption, and processing these 16 most sensitive bands can eliminate some interference, thereby improving the accuracy of the detection results.

[0038] (2) Three-dimensional convolutional neural network module; To rapidly detect surface defects in coated iron, this invention employs an inverted residual 3D convolutional neural algorithm as its backbone network. Its core feature is an inverted bottleneck structure of "low-dimensional upscaling - grouped depthwise convolution - low-dimensional reduction." Furthermore, based on the dimensionality characteristics of the coated iron hyperspectral imaging data, traditional 2D operations are extended to 3D along the spectral band direction, forming a continuous chain of operations: 1×1×1 upscaling, 3×3×3 grouped depthwise separable convolution, and 1×1×1 reduction. Its structure is as follows: Figure 5 As shown. The specific working process of this network is as follows: The 128-layer (16-band × 4-polarization × 2-camera) tensor data output from the dynamic band selector is first upscaled by 1×1×1 to... R aisle, R The imaging resolution is then processed into two sets of inverted residual blocks. Each set uses four 3×3×3 depth convolutions to capture high-order spectral-polarization correlation features in the "height×width×band" three-dimensional space. Subsequently, dimensionality reduction is performed using a 1×1×1 method, and low-dimensional skip connections are made with the features before dimensionality upscaling. This preserves the spectral fingerprint while avoiding gradient vanishing. The entire network only stacks two inverted blocks, resulting in fewer parameters and further improving computational efficiency. It also outputs both spatial and spectral data for subsequent feature fusion.

[0039] (3) Squeeze-Excitation Weighted Addition module; like Figure 6 The Squeeze-Excitation Weighted Addition (SE) module aims to enable the network to automatically weigh the importance of the spatial morphology and spectral fingerprint of the coated iron surface, and to fuse the two branches in a "weighted summation" manner. First, the spatial branch and spectral branch output from the inverted residual 3D convolutional neural network are processed... R Perform global average pooling on each of the two branches (×1×128) to obtain their respective... RThe system generates a 1×128 channel vector. Then, through a sequence of fully connected dimensionality reduction, ReLU activation function, fully connected dimensionality increase, and Sigmoid activation function, channel weight vectors w1 and w2 for each of the two branches are generated. This process is the "flattening-scoring" stage of Squeeze-Excitation. Finally, the weights are multiplied with the corresponding features channel by channel and then pixel by pixel Add is performed instead of traditional stitching. This achieves adaptive fusion of spatial-spectral features without increasing spatial dimension, greatly improving the accuracy of detection.

[0040] After the data is superimposed and fused, it is input into the decoding head and outputs the defect probability and depth index corresponding to each pixel through two channels. The defect probability refers to the probability that the corresponding pixel is a defect or background, while the depth index represents the type of defect, including on the film, in the film, and under the film, thereby realizing multi-level detection of surface defects in the coated iron process.

[0041] III. Network Training and Validation.

[0042] To verify the detection capability of the neural network used in this invention for defects on the coated iron surface, a high-speed production line dataset of defects on the coated surface and under the coating was constructed. Data was acquired using an 8k line scan camera, with each row collecting 8192×1 pixels, 32 bands (420-1000 nm), and 4 polarization angles × 2 cameras (0° specular + 60° scattering), forming a 256-layer hyperspectral-polarization cube. On-site annotation employed online electrolytic colorimetry and scanning electron microscopy cross-section verification, annotating each row with four types of defects: scratches, bubbles, inclusions, and exposed iron, along with depth labels (on / in / under the coating). A total of 800,000 training rows, 100,000 validation rows, and 100,000 sealed test rows were accumulated, totaling 24 million pixel-level samples. Training was performed using the AdamW optimizer, with a single-frame training time of 0.9 ms. Table 1 compares common surface defect detection algorithms with the method of this invention, where the Dice coefficient measures the similarity between image segmentation and defect detection, and F1 reflects the overall accuracy of the model. As can be seen, the method of the present invention has a high accuracy rate in the detection of surface defects in coated iron processes, and has significant advantages in terms of resource consumption and computational cost.

[0043] Table 1. Comparison of detection performance of different methods on coated iron surfaces Model Architecture Dice coefficient F1 (%) Single frame (ms) GPU memory (GB) Mask R-CNN 0.936 97.8 1.8 8.0 YOLOv3+FPN 0.968 97.1 1.2 3.5 Level 4 3D-CNN 0.986 98.3 1.1 6.0 Method of the present invention 0.979 98.3 0.9 3.2 The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging, characterized in that: Includes the following steps: A linearly polarized beam of 420 nm–1000 nm is emitted onto the coated iron surface at a fixed incident angle of 30°. The imaging module simultaneously receives 0° specular reflected light and 60° scattered light. After polarization beam splitting and tunable filtering, hyperspectral-polarized cubic data of the coated iron surface are obtained in the same exposure cycle. Dynamic band selection is performed on the cube data to retain defect-sensitive bands, resulting in simplified data; The simplified data is input into a three-dimensional convolutional neural network to extract spectral-polarization joint features. The joint features are then separated into spatial and spectral branches. The SE module performs weighted fusion of the two branches to obtain the fused features. Based on the fusion features, the defect probability and the depth index used to distinguish between the upper, middle and lower parts of the film are output pixel by pixel to complete the detection of defects on the coated iron surface.

2. The method for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging as described in claim 1, characterized in that: When performing dynamic band selection on hyperspectral-polarization cube data, band weights are generated through 1×1×32 convolution and Softmax operation, and the 16 bands with the highest weights are retained to form the simplified data.

3. The method for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging as described in claim 1, characterized in that: The three-dimensional convolutional neural network is an inverted residual structure, which includes 1×1×1 dimensionality increase, 3×3×3 grouped depthwise convolution and 1×1×1 dimensionality reduction, and is used to extract the spectral-polarization joint features.

4. The method for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging as described in claim 1, characterized in that: The SE module first performs global average pooling on the spatial branch and the spectral branch respectively, then generates channel weights through fully connected compression-excitation, and finally adds the weighted features of the two branches pixel by pixel to obtain the fused features.

5. The method for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging as described in claim 1, characterized in that: The defect probability and depth index are output in parallel by the same decoding head, wherein the depth index includes three categories: above the membrane, in the membrane, and below the membrane.

6. The method for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging as described in claim 1, characterized in that: The linearly polarized beam of 420 nm–1000 nm is formed by a mixed array of white LEDs and near-infrared LEDs, followed by a collimating rod lens, a linear polarizer, and a cylindrical lens.

7. A device for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging, characterized in that: Along the optical path, the following are included in sequence: An illumination module for emitting a linearly polarized beam of 420 nm–1000 nm onto the coated iron surface at a fixed incident angle of 30°; The imaging module is used to simultaneously receive 0° specular reflected light and 60° scattered light. After polarization beam splitting and tunable filtering, hyperspectral-polarization cubic data of the coated iron surface are obtained within the same exposure cycle. A dynamic band selection module, which is connected to the imaging module, is used to dynamically select bands for the cube data and output simplified data that retains defect-sensitive bands. A three-dimensional convolutional neural network module is data-connected to the dynamic band selection module, used to extract spectral-polarization joint features from the simplified data, and separate the joint features into spatial branches and spectral branches; The SE module is data-connected to the three-dimensional convolutional neural network module and is used to perform weighted fusion of the spatial branch and the spectral branch and output the fusion feature. The decoding head module is connected to the SE module for data transmission and is used to output the defect probability pixel by pixel based on the fusion features and the depth index used to distinguish between the upper, middle and lower parts of the film, thereby completing the detection of defects on the coated iron surface.

8. The device for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging as described in claim 7, characterized in that: The lighting module includes a mixed array of white LEDs and near-infrared LEDs arranged sequentially along the light emission direction, a collimating rod lens, a linear polarizer, and a cylindrical lens. The extinction ratio of the linear polarizer is ≥1000:1, and it can be rotated ±90°.

9. The device for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging as described in claim 7, characterized in that: The imaging module includes a 50:50 beam splitter and two parallel channels, which correspond to the reception of 0° mirror reflection light and 60° scattered light, respectively. Each channel is equipped with a lens, a polarizing beam splitter, a tunable filter, and an imaging surface.

10. The device for detecting defects on coated iron surfaces based on polarization-hyperspectral joint imaging as described in claim 7, characterized in that: It also includes an encoder synchronization module, which is used to trigger the illumination module and the imaging module to keep the emission of the linearly polarized beam and the acquisition of cube data in line frequency synchronization.

Images