A high-reflectivity metal zirconium strip surface defect detection method based on multi-source data

Abstract

The application discloses a high-reflective metal zirconium strip surface defect detection method based on multi-source data, relates to the technical field of metal zirconium strips, and comprises the following steps: collecting multi-source data of a metal zirconium strip surface to obtain metal zirconium surface data; inputting the metal zirconium surface data into a multi-modal encoder model to output metal zirconium surface fusion feature maps; inputting the metal zirconium surface fusion feature maps into an improved generative adversarial network model to output standard template feature maps; comparing the metal zirconium surface fusion feature maps with the standard template feature maps to obtain metal zirconium surface residual maps, decoding the metal zirconium surface residual maps, and obtaining metal zirconium surface defect positions, so that the defect detection of the metal zirconium surface is realized.

Description

Technical Field

[0001] This application relates to the field of zirconium metal ribbon technology, and in particular to a method for detecting surface defects in highly reflective zirconium metal ribbons based on multi-source data. Background Technology

[0002] As a core basic material in fields such as nuclear industry and high-end precision manufacturing, the surface quality of high-reflectivity zirconium strip directly determines the mechanical properties, corrosion resistance and service safety of the product. Surface scratches, pits, compositional segregation and other defects are key quality indicators that need to be strictly controlled in the production process. Efficient and accurate surface defect detection technology is an important prerequisite for ensuring that zirconium strip products are qualified to leave the factory.

[0003] Traditional methods for detecting surface defects in highly reflective zirconium strips often employ two-dimensional optical imaging. These methods involve using an industrial camera to capture images of the reflected light intensity on the zirconium strip surface and relying on traditional image processing algorithms such as grayscale thresholding and edge detection to identify and locate surface defects.

[0004] However, traditional methods rely on a single data source of two-dimensional optical imaging data, which is easily affected by the high reflectivity of zirconium strips. This makes it impossible to accurately and completely reflect the three-dimensional morphology and material composition of the zirconium strip surface, resulting in a large number of missed and false detections during defect detection. Ultimately, it is difficult to achieve high-precision and high-reliability detection of defects on the surface of highly reflective zirconium strips. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for detecting surface defects in highly reflective metallic zirconium ribbons based on multi-source data, thereby resolving the problems existing in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting surface defects in highly reflective metallic zirconium ribbons based on multi-source data, comprising the following steps: S1: Multi-source data acquisition is performed on the surface of the zirconium strip to obtain zirconium surface data; S2: Input the zirconium surface data into the multimodal encoder model and output the zirconium surface fusion feature map; S3: Input the fused feature map of the zirconium surface into the improved adversarial generative network model and output the standard template feature map; S4: Compare the fused feature map of the zirconium surface with the standard template feature map to obtain the zirconium surface residual map. Decode the zirconium surface residual map to obtain the location of the zirconium surface defects, thereby realizing the defect detection of the zirconium surface.

[0007] This invention provides a method for detecting surface defects in highly reflective metallic zirconium ribbons based on multi-source data, involving machine learning and deep learning technologies, which has the following beneficial effects: (1) This invention solves the problems of false positives and false negatives caused by high reflectivity and limited information dimension in traditional single two-dimensional optical imaging. It employs three dedicated encoder models to extract depth features from spatially registered two-dimensional reflected light intensity images, three-dimensional height maps, and spectral data cubes: the image feature encoder extracts surface visual texture features, the geometric feature encoder extracts three-dimensional morphological features unaffected by reflectivity, and the spectral feature encoder extracts material element composition features. By stitching and fusing the channel dimensions to form a unified multimodal fusion feature map, the essential complementarity of appearance, morphology, and composition information is achieved, constructing a comprehensive and robust surface state perception capability. This lays a solid feature foundation for subsequent accurate defect detection, significantly reducing the false positive rate caused by high reflectivity and the false negative rate of morphology and composition defects.

[0008] (2) This invention solves the industry problem of high cost of labeling defect samples and difficulty in pre-modeling due to their varied forms in industrial quality inspection. It introduces a dual-domain unsupervised adversarial generative network model, which is trained using only a set of defect-free qualified samples and an unlabeled set of test samples, without requiring any pixel-level or image-level defect labeling. Through adversarial training between the generator and discriminator, the generator learns to map the fused feature map of the test sample, which may contain various defects, to the feature distribution space of qualified samples, generating a unique defect-free standard template for each test sample. Therefore, without pre-defining or exhaustively listing all defect types, a high-quality detection reference system can be constructed, significantly reducing the data and time costs of model training and enabling rapid adaptation to new defects constantly emerging on the production line.

[0009] (3) Through key improvements to the basic adversarial generative network, the generation quality of defect-free templates and the accuracy of defect detection are significantly improved. This invention adopts a multi-scale deep residual recurrent generative adversarial network structure, which specifically solves the problems of traditional generators being unable to handle multi-scale defects, prone to mode collapse, and having low background fidelity of generated templates: The generator adopts a multi-scale structure with three parallel branches: basic scale, dilated convolution, and large convolution kernel, which can simultaneously capture local details, medium-range context, and large-range structural information, and achieve accurate repair of defects of different scales such as small scratches, medium-sized pits, and large-range component segregation; The introduction of recurrent consistency loss constraint forces bidirectional reversibility of inter-domain mapping, effectively preventing mode collapse and ensuring high fidelity between the generated template and the background structure of the input sample, fundamentally avoiding false detections caused by background differences; Combined with feature similarity loss, the structural consistency between the generated template and the real qualified sample is further constrained from the perception level, improving the authenticity of the generated template and the repair accuracy of defect areas. By assigning optimized weight coefficients to the adversarial loss, recurrent consistency loss, and feature similarity loss, the network can be stably trained and converged. The final generated defect-free standard template has high fidelity and strong generalization ability, which significantly improves the sensitivity and accuracy of subsequent residual comparison defect detection.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart illustrating a method for detecting surface defects in highly reflective metallic zirconium strips based on multi-source data, provided in an embodiment of this application. Detailed Implementation

[0013] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0014] The following describes an embodiment of the present application of a method for detecting surface defects in highly reflective metallic zirconium ribbons based on multi-source data, with reference to the accompanying drawings.

[0015] Figure 1 This is a schematic flowchart illustrating a method for detecting surface defects in highly reflective metallic zirconium strips based on multi-source data, provided in an embodiment of this application.

[0016] like Figure 1 As shown, the method includes the following steps: S1, Multi-source data acquisition is performed on the surface of the zirconium strip to obtain zirconium surface data.

[0017] In some embodiments, this step applies to both model training and online detection stages. In the training stage, a set of defect-free qualified zirconium strip samples and a set of unlabeled zirconium strip samples that may contain defects are prepared. The fixation and data acquisition process for the two types of samples is completely consistent. In the detection stage, a single zirconium strip sample is prepared for testing.

[0018] A vacuum adsorption combined with a specialized clamp is used to flatten and fix the zirconium strip sample onto a stable sample stage. During fixation, the detection area—the effective area on the zirconium strip surface where defect detection is required—is avoided to prevent obstruction or sample deformation. After fixation, a laser displacement sensor is used to perform multi-point measurements outside the detection area, such as at the four corners and the center, to verify that the sample surface flatness error is ≤0.1mm. To obtain multi-source surface data of the zirconium strip, a collaborative data acquisition device set is deployed. A high-resolution industrial area array camera with a resolution of at least 5 megapixels (e.g., 2448×2048) is selected, using global exposure mode, paired with a ring LED light source homogenized with a diffuser plate, a color temperature of 6500K, and a color rendering index >90. Data is acquired along a direction perpendicular to the sample surface, with a lens working distance of 150~250mm, obtaining a two-dimensional reflected light intensity image of the zirconium strip surface. This image completely records the texture features and grayscale information of the zirconium strip surface.

[0019] A laser line-scanning 3D profilometer is configured synchronously. Its laser emitter and the accompanying industrial camera are positioned at a pre-calibrated 30° fixed angle using a stereo calibration plate. The Z-axis measurement accuracy is ≤0.01mm, and the line laser width is ≤0.1mm. Before data acquisition, a high-precision calibration plate (such as a checkerboard) is used for joint calibration using multiple poses to obtain a high-precision transformation matrix between the two coordinate systems, ensuring that the spatial registration error between the two data sets is ≤1 pixel. During the sample stage's uniform movement at 5~10mm / s (or the scanner's movement, with the same speed and accuracy requirements as the sample stage), hardware synchronization triggering is used, with a time synchronization accuracy ≤1μs, and the profilometer scans the surface line by line. The acquired original 3D point cloud P is reprojected onto the area array camera coordinate system using the aforementioned calibration matrix, generating a 2D height map D that is spatially aligned pixel by pixel with a registration error ≤1 pixel.

[0020] Furthermore, a laser-induced breakdown spectroscopy detection device is configured, with a spectrometer wavelength range of 200~900nm and a spectral resolution ≤0.1nm. The laser beam emitted by the pulsed laser in the device is focused by a focusing lens and projected onto the sample surface, exciting and generating plasma. The emitted light signal of the plasma is collected by the spectrometer and decomposed into spectral signals. A high-precision motion control card achieves two-dimensional scanning synchronization between the laser focus and the sample surface: whenever the area array camera completes an exposure, a synchronization trigger signal is sent to the pulsed laser and the spectrometer. This allows the acquisition of the characteristic spectral curve of each spatial sampling point, and based on the position of the motion encoder at the trigger moment, a spectral data cube S that is spatially strictly registered with the aforementioned image is constructed.

[0021] The host computer performs synchronous calibration and control of time and position to ensure that the two-dimensional reflected light intensity image, three-dimensional contour point cloud data and spectral data cube of the same metal zirconium strip surface area are spatially corresponding pixel by pixel during a single scanning operation, with a time deviation of no more than 1µs.

[0022] After spatially registering the two-dimensional reflected light intensity image, the three-dimensional contour point cloud data, and the spectral data cube, we obtain two-dimensional reflected light intensity images with the same image height H and width W. Single-channel pixel values ​​characterize the grayscale and texture information of the zirconium ribbon surface reflection; a two-dimensional height map is generated from the three-dimensional point cloud through coordinate pixel mapping. Each pixel corresponds to the relative height information of a point on the zirconium ribbon surface; spectral data cube , where L is the total number of spectral acquisition channels, typically ranging from 2048 to 4096.

[0023] The two-dimensional reflected light intensity image Using a single-channel grayscale image format, the mathematical expression is as follows: The height H and width W represent the pixel resolution of the image, determining the overall spatial detail of the image. The third dimension represents a single-channel attribute. The pixel value I(i,j) corresponding to any pixel coordinate position (i,j) in the image is a scalar value, typically ranging from 0 to 1 after normalization. Physically, it represents the intensity of visible light reflected from the corresponding spatial point on the surface of the zirconium strip to the camera. Higher pixel brightness indicates greater surface reflection intensity, while lower brightness indicates weaker reflection intensity. This image mainly carries information on macroscopic textures, scratches, stains, and other appearance defects on the zirconium strip surface.

[0024] The two-dimensional height map Generated from 3D contour point cloud data, each sampling point within the original point cloud contains complete 3D coordinate information (X, Y, Z). Data transformation is performed using a coordinate-pixel mapping method, as expressed in the following formula: ; The goal of this transformation process is to convert disordered, unstructured 3D point cloud data into a regular meshed 2D height map aligned with the 2D image space. M represents the total number of point cloud points. First, the output mesh is defined, and the spatial resolution of the output height map is preset, namely the height H and the width W. This resolution is consistent with the resolution of the synchronously acquired two-dimensional reflected light intensity image I to ensure spatial alignment during subsequent feature fusion.

[0025] Determine the spatial extent of the 3D point cloud on the XY horizontal plane, and calculate the minimum and maximum X and Y coordinate values ​​of all points in the point cloud: ; ; in, The minimum value of all X coordinates in the point cloud. The maximum value of all X coordinates in the point cloud. The minimum value of all Y coordinates in the point cloud. Let be the maximum value of all Y coordinates in the point cloud. `min{}` and `max{}` represent the mathematical operators for finding the minimum and maximum values ​​in the set, respectively. Let x be the x-coordinate of the m-th point in the point cloud. Let y be the Y coordinate of the m-th point cloud point.

[0026] Establish a linear mapping from point cloud (X,Y) coordinates to heightmap pixel (i,j) coordinates: ; ; Here, `floor()` is a floor function, `i` is the row index of the pixel, `j` is the column index of the pixel, and `(i,j)` are the calculated integer pixel coordinates within the interval [0,H-1] and [0,W-1]. This mapping normalizes the spatial distribution of the point cloud and completely fills it into a preset H×W grid, where H is the height and W is the width. It should be noted that the Y-axis coordinate mapping is reversed here to match the difference between the image coordinate system (origin at the top left corner) and the point cloud coordinate system (origin at the bottom left corner), ensuring a one-to-one correspondence between the height map and the pixel coordinates of the two-dimensional reflected light intensity image.

[0027] Pixel value assignment: Traverse all 3D points For each point, the corresponding target pixel position (i,j) is calculated using the mapping formula described above. Since multiple 3D points will map to the same pixel (i,j), the Z-coordinate height values ​​of all points corresponding to that pixel need to be aggregated to obtain a unique pixel height value. Before aggregation, statistical methods (such as the 3σ criterion) are used to remove outliers in the Z-coordinate of the point cloud corresponding to that pixel to avoid noise points affecting the accuracy of the height value. Let the set of indices of all points mapped to pixel (i,j) be denoted as . The formula for calculating the pixel height value is: ; Where D(i,j) represents the pixel value located in the i-th row and j-th column of the two-dimensional heightmap D. Representative set The total number of three-dimensional points contained within. This represents the Z-coordinate value of the m-th point in the point cloud. A set of indices representing points in a point cloud.

[0028] After assigning pixel height values, some pixels (i,j) may not have a corresponding 3D point mapping, i.e., the index set... Empty pixels create data holes in the height map. To obtain a complete and continuous 2D height map, interpolation is performed on the hole pixels. An appropriate interpolation algorithm is used based on the hole area. If the area of ​​consecutive holes exceeds a certain proportion of the detection area, such as 5%, the data in that area is considered invalid and needs to be re-acquired. After the above processing, the required 2D height map D is finally generated. The scalar value D(i,j) at each pixel (i,j) in the figure can accurately represent the relative or absolute height information of the corresponding spatial region on the surface of the zirconium strip, and the spatial dimensions H and W are strictly aligned with the other modal data, thus completing the entire process of coordinate pixel mapping transformation from a 3D point cloud to a 2D height map.

[0029] It's important to note that 3D point clouds, composed of numerous disordered 3D coordinates, represent a completely different data format from 2D images. This presents challenges related to data structure heterogeneity and spatial alignment, making direct pixel-level feature fusion impossible. Converting the disordered 3D point cloud into a 2D height map, precisely aligned with the 2D image, via coordinate pixel mapping effectively solves these problems. This conversion process acts as a data bridge, resampling and normalizing the unstructured geometric information characterizing the 3D surface morphology into a structured image—the height map—with the same spatial resolution (H, W) and precise location correspondence as the visible light texture image. Mapping the height values ​​(Z-coordinates) of the 3D point cloud to the 2D height map, each pixel value represents the relative or absolute height of the metal surface at that point. This value directly encodes the 3D morphology and microscopic geometric features of the surface. For example, a dented scratch or pit will cause the height value of a local pixel to be significantly lower than the surrounding area; a raised weld scar or material accumulation will cause the height value to be significantly higher; and surface smoothness, curvature, or warping will manifest as a gradient change pattern of height values ​​over a large area. This geometric morphology information cannot be fully captured by simple two-dimensional grayscale images (which only record the intensity of reflected light), especially for highly reflective metal surfaces. Two-dimensional images are prone to losing the true surface texture due to overexposure or specular reflection. However, three-dimensional height measurement, based on triangulation or structured light principles, is unaffected by surface reflectivity and can stably provide reliable geometric information. This allows for the reliable detection of defects that primarily cause changes in three-dimensional morphology, such as deep scratches, pits, protrusions, and indentations, enhancing the robustness of the detection.

[0030] The spectral data cube S is mathematically expressed as: The first two dimensions of this data, H W represents the spatial dimension, maintaining strict physical registration with the aforementioned reflected light intensity image and two-dimensional height map. The same pixel coordinate (i,j) corresponds to the same actual sampling point on the zirconium strip surface. The third dimension, L, represents the spectral dimension, indicating the total number of spectral channels acquired. The value of the spectral dimension depends on the hardware specifications of the spectral acquisition equipment used. For any spatial position (i,j) within the spectral data cube, there exists a one-dimensional spectral feature vector of length L. This vector contains complete spectral curve data, and each element in the vector... The spectral emission intensity detected at the len-th specific wavelength after the material at the sampling point (i,j) on the zirconium ribbon surface is excited.

[0031] It should be noted that using a spectral data cube as the data source is problematic. For highly reflective zirconium ribbons, strong specular reflection can severely interfere with traditional optical imaging, leading to the loss of texture and color features. However, the spectral cube based on laser-induced breakdown spectroscopy (LIBS) technology directly reflects the intensity of a specific wavelength emitted by excited plasma at each spatial point. This information is determined by the material's elemental composition and chemical state, and is completely unaffected by surface light reflection characteristics. Therefore, it can uniquely identify defects in the material itself, such as microscopic contamination, compositional segregation, oxide layers, or uneven heat treatment, even when the appearance and morphology have not changed significantly. This complements two-dimensional reflected light intensity images and two-dimensional height maps, jointly constructing a comprehensive and robust perception capability for surface conditions.

[0032] The two-dimensional reflected light intensity image, the two-dimensional height map, and the spectral data cube all maintain a uniform spatial pixel size H. W enables precise alignment of spatial dimensions, allowing for simultaneous joint observation of the appearance features, three-dimensional morphology, and material composition of the same surface area of ​​a zirconium strip. Each pixel in the reflected light intensity image represents the grayscale reflectance intensity of the surface, and each pixel in the two-dimensional height map represents the spatial height value of the surface. The spectral data cube corresponds to a complete spectral feature vector at each spatial pixel position. This clear and differentiated data dimension and physical definition is an important foundation for subsequent multimodal targeted feature encoding and feature fusion.

[0033] The three types of spatially registered data are standardized to ensure consistent data distribution input to the multimodal encoder model. For the two-dimensional reflected light intensity image: pixel values ​​are linearly normalized to the range of 0-1, and denoising is performed using a Gaussian kernel; for the two-dimensional height map: planar correction is first performed to remove overall tilt, and then height values ​​are normalized to the range of 0-1; for the spectral data cube: background subtraction, wavelength calibration, and intensity normalization are performed sequentially to eliminate the effects of environmental noise and equipment drift. The final zirconium surface data includes the two-dimensional reflected light intensity image, the two-dimensional height map, and the spectral data cube.

[0034] S2, input the zirconium surface data into the multimodal encoder model, and output the zirconium surface fusion feature map.

[0035] In some embodiments, after obtaining the spatially aligned two-dimensional reflected light intensity image I, the two-dimensional height map D, and the spectral data cube S, they need to be mapped to a high-dimensional feature space using three structure-specific encoder models. The multimodal encoder model includes an image feature encoder. Geometric feature encoder and spectral feature encoder : Image feature encoder : The input layer receives a batch of two-dimensional reflected light intensity images I as input tensors, with the batch size typically set to 32. The input tensor dimension is... Where 32 represents the batch size, H and W are the original height and width of the two-dimensional reflected light intensity image, respectively, and 1 represents a single grayscale channel. This input tensor first passes through an initial convolutional layer, which uses a two-dimensional convolution operation with 64 kernels, a size of 7×7, and a stride of 2. After the convolution operation, a batch normalization layer and a ReLU activation function are sequentially connected, resulting in an output feature map with a shape of (32... H / 2 W / 2 64), where H / 2 and W / 2 are the spatial dimensions obtained after the input image height and width are convolutional operations with a stride of 2, respectively, and 64 is the number of output channels of this convolutional layer. Subsequently, this feature map passes through a 3×3 max pooling layer with a stride of 2. The pooling operation only downsamples the spatial dimensions and does not change the number of channels, resulting in an output feature map shape of (32). H / 4 W / 4 64). Subsequently, the feature map undergoes deep feature extraction through 3 residual stages and a total of 9 pre-activated residual blocks. Each pre-activated residual block is constructed in the order of batch normalization → ReLU activation → 3×3 convolution → batch normalization → ReLU activation → 3×3 convolution. The kernel size within each residual block is 3×3, and each residual block contains residual skip connections to alleviate the gradient vanishing problem during deep network training. The specific configuration is as follows: Residual stage 1 contains 3 pre-activated residual blocks. The first residual block uses a convolution with a stride of 2 for spatial downsampling, while the latter two residual blocks use a convolution with a stride of 1 without spatial downsampling. The output channel number for this stage is set to 128. After processing by stage 1, the feature map shape becomes (32...). H / 8 W / 8 128); Residual stage 2 contains 4 pre-activated residual blocks. The first residual block uses a convolution with a stride of 2 for spatial downsampling, while the latter three residual blocks use a convolution with a stride of 1 without spatial downsampling. The number of output channels in this stage is set to 256. After processing in stage 2, the feature map shape becomes (32 H / 16 W / 16 256); Residual stage 3 contains two pre-activated residual blocks. Both residual blocks use convolution with a stride of 1. No spatial downsampling is performed; only the number of channels is adjusted through convolution operations. The output channel number of this stage is set to 512. After processing by stage 3, the feature map spatial size remains (32). H / 16 W / 16), the number of channels becomes 512. The above network uses four 2x downsampling steps (initial convolution stride 2, max pooling stride 2, residual stage 1 stride 2, residual stage 2 stride 2) to unify the feature map space size to 1 / 16 of the original image, consistent with the output size of the geometric feature encoder and spectral feature encoder, laying the foundation for subsequent channel dimension fusion of multimodal features. Finally, Output image feature map Its tensor shape is (32, H′, W′, 512), that is ,in , , The number of feature channels is [value]. This network effectively mitigates gradient vanishing through residual connections and can stably extract appearance features from low-level texture to high-level semantics on the surface of zirconium strips, providing reliable appearance feature support for subsequent multimodal feature fusion.

[0036] Geometric feature encoder Specifically designed for extracting geometric features of a surface's three-dimensional topography; the input is a two-dimensional height map. To enhance the response to geometric abrupt changes such as edges and steps, input augmentation is first applied to D, and its first-order Sobel gradient maps in the X and Y directions are calculated in parallel to obtain... and The original height map D is then concatenated with the two gradient maps along the channel dimension to form a 3-channel augmented input tensor. (Where 32 is the preset batch size, consistent with other encoders). This encoder A U-Net encoder structure consisting of four sequentially connected downsampling blocks is adopted. Each downsampling block consists of two consecutive convolutional units and a max-pooling layer. The execution order of each convolutional unit is: 2D convolutional layer → batch normalization layer → ReLU activation function. The specific configuration is as follows: Downsampling block 1 receives 3 channels of input. Its first convolutional unit uses 32 3×3 convolutional kernels with a stride of 1 and edge padding of 1. The second convolutional unit also uses 32 3×3 convolutional kernels with a stride of 1 and padding of 1. Then it passes through a 2×2 window max-pooling layer with a stride of 2. The output feature map shape of this block is (32, H / 2, W / 2, 32). Downsampling block 2 has 32 input channels. Its two convolutional units each have 64 convolutional kernels, each kernel size is 3×3, each stride is 1, and each padding is 1. The pooling layer parameters are the same as above, and the output feature map shape of this block is (32, H / 4, W / 4, 64). Downsampling block 3 has 64 input channels, and its two convolutional units each have 128 kernels, with kernel size, stride, and padding remaining unchanged. After pooling, the output feature map shape is (32, H / 8, W / 8, 128). Downsampling block 4 has 128 input channels, and its two convolutional units each have 256 kernels, with the same parameter configuration as above. After final pooling, the output feature map shape is (32, H / 16, W / 16, 256). Thus, the encoder has completed 16x spatial downsampling of the input image and 256-dimensional deep geometric feature extraction, outputting a geometric feature map. ,in , This structure performs two convolution operations before each 2x spatial downsampling, which helps to retain richer local geometric details and positional information, thus providing high-quality feature representations for subsequent accurate detection and localization of surface morphology defects such as pits and protrusions.

[0037] Spectral feature encoder Cube for processing spectral data The specific network structure and parameters are as follows. The encoder input is a four-dimensional tensor with shape (32, H, W, L), where 32 is the preset batch size, H and W are the spatial height and width, and L is the number of spectral channels. The first stage performs spectral dimension encoding: for each spatial location (i, j) in the input tensor, a one-dimensional spectral vector of length L is independently passed through a sub-network consisting of two one-dimensional convolutional layers (Conv1D), and all spatial locations share the weight parameters of this sub-network. The first 1D convolutional layer uses 16 kernels of size 7 with a stride of 1, followed by batch normalization, ReLU activation, and a max pooling layer with a kernel size of 2 and a stride of 2, resulting in an output shape of (32, H, W, 16, L / 2). The second 1D convolutional layer uses 32 kernels of size 5 with a stride of 1, followed by batch normalization, ReLU activation, and a max pooling layer with a kernel size of 2 and a stride of 2, resulting in an output shape of (32, H, W, 32, L / 4). Subsequently, a flattening operation and a fully connected layer (using ReLU activation) map the features of each point to a 64-dimensional vector, resulting in an intermediate spectral feature map of shape (32, H, W, 64). The second stage involves spatial dimension aggregation and size alignment: First, the feature map is passed through a spatial convolutional unit containing a 3×3 two-dimensional convolutional layer with a stride of 2 and 128 output channels. This is followed by batch normalization and ReLU activation to downsample the feature map to (32, H / 2, W / 2, 128). Next, it is passed through a second identical spatial convolutional unit (stride 2, maintaining 128 channels) for further downsampling to (32, H / 4, W / 4, 128). Finally, to align with the spatial dimensions (H / 16, W / 16) of other encoder outputs, the feature map is concatenated. A downsampling module: This module first performs a 3×3 convolution (stride 2, output channels 256), followed by batch normalization and ReLU, with an output shape of (32, H / 8, W / 8, 256). Next, a 2×2 max-pooling layer with a stride of 2 is applied, resulting in an output shape of (32, H / 16, W / 16, 256). This spatial dimension processing is achieved through four 2x downsampling operations (two spatial convolutional units with a stride of 2, the downsampling module's convolution with a stride of 2, and the max-pooling with a stride of 2), unifying the feature map's spatial size to 1 / 16 of the original image, perfectly aligning it with the output sizes of other encoders. Finally, a 1×1 convolutional layer compresses the number of channels from 256 to 128, yielding the final spectral feature map. ,in , This structure effectively extracts the fingerprint features of material composition through dedicated convolution in the spectral dimension, and achieves spatial context aggregation and size uniformity through multi-layer spatial convolution and downsampling.

[0038] After completing the normalization and alignment of the feature range and feature distribution of each channel (for each encoder's output feature map, batch normalization is performed along the channel dimension to adjust the feature value distribution to a mean of 0 and a variance of 1, ensuring that the value range and distribution of the three features are consistent), the three feature maps maintain complete consistency in batch size, spatial height, and spatial width, with only the channel dimensions being independent of each other. The image feature maps... Geometric feature map Spectral feature map Feature splicing and fusion operations are performed strictly according to the channel dimension, and the mathematical expression is: Concat( The symbol represents the concatenation operation along the channel dimension, ultimately resulting in a fused feature map of the zirconium strip surface. ,in, , , Image feature maps Geometric feature map Spectral feature map The corresponding channel dimension. The dimension at any spatial location (h, w) in this fused feature map. + + The 3D feature vector can simultaneously fuse multi-dimensional characterization information of the corresponding surface region, including visual grayscale texture, 3D morphological height, and material spectral composition.

[0039] S3, the fused feature map of the zirconium surface is input into the improved adversarial generative network model, and the standard template feature map is output.

[0040] In some embodiments, step S3 is a defect-free standard template feature generation step, specifically implemented by constructing and training an improved multi-scale deep residual recurrent generative adversarial network. This network consists of two generators. , and two discriminators , Composition. The generator is one of the components. The generator is responsible for learning the mapping from a fused feature map containing potential defects to a normal standard template feature map. This module is responsible for constructing cycle consistency constraints. Two discriminators are used to distinguish between real and generated features in the corresponding domain. The core objective is to learn the mapping relationship between the fused feature map containing potential defects and the normal standard template feature map. Cycle consistency constraints are used to ensure the realism and consistency of the generated template. Specific implementation details are as follows: Let the surface fusion feature map of the zirconium ribbon output in step S2 be as follows: 32 represents the batch size. , To standardize the spatial dimensions after downsampling, .

[0041] Domain A is formed by the fusion feature maps of defect-free, standard-compliant zirconium strip samples, and the samples are denoted as... 32 represents the preset batch size. , To standardize the spatial dimensions after downsampling, =896 represents the total number of channels for the fused features. The target domain B comprises visual, geometric, and spectral multimodal features of a normal surface, without any defect information; it is formed by the fused feature maps of unlabeled zirconium strip samples collected from the production line (which may contain various defects such as scratches, pits, and cracks). The samples are denoted as... , It includes features from both normal and defective regions. During the inference phase, only the fused feature map of a single test sample is input as the domain B sample.

[0042] The generator's multi-scale branch basic channel count is The default value is 64, ensuring... The number of channels should be an integer to avoid issues with non-integer channel counts; the number of various residual blocks is... The default value is 3, which balances feature extraction capability and training efficiency, and can be fine-tuned according to the actual training effect; the dilation rate of the second branch dilated convolution is defaulted to 2, which expands the receptive field without reducing feature resolution and is suitable for feature extraction of medium-scale defects.

[0043] The total number of output channels for the three branch paths is This lays the foundation for subsequent feature fusion and dimension alignment.

[0044] generator A multi-scale deep residual structure is employed, with the core function being the mapping from domain B to domain A, i.e., mapping the defective fused feature map to a defect-free standard template feature map. It includes three parallel branches, and the output feature map sizes of all three branches are maintained. Without changing the structure, channel splicing can be performed directly, as shown below: The first branch performs basic scale feature extraction: [This involves] processing the input feature map... The convolution kernel size is 3×3 and the number of channels is A 64-bit 2D convolution operation is used to extract basic features. It is used to repair small-scale defects such as minute scratches and pitting; then it undergoes... Deep feature extraction is performed on three pre-activated residual blocks with skip connections. Within each residual block, the execution order is batch normalization → ReLU activation → 3×3 convolution → batch normalization → ReLU activation → 3×3 convolution. When the input and output channel numbers of the residual blocks are inconsistent, 1×1 convolutions are used for dimension alignment in the skip connections; otherwise, identity mapping is used. The final output is the first branch feature. .

[0045] The second branch performs dilated convolution multi-scale feature extraction: firstly, ... The convolution kernel size is 3×3 and the number of channels is A dilated convolution operation with a dilation rate of 2 is used to obtain features. It is used to repair medium-scale defects such as dents and indentations; then through The first branch uses three 1×1 convolutional kernels as pre-activated residual blocks. The internal structure of these residual blocks is the same as the first branch, except that the kernel size is changed to 1×1 for channel adjustment and feature compression. The final output is the second branch's features. .

[0046] The third branch performs multi-scale feature extraction with large convolutional kernels: first, ... The convolution kernel is 3×3 and the number of channels is Two-dimensional convolution is used to obtain features. It is used to repair large-scale defects such as component segregation and oxide layers; then through The third branch uses three 5×5 convolutional kernels to pre-activate residual blocks. The internal structure of the residual blocks is the same as the first branch, except that the kernel size is changed to 5×5 to capture a wider range of spatial features. The final output is the third branch's features. .

[0047] Branch fusion and output reconstruction layer: This layer reconstructs the final outputs of the three branches. , , By stitching and blending along the channel dimension, the fused features are obtained. Subsequently, a 1×1 convolutional layer (with a stride of 1 and the same padding) was used to reduce the dimensionality from 128 to [missing information]. =896, realizing feature dimension and input Alignment is performed, followed by channel dimensionality reduction and feature smoothing; then a 3×3 convolutional layer (stride 1, same padding) completes feature reconstruction, and finally the generator outputs the final result. Output the corresponding standard template feature map ,in For generator All trainable parameters, including the weights and biases of each convolutional layer and residual block, and The dimensions are completely consistent with the input fused feature map.

[0048] generator and The structure is completely symmetrical, and the core is used to realize the mapping from domain A to domain B, learning the transformation relationship from the defect-free feature distribution to the feature distribution to be tested, and... Together, they constitute a bidirectional cyclic consistency constraint. Its multi-scale branch number, convolution kernel parameters, residual block number, channel number settings, and output dimension are all related to... Completely identical, except for samples whose input is domain A. The output is a simulated feature map containing defects. ,in for The trainable parameters, and They should be trained independently and separately.

[0049] The discriminator employs a Markov discriminator structure, its core function being to determine the authenticity of the input feature map. Compared to a global discriminator, it can better capture subtle defect features. Two discriminators are used. , The structures are completely identical, differing only in the input field. The specific structures are as follows: For example: It consists of 4 sequentially connected convolutional units. The execution order of each convolutional unit is: 2D convolution → batch normalization → LeakyReLU activation. The LeakyReLU slope is preset to 0.2 to avoid gradient vanishing. The specific parameter configuration is as follows: 1. First convolutional layer: kernel size 4×4, stride 2, number of input channels =896, 64 output channels, no padding, output feature map shape is 2. Second convolutional layer: kernel size 4×4, stride 2, input channels 64, output channels 128, no padding, output feature map shape is... 3. Third convolutional layer: kernel size 4×4, stride 2, input channels 128, output channels 256, no padding, output feature map shape is... 4. Fourth convolutional layer: kernel size 4×4, stride 1, input channels 256, output channels 1, no padding, output is a single-channel probability map. In the probability map, each pixel value corresponds to the probability that the local region of the input feature map comes from the real domain A, and the value ranges from 0 to 1.

[0050] Discriminator and The network structure and hyperparameter configurations are completely identical. The discriminator... The fixed input contains two classes: one is the features of real, defect-free samples in domain A. Another type is generators. Output generated standard template features This is used to distinguish whether the input feature map belongs to a real sample in domain A or a fake sample generated by the network. Discriminator The fixed input also contains two types: one is the features of the real test samples in domain B. Another type is generators. Output generator domain B features It is used to determine whether the input feature map belongs to a real sample in domain B or a fake sample generated by the network.

[0051] The network is trained by jointly optimizing three loss functions, including adversarial loss. To ensure that the generated features are indistinguishable from the real qualified features in terms of distribution, the total adversarial loss is the sum of the adversarial losses in domain A and domain B: ; in, Represents the adversarial loss function. For the adversarial losses of domain A, The adversarial loss for domain B.

[0052] The adversarial loss expression for domain A is: ; The adversarial loss expression for domain B is: ; E represents the expected value, p() represents the data distribution, and p( ) indicates a defect-free and qualified characteristic. The probability distribution it follows, p( ) indicates a feature to be tested that contains defects. The probability distribution it follows ( ) represents the discriminator For true defect-free features The output confidence probability. This represents a generator that maps from domain B to domain A. This represents a generator that maps from domain A to domain B. () indicates the discriminator The output confidence probability. () indicates the discriminator The output confidence probability. This represents the multimodal fusion feature map of a defect-free, qualified zirconium metal ribbon sample in domain A. The multimodal fusion feature map of the zirconium ribbon sample under test containing potential defects in domain B.

[0053] Cyclic consistency loss is used to constrain the invertibility of bidirectional mappings, ensuring that the generator only modifies anomalous features without changing the normal background structure. Its expression is: ; in, Represents the cycle consistency loss function. Representation of features First through generator From domain B to domain A, then through the generator The feature map is reconstructed after one forward loop by returning from domain A to domain B. Representation of features First through generator Transformation from domain A to domain B, then through the generator The process involves returning from domain B to domain A, completing one reverse loop reconstruction of the feature map. This represents the L1 norm.

[0054] Feature similarity loss To constrain the structural similarity between the generated feature map and the corresponding qualified feature map, drawing on the concept of structural similarity, it is calculated only on samples in domain A, and its expression is: ; in, This represents the feature similarity loss function, where N is the total number of samples involved in the loss calculation in a single training batch, and n represents the sample index. Let represent the mean of all feature values ​​of the nth true qualified feature map in the spatial dimension. This represents the mean of all feature values ​​in the spatial dimension of the nth generated standard template feature map. The covariance between the nth true qualified feature map and its corresponding generated template feature map represents the degree of linear correlation between their spatial structures. This represents the variance of the eigenvalues ​​in the nth true qualified feature map space. This represents the variance of the eigenvalues ​​in the feature map space of the nth generated standard template. , The purpose of setting a small, fixed constant is to prevent the denominator of the formula from being zero. It is typically set to... .

[0055] The network's total loss function is a weighted sum of the three loss functions mentioned above: ; in, For the total loss function, To counter the loss function, Let the cycle consistency loss function be... The feature similarity loss function is... , , These are the weight coefficients for the adversarial loss function, the cycle consistency loss function, and the feature similarity loss function, respectively. =10.0, the numerical magnitude of the cycle consistency loss is usually significantly smaller than that of the adversarial loss LGAN. By assigning a higher weight, it ensures that the constraint of cycle reconstruction dominates the optimization process, forcing the model to learn a bidirectional, reversible, and background-preserving mapping, which is the foundation of the correctness of the defect repair task. =1.0, serving as the baseline weight, the adversarial loss drives the generated distribution to approximate the true distribution, which is the driving force behind GAN training. =10 to form a stable training balance. =5.0, a medium weight, serves as a supplement to the cycle consistency loss at the perceptual level, ensuring the structural similarity of generated features while avoiding excessive constraints that could lead to a decline in the model's generalization ability.

[0056] By simultaneously using a flawless, qualified sample set (domain A) and an unlabeled sample set to be tested (domain B), the network parameters were evaluated. , , , Iterative optimization is performed until convergence. No defect annotation information is needed during training; only the domain labels (qualified / to be tested) of the samples are required. The trained generator... That is, it has the ability to fuse the input feature maps Convert to its corresponding standard template feature map The ability to perform model training is described. During training, 20% of the sample sets from domain A and domain B are pre-defined as validation sets, which are not used for parameter updates but only for monitoring model performance. At the end of each training round, the generator is calculated. Cycle consistency loss on the validation set The value of P is used to determine if the validation loss has converged and terminates training when it no longer decreases within P consecutive training epochs and the decrease is less than a preset threshold Q. Simultaneously, to prevent excessively long training times, a maximum number of training epochs, Emax, needs to be set; if the training reaches Emax epochs without triggering the early stopping condition, training is also forcibly terminated. Here, P, Q, and Emax are hyperparameters that need to be predefined; P is preferably 10, and Q is preferably... The optimal value for Emax is 200. This method determines whether the model has learned a stable and generalized mapping ability by monitoring the trend of the validation loss, thus avoiding overfitting on the training set and ensuring the objective termination of the training process and the reproducibility of the results.

[0057] Generator after training It has stable mapping capabilities and can fuse feature maps of any input zirconium ribbon surface. Convert to the corresponding defect-free standard template feature map This provides a standard reference for subsequent defect difference comparison and defect segmentation detection.

[0058] S4. The fused feature map of the zirconium surface is compared with the standard template feature map to obtain the zirconium surface residual map. The zirconium surface residual map is decoded to obtain the location of the zirconium surface defects, thereby realizing the defect detection of the zirconium surface.

[0059] In some embodiments, S4 is the feature residual comparison and defect decoding step, the specific implementation and mathematical expression of which are as follows: First, the residual feature map between the fused feature map of the zirconium surface to be tested and the standard template feature map is calculated element-wise in each channel dimension at the corresponding spatial location. The dimension of the residual feature map during the training phase is... The reasoning stage is The calculation formula is: ; Where R is the residual map of the zirconium surface, which contains at each spatial location (h, w) The residual vector represents the numerical difference between the test sample and the standard template in each channel dimension of the multimodal feature space. Since the generator only repairs defective regions that do not conform to the normal distribution and retains the features of normal regions as is, the values ​​of each dimension of the residual vector in normal regions are close to 0, while the values ​​of each dimension of the residual vector in defective regions are significantly greater than 0. This is a fusion feature map of zirconium metal surfaces. This is a standard template feature map.

[0060] To transform the high-dimensional residual vector into a single anomaly quantification index and comprehensively measure the overall anomaly degree at each spatial location, for each spatial location (h,w)... The 2D residual vector is mapped to scalar anomaly intensity using the L2 Euclidean norm to generate a 2D residual anomaly map. Then the anomaly intensity at that spatial location (h, w) is... The calculation formula is: ; in, This represents the anomaly intensity at spatial location (h, w). A larger value indicates a higher probability that the location is a defect. The first position at spatial location (h, w) The residual values ​​for each channel, where c is the channel index. This represents the total number of residual channels.

[0061] The reason for choosing the L2 Euclidean norm is that it is more sensitive to large numerical residuals, can effectively amplify the difference between defective and normal regions, and suppress the influence of small-amplitude random noise, making it more suitable for the abnormal quantization requirements of multimodal feature residuals in this invention.

[0062] After generating the residual anomaly map, it is first smoothed and denoised using a fixed Gaussian kernel of size 3×3 and standard deviation of 1.0 to suppress pseudo-anomalies caused by feature noise. Then, adaptive threshold segmentation is applied to the denoised residual anomaly map to complete defect decoding and localization. The adaptive segmentation threshold is set as follows: ; in, , These are residual abnormal diagrams. The mean and standard deviation of all pixel intensities; k is an adjustable sensitivity coefficient, typically ranging from 1.5. 3.0. The larger the value of k, the higher the detection threshold and the fewer false detections, but the more likely it is to miss detections. The smaller the value of k, the more likely it is to miss detections. When it is necessary to prioritize the detection of small defects, k is 1.5~2.0. When it is necessary to prioritize the reduction of false detection rate, k is 2.5~3.0. It can be flexibly adjusted according to the actual detection scenario.

[0063] Based on adaptive threshold Generate a binary mask image of surface defects on a zirconium ribbon. The pixel discrimination rule is: for any spatial position (h, w) in the residual abnormal image, if... If a defect is determined to exist at that location, then =1; if If it is determined to be a normal background area, then =0. This defect mask is a binary image in a low-resolution feature space, with a spatial size of 1 / 16 of the original acquired image. To further improve detection accuracy, post-processing is required on the mask: connected component analysis is used to remove isolated small connected components with an area of ​​less than 5 pixels, eliminating false detections caused by residual noise points. The area with a pixel value of 1 in the mask is the detected defect target area.

[0064] Finally, nearest neighbor interpolation is used to map the defect mask generated in the low-resolution feature space back to the coordinate space of the original acquired image. Because the image, geometry, and spectral feature encoders in step S2 all use a 4x 2x downsampling design, resulting in an overall downsampling factor of 16x, the resolution of the defect mask in the feature space is much lower than that of the original input image. Direct use of this would lead to coarse defect localization granularity, failing to meet the requirements of pixel-level accurate detection and on-site workstation positioning. Nearest neighbor interpolation is chosen because it strictly preserves the binary attributes of the defect mask, does not introduce blurring or intermediate grayscale values, ensures the clarity of defect edges, and is most suitable for upsampling binary masks. By establishing a coordinate correspondence between the feature space and the original image space based on a fixed downsampling ratio of 16x using nearest neighbor interpolation, the upsampling and coordinate alignment of the defect mask are completed without introducing additional numerical interference, while maintaining the binary attributes of the defect mask. This ultimately generates a defect binary mask with the same size as the original acquired image, accurately reproducing the contour range and pixel position of the defect in the original image. Based on the aligned defect location results, the defect area can be intuitively marked by overlaying it with the original image. Alternatively, the image pixel coordinates can be converted into the physical operating coordinates of the equipment by using the pre-calibrated pixel physical size (such as each pixel corresponding to an actual area of ​​0.01mm×0.01mm), thereby achieving pixel-level accurate detection and engineering placement of defects on the surface of zirconium strips.

[0065] It should be noted that the various numerical designations such as "first," "second," etc., involved in this application are only for the convenience of description and are not used to limit the scope of the embodiments of this application, nor do they indicate the order of events.

[0066] At least one in this application can also be described as one or more, and multiple can be two, three, four or more, and this application does not impose any limitation. In the embodiments of this application, for a technical feature, the technical features in that technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", and there is no order or size among the technical features described by "first", "second", "third", "A", "B", "C" and "D".

[0067] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0068] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting surface defects in highly reflective metallic zirconium ribbons based on multi-source data, characterized in that, include: S1: Multi-source data acquisition is performed on the surface of the zirconium strip to obtain zirconium surface data; S2: Input the zirconium surface data into the multimodal encoder model and output the zirconium surface fusion feature map; S3: Input the fused feature map of the zirconium surface into the improved adversarial generative network model and output the standard template feature map; S4: Compare the fused feature map of the zirconium surface with the standard template feature map to obtain the zirconium surface residual map. Decode the zirconium surface residual map to obtain the location of the zirconium surface defects, thereby realizing the defect detection of the zirconium surface.

2. The method for detecting surface defects of highly reflective metallic zirconium ribbons based on multi-source data according to claim 1, characterized in that, The zirconium surface data includes a two-dimensional reflected light intensity image, a two-dimensional height map, and a spectral data cube.

3. The method for detecting surface defects of highly reflective metallic zirconium ribbons based on multi-source data according to claim 1, characterized in that, The multimodal encoder model includes an image feature encoder, a geometric feature encoder, and a spectral feature encoder.

4. The method for detecting surface defects of highly reflective metallic zirconium ribbons based on multi-source data according to any one of claims 1-3, characterized in that, The step of inputting the zirconium surface data into a multimodal encoder model and outputting a fused feature map of the zirconium surface includes: The two-dimensional reflected light intensity image is input into the image feature encoder, and an image feature map is output. The two-dimensional height map is input into the geometric feature encoder, and a geometric feature map is output. The spectral data cube is input into the spectral feature encoder, and a spectral feature map is output. The image feature map, the geometric feature map, and the spectral feature map are combined to generate the zirconium surface fusion feature map.

5. The method for detecting surface defects of highly reflective metallic zirconium ribbons based on multi-source data according to claim 1, characterized in that, The process of inputting the fused feature map of the zirconium surface into an improved adversarial generative network model and outputting a standard template feature map includes: Construct the improved adversarial generative network model; The fused feature map of the zirconium surface is input into the improved generative adversarial network model, and the standard template feature map is output.

6. The method for detecting surface defects of highly reflective metallic zirconium ribbons based on multi-source data according to claim 5, characterized in that, The construction of the improved adversarial generative network model includes: The structure of the generator and the structure of the discriminator are constructed. The structure of the adversarial generative network model includes two generators and two discriminators. Construct the total loss function for the model.

7. The method for detecting surface defects of highly reflective metallic zirconium ribbon based on multi-source data according to claim 6, characterized in that, The generator structure includes: a first branch, a second branch, a third branch, and a branch fusion and output reconstruction layer.

8. The method for detecting surface defects of highly reflective metallic zirconium ribbons based on multi-source data according to claim 6, characterized in that, The discriminator structure includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer.

9. The method for detecting surface defects of highly reflective metallic zirconium ribbon based on multi-source data according to claim 6, characterized in that, The total loss function is: ； in, For the total loss function, To counter the loss function, Let the cycle consistency loss function be... The feature similarity loss function is... , , These are the weight coefficients for the adversarial loss function, the cycle consistency loss function, and the feature similarity loss function, respectively.

10. The method for detecting surface defects of highly reflective metallic zirconium ribbon based on multi-source data according to claim 1, characterized in that, The formula for calculating the residual map of the zirconium metal surface is as follows: ； Where R is the residual map of the zirconium metal surface. This is a fusion feature map of zirconium metal surfaces. This is a standard template feature map.

Images