Method for detecting black spots on galvanized steel sheets

Abstract

The application discloses a black spot detection method for a galvanized steel plate, comprising the following steps: acquiring an initial image of the galvanized steel plate; inputting the initial image into a black spot detection model, performing image segmentation on the initial image through a segmentation embedding module to obtain a plurality of image blocks, and linearly mapping each image block to a high-dimensional space to obtain an embedding vector sequence; capturing, by an encoder, long-distance dependency relationships among black spots and backgrounds, black spots and noise points, and each black spot according to the embedding vector sequence, applying attention weights on feature dimensions to obtain a target feature sequence with enhanced context information; performing pixel-level feature restoration on the target feature sequence by a decoder to obtain a feature map matching the size of the initial image, wherein the feature map comprises a plurality of pixel feature vectors; performing classification detection on each pixel feature vector by a classification module to output a black spot detection result; and visually displaying the black spot detection result. The application can improve the black spot detection accuracy of the galvanized steel plate.

Description

Technical Field

[0001] This application belongs to the field of steel plate defect detection technology, and particularly relates to a method for detecting black spots on galvanized steel plates. Background Technology

[0002] Galvanized automotive exterior panels have strict standards for surface quality. Among these standards, black spot defects (both in number and size) are a crucial control point in the panel production process. They directly affect the surface uniformity and appearance after painting, potentially leading to workpiece rejection. Black spot defects are usually caused by surface micro-oxidation during annealing or by zinc dross adhering during galvanizing.

[0003] Traditional methods for detecting black spot defects primarily rely on manual visual inspection. Quality inspectors first select a sample at the export section, move it to the offline inspection table, wipe it clean, mark the area to be inspected with squares on the sample, then visually identify the location of each black spot, measure its size with a ruler, and finally record and classify the data. This method is not only time-consuming and inefficient, but also easily limited by human subjective judgment and physical and mental state, making it difficult to maintain a high degree of accuracy and consistency. Manual inspection is also easily affected by lighting conditions, observation angle, and personal experience, leading to missed or false detections. Summary of the Invention

[0004] The embodiments of this application provide a method for detecting black spots on galvanized steel sheets, which can at least to a certain extent effectively retain the feature information of small-sized black spots, improve the detection rate of small targets, and improve the detection accuracy of black spots on galvanized steel sheets.

[0005] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0006] The first aspect of this application provides a method for detecting black spots on galvanized steel sheets, including: Obtain the initial image of the galvanized steel sheet; The initial image is input into a black spot detection model, which performs the following steps: The initial image is segmented into several image patches using the segmentation and embedding module of the black spot detection model; each image patch is linearly mapped to a high-dimensional space to obtain an embedding vector sequence; the encoder of the black spot detection model captures the long-distance dependencies between black spots and background, black spots and noise points, and individual black spots based on the embedding vector sequence, and applies attention weights to the feature dimension to obtain a target feature sequence that enhances contextual information; the decoder performs pixel-level feature reconstruction on the target feature sequence using cross-attention and upsampling operations to obtain a feature map matching the size of the initial image, the feature map including multiple pixel feature vectors; the classification module of the black spot detection model classifies and detects each pixel feature vector and outputs the black spot detection result. Obtain the black spot detection results and visualize the black spot detection results.

[0007] Optionally, the segmentation and embedding module of the black dot detection model performs image segmentation on the initial image to obtain several image patches, and linearly maps each image patch to a high-dimensional space to obtain an embedding vector sequence, including: The following steps are performed using the segmentation and embedding module of the black dot detection model: Based on preset block size parameters, the initial image is divided into several non-overlapping image blocks using a non-overlapping sliding window cutting method; For each image block, the two-dimensional pixel matrix within the image block is flattened into a one-dimensional pixel vector, and the one-dimensional pixel vector is mapped from the original pixel space to a high-dimensional space of a preset dimension based on a linear projection matrix to obtain an initial embedding vector. Add position encoding to the initial embedding vector to obtain an embedding vector with spatial position information; The embedding vectors are arranged in order of their spatial positions in the initial image to obtain the embedding vector sequence.

[0008] Optionally, the segmentation and embedding module of the black dot detection model performs image segmentation on the initial image to obtain several image patches, and linearly maps each image patch to a high-dimensional space to obtain an embedding vector sequence, including: The following steps are performed using the segmentation and embedding module of the black dot detection model: Based on a preset block size parameter, the initial image is divided into several image blocks using a sliding window step size smaller than the block size parameter, and adjacent image blocks at least partially overlap. For each image block, the two-dimensional pixel matrix within the image block is flattened into a one-dimensional pixel vector. The one-dimensional pixel vector is mapped from the original pixel space to a high-dimensional space of a preset dimension based on a linear projection matrix to obtain an initial embedding vector. The initial embedding vector is then position-encoded to obtain an embedding vector with position-encoded information. The embedding vectors are arranged in order of their spatial positions in the initial image to obtain the embedding vector sequence.

[0009] Optionally, the encoder employs a multi-head self-attention mechanism and a feedforward neural network stacked alternately. The encoder using the black spot detection model captures the relationships between black spots and background, black spots and noise points, and long-distance dependencies between individual black spots based on the embedded vector sequence, and applies attention weights to the feature dimension to obtain a target feature sequence that enhances contextual information, including: The encoder of the black dot detection model performs the following steps: The embedded vector sequence is mapped to a query vector, a key vector, and a value vector. The similarity between the query vector and the key vector is calculated through a multi-head self-attention mechanism and then normalized to obtain the attention weight. The value vector is weighted and summed based on the attention weights to complete global context modeling, capture the long-distance dependencies between black points and background, black points and noise points, and each black point, and obtain a target feature sequence that enhances contextual information.

[0010] Optionally, the step of performing global context modeling on each of the embedding vectors in the embedding vector sequence to obtain the attention weights of each embedding vector includes: For each embedding vector, obtain the correlation evaluation value between the embedding vector and all key vectors in the embedding vector sequence, and normalize each correlation evaluation value to obtain the attention weight of the embedding vector.

[0011] Optionally, before the initial image is segmented into several image patches using the segmentation embedding module of the black dot detection model, the method further includes: The image preprocessing module of the black spot detection model performs the following steps: The initial image is subjected to distortion correction and illumination equalization processing; The CLAHE algorithm is used to enhance the contrast between black points and the background to obtain the preprocessed initial image.

[0012] Optionally, the encoder includes sub-encoders with different depths, the black spot detection model further includes a multi-scale feature fusion module, and the method further includes: The following steps are performed by the multi-scale feature fusion module: A first feature map is obtained from a first depth layer of the encoder, a second feature map is obtained from a second depth layer of the encoder, and a third feature map is obtained from a third depth layer of the encoder, wherein the depth of the first depth layer is less than the depth of the second depth layer, and the depth of the second depth layer is less than the depth of the third depth layer. The second feature map and the third feature map are upsampled to the resolution of the first feature map by bilinear interpolation or transpose convolution, so that each feature map has the same spatial size; The first feature map, the second feature map, and the third feature map are concatenated or added element by element along the channel dimension, and a fused feature map is output after channel fusion through a convolutional layer.

[0013] Optionally, during the process of inputting the initial image into the black spot detection model and performing the following steps through the black spot detection model, the method further includes: The inference process of the black spot detection model is accelerated using the TensorRT inference optimization engine.

[0014] Optionally, visualizing the black spot detection results includes: The black spot detection results are then overlaid with the initial image and displayed visually.

[0015] Optionally, after visualizing the black spot detection results, the method further includes: Generate a black spot detection report for the galvanized steel sheet. The black spot detection report includes one or more of the following: the number, distribution location, and distribution density of black spots of various sizes in the initial image.

[0016] A second aspect of this application provides a black spot detection device for galvanized steel sheets, comprising: The acquisition unit is used to acquire an initial image of the galvanized steel sheet; The detection unit is used to input the initial image into a black spot detection model, and the black spot detection model performs the following steps: The initial image is segmented into several image patches by the segmentation and embedding module of the black spot detection model; each image patch is linearly mapped to a high-dimensional space to obtain an embedding vector sequence; the encoder of the black spot detection model captures the long-distance dependencies between black spots and background, black spots and noise points, and between individual black spots based on the embedding vector sequence, and applies attention weights to the feature dimension to obtain a target feature sequence that enhances contextual information; the decoder performs pixel-level feature reconstruction on the target feature sequence using cross-attention and upsampling operations to obtain a feature map matching the size of the initial image, the feature map including multiple pixel feature vectors; the classification module of the black spot detection model classifies and detects each pixel feature vector and outputs the black spot detection result. The display unit is used to acquire the black spot detection results and to visualize the black spot detection results. A third aspect of this application provides a computer-readable storage medium storing at least one computer program instruction, which is loaded and executed by a processor to perform the operations described in any of the methods described in the first aspect.

[0017] A fourth aspect of this application provides an electronic device including one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to perform the operation as described in any of the methods in the first aspect.

[0018] The embodiments of the present invention provide one or more technical solutions that achieve at least the following technical effects or advantages: The black spot detection method for galvanized steel sheets according to embodiments of this application includes: acquiring an initial image of the galvanized steel sheet; inputting the initial image into a black spot detection model, and performing the following steps through the black spot detection model: segmenting the initial image into several image blocks using the segmentation and embedding module of the black spot detection model, linearly mapping each image block to a high-dimensional space to obtain an embedding vector sequence; capturing the long-distance dependencies between black spots and background, black spots and noise points, and each black spot using the embedding vector sequence through the encoder of the black spot detection model, and applying attention weights on the feature dimension to obtain a target feature sequence that enhances contextual information; performing pixel-level feature restoration on the target feature sequence through the decoder using cross-attention and upsampling operations to obtain a feature map matching the size of the initial image, the feature map including multiple pixel feature vectors; classifying and detecting each pixel feature vector through the classification module of the black spot detection model, and outputting the black spot detection result; acquiring the black spot detection result, and visualizing the black spot detection result. Therefore, the embodiments of this application segment the initial image into multiple image blocks through a black spot detection model, thereby increasing the proportion of black spots in the image. High-dimensional embedding is performed on each image block to effectively preserve the feature information of small-sized black spots, improve the detection rate of small targets, and improve the black spot detection accuracy of galvanized steel plates.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 A flowchart of a black spot detection method for galvanized steel sheets according to an embodiment of this application is shown; Figure 2 A structural diagram of a black spot detection device for galvanized steel sheets according to an embodiment of this application is shown; Figure 3 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0022] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0023] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different models and / or processor devices and / or microcontroller devices.

[0024] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0025] It should also be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such uses of these terms can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described.

[0026] Galvanized automotive exterior panels have strict standards for surface quality. Among these standards, black spot defects (both in number and size) are a crucial control point in the panel production process. They directly affect the surface uniformity and appearance after painting, potentially leading to workpiece rejection. Black spot defects are usually caused by surface micro-oxidation during annealing or by zinc dross adhering during galvanizing.

[0027] Traditional methods for detecting black spot defects primarily rely on manual visual inspection. Quality inspectors first select a sample at the export section, move it to the offline inspection table, wipe it clean, mark the area to be inspected with squares on the sample, then visually identify the location of each black spot, measure its size with a ruler, and finally record and classify the data. This method is not only time-consuming and inefficient, but also easily limited by human subjective judgment and physical and mental state, making it difficult to maintain a high degree of accuracy and consistency. Manual inspection is also easily affected by lighting conditions, observation angle, and personal experience, leading to missed or false detections.

[0028] In view of this, the present application provides a method for detecting black spots on galvanized steel sheets. This method divides the initial image into multiple image blocks using a black spot detection model to increase the proportion of black spots in the image. High-dimensional embedding is performed on each image block to effectively preserve the feature information of small-sized black spots, improve the detection rate of small targets, and improve the black spot detection accuracy of galvanized steel sheets.

[0029] The black spot detection method of this application embodiment will be described below with reference to the accompanying drawings.

[0030] Figure 1 A flowchart of a black spot detection method according to an embodiment of this application is shown.

[0031] The first aspect of this application provides a method for detecting black spots on galvanized steel sheets, including but not limited to: Step S10. Obtain the initial image of the galvanized steel sheet; For example, a high-precision line scan camera, with a resolution of 4096 pixels per line and a line frequency of 40kHz, is deployed at the exit section of the galvanizing line. The camera can be mounted on a traverse mechanism and can automatically adjust its shooting position according to the strip width (800-1600mm). The camera is positioned at a certain distance from the strip surface, for example, 500mm, using an LED linear light source for supplemental lighting to ensure image uniformity. Images are acquired synchronously at a strip speed of 1m / s, generating approximately 200MB of image data per meter of strip.

[0032] In some embodiments, after the image is acquired based on the camera, the method further includes: The acquired images undergo distortion correction and illumination equalization. For example, Gaussian filtering is used to remove image noise. Adaptive lighting adjustment and image distortion correction are performed; CLAHE (Contrast Limiting Adaptive Histogram Equalization) is used to enhance the contrast between black points and the background, thereby enhancing black point features.

[0033] Step S20. Input the initial image into the black spot detection model, and perform the following steps through the black spot detection model: The initial image is segmented into several image blocks by the segmentation and embedding module of the black spot detection model; each image block is linearly mapped to a high-dimensional space to obtain an embedding vector sequence; the encoder of the black spot detection model captures the long-distance dependencies between black spots and background, black spots and noise points, and between individual black spots based on the embedding vector sequence, and applies attention weights to the feature dimension to obtain a target feature sequence that enhances contextual information; the decoder of the black spot detection model uses cross-attention and upsampling operations to perform pixel-level feature restoration on the target feature sequence to obtain a feature map matching the size of the initial image, the feature map including multiple pixel feature vectors; the classification module of the black spot detection model classifies and detects each pixel feature vector and outputs the black spot detection result. It is understood that the black spot detection model in this application embodiment is obtained by training the network to be trained using training samples. The training samples can be: a dataset of galvanized steel plates containing several labeled images, such as 5000 labeled images; the network to be trained can be a Transformer network.

[0034] It should be noted that the black spot detection model includes a segmentation and embedding module, an encoder, a decoder, and a classification module connected in sequence. The segmentation and embedding module is used to segment the labeled image into several image patches, and linearly map each image patch to a high-dimensional space to obtain an embedding vector sequence. The encoder is used to capture the long-distance dependencies between black spots and background, black spots and noise points, and each black spot according to the embedding vector sequence, and apply attention weights on the feature dimension to obtain a target feature sequence that enhances contextual information. The decoder is used to obtain a feature map according to the target feature sequence, and the feature map includes multiple pixel feature vectors. The classification module is used to classify and detect each pixel feature vector and output the black spot detection result.

[0035] The training process of the black spot detection model is explained below: During model training, the input for each iteration is a set of data-augmented images of galvanized steel plates (with a preset resolution, such as 512×512 pixels), and the corresponding ground truth labels are pixel-level segmentation masks of the same size (each pixel is labeled as a black dot or background). During forward propagation, the image passes through the segmentation embedding module, encoder, decoder, and classification module, and outputs the predicted segmentation mask. The loss function uses a hybrid loss function combining cross-entropy loss and Dice loss to address the class imbalance problem between black dots and background. During backpropagation, the SGD optimizer is used to dynamically update the network parameters. The initial learning rate is set to 0.01, and a cosine annealing scheduling strategy is used to gradually decay the learning rate to near 0 during training, with weight decay. The coefficient was set to 0.0001 for regularization to prevent overfitting, and the gradient clipping threshold was set to 1.0 to limit the gradient norm and avoid gradient explosion. For hyperparameter tuning, the optimal block size (8×8, 16×16, or 32×32), number of encoder layers (6, 8, or 12), and number of attention heads (6, 8, or 12) were determined through experiments on a small-scale validation set. Finally, the configuration with the highest segmentation accuracy (mIoU) on the validation set was selected. The training lasted for a total of 200 epochs. After each epoch, the validation set loss was calculated and the current optimal model weights were saved. This yielded a minimum diameter range of 0.1mm-2.00mm that could detect black spots, thus achieving the detection of small targets.

[0036] In some embodiments, the segmentation and embedding module of the black spot detection model performs image segmentation on the initial image to obtain several image patches, and linearly maps each image patch to a high-dimensional space to obtain an embedding vector sequence, including: The following steps are performed using the segmentation and embedding module of the black dot detection model: Step S211. Based on the preset block size parameters, the initial image is divided into several non-overlapping image blocks using a non-overlapping sliding window cutting method; For example, the initial image is captured by a camera with 4096 pixels per line and a shooting distance of 500mm. A small target of 0.1mm corresponds to about 2-3 pixels. Therefore, the initial image is divided into 25 image blocks, 30 image blocks, etc., to ensure that the small target falls completely within a single image block.

[0037] Step S212. For each image block, flatten the two-dimensional pixel matrix within the image block into a one-dimensional pixel vector, and map the one-dimensional pixel vector from the original pixel space to a high-dimensional space of a preset dimension based on a linear projection matrix to obtain an initial embedding vector; Step S213. Add position encoding to the initial embedding vector to obtain an embedding vector with spatial position information; Step S214. Arrange each of the embedding vectors according to their spatial position in the initial image to obtain the embedding vector sequence.

[0038] In steps S211-S214, the segmentation and embedding module first performs adaptive resolution normalization on the initial image, scaling the galvanized steel plate images acquired under different acquisition conditions to a preset standard size. Then, based on preset block size parameters (e.g., 16×16 pixels), a non-overlapping sliding window cutting strategy is used to uniformly divide the normalized image into multiple regular and non-overlapping image blocks. The block size parameters are dynamically configured according to the typical pixel ratio range of black spots on the galvanized steel plate. For scenes with small or sparsely distributed black spot targets, a smaller block size (e.g., 8×8 pixels) is used to enhance the preservation of local details, while for scenes with relatively large black spot targets, a larger block size (e.g., 32×32 pixels) is used. To balance computational efficiency, after image segmentation, for each image block, its internal two-dimensional pixel matrix is ​​flattened into a one-dimensional pixel vector, and the one-dimensional pixel vector is mapped from the original pixel space to a high-dimensional space of a preset dimension through a learnable linear projection matrix to obtain an initial embedding vector. Then, a positional encoding is added to each initial embedding vector to carry the spatial position information of the image block in the original image, thereby obtaining an embedding vector with spatial position information. Finally, the embedding vectors corresponding to all image blocks are arranged in order of their spatial position in the initial image to form an embedding vector sequence. This sequence contains both the visual content features of the local image blocks and retains the global spatial layout information.

[0039] In some embodiments, the segmentation and embedding module of the black spot detection model performs image segmentation on the initial image to obtain several image patches, and linearly maps each image patch to a high-dimensional space to obtain an embedding vector sequence, including: The following steps are performed using the segmentation and embedding module of the black dot detection model: Step S214. Based on the preset block size parameter, the initial image is divided into several image blocks using a sliding window step size smaller than the block size parameter, and the adjacent image blocks overlap at least partially; Step S215. For each image block, flatten the two-dimensional pixel matrix in the image block into a one-dimensional pixel vector, and map the one-dimensional pixel vector from the original pixel space to a high-dimensional space of a preset dimension based on the linear projection matrix to obtain an initial embedding vector. Then, perform position encoding on the initial embedding vector to obtain an embedding vector with position encoding information. Step S216. Arrange each of the embedding vectors according to their spatial position in the initial image to obtain the embedding vector sequence.

[0040] Understandably, in steps S214-S216, by setting a sliding window step size smaller than the block size, there is a partial overlap between adjacent image blocks, which effectively avoids the black point features located on the block boundary being cut off or lost. At the same time, combined with the position encoding overlay operation, learnable absolute position encoding or relative position encoding is added to each embedding vector, so that the embedding vector sequence not only contains the local texture and color features of the black points, but also retains the spatial relative positional relationship between each image block, laying the foundation for the subsequent encoder to capture the long-distance dependency relationship of black points across regions.

[0041] In some embodiments, the encoder employs a multi-head self-attention mechanism and a feedforward neural network stacked alternately. The encoder using the black spot detection model captures the relationships between black spots and background, black spots and noise points, and long-distance dependencies between individual black spots based on the embedded vector sequence, and applies attention weights to the feature dimension to obtain a target feature sequence that enhances contextual information, including: The encoder of the black dot detection model performs the following steps: Step S221. Map the embedded vector sequence to a query vector, a key vector, and a value vector. Calculate the similarity between the query vector and the key vector using a multi-head self-attention mechanism and obtain the attention weights after normalization. Step S222. Based on the attention weights, perform weighted summation on the value vector to complete global context modeling, capture the long-distance dependencies between black points and background, black points and noise points, and each black point, and obtain the target feature sequence with enhanced context information.

[0042] In steps S221-S222, the encoder first maps the embedded vector sequence into query vectors, key vectors, and value vectors through a linear transformation. Then, it calculates the similarity between each query vector and all key vectors using a multi-head self-attention mechanism, and obtains attention weights by normalization, for example, using a Softmax function. Finally, it performs a weighted summation of the value vectors based on these attention weights, thereby completing global context modeling. During this process, it can effectively capture the long-distance dependency between black spots and background areas in galvanized steel plate images. For example, when an image patch exhibits a dark, circular feature, the attention weights assign lower weights to the surrounding metallic-looking background areas, while assigning higher weights to other suspected black spot areas. This allows the global context to confirm that the area is indeed a black spot rather than a surface scratch or oil stain. Simultaneously, the attention... The attention mechanism can also distinguish between black spots and noise spots. Since real black spots are usually continuous on the steel plate surface and have certain area and shape regularity, while noise spots (such as dust and water stains) are often isolated, irregular or abnormal in color, the encoder can learn the pattern that real black spots have high statistical consistency with the surrounding area and large differences in statistical features from noise spots by calculating the attention weight distribution between image patches at different locations. Thus, it can automatically apply differentiated weights on the feature dimension: assign high attention weights to feature channels related to the semantics of black spots (such as dark tone, roundness, and edge blur), assign medium weights to feature channels related to the background metallic luster and texture regularity, and assign near-zero suppression weights to feature channels related to random noise and acquisition artifacts. Finally, it outputs a target feature sequence that enhances contextual information.

[0043] Based on multi-head attention, the encoder performs nonlinear transformation and feature mapping on the attention output through a feedforward neural network to enhance the model's expressive power. Furthermore, it ensures gradient flow stability and prevents deep network degradation through residual connections and layer normalization operations. After stacking multiple encoders, the embedding vector of each image block not only aggregates the original pixel information of its own local region, but also fully integrates the global contextual information of black spots, background, and noise in all other locations in the entire steel plate image, outputting a target feature sequence with strong discriminativeness and global semantic consistency.

[0044] In some embodiments, performing global context modeling on each of the embedding vectors in the embedding vector sequence to obtain the attention weights for each embedding vector includes: For each embedding vector, obtain the correlation evaluation value between the embedding vector and all key vectors in the embedding vector sequence, and normalize each correlation evaluation value to obtain the attention weight of the embedding vector.

[0045] In some embodiments, before the initial image is segmented into several image patches using the segmentation embedding module of the black dot detection model, the method further includes: The image preprocessing module of the black spot detection model performs the following steps: Step S201. Perform distortion correction and illumination equalization processing on the initial image; Step S202. Use the CLAHE algorithm to enhance the contrast between black points and the background to obtain the preprocessed initial image.

[0046] It should be noted that in steps S201-S202, the image preprocessing module first performs distortion correction and illumination equalization on the initial image to eliminate interference caused by lens distortion and uneven illumination. Then, the CLAHE algorithm is used to enhance the contrast between black points and the background to obtain the preprocessed initial image. It should also be noted that in the data preprocessing stage, the image enhancement module performs various transformation operations on the original galvanized steel plate image to expand sample diversity and improve model robustness. For example, a random rotation strategy is used to randomly rotate the image within an angle range of -30° to 30°. The model undergoes several transformations to simulate the slight deflection or tilting of steel plates during actual production line acquisition. Next, HSV color space adjustment is performed, randomly changing the values ​​of the hue, saturation, and brightness channels within a preset reasonable range to simulate the impact of different light intensities, light source color temperatures, and differences in steel plate surface reflectivity on image quality. Mosaic stitching enhancement is introduced, where multiple different steel plate images are scaled and stitched into a single composite image, enabling the model to simultaneously learn the black dot feature distribution of multiple different regions in a single forward propagation, thus enhancing the model's adaptability to mixed scenes with both dense and sparse black dot regions.

[0047] In some embodiments, the encoder includes sub-encoders with different depths, the black spot detection model further includes a multi-scale feature fusion module, and the method further includes: The following steps are performed by the multi-scale feature fusion module: Step S204. Obtain a first feature map from the first depth layer of the encoder, a second feature map from the second depth layer of the encoder, and a third feature map from the third depth layer of the encoder, wherein the depth of the first depth layer is less than the depth of the second depth layer, and the depth of the second depth layer is less than the depth of the third depth layer; Step S205. Perform bilinear interpolation or transpose convolution on the second feature map and the third feature map to upsample to the resolution of the first feature map, so that each feature map has the same spatial size; Step S206. The first feature map, the second feature map, and the third feature map are concatenated or added element by element in the channel dimension, and a fused feature map is output after channel fusion through a convolutional layer.

[0048] In steps S204-S206, the multi-scale feature fusion module extracts multi-scale feature maps from different depth layers of the Transformer encoder. The shallow encoder outputs feature maps with higher resolution but lower semantic abstraction, used to preserve details such as edges, textures, and shapes of black dots. The mid-layer encoder outputs feature maps with medium resolution, and the deep encoder outputs feature maps with lower resolution but rich semantic information, used to distinguish the category attributes of black dots from the background. Then, the small-sized deep feature maps are upsampled to the resolution of the shallow feature maps through bilinear interpolation or transposed convolution, so that all feature maps have the same spatial size. These aligned feature maps are concatenated or element-wise added along the channel dimension, and channel fusion is performed through convolutional layers to output a fused feature map that contains both edge localization capabilities and strong semantic discriminativeness, for subsequent upsampling and pixel-level classification by the decoder.

[0049] In some embodiments, during the process of inputting the initial image into a black spot detection model and performing the following steps through the black spot detection model, the method further includes: The inference process of the black spot detection model is accelerated using the TensorRT inference optimization engine.

[0050] Understandably, the TensorRT model can serve as a compiler or optimizer for black spot detection models. TensorRT optimizes inference for black spot detection models specifically for GPU hardware, allowing them to run faster and consume less GPU memory on that hardware. For example, the inference time for a single 204×204 image is approximately 8ms, and the total inference time for 25 small images is approximately 200ms, meeting the requirements for online real-time detection (detection time per meter <300ms when strip speed ≤2m / s).

[0051] In some embodiments, after outputting the black spot detection results, the method further includes: Connectivity analysis was performed on the black spot detection results to remove duplicates and isolated noise points. A confidence threshold of 0.85 was set to filter false positives; and / or The black spot detection results are output according to the diameter of the black spot. For example, the black spot diameter is classified as follows: <0.3mm is a tiny black spot, 0.3-0.6mm is a small black spot, 0.6-1.0mm is a medium black spot, and >1.0mm is a large black spot.

[0052] Step S30. Obtain the black spot detection results and visualize the black spot detection results.

[0053] In some embodiments, visualizing the black spot detection results includes: The black spot detection results are then overlaid with the initial image and displayed visually.

[0054] In some embodiments, after visualizing the black spot detection results, the method further includes: Generate a black spot detection report for the galvanized steel sheet. The black spot detection report includes one or more of the following: the number, distribution location, and distribution density of black spots of various sizes in the initial image.

[0055] Based on the above disclosure, the black spot detection method for galvanized steel sheets according to embodiments of this application includes: acquiring an initial image of the galvanized steel sheet; inputting the initial image into a black spot detection model, and performing the following steps through the black spot detection model: segmenting the initial image into several image blocks using the segmentation and embedding module of the black spot detection model, linearly mapping each image block to a high-dimensional space to obtain an embedding vector sequence; capturing the long-distance dependencies between black spots and background, black spots and noise points, and each black spot using the embedding vector sequence through the encoder of the black spot detection model, and applying attention weights on the feature dimension to obtain a target feature sequence that enhances contextual information; performing pixel-level feature restoration on the target feature sequence through the decoder using cross-attention and upsampling operations to obtain a feature map matching the size of the initial image, the feature map including multiple pixel feature vectors; classifying and detecting each pixel feature vector through the classification module of the black spot detection model, and outputting the black spot detection result; acquiring the black spot detection result, and visualizing the black spot detection result. Therefore, the embodiments of this application segment the initial image into multiple image blocks through a black spot detection model, thereby increasing the proportion of black spots in the image. High-dimensional embedding is performed on each image block to effectively preserve the feature information of small-sized black spots, improve the detection rate of small targets, and improve the black spot detection accuracy of galvanized steel plates.

[0056] Figure 2 A structural diagram of a black spot detection device for galvanized steel sheets according to an embodiment of this application is shown.

[0057] A second aspect of this application provides a black spot detection device 200 for galvanized steel sheets, comprising: Acquisition unit 201 is used to acquire an initial image of the galvanized steel sheet; The detection unit 202 is used to input the initial image into a black spot detection model, and the black spot detection model performs the following steps: the initial image is segmented into several image blocks by the segmentation and embedding module of the black spot detection model, and each image block is linearly mapped to a high-dimensional space to obtain an embedding vector sequence; the encoder of the black spot detection model captures the long-distance dependencies between black spots and background, black spots and noise points, and each black spot according to the embedding vector sequence, and applies attention weights on the feature dimension to obtain a target feature sequence that enhances contextual information; the decoder performs pixel-level feature restoration on the target feature sequence using cross-attention and upsampling operations to obtain a feature map that matches the size of the initial image, the feature map including multiple pixel feature vectors; the classification module of the black spot detection model classifies and detects each pixel feature vector, and outputs the black spot detection result; Display unit 203 is used to acquire the black spot detection results and to visualize the black spot detection results. A third aspect of this application provides a computer-readable storage medium storing at least one computer program instruction, which is loaded and executed by a processor to perform the operations as described in any of the methods in the first aspect.

[0058] Computer-readable storage media may be portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the computer-readable storage medium of this application is not limited thereto. In this application, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0059] A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0060] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0061] A fourth aspect of this application provides an electronic device including one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to perform the operation as described in any of the methods in the first aspect.

[0062] like Figure 3 As shown, the electronic device 400 is manifested in the form of a general-purpose computing device. The components of the electronic device 400 may include, but are not limited to: at least one processing unit 410, at least one storage unit 420, and a bus 430 connecting different system components (including storage unit 420 and processing unit 410).

[0063] The storage unit stores program code, which can be executed by the processing unit 410, causing the processing unit 410 to perform the steps described in the "Embodiment Method" section above according to various exemplary embodiments of this application.

[0064] Storage unit 420 may include readable media in the form of volatile storage units, such as random access memory (RAM) 421 and / or cache 422, and may further include read-only memory (ROM) 423.

[0065] Storage unit 420 may also include a program / utility 424 having a set (at least one) of program modules 425, such program modules 425 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0066] Bus 430 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0067] Electronic device 400 can also communicate with one or more external devices 500 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 400, and / or with any device that enables electronic device 400 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed through I / O (input / output) interface 450, which can also be connected to display unit 440 to display the communication content. Furthermore, electronic device 400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) through network adapter 460. As shown, network adapter 460 communicates with other modules of electronic device 400 via bus 430. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0068] The functions described herein can be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions can be stored as one or more instructions or codes on or transmitted via a computer-readable medium. Other examples and embodiments are within the scope and spirit of this invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Furthermore, the functional units can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit.

[0069] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0070] The units described as separate components may or may not be physically separate. Similarly, the components of the control device may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0071] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0072] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting black spots on galvanized steel sheets, characterized in that, include: Obtain the initial image of the galvanized steel sheet; The initial image is input into the black spot detection model, which performs the following steps: the initial image is segmented into several image patches by the segmentation and embedding module of the black spot detection model, and each image patch is linearly mapped to a high-dimensional space to obtain an embedding vector sequence; the encoder of the black spot detection model captures the long-distance dependencies between black spots and background, black spots and noise points, and each black spot according to the embedding vector sequence, and applies attention weights on the feature dimension to obtain a target feature sequence that enhances contextual information; The target feature sequence is restored at the pixel level by using cross-attention and upsampling operations through the decoder to obtain a feature map that matches the size of the initial image. The feature map includes multiple pixel feature vectors. The classification module of the black spot detection model is used to classify and detect each pixel feature vector and output the black spot detection result. Obtain the black spot detection results and visualize the black spot detection results.

2. The method according to claim 1, characterized in that, The segmentation and embedding module using the black dot detection model performs image segmentation on the initial image to obtain several image patches, and linearly maps each image patch to a high-dimensional space to obtain an embedding vector sequence, including: The following steps are performed using the segmentation and embedding module of the black dot detection model: Based on preset block size parameters, the initial image is divided into several non-overlapping image blocks using a non-overlapping sliding window cutting method; For each image block, the two-dimensional pixel matrix within the image block is flattened into a one-dimensional pixel vector, and the one-dimensional pixel vector is mapped from the original pixel space to a high-dimensional space of a preset dimension based on a linear projection matrix to obtain an initial embedding vector. Add position encoding to the initial embedding vector to obtain an embedding vector with spatial position information; The initial embedding vectors are arranged in order of their spatial positions in the initial image to obtain the embedding vector sequence.

3. The method according to claim 1, characterized in that, The segmentation and embedding module using the black dot detection model performs image segmentation on the initial image to obtain several image patches, and linearly maps each image patch to a high-dimensional space to obtain an embedding vector sequence, including: The following steps are performed using the segmentation and embedding module of the black dot detection model: Based on a preset block size parameter, the initial image is divided into several image blocks using a sliding window step size smaller than the block size parameter, and adjacent image blocks at least partially overlap. For each image block, the two-dimensional pixel matrix within the image block is flattened into a one-dimensional pixel vector. The one-dimensional pixel vector is mapped from the original pixel space to a high-dimensional space of a preset dimension based on a linear projection matrix to obtain an initial embedding vector. The initial embedding vector is then position-encoded to obtain an embedding vector with position-encoded information. The embedding vectors are arranged in order of their spatial positions in the initial image to obtain the embedding vector sequence.

4. The method according to claim 1, characterized in that, The encoder employs a multi-head self-attention mechanism and a feedforward neural network stacked alternately. The encoder using the black spot detection model captures the relationships between black spots and background, black spots and noise points, and long-distance dependencies between individual black spots based on the embedded vector sequence. Attention weights are applied to the feature dimension to obtain a target feature sequence that enhances contextual information, including: The encoder of the black dot detection model performs the following steps: The embedded vector sequence is mapped to a query vector, a key vector, and a value vector. The similarity between the query vector and the key vector is calculated through a multi-head self-attention mechanism and then normalized to obtain the attention weight. The value vector is weighted and summed based on the attention weights to complete global context modeling, capture the long-distance dependencies between black points and background, black points and noise points, and each black point, and obtain a target feature sequence that enhances contextual information.

5. The method according to claim 4, characterized in that, The step of performing global context modeling on each of the embedding vectors in the embedding vector sequence to obtain the attention weight of each embedding vector includes: For each embedding vector, obtain the correlation evaluation value between the embedding vector and all key vectors in the embedding vector sequence, and normalize each correlation evaluation value to obtain the attention weight of the embedding vector.

6. The method according to claim 1, characterized in that, Before the initial image is segmented into several image patches using the segmentation embedding module of the black dot detection model, the method further includes: The image preprocessing module of the black spot detection model performs the following steps: The initial image is subjected to distortion correction and illumination equalization processing; The CLAHE algorithm is used to enhance the contrast between black points and the background to obtain the preprocessed initial image.

7. The method according to claim 1, characterized in that, The encoder includes sub-encoders with different depths, the black dot detection model further includes a multi-scale feature fusion module, and the method further includes: The following steps are performed by the multi-scale feature fusion module: A first feature map is obtained from a first depth layer of the encoder, a second feature map is obtained from a second depth layer of the encoder, and a third feature map is obtained from a third depth layer of the encoder, wherein the depth of the first depth layer is less than the depth of the second depth layer, and the depth of the second depth layer is less than the depth of the third depth layer. The second feature map and the third feature map are upsampled to the resolution of the first feature map by bilinear interpolation or transpose convolution, so that each feature map has the same spatial size; The first feature map, the second feature map, and the third feature map are concatenated or added element by element along the channel dimension, and a fused feature map is output after channel fusion through a convolutional layer.

8. The method according to claim 1, characterized in that, In the process of inputting the initial image into the black spot detection model and performing the following steps through the black spot detection model, the method further includes: The inference process of the black spot detection model is accelerated using the TensorRT inference optimization engine.

9. The method according to claim 1, characterized in that, The step of visualizing the black spot detection results includes: The black spot detection results are then overlaid with the initial image and displayed visually.

10. The method according to claim 1, characterized in that, After visualizing the black spot detection results, the method further includes: Generate a black spot detection report for the galvanized steel sheet. The black spot detection report includes one or more of the following: the number, distribution location, and distribution density of black spots of various sizes in the initial image.

Images