A deep learning-based strong reflection metal surface defect identification method and system

Abstract

The application relates to the technical field of intelligent detection, and discloses a strong-reflection metal surface defect recognition method and system based on deep learning. The method is as follows: image data of a metal surface is acquired; the image data is preprocessed, multi-level feature extraction is performed on the preprocessed metal image, a plurality of original feature maps corresponding to a plurality of levels are generated, a light reflection mask map corresponding to each original feature map is generated, a feature weight map corresponding to each original feature map is dynamically generated, the feature weight map is multiplied by the corresponding original feature map element by element, a plurality of weighted feature maps are obtained, feature compensation is performed on the plurality of weighted feature maps, and the plurality of compensated feature maps are fused and enhanced; positioning and classification of metal surface defects are performed based on the enhanced feature maps; and position, category and confidence information of defects are output. The application greatly improves the detection performance in a strong-reflection scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology, and in particular to a method and system for identifying defects on highly reflective metal surfaces based on deep learning. Background Technology

[0002] Metal surface defect detection is a crucial aspect of industrial production and equipment maintenance. In the power industry, metal fittings in high-voltage transmission lines are subjected to immense mechanical stress and environmental corrosion. Components such as tension clamps, suspension clamps, bolts, and pins are prone to defects like loose bolts, missing pins, and deformed crimping pipes. Failure to detect and address these defects promptly can lead to serious power grid accidents. In manufacturing, surface defects in metal parts and products, such as scratches, dents, and cracks, can affect product quality and lifespan, necessitating rigorous inspection during the production process.

[0003] Traditional metal surface defect detection mainly relies on manual visual inspection, which suffers from drawbacks such as low efficiency, high subjectivity, and susceptibility to fatigue. With the development of computer vision technology, automated detection methods based on image processing are gradually being applied in practice. Early methods were primarily based on traditional image processing techniques, such as edge detection, morphological manipulation, and texture analysis. These methods extract geometric or texture features from the metal surface and combine them with threshold judgments or classifiers for defect identification. For example, some researchers have proposed a bolt loosening detection method based on Canny edge detection and Hough transform, which determines the presence of looseness by detecting the integrity of the bolt contour. Other researchers have used Gabor filters to extract texture features from the metal surface and combined them with support vector machines for scratch and dent classification.

[0004] Traditional image processing methods have significant limitations. First, these methods rely on manually designed feature extraction algorithms, which are poorly adaptable to complex and varied defect patterns. Second, traditional methods are highly sensitive to lighting conditions; under uneven lighting or with strong reflections, the extracted features are often severely distorted, leading to a significant drop in detection performance. Particularly in strong outdoor lighting environments, metal surfaces exhibit severe specular reflection, appearing as bright, saturated white spots in the image. This strong reflection obscures the true texture information of the metal surface, rendering edge- or texture-based detection methods completely ineffective.

[0005] In recent years, deep learning technology has made groundbreaking progress in the field of computer vision. Object detection and image classification methods based on convolutional neural networks have achieved performance surpassing human capabilities in multiple visual tasks. Researchers have begun to apply deep learning methods to metal surface defect detection. Currently, mainstream methods employ general object detection frameworks such as YOLO, Faster R-CNN, and RetinaNet, training the network on metal defect datasets to learn the visual features of defects and perform automatic detection. These methods perform well under normal lighting conditions, achieving detection accuracy rates exceeding 90%.

[0006] However, existing deep learning-based defect detection methods still have significant shortcomings when dealing with strong reflective interference. General object detection networks employ standard convolutional operations in the feature extraction stage. This operation treats all regions in the image equally, failing to distinguish which regions have reliable features and which contain noise information due to reflection. In highly reflective scenes, convolutional networks often mistakenly identify the bright features of reflective areas as salient features, drawing the network's attention to these reflective areas while neglecting normal regions that actually contain defect information. This leads to a significant drop in detection accuracy under highly reflective environments, with an average decrease of 20% to 30%.

[0007] Some researchers have recognized this problem and attempted to suppress the effects of reflections through image preprocessing. Common methods include histogram equalization, adaptive contrast enhancement, and high dynamic range imaging. Histogram equalization enhances contrast by redistributing the gray values ​​of an image, but this global operation can lead to over-enhancing of normally functioning areas, introducing new noise. Adaptive contrast enhancement uses local windows to adjust contrast, achieving better results than histogram equalization, but for highly saturated reflective areas, the image information is completely lost, making recovery difficult with any post-processing method. High dynamic range imaging requires acquiring and fusing multiple images with different exposures. While it effectively suppresses reflections, it requires specialized hardware and a long acquisition time, making it difficult to apply in real-time detection scenarios.

[0008] Another prominent issue is the insufficient detection accuracy for small target defects. In metal surface defect detection, components such as bolts and pins are typically small in size, occupying only a few dozen pixels in a panoramic image, making them a typical small target detection problem. General-purpose target detection networks face the challenge of insufficient feature information when processing small targets. Due to the low resolution of deep feature maps in convolutional networks, small targets may completely disappear or become severely distorted after multiple downsampling. Although multi-scale detection methods such as feature pyramid networks alleviate this problem to some extent, these methods mainly focus on feature fusion between objects of different scales, lacking targeted feature enhancement mechanisms for small targets with specific geometric structures and texture features.

[0009] Furthermore, the environmental adaptability and generalization ability of existing methods need improvement. Different types of power towers, different shooting angles, and different weather conditions can all lead to significant changes in image features. If the scene coverage of the training data is insufficient, or if the network structure is not robust enough to scene changes, the model is prone to performance degradation in practical applications. Some researchers have attempted to improve the generalization ability of the model through data augmentation, but simple geometric transformations and color perturbations are difficult to simulate real strong reflection phenomena, and the effects are limited.

[0010] In summary, existing metal surface defect detection technologies have significant shortcomings in suppressing strong reflection interference, improving the accuracy of small target detection, and enhancing environmental adaptability. There is an urgent need to develop a targeted technical solution to address these issues. Summary of the Invention

[0011] To address the significant shortcomings of existing metal surface defect detection technologies in suppressing strong reflection interference, improving the accuracy of small target detection, and enhancing environmental adaptability, this invention provides a deep learning-based method and system for identifying strong reflection metal surface defects.

[0012] To achieve the above objectives, the present invention adopts the following technical solution, including: A deep learning-based method for identifying defects on highly reflective metal surfaces includes the following steps: S1, acquire image data of the metal surface; S2, preprocess the image data to obtain the preprocessed metal image; S3, performs multi-level feature extraction on the metal image, and generates multiple original feature maps corresponding to multiple levels; S4. For each original feature map, identify the reflective areas it contains, and generate a reflective mask map corresponding to the feature map based on all reflective areas within the same feature map. S5. Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map, and multiply the feature weight map with the corresponding original feature map element by element to obtain multiple weighted feature maps. Perform feature compensation on the multiple weighted feature maps to obtain multiple compensated feature maps. S6, fuse multiple compensated feature maps to obtain an enhanced feature map; S7, based on the enhanced feature map, locates and classifies metal surface defects, and outputs the location, category, and confidence information of the defects.

[0013] Preferably, step S1 involves capturing images of the metal surface using a fixedly installed or mobile camera device to obtain image data of the metal surface.

[0014] Preferably, step S2 uses an edge computing device or a cloud server to preprocess the image data; Preprocessing operations include: size adjustment and normalization.

[0015] Preferably, the specific steps of step S4 are as follows: S41, For each original feature map, use image analysis algorithms or neural networks to extract regions in the original feature map whose brightness values ​​exceed the first preset threshold and whose gradient changes are lower than the second preset threshold as candidate reflective regions. S42, perform connected component analysis on the candidate reflective regions to remove interfering regions that do not meet the conditions, thereby determining the final reflective regions; S43, Generate a reflective mask based on the final reflective area.

[0016] Preferably, the specific operation steps of step S5 are as follows: S51: Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map; for regions in the original feature map that are judged to be reflective, reduce their feature weight; for normal regions in the original feature map, increase their feature weight.

[0017] S52, multiply the feature weight map element-wise with the corresponding original feature map to obtain multiple weighted feature maps; S53. For pixels in the weighted feature map that are determined to be highly reflective, compensate for the feature loss caused by reflection occlusion to obtain a compensated feature map.

[0018] Preferably, the formula for calculating the feature weight map in step S51 is: ; in, Let (i,j) be the feature weight at pixel (i,j). Let (i,j) be the reflectivity probability or normalized reflectivity of pixel (i,j). , The feature activation intensity or normalized gradient intensity of pixel (i,j) is the intensity of the feature activation intensity. α and β are adjustable weighting coefficients.

[0019] Preferably, the feature compensation method in step S53 includes: Interpolation compensation method: Use bilinear interpolation or cubic spline interpolation algorithm to fill missing or sparse feature regions with pixel-level or feature-level numerical values; Diffusion compensation method: Based on anisotropic diffusion or solving the Poisson equation, the characteristic structure of the known region is smoothly propagated to the neighboring unknown region while preserving the edges. Generative compensation method: Utilize deep learning models such as generative adversarial networks or autoencoders to learn the intrinsic distribution of features, thereby repairing the missing feature maps; Cross-domain attention compensation method: Introduce an attention mechanism, and by calculating the similarity between feature blocks, non-locally propagate the feature information of the effective feature region to the missing or invalid feature region.

[0020] Preferably, the feature enhancement methods in step S6 include: A multi-scale feature fusion strategy is adopted to fuse feature maps at different resolution levels while preserving detailed information; Dilated convolution or deformable convolution is used to expand the receptive field and capture global contextual information at multiple scales; An attention mechanism is used to enhance the weights of feature channels or spatial regions associated with defects; Feature propagation from bottom to top and from top to bottom can be achieved by using feature pyramid networks or path aggregation networks.

[0021] This invention also proposes a deep learning-based system for identifying defects on highly reflective metal surfaces, which is applied to the aforementioned deep learning-based method for identifying defects on highly reflective metal surfaces. The metal surface defect identification system includes: Data acquisition module: Acquires image data of the metal surface; Preprocessing module: preprocesses the image data to obtain a preprocessed metal image; The backbone feature extraction module performs multi-level feature extraction on the metal image, generating multiple original feature maps corresponding to the multiple levels; Reflective region recognition network: For each original feature map, the reflective regions contained therein are identified, and based on all reflective regions within the same feature map, a reflective mask map corresponding to that feature map is generated; Feature compensation module: Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map, and multiply the feature weight map with the corresponding original feature map element by element to obtain multiple weighted feature maps. Perform feature compensation on multiple weighted feature maps to obtain multiple compensated feature maps. Fine-grained feature enhancement module: fuses multiple compensated feature maps to obtain an enhanced feature map; Defect detection module: Based on the enhanced feature map, it locates and classifies defects on the metal surface, and outputs the location, category and confidence information of the defects.

[0022] Preferably, the metal surface defect identification system comprises: Data storage device: used to store acquired images, detection results, and historical records; The data storage device supports local storage and cloud storage of data.

[0023] The advantages of this invention are: 1. This invention effectively suppresses the interference of strong reflections on feature extraction through a specially designed reflective area recognition and feature compensation mechanism. It maintains high detection accuracy even when reflective areas are partially obscured, significantly improving detection performance in high-reflection scenarios. Experimental data shows that in simulated direct sunlight test scenarios, the average accuracy of the method of this invention reaches 85.6%, while the traditional YOLO v5 method is only 72.4%, and the Faster R-CNN method is 68.9%. This invention represents an improvement of 13.2 percentage points compared to the best existing methods, an improvement of 18.2%. In extreme high-reflection scenarios, where more than 30% of the image area is obscured by reflections, the detection accuracy of the method of this invention can still be maintained above 78%, while existing methods generally drop below 60%. This significant improvement enables this invention to work reliably in outdoor high-light environments, expanding its application scope.

[0024] 2. This invention enhances the detection capability of small-sized components such as bolts and pins through multi-scale feature fusion and attention mechanism enhancement, significantly improving the recall rate of small target defects. For defects smaller than 16×16 pixels, the recall rate of this invention reaches 88.4%, while YOLO v5 achieves only 71.2% and Faster R-CNN 74.8%. This invention represents a 13.6 percentage point improvement over the best existing methods, an improvement of 18.2%. For extremely small targets between 8×8 and 16×16 pixels, the recall rate of this invention still reaches 82.3%, while existing methods are generally below 65%. High recall means fewer missed detections, which is of great significance for safety-critical applications such as power transmission line inspection, effectively reducing safety hazards caused by missed detections.

[0025] 3. This invention utilizes samples under various lighting conditions during the training phase, and the deep learning network possesses the ability to autonomously recognize and adapt to changes in lighting. This enables the invention to exhibit better generalization performance when facing new scenarios, enhancing its generalization ability and environmental adaptability. Cross-validation experiments show that when the training and test sets come from different types of power towers or different geographical locations, the performance degradation of the method in this invention is only 5% to 8%, while the degradation of existing methods is generally between 15% and 25%. This means that the model of this invention has a lower dependence on training data, requires less scenario adaptation work in actual deployment, and reduces application costs.

[0026] 4. This invention provides a complete solution from data annotation and model training to system deployment, facilitating engineering applications. The open-source annotation tools and training code allow users to customize training models according to their specific application scenarios. The modular design of the network architecture allows for flexible combination or replacement of components; for example, the backbone network can be configured with different architectures based on computational resources and accuracy requirements, and the detection head can be selected using a single-stage or two-stage method depending on the specific task. Furthermore, the core idea of ​​this invention—the feature compensation mechanism for strongly reflective objects—is not only applicable to metal fittings detection but can also be extended to other visual inspection tasks with strong reflection interference, such as glass surface defect detection and coating quality inspection, demonstrating broad application prospects. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the metal surface defect identification system in Example 1.

[0028] Figure 2 This is a schematic diagram of the metal surface defect identification system in Example 1.

[0029] Figure 3 for Figure 1 A schematic diagram of the reflective area recognition network and feature compensation module.

[0030] Figure 4 This is a schematic diagram of the fine-grained feature enhancement module in this embodiment.

[0031] Figure 5 for Figure 4 Internal structure diagram of the attention module.

[0032] Figure 6 This is a comparison chart of the convergence curves of the multi-task loss function in Example 1.

[0033] Figure 7 This is a flowchart of the metal surface defect identification method in Example 2. Detailed Implementation

[0034] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] It should be noted that when a component is said to be "installed on" another component, it can be directly on the other component or it may be in a component that is centered on it. When a component is said to be "set on" another component, it can be directly set on the other component or it may also be in a component that is centered on it. When a component is said to be "fixed to" another component, it can be directly fixed to the other component or it may also be in a component that is centered on it.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0037] Example 1

[0038] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of a module for a deep learning-based high-reflectivity metal surface defect recognition system provided in this embodiment. Figure 2 This is a schematic diagram of the overall metal surface defect identification system in Example 1. The metal surface defect identification system includes the following modules: I. Data Acquisition Module.

[0039] Acquire image data of the metal surface.

[0040] II. Preprocessing Module.

[0041] The image data is preprocessed to obtain a preprocessed metal image.

[0042] III. Main Feature Extraction Module.

[0043] Multi-level feature extraction is performed on metal images to generate multiple original feature maps corresponding to multiple levels.

[0044] IV. Reflective Area Recognition Network.

[0045] For each original feature map, the reflective regions contained therein are identified, and a reflective mask map corresponding to the feature map is generated based on all reflective regions within the same feature map.

[0046] V. Feature Compensation Module.

[0047] Based on each reflective mask, a feature weight map corresponding to each original feature map is dynamically generated. The feature weight map is then multiplied element-wise with the corresponding original feature map to obtain multiple weighted feature maps. Feature compensation is then performed on the multiple weighted feature maps to obtain multiple compensated feature maps.

[0048] VI. Fine-grained feature enhancement module.

[0049] Multiple compensated feature maps are fused to obtain an enhanced feature map.

[0050] VII. Defect Detection Module.

[0051] Based on the enhanced feature map, the system locates and classifies metal surface defects, and outputs the location, category, and confidence level information of the defects.

[0052] This embodiment 1 addresses the significant shortcomings of existing metal surface defect detection technologies in terms of strong reflection interference suppression, small target detection accuracy, and environmental adaptability. It proposes a deep learning-based strong reflection metal surface defect recognition system, which effectively improves the defect detection accuracy and recall rate for small target defects in strong reflection environments.

[0053] The following will provide a detailed introduction to each module: I. Data Acquisition Module.

[0054] Acquire image data of the metal surface.

[0055] In this embodiment, image data of the metal surface is obtained by capturing images of the metal surface using a fixedly installed or mobile camera device.

[0056] The image data covers different lighting conditions, including normal lighting, direct strong light, side lighting, and shadow environments; at the same time, the image data covers different shooting angles and distances to simulate diverse scenarios in real-world applications.

[0057] Image data of metal surfaces include: metal fittings in high-voltage transmission lines, metal parts or products in industrial manufacturing.

[0058] Metal fittings include: tension clamps, suspension clamps, bolts, pins, crimping pipes, and connecting plates; and the types of defects in metal fittings include loose bolts, missing pins, deformed crimping pipes, cracked clamps, and corrosion and wear.

[0059] Defect types in metal parts or products manufactured in industry include: surface scratches, dents, cracks, stains, oxidation, and discoloration.

[0060] II. Preprocessing Module.

[0061] The image data is preprocessed to obtain a preprocessed metal image.

[0062] In this embodiment, the image data is resized and normalized to obtain a preprocessed metal image.

[0063] Preprocessing operations include: size adjustment and normalization.

[0064] The preprocessing module uses edge computing devices or cloud servers to preprocess image data.

[0065] Edge computing devices include: embedded processors, graphics processors, or dedicated neural network accelerators.

[0066] III. Main Feature Extraction Module.

[0067] Multi-level feature extraction is performed on metal images to generate multiple original feature maps corresponding to multiple levels.

[0068] In this embodiment, the backbone feature extraction module is responsible for extracting multi-level feature representations from the preprocessed metal image and generating multiple original feature maps corresponding to multiple levels.

[0069] The backbone network that can be used in the main feature extraction module includes ResNet, CSPDarknet, or EfficientNet.

[0070] IV. Reflective Area Recognition Network.

[0071] For each original feature map, the reflective regions contained therein are identified, and a reflective mask map corresponding to the feature map is generated based on all reflective regions within the same feature map.

[0072] In this embodiment, the reflective region recognition network identifies reflective regions in each original feature map and generates a reflective mask map corresponding to the feature map based on all reflective regions within the same feature map, such as... Figure 3 As shown.

[0073] The specific steps for identifying the reflective area are as follows: S41, For each original feature map, use image analysis algorithms or neural networks to extract regions in the original feature map whose brightness values ​​exceed the first preset threshold and whose gradient changes are lower than the second preset threshold as candidate reflective regions. S42, perform connected component analysis on the candidate reflective regions to remove interfering regions that do not meet the conditions, thereby determining the final reflective regions; S43, Generate a reflective mask based on the final reflective area.

[0074] The reflective mask is used to characterize the reflective probability value or reflective intensity value of each pixel.

[0075] The method for identifying reflective areas using a neural network described in step S41 employs a semantic segmentation network. The original feature map obtained by the backbone feature extraction module is input into the semantic segmentation network, and a multi-channel semantic segmentation mask is output. The multi-channel semantic segmentation mask contains at least one channel for representing reflective areas.

[0076] V. Feature Compensation Module.

[0077] Based on each reflective mask, a feature weight map corresponding to each original feature map is dynamically generated. The feature weight map is then multiplied element-wise with the corresponding original feature map to obtain multiple weighted feature maps. Feature compensation is then applied to these multiple weighted feature maps to obtain multiple compensated feature maps, such as... Figure 3 As shown.

[0078] The specific operation steps of the feature compensation module are as follows: S51: Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map; for regions in the original feature map that are identified as reflective, reduce their feature weight to avoid interference from reflective noise to subsequent detection; for normal regions in the original feature map, especially edge regions with strong gradient features, increase their feature weight to enhance the expression of useful information.

[0079] S52, multiply the feature weight map element-wise with the corresponding original feature map to obtain multiple weighted feature maps; S53. For pixels in the weighted feature map that are determined to be highly reflective, compensate for the feature loss caused by reflection occlusion to obtain a compensated feature map.

[0080] The formula for calculating the feature weight map mentioned in step S51 is as follows: ; in, Let (i,j) be the feature weight at pixel (i,j). Let (i,j) be the reflectivity probability or normalized reflectivity of pixel (i,j). , This represents the feature activation intensity or normalized gradient intensity of pixel (i,j). α and β are adjustable weighting coefficients.

[0081] Reflection probability: The predicted value output by the deep learning semantic segmentation network (such as a [0,1] floating-point number after Sigmoid activation) represents the confidence of the network in determining that the point belongs to the reflective region based on the context semantics.

[0082] Normalized reflectivity: Using traditional digital image processing methods, the brightness / grayscale value of the pixel is directly extracted and linearly mapped to the [0,1] interval using an extreme value normalization formula (such as Value / 255).

[0083] When deployed on cloud servers or edge devices with sufficient computing power, the system preferentially uses "reflection probability" because it has strong anti-interference capabilities and can accurately distinguish between real and false reflections. When deployed on low-end embedded devices with extremely limited computing power and requiring extremely high real-time performance, the system can choose to use "normalized reflection intensity," sacrificing a little accuracy for ultimate inference speed.

[0084] The reflectivity probability is a semantic prediction value output by a deep learning segmentation network (such as a Sigmoid layer), which is more intelligent and can distinguish between real and fake reflectivity. "Normalized reflectivity intensity" is a direct mathematical normalization of the brightness of image pixels (such as Min-Maxscaling), which is fast to calculate but rather rigid.

[0085] "Feature activation intensity" is the response value extracted from the deep feature map of the neural network (representing the semantic defect regions that the network considers important); "normalized gradient intensity" is the rate of change of physical edges extracted using traditional operators.

[0086] In step S53, the feature compensation is achieved through any one or more combinations of the following methods: Interpolation compensation method: Use bilinear interpolation or cubic spline interpolation algorithm to fill missing or sparse feature regions with pixel-level or feature-level numerical values; Diffusion compensation method: Based on anisotropic diffusion or solving the Poisson equation, the characteristic structure of the known region is smoothly propagated to the neighboring unknown region while preserving the edges. Generative compensation method: Utilize deep learning models such as Generative Adversarial Networks (GAN) or Autoencoders (AE) to learn the intrinsic distribution of features, thereby repairing the missing feature maps; Cross-domain attention compensation method: Introduce an attention mechanism, and by calculating the similarity between feature blocks, non-locally propagate the feature information of the effective feature region to the missing or invalid feature region.

[0087] In this embodiment, the feature compensation module includes: Weight generation unit: Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map; for regions in the original feature map that are identified as reflective, reduce their feature weight to avoid interference from reflective noise to subsequent detection; for normal regions in the original feature map, especially edge regions with strong gradient features, increase their feature weight to enhance the expression of useful information.

[0088] Feature fusion unit: Multiplies the feature weight map element-wise with the corresponding original feature map to obtain multiple weighted feature maps; Feature compensation unit: For pixels in the weighted feature map that are determined to be highly reflective, compensate for the feature loss caused by reflection occlusion to obtain a compensated feature map.

[0089] VI. Fine-grained feature enhancement module.

[0090] Multiple compensated feature maps are fused and enhanced to obtain an enhanced feature map.

[0091] The fine-grained feature enhancement module fuses multiple compensated feature maps to obtain a fused feature map, and then enhances the fused feature map to obtain an enhanced feature map. The fine-grained feature enhancement module enhances feature representation through at least one of the following methods: Method 1: Employ a multi-scale feature fusion strategy to fuse feature maps at different resolution levels while preserving detailed information. Method 2: Use dilated convolution or deformable convolution to expand the receptive field in order to capture global contextual information at multiple scales; Method 3: Employ attention mechanisms to enhance the weights of feature channels or spatial regions related to defects; Method 4: Use feature pyramid networks or path aggregation networks to achieve bottom-up and top-down feature propagation.

[0092] The specific implementation steps of Method 1 include: (1) The backbone feature extraction module extracts multiple original feature maps corresponding to different levels of the metal image, and the original feature maps have different spatial resolutions and semantic levels; Shallow feature maps have high resolution and contain rich details, which is beneficial for locating small targets; deep feature maps have high semantic level, which is beneficial for target classification and recognition.

[0093] (2) Upsample the original feature map with low resolution and high semantics, and downsample the original feature map with high resolution and low semantics, so that the feature maps of different levels reach the same spatial size. (3) The original feature maps of different levels are fused by splicing, addition or weighted fusion; By using bottom-up and top-down feature propagation paths, the original feature maps at different levels are organically fused, so that the final feature map used for detection retains both details and strong semantics.

[0094] (4) Perform convolution processing on the fused feature map to generate the final feature map used for defect detection, i.e., the enhanced feature map.

[0095] Method 3 describes an attention mechanism that includes channel attention, spatial attention, and a combination of channel and spatial attention mechanisms. The channel attention mechanism learns the importance weights of different feature channels, enhancing channels relevant to metallic textures and suppressing irrelevant background channels. The spatial attention mechanism learns the importance weights of each spatial location on the feature map, highlighting key areas suspected of defects.

[0096] Please see Figure 4 and Figure 5 , Figure 4 This is a schematic diagram of the fine-grained feature enhancement module in this embodiment. Figure 5 for Figure 4 Internal structure diagram of the attention module (AM) Figure 4 C3, C4, and C5 are multiple compensated feature maps, while P3, P4, and P5 are enhanced feature maps.

[0097] In this embodiment, methods one, two, three, and four are used, and their specific correspondences are as follows: (1) Method 1 and Method 4 (collaborative work): “Multi-scale feature fusion” (Method 1) is mainly achieved by constructing “Feature Pyramid Network (FPN) or Path Aggregation Network (PANet)” (Method 4), which are represented in the network structure as upsampling and concatenation of feature maps.

[0098] (2) Method 2 (deformable convolution): It is usually placed at the end of the fusion enhancement before being output to the detection head to further adapt to the irregular shape of the defect.

[0099] (3) Method 3 (Attention Mechanism): It is embedded as a module (AM module) at the node of feature fusion to adjust the weights of the channels and space before and after fusion.

[0100] VII. Defect Detection Module.

[0101] Based on the enhanced feature map, the system locates and classifies metal surface defects, and outputs the location, category, and confidence level information of the defects.

[0102] In this embodiment, the defect detection module can use a single-stage detection head or a two-stage detection head to locate and classify metal surface defects in the enhanced feature map, and output the location, category and confidence information of the defects.

[0103] Single-stage detection heads include: YOLO's detection head; The two-stage detection head includes: the region proposal network of Faster R-CNN and the classification and regression head.

[0104] In other preferred embodiments, the metal surface defect identification system further includes: Data storage device: Used to store acquired images, detection results and historical records, and the data storage device supports local storage or cloud storage of data.

[0105] Defect alarm device: used for real-time data monitoring, and immediately triggers an audible and visual alarm when a defect is detected.

[0106] Display: Used to show the location, type, and confidence level information of defects.

[0107] To improve detection accuracy, this invention employs a multi-task learning strategy to train the metal surface defect identification system offline. The offline training phase includes the following steps: 1. Constructing the training dataset: The training dataset contains images of metal surfaces and their corresponding defect annotation information. The annotation information includes the location bounding box and category label of the defect, as well as the semantic segmentation annotation of the strong reflective area. This dual annotation strategy enables the network to learn how to identify and process reflective areas while learning the defect detection task, providing a supervision signal for subsequent reflection interference suppression.

[0108] 2. Define the loss function: The total loss function is defined as the weighted sum of the reflective area segmentation loss, defect classification loss, and defect location loss. Please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a comparison chart of the convergence curves of the multi-task loss function.

[0109] The formula for calculating the total loss function is: ; in, For semantic segmentation loss of reflective regions, Loss is classified as defect. To locate the loss of the defect, , , These are the weighting coefficients for each loss term.

[0110] Reflective area segmentation loss Cross-entropy loss, Dice loss, or a combination of cross-entropy loss and Dice loss are used to supervise the training of the reflective region recognition module; defect classification loss is also employed. By employing cross-entropy loss, Focal loss, or label smoothing loss, the weights of easily classified samples are reduced, causing the network to focus more on difficult-to-classify defective samples, thus alleviating the sample imbalance problem; defect localization loss... The regression accuracy of bounding boxes can be optimized by using IoU loss, GIoU loss, DIoU loss, or CIoU loss.

[0111] The weight coefficients of the three loss terms were determined experimentally. Typically, the weights are set as follows: reflectance segmentation loss weight 1, defect classification loss weight 3, and defect localization loss weight 2, reflecting the dominant role of defect detection and the auxiliary role of reflectance recognition.

[0112] When using a metal surface defect recognition system, the trained system is deployed on an edge computing device or cloud server to process the acquired images in real time. The pre-processed images are then fed into the network, sequentially passing through modules for backbone feature extraction, reflective area recognition, feature compensation, fine-grained feature enhancement, and defect detection, ultimately outputting the detection results.

[0113] To meet real-time requirements, this invention employs various model optimization techniques. Regarding network structure, the reflective region recognition module adopts a lightweight design, using depthwise separable convolutions to reduce computational load. For model deployment, model quantization converts floating-point operations to fixed-point operations, model pruning removes redundant parameters, and knowledge distillation transfers knowledge from a large model to a smaller model. Through these optimizations, the model maintains detection accuracy while achieving an inference speed of over 30 frames per second, meeting real-time detection requirements.

[0114] Example 2

[0115] This embodiment 2 proposes a method for identifying defects on highly reflective metal surfaces based on deep learning. Please refer to [link to relevant documentation]. Figure 7 , Figure 7 This is a flowchart of the metal surface defect identification method in Example 2. The operation steps of the metal surface defect identification method are as follows: S1, acquire image data of the metal surface; S2, preprocess the image data to obtain the preprocessed metal image; S3, performs multi-level feature extraction on the metal image, and generates multiple original feature maps corresponding to multiple levels; S4. For each original feature map, identify the reflective areas it contains, and generate a reflective mask map corresponding to the feature map based on all reflective areas within the same feature map. S5. Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map, and multiply the feature weight map with the corresponding original feature map element by element to obtain multiple weighted feature maps. Perform feature compensation on the multiple weighted feature maps to obtain multiple compensated feature maps. S6, fuse multiple compensated feature maps to obtain an enhanced feature map; S7, based on the enhanced feature map, locates and classifies metal surface defects, and outputs the location, category, and confidence information of the defects.

[0116] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying defects on highly reflective metal surfaces based on deep learning, characterized in that, Includes the following steps: S1, acquire image data of the metal surface; S2, preprocess the image data to obtain the preprocessed metal image; S3, performs multi-level feature extraction on the metal image, and generates multiple original feature maps corresponding to multiple levels; S4. For each original feature map, identify the reflective areas it contains, and generate a reflective mask map corresponding to the feature map based on all reflective areas within the same feature map. S5. Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map, and multiply the feature weight map with the corresponding original feature map element by element to obtain multiple weighted feature maps. Perform feature compensation on the multiple weighted feature maps to obtain multiple compensated feature maps. S6, fuse multiple compensated feature maps to obtain an enhanced feature map; S7, based on the enhanced feature map, locates and classifies metal surface defects, and outputs the location, category, and confidence information of the defects.

2. The method for identifying defects on highly reflective metal surfaces based on deep learning according to claim 1, characterized in that, Step S1 involves capturing images of the metal surface using a fixed or mobile camera device to obtain image data of the metal surface.

3. The method for identifying defects on highly reflective metal surfaces based on deep learning according to claim 1, characterized in that, Step S2 uses edge computing devices or cloud servers to preprocess the image data; Preprocessing operations include: size adjustment and normalization.

4. The method for identifying defects on highly reflective metal surfaces based on deep learning according to claim 1, characterized in that, The specific steps for step S4 are as follows: S41, For each original feature map, use image analysis algorithms or neural networks to extract regions in the original feature map whose brightness values ​​exceed the first preset threshold and whose gradient changes are lower than the second preset threshold as candidate reflective regions. S42, perform connected component analysis on the candidate reflective regions to remove interfering regions that do not meet the conditions, thereby determining the final reflective regions; S43, Generate a reflective mask based on the final reflective area.

5. The method for identifying defects on highly reflective metal surfaces based on deep learning according to claim 1, characterized in that, The specific steps for step S5 are as follows: S51: Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map; for regions in the original feature map that are identified as reflective, reduce their feature weight. For normal regions in the original feature map, increase their feature weights; S52, multiply the feature weight map element-wise with the corresponding original feature map to obtain the weighted feature map; S53. For pixels in the weighted feature map that are determined to be highly reflective, compensate for the feature loss caused by reflection occlusion to obtain a compensated feature map.

6. The method for identifying defects on highly reflective metal surfaces based on deep learning according to claim 5, characterized in that, The formula for calculating the feature weight map in step S51 is: ； in, Let (i,j) be the feature weight at pixel (i,j). Let (i,j) be the reflectivity probability or normalized reflectivity of pixel (i,j). , This represents the feature activation intensity or normalized gradient intensity of pixel (i,j). α and β are adjustable weighting coefficients.

7. The method for identifying defects on highly reflective metal surfaces based on deep learning according to claim 5, characterized in that, The feature compensation in step S53 is achieved through any one or more combinations of the following methods: Interpolation compensation method: Use bilinear interpolation or cubic spline interpolation algorithm to fill missing or sparse feature regions with pixel-level or feature-level numerical values; Diffusion compensation method: Based on anisotropic diffusion or solving the Poisson equation, the characteristic structure of the known region is smoothly propagated to the neighboring unknown region while preserving the edges. Generative compensation method: Utilize deep learning models such as generative adversarial networks or autoencoders to learn the intrinsic distribution of features, thereby repairing the missing feature maps; Cross-domain attention compensation method: Introduce an attention mechanism, and by calculating the similarity between feature blocks, non-locally propagate the feature information of the effective feature region to the missing or invalid feature region.

8. The traffic flow prediction method based on dual attention and depthwise separable graph convolution according to claim 1, characterized in that, Step S6 enhances the features in at least one of the following ways: Method 1: Employ a multi-scale feature fusion strategy to fuse feature maps at different resolution levels while preserving detailed information. Method 2: Use dilated convolution or deformable convolution to expand the receptive field in order to capture global contextual information at multiple scales; Method 3: Employ attention mechanisms to enhance the weights of feature channels or spatial regions related to defects; Method 4: Use feature pyramid networks or path aggregation networks to achieve bottom-up and top-down feature propagation.

9. A deep learning-based system for identifying defects on highly reflective metal surfaces, characterized in that, A deep learning-based method for identifying defects on highly reflective metal surfaces, applicable to any one of claims 1-8, comprises a metal surface defect identification system including: Data acquisition module: Acquires image data of the metal surface; Preprocessing module: preprocesses the image data to obtain a preprocessed metal image; The backbone feature extraction module performs multi-level feature extraction on the metal image, generating multiple original feature maps corresponding to the multiple levels; Reflective region recognition network: For each original feature map, the reflective regions contained therein are identified, and based on all reflective regions within the same feature map, a reflective mask map corresponding to that feature map is generated; Feature compensation module: Based on each reflective mask, dynamically generate a feature weight map corresponding to each original feature map, and multiply the feature weight map with the corresponding original feature map element by element to obtain multiple weighted feature maps. Perform feature compensation on multiple weighted feature maps to obtain multiple compensated feature maps. Fine-grained feature enhancement module: fuses multiple compensated feature maps to obtain an enhanced feature map; Defect detection module: Based on the enhanced feature map, it locates and classifies defects on the metal surface, and outputs the location, category and confidence information of the defects.

10. A deep learning-based system for identifying defects on highly reflective metal surfaces according to claim 9, characterized in that, Metal surface defect identification systems include: Data storage device: used to store acquired images, detection results, and historical records; the data storage device supports local storage and cloud storage of data.

Images