Plate surface defect detection method and system

Abstract

The invention provides a plate surface defect detection method and system, and relates to the technical field of image processing, and the method comprises the steps: obtaining an original multi-exposure image sequence of a to-be-detected plate under different exposure parameters, and carrying out precise alignment and high dynamic range image synthesis processing to generate a high dynamic range irradiance image; performing tone mapping and contrast enhancement processing on the high dynamic range irradiance image to generate an enhanced dynamic range representation image; constructing a plate surface defect detection model based on the neural network; taking the enhanced dynamic range representation image as the input of the model, and outputting a preliminary detection result containing a plurality of bounding box coordinates, category confidence and category probability; calculating a feature focusing score of each bounding box, and performing re-calibration and segmented screening on the preliminary detection result based on each feature focusing score to obtain a refined detection result; and the refined detection result is mapped and drawn to the enhanced dynamic range representation image, and a visual detection result graph is generated.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for detecting surface defects in sheet metal. Background Technology

[0002] During the production, processing, and finished product inspection of sheet materials, various defects such as scratches, pits, cracks, indentations, oxide spots, and foreign matter adhesion can easily occur on the surface of the sheet materials due to factors such as raw material quality, processing technology, equipment condition, and environmental factors. Therefore, rapid, accurate, and stable detection of surface defects in sheet materials is a crucial technical aspect for achieving product quality control and intelligent manufacturing.

[0003] Currently, the detection of surface defects in sheet materials mainly employs manual visual inspection, traditional machine vision inspection, and rule-based automated inspection methods. Among these, manual visual inspection relies on the operator's experience to observe and judge the sheet material surface; traditional machine vision methods typically use images acquired under fixed light source conditions and algorithms such as grayscale threshold segmentation, edge detection, and texture feature extraction to identify and locate defect areas; some automated inspection systems also combine preset rules or template matching methods to distinguish specific types of defects.

[0004] However, existing surface defect detection technologies for sheet materials are highly sensitive to changes in lighting conditions, the complexity of sheet surface textures, and noise interference. When defect morphologies are diverse, boundaries are unclear, or the contrast between defects and the background is low, it is difficult to reliably extract effective features, leading to a decrease in detection accuracy. Furthermore, detection methods based on fixed rules or templates rely on human experience and prior settings, resulting in poor versatility and scalability. They struggle to adapt to variations in the types and manifestations of surface defects under different materials and processing conditions, easily leading to missed detections and false positives. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method for detecting surface defects in sheet metal. This method addresses the problems of existing sheet metal surface defect detection technologies being sensitive to changes in lighting conditions, the complexity of sheet metal surface textures, and noise interference. Furthermore, it struggles to reliably extract effective features when defects have diverse morphologies, unclear boundaries, or low contrast with the background, leading to decreased detection accuracy. Additionally, detection methods based on fixed rules or templates rely on human experience and prior settings, resulting in poor versatility and scalability. They are ill-suited to adapting to variations in the types and manifestations of surface defects under different materials and processing conditions, easily leading to missed detections and false detections.

[0006] A first aspect of this invention provides a method for detecting surface defects in sheet metal, comprising: S1: Obtain the original multi-exposure image sequence of the material to be tested under different exposure parameters; S2: Perform precise alignment and high dynamic range image synthesis processing on the original multi-exposure image sequence to generate a high dynamic range irradiance image; S3: Perform tone mapping and contrast enhancement processing on the high dynamic range irradiance image to generate an enhanced dynamic range characterization image; S4: Construct a neural network-based plate surface defect detection model, wherein the neural network-based plate surface defect detection model includes a fusion channel attention feature enhancement module and a global feature adaptive focusing module; S5: Using the enhanced dynamic range characterization image as input to the neural network-based material surface defect detection model of the board, the output includes preliminary detection results containing multiple bounding box coordinates, class confidence and class probability; S6: Calculate the feature focus score of each bounding box, and based on each feature focus score, recalibrate and segment the preliminary detection results to obtain refined detection results; S7: Map and plot the refined detection results onto the enhanced dynamic range characterization image to generate a visualized detection result map containing defect category and location annotations.

[0007] A second aspect of the present invention provides a surface defect detection system for sheet metal, comprising: a processor and a memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the plate surface defect detection method as described in the first aspect.

[0008] A third aspect of the present invention provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the plate surface defect detection method as described in the first aspect.

[0009] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: In this embodiment of the invention, by constructing a multi-exposure image acquisition and high dynamic range image synthesis process, the contrast expression and detail display capabilities of sheet metal defects under strong reflection, weak illumination, or complex texture backgrounds are effectively enhanced, significantly improving the detection accuracy under complex lighting conditions. Simultaneously, a neural network model including a fusion-based channel attention feature enhancement module and a global feature adaptive focusing module is introduced, enhancing the robust recognition capability for complex defects such as blurred boundaries and diverse shapes. Furthermore, by calculating feature focusing scores and recalibrating and segmenting the preliminary detection results, the bounding box positioning accuracy is optimized, reducing the risk of false detections and missed detections caused by feature interference. This significantly improves the accuracy, stability, and general adaptability of the sheet metal surface defect detection system under complex working conditions. Attached Figure Description

[0010] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0011] Figure 1 This is a flowchart illustrating a method for detecting surface defects in sheet metal provided in an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of an improved YOLOv5s architecture provided in an embodiment of the present invention.

[0013] Figure 3 This is a schematic diagram of the structure of a plate surface defect detection system provided in an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] The method for detecting surface defects in sheet metal provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0016] Reference manual attached Figure 1 The diagram shows a flowchart of a method for detecting surface defects in sheet metal provided by an embodiment of the present invention.

[0017] This invention provides a method for detecting surface defects in sheet metal, which may include the following steps: S1: Obtain the original multi-exposure image sequence of the material to be tested under different exposure parameters.

[0018] Specifically, in the process of detecting surface defects in sheet metal, the sheet metal to be inspected is placed in a fixed imaging position. While maintaining a constant relative position between the camera and the sheet metal, multiple consecutive images are taken of the same surface area. During the imaging process, the camera's aperture and ISO parameters are kept constant. By adjusting the exposure time or equivalent exposure parameters, multiple original images with different exposure levels are sequentially acquired within a preset exposure range, thus forming a multi-exposure image sequence covering low, medium, and high exposure ranges.

[0019] Specifically, the multi-exposure image sequence is used to simultaneously record the brightness changes and texture details of the board surface in both strong and weak reflective areas, so as to provide complete raw input data for subsequent high dynamic range image synthesis and illumination removal processing.

[0020] It should be noted that those skilled in the art can set the size of the preset exposure range according to actual needs, and this invention does not limit this.

[0021] S2: Perform precise alignment and high dynamic range image synthesis on the original multi-exposure image sequence to generate a high dynamic range irradiance image.

[0022] Precise alignment processing refers to the process of correcting the spatial correspondence between multiple exposure images acquired on the same material surface before fusing them. This process ensures that the material surface areas at the same physical location in different exposure images remain spatially consistent by eliminating pixel-level offsets caused by factors such as minor jitter in the imaging system, trigger delays, or mechanical vibrations.

[0023] High dynamic range image synthesis refers to the process of constructing an irradiance image that can simultaneously characterize the details of both bright and low-brightness areas by comprehensively utilizing the brightness information contained in the images under different exposure conditions, based on the accurate alignment of multi-exposure images.

[0024] In one possible implementation, S2 specifically includes: S201: The original multi-exposure image sequence is sorted sequentially, and the geometric consistency of each frame is ensured according to the acquisition constraints of the fixed camera position, so as to obtain the target multi-exposure input sequence for fusion.

[0025] S202: Read the exposure time set and bind the exposure time set with the corresponding target multi-exposure input sequence.

[0026] S203: Based on the target multi-exposure input sequence after binding processing, the Debevec merging algorithm is used to fuse pixel observations under different exposures to generate a high dynamic range irradiance image.

[0027] Among them, the Debevec merging algorithm is a classic method for high dynamic range image synthesis. Its core idea is to use multiple images of the same scene acquired at different exposure times, and map the gray values ​​of pixels to the real scene irradiance by estimating the camera's response function.

[0028] It should be noted that the Debevec merging algorithm is existing technology, and will not be elaborated upon here.

[0029] Specifically, the acquired original multi-exposure image sequence is arranged in ascending order of exposure time, ensuring that each exposure image is geometrically consistent, thus forming the target multi-exposure input sequence for subsequent fusion processing. Next, the exposure time information corresponding to each exposure image is read to form an exposure time set, and this set is bound one-to-one with each frame in the target multi-exposure input sequence to define the exposure parameters of each image. After binding the image sequence with the exposure time, the Debevec merging algorithm is used to fuse pixel observations under different exposure conditions based on the target multi-exposure input sequence. By weighting the responses of the same pixel in different exposure images, its true irradiance information is recovered, thereby generating a high dynamic range irradiance image that simultaneously preserves details in both bright and dark areas.

[0030] In this embodiment of the invention, by performing precise alignment processing on the original multi-exposure image sequence, pixel-level misalignment caused by factors such as minor jitter of the imaging system, triggering errors or mechanical vibrations is effectively eliminated, avoiding ghosting, false edges or brightness distortion in the subsequent image fusion process, thereby ensuring the spatial consistency of the multi-exposure images.

[0031] S3: Perform tone mapping and contrast enhancement processing on the high dynamic range irradiance image to generate an enhanced dynamic range characterization image.

[0032] It should be noted that in the scenario of detecting defects on the surface of sheet materials, due to the common strong reflection and uneven illumination on the surface of sheet materials, existing processing methods based on single tone mapping or uniform contrast enhancement are difficult to simultaneously take into account the defect representation effect of bright spot areas and non-bright spot areas: when using global tone mapping, small defects in bright spot areas are easily over-compressed and lost. On the other hand, when using local tone mapping or uniform enhancement strategies, the background texture in non-reflective areas is easily over-magnified, thereby introducing a large number of pseudo-structures and leading to false detections.

[0033] Therefore, this invention innovatively introduces adaptive reflection intensity weights to continuously model the reflection characteristics of different brightness regions. Based on this, it combines a local contrast enhancement mechanism for reflection perception to prioritize the preservation of real defect details and suppress over-enhancement effects in bright spot regions, while enhancing the separability between defects and the background in non-bright spot regions. This provides more accurate image data for the detection model, thereby improving defect visibility and detection stability under complex lighting conditions and effectively reducing the risk of false detection and missed detection.

[0034] In one possible implementation, S3 specifically includes: S301: Extract the luminance and color components from the high dynamic range irradiance image to form a tone mapping input.

[0035] Specifically, the high dynamic range irradiance image is converted from RGB representation to a luminance-color separated representation. The luminance component is directly extracted from the representation as the processing object for tone mapping, while the corresponding color component is retained for subsequent color reconstruction.

[0036] S302: Based on tone mapping input, construct tone mapping basic quantities through tone mapping algorithm.

[0037] It should be noted that the tone mapping baseline refers to the intermediate brightness characterization result obtained after preliminary tone mapping of a high dynamic range image, which is used for subsequent brightness analysis and further adjustment.

[0038] Optionally, the tone mapping algorithm selects a global TMO.

[0039] Global TMO is an image processing method that performs tone mapping based on the overall brightness distribution of the entire image. During the processing, it uses a uniform mapping function for all pixels to compress the brightness values ​​in high dynamic range images to a low dynamic range that can be displayed or processed.

[0040] Specifically, for the luminance component, a preset tone mapping algorithm is used to perform dynamic range compression processing, mapping the high dynamic range luminance to a low dynamic range suitable for subsequent processing, thereby obtaining a mapping result with continuous and overall stable luminance distribution. After completing the luminance mapping, the original color components are retained unchanged, or the color components are adjusted proportionally according to the mapped luminance components, thus maintaining the overall color consistency of the image. Through the above processing, an intermediate result containing preliminary compressed luminance information and maintaining consistent color relationships is obtained, serving as the basis for subsequent tone mapping calculations of reflection intensity weights and luminance component branch processing.

[0041] S303: Calculate adaptive reflectance intensity weights based on the luminance component in the hue mapping fundamentals. in, This represents the adaptive reflection intensity weight at pixel location x, where x represents the pixel coordinates in the image. This represents the Sigmoid mapping function. This represents the brightness value at pixel position x. Indicates the brightness threshold. Indicates the smoothing control parameters. This represents an exponential function.

[0042] It should be noted that the brightness threshold and smoothing control parameters are set based on the experience of those skilled in the art.

[0043] In this embodiment of the invention, by calculating an adaptive reflection intensity weight, pixel brightness information can be continuously mapped to a reflection intensity level, creating a smooth transition between reflective and non-reflective areas. This avoids the boundary abruptness and noise sensitivity issues caused by hard threshold segmentation. This weight reflects both the deviation of pixel brightness from the reflection threshold and adjusts the sensitivity of weight changes through smooth control parameters. This provides a stable and controllable weight basis for subsequent adaptive fusion of brightness components and contrast enhancement, effectively improving the robustness of brightness processing under complex lighting conditions.

[0044] S304: Calculate the stable luminance component using the Ward histogram method.

[0045] Among them, the Ward histogram method is a tone mapping adjustment method based on the distribution of brightness histogram. Its core idea is to perform statistical analysis on the brightness histogram of high dynamic range image and adaptively compress and remap the brightness distribution while maintaining overall visibility.

[0046] S305: Calculates the detail luminance component using the Reinhard tone mapping operator.

[0047] Among them, the Reinhard tone mapping operator is a commonly used high dynamic range image brightness mapping method. Its core idea is to simulate the human eye's perception of brightness changes and nonlinearly compress the brightness values ​​in the high dynamic range image so that the details of the bright and dark areas can be expressed simultaneously within a limited dynamic range.

[0048] It should be noted that the Ward histogram method and the Reinhard tone mapping operator are existing technologies, and will not be elaborated upon here.

[0049] S306: Based on adaptive reflection intensity weighting, the stable luminance component and the detail luminance component are adaptively fused to obtain the fused luminance value: in, This represents the blended brightness value at pixel position x. This represents the detail luminance component at pixel position x. This represents the stable luminance component at pixel position x.

[0050] Specifically, bright spots / highly reflective areas More often Preserve details. Non-bright spot areas. More often Maintain stability and low false positives.

[0051] It should be noted that, in order to avoid over-enhancing of bright spots, this invention innovatively uses local contrast enhancement to process the blended brightness, thereby improving the separability of defects from the background.

[0052] S307: Perform local contrast enhancement on the blended brightness value to obtain adaptive enhanced brightness: in, This represents the adaptive enhanced brightness value at pixel position x. This represents the average brightness within the local neighborhood of pixel position x. This represents the local contrast enhancement coefficient at pixel location x. This represents the preset minimum contrast enhancement factor. This indicates the preset maximum contrast enhancement factor.

[0053] It should be noted that the preset minimum contrast enhancement coefficient and the preset maximum contrast enhancement coefficient are set by those skilled in the art based on experience.

[0054] In this embodiment of the invention, by using the local brightness average as a reference to enhance the contrast of the fused brightness, the local brightness difference between the defect area and the background can be effectively amplified without changing the overall brightness level. Simultaneously, by introducing a contrast enhancement coefficient associated with the adaptive reflection intensity weight, a smaller enhancement intensity is applied to highly reflective areas, while a larger enhancement intensity is applied to non-reflective areas. This avoids over-enhancing bright spots and improves the separability between defects in non-reflective areas and the background.

[0055] S308: Performs Gamma correction and clamp truncation on the adaptive enhanced brightness value to generate an enhanced dynamic range characterization image.

[0056] Gamma correction is a nonlinear mapping method used to adjust the brightness distribution of an image. Its main function is to change the correspondence between image brightness and pixel values ​​to better match the human eye's perception of brightness changes.

[0057] Clamping is a constraint operation used to limit the range of pixel values. It prevents the brightness or color components from being abnormally amplified or overflowed during processing by limiting the pixel values ​​in the image to the range between the minimum and maximum values.

[0058] Specifically, by applying Gamma correction to the adaptively enhanced brightness values, the perceptual weights of brightness changes can be redistributed, making the brightness differences related to defects more clearly distinguishable within a limited dynamic range. Simultaneously, clamping is used to limit the brightness values ​​to a set range, avoiding extreme numerical interference introduced by local enhancement or nonlinear mapping. After these processes, the generated image meets the input requirements of subsequent defect detection models in terms of brightness distribution and numerical range, thus forming an enhanced dynamic range characterization image for detecting defects on sheet metal surfaces.

[0059] It should be noted that Gamma correction and clamp truncation are existing technologies, and will not be elaborated upon here.

[0060] In this embodiment of the invention, by introducing adaptive reflection intensity weights and combining adaptive fusion and enhancement processing of stable brightness and detail brightness, the invention can dynamically adjust the brightness characterization method according to the reflection characteristics of different areas on the board surface. While suppressing over-enhancement of strong reflection areas, it improves the separability between defects and background in non-reflective areas. Thus, under complex lighting conditions, it can simultaneously ensure brightness stability and defect detail integrity, reduce the risk of false detection and missed detection, and provide more robust and reliable image input for subsequent board surface defect detection.

[0061] Reference manual attached Figure 2 The diagram shows an architecture schematic of an improved YOLOv5s provided by an embodiment of the present invention.

[0062] S4: Construct a neural network-based surface defect detection model for sheet metal, which includes a fusion channel attention feature enhancement module and a global feature adaptive focusing module.

[0063] In one possible implementation, the neural network-based plate surface defect detection model uses an improved YOLOv5s framework, which includes a backbone network, a neck network, and a detection head.

[0064] In the backbone network, the C3 module in the conventional backbone network is replaced with a fusion channel attention feature enhancement module.

[0065] It should be noted that the C3 module in a conventional backbone network is mainly used to improve feature reuse and computational efficiency. However, it treats all channel features equally during feature extraction and lacks the ability to explicitly model channels relevant to defect discrimination. In scenarios with complex surface textures, small defect sizes, and low contrast, defect features are easily submerged by background features during channel fusion. This invention replaces the conventional C3 module with a fusion-based channel attention feature enhancement module. Through multi-branch feature extraction and channel attention weighting mechanisms, it guides the network to adaptively enhance feature channels with higher relevance to defect discrimination while suppressing background texture interference. Without changing the overall network topology, it can significantly improve the feature representation ability and detection stability for small and multi-shaped surface defects of sheet materials.

[0066] In the backbone network, a global feature adaptive focusing module is added to jointly model the channel dimension and spatial dimension of the intermediate feature map of the backbone network and adaptively allocate the weight of each dimension feature to the output result.

[0067] It should be noted that conventional YOLOv5s detection models primarily rely on local convolutional features and multi-scale feature fusion for target discrimination, lacking explicit modeling of the importance of different spatial regions in the entire image. When the background texture of the board surface is complex and the defect distribution is sparse, it is prone to false responses to local noise areas. This invention introduces a global feature adaptive focusing module into the backbone network. By jointly modeling the channel and spatial dimensions of the feature map, it guides the model to adaptively adjust the contribution of each spatial region's features to the detection results globally, suppressing background and reflection interference and highlighting true defect areas, thus significantly improving the stability and reliability of the detection results.

[0068] In the neck network, the C3 module in the conventional neck network is replaced with a fusion channel attention feature enhancement module.

[0069] In one possible implementation, the specific architecture of the neural network-based plate surface defect detection model is as follows: The input end of the backbone network is connected to the first CBS module, and the input end of the backbone network is used to input the enhanced dynamic range characterization image.

[0070] The first CBS module is sequentially connected to the second CBS module, the first fusion channel attention feature enhancement module, the third CBS module, the second fusion channel attention feature enhancement module, the fourth CBS module, the third fusion channel attention feature enhancement module, the fifth CBS module, the fourth fusion channel attention feature enhancement module, and the SPPF module. The output of the SPPF module is connected to the Global Feature Adaptive Focusing module (GAM Attention). In the GAM Attention module, the channel dimension and spatial dimension of the intermediate feature map output by the backbone network are jointly modeled, and the weights of each dimension feature are adaptively assigned to the output result.

[0071] The CBS module and SPPF module are existing technologies, and will not be described in detail here.

[0072] The second fusion channel attention feature enhancement module, the third fusion channel attention feature enhancement module, and the global feature adaptive focusing module are connected. The input end of the neck network is used to input the first scale feature map (P3) output by the second fusion channel attention feature enhancement module, the second scale feature map (P4) output by the third fusion channel attention feature enhancement module, and the third scale feature map (P5) output by the global feature adaptive focusing module.

[0073] The global feature adaptive focusing module is sequentially connected to the sixth CBS module and the first upsampling module. The output feature map of the first upsampling module is concatenated with the second-scale feature map (P4), and the concatenated feature is input to the fifth fusion channel attention feature enhancement module. The fifth fusion channel attention feature enhancement module is sequentially connected to the seventh CBS module and the second upsampling module. The output feature map of the second upsampling module is concatenated with the first-scale feature map (P3), and the concatenated feature is input to the sixth fusion channel attention feature enhancement module. The sixth fusion channel attention feature enhancement module is connected to the eighth CBS module. The output feature map of the eighth CBS module is concatenated with the output feature map of the seventh CBS module, and the concatenated feature is input to the seventh fusion channel attention feature enhancement module. The seventh fusion channel attention feature enhancement module is connected to the ninth CBS module. The output feature map of the ninth CBS module is concatenated with the output feature map of the sixth CBS module, and the concatenated feature is input to the eighth fusion channel attention feature enhancement module. The output feature maps of the sixth, seventh, and eighth fusion channel attention feature enhancement modules are used as the multi-scale fusion feature outputs of the neck network and are respectively input to the input end of the corresponding scale of the detection head.

[0074] Optionally, the CBS module consists of a convolutional layer, a batch normalization layer, and an activation function layer (SiLU) connected sequentially. The SPPF module consists of a channel-compressed convolutional layer, multiple serial max pooling layers, and a channel-integrated convolutional layer connected sequentially.

[0075] In one possible implementation, the fusion channel attention feature enhancement module includes a first convolutional branch and a second convolutional branch.

[0076] The first convolutional branch connects the first convolutional layer and the CSPNeXtBlock module (a NeXt feature block with cross-stage connections) sequentially. The second convolutional branch includes a second convolutional layer. The output feature map of the CSPNeXtBlock module is concatenated with the output feature map of the second convolutional layer along the channel dimension. The concatenated feature map is then fed into the channel attention module and the convolutional module sequentially. The output feature map of the convolutional module serves as the output feature map of the fused channel attention feature enhancement module.

[0077] The channel attention module is used to adaptively weight the responses of each channel of the concatenated features, and the convolution module is used to integrate and output the weighted features.

[0078] It should be noted that by adopting a multi-branch feature extraction and splicing strategy, combined with a channel attention mechanism, the model is guided to automatically focus on the channel information of discriminative defect regions, thereby improving the overall feature expression capability and detection accuracy.

[0079] In this embodiment of the invention, by introducing a fusion-based channel attention feature enhancement module and a global feature adaptive focusing module into the neural network-based board surface defect detection model, the model's ability to express minute defect features against complex texture backgrounds is effectively enhanced without changing the overall topology of the original detection framework. Simultaneously, combined with processed high-quality image input, the model's discrimination of board surface defects is more stable under complex lighting and texture conditions, further improving the overall accuracy and reliability of board surface defect detection.

[0080] S5: The enhanced dynamic range representation image is used as the input to the neural network-based plate surface defect detection model, and the output includes preliminary detection results containing multiple bounding box coordinates, class confidence, and class probability.

[0081] Specifically, after generating the enhanced dynamic range representation image, it is input into the constructed sheet metal surface defect detection model. The enhanced dynamic range representation image first undergoes multi-layer feature extraction and multi-scale feature fusion through the model's backbone and neck networks. During this process, the model can progressively extract structural features and semantic information related to sheet metal surface defects from the image. Subsequently, based on the fused multi-scale features, the detection head predicts candidate regions at different spatial locations and scales, outputting the corresponding bounding box coordinates, defect category confidence scores, and probability information for each category, thus forming a preliminary detection result containing multiple candidate defect targets.

[0082] S6: Calculate the feature focus score of each bounding box, and based on the feature focus score, recalibrate and segment the preliminary detection results to obtain refined detection results.

[0083] Feature focus score is an evaluation metric used to measure the degree to which a detection model pays attention to the corresponding region of a candidate target in the feature map when predicting the target. This score reflects the degree of matching between the candidate target region and defect-related features at both spatial and semantic levels by utilizing attention information generated within the network.

[0084] In one possible implementation, S6 specifically includes: S601: During the inference process of the neural network-based plate surface defect detection model, obtain the spatial attention weights output by the global feature adaptive focusing module at the corresponding feature layer.

[0085] S602: Based on the coordinates of each bounding box, scale interpolation and spatial alignment are performed on each spatial attention weight to obtain aligned spatial attention weights.

[0086] Specifically, based on the candidate box coordinate scale output by the plate surface defect detection model, the spatial attention weights are subjected to scale interpolation and spatial alignment processing to obtain an attention weight map consistent with the candidate box coordinate system, so as to perform pixel-level statistics in the candidate box area in the future.

[0087] S603: Extract the attention weights corresponding to the bounding box coordinates from the alignment space attention weights, and calculate the local attention mean of the current bounding box based on each attention weight.

[0088] Specifically, the average value of all pixels within the captured attention weight region is calculated, and the resulting average value is used as the local attention mean value corresponding to the candidate bounding box, which reflects the overall attention of the model to defect-related features within the bounding box region.

[0089] S604: Based on the alignment space attention weights, calculate the global attention mean as the reference benchmark for the whole image attention, and combine the local attention mean and the global attention mean to calculate the feature focus score of each bounding box.

[0090] Specifically, the attention weights of all pixel positions in the aligned spatial attention weight map are statistically analyzed, and the average of the attention weights of all pixels is taken to obtain the global attention mean, which is used to characterize the overall feature attention level of the model in the current image and serves as a reference benchmark for subsequent calculation of the feature focus degree of each candidate bounding box.

[0091] Optionally, the feature focus score is calculated as follows: in, This represents the feature focus score corresponding to the i-th bounding box. This represents the mean local attention value within the i-th bounding box region. This represents the mean of global attention. This represents a constant value, set based on experience.

[0092] S605: Based on the feature focus scores, the class confidence of the corresponding bounding box is recalibrated to obtain the recalibrated class confidence: in, This represents the class confidence score after the i-th bounding box has been recalibrated. This represents the original category confidence score of the plate surface defect detection model for the i-th candidate bounding box. This represents the mapping function used to map feature-focused scores to confidence adjustment coefficients.

[0093] Optionally, a piecewise linear mapping approach can be used for adjustment. Specifically, based on the feature focus score, the bounding box is divided into three cases: low focus, medium focus, and high focus. When the feature focus score is lower than the first focus threshold, the corresponding mapping coefficient is set to a small fixed value to reduce the confidence of the bounding box and suppress detection results with low focus. When the feature focus score is between the first and second focus thresholds, the mapping coefficient is set to a neutral value to keep the original confidence of the bounding box unchanged. When the feature focus score is not less than the second focus threshold, the mapping coefficient is set to a fixed value greater than 1 to enhance the confidence of high-focus bounding boxes.

[0094] Specifically, the first focusing threshold is less than the second focusing threshold.

[0095] It should be noted that those skilled in the art can set the values ​​of the first focus threshold and the second prefocus threshold according to actual needs, and this invention does not limit such settings.

[0096] In this embodiment of the invention, the segmented mapping method can achieve differentiated adjustment of detection results with different degrees of focus without changing the detection model structure, thereby increasing the retention probability of high-confidence detection results and reducing the interference of low-confidence detection results on the final detection output.

[0097] S606: Using a focus threshold based on brightness perception, the bounding boxes are segmented and filtered to obtain a set of target bounding boxes.

[0098] It should be noted that existing methods for detecting surface defects in sheet materials typically employ a uniform confidence threshold or non-maximum suppression strategy during the result screening stage. These methods fail to distinguish between reflective and non-reflective areas in terms of imaging characteristics and false detection risk, and do not fully utilize the attention information already learned within the network. Consequently, they are prone to generating high-confidence false detections in highly reflective areas and missing genuine minute defects in non-reflective areas.

[0099] Therefore, this invention innovatively introduces a feature focus threshold mechanism based on brightness perception in the detection result screening stage. It first distinguishes the reflectivity of the candidate bounding box, and then combines the feature focus score calculated by the network attention information to adopt a differentiated screening strategy for the detection results of different regions. This enables fine-grained control over the reliability of the detection results, effectively reduces the false detection rate under strong reflection conditions, and improves the overall detection reliability.

[0100] In one possible implementation, S606 specifically includes: S6061: Generate a brightness map based on the enhanced dynamic range representation image.

[0101] Specifically, the enhanced dynamic range characterization image is converted into a luminance and color separation representation, and the luminance component is extracted as a luminance map for subsequent luminance analysis.

[0102] S6062: Extract the pixel region of the luminance map and the corresponding bounding box of the luminance map, and calculate the average luminance value within the pixel region of the bounding box.

[0103] S6063: Determine whether the average brightness value is greater than the preset brightness value. If yes, classify the bounding box pixel area as a reflective area bounding box and proceed to S6064. Otherwise, classify the bounding box pixel area as a non-reflective area bounding box and proceed to S6065.

[0104] It should be noted that those skilled in the art can set the preset brightness value according to actual needs, and this invention does not limit that.

[0105] S6064: Determine whether the feature focus score of each reflective region bounding box is greater than or equal to the reflective region threshold. If yes, retain the reflective region bounding box. Otherwise, discard the reflective region bounding box.

[0106] S6065: Determine whether the feature focus score of each non-reflective region bounding box is greater than or equal to the non-reflective region threshold. If yes, retain the non-reflective region bounding box. Otherwise, discard the non-reflective region bounding box.

[0107] S6066: Combine the retained bounding boxes of reflective and non-reflective areas to obtain a set of target bounding boxes.

[0108] Optionally, the reflective area threshold is greater than the non-reflective area threshold.

[0109] It should be noted that those skilled in the art can set the size of the reflective area threshold and the non-reflective area threshold according to actual needs, and this invention does not limit this.

[0110] Specifically, firstly, a brightness map is generated based on the enhanced dynamic range representation image. Then, based on the candidate bounding box positions output by the detection model, the corresponding pixel regions are extracted from the brightness map, and the average brightness value of each bounding box is calculated. Next, the average brightness value is compared with a brightness threshold. Bounding boxes with values ​​greater than the threshold are identified as reflective region bounding boxes, while the rest are identified as non-reflective region bounding boxes. For both reflective and non-reflective region bounding boxes, their retention is determined based on corresponding feature focus thresholds; bounding boxes that do not meet the threshold conditions are discarded. Finally, the retained bounding boxes are merged to obtain a bounding box set.

[0111] S607: Sort the calibration category confidence scores of the target bounding box set in descending order, and perform non-maximum suppression based on the sorting results to obtain refined detection results.

[0112] Specifically, the target bounding boxes are sorted from highest to lowest confidence level according to their calibrated categories. The bounding box with the highest current confidence level is then selected as the retained bounding box. For each retained bounding box and each subsequent bounding box in the sorted sequence, the overlap between any two bounding boxes is calculated. If the overlap is greater than a preset overlap, the lower-ranked bounding boxes are suppressed. If the overlap is not greater than the preset overlap, the lower-ranked bounding boxes are retained. This process is repeated until all bounding boxes have been traversed, and the set of unsuppressed bounding boxes is output as the refined detection result.

[0113] It should be noted that those skilled in the art can set the preset overlap size according to actual needs, and this invention does not limit this.

[0114] S7: Map and plot the refined inspection results onto the enhanced dynamic range representation image to generate a visualized inspection result map containing defect category and location annotations.

[0115] Specifically, based on the bounding box location, defect category, and corresponding calibration confidence level contained in the detection results, bounding boxes are drawn at the corresponding positions in the enhanced dynamic range representation image, and the corresponding defect category labels and confidence level information are overlaid and displayed near the bounding boxes. Different defect categories are distinguished through a unified color coding or annotation style, thereby generating a visual detection result image that intuitively reflects the defect location and category information, facilitating subsequent manual inspection, quality assessment, or result storage and analysis.

[0116] In this embodiment of the invention, by mapping and plotting the refining test results onto an enhanced dynamic range characterization image, the spatial location, category information, and confidence level of defects on the plate surface can be presented in an intuitive way, making it easy for operators to quickly understand and verify the test results. This visualization method helps reduce the difficulty of manual interpretation and improves the interpretability and usability of the test results.

[0117] Reference manual attached Figure 3 The diagram shows a structural schematic of a plate surface defect detection system provided in an embodiment of the present invention.

[0118] This invention provides a plate surface defect detection system 20, including: a processor 201 and a memory 202; The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-described method for detecting surface defects in sheet metal and achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0119] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0120] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0121] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0122] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0123] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0125] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0128] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] This invention provides a readable storage medium comprising: storing a program or instructions on the readable storage medium, wherein when the program or instructions are executed by a processor, the program or instructions implement the steps of the above-described method for detecting surface defects in sheet metal, and can achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting surface defects in sheet metal, characterized in that, include: S1: Obtain the original multi-exposure image sequence of the material to be tested under different exposure parameters; S2: Perform precise alignment and high dynamic range image synthesis processing on the original multi-exposure image sequence to generate a high dynamic range irradiance image; S3: Perform tone mapping and contrast enhancement processing on the high dynamic range irradiance image to generate an enhanced dynamic range characterization image; S4: Construct a neural network-based plate surface defect detection model, wherein the neural network-based plate surface defect detection model includes a fusion channel attention feature enhancement module and a global feature adaptive focusing module; S5: Using the enhanced dynamic range characterization image as the input to the neural network-based plate surface defect detection model, the output includes preliminary detection results containing multiple bounding box coordinates, class confidence, and class probability; S6: Calculate the feature focus score of each bounding box, and based on each feature focus score, recalibrate and segment the preliminary detection results to obtain refined detection results; S7: Map and plot the refined detection results onto the enhanced dynamic range characterization image to generate a visualized detection result map containing defect category and location annotations.

2. The method for detecting surface defects in sheet metal according to claim 1, characterized in that, S2 specifically includes: S201: The original multi-exposure image sequence is sorted sequentially, and the imaging geometry of each frame is ensured to be consistent according to the acquisition constraints of the fixed camera position, so as to obtain the target multi-exposure input sequence for fusion; S202: Read the exposure time set and bind the exposure time set with the corresponding target multi-exposure input sequence; S203: Based on the target multi-exposure input sequence after binding processing, the pixel observations under different exposures are fused using the Debevec merging algorithm to generate the high dynamic range irradiance image.

3. The method for detecting surface defects in sheet metal according to claim 1, characterized in that, S3 specifically includes: S301: Extract the luminance component and color component from the high dynamic range irradiance image to form a tone mapping input; S302: Based on the tone mapping input, construct the tone mapping basic quantities through the tone mapping algorithm; S303: Calculate the adaptive reflection intensity weight based on the luminance component in the hue mapping base quantity; S304: Calculate the stable luminance component using the Ward histogram method; S305: Calculates detail luminance components using the Reinhard tone mapping operator; S306: Based on the adaptive reflection intensity weight, the stable luminance component and the detail luminance component are adaptively fused to obtain a fused luminance value; S307: Perform local contrast enhancement on the fused brightness value to obtain an adaptive enhanced brightness value; S308: Perform Gamma correction and clamp truncation on the adaptive enhanced brightness value to generate the enhanced dynamic range characterization image.

4. The method for detecting surface defects in sheet metal according to claim 1, characterized in that, The neural network-based plate surface defect detection model uses an improved YOLOv5s as its framework, which includes a backbone network, a neck network, and a detection head. In the backbone network, the C3 module in the conventional backbone network is replaced by the fusion channel attention feature enhancement module; In the backbone network, the global feature adaptive focusing module is added to jointly model the channel dimension and spatial dimension of the intermediate feature map of the backbone network and adaptively allocate the weight of each dimension feature to the output result. In the neck network, the C3 module in the conventional neck network is replaced with the fusion channel attention feature enhancement module.

5. The method for detecting surface defects in sheet metal according to claim 4, characterized in that, The specific architecture of the neural network-based plate surface defect detection model is as follows: The input end of the backbone network is connected to the first CBS module, and the input end of the backbone network is used to input the enhanced dynamic range characterization image; The first CBS module is sequentially connected to the second CBS module, the first fusion channel attention feature enhancement module, the third CBS module, the second fusion channel attention feature enhancement module, the fourth CBS module, the third fusion channel attention feature enhancement module, the fifth CBS module, the fourth fusion channel attention feature enhancement module, and the SPPF module; the output of the SPPF module is connected to the global feature adaptive focusing module, in which the channel dimension and spatial dimension of the intermediate feature map output by the backbone network are jointly modeled, and the weights of each dimension feature to the output result are adaptively allocated; The second fusion channel attention feature enhancement module, the third fusion channel attention feature enhancement module, and the global feature adaptive focusing module are connected. The input end of the neck network is used to input the first scale feature map output by the second fusion channel attention feature enhancement module, the second scale feature map output by the third fusion channel attention feature enhancement module, and the third scale feature map output by the global feature adaptive focusing module. The global feature adaptive focusing module is connected to the sixth CBS module and the first upsampling module in sequence. The output feature map of the first upsampling module is concatenated with the second scale feature map, and the concatenated feature is input to the fifth fusion channel attention feature enhancement module. The fifth fusion channel attention feature enhancement module is connected in sequence to the seventh CBS module and the second upsampling module. The output feature map of the second upsampling module is spliced ​​with the first scale feature map, and the spliced ​​feature is input to the sixth fusion channel attention feature enhancement module. The sixth fusion channel attention feature enhancement module is connected to the eighth CBS module. The output feature map of the eighth CBS module is spliced ​​with the output feature map of the seventh CBS module, and the spliced ​​feature is input to the seventh fusion channel attention feature enhancement module. The seventh fusion channel attention feature enhancement module is connected to the ninth CBS module. The output feature map of the ninth CBS module is spliced ​​with the output feature map of the sixth CBS module, and the spliced ​​feature is input to the eighth fusion channel attention feature enhancement module. The output feature maps of the sixth, seventh, and eighth fusion channel attention feature enhancement modules are respectively used as multi-scale fusion feature outputs of the neck network and are respectively input to the input end of the corresponding scale of the detection head.

6. The method for detecting surface defects in sheet metal according to claim 4, characterized in that, The fusion channel attention feature enhancement module includes a first convolutional branch and a second convolutional branch; The first convolutional branch is connected to the first convolutional layer and the CSPNEXtBlock module in sequence; the second convolutional branch includes the second convolutional layer; the output feature map of the CSPNEXtBlock module and the output feature map of the second convolutional layer are concatenated, and the concatenated features are sequentially input to the channel attention module and the convolutional module; the output feature map of the convolutional module is used as the output feature map of the fusion channel attention feature enhancement module.

7. The method for detecting surface defects in sheet metal according to claim 1, characterized in that, S6 specifically includes: S601: During the inference process of the neural network-based plate surface defect detection model, obtain the spatial attention weights output by the global feature adaptive focusing module at the corresponding feature layer; S602: Based on the coordinates of each bounding box, perform scale interpolation and spatial alignment processing on each spatial attention weight to obtain aligned spatial attention weights; S603: Extract the attention weights corresponding to the bounding box coordinates from the alignment space attention weights, and calculate the local attention mean of the current bounding box based on each of the attention weights; S604: Based on the alignment space attention weights, calculate the global attention mean as the reference benchmark for the whole image attention, and combine the local attention mean and the global attention mean to calculate the feature focus score of each bounding box; S605: Based on the feature focus scores of each feature, the class confidence of the corresponding bounding box is recalibrated to obtain the calibrated class confidence. S606: Using a focus threshold based on brightness perception, each bounding box is segmented and filtered to obtain a set of target bounding boxes; S607: Sort the calibration category confidence scores corresponding to the target bounding box set in descending order, and perform non-maximum suppression based on the sorting results to obtain the refined detection results.

8. The method for detecting surface defects in sheet metal according to claim 7, characterized in that, Specifically, S606 includes: S6061: Generate a brightness map based on the enhanced dynamic range characterization image; S6062: Extract the pixel region of the luminance map and the corresponding bounding box of the luminance map, and calculate the average luminance value within the pixel region of the bounding box; S6063: Determine whether the average brightness value is greater than the preset brightness value; if so, determine the bounding box pixel area as a reflective area bounding box and proceed to S6064; otherwise, determine the bounding box pixel area as a non-reflective area bounding box and proceed to S6065. S6064: Determine whether the feature focus score of each of the reflective region bounding boxes is greater than or equal to the reflective region threshold; if so, retain the reflective region bounding box; otherwise, discard the reflective region bounding box. S6065: Determine whether the feature focus score of each of the non-reflective region bounding boxes is greater than or equal to the non-reflective region threshold; if so, retain the non-reflective region bounding box; otherwise, discard the non-reflective region bounding box. S6066: Combine the retained reflective area bounding boxes and non-reflective area bounding boxes to obtain the target bounding box set.

9. A surface defect detection system for sheet metal, characterized in that, include: Processor and memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the plate surface defect detection method as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the plate surface defect detection method as described in any one of claims 1 to 8.

Images