Method, apparatus and storage medium for defect detection of steel material

Abstract

This application discloses a method, device, and storage medium for detecting defects in steel, belonging to the field of image recognition technology. The method includes: acquiring an image of the steel to be detected; performing a step-by-step frequency decomposition of the image using wavelet transform to generate low-frequency components and high-frequency components in multiple directions; performing feature extraction on each level of low-frequency components to form low-frequency feature components; and performing feature extraction on each level of high-frequency components to form high-frequency feature components; linearly combining the high-frequency and low-frequency feature components using inverse wavelet transform to generate a feature map; and identifying steel defect information in the image based on the feature map. This application, through wavelet decomposition, enables the model to capture contextual information covering a larger area of ​​the image at a shallow level, thereby effectively understanding the global morphology of defects and their relationship to the background, while retaining the ability to capture key details of minute defects, reducing the computational complexity and number of parameters of the model.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, device and storage medium for detecting defects in steel. Background Technology

[0002] To ensure product quality, deep learning-based target detection models, such as Faster R-CNN and YOLO, are commonly used in industrial manufacturing processes to automatically learn the visual features of steel defects and output their category and location. In related technologies, steel defect detection typically relies on stacked deep convolutional neural networks to construct a feature extraction backbone network. Through layer-by-layer propagation and downsampling, the receptive field is expanded and semantic information is abstracted to achieve the identification of defects of different shapes.

[0003] However, to obtain a sufficiently large receptive field to capture global information and context in steel images, the model must be designed to be deep enough or use large-scale convolutional kernels. This leads to a surge in the number of model parameters and an exponential increase in computational complexity. Therefore, existing high-precision models are often computationally burdensome and difficult to implement in real-time inference on mobile or embedded devices with limited computing resources.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a method, device, and storage medium for detecting defects in steel, aiming to solve the technical problem that defect detection algorithms based on steel images are difficult to implement real-time inference on mobile or embedded devices with limited computing resources.

[0006] To achieve the above objectives, this application provides a method for detecting defects in steel, the method comprising the following steps: An image of the steel to be detected is acquired, and the image is decomposed into low-frequency components and high-frequency components in multiple directions by wavelet transform. Perform feature extraction on each of the low-frequency components to form low-frequency feature components; Furthermore, feature extraction operations are performed on the high-frequency components at each level to form high-frequency feature components; By using inverse wavelet transform, the high-frequency feature components and the low-frequency feature components are linearly combined to generate a feature map; Based on the feature map, steel defect information in the image of the steel to be detected is identified.

[0007] In one embodiment, the steps of acquiring an image of the steel to be detected, performing multi-level frequency decomposition on the image of the steel to be detected, and generating low-frequency components and high-frequency components in multiple directions step by step include: The image of the steel to be detected is obtained, the image of the steel to be detected is selected as the image to be processed, and the image to be processed is input into the first level of the image decomposition layer in the wavelet transform module; The low-frequency component of the target and the high-frequency component of the target in the preset direction are extracted from the image under processing by using the low-pass filter and the high-pass filter in the image decomposition layer, respectively. The target low-frequency component is selected as the image to be processed and input into the next level of the image decomposition layer in the wavelet transform module. Then, the step of executing the step of extracting the target low-frequency component and the target high-frequency component in the preset direction of the image to be processed by passing through the low-pass filter and high-pass filter in the image decomposition layer, respectively. The low-frequency component and the high-frequency component in multiple directions are determined based on the target low-frequency component and the target high-frequency component in different image decomposition layers.

[0008] In one embodiment, the step of performing feature extraction on each level of the low-frequency component to form low-frequency feature components includes: For each level of low-frequency components, multi-scale convolution kernels are used for feature extraction to obtain low-frequency features at different scales. The low-frequency features are normalized to form the low-frequency feature components.

[0009] In one embodiment, the step of performing feature extraction operations on each of the high-frequency components to form high-frequency feature components includes: Determine the directional characteristics corresponding to the high-frequency components; High-frequency features in the high-frequency components are extracted using the direction-sensitive operator corresponding to the directional characteristics. The high-frequency features are subjected to image enhancement processing to form the high-frequency feature components.

[0010] In one embodiment, the step of extracting high-frequency features from the high-frequency components using the direction-sensitive operator corresponding to the direction characteristics includes: The high-frequency components are input into the corresponding directional convolution units in the horizontal, vertical and diagonal directions respectively to obtain the directional high-frequency features in the three directions; The high-frequency features in the aforementioned directions are spliced ​​together along the channel dimension to obtain spliced ​​high-frequency features; In one embodiment, the step of generating a feature map by linearly combining the high-frequency feature components and the low-frequency feature components through inverse wavelet transform includes: High-frequency feature components and low-frequency feature components of the same level are spliced ​​together in the channel dimension to form a composite tensor; Perform an inverse wavelet transform on the composite tensor to obtain the reconstructed feature map; The reconstructed feature map is added element by element to the target reconstructed feature map of the adjacent layer to obtain the fused feature map; When the resolution of the fused feature map is the same as the resolution of the steel image to be detected, the fused feature map is used as the feature map.

[0011] In one embodiment, the step of identifying steel defect information in the image of the steel to be detected based on the feature map includes: The feature map is reorganized using a non-step spatial reconstruction method, which converts the spatial dimension information to the channel dimension to obtain the reorganized features. The recombined features are convolved point by point to obtain refined features; The refining features are input into the detection head, and the steel defect information is output.

[0012] In one embodiment, after the step of identifying steel defect information in the steel image to be detected based on the feature map, the method further includes: Obtain the predicted value of the defect category, the center point of the defect bounding box, and the predicted distribution of the bounding box position from the steel defect information; The predicted defect category value and the labeled category value are subjected to binary cross-entropy calculation to obtain the category loss; The Euclidean distance between the center point of the defect bounding box and the center point of the labeled bounding box is calculated to obtain the positioning loss. The distribution focus loss is obtained by performing distribution focus calculation on the predicted distribution of bounding box locations and the labeled distribution. The category loss, localization loss, and distribution focus loss are added together with preset weights to form a loss function; Based on the loss function, adjust the model parameters in the defect detection model.

[0013] In addition, to achieve the above objectives, this application also provides a steel defect detection device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the steel defect detection method as described above.

[0014] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the steel defect detection method as described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: This application fundamentally changes the paradigm of traditional models that rely on stacked spatial convolutions to expand the receptive field by introducing wavelet transform for stepwise frequency decomposition of images. The method acquires an image of the steel to be detected and performs stepwise frequency decomposition using wavelet transform, generating low-frequency components containing the overall image structure and high-frequency components containing edge details in multiple directions. Subsequently, features are extracted from both the low-frequency and high-frequency components. Inverse wavelet transform is then used to linearly combine the processed high-frequency and low-frequency feature components to generate a feature map. This allows the model to simultaneously obtain a broad receptive field and refined multi-scale feature representation without significantly increasing network depth or the number of parameters. The inherent multi-scale analysis capability of wavelet decomposition enables the model to capture contextual information covering a larger area of ​​the image at a shallow level, effectively understanding the global morphology of defects and their relationship to the background. Simultaneously, the processing of high-frequency components ensures the capture of key details of minute defects. Therefore, this application significantly reduces the computational complexity and number of parameters while still accurately identifying steel defect information based on the generated feature map, ultimately achieving an effective balance between detection accuracy and inference efficiency, making it suitable for resource-constrained industrial mobile and embedded devices. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the defect detection method for steel according to this application; Figure 2 This is a schematic diagram of the working process of the steel defect detection system involved in the first embodiment of the steel defect detection method of this application; Figure 3 This is a flowchart illustrating the second embodiment of the defect detection method for steel according to this application; Figure 4 This is a schematic diagram of the structure of a steel defect detection device in the hardware operating environment involved in the embodiments of this application.

[0019] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0021] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application embodiment is as follows: acquire an image of the steel to be detected, and perform a step-by-step frequency decomposition of the image of the steel to be detected through wavelet transform to generate low-frequency components and high-frequency components in multiple directions. Perform feature extraction operation on each level of low-frequency components to form low-frequency feature components, and perform feature extraction operation on each level of high-frequency components to form high-frequency feature components. Perform inverse wavelet transform to linearly combine the high-frequency feature components and low-frequency feature components to generate a feature map. Based on the feature map, identify steel defect information in the image of the steel to be detected.

[0023] To ensure product quality, current industrial manufacturing processes typically employ deep learning-based target detection models, such as Faster R-CNN and YOLO, to automatically learn the visual features of steel defects and output their category and location. In these technologies, steel defect detection usually relies on stacked deep convolutional neural networks to construct a feature extraction backbone network. Through layer-by-layer propagation and downsampling, the receptive field is expanded and semantic information is abstracted to identify defects of different shapes. However, to obtain a sufficiently large receptive field to capture global information and context in steel images, the model must be designed to be sufficiently deep or use large-scale convolutional kernels. This leads to a surge in the number of model parameters and an exponential increase in computational complexity. Therefore, existing high-precision models often have a heavy computational burden, making real-time inference difficult to achieve on mobile or embedded devices with limited computing resources.

[0024] This application fundamentally changes the paradigm of traditional models that rely on stacked spatial convolutions to expand the receptive field by introducing wavelet transform for stepwise frequency decomposition of images. The method acquires an image of the steel to be detected and performs stepwise frequency decomposition using wavelet transform, generating low-frequency components containing the overall image structure and high-frequency components containing edge details in multiple directions. Subsequently, features are extracted from both the low-frequency and high-frequency components. Inverse wavelet transform is then used to linearly combine the processed high-frequency and low-frequency feature components to generate a feature map. This allows the model to simultaneously obtain a broad receptive field and refined multi-scale feature representation without significantly increasing network depth or the number of parameters. The inherent multi-scale analysis capability of wavelet decomposition enables the model to capture contextual information covering a larger area of ​​the image at a shallow level, effectively understanding the global morphology of defects and their relationship to the background. Simultaneously, the processing of high-frequency components ensures the capture of key details of minute defects. Therefore, this application significantly reduces the computational complexity and number of parameters while still accurately identifying steel defect information based on the generated feature map, ultimately achieving an effective balance between detection accuracy and inference efficiency, making it suitable for resource-constrained industrial mobile and embedded devices.

[0025] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0026] It should be noted that the executing entity in this embodiment can be a steel defect detection system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or steel defect detection device capable of the above functions. This embodiment does not specifically limit the specific implementation. The following uses a steel defect detection system as an example to describe this embodiment and the following embodiments.

[0027] Based on this, embodiments of this application provide a method for detecting defects in steel, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the defect detection method for steel according to this application.

[0028] In this embodiment, the defect detection method for steel includes steps S10 to S50: Step S10: Acquire an image of the steel to be detected, and perform a step-by-step frequency decomposition on the image of the steel to be detected through wavelet transform to generate low-frequency components and high-frequency components in multiple directions; It should be noted that wavelet transform (WT) is a transform analysis method that performs time-frequency analysis and processing of signals by providing a "time-frequency" window that changes with frequency. Its main characteristics are that it fully highlights certain features of the problem through transformation, enabling localized analysis of time or spatial frequency, and progressively refining signals and functions at multiple scales through scaling and translation operations. Ultimately, it achieves time subdivision at high frequencies and frequency subdivision at low frequencies, automatically adapting to the requirements of time-frequency signal analysis, thus allowing focus on any detail of the signal.

[0029] In this embodiment, the steel defect detection system decomposes the signal into different frequency sub-bands and scales by scaling and translating the mother wavelet function, thereby acquiring the signal's frequency information and temporal or spatial location information, and capturing local features at different scales and directions in the image. After acquiring and preprocessing the steel surface image, wavelet transform is used to map the image from a single spatial domain to a more analytical time-frequency domain, deconstructing the input image into a series of sub-band signals representing different frequency components and directional features, namely low-frequency components and high-frequency components. The low-frequency components correspond to slowly changing signals in the image, such as large color areas, background, and overall brightness. The high-frequency components correspond to rapidly changing signals in the image, such as edges, textures, noise, and details.

[0030] In one embodiment, the steel defect detection system acquires an image of the steel to be detected, selects this image as the image to be processed, and inputs it into the first-level image decomposition layer of the wavelet transform module. The image decomposition layer uses low-pass and high-pass filters to extract the target low-frequency components and the target high-frequency components in a preset direction. The target low-frequency components are then selected as the next-level image decomposition layer in the wavelet transform module to achieve step-by-step frequency decomposition of the steel image based on wavelet transform. After inputting into the next-level image decomposition layer, the steel defect detection system further extracts the target low-frequency components and the target high-frequency components in a preset direction based on the low-pass and high-pass filters in that level. After completing the wavelet transform at a preset number of levels, the system determines the low-frequency components and high-frequency components in multiple directions based on the target low-frequency and target high-frequency components at different levels of image decomposition.

[0031] Specifically, the steel defect detection system uses a high-resolution industrial linear or area array camera to acquire raw digital images of the steel surface under stable lighting conditions. These images typically contain widths and heights ranging from hundreds to thousands of pixels. The system performs necessary preprocessing on the raw image, such as grayscale conversion or color space transformation, and normalizes pixel values ​​to a fixed range, such as between zero and one, to improve the numerical stability of the model. The preprocessed image is then fed into a specially constructed wavelet decomposition module. This module operates iteratively, taking the input image of the current level (initially the raw image) and passing it through a set of pre-selected low-pass and high-pass filters. The low-pass filter removes high-frequency details from the image while preserving its general outline and overall brightness variations, thus outputting a low-frequency approximation component. A set of three directional high-pass filters captures abrupt changes in the image along the horizontal, vertical, and diagonal directions, respectively. These changes typically correspond to details such as edges, scratches, or textures, thus outputting high-frequency detail components in the corresponding directions. Each filtering operation is followed by a downsampling step, typically reducing the image's width and height by half. This conforms to wavelet transform principles and significantly reduces the amount of data required for subsequent computations. The low-frequency components generated in this decomposition are then used as input for the next decomposition layer, repeating the filtering and downsampling process. This cycle continues until a preset number of decomposition layers is reached, such as two or three, forming a frequency pyramid composed of coarse low-frequency components from the bottom layer and multi-directional high-frequency components from each layer.

[0032] Optionally, the steel defect detection system incorporates the WTConv module, a wavelet transform module. This module utilizes wavelet transform to achieve collaborative modeling of spatial and frequency features. The WTConv module progressively refines the frequency information of the image through wavelet transform, thereby enhancing the model's ability to capture global features. Specifically, the WTConv module uses filters to extract the low-frequency (LL) component and the high-frequency components in the horizontal (LH), vertical (HL), and diagonal (HH) directions of the image. The low-frequency components contain the overall structure and average information of the image, while the high-frequency components contain edge information and details in each direction. Through multi-level wavelet decomposition, the WTConv module progressively amplifies the low-frequency information and performs convolution processing. In each level of wavelet decomposition, downsampling is applied to progressively reduce the image resolution. This operation not only helps provide a multi-scale representation of the image but also ensures the maintenance of computational efficiency.

[0033] Step S20: Perform feature extraction operation on the low-frequency components of each level to form low-frequency feature components; Step S30: Perform feature extraction operation on each of the high-frequency components to form high-frequency feature components; In this embodiment, based on image feature extraction, the steel defect detection system uses convolution operations to extract defect-related information from frequency components. Low-frequency feature components primarily carry the macroscopic structure and contour information of the image, while high-frequency feature components contain detailed information such as the edges and textures of the defects.

[0034] Specifically, for each level of decomposition, the system uses a convolutional module for processing the low-frequency components. This module may contain multiple convolutional kernels of different scales, such as 3x3 and 5x5 kernels working in parallel, to capture structural features under different receptive fields. A normalization layer, such as batch normalization, is typically applied after convolution to stabilize the training process and accelerate convergence, outputting enhanced low-frequency feature components. For high-frequency components, the processing is more targeted. The steel defect detection system identifies the directional characteristics of each high-frequency component: horizontal high-frequency (LH) is mainly activated by horizontal edges, vertical high-frequency (HL) by vertical edges, and diagonal high-frequency (HH) by diagonal edges. The system uses direction-sensitive convolutional operators that match the directional characteristics to convolve these components separately, enhancing edge and texture features in specific directions. These processed directional features are then concatenated to form a comprehensive high-frequency feature representation.

[0035] In one embodiment, the steel defect detection system uses multi-scale convolution kernels to extract features from the low-frequency components at each level, obtains low-frequency features at different scales, and normalizes the low-frequency features to form low-frequency feature components.

[0036] Optionally, for the low-frequency components generated at each level of decomposition, which contain the main structure and contours of the image but may be relatively smooth and lack texture, the steel defect detection system employs a multi-scale feature extraction unit. This unit uses multiple convolutional kernels of different sizes in parallel, such as smaller 3x3 kernels and larger 5x5 kernels, to perform convolution operations on the low-frequency components simultaneously. Smaller kernels focus on capturing small local structural changes, while larger kernels can integrate contextual information from a larger area, thereby understanding the morphology of the defect at different scales. The feature maps generated by the convolution operation are immediately passed through an activation function, such as the ReLU function, to introduce non-linearity. Subsequently, a normalization layer, such as a batch normalization layer, is used to standardize these features, adjust their data distribution, and output enhanced, semantically richer low-frequency feature components.

[0037] In another embodiment, the steel defect detection system determines the directional characteristics corresponding to the high-frequency components, extracts the high-frequency features from the high-frequency components using the directional sensitive operator corresponding to the directional characteristics, and performs image enhancement processing on the high-frequency features to form high-frequency feature components.

[0038] Optionally, the steel defect detection system inputs the high-frequency components into the corresponding directional convolution units in three directions: horizontal, vertical, and diagonal, to obtain directional high-frequency features in the three directions. The directional high-frequency features are then concatenated along the channel dimension to obtain concatenated high-frequency features. Non-step convolution is then performed on the concatenated high-frequency features to obtain high-frequency feature components.

[0039] Optionally, the steel defect detection system identifies the dominant directional characteristics of each high-frequency component. For example, the horizontal high-frequency component mainly responds to edges in the horizontal direction. It then invokes pre-designed direction-sensitive operators for different directions. These direction-sensitive operators are essentially convolutional kernels with specific weight patterns, capable of generating stronger activations for edges and textures in specific directions. The high-frequency components in the three directions are processed by their corresponding directional convolutional kernels, generating three directional feature maps. These directional feature maps are concatenated along the channel dimension to form a comprehensive feature tensor rich in multi-directional details. To fuse this information from different directions and control the number of channels, a subsequent non-strut convolutional layer (i.e., a convolutional layer with a stride of one) is applied to this concatenated tensor for feature fusion and compression, ultimately outputting refined high-frequency feature components.

[0040] It's important to note that the WTConv module amplifies and convolves low-frequency information step-by-step through multi-level wavelet decomposition. Low-frequency information is typically related to the shape and structure of an image, and this information is more easily preserved under noise interference. Therefore, the WTConv module enhances the model's robustness to noise. In each level of wavelet decomposition, downsampling is applied to progressively reduce the image resolution. This operation not only helps provide a multi-scale representation of the image but also ensures that computational efficiency is maintained. By recursively decomposing the low-frequency components of the image, the WTConv module can further highlight the main structure of the image and reveal more layers of detail, enabling the model to more accurately observe the features of the image at different frequency levels.

[0041] Step S40: By performing inverse wavelet transform, the high-frequency feature components and the low-frequency feature components are linearly combined to generate a feature map; It should be noted that the inverse wavelet transform is the inverse process of the wavelet transform. It can reassemble the processed and decomposed frequency sub-band components and reconstruct the signal back to the original domain, i.e. the image spatial domain, for use in synthesizing feature maps that fuse multi-scale frequency information.

[0042] In this embodiment, the steel defect detection system re-integrates the frequency features previously processed at different scales and directions into a unified, information-complete spatial domain feature map. Utilizing the reversibility of inverse wavelet transform, it feeds the analysis results from the frequency domain back into the spatial domain, generating a fused feature representation that contains both deep semantics and retains shallow details, thus forming the feature map.

[0043] Specifically, the steel defect detection system splices high-frequency feature components and low-frequency feature components of the same level in the channel dimension to form a composite tensor. An inverse wavelet transform is performed on the composite tensor to obtain a reconstructed feature map. The reconstructed feature map is added element by element to the target reconstructed feature map of the adjacent level to obtain a fused feature map. When the resolution of the fused feature map is the same as the resolution of the steel image to be detected, the fused feature map is used as the feature map.

[0044] Furthermore, the steel defect detection system employs a bottom-up reconstruction process, starting from the deepest layer, i.e., the components after the last level of decomposition. The system first concatenates the enhanced low-frequency and high-frequency feature components at the same level along the channel dimension, combining them into a composite tensor. This tensor contains all structural and detailed information at that scale. By performing an inverse wavelet transform algorithm on this composite tensor, each channel of the input tensor is upsampled, typically doubling its size. A set of reconstruction filters corresponding to those used in wavelet decomposition is then used for convolution, cleverly fusing the frequency sub-band signals and reconstructing them back into the spatial domain, generating a fused feature map. This fused feature map is then element-wise added to the feature map reconstructed from the previous, shallower layer through the same process, achieving mutual supplementation and enhancement of detailed information. This process is recursively performed, starting from the deepest layer and gradually fusing upwards until a final feature map matching the spatial dimensions of the original input image is reconstructed. This final feature map deeply integrates all key information from the global contour to the local edges.

[0045] For example, the linear properties of WT and inverse wavelet transform (IWT) are finally utilized to linearly combine the outputs of different frequency components, thereby generating the final feature map:

[0046] Among them, f LL For a low-pass filter, f LH f HL f HH It is a set of high-pass filters. i represents the wavelet transform level, and W(i) is a weight tensor specifically designed for the i-th level, used to perform convolution operations in the wavelet domain of the i-th level.

[0047] The steel defect detection system introduces the SPD-Conv module, achieving efficient feature transformation. Its spatial reconstruction layer cleverly concatenates sub-feature maps along the channel dimension, significantly increasing the channel dimension while reducing spatial resolution, achieving downsampling without information loss. Specifically, the feature map size is transformed from H×W×C to H / scale×W / scale×scale²×C, where scale is the downsampling factor. Channel compression and semantic enhancement are performed through non-strut convolution, forming an optimized link between spatial reconstruction and feature refinement. The loss for the detection task is jointly guided by class loss and localization loss during training to ensure prediction accuracy.

[0048] Step S50: Based on the feature map, identify steel defect information in the steel image to be detected.

[0049] In this embodiment, the steel defect detection system decodes the feature map that integrates multi-scale frequency information and transforms it into specific defect detection results. The system also includes a detection head module, which optimizes the feature map by introducing a lossless downsampling technique, thereby accurately outputting the type and location of the defect.

[0050] Specifically, the system performs non-step spatial reconstruction on the feature map, transforming the spatial dimension information into the channel dimension to obtain reconstructed features. The reconstructed features are then convolved point by point to obtain refined features. The refined features are then input into the detection head to output steel defect information.

[0051] It should be noted that the steel defect detection system also includes a multi-layer feature refinement module. This module, located after feature map generation and before the detection head, further optimizes and enhances the feature map, which incorporates multi-scale frequency information, to improve its feature discrimination capability. Its core principle is to achieve efficient feature compression and semantic extraction while preserving all spatial information through the collaborative operation of space-to-depth and non-strided convolution. The space-to-depth layer divides the feature map into several sub-blocks in the spatial dimension and rearranges the pixel values ​​within each sub-block along the channel dimension, thereby reducing spatial resolution and increasing the number of channels without losing any details, achieving lossless downsampling. The non-strided convolution layer fuses and compresses the recombined high-dimensional features between channels, filtering out redundant responses and strengthening the semantic information related to defects. This multi-layer processing ensures that the feature map, while maintaining the integrity of its details, has a more focused and discriminative semantic expression.

[0052] Specifically, after generating a feature map that integrates multi-scale frequency information, this feature map is further processed by a multi-layer feature refinement module to obtain a more discriminative feature map. Specifically, firstly, the feature map undergoes non-strided spatial reconstruction (space-to-depth), recombining local neighborhood information from the spatial dimension to the channel dimension. This reduces spatial resolution while increasing the number of channels without loss, achieving efficient downsampling. Subsequently, non-strided convolution layers are used to perform channel compression and semantic fusion on the recombined high-dimensional features, filtering out redundant information and enhancing key features related to defects. The feature map obtained after this multi-step refinement process has more concise semantic information and stronger discriminative power. Finally, it is input into the detection head module to accurately output the category and location information of steel defects.

[0053] Optionally, the steel defect detection system employs a spatial reconstruction layer to optimize and reconstruct the spatial dimension of the high-resolution feature map. This layer spatially divides the feature map into several non-overlapping local blocks, such as 2x2 grids. The four pixel values ​​within each grid are rearranged to the channel dimension, making previously spatially adjacent information now channel-adjacent, quadrupling the number of channels and achieving efficient downsampling without losing any pixel information. Through non-stretch convolutional layers, the feature map undergoes channel compression and semantic refinement. Learnable parameter weights effectively fuse and reduce the dimensionality of the surged features across the channel dimension, further extracting features strongly relevant to the detection task. The refined features are then fed into two parallel branches of the detection head. The first branch is a classification branch, which maps each anchor point of the feature map to a series of scores through convolutional layers, and then passes them through activation functions such as Softmax to output the probability that each potential target belongs to each defect category, such as scratches, inclusions, or holes. The second branch is a regression branch, which, also through convolutional layers, predicts the offset of the bounding box for each anchor point, including the coordinates of the center point as well as the width and height of the box, thereby accurately framing the location of the defect.

[0054] For example, such as Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the workflow of the steel defect detection system according to an embodiment of this application. (Refer to...) Figure 2The steel defect detection system comprises a backbone network, a neck network, and a head. In the backbone, the convolutional layers (Conv) are used to initially extract basic image features. The WTConv module decomposes and enhances multi-scale details of the features through wavelet transform (WT), followed by inverse wavelet transform (IWT) for reassembly. The C2f module fuses residuals and dense connections to strengthen feature discriminative power. The SPPF module aggregates defect information of different sizes through multi-scale pooling. Fusion1 and Fusion2 enable cross-layer feature interaction. Specifically, the ORF Block in the neck enhances details through upsampling (Upsample), spatial detail enhancement convolution (SPDConv), and feature concatenation (Concat). Upsampling (UP) improves feature resolution, and the feature concatenation (Concat) and C2f modules achieve cross-scale feature fusion. The Detect module in the head is responsible for defect category and location prediction. The process involves the following steps: the image of the steel to be inspected is first processed by Backbone's Conv, WTConv, C2f, SPPF and feature fusion operations to extract multi-scale features; then, the Neck's ORF Block, UP, Concat and C2f modules refine the features and fuse them across scales; and finally, the Head's Detect module outputs the detection results, such as defect category and bounding box.

[0055] This application's embodiments fundamentally change the paradigm of traditional models that rely on stacked spatial convolution to expand the receptive field by introducing wavelet transform for stepwise frequency decomposition of images. The method acquires an image of the steel to be detected and performs stepwise frequency decomposition using wavelet transform, generating low-frequency components containing the overall image structure and high-frequency components containing edge details in multiple directions. Subsequently, feature extraction is performed on both the low-frequency and high-frequency components. Inverse wavelet transform is then used to linearly combine the processed high-frequency and low-frequency feature components to generate a feature map. This allows the model to simultaneously obtain a broad receptive field and refined multi-scale feature representation without significantly increasing network depth or the number of parameters. The inherent multi-scale analysis capability of wavelet decomposition enables the model to capture contextual information covering a larger area of ​​the image at a shallow level, effectively understanding the global morphology of defects and their relationship to the background. Simultaneously, the processing of high-frequency components ensures the capture of key details of minute defects. Therefore, this application significantly reduces the computational complexity and number of parameters while still accurately identifying steel defect information based on the generated feature map, ultimately achieving an effective balance between detection accuracy and inference efficiency, making it suitable for resource-constrained industrial mobile and embedded devices.

[0056] Based on the same inventive concept, this application also provides a second embodiment, referring to... Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the defect detection method for steel in this application.

[0057] In this embodiment, the defect detection method for steel further includes steps S51 to S56: Step S51: Obtain the predicted value of the defect category, the center point of the defect bounding box, and the predicted distribution of the bounding box position from the steel defect information; Step S52: Perform binary cross-entropy calculation on the defect category prediction value and the category label value to obtain the category loss; Step S53: Calculate the Euclidean distance between the center point of the defect bounding box and the center point of the labeled bounding box to obtain the positioning loss; Step S54: Perform distribution focus calculation on the bounding box location prediction distribution and the labeled distribution to obtain the distribution focus loss; Step S55: Add the category loss, localization loss, and distribution focus loss according to preset weights to form a loss function; Step S56: Adjust the model parameters in the defect detection model based on the loss function.

[0058] In this embodiment, the steel defect detection system also defines a multi-task loss function for training and optimizing the defect detection model. By acquiring the defect category predictions, bounding box center points, and location prediction distributions output by the model, the category predictions are compared with the ground truth labels using binary cross-entropy to obtain the category loss. The predicted bounding boxes are compared with the ground truth boxes using comprehensive geometric metrics to obtain the localization loss. The location prediction distributions are compared with the ground truth distributions using distribution focus operations to obtain the distribution focus loss. These losses are then added together with preset weights to form a comprehensive loss function. Based on this function, all parameters in the model are adjusted using a backpropagation algorithm.

[0059] Specifically, the category loss uses binary cross-entropy to severely penalize the model's incorrect classification confidence, directly driving the model to improve defect recognition accuracy. The localization loss employs an improved Complete Intersection over Union (CIoU) loss, which not only calculates the overlap between the predicted and ground truth boxes but also comprehensively considers center point distance and aspect ratio consistency, thus simultaneously refining the position, size, and shape of the bounding boxes. The distribution focus loss further treats bounding box localization as a classification problem, encouraging the model to focus predicted probabilities on discrete anchor points near the ground truth values, utilizing richer gradient information to achieve stable and accurate localization, especially for small defects. Finally, the system balances these three losses with different dimensions using pre-defined weights, forming a unified objective function. Optimizers such as stochastic gradient descent iteratively update the network weights based on the gradients calculated from this function, thereby systematically improving the model's overall performance in classification and localization tasks.

[0060] For example, the system minimizes the difference between the model's predictions and the ground truth labels using a loss function. This loss function primarily consists of class loss and localization loss.

[0061] Among them, L DFL For the distribution focus loss, L CIoU To locate the loss, L cls ρ represents the class loss, and Loss represents the total loss. 2 (b, b) gt ) is the Euclidean distance between the center points of the predicted bounding box and the center points of the ground truth bounding box. c is the diagonal length of the smallest closure bounding box containing both the predicted and ground truth bounding boxes. v is the aspect ratio consistency loss, calculated as follows:

[0062] Where α is the weighting parameter used to balance the contribution of v, and w and h are the width and height of the prediction box, respectively. gt and h gt These are the width and height of the true bounding box. The function `arctan` is used to calculate the angle corresponding to the aspect ratio of the bounding box.

[0063] Images from the NEU-DET and GC10-DET large-scale steel defect detection datasets were used to fine-tune the detection model, adjusting the model parameters to suit the steel defect detection task.

[0064] Since the system described in Embodiment 2 of this application is a system used to implement the method of Embodiment 1 of this application, those skilled in the art can understand the specific structure and variations of the system based on the method described in Embodiment 1 of this application, and therefore will not be described again here. All systems used in the method of Embodiment 1 of this application fall within the scope of protection of this application.

[0065] This application provides a steel defect detection device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steel defect detection method of the above embodiment 1.

[0066] The following is for reference. Figure 4The diagram illustrates a structural schematic of a steel defect detection device suitable for implementing embodiments of this application. The steel defect detection device in this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The steel defect detection equipment shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0067] like Figure 4 As shown, the steel defect detection device may include a processing unit 1001 (e.g., a core processor, graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the steel defect detection device. The processing unit 1001, the ROM 1002, and the RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the steel defect detection equipment to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a steel defect detection equipment with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0068] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0069] The steel defect detection device provided in this application, employing the steel defect detection method described in the above embodiments, can solve the technical problem that defect detection algorithms based on steel images are difficult to implement in real-time inference on mobile or embedded devices with limited computing resources. Compared with the prior art, the beneficial effects of the steel defect detection device provided in this application are the same as those of the steel defect detection method provided in the above embodiments, and other technical features of this steel defect detection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0070] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0071] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0072] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the steel defect detection method in the above embodiments.

[0073] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0074] The aforementioned computer-readable storage medium may be included in the steel defect detection equipment; or it may exist independently and not assembled into the steel defect detection equipment.

[0075] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a steel defect detection device, the steel defect detection device: acquires an image of the steel to be detected; performs a step-by-step frequency decomposition of the image using wavelet transform to generate low-frequency components and high-frequency components in multiple directions; performs feature extraction on each low-frequency component to form low-frequency feature components; performs feature extraction on each high-frequency component to form high-frequency feature components; performs inverse wavelet transform to linearly combine the high-frequency and low-frequency feature components to generate a feature map; and identifies steel defect information in the image of the steel to be detected based on the feature map.

[0076] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0077] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0078] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0079] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described steel defect detection method. This solves the technical problem that defect detection algorithms based on steel images are difficult to implement in real-time inference on mobile or embedded devices with limited computing resources. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the steel defect detection method provided in the above embodiments, and will not be repeated here.

[0080] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for detecting defects in steel, characterized in that, The method includes the following steps: An image of the steel to be detected is acquired, and the image is decomposed into low-frequency components and high-frequency components in multiple directions by wavelet transform. Perform feature extraction on each of the low-frequency components to form low-frequency feature components; Furthermore, feature extraction operations are performed on the high-frequency components at each level to form high-frequency feature components; By using inverse wavelet transform, the high-frequency feature components and the low-frequency feature components are linearly combined to generate a feature map; Based on the feature map, steel defect information in the image of the steel to be detected is identified.

2. The defect detection method for steel as described in claim 1, characterized in that, The steps of acquiring an image of the steel to be detected, performing multi-level frequency decomposition on the image, and generating low-frequency components and high-frequency components in multiple directions step by step include: The image of the steel to be detected is obtained, the image of the steel to be detected is selected as the image to be processed, and the image to be processed is input into the first level of the image decomposition layer in the wavelet transform module; The low-frequency component of the target and the high-frequency component of the target in the preset direction are extracted from the image under processing by using the low-pass filter and the high-pass filter in the image decomposition layer, respectively. The target low-frequency component is selected as the image to be processed and input into the next level of the image decomposition layer in the wavelet transform module. Then, the step of executing the step of extracting the target low-frequency component and the target high-frequency component in the preset direction of the image to be processed by passing through the low-pass filter and high-pass filter in the image decomposition layer, respectively. The low-frequency component and the high-frequency component in multiple directions are determined based on the target low-frequency component and the target high-frequency component in different image decomposition layers.

3. The defect detection method for steel as described in claim 1, characterized in that, The step of performing feature extraction on each level of the low-frequency component to form low-frequency feature components includes: For each level of low-frequency components, multi-scale convolution kernels are used for feature extraction to obtain low-frequency features at different scales. The low-frequency features are normalized to form the low-frequency feature components.

4. The defect detection method for steel as described in claim 1, characterized in that, The step of performing feature extraction operations on each of the high-frequency components to form high-frequency feature components includes: Determine the directional characteristics corresponding to the high-frequency components; High-frequency features in the high-frequency components are extracted using the direction-sensitive operator corresponding to the directional characteristics. The high-frequency features are subjected to image enhancement processing to form the high-frequency feature components.

5. The defect detection method for steel as described in claim 4, characterized in that, The step of extracting high-frequency features from the high-frequency components using the direction-sensitive operator corresponding to the direction characteristics includes: The high-frequency components are input into the corresponding directional convolution units in the horizontal, vertical and diagonal directions respectively to obtain the directional high-frequency features in the three directions; The high-frequency features in the aforementioned directions are spliced ​​together along the channel dimension to obtain spliced ​​high-frequency features; The spliced ​​high-frequency features are subjected to non-stride convolution to obtain the high-frequency feature components.

6. The defect detection method for steel as described in claim 1, characterized in that, The step of generating a feature map by linearly combining the high-frequency feature components and the low-frequency feature components through inverse wavelet transform includes: High-frequency feature components and low-frequency feature components of the same level are spliced ​​together in the channel dimension to form a composite tensor; Perform an inverse wavelet transform on the composite tensor to obtain the reconstructed feature map; The reconstructed feature map is added element by element to the target reconstructed feature map of the adjacent layer to obtain the fused feature map; When the resolution of the fused feature map is the same as the resolution of the steel image to be detected, the fused feature map is used as the feature map.

7. The defect detection method for steel as described in claim 1, characterized in that, The step of identifying steel defect information in the steel image to be detected based on the feature map includes: The feature map is reorganized using a multi-layer feature refinement module, which performs non-step spatial reconstruction to convert spatial dimension information to channel dimension, thereby obtaining reorganized features. Furthermore, the recombined features are convolved point-by-point to obtain refined features; The refining features are input into the detection head, and the steel defect information is output.

8. The defect detection method for steel as described in claim 1, characterized in that, After the step of identifying steel defect information in the image of the steel to be detected based on the feature map, the method further includes: Obtain the predicted value of the defect category, the center point of the defect bounding box, and the predicted distribution of the bounding box position from the steel defect information; The predicted defect category value and the labeled category value are subjected to binary cross-entropy calculation to obtain the category loss; The Euclidean distance between the center point of the defect bounding box and the center point of the labeled bounding box is calculated to obtain the positioning loss. The distribution focus loss is obtained by performing distribution focus calculation on the predicted distribution of bounding box locations and the labeled distribution. The category loss, localization loss, and distribution focus loss are added together with preset weights to form a loss function; Based on the loss function, adjust the model parameters in the defect detection model.

9. A defect detection device for steel, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the defect detection method for steel as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the defect detection method for steel as described in any one of claims 1 to 8.

Images