A steel flaw detection method, device, medium and equipment

Abstract

The application discloses a kind of steel flaw detection method, device, medium and equipment, it is related to the technical field of image recognition.The method comprises: obtaining the image training dataset of steel and the image of steel to be detected;The image in image training dataset is used as input, and the denoised steel image is used as output, the generative adversarial network is trained, and the denoising model is obtained;The image of steel to be detected is input into the denoising model, and the denoised image is obtained;Denoised steel image is input into pre-trained swin transformer model and pre-trained convolutional neural network model for feature extraction, to obtain the first feature and the second feature of denoised image, the first feature and the second feature are fused, to obtain the feature map about steel to be detected;The feature map is processed using region proposal network and region of interest network, to obtain the flaw type and flaw position of steel to be detected.Through the above scheme, the accuracy of defect detection on steel can be improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, medium, and equipment for detecting defects in steel. Background Technology

[0002] Steel, as a core material for construction and infrastructure, is not only the cornerstone of modern industry but also a crucial factor driving social development and economic growth. However, the surface of steel is often affected by various factors. For example, steel surfaces frequently possess complex textures or reflections, especially metallic surfaces which can exhibit specular reflections, making it difficult to distinguish defects from the background. Furthermore, during image acquisition, factors such as lighting and sensor input can introduce irrelevant noise into the images, impacting the model's defect recognition capabilities.

[0003] Traditional steel defect detection relies primarily on manual inspection, a labor-intensive process. With the rapid development of deep learning and improved computing power, deep convolutional neural networks (CNNs) have achieved remarkable results in multiple fields and are currently the mainstream approach for surface defect detection in industrial products. Existing methods for steel surface defect detection based on CNNs require finding prior information about the image and then using optimization algorithms to iteratively solve the model. This complex optimization process is time-consuming and computationally expensive. Furthermore, it can easily lead to image blurring and loss of detail. In addition, existing denoising methods mainly utilize synthetically generated noisy images, resulting in relatively simple noise types. However, images captured during steel inspection may contain multiple types of noise superimposed, with complex distributions and unknown intensities.

[0004] With the application of deep learning in image denoising, the denoising effect has been improved. However, when processing real noisy images, due to noise interference, the variety of steel surface defects, different shapes, large scale differences, and low contrast between defects and background, it is difficult to distinguish complex noise from natural textures. This leads to problems such as loss of image edge information and overly smooth details, resulting in low accuracy of steel defect detection. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, medium, and equipment for detecting steel defects, addressing the technical problem that existing technologies do not have high accuracy in detecting steel defects.

[0006] The present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for detecting defects in steel, the method comprising:

[0008] Obtain a training dataset of images related to steel, and obtain images of the steel to be detected;

[0009] The images in the image training dataset are used as input, and the denoised steel images are used as output to train the generative adversarial network to obtain the denoised network model.

[0010] The image of the steel to be detected is input into the denoising network model to obtain a denoised image of the steel to be detected. The denoised image of the steel to be detected is then input into a pre-trained swin transformer model and a pre-trained convolutional neural network model for feature extraction to obtain the first feature and the second feature of the denoised image of the steel to be detected. The first feature and the second feature are then fused at multiple scales to obtain a feature map of the steel to be detected.

[0011] The feature map is processed using a region proposal network and a region of interest network to obtain the defect type and defect location of the steel to be detected.

[0012] Furthermore, the multi-scale feature fusion of the first feature and the second feature is achieved through an adaptive multi-scale feature fusion module. The expression for adaptive multi-scale feature fusion is as follows:

[0013] ;

[0014] Among them, P i For the features after fusion, C i Let C be the feature map of the i-th layer. i+1 Let be the feature map of the (i+1)th layer.

[0015] Furthermore, the adaptive multi-scale feature fusion module includes a cross-attention branch and a selectable convolutional kernel branch;

[0016] The cross-attention branch is used to determine the association between adjacent convolutional layers and to fuse the feature information of different convolutional layers through a cross-layer attention mechanism.

[0017] The selectable convolution kernel branch is used to adaptively select different convolution kernels in each layer of feature maps and adjust the receptive field of the convolution target in each layer of feature maps.

[0018] Furthermore, the process of using a region proposal network and a region of interest network to process the feature map to obtain the defect type and defect location of the steel to be detected specifically includes:

[0019] The feature map is input into the region proposal network to obtain the processing result of the feature map;

[0020] The processing result of the feature map is input into the region of interest network for further processing to obtain the defect type and defect location of the steel to be detected.

[0021] Furthermore, using the images in the image training dataset as input and the denoised steel image as output, a generative adversarial network is trained to obtain a denoising network model, specifically including:

[0022] The Nth image in the image training dataset is input into the generator of the generative adversarial network to obtain the first denoised image of the Nth image, where N is a positive integer;

[0023] The Nth image and its first denoised image are input into the discriminator of the generative adversarial network to obtain the similarity probability between the Nth image and its first denoised image.

[0024] The similarity probability is input into the loss function of the generative adversarial network to obtain the loss value, and the loss value is input into the generator for adaptive moment estimation iterative training to obtain the denoising network model.

[0025] Furthermore, the loss function is obtained by weighted summation of the adversarial loss, perceptual loss, and the loss function combined with the image texture feature matching of the discriminator, and its calculation expression is as follows:

[0026] ;

[0027] Where Loss is the output of the loss function. To combat the losses, In order to perceive loss, The loss function is used to match the image texture features of the discriminator. yes The weight, yes The weight, yes weight

[0028] The The calculation expression is:

[0029] ;

[0030] in, Let G represent the i-th convolutional layer of the discriminator, and let G represent the generator. This represents the data distribution of the original noise samples. Let z represent the data distribution of samples without noise, where z represents noise and T represents the number of layers in the discriminator. This represents the number of elements in each layer, and k represents the criterion. Represents the k-th discriminant metric .

[0031] Furthermore, the pre-trained convolutional neural network model is obtained by training based on EfficientNet.

[0032] Secondly, the present invention provides a steel defect detection device, comprising:

[0033] The acquisition module is used to acquire a training dataset of images related to steel and to acquire images of the steel to be detected.

[0034] The training module is used to train the generative adversarial network by taking the images in the image training dataset as input and the denoised steel image as output, so as to obtain the denoised network model.

[0035] The feature extraction module is used to input the image of the steel to be detected into the denoising network model to obtain a denoised image of the steel to be detected; input the denoised image of the steel to be detected into a pre-trained swintransformer model and a pre-trained convolutional neural network model respectively for feature extraction to obtain a first feature and a second feature of the denoised image of the steel to be detected; and perform multi-scale feature fusion on the first feature and the second feature to obtain a feature map of the steel to be detected.

[0036] The detection module is used to process the feature map using a region proposal network and a region of interest network to obtain the defect type and defect location of the steel to be detected.

[0037] The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steel defect detection method.

[0038] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steel defect detection method.

[0039] The at least one technical solution adopted in this invention can achieve the following beneficial effects: This invention uses a training dataset of steel-related images as input and a denoised steel image as output to train a generative adversarial network. This improves denoising performance while preserving image details, solving the problem of losing high-frequency features during denoising. Furthermore, a pre-trained Swin transformer model and a pre-trained convolutional neural network model are used to extract the first and second features of the denoised steel image to be detected, respectively. Multi-scale feature fusion is then performed on the first and second features, achieving the fusion of local and global features of the steel image to be detected. This solves the problems of feature loss and feature redundancy during image feature extraction, thereby optimizing the accuracy of feature extraction from the steel image to be detected. Finally, a region proposal network and a region of interest network are used to process the feature map, accurately determining the defect type and location of the steel to be detected, thus improving the accuracy of steel defect detection. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0041] Figure 1 This invention provides a schematic flowchart of a steel defect detection method.

[0042] Figure 2 A schematic diagram of the technical route for a steel defect detection method provided by the present invention;

[0043] Figure 3 This is a schematic diagram of the denoising network model architecture provided by the present invention;

[0044] Figure 4 A schematic diagram of the structure of the pre-trained convolutional neural network model and the pre-trained swin transformer model provided by the present invention;

[0045] Figure 5 This is a schematic diagram of the structure of the adaptive multi-scale fusion network provided by the present invention;

[0046] Figure 6 This is a schematic diagram of the structure of the selectable convolution kernel module provided by the present invention;

[0047] Figure 7 This is a schematic diagram of the MBconv network structure provided by the present invention;

[0048] Figure 8 This is a schematic diagram of a steel defect detection device provided by the present invention;

[0049] Figure 9 This is a schematic diagram of a computer device for implementing a method for detecting defects in steel, as provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0051] The server mentioned in this invention can be a server set up on a business platform, or a device such as a desktop computer or laptop computer capable of executing the solution of this invention. For ease of explanation, the following description focuses solely on the server as the executing entity. The technical solutions provided by various embodiments of this invention are described in detail below with reference to the accompanying drawings.

[0052] refer to Figure 1 The steel defect detection method of the present invention specifically includes the following steps:

[0053] S10: Obtain the training dataset of images related to steel, and obtain the image of the steel to be detected.

[0054] In this embodiment, the steel-related image training dataset is a raw image dataset containing noise, and the image of the steel to be detected is obtained by taking pictures of the steel that needs to be defect detected.

[0055] S20: Using the images in the image training dataset as input and the denoised steel image as output, train the generative adversarial network to obtain a denoising network model. The generator is used to denoise the images in the image training dataset to obtain images without noise; the discriminator is used to calculate the similarity probability between the images in the image training dataset and the images without noise.

[0056] In this embodiment, reference Figure 2 The image training dataset is used as input, and the denoised steel image is used as output to train the Generative Adversarial Network (GAN) to obtain the denoised network model.

[0057] In this embodiment, images from the image training dataset are used as input, and denoised steel images are used as output to train a generative adversarial network, resulting in a denoised network model. Specifically, this includes:

[0058] The Nth image in the image training dataset is input into the generator of the generative adversarial network to obtain the first denoised image of the Nth image, where N is a positive integer.

[0059] The Nth image and its first denoised image are input into the discriminator of the generative adversarial network to obtain the similarity probability between the Nth image and its first denoised image.

[0060] The similarity probability is input into the loss function of the generative adversarial network to obtain the loss value. The loss value is then input into the generator for iterative training using adaptive moment estimation (Adam) to obtain the denoising network model.

[0061] Specifically, the structure of generative adversarial networks is as follows: Figure 3 As shown, the model includes a generator network, a discriminator network, and a loss function. The generator network contains a generator G, which removes noise from noisy images (each image in the image training dataset) to obtain a denoised image (the first denoised image). The discriminator network contains a discriminator D, which takes the first denoised image generated by generator G and the noisy image as input. The output is a value in the range [0, 1], representing the similarity probability between the denoised image and the noisy image, and this similarity probability is used as input to the loss function. The loss function determines the loss value based on the similarity probability between the noisy image and the first denoised image. This loss value is then input into generator G for iterative training using adaptive moment estimation to obtain the denoising network model. Using this denoising network model, noise in the noisy image can be removed, resulting in the final denoised image. The first denoised image refers to the image output by generator G, and the denoised image of the steel to be detected refers to the final denoised image output by the denoising network model.

[0062] Specifically, the generator G consists of a parallel neural network with residual dense blocks, which can extract more feature information without increasing time cost. The discriminator network uses a fully convolutional network structure to achieve pixel-level classification of images, thereby improving the performance of the discriminator D.

[0063] S30: Input the image of the steel to be detected into the denoising network model to obtain the denoised image of the steel to be detected; input the denoised image of the steel to be detected into the pre-trained swin transformer model and the pre-trained convolutional neural network model respectively for feature extraction to obtain the first feature and the second feature of the steel to be detected, and perform multi-scale feature fusion on the first feature and the second feature to obtain the feature map of the steel to be detected.

[0064] In this embodiment, reference Figure 2The denoised image of the steel to be detected is input into a pre-trained Swin Transformer model and a pre-trained convolutional neural network model for feature extraction. The first feature refers to the global feature extracted from the denoised image of the steel to be detected using the pre-trained Swin Transformer model, and the second feature refers to the local feature extracted from the denoised image of the steel to be detected using the pre-trained convolutional neural network model.

[0065] For details, please refer to Figure 4 This method uses the Swing Transformer and CNN as the backbone networks of two branches, dynamically selecting, weighting, and fusing features from both branches. By leveraging local features extracted by the CNN and global features extracted by the Swing Transformer, it avoids information loss or feature redundancy problems that may exist in traditional methods. It adaptively adjusts the importance of different features according to task requirements, thereby optimizing detection performance. The Swing Transformer model includes a patch partition module, a linear embedding module, a first shift-window transformer module (Swin Transformer Block), a first path fusion module (PatchMerging), a second shift-window transformer module (Swin Transformer Block), a second path fusion module (Patch Merging), a third shift-window transformer module (Swin Transformer Block), a third path fusion module (Patch Merging), and a fourth shift-window transformer module (Swin Transformer Block).

[0066] The convolutional neural network model consists of five convolutional layers. The second, third, fourth, and fifth convolutional layers are each composed of three residual blocks.

[0067] S40: The feature map is processed using a region proposal network and a region of interest network to obtain the defect type and defect location of the steel to be detected.

[0068] based on Figure 1This invention presents a steel defect detection method. It uses a training dataset of steel-related images as input and a denoised steel image as output to train a generative adversarial network. This improves denoising performance while preserving image details, addressing the problem of losing high-frequency features during denoising. Furthermore, a pre-trained Swing Transformer model and a pre-trained Convolutional Neural Network model extract the first and second features of the denoised steel image, respectively. Multi-scale feature fusion is then performed on the first and second features, achieving the fusion of local and global features of the steel image. This solves the problems of feature loss and redundancy during feature extraction, thereby optimizing the accuracy of feature extraction from the steel image. Finally, a Region Proposal Network (RPN) and a Region of Interest (ROI) network are used to process the feature map, accurately determining the defect type and location of the steel, thus improving the accuracy of steel defect detection.

[0069] When applying the steel defect detection method provided by this invention, it is not necessary to rely on... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this invention does not impose any restrictions on it.

[0070] Furthermore, in one or more embodiments of the present invention, the feature map is processed using a region proposal network and a region of interest network to obtain the defect type and defect location of the steel to be detected, specifically including:

[0071] The feature map is input into the region proposal network to obtain the processed feature map.

[0072] The processing results of the feature map are input into the region of interest network for further processing to obtain the defect type and defect location of the steel to be detected.

[0073] In this embodiment, reference Figure 2 The feature map is first input into the Region Proposal Network (RPN) for preliminary processing, and then the pre-processed feature map is input into the Region of Interest (ROI) for further processing to obtain the defect type and defect location of the steel to be detected.

[0074] This embodiment combines the advantages of RPN and ROI (ROI can focus on specific regions in an image, avoiding indiscriminate feature extraction of the entire image, thereby improving the efficiency and targeting of feature extraction, and helping to more accurately identify and locate target objects; RPN uses sliding window or convolution to quickly calculate on the image feature map when generating candidate regions, which can generate a large number of candidate regions in a short time), improving the accuracy and efficiency of obtaining the defect type and defect location of the steel to be detected.

[0075] The region proposal network includes an anchor box layer, a classification and regression layer, and a non-maximum suppression layer; the region of interest network includes a region of interest pooling layer, a fully connected layer, and a classification and regression layer.

[0076] Furthermore, in one or more embodiments of the present invention, the loss function is obtained by weighted summation of adversarial loss, perceptual loss, and image texture feature matching loss function combined with discriminator, and its calculation expression is as follows:

[0077] ;

[0078] Where Loss is the output of the loss function. To combat the losses, In order to perceive loss, The loss function is used to match the image texture features of the discriminator. yes The weight, yes The weight, yes The weight.

[0079] In this embodiment, , and The specific values ​​were determined through experiments. First, we observed the training of each loss function individually. When each loss function was close to convergence, we set the baseline weights based on the order of magnitude of the loss functions. For example, assuming the first loss function is around 0.02, and the second and third loss functions are around 0.2, then we set the baseline weights to 1:10.

[0080] This embodiment constructs a composite loss function from aspects such as pixel, structural and semantic features of the image, so that the denoising network model has network stability, denoising effectiveness and maintains the stability of image content.

[0081] Specifically, The calculation expression is:

[0082] ;

[0083] in, Let G represent the i-th convolutional layer of the discriminator, and let G represent the generator. This represents the data distribution of the original noise samples. Let z represent the data distribution of samples without noise, where z represents noise and T represents the number of layers in the discriminator. This represents the number of elements in each layer, and k represents the criterion. Represents the k-th discriminant metric G represents.

[0084] In this embodiment, Euclidean distance is used to measure the feature representation of the noisy image and the denoised image at different scales, and the comparison results are summarized into the total loss.

[0085] Furthermore, in one or more embodiments of the present invention, multi-scale feature fusion of the first feature and the second feature is implemented by an adaptive multi-scale feature fusion module, the expression of which is:

[0086] ;

[0087] Among them, P i For the features after fusion, C i Let C be the feature map of the i-th layer. i+1 Let be the feature map of the (i+1)th layer.

[0088] In this embodiment, an Adaptive Multi-Scale Feature Fusion (AMFF) module is constructed based on the Feature Pyramid Network (FPN). AMFF not only inherits the core ideas of FPN, but also introduces an adaptive receptive field adjustment mechanism and enhances the interaction between the current layer and adjacent layers, thereby more effectively capturing the diverse features of the target.

[0089] For details, please refer to Figure 5 The adaptive multi-scale feature fusion module includes a cross-attention branch (CEAM) and a selectable convolutional kernel (LSK) branch.

[0090] The cross-attention branch is used to determine the relationship between adjacent convolutional layers and to fuse the feature information of different convolutional layers through the cross-layer attention mechanism.

[0091] In this embodiment, the CEAM branch aims to seek the intrinsic connections between adjacent convolutional layers, dynamically fusing feature information from different layers through a cross-layer attention mechanism. The specific expression is as follows:

[0092] ;

[0093] in, It is the output of the CEAM branch, where sig is the sigmoid activation function. This is a pixel-wise multiplication. Guided by the features of the i-th layer, this branch provides more valuable contextual interaction information for fusion. Branch ensemble: After the above effective operations, the outputs of LSK and CEAM are added to the original i-th layer feature map as residuals.

[0094] ;

[0095] in It involves pixel-wise summation. In summary, the features of the i-th layer are adaptively adjusted through intra-layer and cross-layer attention mechanisms. Compared to the traditional FPN, the AMFF module offers greater flexibility and adaptability. It can adaptively adjust the fusion method and receptive field size according to the specific characteristics of the target, thereby capturing the target's feature information more accurately. Simultaneously, the AMFF module enhances the model's ability to detect multi-scale targets by strengthening inter-layer interactions.

[0096] The LSK branch is used to adaptively select different convolution kernels in each layer of feature maps and adjust the receptive field of the convolution target in each layer of feature maps.

[0097] refer to Figure 6 The LSK branch allows the adaptive multi-scale feature fusion module to adaptively select different large convolutional kernels on the i-th layer feature map and adjust the receptive field of each convolutional target in the space of each layer feature map as needed. This structure mainly consists of large kernel convolution and spatial selection mechanisms.

[0098] To achieve efficient modeling and adaptive selection of multiple long-range contexts, the LSK branch constructs a series of deep convolutional sequences with large growth kernels and increasing dilation rates.

[0099] Specifically, the kernel size k, dilation rate d, and receptive field RF dilation of the i-th depthwise convolution in this series are defined as follows:

[0100] ;

[0101] ;

[0102] Increasing the kernel size and dilation rate ensures the receptive field dilates quickly enough, while setting an upper limit on the dilation rate ensures that dilated convolutions do not introduce gaps between feature maps. When extracting rich contextual information from different ranges of the input, the LSK branch uses a sequence of deep convolutions containing different receptive fields, rather than a single large convolutional kernel. This approach generates multiple features with different large receptive fields, facilitating subsequent kernel selection. Furthermore, through sequential decomposition, i.e., using a stack of multiple small convolutional kernels, compared to a single large convolutional kernel, the number of parameters is significantly reduced while maintaining the same theoretical receptive field, improving the model's efficiency and flexibility.

[0103] Furthermore, in one or more embodiments of the present invention, the pre-trained convolutional neural network model is constructed and trained based on EfficientNet.

[0104] In this embodiment, EfficientNet is a lightweight model designed by Google for edge computing devices. As an efficient CNN architecture, EfficientNet offers a good balance between performance and computational resources, making it particularly suitable as a lightweight CNN branch for steel defect detection. Through composite scaling and lightweight design, EfficientNet reduces computational overhead and memory usage while ensuring high accuracy, making it ideal for real-time inference in industrial environments. Combining EfficientNet with the Swin Transformer further enhances feature extraction and global modeling capabilities, thus better addressing the complex scenarios in steel defect detection tasks.

[0105] For details, please refer to Figure 7 The EfficientNet feature extraction network consists of MBConv blocks, which are composed of SENet (Squeeze and Excitation Network) and a deep convolutional network. In the MBConv block structure, 1×1 convolutions are first used to increase the image dimensionality, followed by deep convolutions and SENet sequentially, and finally, 1×1 convolutions are used to reduce the dimensionality of the output image. EfficientNet achieves a more balanced network architecture by balancing the three dimensions of network depth, network width, and image resolution.

[0106] The above describes a method for detecting steel defects according to one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding device for detecting steel defects, such as... Figure 8 As shown, it includes:

[0107] The acquisition module is used to acquire a training dataset of images related to steel and to acquire images of the steel to be detected.

[0108] The training module is used to train the generative adversarial network by taking the images in the image training dataset as input and the denoised steel image as output, and obtaining the denoised network model.

[0109] The feature extraction module is used to input the image of the steel to be detected into the denoising network model to obtain the denoised image of the steel to be detected; the denoised image of the steel to be detected is then input into the pre-trained Swin transformer model and the pre-trained convolutional neural network model for feature extraction to obtain the first feature and the second feature of the denoised image of the steel to be detected; and the first feature and the second feature are then fused at multiple scales to obtain the feature map of the steel to be detected.

[0110] The detection module is used to process the feature map using a region proposal network and a region of interest network to obtain the defect type and defect location of the steel to be detected.

[0111] For specific limitations regarding a steel defect detection device, please refer to the limitations of a steel defect detection method described above, which will not be repeated here. Each module in the steel defect detection device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0112] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-mentioned... Figure 1 A method for detecting defects in steel is provided.

[0113] The present invention also provides Figure 9 The schematic diagram of the computer device shown is as follows: Figure 9 As shown, at the hardware level, this computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the aforementioned... Figure 1 A method for detecting defects in steel is provided.

[0114] Those skilled in the art will understand that implementing all or part of the processes in the methods of the embodiments can be accomplished by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.

Claims

1. A method for detecting flaws in steel products, characterized by, The method comprises the following steps: An image training data set related to steel is acquired, and an image of a steel to be detected is acquired; A generative adversarial network is trained by taking an image in the image training data set as input and taking a denoised steel image as output to obtain a denoising network model, specifically comprising: inputting an Nth image in the image training data set into a generator of the generative adversarial network to obtain a first denoised image of the Nth image, N being a positive integer; inputting the Nth image and the first denoised image of the Nth image into a discriminator of the generative adversarial network to obtain a similarity probability between the Nth image and the first denoised image of the Nth image; inputting the similarity probability into a loss function of the generative adversarial network to obtain a loss value, and inputting the loss value into the generator for adaptive moment estimation iterative training to obtain the denoising network model; The loss function is obtained by weighted summation of an adversarial loss, a perception loss and a loss function combining image texture features of the discriminator, and the calculation expression is: ； In the formula, Loss is the output of the loss function, is the adversarial loss, is the perceptual loss, is the loss function of the image texture feature matching combined with the discriminator, is is the weight of is is the weight of is is the weight of The The computational expression is: ； In the formula, represents the i-th convolution layer of the discriminator, G represents the generator, represents the data distribution of the original noise sample, represents the data distribution of the sample without noise, z represents noise, and T represents the number of layers of the discriminator, represents the number of elements of each layer, and k represents the scale of discrimination, represents the number of elements of each layer, and k represents the scale of discrimination, ; The image of the steel to be detected is inputted into the denoising network model to obtain a denoised image of the steel to be detected; the denoised image of the steel to be detected is inputted into a pre-trained swin transformer model and a pre-trained convolutional neural network model respectively for feature extraction to obtain a first feature and a second feature of the denoised image of the steel to be detected, and the first feature and the second feature are subjected to multi-scale feature fusion to obtain a feature map of the steel to be detected; The feature map is processed by a region proposal network and a region of interest network to obtain a defect type and a defect position of the steel to be detected.

2. The method for detecting defects in steel as described in claim 1, characterized in that, The multi-scale feature fusion of the first feature and the second feature is realized by an adaptive multi-scale feature fusion module, and the expression of the adaptive multi-scale feature fusion is: ； Wherein, P i is the fused feature, C i is the feature map of the i-th layer, C i+1 is the feature map of the i+1-th layer.

3. A method of inspecting a steel material for defects as claimed in claim 2, wherein The adaptive multi-scale feature fusion module comprises a cross-attention branch and a selectable convolution kernel branch; The cross-attention branch is used to determine the correlation between adjacent convolution layers and fuse feature information of different convolution layers through a cross-layer attention mechanism; The selectable convolution kernel branch is used to adaptively select different convolution kernels in each layer feature map and adjust the receptive field of the convolution target in each layer feature map.

4. The method for detecting defects in steel as described in claim 1, characterized in that, The processing of the feature map by the region proposal network and the region of interest network to obtain the defect type and the defect position of the steel to be detected specifically comprises: The feature map is inputted into the region proposal network to obtain a processing result of the feature map; The processing result of the feature map is inputted into the region of interest network for further processing to obtain the defect type and the defect position of the steel to be detected.

5. The method for detecting defects in steel as described in claim 1, characterized in that, The pre-trained convolutional neural network model is obtained based on EfficientNet.

6. A steel material flaw detection device characterized by comprising: The method comprises the following steps: An image training data set related to steel is acquired, and an image of a steel to be detected is acquired; The training module is configured to train the generative adversarial network by taking images in the image training dataset as input and taking denoised steel images as output, to obtain a denoising network model, and specifically includes: inputting an Nth image in the image training dataset into a generator of the generative adversarial network to obtain a first denoised image about the Nth image, where N is a positive integer; inputting the Nth image and the first denoised image of the Nth image into a discriminator of the generative adversarial network to obtain a similarity probability between the Nth image and the first denoised image of the Nth image; inputting the similarity probability into a loss function of the generative adversarial network to obtain a loss value, and inputting the loss value into the generator for adaptive moment estimation iterative training to obtain the denoising network model; The loss function is obtained by weighted summation of an adversarial loss, a perception loss and a loss function combining image texture feature matching of the discriminator, and has a calculation expression as follows: ； In the formula, Loss is the output of the loss function, is the adversarial loss, is the perceptual loss, is the loss function of the image texture feature matching combined with the discriminator, is the weight of is the weight of is the weight of The The computational expression is: ； In the formula, represents the i-th convolution layer of the discriminator, G represents the generator, represents the data distribution of the original noise sample, represents the data distribution of the sample without noise, z represents noise, T represents the number of layers of the discriminator, represents the number of elements of each layer, k represents the scale of discrimination, represents the number of elements of each layer, k represents the scale of discrimination, ; The feature extraction module is configured to input the image of the steel to be detected into the denoising network model to obtain a denoised image of the steel to be detected; input the denoised image of the steel to be detected into a pre-trained swintransformer model and a pre-trained convolutional neural network model respectively for feature extraction to obtain a first feature and a second feature of the denoised image of the steel to be detected, and perform multi-scale feature fusion on the first feature and the second feature to obtain a feature map about the steel to be detected. The detection module is configured to process the feature map by using a region proposal network and a region of interest network to obtain a flaw type and a flaw position of the steel to be detected.

7. A computer readable storage medium characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the steel flaw detection method according to any one of claims 1-5.

8. A computer device, comprising: The computer program is stored in the memory and executable on the processor, and the processor implements the steel flaw detection method according to any one of claims 1-5 when executing the program.

Images