A steel surface defect detection method and system based on PPY-YOLO

Abstract

The present invention discloses a steel surface defect detection method and system based on PPY-YOLO, the method comprising step 1: preprocessing of steel surface defect images; step 2: obtaining a PPY-YOLO target detection model by introducing a neck structure Faster-Neck, a deep learning module C3K2_Faster and a spatial pyramid pooling module SPPFSim in a YOLOv11 network; step 3: training the PPY-YOLO target detection model; step 4: obtaining a steel image, inputting the trained PPY-YOLO target detection model, and obtaining the category and positioning information of the steel surface defects. The present invention constructs a new model, by improving a backbone network and a neck network, the backbone network is the input end of a C2PSA module connected to the output end of a SPPFSim module, and the neck network adds an upsampling group and a downsampling group, so that the model has a better detection effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of defect detection, and in particular relates to a steel surface defect detection method and system based on PPY-YOLO. Background Art

[0002] In the production process of steel, various defects are inevitable, such as surface scratches, cracks, dents, and internal inclusions or voids. If these defects are not discovered and handled in time, they may have a significant adverse effect on the quality and performance of steel, and then affect important indicators such as the appearance, mechanical properties, and corrosion resistance of the product, and may even cause serious safety hazards in certain key application fields (such as construction, automobiles, aerospace, etc.). Therefore, steel defect detection technology has become particularly important in modern industrial production, which can effectively ensure the quality of steel and reduce the safety risks caused by defective products.

[0003] Traditional steel defect detection methods, such as manual visual inspection, have long been widely used in the industrial field. However, these traditional methods have many limitations. First, manual inspection is inefficient and easily affected by human factors, such as inspector fatigue, emotions, and experience differences, which can lead to insufficient inspection accuracy and consistency. Second, manual inspection is highly dependent on external conditions such as lighting and environment, and cannot meet the needs of high precision and high efficiency in large-scale production processes.

[0004] Compared with traditional methods, machine learning-based detection technologies are considered to be potential solutions in the field of steel defect detection. The advantages of these methods are that they can realize automated detection, process large amounts of data, and reduce human intervention. However, machine learning-based detection methods also face some challenges. First, in complex scenarios, it is often difficult for the model to effectively capture the subtle features of the defects, resulting in unsatisfactory detection results. Secondly, many machine learning models have poor generalization capabilities, especially when processing data of different types or from different sources, the performance of the model may be greatly reduced. Furthermore, the computational efficiency of some machine learning models is low, and it is difficult to meet the needs of practical applications in high-speed production lines that require real-time detection.

[0005] In recent years, with the rapid development of deep learning technology, convolutional neural networks (CNNs) have been widely used in various fields, especially in image recognition and detection. Traditional detection methods are inefficient and have limited accuracy, and the introduction of deep learning technology has brought new solutions and breakthroughs to steel surface defect detection. Among them, the YOLO series of algorithms have attracted widespread attention because of their high speed and high accuracy. In recent years, it has been updated to YOLOv11, but YOLOv11 still has some limitations in the application of steel surface defect detection, such as low detection accuracy, etc. That is, there is still a better balance to be achieved between detection accuracy, detection efficiency and algorithm lightweight. Summary of the invention

[0006] The purpose of the present invention is to solve the problems existing in the above-mentioned prior art and propose a steel surface defect detection method and system based on PPY-YOLO to improve the detection accuracy of steel surface defect detection.

[0007] To this end, the present invention provides the following technical solutions:

[0008] On the one hand, a steel surface defect detection method based on PPY-YOLO includes:

[0009] Step 1: Preprocess the acquired steel surface defect image;

[0010] Step 2: Obtain the PPY-YOLO target detection model by introducing the neck structure Faster-Neck, the deep learning module C3K2_Faster, and the spatial pyramid pooling module SPPFSim into the YOLOv11 network;

[0011] PPY stands for Preprocessing Pyramid Yield-YOLO;

[0012] The PPY-YOLO target detection model includes a backbone network, a neck network and a detection head connected in series, wherein the input end of the C2PSA module of the backbone network is connected to the output end of the SPPFSim module, and the neck structure Faster-Neck adds a group of upsampling groups and a group of downsampling groups on the basis of the existing neck network;

[0013] Step 3: Input the training set obtained by preprocessing in step 1 into the PPY-YOLO target detection model for training, and input the validation set during the training process, and verify the training effect through the test set to obtain the trained PPY-YOLO target detection model;

[0014] Step 4: Use a camera to obtain steel images during the industrial steel production process, input the trained PPY-YOLO target detection model, and obtain the category and location information of steel surface defects and the result confidence.

[0015] By constructing a new model, an improved backbone network structure and a neck network structure are used. The improved backbone network structure is that the input end of the C2PSA module is connected to the output end of the SPPFSim module. The improved neck network structure adds upsampling groups and downsampling groups, which makes the network show higher sensitivity when detecting targets with obvious edge features such as steel surface defects. The model formed in this way has a better detection effect.

[0016] The acquired steel surface defect images are preprocessed, specifically, the data set images are processed by histogram equalization, piecewise linear grayscale transformation and Gamma transformation data enhancement methods. Through these data enhancement processing methods, the data set can be denoised and enhanced, thereby initially expanding the data set, and then the data set is further expanded by generating adversarial networks (GANs).

[0017] In the detection head, an anchor-free decoupled detection head is adopted to separate the classification and positioning tasks.

[0018] Furthermore, the backbone network is composed of two Conv modules connected in series, and then connected in series, a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a SPPFSim module, and a C2PSA module;

[0019] The Conv module includes a standard two-dimensional convolutional layer, a BatchNormalization layer, and an activation function SILU layer connected in series.

[0020] The C3K2 module includes a Conv module and a C3K module, and the C2PSA module includes a Conv module and a PSA module connected in series.

[0021] Furthermore, in the neck structure Faster-Neck, the newly added upsampling group on the existing neck network includes an upsampling module, a feature fusion module, and a deep learning module C3K2_Faster connected in series in sequence, and the newly added downsampling group includes a Conv module, a deep learning module C3K2_Faster, and a feature fusion module connected in series in sequence, and the C3K2 module in the existing neck network is replaced by the deep learning module C3K2_Faster;

[0022] The deep learning module C3K2_Faster includes a Conv module, a Split operation layer, several FasterBlock modules or C3K Faster modules, a Concat operation layer and a Conv module connected in series in sequence, and the output ends of the first Conv module, the Split operation layer and several FasterBlock modules or C3K Faster modules are all connected to the input end of the Concat operation layer.

[0023] A new set of upsampling groups can amplify the resolution of feature maps, so that shallow features containing local texture or edge information that can clearly capture defects such as scratches, cracks, and pits in steel can be spatially aligned and fused with higher-level features. This process not only retains the detailed information in shallow features, but also provides richer spatial context information for subsequent detection of the overall shape, category, location, and distribution pattern of steel defects by improving the resolution of feature maps.

[0024] The newly added downsampling group can combine features at different levels while extracting features, and can comprehensively utilize the advantages of features at each level to form a more comprehensive and representative feature representation. Through the newly added upsampling layer and feature fusion layer, shallow features containing texture and edge information, which can help the model preliminarily distinguish possible defect areas and backgrounds at lower resolutions, and deep features containing the overall shape, position and distribution pattern of defects, which can help the model accurately judge the category and nature of defects, can be organically combined. The new deep learning module C3K2_Faster optimizes multi-scale feature extraction and suppresses background noise, so that the network can provide higher detection accuracy in complex scenarios. At the same time, through a more efficient feature processing mechanism, it reduces the computational burden and achieves a good balance between performance and resource utilization. That is, the C3K2_Faster module optimizes the network computing efficiency and reduces network parameters. The new neck structure Faster-Neck proposed in the present invention can achieve more comprehensive and accurate identification and positioning of small target defects in steel.

[0025] Furthermore, the spatial pyramid pooling module SPPFSim module integrates the SPPF module with the SimAM attention mechanism, and includes two standard convolutional layers and three maximum pooling layers, a SimAM attention mechanism and a Concat layer, that is, the input feature map is subjected to the attention mechanism of the SimAM module to extract feature information, and the feature information is divided into three, which are processed by three maximum pooling layers respectively, and the outputs of the three pooling layers are spliced ​​to finally obtain the final feature map.

[0026] The structure of SPPFSim module is as follows Figure 8As shown in Figure 1, it can not only retain the multi-scale feature fusion and feature map semantic features of the original SPPF module, but also enable the network to adaptively strengthen the focus on key areas and suppress the interference of irrelevant areas. The SimAM module is a parameter-free attention mechanism with a structure as follows: Fig. 9 As shown in the figure, effective features can be further selected without adding additional calculation parameters and calculation amount. The new spatial pyramid pooling module SPPFSim not only integrates multi-scale features and enriches the semantic features of feature maps, but also significantly enhances the model's ability to focus on key areas in the image;

[0027] Furthermore, the loss functions used in the training of the PPY-YOLO target detection model include classification loss function and positioning loss function, where the classification loss function uses cross entropy loss, and the positioning loss function uses CIOU and distributed focus DFL loss functions;

[0028] The overall loss function calculation formula is as follows:

[0029] ;

[0030] Among them, L cls represents the classification cross entropy loss function, L reg To locate the CloU loss function, L dfl is the distributed focus DFL loss function, α is the balance coefficient of classification loss, The balance coefficient of positioning loss, γ is the balance coefficient of positioning distributed focus loss, is the number of cells assigned as positive samples.

[0031] In the second aspect, a steel surface defect detection system based on PPY-YOLO comprises:

[0032] Preprocessing module: preprocess the acquired steel surface defect images;

[0033] Model building module: The PPY-YOLO target detection model is obtained by introducing the neck structure Faster-Neck, the deep learning module C3K2_Faster, and the spatial pyramid pooling module SPPFSim into the YOLOv11 network;

[0034] The PPY-YOLO target detection model includes a backbone network, a neck network and a detection head connected in series, wherein the input end of the C2PSA module of the backbone network is connected to the output end of the SPPFSim module, and the neck structure Faster-Neck adds a group of upsampling groups and a group of downsampling groups on the basis of the existing neck network;

[0035] Training module: The training set obtained by preprocessing the preprocessing module is input into the PPY-YOLO target detection model for training, and the validation set is input during the training process. The training effect is verified by the test set, thereby obtaining a trained PPY-YOLO target detection model;

[0036] Detection module: In the process of industrial steel production, cameras are used to obtain steel images, and the trained PPY-YOLO target detection model is input to obtain the category and location information of steel surface defects and the confidence of the results.

[0037] Furthermore, the backbone network is composed of two Conv modules connected in series, and then connected in series, a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a SPPFSim module, and a C2PSA module;

[0038] The Conv module includes a standard two-dimensional convolutional layer, a BatchNormalization layer, and an activation function SILU layer connected in series.

[0039] The C3K2 module includes a Conv module and a C3K module, and the C2PSA module includes a Conv module and a PSA module connected in series.

[0040] Furthermore, in the neck structure Faster-Neck, the newly added upsampling group on the existing neck network includes an upsampling module, a feature fusion module, and a deep learning module C3K2_Faster connected in series in sequence, and the newly added downsampling group includes a Conv module, a deep learning module C3K2_Faster, and a feature fusion module connected in series in sequence, and the C3K2 module in the existing neck network is replaced by the deep learning module C3K2_Faster;

[0041] The deep learning module C3K2_Faster includes a Conv module, a Split operation layer, several FasterBlock modules or C3K Faster modules, a Concat operation layer and a Conv module connected in series in sequence, and the output ends of the first Conv module, the Split operation layer and several FasterBlock modules or C3K Faster modules are all connected to the input end of the Concat operation layer.

[0042] Furthermore, the spatial pyramid pooling module SPPFSim module integrates the SPPF module with the SimAM attention mechanism, and includes two standard convolutional layers and three maximum pooling layers, a SimAM attention mechanism and a Concat layer, that is, the input feature map is subjected to the attention mechanism of the SimAM module to extract feature information, and the feature information is divided into three, which are processed by three maximum pooling layers respectively, and the outputs of the three pooling layers are spliced ​​to finally obtain the final feature map.

[0043] In a third aspect, a computer-readable storage medium stores a computer program, wherein the computer program is called by a processor to implement:

[0044] The above steps of a steel surface defect detection method based on PPY-YOLO.

[0045] Beneficial Effects

[0046] Compared with the existing method, the advantages of the present invention are:

[0047] 1. The steel surface defect detection method based on PPY-YOLO described in the technical solution of the present invention uses a data enhancement method when preprocessing the data set, and also uses a generative adversarial network GAN to further expand the data set, so that the model can be better trained and the overfitting problem caused by insufficient number of data set samples that may occur during the model training process is reduced, thereby improving the robustness of the model and further improving the detection performance of the model.

[0048] 2. The steel surface defect detection method based on PPY-YOLO described in the technical solution of the present invention, on the basis of the existing YOLOv11 network, optimizes the backbone network SPPF module into a new spatial pyramid pooling module SPPFSim, which not only retains the ability of the original SPPF module to fuse shallow features with high-level features so that the model can obtain more complete feature information, but also calculates the similarity between each pixel and adjacent pixels between feature images, thereby generating a corresponding attention weight for each neuron, so that the network is more sensitive to small steel surface defect areas with large local area differences, and suppresses the interference of different lighting and angles on the detection effect in the steel surface defect detection scenario, thereby improving the model's detection performance for steel surface defects.

[0049] 3. The model designed by the technical solution of the present invention introduces Faster-Neck, so that the extracted detection information contains texture and edge information. The newly added upsampling group can help the model to enlarge the resolution of the feature map, so that the shallower features can be aligned and fused in space, and the newly added downsampling group can further integrate the shallow features with the deep features. The network can comprehensively utilize the advantages of features at all levels to form a more comprehensive and representative feature representation, so that the network can better combine the shallow features containing the overall shape, position and distribution pattern of the defect with the deep features that can help the model accurately judge the category and nature of the defect, so as to achieve more comprehensive and accurate identification and positioning of small target defects in steel.

[0050] 4. The model designed by the technical solution of the present invention introduces the C3K2_Faster module, which not only retains the feature extraction capability of the original C3K2 module, but also enables the network to converge faster, so that the model can also perform well under limited steel surface defect data sets, and can more fully extract the local details and global patterns of steel defects, strengthen the model's attention to tiny features, and the C3K2_Faster module also reduces the computational complexity and speeds up the reasoning speed. This is particularly important for steel surface detection applications that require real-time detection. The improved PPY-YOLO steel surface defect detection method has a higher detection accuracy while reducing the number of parameters and increasing the detection efficiency compared to the traditional YOLOv11 steel surface defect detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a principle flow chart of the technical solution of the present invention;

[0052] Figure 2 This is a schematic diagram of the PPY-YOLO steel surface defect detection model of the technical solution of the present invention;

[0053] Figure 3 This is a schematic diagram of the C3K2 module in the technical solution of the present invention;

[0054] Figure 4 It is a schematic diagram of the C3K2_Faster module in the technical solution of the present invention;

[0055] Figure 5 It is a schematic diagram of the FasterBlock module in the technical solution of the present invention;

[0056] Figure 6 It is a schematic diagram of the PConv module in the technical solution of the present invention;

[0057] Figure 7 Schematic diagram of the C2PSA module in the technical solution of the present invention;

[0058] Figure 8 It is a schematic diagram of the SPPFSim module in the technical solution of the present invention;

[0059] Fig. 9 It is a schematic diagram of the SimAM module in the technical solution of the present invention;

[0060] Fig.10 It is a schematic diagram of the Faster-Neck structure in the technical solution of the present invention;

[0061] Fig.11 It is a schematic diagram comparing the detection results of steel surface defect detection samples by the PPY-YOLO model of the present invention and the baseline model. DETAILED DESCRIPTION

[0062] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0063] like Figure 1 As shown, a steel surface defect detection method based on PPY-YOLO includes:

[0064] Step 1: Preprocess the acquired steel surface defect image;

[0065] The steel production process is captured using mobile devices to obtain and construct relevant data sets. In this example, the NEU-DET data set is used to replace the captured samples that have been acquired. This data set contains six typical steel surface defects, namely rolled-in scale (RS), patches (Pa), cracks (Cr), pitted surface (PS), inclusions (In) and scratches (Sc).

[0066] This dataset includes 1,800 grayscale images. First, the images in this dataset are subjected to denoising. The image enhancement method used is histogram equalization. The core idea of ​​histogram equalization is to redistribute the grayscale values ​​of the image so that the distribution of pixel values ​​is more uniform. This is done through the cumulative distribution function (CDF), which maps the original pixel values ​​to new grayscale values ​​so that the distribution of pixel grayscale values ​​of the entire image is closer to a uniform distribution, thereby enhancing the contrast of the image and making the distribution of bright and dark areas of the image more even, making the details clearer and achieving the purpose of denoising. The mathematical expression of histogram equalization is:

[0067] ;

[0068] in, Represents the grayscale output after mapping, Represents the conversion function of the kth gray level in the original image; is the number of pixels with gray levels 0-j, N is the total number of pixels, is a probability function and s∈[0,1], Represents the grayscale level of the output after mapping .

[0069] Then, the data set is initially expanded by performing data enhancement methods such as rotating the image to a specific angle, flipping the image, scaling the image, and blurring the image.

[0070] Since there are few images in the steel data set, the model may have problems such as difficulty in convergence during training, poor robustness and poor generalization ability. Therefore, the technical solution of the present invention further introduces generative adversarial networks (GAN) to further expand the data set. The generative adversarial network consists of two network parts: a generator and a discriminator. The generator network accepts random noise input and tries to generate data samples similar to the distribution of real data. Its goal is to generate sufficiently realistic data so that it can "fool" the discriminator network. The task of the discriminator network is to distinguish between real samples and generated samples. It accepts a batch of data samples (including both real samples and generated samples generated by the generator network) as input and outputs a probability value indicating the possibility that the sample is real data. The goals of the two are adversarial, that is, the generator network hopes that the generated samples can "fool" the discriminator network so that it cannot correctly distinguish between real and generated samples. The discriminator network hopes to distinguish which samples are real and which are generated as accurately as possible. After multiple iterations, the generator network and the discriminator network will confront each other and continuously improve their respective capabilities. Finally, when GAN converges, the generative network can generate realistic samples, and the discriminative network will not be able to easily distinguish between real samples and generated samples. That is, the output of the discriminative network is close to 0.5, indicating that the generated samples are basically close to the real samples, thereby achieving the effect of further expanding the data set.

[0071] Step 2: By introducing the neck structure Faster-Neck, the deep learning module C3K2_Faster, and the spatial pyramid pooling module SPPFSim into the YOLOv11 network, the PPY-YOLO target detection model is obtained. The model structure is as follows: Figure 2 As shown;

[0072] PPY stands for Preprocessing Pyramid Yield-YOLO;

[0073] The PPY-YOLO target detection model includes a backbone network, a neck network and a detection head connected in series, wherein the input end of the C2PSA module of the backbone network is connected to the output end of the SPPFSim module, and the neck structure Faster-Neck adds a group of upsampling groups and a group of downsampling groups on the basis of the existing neck network;

[0074] In this example, a decoupled detection head without anchor boxes is used to separate the classification and positioning tasks.

[0075] The backbone network is composed of two Conv modules connected in series, and then connected in series with a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a SPPFSim module and a C2PSA module;

[0076] The Conv module includes a standard two-dimensional convolutional layer, a BatchNormalization layer, and an activation function SILU layer connected in series.

[0077] The C3K2 module includes a Conv module and a C3K module, the C2PSA module includes a Conv module and a PSA module connected in series, and the structure of the C3K2 module is as follows: Figure 3As shown. The feature map first passes through the Conv module. After that, each time it passes through a module, it is divided into two feature maps using a Split operation. Then one of the feature maps passes through n C3K modules. Finally, the obtained feature map is combined with the feature map separated by the previous Split operation using the Concat operation. Finally, the features are integrated through a Conv module to obtain the final feature map. There is a C3K parameter in the network. When C3K is equal to True, it means that C3K2 in the network will enable n C3K modules, otherwise n BottleNeck modules will be used. The C3K module is composed of three Conv modules and one BottleNeck module. The essence of both the BottleNeck module and the C3K module is to combine the residual structure with the idea of ​​gradient diversion, so that the network can obtain rich gradient flow feature information in the feature extraction stage. However, due to the compression of the number of channels, the BottleNeck module may cause some feature information to be lost, and the BottleNeck module tends to be simplified in the expression of feature levels, which may not be sufficient for the extraction of subtle and local detail features, especially in the steel surface defect detection scenario, where the defect target is small, which will lead to unsatisfactory detection results. Moreover, if too many BottleNeck modules are stacked, the accumulated computational cost will be large as the model depth increases, and the memory and hardware requirements will be high. In the steel surface defect detection scenario, the computing power and storage capacity of mobile detection equipment are often poor, so it is difficult to meet the actual application requirements.

[0078] The neck structure Faster-Neck adds an upsampling group to the structure of the existing neck network, including an upsampling module, a feature fusion module, and a C3K2_Faster module connected in series in sequence; the newly added downsampling group includes a Conv module, a deep learning module C3K2_Faster, and a feature fusion module connected in series in sequence, and the C3K2 module in the existing neck structure is replaced by the deep learning module C3K2_Faster;

[0079] The deep learning module C3K2_Faster includes a Conv module, a Split operation layer, several FasterBlock modules or C3K Faster modules, a Concat operation layer and a Conv module connected in series, and the output ends of the first Conv module, the Split operation layer and several FasterBlock modules or C3K Faster modules are all connected to the input end of the Concat operation layer. The structure of the C3K2_Faster module is as follows: Figure 4As shown in the figure, the feature map first passes through the Conv module, and then each time it passes through a module, it is divided into two feature maps using a Split operation, and then one of the feature maps passes through n C3K_Faster modules, and finally the obtained feature map is combined with the feature map obtained by the previous Split operation using the Concat operation, and finally a Conv module is used to integrate the features to obtain the final feature map.

[0080] This example uses the FasterBlock module to optimize the BottleNeck module contained in the C3K module in the C3K2 module, or the BottleNeck module when the C3K module is not enabled. The structure of the FasterBlock module is as follows: Figure 5 As shown in the figure, it contains a PConv module and two Conv modules. The two Conv modules contain a BatchNormalization layer and an activation function SILU layer. The feature map first passes through the PConv module and is split into two feature maps. One of them passes through two Conv modules and a BatchNormalization layer, an activation function SILU layer, and finally adds the feature map separated by the Split operation to obtain the final feature map. The PConv module is a partial convolution. Its main function is to perform a convolution operation on a specific area in the input feature map, rather than processing the entire input feature map. The structure of the PConv module is as follows: Figure 6 As shown, the calculation amount and memory access calculation formulas of the PConv module are as follows:

[0081] = ;

[0082] ;

[0083] Among them, h represents the height of the feature map, w represents the width of the feature map, k represents the number of convolution kernels, Cp represents the number of channels of the feature map, FLOPs represents the amount of computation, and M represents the amount of memory access. Compared with the standard convolution, the computation amount of PConv partial convolution is 1 / 16 of that of the standard convolution, while the memory access amount is 1 / 4 of that of the standard convolution.

[0084] The subsequent standard convolution can not only be used to enhance the part processed by the previous PConv module, but also to process the part not processed by the PConv module, including the BatchNormalization layer between the two standard convolution layers and an activation function SILU layer, which is used to increase the nonlinear expression of the module. The main idea of ​​the FasterBlock module is to divide and conquer, focusing on the edge or area of ​​interest, thereby enhancing the pertinence of feature extraction, improving the model's detection effect in processing targets with obvious edge and texture features such as steel surface defects, and effectively reducing redundant calculations, making the calculation more efficient. That is, the FasterBlock module is a lighter module, replacing the original BottleNeck module, which can effectively reduce the number of model parameters and the complexity of the model, making the model more sensitive to fine textures, the difference between defects and surrounding normal areas, defect locations, the overall shape of defects, and the category of steel defects. At the same time, it reduces the amount of model calculations, reduces the number of model parameters, and reduces the amount of calculations, making the model easier to deploy on removable devices with low memory and computing performance, which is more in line with the actual application requirements of steel surface defect detection scenarios.

[0085] The structure of C2PSA module is as follows Figure 7 As shown in the figure, the structure of the C2PSA module is similar to that of the C3K2 module, that is, the C3K module of the C3K2 module is replaced by the PSABlock module. The essential idea is to combine the residual structure with the gradient splitting. The PSABlock module includes a multi-head attention mechanism layer and two standard convolutional layers. The C2PSA module can make the network better focus on effective features, better capture the semantically related areas in the image, and improve the global feature expression. Moreover, its unique structure can reduce the computational cost while maintaining the feature capture capability, which is particularly suitable for images with a large number of details.

[0086] The spatial pyramid pooling module SPPFSim module integrates the SPPF module with the SimAM attention mechanism, and includes two standard convolutional layers and three maximum pooling layers, a SimAM attention mechanism and a Concat layer, that is, the input feature map is subjected to the attention mechanism of the SimAM module to extract feature information, and the feature information is divided into three, which are processed by three maximum pooling layers respectively, and the outputs of the three pooling layers are spliced ​​to finally obtain the final feature map.

[0087] The structure of SPPFSim module is as follows Figure 8 As shown in Figure 1, it can not only retain the multi-scale feature fusion and feature map semantic features of the original SPPF module, but also enable the network to adaptively strengthen the focus on key areas and suppress the interference of irrelevant areas. The SimAM module is a parameter-free attention mechanism with a structure as follows: Fig. 9As shown in the figure, effective features can be further selected without adding additional calculation parameters and calculation amount. The new spatial pyramid pooling module SPPFSim not only integrates multi-scale features and enriches the semantic features of feature maps, but also significantly enhances the model's ability to focus on key areas in the image;

[0088] The new spatial pyramid pooling module SPPFSim introduces a similarity score at each pixel point in the feature map to measure the similarity between the pixel and its surrounding pixels, thereby obtaining the attention weight. The surface defects of steel are characterized by being small, irregular, and having large differences in local areas. SimAM can enhance the feature representation of these subtle differences, helping the model to better distinguish defects from the background, thereby enhancing the model's feature expression, improving the model's ability to recognize small defects in steel, and reducing the interference of background, lighting, and angles on the detection effect, making the model more robust. The calculation formula of SimAM is:

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] in: is the minimum energy function, and its value is inversely proportional to the feature difference between a neuron and its adjacent neurons. That is, the smaller the value, the greater the feature difference between a neuron and its adjacent neurons in the steel surface defect sample feature map, and the higher the attention weight of the neuron. t is the value of the neuron, and λ is the regularization coefficient. is the mean value of all neurons in each channel of the steel surface defect sample feature map, is the variance of all neurons in each channel of the steel surface defect sample feature map, M is the number of channels, represents other neurons in the i-th channel, X is the input feature map, For the enhanced feature map, The activation function is used to limit the value and obtain the final attention feature map. e is the base of the natural logarithm function.

[0094] Improved neck structure Faster-Neck, its structure is as follows Fig.10As shown in the Faster-Neck part of the figure, the image is sent to the neck network after the backbone network extracts features. After three groups of upsample, C3k2_Faster, Concat and three groups of Conv, C3k2_Faster, Concat, the three feature maps of different scales are sent to the detection head to obtain the detection results. The Faster-Neck neck structure still adopts the PAFPN architecture combining FPN and PAN, including the FPN part and the PAN part. The FPN part transfers features from high dimensions to low dimensions, while the PAN part transfers features from low dimensions to high dimensions. The entire neck network presents a symmetrical V shape, and these two paths will fuse adjacent features of the same level during the feature extraction process. This allows the network to fully integrate and utilize feature information at all levels, thereby improving the detection accuracy of the model. In the actual application scenarios of steel defect detection, the surface of the strip is often accompanied by complex background textures, lighting changes, and possible mechanical scratches and other interference factors. These complex backgrounds greatly increase the difficulty of detecting common small target defects in steel defect detection. Therefore, this example adds a group of upsampling groups including upsampling module, C3k2_Faster module, feature fusion module, and a group of downsampling groups including C3k2_Faster module, feature extraction module, feature fusion module in the existing neck structure new path, and improves the C3k2 module in the neck structure to C3k2_Faster module to obtain a new neck structure Faster-Neck. The newly added group of upsampling groups can enlarge the resolution of the feature map, so that the shallow features containing local texture or edge information and can clearly capture defects such as scratches, cracks, pits, etc. of steel can be aligned and fused with higher-level features in space. This process not only retains the detailed information in the shallow features, but also provides richer spatial context information for the subsequent detection of the overall shape, category, location and distribution pattern of steel defects by improving the resolution of the feature map. The newly added group of downsampling groups can combine features at different levels while extracting features, and can comprehensively utilize the advantages of features at each level to form a more comprehensive and representative feature representation. In summary, through the newly added upsampling group and downsampling group, shallow features containing texture and edge information can help the model preliminarily distinguish possible defect areas and backgrounds at a lower resolution, and deep features containing the overall shape, position and distribution pattern of the defects can help the model accurately judge the category and nature of the defects. They can be organically combined, while C3k2_Faster optimizes the network calculation efficiency and reduces the network parameters. The Faster-Neck proposed in this example can achieve more comprehensive and accurate recognition and positioning of small target defects in steel.

[0095] Step 3: Input the training set obtained by preprocessing in step 1 into the PPY-YOLO target detection model for training, and input the validation set during the training process, and verify the training effect through the test set to obtain the trained PPY-YOLO target detection model;

[0096] The obtained dataset is divided into training set, validation set and test set in a ratio of 8:1:1. Construct the data.yaml file corresponding to the dataset, fill in the corresponding dataset location, category type and number of categories, so that the network can identify the details of the dataset and its location, and then adjust the parameter file default.yaml file, which can adjust the corresponding hyperparameters, such as input image size, number of training rounds, etc. The corresponding parameters can be set according to the actual environment. The experimental environment of this example is: CPU is 13th Gen Intel(R) Core(TM) i9-13900HX, GPU is NVIDIAGeForce RTX 4060 Laptop GPU, Python version is 3.8, and deep learning framework is Pytorch1.13.1+Cuda11.3. The parameters set are as follows: the initial learning rate is set to 0.01, the momentum decay parameter value is set to 0.937, and the weight decay parameter value is set to 0.000 5. The number of training rounds is set to 150, the batch size is set to 8, and the SGD optimizer is used. The resolution of the input image is 640×640 after preprocessing. After the setting is completed, the training set and the validation set are input into the PPY-YOLO model for training. In each round of training, the input image is passed into the model, and the features are extracted through the backbone network, the neck network fuses the features, and the output is input to the detection head, and finally the detection result is output, and the detection result is compared with the label data. The loss is calculated through the loss function, and the weights of the unfrozen part of the network are updated, and this is repeated until the set number of training rounds is reached. , the results can be verified according to the validation set to adjust the hyperparameters to make the network training effect better. It is worth noting that this example will use mosaic data enhancement during the training process, that is, splicing 2 or 4 different images together to form a new image, which can not only enhance the diversity and quantity of the data set, but also increase the batchsize during network training, which is conducive to improving the training efficiency of the network. However, mosaic data enhancement may cause the distribution of training data to deviate, resulting in errors in network fitting, so this example will turn off mosaic data enhancement in the last 20 epochs and slightly correct the regression distribution of the network. Finally, the trained PPY-YOLO steel surface defect detection model is obtained, and the trained weight file best.pt is saved.

[0097] The loss functions used in training the PPY-YOLO target detection model include classification loss function and positioning loss function. The classification loss function uses cross entropy loss, while the positioning loss function uses CIOU and distributed focus DFL loss functions.

[0098] The overall loss function calculation formula is as follows:

[0099] ;

[0100] Among them, L cls represents the classification cross entropy loss function, L reg To locate the CloU loss function, L dfl is the distributed focus DFL loss function, α is the balance coefficient of classification loss, The balance coefficient of positioning loss, γ is the balance coefficient of positioning distributed focus loss, and the values ​​of each coefficient are determined according to the experimental training results. is the number of cells assigned as positive samples.

[0101] Step 4: Use a camera to obtain steel images during the industrial steel production process, input the trained PPY-YOLO target detection model, and obtain the category and location information of steel surface defects and the result confidence.

[0102] The final test results are as follows Fig.11 As shown, it can be clearly seen that the target detection model described in the technical solution of this example has better detection accuracy and lower missed detection rate and false detection rate in steel surface defect detection than the existing YOLOv11 model.

[0103] Finally, in order to verify the effectiveness of the PPY-YOLO model proposed in this example technical solution, the PPY-YOLO model (technical solution of the present invention) is compared with the baseline model YOLOv11 in the experiment using the same equipment, data set and experimental parameters. The performance indicators include mean average precision (mAP), inference speed (FPS) and parameter quantity (Params).

[0104] The mean average precision (mAP) is used to measure the accuracy and comprehensiveness of detection of different categories. The formula is as follows:

[0105] ;

[0106] ;

[0107] Among them, AP represents the accuracy of a single category, p(r) represents the precision p corresponding to a specific recall rate r, mAP is the result of integrating the area under the PR curve, and represents the mean accuracy of all categories.i represents the precision of the i-th category, and n is the number of categories.

[0108] FPS represents the inference speed of the mode, where t1 represents the image preprocessing time, t2 represents the image inference time, and t3 represents the image postprocessing time.

[0109] ;

[0110] Parameters refer to the parameters that need to be learned during the network model training process, which represents the spatial complexity of the model. The larger the parameters, the more memory the model occupies.

[0111] The experimental results are shown in Table 1. Compared with the baseline model YOLOv11, the PPY-YOLO model of the technical solution of the present invention can slightly increase the FPS while keeping the number of parameters slightly reduced, and the detection accuracy is improved by 2.5%. It not only reduces the number of model parameters, but also ensures that the model can meet the real-time speed requirements of steel surface defect detection, and the accuracy is improved. It can be seen that this implementation is superior in accuracy and speed.

[0112] Table 1 Algorithm comparison experimental results

[0113]

[0114] Example 2

[0115] A steel surface defect detection system based on PPY-YOLO, comprising:

[0116] Preprocessing module: preprocess the acquired steel surface defect images;

[0117] Model building module: The PPY-YOLO target detection model is obtained by introducing the neck structure Faster-Neck, the deep learning module C3K2_Faster, and the spatial pyramid pooling module SPPFSim into the YOLOv11 network;

[0118] The PPY-YOLO target detection model includes a backbone network, a neck network and a detection head connected in series, wherein the input end of the C2PSA module of the backbone network is connected to the output end of the SPPFSim module, and the neck structure Faster-Neck adds a group of upsampling groups and a group of downsampling groups on the basis of the existing neck network;

[0119] Training module: The training set obtained by preprocessing the preprocessing module is input into the PPY-YOLO target detection model for training, and the validation set is input during the training process. The training effect is verified by the test set, thereby obtaining a trained PPY-YOLO target detection model;

[0120] Detection module: In the process of industrial steel production, cameras are used to obtain steel images, and the trained PPY-YOLO target detection model is input to obtain the category and location information of steel surface defects and the confidence of the results.

[0121] The backbone network is composed of two Conv modules connected in series, and then connected in series with a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a SPPFSim module and a C2PSA module;

[0122] The Conv module includes a standard two-dimensional convolutional layer, a BatchNormalization layer, and an activation function SILU layer connected in series.

[0123] The C3K2 module includes a Conv module and a C3K module, and the C2PSA module includes a Conv module and a PSA module connected in series.

[0124] The neck structure Faster-Neck has a newly added upsampling group in the existing neck network, including an upsampling module, a feature fusion module, and a deep learning module C3K2_Faster connected in series, and a newly added downsampling group includes a Conv module, a deep learning module C3K2_Faster, and a feature fusion module connected in series, and the C3K2 module in the existing neck structure is replaced by a C3K2_Faster module;

[0125] The deep learning module C3K2_Faster includes a Conv module, a Split operation layer, several FasterBlock modules or C3K Faster modules, a Concat operation layer and a Conv module connected in series in sequence, and the output ends of the first Conv module, the Split operation layer and several FasterBlock modules or C3K Faster modules are all connected to the input end of the Concat operation layer.

[0126] The spatial pyramid pooling module SPPFSim module integrates the SPPF module with the SimAM attention mechanism, and includes two standard convolutional layers and three maximum pooling layers, a SimAM attention mechanism and a Concat layer, that is, the input feature map is subjected to the attention mechanism of the SimAM module to extract feature information, and the feature information is divided into three, which are processed by three maximum pooling layers respectively, and the outputs of the three pooling layers are spliced ​​to finally obtain the final feature map.

[0127] It should be understood that the specific implementation process of each module please refer to the above method content, the present invention will not be repeated here, and the division of the above functional modules is only for example illustration. In some embodiments, some functional modules can be merged, and some functional modules can be split. Each functional module can be implemented in software or hardware or a combination of software and hardware. Among them, the hardware and software equipment includes but is not limited to general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0128] Example 3

[0129] A computer-readable storage medium stores a computer program, wherein the computer program is called by a processor to implement:

[0130] The above steps of a steel surface defect detection method based on PPY-YOLO.

[0131] For the specific implementation process of each step, please refer to the description of the above method.

[0132] It should be understood that in the embodiments of the present invention, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the type of device.

[0133] Embodiment 4:

[0134] The present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is called by a processor to implement: steps of a one-dimensional magnetotelluric anisotropy inversion method based on Fourier series.

[0135] For the specific implementation process of each step, please refer to the description of the above method.

[0136] The readable storage medium is a computer-readable storage medium, which may be an internal storage unit of the software and hardware device described in any of the aforementioned embodiments, such as a hard disk or memory of a controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the controller. Furthermore, the readable storage medium may also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0137] Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0138] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is a device for realizing the function specified in one flow or multiple flows of the flow chart and / or one or multiple blocks of the block diagram with reference to the instructions executed by the processor according to the method, device (system) and computer program product of the embodiment of the present application. These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which realizes the function specified in one flow or multiple flows of the flow chart and / or one or multiple blocks of the block diagram. These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0139] It should be emphasized that the examples described in the present invention are illustrative rather than restrictive, and therefore the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solution of the present invention that do not depart from the purpose and scope of the present invention, whether modified or replaced, also fall within the scope of protection of the present invention.

Claims

1. A steel surface defect detection method based on PPY-YOLO, characterized in that: include: Step 1: Preprocess the acquired steel surface defect image; Step 2: Obtain the PPY-YOLO target detection model by introducing the neck structure Faster-Neck, the deep learning module C3K2_Faster, and the spatial pyramid pooling module SPPFSim into the YOLOv11 network; The PPY-YOLO target detection model includes a backbone network, a neck network and a detection head connected in series, wherein the input end of the C2PSA module of the backbone network is connected to the output end of the SPPFSim module, and the neck structure Faster-Neck adds a group of upsampling groups and a group of downsampling groups on the basis of the existing neck network; Step 3: Input the training set obtained by preprocessing in step 1 into the PPY-YOLO target detection model for training, and input the validation set during the training process, and verify the training effect through the test set to obtain the trained PPY-YOLO target detection model; Step 4: Use a camera to obtain steel images during the industrial steel production process, input the trained PPY-YOLO target detection model, and obtain the category and location information of steel surface defects and the confidence of the results; In the neck structure Faster-Neck, the newly added upsampling group on the existing neck network includes an upsampling module, a feature fusion module, and a deep learning module C3K2_Faster connected in series in sequence, and the newly added downsampling group includes a Conv module, a deep learning module C3K2_Faster, and a feature fusion module connected in series in sequence, and the C3K2 module in the existing neck network is replaced by the deep learning module C3K2_Faster; The deep learning module C3K2_Faster includes a Conv module, a Split operation layer, several FasterBlock modules or C3K_Faster modules, a Concat operation layer and a Conv module connected in series in sequence, and the output ends of the first Conv module, the Split operation layer and several FasterBlock modules or C3K_Faster modules are all connected to the input end of the Concat operation layer.

2. The method according to claim 1, characterized in that The backbone network is composed of two Conv modules connected in series, and then connected in series with a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a SPPFSim module and a C2PSA module; The Conv module includes a standard two-dimensional convolutional layer, a BatchNormalization layer, and an activation function SILU layer connected in series. The C3K2 module includes a Conv module and a C3K module, and the C2PSA module includes a Conv module and a PSA module connected in series.

3. The method according to claim 1, characterized in that The spatial pyramid pooling module SPPFSim module integrates the SPPF module with the SimAM attention mechanism, and includes two standard convolutional layers and three maximum pooling layers, a SimAM attention mechanism and a Concat layer, that is, the input feature map is subjected to the attention mechanism of the SimAM module to extract feature information, and the feature information is divided into three, which are processed by three maximum pooling layers respectively, and the outputs of the three pooling layers are spliced ​​to finally obtain the final feature map.

4. The method according to claim 1, characterized in that: The loss functions used in training the PPY-YOLO target detection model include classification loss function and positioning loss function. The classification loss function uses cross entropy loss, while the positioning loss function uses CIOU and distributed focus DFL loss functions. The overall loss function calculation formula is as follows: ； Among them, L cls represents the classification cross entropy loss function, L reg To locate the CloU loss function, L dfl is the distributed focus DFL loss function, α is the balance coefficient of classification loss, The balance coefficient of positioning loss, γ is the balance coefficient of positioning distributed focus loss, is the number of cells assigned as positive samples.

5. A steel surface defect detection system based on PPY-YOLO, characterized in that: include: Preprocessing module: preprocess the acquired steel surface defect images; Model building module: The PPY-YOLO target detection model is obtained by introducing the neck structure Faster-Neck, the deep learning module C3K2_Faster, and the spatial pyramid pooling module SPPFSim into the YOLOv11 network; The PPY-YOLO target detection model includes a backbone network, a neck network and a detection head connected in series, wherein the input end of the C2PSA module of the backbone network is connected to the output end of the SPPFSim module, and the neck structure Faster-Neck adds a group of upsampling groups and a group of downsampling groups to the upsampling module and downsampling module of the existing neck network respectively; Training module: The training set obtained by preprocessing the preprocessing module is input into the PPY-YOLO target detection model for training, and the validation set is input during the training process. The training effect is verified by the test set, thereby obtaining a trained PPY-YOLO target detection model; Detection module: In the process of industrial steel production, cameras are used to obtain steel images, and the trained PPY-YOLO target detection model is input to obtain the category and location information of steel surface defects and the confidence of the results; In the neck structure Faster-Neck, the newly added upsampling group on the existing neck network includes an upsampling module, a feature fusion module, and a deep learning module C3K2_Faster connected in series in sequence, and the newly added downsampling group includes a Conv module, a deep learning module C3K2_Faster, and a feature fusion module connected in series in sequence, and the C3K2 module in the existing neck network is replaced by the deep learning module C3K2_Faster; The deep learning module C3K2_Faster includes a Conv module, a Split operation layer, several FasterBlock modules or C3K Faster modules, a Concat operation layer and a Conv module connected in series in sequence, and the output ends of the first Conv module, the Split operation layer and several FasterBlock modules or C3K Faster modules are all connected to the input end of the Concat operation layer.

6. The system according to claim 5, characterized in that The backbone network is composed of two Conv modules connected in series, and then connected in series with a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a Conv module, a C3K2 module, a SPPFSim module and a C2PSA module; The Conv module includes a standard two-dimensional convolutional layer, a BatchNormalization layer, and an activation function SILU layer connected in series. The C3K2 module includes a Conv module and a C3K module, and the C2PSA module includes a Conv module and a PSA module connected in series.

7. The system according to claim 5, characterized in that The spatial pyramid pooling module SPPFSim module integrates the SPPF module with the SimAM attention mechanism, and includes two standard convolutional layers and three maximum pooling layers, a SimAM attention mechanism and a Concat layer, that is, the input feature map is subjected to the attention mechanism of the SimAM module to extract feature information, and the feature information is divided into three, which are processed by three maximum pooling layers respectively, and the outputs of the three pooling layers are spliced ​​to finally obtain the final feature map.

8. A computer-readable storage medium, characterized in that: A computer program is stored, which is called by a processor to implement: The steps of the method according to any one of claims 1 to 4.

Images