Industrial pure iron metallographic specimen defect detection optimization method based on YOLOv8obb

Abstract

The invention relates to the technical field of material detection, in particular to an optimization method for industrial pure iron metallographic specimen defect detection based on YOLOv8obb. The method comprises the following steps: acquiring an industrial pure iron metallographic sample defect image; preprocessing the metallographic image; constructing a YOLOv8obb defect detection network, and optimizing the defect detection network to obtain an optimized defect detection model; according to the model, a DynamicConv is used as a convolutional layer of Bottleneck in a C2f module, a SimAM attention mechanism module is introduced into a backbone network, a tiny target detection head is added, and an NEFPN feature pyramid network is used as a neck multi-scale feature fusion network; and the preprocessed sample data image pair is used for performing defect detection model training, and the trained optimized defect detection model is used for performing industrial pure iron metallographic image defect detection, so that the detection precision and the detection efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of material detection, and in particular to an industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb. Background Art

[0002] Metallographic analysis of industrial pure iron is an important method to study the relationship between material microstructure and related properties, which is crucial for understanding material properties and controlling product quality. However, during the preparation of metallographic samples, defects (such as scratches, black spots, stains, etc.) caused by improper operation will seriously affect the accurate analysis of the material microstructure. The metallographic analysis method that relies on manual observation has problems such as subjective judgment, high labor intensity and low accuracy, and it is difficult to adapt to the detection requirements of modern pure iron production enterprises. At the same time, due to the lack of sufficient labeled data and poor model generalization methods, the application of deep learning-based detection methods in metallographic analysis has not been fully explored to a large extent. Therefore, it is necessary to study an intelligent, real-time and high-precision visual detection method for defects in industrial pure iron metallographic samples to overcome the above-mentioned problems in the existing industrial pure iron metallographic analysis. Summary of the invention

[0003] In order to achieve the purpose of intelligent, real-time and high-precision visual detection of industrial pure iron metallographic sample defects, the present invention provides an industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb, which mainly includes:

[0004] S1: Collect defect images of industrial pure iron metallographic samples;

[0005] S2: preprocessing the metallographic images collected by S1;

[0006] S3: Build the YOLOv8_obb defect detection network, optimize the defect detection network, and obtain the optimized defect detection model. The model uses DynamicConv as the convolution layer of Bottleneck in the C2f module, introduces the SimAM attention mechanism module in the backbone network, adds a small target detection head, and uses the NEFPN feature pyramid network as the neck multi-scale feature fusion network;

[0007] S4: Use the preprocessed sample data images to train the defect detection model, and use the trained defect detection model to perform industrial pure iron metallographic image defect detection.

[0008] A storage device stores instructions and data for implementing an industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb.

[0009] An industrial pure iron metallographic sample defect detection optimization device based on YOLOv8_obb comprises: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement an industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb.

[0010] The technical solution provided by the present invention has the following beneficial effects: compared with the prior art, the present invention optimizes the YOLOv8_obb model framework to perform defect detection and classification tasks. In the process of industrial pure iron detection, the image library of the industrial pure iron metallographic sample image signal is expanded by preprocessing, and DynamicConv is used as the convolution layer of Bottleneck in the C2f module to enhance the feature extraction capability; the SimAM attention mechanism module is introduced into the backbone network to enhance the ability to capture features of different dimensions; a small target detection head is added to detect by splicing low-level feature maps and high-level feature maps; the NEFPN feature pyramid network is used as the neck multi-scale feature fusion network to improve the object detection performance. The optimized model is used to extract the multi-scale features of the defects of the industrial pure iron metallographic sample image and fuse them, and the fused multi-scale feature map is further used to complete the defect detection and classification task of the industrial pure iron metallographic sample. The average accuracy of the algorithm using this method is 82.9%, and the detection time of a single frame image is 7.2ms. Compared with the FasterRCNN, RetinaNet, R3Det, ​​OrientedRCNN, and YOLOv8s_obb network models, the detection accuracy is improved by 53.6%, 52.6%, 51%, 53.6%, and 36.1%, respectively, and the detection speed is improved by 47.4ms, 47.5ms, 54.9ms, and 49.5ms, respectively. The optimized defect detection network is 0.7ms slower than the unoptimized model, but due to the significant improvement in detection accuracy, the slight reduction in detection speed is within an acceptable range. The network structure can fully analyze the fused multi-scale features and has a relatively simple structure. The model has the advantages of short training time, fast computing speed, high detection accuracy, and low spatial complexity, indicating that the algorithm can meet the requirements of industrial pure iron metallographic sample defect detection in actual production requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0012] Figure 1 It is a flow chart of an industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb in an embodiment of the present invention;

[0013] Figure 2 It is a schematic diagram of a sliding window slice preprocessing method in an industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb in an embodiment of the present invention;

[0014] Figure 3 is a network structure diagram of an improved defect detection model in an embodiment of the present invention;

[0015] Figure 4 is a schematic diagram of the DynamicConv module structure used in an embodiment of the present invention;

[0016] Figure 5 is a structural design diagram of four detection heads used in an embodiment of the present invention;

[0017] Figure 6 : is a diagram of the NEFPN neck multi-scale feature fusion network structure used in an embodiment of the present invention;

[0018] Figure 7 is a schematic diagram of a metallographic sample image used as input in an embodiment of the present invention;

[0019] Figure 8 It is a schematic diagram of the image defect detection result of the industrial pure iron metallographic sample output in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0021] The main application scenarios of the present invention are: when performing metallographic analysis of industrial pure iron, in the process of metallographic sample preparation, various defects will inevitably appear in the metallographic sample due to improper operation, and the defects will affect the observation of the microstructure of the material to a certain extent, thereby leading to incorrect analysis of the mechanical properties, physical properties, and processing technology of industrial pure iron. Therefore, in the metallographic analysis of industrial pure iron, it is necessary to abandon the interference of defects to achieve accurate analysis of the microstructure of industrial pure iron. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows: a defect detection optimization method for industrial pure iron metallographic samples based on YOLOv8_obb, the main work of which is the algorithm research and optimization of industrial pure iron metallographic sample defect detection and identification based on deep learning, and the preprocessing of sample data images, and finally improve the detection accuracy of two types of defects: black spots, stains, and scratches.

[0022] Example 1

[0023] Please refer to Figure 1 , Figure 1 It is a flow chart of an industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb in an embodiment of the present invention, which specifically includes:

[0024] S1: Collect defect images of industrial pure iron metallographic samples;

[0025] Pick The metal microstructure of the cylindrical industrial pure iron sample was obtained by rough grinding, fine grinding, polishing and corrosion. The Nikon TS2-S-SM microscope was used to collect images on the industrial pure iron sample, and the image resolution was processed by the corresponding TCapture software. A total of 504 metallographic images with a size of 1280×960px were collected under natural light and three magnifications (100×, 200×, 500×).

[0026] S2: Preprocess the metallographic images collected in S1, including data expansion methods such as horizontal flipping, vertical flipping, grayscale, and color enhancement, as well as a preprocessing method using a sliding window slicing method to obtain a size that meets the model input size without losing feature detail information. The specific implementation steps include:

[0027] S2.1: Use roLabelImg software to label the metallographic image with a rotating target frame in Pascal VOC format. The completed label is saved as a text file, with one label file for each image. These label files contain the defect category information in the image, the location and size of the target frame.

[0028] S2.2: Four data augmentation methods are used for the sample data: horizontal flip, vertical flip, grayscale, and color enhancement. The original images are expanded to 2520. The category and number of metallographic sample defects in each image are random, but each image has at least one defect object. This image dataset includes two types of defect objects, namely black spots and scratches.

[0029] S2.3: Use the sliding window slicing method to slice the image data, such as Figure 2 shown.

[0030] The detailed steps include:

[0031] S2.3.1: Based on the size of the original image and the required standard image size, define a sliding window of 800×800px and set a non-blank threshold to determine whether the image in the sliding window is a blank area. If the pixel value in the sliding window is lower than the non-blank threshold, the window is considered to be a blank area and no processing is performed.

[0032] S2.3.2: Get the image path and label file path from the input, and use the corresponding library function to traverse each image file in the original dataset. Find the corresponding label file according to the image file name or index, ensure that the label file exists and matches the image file, and then process the single image.

[0033] S2.3.3: Read the original image file and the corresponding label file, and extract the required defect information from the label file, such as the location, type, and size of the defect. In this embodiment, the window is moved from the upper left corner along the row and column direction to the lower right for slicing. According to the non-blank threshold principle in S2.3.1, the pixel values ​​of the slice sub-image and the sliding window image are compared to obtain a valid slice sub-image.

[0034] S2.3.4: For each slice sub-image, first determine the size and position of the sub-image in the original image. Set up a function to traverse all target defect boxes in the original image and check whether they are in the image area of ​​the current slice or intersect with it (coordinate point value comparison). Then use the area of ​​the old and new target defect boxes and their intersection area to calculate the intersection and union ratio, and compare the intersection and union ratio calculation result with the set threshold to determine whether the target defect exists in the extracted image area. If so, extract the category, position and size data information of the target defect in the original image label, and then calculate the new normalized coordinates of the target defect in the slice sub-image. Fill the new normalized coordinates into the new label file corresponding to the slice sub-image. The new label file should contain the category, name and normalized coordinate data information. If the target defect box is incomplete, remove the data information from the new label file.

[0035] S2.3.5: Traverse all images in the original dataset, save all valid slice sub-images and corresponding new label files to the specified directory, and print processing statistics.

[0036] S2.4: Obtain sample data images through S2.1-S2.3, and divide the preprocessed sample data images into training set, validation set and test set in a ratio of 8:1:1. The training set is used to train the defect detection model parameters, the validation set is used to adjust the model's hyperparameters and to perform a preliminary evaluation of the model's capabilities, and the test set is used to evaluate the generalization ability of the final model.

[0037] S3: Build the YOLOv8_obb defect detection network, optimize the defect detection network in a targeted manner, and obtain the optimized defect detection model; including:

[0038] An industrial pure iron metallographic sample defect detection model is constructed based on the YOLOv8_obb algorithm, and the industrial pure iron metallographic sample defect detection model includes a backbone feature extraction network, a neck multi-scale feature fusion network and a head detection network. The optimized YOLOv8_obb network model structure provided by the present invention is as follows Figure 3 shown.

[0039] S3.1: DynamicConv is used to replace the ordinary convolution of Bottleneck in the C2f module of the backbone feature extraction network for feature extraction.

[0040] The network structure of the original YOLOv8_obb model includes a C2f module, which contains a 1x1 Conv layer, n Bottleneck layers, and a connected Conv layer. Each Bottleneck block contains two convolutional layers. This standard convolution usually only focuses on the local area and cannot effectively capture small or detailed defect features. It also consumes a lot of computing resources. Therefore, in order to adapt to features of different scales and enhance the model's ability to capture defect detail features, it is decided to use DynamicConv to improve the module. Figure 4 This is the DynamicConv structure diagram.

[0041] DynamicConv performs a global average pooling operation on the input feature map x, aggregating the features of each channel into a single value, thereby obtaining a global spatial feature vector. After that, the output of the global average pooling is passed through two fully connected layers. The first FC layer maps the global features to an intermediate dimension, and the second FC layer maps this intermediate dimension to K dimensions, which is the final number of attention weights.

[0042]

[0043] Among them, y represents the number of output attention weights, x represents the input feature map, and g represents the activation function. represents the weight parameter of the kth convolution kernel, represents the bias parameter of the kth convolution kernel, K represents the total number of different convolution kernels involved in the convolution process, k represents the index of the convolution kernel currently being processed, k = 1, 2, 3, ..., K, K represents the number of convolution kernel indices;

[0044] Then apply the Softmax function to the output of the second FC layer to normalize all values ​​into the form of probability distribution, ensuring that all π k The (x) values ​​are all between 0 and 1, and their sum is 1.

[0045] st0≤π k (x)≤1 (4)

[0046]

[0047] Among them, π k (x) is the weight of the corresponding convolution kernel, W, b, and g represent weight, bias, and activation function respectively; the formula for summing K convolution kernels is shown in equations (1)-(5). After obtaining these attention weights, they are used to weight the corresponding convolution kernels. Finally, the model dynamically adjusts the convolution kernel according to the input feature map x, better adapts to different features, and processes complex data more flexibly and effectively.

[0048] The improved Bottleneck block contains two DynamicConv convolutional layers and residual connections. The first 1×1 convolutional layer reduces the number of input channels c1 to an intermediate number of channels c_ (usually a proportion of the output channel number c2), and the second 3×3 convolutional layer restores the intermediate number of channels c_ to the output channel number c2, and group convolution can be applied according to the configuration. If the number of input channels c1 is the same as the number of output channels c2, and shortcut=True, the residual connection is enabled, that is, the input features are directly added to the output features to form a jump connection. If shortcut=False, only the convolved feature map is returned. By using the improved Bottleneck block, the model is able to calculate its own unique convolution kernel parameters and adaptively fuse these parameters for more efficient feature extraction and representation.

[0049] S3.2: Introduce the SimAM attention mechanism module into the backbone network.

[0050] Since the targets detected by the present invention are all defects of industrial pure iron metallographic samples, most of these defects are small in size and widely distributed, and the current detection algorithm is still insufficient in detecting small targets. However, YOLOv8_obb focuses on global features, which not only consumes a lot of computing resources but also has insufficient defect feature extraction capabilities of the model. Therefore, the present invention introduces the SimAM attention mechanism module into the 9th layer of the backbone network.

[0051] Most existing attention modules can only redefine features along the channel or spatial dimension, which limits their flexibility in learning attention weights that vary in channels and space. Unlike existing attention modules, SimAM, also known as a simple parameter-free attention mechanism, is inspired by the activity of neurons in the human brain and innovatively proposes the phenomenon of spatial inhibition: the most informative features in the image show unique expression patterns than the surrounding feature information, and may also inhibit the expression of the surrounding feature information, presenting an obvious spatial inhibition phenomenon. Therefore, feature information with spatial inhibition should be given a higher priority. To measure the linear separability between a feature information with spatial inhibition and other feature information, each feature information is defined as the following energy function:

[0052]

[0053] Among them, t is the most informative feature and surrounding features in a single channel of the input feature X, x i is the surrounding features of the most informative feature in a single channel of the input feature X, i is the index on the spatial dimension, M is the number of all feature information on this channel, and w t and b tis the weight and bias transformation, λ is used to control the weight w t The lower the energy, the greater the difference between the feature information with spatial suppression phenomenon and other feature information, that is, the more linearly separable the two feature information is, the higher the importance is. The features are then processed according to the following function, where E takes all e t * Grouping across channels and spatial dimensions.

[0054]

[0055] Among them, E is used to convert all e t * Grouping across channels and spatial dimensions; e t * represents the minimum energy difference between the target neuron and other neurons in each channel, X represents the input feature, Represents the features after function processing.

[0056] Through the above calculations, SimAM infers the 3D attention weights of the feature map and optimizes the energy function to find the importance of each neuron, thereby improving the model performance.

[0057] S3.3: Derive the P2 low-level feature map in the second layer of the backbone network and add a small object detection head.

[0058] The head detection network is responsible for further processing and detection and identification of the feature maps after the neck network is spliced. The head detection network first uses convolutional layers to extract features. These convolutional layers capture local features in the image through different filters, which are then used to classify and locate the target. After the convolutional layer, the head detection network uses maximum pooling or average pooling to reduce the dimension of features and compress spatial information while retaining key information, which helps to reduce the amount of computation and improve the abstraction ability of the network. Next, the fully connected layer expands the features by connecting all neurons, and applies weights and biases to generate the probability distribution of the target category and the location and size of the bounding box as the final output. Figure 5As shown in the figure, there are three feature maps of different scales (P3, P4, and P5) in the YOLOv8_obb network for detecting targets of different sizes. Among them, P3 is used to detect targets with a feature size of 8×8 or more, P4 is used to detect targets with a feature size of 16×16 or more, and P5 is used to detect targets with a feature size of 32×32 or more. This multi-scale feature map design allows the network to detect targets of different sizes at the same time. The second layer in the backbone network leads to the P2 low-level feature map. P2 is a newly added small target detection head, corresponding to a 160×160 detection feature map, which is used to detect targets larger than 4×4. This design expands the detection range of the network, enabling it to detect smaller targets and enhancing the network's ability to detect small targets.

[0059] S3.4: Adopt NEFPN as the neck network.

[0060] The PANet structure used in the neck network of YOLOv8_obb is an efficient multi-scale feature fusion network, which realizes the bidirectional fusion of feature maps by combining top-down and bottom-up paths. PANet first uses a convolutional neural network to extract high-level features from the input image, and then in the bottom-up path, the low-level feature map is fused with the high-level feature map through the feature pyramid network (FPN) structure to capture the details and context information in the image. This process ensures that the precise positioning information of the low-level feature map can be passed to higher layers, enhancing the positioning expression ability of the feature pyramid. In the top-down path, PANet combines the high-level semantic information with the low-level spatial information through upsampling, retaining the abstract information of the high-level while integrating the precise location information of the low-level. In addition, PANet also establishes lateral connections between feature maps of different scales to achieve information complementarity. Finally, PANet upsamples the fused feature map to the resolution of the original image and generates a classification probability for each pixel through the convolution layer. Although the PANet structure performs well in target detection tasks and its powerful multi-scale feature fusion capability improves the performance of the model, its high computing resource consumption and complex feature fusion process limit its application in defect detection of industrial pure iron metallographic samples. To this end, the present invention specifically uses NEFPN as the neck network. NEFPN is a network structure optimized on the basis of HSFPN. HSFPN itself is a network structure specifically used to enhance the model's ability to detect targets of different scales. It improves the accuracy and robustness of target detection through a multi-scale feature fusion strategy. The feature selection module in HSFPN uses the Coordinate attention (CA) attention mechanism to capture the global spatial information of the image to enhance features. CA first divides the input feature map into two directions, width and height, and performs global average pooling to obtain feature maps in the width and height directions respectively:

[0061]

[0062] in, represents the width feature map, represents the height feature map; W represents the width of the feature map, H represents the height of the feature map; x c (h,i) represents the element of the input feature map at channel c, height h and width i, x c (j,w) represents the element of the input feature map at channel c, height j, and width w.

[0063] The feature maps of the width and height of the global receptive field are concatenated together, and then sent to a convolution module with a shared convolution kernel of 1×1 to reduce its dimension to the original C / r. The batch normalized feature map F1 is then sent to the Sigmoid activation function to obtain a feature map f of the shape of 1×(W+H)×C / r. The formula is as follows:

[0064] f=δ(BN(F1([z h ,z w ]))) (10)

[0065] Among them, h, w, c represent the height, width and channel dimensions, F1 represents a 2D convolution, BN represents batch normalization, δ represents the Sigmoid nonlinear activation function, C represents the number of channels of the input feature map, and r represents the downsampling ratio of the control module size.

[0066] The feature map f is convolved with a kernel of 1×1 according to the original height and width to obtain the feature map F with the same number of channels as the original one. h and F w , after the Sigmoid activation function, the attention weights g of the feature maps in the height direction are obtained respectively h and the attention weight g in the width direction w :

[0067] g h =σ(F h (f h )) (11)

[0068] g w =σ(F w (f w )) (12)

[0069] Among them, f h represents the feature map obtained by segmentation along the height direction, f w Represents the feature map obtained by segmentation along the width direction.

[0070] By multiplying the original feature map with weights, we get the feature map y with attention weights in the width and height directions. c (i,j):

[0071]

[0072] Among them, y c (i,j) represents the final feature map, Represents the attention weight for the i-th height direction on channel c, Represents the attention weight for the jth height direction on channel c.

[0073] However, this mechanism encountered challenges when processing scratch defects in industrial pure iron metallographic images. The width of the scratch defect is very fine, the length span is large, and the position is uncertain. These characteristics make it difficult to accurately capture the local details of the defect by relying solely on global information. To solve this problem, the present invention uses the Efficient Local Attention (ELA) attention mechanism to replace the CA attention mechanism. A new NEFPN neck multi-scale feature fusion network is formed, such as Figure 6 As shown. ELA enhances the position information by applying one-dimensional convolution to the position information on the horizontal and vertical feature vectors of the spatial dimension, and then uses group normalization Gn to process the enhanced position information, so as to obtain the representation of position attention in the horizontal and vertical directions. The formula is as follows:

[0074] y h =σ(G n (F h (z h ))) (14)

[0075] y w =σ(G n (F w (z w ))) (15)

[0076] Among them, y h Represents the vertical position attention, y w represents the horizontal position attention, σ represents the Sigmoid activation function, F h Represents the feature map after the convolution operation, z h represents the feature representation obtained after global average pooling of the input feature map in the height direction, z w Represents the feature representation obtained after global average pooling of the input feature map in the width direction.

[0077] This design can maintain a narrow kernel shape to capture long-range dependencies while preventing irrelevant regions from interfering with label prediction results. ELA processes the feature vectors in each direction independently to obtain more accurate attention predictions, and then combines these attention predictions through a dot multiplication operation to achieve precise positioning of the region of interest. This method not only improves the recognition accuracy of scratch defects, but also reduces the number of model parameters and computational complexity, making it very suitable for industrial pure iron metallographic image defect detection tasks.

[0078] S4: Use the preprocessed sample data images for training, and use the trained optimized defect detection model to perform industrial pure iron metallographic image defect detection;

[0079] S4.1: After obtaining the optimized defect detection model, the model needs to be trained.

[0080] S4.2: First, in the process of training the optimized defect detection model, the pre-processed industrial pure iron metallographic sample defect sample data image is scaled to the set value and used as the input data of the model for training. The specific process is as follows: the input image size is set to 640×640px, the initial learning rate is 0.01, the training process epoch is 500, the batch size is 16, and the weight decay is 0.0005.

[0081] S4.3 During the model training process, various evaluation indicators are used to determine the performance of the optimized defect detection model. The performance evaluation indicators specifically include Precision, Recall, mAP (average precision), Params (parameter quantity), and GFLOPs (computational complexity). Precision indicates the proportion of samples predicted to be positive that are actually positive, Recall indicates the proportion of actual positive samples in the predicted positive samples to the positive samples in the total samples, mAP indicates the average value of the maximum precision under different recall values, Params indicates the total number of parameters that need to be trained in the network model, and GFLOPs indicates the number of billion floating-point operations per second involved in executing an algorithm or model. The evaluation indicator calculation formula is as follows:

[0082]

[0083] Among them, TP indicates that the true value of the data is a positive example, and the predicted value is also a positive example. TN indicates that the true value of the data is a negative example, and the predicted value is also a negative example. FP indicates that the true value of the data is a negative example, but it is mistakenly predicted as a positive example. FN indicates that the true value of the data is a positive example, but it is mistakenly predicted as a negative example. For a given category c and threshold, AP is the area under the PR curve for that category. When there are multiple categories N in the data setclasses When mAP is the average of all AP values ​​of all categories, the training results of the unoptimized defect detection model and the optimized defect detection model are compared. The results show that the optimized defect detection model shows better detection performance in the defect detection of industrial pure iron metallographic samples. The results are shown in Tables 1 and 2. Table 1 is an ablation experiment conducted to prove the effectiveness of the optimization algorithm, and Table 2 is a comparison of the detection performance of the optimized defect detection model and other detection models.

[0084] Table 1 Ablation experiment data

[0085]

[0086] Table 2 Comparison of different defect detection models

[0087]

[0088] S4.4: After the optimization of the defect detection model is completed, the image of the industrial pure iron metallographic sample to be detected (such as Figure 7 ) Input the trained optimized defect detection model to obtain the predicted defect image of industrial pure iron metallographic sample (such as Figure 8 ), where the defect detection image contains the type and location marks of each defect.

[0089] For the situation that the artificial microscopic detection method is too poor for the defect detection of industrial pure iron metallographic samples, the present invention provides an optimization method for defect detection of industrial pure iron metallographic samples based on YOLOv8_obb, obtains sample data, and divides the labeled sample data into a training set, a verification set, and a test set, and completes the preprocessing of the sample data. The preprocessing of the sample data includes: the image in the sample data set is a jpg image of 1280×960px, and the sample data image is labeled using roLabelImg software, and the label format is a txt label file. And the data is expanded and sliced ​​to obtain the training image, and the optimized defect detection network is trained using the preprocessed sample data; the use of the trained optimized defect detection network to perform defect detection on the industrial pure iron metallographic sample image includes: inputting the image to be detected into the trained optimized defect detection model, and outputting the industrial pure iron metallographic sample defect detection image. The defect detection image contains the type and position mark of each defect. The trained optimized defect detection network is used to perform defect detection on the industrial pure iron metallographic sample image. The effectiveness of the currently popular deep learning method in the defect detection task of industrial pure iron metallographic samples was verified.

[0090] Example 2

[0091] A YOLOv8_obb-based industrial pure iron metallographic sample defect detection optimization device comprises: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the YOLOv8_obb-based industrial pure iron metallographic sample defect detection optimization method.

[0092] Example 3

[0093] A storage device, wherein the storage device stores instructions and data for implementing the industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A defect detection optimization method for industrial pure iron metallographic samples based on YOLOv8_obb, characterized in that: include: S1: Collect defect images of industrial pure iron metallographic samples; S2: preprocessing the metallographic images collected by S1; S3: Build the YOLOv8_obb defect detection network, optimize the defect detection network, and obtain the optimized defect detection model. The model uses DynamicConv as the convolution layer of Bottleneck in the C2f module, introduces the SimAM attention mechanism module in the backbone network, adds a small target detection head, and uses the NEFPN feature pyramid network as the neck multi-scale feature fusion network; S4: Use the preprocessed sample data images to train the defect detection model, and use the trained defect detection model to perform industrial pure iron metallographic image defect detection.

2. The method for optimizing defect detection of industrial pure iron metallographic samples based on YOLOv8_obb according to claim 1, characterized in that: The preprocessing process in S2 is: S2.1: Use roLabelImg software to mark the rotating target frame of the metallographic image. The completed label is saved as a text file. Each image corresponds to a label file; these label files contain the defect category information in the image, the position and size of the target frame; S2.2: The metallographic images are augmented using four data augmentation methods: horizontal flipping, vertical flipping, grayscale, and color enhancement. The category and number of metallographic sample defects in each image are random, but each image has at least one defect object; this image dataset includes two types of defect objects, namely black spots and scratches; S2.3: Slice the image data using a sliding window slicing method; S2.4: Obtain sample data images through S2.1-S2.3, and divide the preprocessed sample data images into training set, validation set and test set according to a preset ratio. The training set is used to train the defect detection model parameters, the validation set is used to adjust the model's hyperparameters and to perform a preliminary evaluation of the model's capabilities, and the test set is used to evaluate the model's generalization ability.

3. The method for optimizing defect detection of industrial pure iron metallographic samples based on YOLOv8_obb as claimed in claim 2, characterized in that: The slicing process in S2.3 is as follows: S2.3.1: Define a preset sliding window based on the size of the original metallographic image and the required standard image size, and set a non-blank threshold to determine whether the image in the sliding window is a blank area. If the pixel value in the sliding window is lower than the non-blank threshold, the window is considered to be a blank area and no processing is performed; S2.3.2: Get the image path and label file path from the input, use the corresponding library function to traverse each image file in the original dataset, find the corresponding label file according to the image file name or index, ensure that the label file exists and matches the image file, and then process the single image; S2.3.3: Read the original metallographic image file and the corresponding label file, and extract the required defect information from the label file, including the location, type and size of the defect; start from the upper left corner and move the window toward the lower right along the row and column direction according to the set step size to perform slicing processing; compare the pixel values ​​of the slice sub-image and the sliding window image according to the non-blank threshold principle in S2.3.1 to obtain a valid slice sub-image; S2.3.4: For each slice sub-image, first determine the size and position of the sub-image in the original image; Set up a function to traverse all target defect frames in the original image and check whether they are in the image area of ​​the current slice or intersect with it. Then use the area of ​​the new and old target defect frames and their intersection area to calculate the intersection and union ratio, and compare the intersection and union ratio calculation result with the set threshold to determine whether the target defect exists in the extracted image area. If so, extract the category, position and size data information of the target defect in the original image label, and then calculate the new normalized coordinates of the target defect in the slice sub-image, and fill the new normalized coordinates into the new label file corresponding to the slice sub-image. The new label file should contain the category, name and normalized coordinate data information. If the target defect frame is incomplete, remove the data information from the new label file; S2.3.5: Traverse all images in the original dataset, save all valid slice sub-images and corresponding new label files to the specified directory, and print processing statistics.

4. The method for optimizing defect detection of industrial pure iron metallographic samples based on YOLOv8_obb according to claim 1, characterized in that: S3 is specifically: S3.1: Using DynamicConv as the convolutional layer of Bottleneck in the C2f module includes: improving the ordinary convolution of Bottleneck in the backbone network C2f module with DynamicConv; S3.2: Introducing the SimAM attention mechanism module in the backbone network includes: introducing the SimAM attention mechanism in the 9th layer of the backbone network; S3.3: Adding a small object detection head includes: extracting the P2 low-level feature map in the second layer of the backbone network and performing multi-scale feature fusion in the neck network; S3.4: Using the NEFPN feature pyramid network as the neck multi-scale feature fusion network includes: using the ELA attention mechanism to optimize the HSFPN network, the optimized HSFPN network is named NEFPN, and used as the neck network of the defect detection model, thereby splicing and fusing feature maps of different scales in the backbone network.

5. The method for optimizing defect detection of industrial pure iron metallographic samples based on YOLOv8_obb as claimed in claim 4, characterized in that: In S3.1, DynamicConv performs a global average pooling operation on the input feature map x, aggregating the features of each channel into a single value to obtain a global spatial feature vector; then, the output of the global average pooling passes through two fully connected layers FC. The first FC layer maps the global features to an intermediate dimension, and the second FC layer maps this intermediate dimension to K dimensions, which is the final number of attention weights: Among them, y represents the number of output attention weights, x represents the input feature map, and g represents the activation function. represents the weight parameter of the kth convolution kernel, represents the bias parameter of the kth convolution kernel, K represents the total number of different convolution kernels involved in the convolution process, k represents the index of the convolution kernel currently being processed, k = 1, 2, 3, ..., K, K represents the number of convolution kernel indices; Then apply the Softmax function to the output of the second FC layer to normalize all values ​​into the form of probability distribution, ensuring that all π k (x) values ​​are all between 0 and 1, and their sum is 1; s.t.0≤π k (x)≤1 (4) Among them, π k (x) is the weight of the corresponding convolution kernel; the formula for summing K convolution kernels is shown in equations (1)-(5). After obtaining these attention weights, they are used to weight the corresponding convolution kernels; finally, the model dynamically adjusts the convolution kernel according to the input feature map x, better adapts to different features, and processes complex data more flexibly and effectively; The improved Bottleneck block contains two DynamicConv convolutional layers and residual connections; the first 1×1 convolutional layer reduces the number of input channels c1 to an intermediate number of channels c_, and the second 3×3 convolutional layer restores the intermediate number of channels c_ to the output number of channels c2, and applies group convolution according to the configuration; if the number of input channels c1 is the same as the number of output channels c2, and shortcut=True, the residual connection will be enabled, that is, the input features are directly added to the output features to form a jump connection; if shortcut=False, only the convolved feature map is returned; by using the improved Bottleneck block, the model can calculate its own unique convolution kernel parameters and adaptively fuse these parameters, so as to more effectively extract and represent features.

6. The method for optimizing defect detection of industrial pure iron metallographic samples based on YOLOv8_obb as claimed in claim 5, characterized in that: In S3.2, when the SimAM attention mechanism module is introduced into the backbone network, in order to measure the linear separability between a feature information with spatial inhibition phenomenon and other feature information, each feature information is defined as the following energy function: Among them, t represents the most informative feature in a single channel of the input feature X, x i is the surrounding features of the most informative feature in a single channel of the input feature X, i is the index on the spatial dimension, M is the number of all feature information on this channel, and w t and b t is the weight and bias transformation, λ is used to control the weight w t The regularization coefficient of size; Features are processed according to the following function: Among them, E is used to convert all e t * Grouping across channels and spatial dimensions; e t * represents the minimum energy difference between the target neuron and other neurons in each channel, X represents the input feature, Represents the features after function processing; Through the above calculations, SimAM infers the 3D attention weights of the feature map and optimizes the energy function to find the importance of each neuron, thereby improving the model performance.

7. The method for optimizing defect detection of industrial pure iron metallographic samples based on YOLOv8_obb as claimed in claim 4, characterized in that: In S3.4, the feature selection module in HSFPN uses the CA attention mechanism to capture the global spatial information of the image to enhance the features; CA first divides the input feature map into two directions, width and height, and performs global average pooling to obtain feature maps in the width w and height h directions respectively: in, represents the width feature map, represents the height feature map; W represents the width of the feature map, H represents the height of the feature map; x c (h,i) represents the element of the input feature map at channel c, height h and width i, x c (j,w) represents the element of the input feature map at channel c, height j and width w; The feature maps of the width and height of the global receptive field are concatenated together, and then sent to a convolution module with a shared convolution kernel of 1×1 to reduce its dimension to the original C / r. The batch normalized feature map F1 is then sent to the Sigmoid activation function to obtain the feature map f. The formula is as follows: Among them, h, w, c represent the height, width and channel dimensions, F1 represents a 2D convolution, BN represents batch normalization, δ represents the Sigmoid nonlinear activation function; C represents the number of channels of the input feature map, and r represents the downsampling ratio of the control module size; The feature map f is convolved with a kernel of 1×1 according to the original height and width to obtain the feature map F with the same number of channels as the original one. h and F w , after the Sigmoid activation function, the attention weights g of the feature maps in the height direction are obtained respectively h and the attention weight g in the width direction w : g h =σ(F h (f h )) (11) g w =σ(F w (f w )) (12) Among them, f h represents the feature map obtained by segmentation along the height direction, f w Represents the feature map obtained by segmentation along the width direction; By multiplying the original feature map with weights, we get the feature map y with attention weights in the width and height directions. c (i, j): Among them, y c (i, j) represents the final feature map, Represents the attention weight for the i-th height direction on channel c, Represents the attention weight for the jth height direction on channel c; The ELA attention mechanism is used to replace the CA attention mechanism to form a new NEFPN neck multi-scale feature fusion network. ELA enhances the information F by applying one-dimensional convolution to the position information on the horizontal and vertical feature vectors of the spatial dimension. h , and then use group normalization Gn to process the enhanced position information to obtain the representation of position attention in the horizontal and vertical directions. The formula is as follows: y h =σ(G n (F h (z h ))) (14) y w =σ(G n (F w (z w ))) (15) Among them, y h Represents the vertical position attention, y w represents the horizontal position attention, σ represents the Sigmoid activation function, F h Represents the feature map after the convolution operation, z h represents the feature representation obtained after global average pooling of the input feature map in the height direction, z w Represents the feature representation obtained after global average pooling of the input feature map in the width direction.

8. The method for optimizing defect detection of industrial pure iron metallographic samples based on YOLOv8_obb as claimed in claim 4, characterized in that: S4 is specifically: S4.1: After obtaining the optimized defect detection model, training the model; S4.2: In the process of training the defect detection model, the pre-processed industrial pure iron metallographic sample defect sample data image is scaled to a set value and used as the input data of the model for training; S4.3: During the model training process, various evaluation indicators are used to determine the performance of the optimized defect detection model. The performance evaluation indicators include Precision, Recall, mAP, Params, and GFLOPs. Precision indicates the proportion of samples predicted to be positive that are actually positive. Recall indicates the proportion of samples predicted to be positive that are actually positive to all positive samples. mAP indicates the average of the maximum precision under different recall values. Params indicates the total number of parameters that need to be trained in the network model. GFLOPs indicates the number of billion floating-point operations per second involved in executing an algorithm or model.

9. A storage device, characterized in that: The storage device stores instructions and data for implementing the industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb as described in any one of claims 1 to 8.

10. An industrial pure iron metallographic sample defect detection optimization device based on YOLOv8_obb, characterized in that: include: A processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the industrial pure iron metallographic sample defect detection optimization method based on YOLOv8_obb as described in any one of claims 1 to 8.

Images