Steel inner surface defect detection method and device

Abstract

The invention relates to the technical field of steel production, in particular to a steel inner surface defect detection method and device, and the method comprises the steps: constructing a steel inner surface detection model, a backbone network comprising a YOLOv8 model, a neck network constructed based on a cross fusion module, and a detection head of the YOLOv8 model; and the cross fusion module adopts a self-attention fusion strategy to carry out integration processing on the two parts of feature fusion and sampling, and adjacent features are fused in a targeted manner in the up-sampling process so as to completely obtain the whole area of the steel inner surface defect. The accuracy of steel inner surface defect detection can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of steel production, and in particular to a method and device for detecting inner surface defects of steel. Background Art

[0002] As a vital material in modern industry and construction, steel faces varying demands for exceptional strength, durability, and versatility in different situations. The overall quality of steel depends on factors such as mechanical properties, shape accuracy, and surface quality. During the production process, limitations of the equipment used at different stages can lead to internal surface defects such as scratches, pits, patches, and corrosion on steel products. These defects can negatively impact the material's overall performance, shorten its service life, and pose significant safety risks. Therefore, accurately detecting and assessing internal surface defects in steel is a critical step in maintaining product quality and ensuring safe use.

[0003] In recent years, the development of advanced imaging technology and artificial intelligence algorithms has made significant breakthroughs in steel surface defect detection methods. In particular, models such as CNN, YOLO series, and ResNet can automatically extract rich feature information from images, effectively overcoming the low accuracy and time-consuming problems of traditional detection methods.

[0004] However, defects on the inner surface of steel are usually complex, diverse, and different. Although existing detection models can extract deeper semantic features, they only use simple cascade or addition methods in the fusion stage of semantic features and shallow details. Semantic information and detail features cannot be effectively integrated, which easily leads to insufficient feature fusion and reduces the detection accuracy of the inner surface of steel. Summary of the Invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the problem that the existing detection model uses simple cascade or addition methods to fuse features, which easily leads to insufficient feature fusion and reduces the detection accuracy of the inner surface of steel.

[0006] To solve the above technical problems, the present invention provides a method for detecting inner surface defects of steel, comprising:

[0007] Input the steel inner surface image to be detected into the steel inner surface detection model, and obtain the first scale feature P1, the second scale feature P2, the third scale feature P3 and the fourth scale feature P′4 through the backbone network;

[0008] Input the fourth scale feature P′4 and the third scale feature P3 into the third cross fusion module to obtain the third fused feature P′3; input the third fused feature P′3 and the second scale feature P2 into the second cross fusion module to obtain the second fused feature P′2; input the second fused feature P′2 and the first scale feature P1 into the first cross fusion module to obtain the first fused feature P′1;

[0009] In each cross-fusion module, the deep feature P′ i Through upsampling operation, its size is similar to the shallow feature P i-1 Consistent, we get the upsampled deep feature P″ i ; The deep features P″ after the above sampling i Generate query sequence, with shallow feature P i-1 Generate key sequence and value sequence; calculate the similarity between query sequence and key sequence; multiply the similarity between query sequence and key sequence with value sequence and pass Sigmoid activation function to obtain fusion gating matrix; weight the fusion gating matrix to shallow feature P i-1 , get the fusion feature P′ i-1 ;

[0010] The first fusion feature P′1, the second fusion feature P′2 and the third fusion feature P′3 are input into the first detection head, the second detection head and the third detection head respectively to obtain the detection results of the inner surface defects of the steel.

[0011] Preferably, the backbone network and detection head of the steel inner surface detection model both adopt the backbone network and detection head of the YOLOv8 model.

[0012] Preferably, the deep feature P″ after the above sampling i Generate query sequence, with shallow feature P i-1 Generate key sequence and value sequence, the formula includes:

[0013] Q=P″ i W Q ,K=P i-1 W K ,V=P i-1 W V

[0014] Among them, Q represents the query sequence, K represents the key sequence, V represents the value sequence; W Q 、W K and W V are the linear projections corresponding to Q, K and V respectively.

[0015] Preferably, the similarity between the query sequence and the key sequence is calculated using the formula:

[0016] C′=σ(Q×K)

[0017] Where Q represents the query sequence, K represents the key sequence, σ represents the Sigmoid activation function, and C′ represents the similarity between the query sequence and the key sequence.

[0018] Preferably, the similarity between the query sequence and the key sequence is multiplied with the value sequence and then passed through a Sigmoid activation function to obtain a fusion gating matrix, which is:

[0019] C″=σ(C′×V

[0020] Where C″ represents the fusion gating matrix, σ represents the Sigmoid activation function, C′ represents the similarity between the query sequence and the key sequence, and V represents the value sequence.

[0021] Preferably, the fusion gating matrix is weighted to the shallow feature P i-1 , get the fusion feature P′ i-1 , the formula is:

[0022] P′ i-1 =C″⊙P i-1

[0023] Where C″ represents the fusion gating matrix and ⊙ represents the pixel-level dot product.

[0024] Preferably, when training the steel inner surface detection model, a CIoU loss function is used as the regression loss;

[0025] The formula of CIoU loss function is expressed as:

[0026]

[0027] Among them, L CIoU represents the CIoU loss function, IoU represents the intersection-over-union ratio between the predicted box and the true box, d represents the distance between the center point of the predicted box and the true box, c represents the diagonal distance between the minimum circumscribed rectangle of the predicted box and the true box, α represents the balance parameter, and v represents the correction factor.

[0028] Preferably, the calculation formula of the correction factor is:

[0029]

[0030] Where v represents the correction factor, w G and h G Represents the width and height of the real box, w p and h p Represent the width and height of the prediction box respectively.

[0031] Preferably, the calculation formula of the balance parameter is:

[0032]

[0033] Among them, α represents the balance parameter, IoU represents the intersection over union ratio of the predicted box to the true box, and v represents the correction factor.

[0034] The present invention also provides a device for detecting defects on the inner surface of steel, comprising:

[0035] The feature extraction module is used to input the steel inner surface image to be detected into the steel inner surface detection model, and obtain the first scale feature P1, the second scale feature P2, the third scale feature P3 and the fourth scale feature P′4 through the backbone network;

[0036] A feature fusion module is configured to input the fourth scale feature P′4 and the third scale feature P3 into a third cross fusion module to obtain a third fused feature P′3; input the third fused feature P′3 and the second scale feature P2 into a second cross fusion module to obtain a second fused feature P′2; and input the second fused feature P′2 and the first scale feature P1 into the first cross fusion module to obtain a first fused feature P′1;

[0037] In each cross-fusion module, the deep feature P′ i Through upsampling operation, its size is similar to the shallow feature P i-1 Consistent, we get the upsampled deep feature P″ i ; The deep features P″ after the above sampling i Generate query sequence, with shallow feature P i-1 Generate key sequence and value sequence; calculate the similarity between query sequence and key sequence; multiply the similarity between query sequence and key sequence with value sequence and pass Sigmoid activation function to obtain fusion gating matrix; weight the fusion gating matrix to shallow feature P i-1 , get the fusion feature P′ i-1 ;

[0038] The detection module is used to input the first fusion feature P′1, the second fusion feature P′2 and the third fusion feature P′3 into the first detection head, the second detection head and the third detection head respectively to obtain the detection results of the inner surface defects of the steel.

[0039] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0040] The present invention discloses a method for detecting inner surface defects of steel. Based on the YOLOv8 model, a neck network is constructed using a cross-fusion module to fuse the multi-scale features output by the backbone network. The cross-fusion module utilizes a self-attention mechanism. During the upsampling process, based on deep semantic features, higher-level information is targetedly fused into shallow features, thereby realizing the holistic processing of upsampling and feature fusion. It can not only fuse the specific features of the contextual information in a targeted manner, thereby effectively obtaining the overall information of the inner surface defect area of the steel, but also indirectly ensure the principle of minimizing information loss in the fusion stage, thereby reducing information loss in the feature fusion stage, and helping to improve the accuracy of inner surface defect detection of steel.

[0041] Furthermore, the present invention adopts the CIoU loss function as the regression loss of the model. The CIoU loss function focuses on the complete intersection between target frames and introduces a correction factor to more accurately measure the similarity between target frames, so that the model can better understand the exact position and specific shape of the target frame during training. It can not only more accurately evaluate the quality of the target frame, but also improve the accuracy of the model in detecting inner surface defects of steel in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0043] Figure 1 It is a structural diagram of the steel inner surface detection model of the present invention;

[0044] Figure 2 It is the structural diagram of the cross-fusion module;

[0045] Figure 3 This is the schematic diagram of the CIoU loss function;

[0046] Figure 4 This is an example of the results of using the YOLOv8 model to detect internal surface defects of steel;

[0047] Figure 5 This is an example of the results of using the C-YOLOv8 model to detect internal surface defects in steel.

[0048] Figure 6 It is a flow chart for detecting internal surface defects of steel. DETAILED DESCRIPTION

[0049] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0050] Existing detection algorithms commonly encounter issues such as misaligned and insufficient feature fusion when faced with irregular defects in complex scenarios. This phenomenon occurs because traditional detection algorithms employ simple operations such as concatenation or addition during the fusion phase, preventing the effective fusion of semantic information and detailed features, indirectly impacting subsequent detection results. Furthermore, traditional detection algorithms employ a fusion-first, then-sampling strategy to compensate for detailed features lost during upsampling. This strategy can easily lead to the loss of subtle features in the fused features during upsampling, making it impossible to comprehensively model the relationship between the two and degrading subsequent detection results.

[0051] In order to solve the problem of insufficient fusion and misalignment of existing detection features, this paper builds a steel inner surface detection model based on the YOLOv8 model, called the C-YOLOv8 model. The C-YOLOv8 model is an end-to-end network using a "codec" architecture. Its structure is similar to Figure 1 As shown, it includes the backbone network, neck network and detection head.

[0052] The backbone network of the steel inner surface detection model adopts the backbone network of the YOLOv8 model. Its purpose is to use a smaller model to extract deeper semantic information.

[0053] The backbone network includes an initial convolutional layer, a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module and an SPPF module connected in sequence.

[0054] The initial convolutional layer is a 3×3 convolutional block.

[0055] Each n×n convolution block in the network structure includes a sequentially connected n×n two-dimensional convolution layer, a normalization layer and a SiLU activation function.

[0056] Each feature extraction module consists of a 3×3 convolutional block and a C2f module connected sequentially.

[0057] The input features of the C2f module first pass through the input convolution block. The output features of the input convolution block are split into two parts through a split operation. One part is processed by multiple Bottleneck blocks, and the other part is retained as the features. The features processed by multiple Bottleneck blocks are concatenated with the retained features and then pass through the output convolution block to obtain the output features of the C2f module. The input and output convolution blocks of the C2f module are both 1×1 convolution blocks.

[0058] The input features of the Bottleneck block pass through the 3×3 convolution block and the 1×1 convolution block in sequence, and then the output features of the 3×3 convolution block and the output features of the 1×1 convolution block are added through the residual connection to obtain the output features of the Bottleneck block.

[0059] Preferably, the first feature extraction module includes 3 Bottleneck blocks, the second feature extraction module includes 6 Bottleneck blocks, the third feature extraction module includes 6 Bottleneck blocks, and the fourth feature extraction module includes 3 Bottleneck blocks.

[0060] The input features of the SPPF module pass through the input convolution block and three series-connected maximum pooling layers. The output features of the input convolution block and the output features of each maximum pooling layer are concatenated and then passed through the output convolution block to obtain the output features of the SPPF module. The input and output convolution blocks of the SPPF module are both 1×1 convolution blocks.

[0061] The neck network of the steel inner surface inspection model consists of three cross fusion modules (CFMs). The cross fusion module uses a self-attention fusion strategy to integrate the fusion and sampling processes. During the upsampling process, adjacent features are specifically integrated to fully capture the entire area of the steel inner surface defect.

[0062] Specifically, the structure of the cross-fusion module refers to Figure 2 As shown. In each cross-fusion module, the deep features First, the upsampling operation is performed to make its size consistent with the shallow features Consistent, get the deep features after upsampling Among them, H i 、W i and C i Represent the height, width and number of channels of deep features, H i-1 、W i-1 and C i-1 They represent the height, width and number of channels of shallow features respectively.

[0063] The deep features P″ after the above sampling i Generate query sequence, with shallow feature P i-1 Generate key sequence and value sequence, the formula is expressed as:

[0064] Q=P″ i W Q ,K=P i-1 W K ,V=P i-1 W V

[0065] in, represents the query sequence, represents a key sequence, Represents a sequence of values; W Q 、WK and W V are the linear projections corresponding to Q, K and V respectively.

[0066] Calculate the similarity between the query sequence and the key sequence. The formula is:

[0067] C′=σ(Q×K)

[0068] Where Q represents the query sequence, K represents the key sequence, σ represents the Sigmoid activation function, and C′ represents the similarity between the query sequence and the key sequence.

[0069] The similarity between the query sequence and the key sequence is multiplied by the value sequence and then passed through the Sigmoid activation function to obtain the fusion gating matrix, which further improves the pixel-level fusion between adjacent stages. The Sigmoid activation function can be used as a gate to filter out irrelevant factors between the two and focus on capturing the areas of common interest. The formula for generating the fusion gating matrix is expressed as:

[0070] C″=σ(C′×V)

[0071] Where C″ represents the fusion gating matrix, σ represents the Sigmoid activation function, C′ represents the similarity between the query sequence and the key sequence, and V represents the value sequence.

[0072] Weight the fused gating matrix to the shallow feature P i-1 , guide the shallow features to focus on the small area after fusion, and obtain the fusion feature P′ i-1 , the formula is:

[0073] P′ i-1 =C″⊙P i-1

[0074] Where ⊙ represents pixel-level dot product.

[0075] The present invention constructs a cross-fusion module and integrates it with the backbone network of the YOLOv8 model. This module employs a self-attention fusion strategy, integrating the fusion and upsampling components based on deep features. This module specifically integrates the specific features of contextual information, effectively capturing the overall information of the steel's inner surface defect areas and indirectly ensuring the principle of minimizing information loss during the fusion phase. This innovation provides a novel and efficient solution for the research of steel inner surface defect detection algorithms in complex environments, with significant practical application value.

[0076] The detection head of the steel inner surface detection model adopts the detection head of the YOLOv8 model, which includes three detection heads, one for detecting large targets, one for detecting medium targets, and one for detecting small targets.

[0077] The detection head consists of a regression branch and a classification branch, each of which consists of two 3×3 convolutional blocks connected in sequence and a 1×1 two-dimensional convolutional layer. The regression branch outputs the position and size of the predicted box; the classification branch outputs the probability that the predicted object belongs to each category.

[0078] The steel inner surface detection model is used to detect steel inner surface defects. The specific steps include:

[0079] The steel inner surface image to be inspected is input into the backbone network of the steel inner surface detection model, and the initial features are obtained through the initial convolution layer. The initial features are then processed by the first feature extraction module to obtain the first scale feature P1. The first scale feature P1 is then processed by the second feature extraction module to obtain the second scale feature P2. The second scale feature P2 is then processed by the third feature extraction module to obtain the third scale feature P3. The third scale feature P3 is then processed by the fourth feature extraction module and the SPPF module to obtain the fourth scale feature P′4.

[0080] Input the fourth scale feature P′4 and the third scale feature P3 into the third cross fusion module to obtain the third fused feature P′3; input the third fused feature P′3 and the second scale feature P2 into the second cross fusion module to obtain the second fused feature P′2; input the second fused feature P′2 and the first scale feature P1 into the first cross fusion module to obtain the first fused feature P′1;

[0081] The first fusion feature P′1, the second fusion feature P′2 and the third fusion feature P′3 are input into the first detection head, the second detection head and the third detection head respectively to obtain the detection results of the inner surface defects of the steel.

[0082] During training, existing models use a simple loss function to evaluate the difference between the predicted and actual bounding boxes. This simple loss function cannot achieve optimal model weights in complex environments, where defect variance is small or defect sizes are irregular. Therefore, it is necessary to introduce a loss function more appropriate for the specific environment to train the model and achieve optimal results.

[0083] Preferably, the present invention uses the CIoU loss function as the regression loss when training the steel inner surface detection model. The CIoU loss function considers the complete intersection between target boxes and introduces a correction factor to more accurately measure the similarity between target boxes. This allows the steel inner surface detection model to better understand the exact position and shape of the target boxes during training.

[0084] Reference Figure 3 As shown, the formula of CIoU loss function is expressed as:

[0085]

[0086] Among them, LCIoU represents the CIoU loss function, IoU represents the intersection-over-union ratio between the predicted box and the true box, d represents the distance between the center point of the predicted box and the true box, c represents the diagonal distance between the minimum circumscribed rectangle of the predicted box and the true box, α represents the balance parameter, and v represents the correction factor.

[0087] The correction factor is calculated as follows:

[0088]

[0089] Among them, w G and h G Represents the width and height of the real box, w p and h p Represent the width and height of the prediction box respectively.

[0090] The calculation formula of the equilibrium parameter is:

[0091]

[0092] During model training, this paper uses the CIoU loss function as a regression loss to address the incomplete intersection between the predicted and ground-truth boxes. This correction factor allows the model to more accurately measure the specific similarity between the target box and the ground-truth box, resolving the issue of the traditional IoU loss function's inability to accurately locate the target box's position and shape. This innovation not only enables more accurate assessment of target box quality but also improves the model's superior performance in complex scenarios.

[0093] Figure 4 and Figure 5 These figures show examples of the results of detecting internal surface defects in steel using the YOLOv8 model and the C-YOLOv8 model. As can be seen from the figures, the C-YOLOv8 model proposed in this paper significantly outperforms the YOLOv8 model. These results highlight the effectiveness of the C-YOLOv8 model in addressing the challenges posed by defect diversity, irregular shapes, and complex backgrounds, providing reliable technical support for the field of internal surface defect detection algorithms in steel.

[0094] In summary, the method for detecting inner surface defects of steel described in the present invention, based on the YOLOv8 model, constructs a neck network with a cross fusion module for fusing the multi-scale features output by the backbone network; the cross fusion module utilizes the self-attention mechanism, and in the process of upsampling, based on deep semantic features, targets higher-level information into shallow features, thereby realizing the overall processing of upsampling and feature fusion, which not only can target the specific features of contextual information, thereby effectively obtaining the overall information of the inner surface defect area of steel, but also indirectly ensures the principle of minimizing information loss in the fusion stage, reduces the information loss in the feature fusion stage, and helps to improve the accuracy of inner surface defect detection of steel.

[0095] Furthermore, the present invention adopts the CIoU loss function as the regression loss of the model. The CIoU loss function focuses on the complete intersection between target frames and introduces a correction factor to more accurately measure the similarity between target frames, so that the model can better understand the exact position and specific shape of the target frame during training. It can not only more accurately evaluate the quality of the target frame, but also improve the accuracy of the model in detecting inner surface defects of steel in complex scenarios.

[0096] In practical applications, refer to Figure 6 As shown in the figure, the detection of internal surface defects of steel mainly includes the following steps:

[0097] Dataset preprocessing includes: labeling the collected steel surface defects with roLableImg to construct a dataset; generating an XML file for the dataset; converting the XML file of the dataset into a TXT file; expanding the dataset through data augmentation operations such as flipping and mirroring; and dividing the dataset into a training set and a test set.

[0098] Build a steel inner surface detection model, including: the backbone network of the YOLOv8 model, the neck network built based on the cross-fusion module, and the detection head of the YOLOv8 model;

[0099] Model training includes: determining the weights required for training the steel inner surface detection model; setting model parameters; training the weights of the steel inner surface detection model using the training set; and obtaining the trained steel inner surface detection model;

[0100] Model prediction includes: inputting the inner surface image of the steel to be inspected to complete the trained steel inner surface inspection model to obtain the inspection results of the steel inner surface defects; randomly sampling within a certain area based on the inspection results to obtain various indicators; extracting the color features of the inspection results; and assigning different RGB values to different categories of defects to indicate the category of the defect and its specific location.

[0101] Based on the above-mentioned method for detecting inner surface defects of steel, this embodiment further provides a device for detecting inner surface defects of steel, comprising:

[0102] The feature extraction module is used to input the steel inner surface image to be detected into the steel inner surface detection model, and obtain the first scale feature P1, the second scale feature P2, the third scale feature P3 and the fourth scale feature P′4 through the backbone network;

[0103] A feature fusion module is configured to input the fourth scale feature P′4 and the third scale feature P3 into a third cross fusion module to obtain a third fused feature P′3; input the third fused feature P′3 and the second scale feature P2 into a second cross fusion module to obtain a second fused feature P′2; and input the second fused feature P′2 and the first scale feature P1 into the first cross fusion module to obtain a first fused feature P′1;

[0104] In each cross-fusion module, the deep feature P′ i Through upsampling operation, its size is similar to the shallow feature P i-1 Consistent, we get the upsampled deep feature P″ i ; The deep features P″ after the above sampling i Generate query sequence, with shallow feature P i-1 Generate key sequence and value sequence; calculate the similarity between query sequence and key sequence; multiply the similarity between query sequence and key sequence with value sequence and pass Sigmoid activation function to obtain fusion gating matrix; weight the fusion gating matrix to shallow feature P i-1 , get the fusion feature P′ i-1 ;

[0105] The detection module is used to input the first fusion feature P′1, the second fusion feature P′2 and the third fusion feature P′3 into the first detection head, the second detection head and the third detection head respectively to obtain the detection results of the inner surface defects of the steel.

[0106] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0110] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for detecting inner surface defects of steel, characterized in that: include: Input the steel inner surface image to be detected into the steel inner surface detection model, and obtain the first scale feature P1, the second scale feature P2, the third scale feature P3 and the fourth scale feature P'4 through the backbone network; The fourth scale feature P'4 and the third scale feature P3 are input into the third cross fusion module to obtain the third fusion feature P'3; the third fusion feature P'3 and the second scale feature P2 are input into the second cross fusion module. Obtain the second fusion feature P'2; input the second fusion feature P'2 and the first scale feature P1 into the first cross fusion module to obtain the first fusion feature P'1; In each cross-fusion module, the deep feature P' i Through upsampling operation, its size is similar to the shallow feature P i-1 Consistent, we get the upsampled deep feature P" i ; The deep features P after the above sampling i Generate query sequence, with shallow feature P i-1 Generate key sequence and value sequence; Calculate the similarity between the query sequence and the key sequence; The similarity between the query sequence and the key sequence is multiplied with the value sequence and then passed through the Sigmoid activation function to obtain the fused gating matrix; the fused gating matrix is weighted to the shallow feature P i-1 , get the fusion feature P' i-1 ; The first fusion feature P'1, the second fusion feature P'2 and the third fusion feature P'3 are input into the first detection head, the second detection head and the third detection head respectively to obtain the detection results of the inner surface defects of the steel.

2. A method for detecting inner surface defects of steel according to claim 1, characterized in that: The backbone network and detection head of the steel inner surface detection model both use the backbone network and detection head of the YOLOv8 model.

3. The method for detecting inner surface defects of steel according to claim 1, characterized in that: The deep features P" after the above sampling i Generate query sequence, with shallow feature P i-1 Generate key sequence and value sequence, the formula includes: Q=P” i W Q ,K=P i-1 W K ,V=P i-1 W V Among them, Q represents the query sequence, K represents the key sequence, V represents the value sequence; W Q 、W K and W V are the linear projections corresponding to Q, K and V respectively.

4. The method for detecting inner surface defects of steel according to claim 1, characterized in that: Calculate the similarity between the query sequence and the key sequence. The formula is: C'=σ(Q×K) Where Q represents the query sequence, K represents the key sequence, σ represents the Sigmoid activation function, and C' represents the similarity between the query sequence and the key sequence.

5. The method for detecting inner surface defects of steel according to claim 1, characterized in that: The similarity between the query sequence and the key sequence is multiplied by the value sequence and then passed through the Sigmoid activation function to obtain the fusion gating matrix. The formula is: C"=σ(C'×V Where C' represents the fusion gating matrix, σ represents the Sigmoid activation function, C' represents the similarity between the query sequence and the key sequence, and V represents the value sequence.

6. The method for detecting inner surface defects of steel according to claim 1, characterized in that: Weight the fused gating matrix to the shallow feature P i-1 , get the fusion feature P' i-1 , the formula is: P' i-1 =C”⊙P i-1 Where c'' represents the fusion gating matrix and ⊙ represents the pixel-level dot product.

7. A method for detecting inner surface defects of steel according to claim 1, characterized in that: When training the steel inner surface detection model, the CIoU loss function is used as the regression loss; The formula of CIoU loss function is expressed as: Among them, L CIoU represents the CIoU loss function, IoU represents the intersection-over-union ratio between the predicted box and the true box, d represents the distance between the center point of the predicted box and the true box, c represents the diagonal distance between the minimum circumscribed rectangle of the predicted box and the true box, α represents the balance parameter, and v represents the correction factor.

8. A method for detecting inner surface defects of steel according to claim 7, characterized in that: The correction factor is calculated as follows: Where v represents the correction factor, w G and h G Represents the width and height of the real box, w p and h p Represent the width and height of the prediction box respectively.

9. A method for detecting inner surface defects of steel according to claim 8, characterized in that: The calculation formula of the equilibrium parameter is: Among them, α represents the balance parameter, IoU represents the intersection over union ratio of the predicted box to the true box, and v represents the correction factor.

10. A device for detecting defects on the inner surface of steel, characterized in that: include: The feature extraction module is used to input the steel inner surface image to be detected into the steel inner surface detection model, and obtain the first scale feature P1, the second scale feature P2, the third scale feature P3 and the fourth scale feature P'4 through the backbone network; A feature fusion module is configured to input the fourth scale feature P'4 and the third scale feature P3 into a third cross fusion module to obtain a third fused feature P'3; input the third fused feature P'3 and the second scale feature P2 into a second cross fusion module to obtain a second fused feature P'2; and input the second fused feature P'2 and the first scale feature P1 into the first cross fusion module to obtain a first fused feature P'1; In each cross-fusion module, the deep feature P' i Through upsampling operation, its size is similar to the shallow feature P i-1 Consistent, we get the upsampled deep feature P" i ; The deep features P after the above sampling i Generate query sequence, with shallow feature P i-1 Generate key sequence and value sequence; Calculate the similarity between the query sequence and the key sequence; The similarity between the query sequence and the key sequence is multiplied with the value sequence and then passed through the Sigmoid activation function to obtain the fused gating matrix; the fused gating matrix is weighted to the shallow feature P i-1 , get the fusion feature P' i-1 ; The detection module is used to input the first fusion feature P'1, the second fusion feature P'2 and the third fusion feature P'3 into the first detection head, the second detection head and the third detection head respectively to obtain the detection results of the inner surface defects of the steel.

Images