A priori modulation dynamic visual self-attention model strip defect detection method

Abstract

The application relates to the field of defect visual detection, and aims to solve the problems of noise ambiguity between foreground defects and background clutter, influencing detection precision, semantic gap between coded image features and object queries, and reducing convergence speed. The application provides a priori modulation dynamic visual self-attention model strip defect detection method, which comprises the following steps: step S10, a backbone network extracts a multi-scale image feature map with different hierarchical resolutions and semantics; and step S20, an encoding network realizes dynamic interaction and aggregation of the multi-scale features based on a gate network, so that the foreground defects and the background clutter are distinguished based on the coded multi-scale feature map. Compared with related technologies, the application can obtain more superior detection effects of defects with different complexities and different morphological distributions, and has the beneficial effects of meeting the operation speed required by a production line and fast training speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual detection of steel surface defects, and particularly to a priori modulation dynamic visual self-attention model strip defect detection method. BACKGROUND

[0002] Hot rolling production lines are often disturbed by high temperature, dust, oil stains, water mist, elastic vibration and other various disturbances, producing complex defects, which brings the following challenges to the task of automatic defect detection:

[0003] 1) Inter-class similarity: due to the uncertainty of the production process, some defects of different reasons are similar in texture and gray scale.

[0004] 2) Intra-class difference: due to the change of material and casting conditions, the shape and scale of the same class of defects often differ greatly.

[0005] 3) Low contrast: the diffusion of dust and the change of illumination result in low contrast between the foreground defects and the background clutter.

[0006] 4) Defect overlap: equipment wear causes multiple defects to occur at the same time, and defect overlap causes information loss, resulting in insufficient characterization of the defect area.

[0007] In order to solve the above problems, the traditional detection method essentially depends on carefully designed hand-crafted features, objective functions and optimization strategies, and is very sensitive to changes in imaging environment and defect types. With the development of deep learning, convolution-based models have made significant progress in strip defect detection. In order to perceive intra-class differences, the model should remain invariant to changes in shape, scale and texture. Some pyramid modules with different pooling kernels aggregate different scale features extracted by the backbone network, but pooling will cause information loss. For inter-class similarity, the model needs to perceive the connection and difference between different kinds of defects as a whole, and some methods perceive more details of the defect location by combining hierarchical features. However, these convolution-based detection methods rely on many hand-crafted components such as non-maximum suppression, anchor points and rule-based training assignment strategies, and their architecture is quite complex and super parameter intensive, which leads to suboptimal expression of detection performance. Methods based on visual self-attention models eliminate the need for hand-crafted components while forming a pure end-to-end detection paradigm, which has obvious competitiveness, but these methods converge slowly, have deficiencies in small defect detection, and lack structural bias, which leads to overfitting on small data.

[0008] In addition, we find that:

[0009] 1) There is noise ambiguity between the foreground defects and the background clutter, which affects the detection accuracy.

[0010] 2) The semantic gap between the encoded image features and the object query reduces the convergence speed.

[0011] 3) Injecting priori in the visual self-attention model can alleviate the lack of samples and guide the convergence to some extent.

[0012] 4) The optimal detection performance of different types of defects is located at different encoder levels.

[0013] Therefore, it is necessary to tap the optimal detection performance potential of the visual self-attention model, and provide a dynamic visual attention model with priori modulation and semantic alignment capable of aiming at the small sample defect data set of the strip steel production line, eliminating the noise ambiguity between the foreground defects and the background noise, accelerating the convergence speed, balancing the detection accuracy and real-time performance, for solving the above problems of strip steel surface defect detection. SUMMARY

[0014] In view of the deficiencies of the prior art, the purpose of the present application is to provide a dynamic visual attention model with priori modulation and semantic alignment capable of adapting to small sample data sets of strip steel, accelerating the training convergence speed, eliminating the noise ambiguity between defects and background with more rich features, ensuring high accuracy and high recall rate in detecting defects, and meeting the speed requirements of the production line.

[0015] The technical scheme adopted by the present application to solve its technical problems is:

[0016] A priori modulation dynamic visual self-attention model strip steel defect detection method, comprising the following steps:

[0017] Step S10, backbone network, extracting multi-scale image feature maps with different hierarchical resolution and semantics;

[0018] Step S20, encoding network, realizing dynamic interaction and aggregation of multi-scale features based on the gating network, so as to realize the distinction between foreground defects and background clutter with the encoded multi-scale feature maps;

[0019] Step S30, semantic aligner, bridging the encoder and the decoder, so that the query in the decoding network and the feature in the encoding network are in the same embedding space;

[0020] Step S40, decoding network, under the modulation of priori, cross-attention is performed on the semantic aligned query and the encoded feature, and multi-scale feature fusion is performed by using the natural stack architecture, and finally the bounding box prediction is realized.

[0021] Preferably, step S10 comprises the following sub-steps:

[0022] Step S11, the input image is subjected to feature extraction by a backbone network ResNet50 to obtain feature maps of different resolutions in the third to fifth stages

[0023] Step S12, the three-level feature maps I1, I2 and I3 obtained are subjected to a transformation with a convolution kernel of 1 and a step of 1 into a dimension of 256 to obtain Z1, Z2 and Z3, and the I1 with the highest resolution is subjected to a transformation with a convolution kernel of 3 and a step of 2 into a dimension of 256 to obtain Z4, and finally four scale image feature maps with the same number of channels are obtained and are combined into a sequence Z.

[0024] Preferably, the step S20 comprises the following sub-steps:

[0025] Step S21, the input feature map x is divided into two parts along the channel dimension d, and then flows into two branches of attention and convolution with channel proportions of α.d and (1-α).d, respectively, wherein the long-term global information is extracted by the attention branch and the diagonal short-term local information is specifically extracted by the convolution;

[0026] Step S22, a reference point P q is applied to guide the deformation self-attention qx . qy The multi-scale feature map with the proportion of α.d flowing in is captured for specifically global feature extraction, represented as

[0027]

[0028] wherein, Z q is the content feature of the query, P q is the reference point, x l is the feature map of the l scales of the channel flowing into the attention, W is the weight, ΔP mlqk is the offset of the reference point, A mlqk is the scalar feature weight, φ l represents re-converting the sequence into multi-scale, and finally the processed global feature map x global is obtained.

[0029] Step S23, a convolution with a kernel of 3 is applied to the feature map of the (1-α).d channel to enhance the local information, and a GLU function and a fully connected layer FC are used for nonlinear transformation to obtain the processed local feature map x local of this branch. The above process is represented as:

[0030] x local =FC(GLU(conv((1-α).d))),

[0031] Step S24, x global is obtained by step S23 local The mixed multi-scale features are interactively aggregated based on convolutional enhancement through the feedforward network FFN.

[0032] Step S25, the selection of the number of encoder stacks is realized by using a lightweight gating network, where the gating network f GC is a lightweight policy network representing the decision to retain or delete each encoder block, and the 3-stage ResNet is selected as the architecture, and a 6-dimensional Bernoulli distribution is defined as the policy behavior, which is represented as:

[0033]

[0034]

[0035] where w is the weight of the parameterized policy network f GC , s is the output of the network after the sigmoid function, and u k = 0 represents discarding the encoder, and u k = 1 represents retaining the encoder.

[0036] Step S26, the use of the minimum encoding block is motivated by giving a large reward for correct prediction and a penalty for incorrect prediction, which is represented as:

[0037]

[0038] where P measures the percentage of encoder usage, and γ controls the trade-off between accuracy and efficiency, so that easy defects can be identified at a shallow level, and more expression capacity is reserved for difficult samples, and finally the multi-scale feature map E after encoding and dynamic selection is obtained.

[0039] Preferably, step S30 includes the following sub-steps:

[0040] Step S31, the region feature F R ∈R 7×7×d is extracted from the multi-scale feature E according to the candidate frame alignment operation RoIAlign using multiple representative key reference frames Bq(Bqx, Bqy, Bqw, Bqh) of each query.

[0041] F R = RoIAling(E, B q )

[0042] Step S32, the position and content of the query embedding Q are subjected to corresponding linear sigmoid operations to form the corresponding intermediate variables Qtemp pos and Qtemp content, which are represented as:

[0043]

[0044] Step S33, in order to predict the spatial offset of the key reference frame, the ConvNet is used to process F R , and then the new frame R(Bqx, Bqy, B'qw, B'qh) is obtained by using the MLP, under the guidance of R, the content weight C R is sampled from F w to the content weight C w by bilinear interpolation, and the position weight P R is obtained by the sine function, which is expressed as:

[0045] R=MLP(ConvNet(F R ))

[0046] C w ={F R [...,x,y]x,y∈R}P w =sin(R)

[0047] Step S34, the new content Qnew content is obtained by the dot product of C w and Qtemp content, and the new position Qnew pos is obtained by the dot product of P w and Qtemp pos, which is expressed as:

[0048]

[0049] Preferably, step S40 includes the following sub-steps:

[0050] Step S41, the priori G is modulated cross attention between the query feature and the input encoded feature map, in order to fully aggregate the partial information of large defects and suppress the background clutter of small defects, the priori G should dynamically generate the scale (Sw, Sh) to adapt to defects of different sizes, and the priori G usually gives higher weight to the position close to the center, and the center Cq(Cx, Cy) is derived from the reference frame Bq, and the scale is obtained by F R in the semantic aligner and learned in parallel, which is expressed as:

[0051]

[0052] Where (i, j) is the spatial coordinate of the attention map, the size of the attention map is [H, W], and β adjusts the bandwidth;

[0053] Step S42, the multi-head part d common attention map modulated by Gi further highlights different backgrounds and improves detection accuracy, and since the hypothesis space is reduced, the convergence speed can be accelerated, which is expressed as:

[0054]

[0055] Step S43, the encoded multi-scale features are fused from coarse to fine using the natural advantages of the encoder stack, the first two layers of decoders receive high-level semantic features E1 at the lowest resolution to reduce the search for initial positioning, the next two layers of decoders use finer features E2 and E3 for access, both of which have the same resolution to achieve more accurate positioning, in addition, the bottom layer features E4 at the highest resolution flow into the last two layers of decoders to perceive the details of minor defects,

[0056] Step S44, after the multi-scale feature fusion, the bounding box is predicted using the prediction head, taking the prior spatial reference (Cx, Cy) as the initial center of the bounding box, and finally calculating the normalized bounding box B, expressed as:

[0057] B={σ(b x +σ -1 (C x )),σ(b y +σ -1 (C y )),σ(b w ),σ(b h )}

[0058] b={b x ,b y ,b w ,b h}

[0059] Where B is predicted by the prediction head, and σ and σ -1 represent sigmoid and inverse sigmoid functions for normalization;

[0060] Step S45, the matching loss measures the similarity between the prediction result and the true value, which includes the category similarity and the bounding box similarity, and the total loss is the linear combination of the category prediction and the bounding box prediction, expressed as:

[0061]

[0062] Where N is the number of samples in the training set, y is the true value, is the predicted value, under the optimal matching χ, is the predicted probability of category C i ; is the bounding box prediction probability, B(i) is the true value bounding box, and λ class is the category loss coefficient, and the bounding box loss is composed of L1 regularization loss L L1 and generalized orthogonal ratio loss L iou , expressed as:

[0063]

[0064] wherein, lambda iou and lambda L1 are the corresponding hyperparameters.

[0065] In summary, compared with the prior art, the prior modulation and semantic alignment based dynamic visual self-attention model prior modulation strip defect detection method provided by the present application has the following advantages:

[0066] (1) The present application proposes a stable and effective defect detection method based on a visual self-attention model to address the challenges of inter-class similarity, inter-class difference, low contrast and overlapping defects on the surface of the strip. By mining the inherent properties of the image, the noise ambiguity between the defects and the clutter can be eliminated.

[0067] (2) The present application introduces priori into the query and encoding features in the same space after alignment operation, modulates cross attention, reduces the hypothesis space, speeds up the convergence speed, and realizes multi-scale feature fusion from coarse to fine.

[0068] (3) The present application proposes a dynamic deep network architecture based on a gating network for actual industrial applications, and gives a practical and feasible comprehensive solution in terms of calculation overhead and classification accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 is a flowchart of the present application;

[0070] Figure 2 is a flowchart of the specific sub-steps of step S10 in the present application;

[0071] Figure 3 is a flowchart of the specific sub-steps of step S20 in the present application;

[0072] Figure 4 is a flowchart of the specific sub-steps of step S30 in the present application;

[0073] Figure 5 is a flowchart of the specific sub-steps of step S40 in the present application;

[0074] Figure 6 is a flowchart of the steps in the present application;

[0075] Figure 7 is a flowchart of steps S21-S24 in the present application;

[0076] Figure 8 is a flowchart of steps S41-S42 in the present application;

[0077] Figure 9 The effect diagram applied to the strip steel defect detection in steps S21-S24 in the present application;

[0078] Figure 10 The effect diagram applied to the strip steel defect detection in steps S41-S42 in the present application. DETAILED DESCRIPTION

[0079] The present application will be described in detail below with reference to the accompanying drawings and examples. The following experimental examples and examples are used to further illustrate but not limit the present application.

[0080] Please refer to Figures 1 to 10 The present application provides a priori modulation dynamic visual self-attention model strip steel defect detection method, specifically comprising the following steps:

[0081] Step S10, backbone network, extracting multi-scale image feature maps with different hierarchical resolution and semantics;

[0082] Wherein, step S10 comprises the following sub-steps:

[0083] Step S11, the input image is subjected to feature extraction by the backbone network ResNet50, and different resolution feature maps of the third stage to the fifth stage are obtained

[0084] Step S12, the three-level feature maps I1, I2, I3 obtained are transformed into 256 dimensions by a convolution kernel of 1 and a step of 1, and Z1, Z2, Z3 are obtained; and the I1 with the highest resolution is transformed into Z4 of 256 dimensions by a convolution kernel of 3 and a step of 2, and finally four-scale image feature maps with the same number of channels are obtained And combine them into a sequence Z.

[0085] Step S20, encoding network, realizing dynamic interaction and aggregation of multi-scale features based on a gating network, and realizing the distinction between foreground defects and background clutter with the encoded multi-scale feature maps;

[0086] Step S20 comprises the following sub-steps:

[0087] Step S21, the input feature map x is divided into two parts along the channel dimension d, and then flows into the attention and convolution two branches with channel ratios of a.d and (1-a).d, respectively. Wherein, the long-term global information is extracted by the attention branch and the diagonal short-term local information is specifically extracted by the convolution.

[0088] Step S22, applying reference point P q (P qx ,P qy) guided deformed self-attention on the multi-scale feature maps of the incoming a.d ratio to capture context for dedicated global feature extraction, resulting in processed global feature maps x global， is represented as:

[0089]

[0090] where Z q is the content feature of the query, P q is the reference point, x l is the feature map of the incoming attention channel of l scales, W is the weight, and ΔP mlqk is the offset of the reference point, A mlqk is the scalar feature weight, and φ l represents the re-conversion of the sequence back to multi-scale.

[0091] Step S23, applying a convolution with a kernel of 3 to the feature map of the (1-α).d channel to enhance local information, and performing nonlinear transformation on it through a GLU function and a fully connected layer FC to obtain the processed local feature map x local of this branch. The above process is represented as:

[0092] x local = FC(GLU(conv((1-α).d))) Step S24, mixing x global obtained in S22 and x local obtained in S23 through a feedforward network FFN to realize multi-scale feature interaction and aggregation based on convolution enhancement.

[0093] Step S25, using a lightweight gating network to select the number of encoder stacks. The gating network f GC is a lightweight policy network, representing the decision to retain or discard each encoder block. A 3-stage ResNet is selected as the architecture, and a 6-dimensional Bernoulli distribution is defined as the policy behavior, represented as:

[0094]

[0095]

[0096] where w is the weight of the parameterized policy network f GC , and s is the output of the network after the sigmoid function. u k = 0 indicates discarding the encoder, and u k = 1 indicates retaining the encoder.

[0097] Step S26, the use of the minimum encoding block is motivated by giving a large reward for correct predictions and a penalty for incorrect predictions, represented as:

[0098]

[0099] Step S30, semantic aligner, bridge encoder and decoder, so that the query in the decoding network and the feature in the encoding network are in the same embedding space.

[0100] Step S30 includes the following sub-steps:

[0101] Step S31, using multiple representative key reference boxes Bq(Bqx, Bqy, Bqw, Bqh) of each query, region features F are extracted from multi-scale features E according to the candidate box alignment operation RoIAlign R ∈R 7×7×d , denoted as:

[0102] F R =RoIAlign(E,B q )

[0103] Step S32, the position and content of the query embedding Q are subjected to corresponding linear sigmoid operations to form the corresponding intermediate variables Qtemp pos and Qtemp content, denoted as:

[0104]

[0105] Step S33, in order to predict the spatial offset of the key reference box, ConvNet is used to process F R , and then MLP is used to obtain a new box R(Bqx, Bqy, B'qw, B'qh). Under the guidance of R, content weights C R are sampled from F w by bilinear interpolation, and position weights P w are obtained by a sine function, denoted as:

[0106] R=MLP(ConvNet(F R ))

[0107] C w ={F R [...,x,y];x,y∈R},P w =Sin(R)

[0108] Step S34, C w dot Qtemp content to obtain new content Qnew content, P w dot Qtemp pos to obtain new position Qnew pos, denoted as:

[0109]

[0110] Step S40, decode network, cross attention between semantic aligned query and encoded features under prior modulation, and use natural stack architecture for multi-scale feature fusion, finally realize bounding box prediction.

[0111] Step S40 includes the following sub-steps:

[0112] Step S41, dynamically generate scale (Sw, Sh) for G prior, to adapt to defects of different sizes. G usually gives higher weight to positions close to the center, and the center Cq (Cx, Cy) is derived from the reference box Bq, and the scale is derived from F in the semantic aligner. R Parallel learning, denoted as:

[0113]

[0114] Where (i, j) is the spatial coordinates of the attention map, the size of the attention map is [H, W], and β adjusts the bandwidth.

[0115] Step S42, multi-head Gi modulated common attention map to further highlight different backgrounds and improve detection accuracy, and convergence speed can be accelerated due to reduced hypothesis space. Denoted as:

[0116]

[0117] Step S43, use the natural advantage of encoder stack to fuse from coarse to fine for the encoded multi-scale features. The first two layers of decoder receive high-level semantic features E1 with the lowest resolution to reduce the search for initial positioning. The next two layers of decoder use more detailed features E2 and E3 for access, both with the same resolution, to achieve more accurate positioning. In addition, the bottom layer features E4 with the highest resolution flow into the last two layers of decoder to perceive the details of small defects.

[0118] Step S44, after multi-scale feature fusion, use the prediction head to predict the bounding box. We take the prior spatial reference (Cx, Cy) as the initial center of the bounding box, and finally calculate the normalized bounding box B, denoted as:

[0119] B = {σ(b x + σ -1 (C x )), σ(b y + σ -1 (C y )), σ(b w ), σ(b h )}

[0120] b = {b x , b y , b w , bh}

[0121] where B is predicted by the prediction head, σ and σ -1 represent sigmoid and inverse sigmoid functions for implementing normalization.

[0122] Step S45, matching loss measures the similarity between the prediction result and the true value, which includes category similarity and bounding box similarity. The total loss is a linear combination of category prediction and bounding box prediction, represented as:

[0123]

[0124] where N is the number of samples in the training set, y is the true value, is the predicted value. At the optimal matching χ, is the predicted probability of category C i ; is the bounding box prediction probability, B(i) is the true bounding box, λ class is the category loss coefficient. The bounding box loss is composed of L1 regularization loss L L1 and generalized orthogonal ratio loss L iou , represented as:

[0125]

[0126] where λ iou and λ L1 are the corresponding hyperparameters.

[0127] Compared with the prior art, the priori modulation dynamic visual self-attention model strip defect detection method completely retains the visual self-attention model which does not depend on the hand-designed component and is completely end-to-end. At the same time, in view of the limitations of the visual self-attention model itself, such as the need for large data driving and slow convergence speed, and the challenges of strip production line surface defect detection, such as inter-class similarity, intra-class difference, low contrast, and defect overlap, by fully tapping the optimal detection performance potential of the visual self-attention model, the method can use the convolution enhanced attention method to eliminate the foreground defect and background noise ambiguity for the small sample defect data set of the strip production line, the priori modulation co-attention can further reduce the hypothesis space to speed up the convergence speed, and the gate network controls the network depth to dynamically balance the detection accuracy and real-time performance. The dynamic visual self-attention model based on priori modulation and semantic alignment is aimed at the defect distribution characteristics of the production line, fully taps the inherent properties of the image and introduces priori to assist in inference. Therefore, compared with the traditional convolution-based and visual self-attention model-based methods, the method can obtain more superior detection effect for defects with different complexity and different morphological distribution, and has the beneficial effects of meeting the running speed required by the production line and fast training speed.

[0128] The above merely describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-described embodiments. Any technical solution falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary skilled persons in the art, some improvements and modifications without departing from the principles of the present application shall also be considered as falling within the protection scope of the present application.

Claims

1. A method for priori modulated dynamic visual self-attention model strip defects detection, characterized in that, Comprise: Step S10, backbone network, extract multi-scale image feature maps with different hierarchical resolution and semantics; Step S20, encoding network, realize dynamic interactive aggregation of multi-scale features based on gating network, and realize the distinction of foreground defects and background clutter based on encoded multi-scale feature maps; Step S30, semantic aligner, contact the encoder and the decoder, so that the query in the decoding network and the feature in the encoding network are in the same embedding space; Step S40, decoding network, cross attention to the semantic aligned query and encoded feature under the modulation of priori, and use natural stack architecture to realize multi-scale feature fusion, and finally realize the prediction of bounding box; Step S10 comprises the following sub-steps: Step S11, feature extraction is performed on the input image through a backbone network ResNet50 to obtain feature maps of different resolutions in the third stage to the fifth stage ( i =1, 2, 3; C 1=512, C 2=1024, C 3=2048); Step S12, the three-level feature maps obtained are transformed by using a convolution kernel of 1 and a step of 1 to obtain a 256-dimensional I 1、 I 2 、I 3 , and the highest-resolution Z 1 is transformed by using a convolution kernel of 3 and a step of 2 to obtain a 256-dimensional Z 2 Z 3 ; and the highest-resolution I 1 is transformed by using a convolution kernel of 3 and a step of 2 to obtain a 256-dimensional Z 4 , and finally a four-scale image feature map with the same number of channels is obtained ( i =1, 2, 3,4; d=256) Z ; Step S20 comprises the following sub-steps: Step S21, given input feature map x Along the channel dimension d Divided into two parts, then respectively with α.d And (, the channel proportion of 1-α ) .d The flow-in attention and convolution two branches, where the long-term global information is extracted by the attention branch, and the diagonal short-term local information is extracted by the convolution. Step S22, application of reference points P q ( P qx ,P qy ) guided deformed self-attention on inflow α.d proportional multi-scale feature maps capture context, denoted as: in, Z q For the characteristics of the content being queried, P q As a reference point, x l It is the flow of attention into the channel. l Feature maps at scales, W is the weight, Δ P mlqk A is the offset of the reference point. mlqk For scalar feature weights, This means converting the sequence back to multi-scale mode, ultimately obtaining the processed global feature map. x global ; Step S23, apply a convolution with a kernel of 3 to (1 α ). d The channel feature map to enhance local information, by GLU Function and fully connected layer FC Nonlinear transformation on it, get processed local feature map x local The above process is represented as: Step S24, mixing the result of step S22 x global with the result of step S23 x local by the feedforward network FFN; Step S25, the selection of the number of encoder stacking layers is implemented by using a lightweight gating network, wherein the gating network f GC The lightweight policy network is a 3-stage ResNet architecture, and a 6-dimensional Bernoulli distribution is defined as the policy behavior, which is represented as: wherein, w is a parameterized policy network f GC with weights, s is the output of the network after the sigmoid function, u k = 0 indicates discarding the encoder, and u k = 1 indicates retaining the encoder; Step S26, stimulate the minimum encoding block by giving a large reward to correct prediction and a penalty to incorrect prediction, represented as: wherein, P measuring the percentage of use of the encoder, control the trade-off between accuracy and efficiency, identify easy defects in shallow layers, and reserve more expressive power for difficult samples, ultimately obtaining the multi-scale feature map after dynamic selection after encoding E .

2. The method of claim 1, wherein the prior-modulated dynamic visual self-attention model for strip defect detection is characterized by, Before step S25, the initial encoder level is 6, and different defects have preference for the number of encoders.

3. The method of claim 1, wherein the prior-modulated dynamic visual self-attention model for strip defect detection is trained using a dataset comprising images of strip defects. Step S30 comprises the following sub-steps: Step S31, using a plurality of representative key reference boxes for each query Bq(Bqx,Bqy,Bqw,Bqh) extracting region features from multi-scale features according to a candidate box alignment operation RoIAlign E extracting region features from multi-scale features according to a candidate box alignment operation RoIAlign is expressed as: Step S32, the position and content of the query embedding Q are subjected to corresponding linear sigmoid operation, forming the corresponding intermediate variables of position and content Qtemp pos and Qtemp content , expressed as: Step S33: Use ConvNet to... F R Process the data, then use MLP to obtain a new bounding box. R ( Bqx,Bqy,B'qw,B'qh ),exist R Guided by this, bilinear interpolation is used to... F R Content weights sampled from the middle C w Position weights are obtained through a sine function. P w , is represented as: Step S34, C w dot product Qtemp content get new content Qnew content , P w dot product Qtemp pos get new position Qnew pos , is expressed as:

4. The method of claim 3, wherein the prior-modulated dynamic visual self-attention model for strip defect detection is trained using a dataset comprising images of strip defects. Before step S40, the semantic alignment operation is performed by using the sub-steps in step S30.

5. The method of claim 1, wherein the prior-modulated dynamic visual self-attention model for strip defect detection is characterized by, Step S40 comprises the following sub-steps: Step S41, injecting prior between query feature and input encoding feature mapping G Modulating cross attention, prior G Should dynamically generate scale ( S w ,S h ), prior G Assign higher weights to positions close to the center, center C q ( C x ,C y ) derived from the reference box Bq, scale derived from the semantic aligner in F R Parallel learning, represented as: wherein, are the spatial coordinates of the attention map, and the size of the attention map is , β adjusting the bandwidth; Step S42, Gi Modulated multi-head d The common attention map in turn highlights different backgrounds, represented as: Step S43, the encoded multi-scale features are fused from coarse to fine by the encoder, the first two layers of decoders receive the high-level semantic features with the lowest resolution E 1, the next two layers of decoders use finer features E 2 and E 3 for access, both have the same resolution; the bottom features E 4 with the highest resolution flow into the last two layers of decoders; Step S44, after the multi-scale feature fusion, the bounding box is predicted using the prediction head, and the prior spatial reference (Cx, Cy) As the initial center of the bounding box, the normalized bounding box is finally calculated B , which is expressed as: wherein, B predicted by the prediction header, σ and σ -1 denote sigmoid and inverse sigmoid functions for implementing normalization; Step S45, the matching loss measures the similarity between the prediction result and the true value, which includes category similarity and bounding box similarity, and the total loss is the linear combination of category prediction and bounding box prediction, represented as: where, N is the number of samples in the training set, y is the ground truth, is the predicted value, in the best match χ below, is the class C i prediction probability; is the bounding box prediction probability, is the ground truth bounding box, λ class is the class loss coefficient, the bounding box loss consists of L1 regularization loss L L1 and generalized cross-bi loss L iou two parts, expressed as: wherein λ iou and λ L1 are the corresponding hyperparameters.

6. The method of claim 4, wherein the prior-modulated dynamic visual self-attention model for strip defect detection is trained using a dataset comprising images of strip defects. Step S45 supervises the training process of the whole detection network.

Images