Industrial Defect Detection Methods, Equipment, and Media Based on Deep Learning Chips

Abstract

This application provides an industrial defect detection method, device, and medium based on a deep learning chip. The method involves stitching together features from different industrial images to be detected to obtain target stitched features; obtaining a first pixel group and a second pixel group from the target stitched features using a sliding window and pooling; obtaining a first attention score and a second attention score based on the first and second pixel groups, and setting a first parameter matrix and a second parameter matrix; determining the spatial relative coordinates and logarithmic interval continuous positional deviation between the second parameter matrix and the query, thereby determining the attention weights of the first pixel group, the attention weights of the second pixel group, and the attention value; performing industrial defect detection based on the attention value; and combining different types of image features extracted by different methods with the corresponding attention features to achieve high-precision detection of small or hidden defects.

Description

Technical Field

[0001] This application relates to the field of industrial inspection, and more particularly to industrial defect detection methods, equipment, and media based on deep learning chips. Background Technology

[0002] Industrial defect detection is a crucial part of industrial production. Currently, there are three common methods for industrial defect detection: First, manual inspection, where workers visually identify defects, which is labor-intensive and inefficient. Second, X-ray and radiographic imaging, using computed tomography (CT) scans, can provide detailed three-dimensional structural information about the internal structure of objects, but CT scanners are expensive. The CT scan process generates a large number of 3D images, consuming significant storage space and time, and ultimately still requires visual identification by workers. Third, deep learning algorithms are used, but currently, deep learning algorithms used for industrial defect detection still have a high error rate for small or hidden defects. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0004] The purpose of this application is to at least partially solve one of the technical problems existing in the related technologies. The embodiments of this application provide an industrial defect detection method, equipment and medium based on deep learning chips, which can achieve high-precision detection of small or hidden defects.

[0005] An embodiment of the first aspect of this application, a method for industrial defect detection based on deep learning chips, includes:

[0006] Acquire the industrial image to be inspected;

[0007] The features of different industrial images to be detected are stitched together to obtain the target stitched features;

[0008] The first pixel group is obtained from the target stitching feature by sliding window, and the second pixel group is obtained from the target stitching feature by pooling;

[0009] A first attention score is obtained based on the first pixel group, and a second attention score is obtained based on the second pixel group;

[0010] A first parameter matrix is ​​set according to the first pixel group, and a second parameter matrix is ​​set according to the second pixel group;

[0011] Determine the spatial relative coordinates between the second parameter matrix and the query of the attention mechanism, and determine the logarithmic interval continuous positional deviation of the spatial relative coordinates;

[0012] Based on the concatenation result of the logarithmic interval of the continuous positional deviation between the first parameter matrix and the spatial relative coordinates, and the concatenation result of the first attention score and the second attention score, the attention weights of the first pixel group and the second pixel group are obtained.

[0013] The attention value is obtained based on the attention weights of the first pixel group and the second pixel group;

[0014] The mask features are obtained by masking based on the attention values;

[0015] The industrial defect detection results are obtained by performing detection based on the mask features.

[0016] According to certain embodiments of the first aspect of this application, the step of stitching together features of different industrial images to be detected to obtain target stitched features includes:

[0017] Extract a first type of feature and a second type of feature from the industrial image to be detected, wherein the first type of feature and the second type of feature are of different types;

[0018] The first type of features of different industrial images to be detected are stitched together to obtain the first stitched sub-feature;

[0019] The second type of features of different industrial images to be detected are stitched together to obtain the second stitched sub-feature;

[0020] Adaptive instance normalization is performed on the first and second splicing sub-features to obtain the third and fourth splicing sub-features;

[0021] The industrial image to be detected, the third stitching sub-feature, and the fourth stitching sub-feature are linearly combined to obtain multiple linear combination features;

[0022] Linear interpolation is performed on multiple linearly combined features based on the bounding box to obtain the target stitched features.

[0023] According to certain embodiments of the first aspect of this application, the target splicing feature is represented by the following formula: x m =T⊙g 11 +(MR c -R s +T)⊙g 22 +(R c -T)⊙g 12 +(R s -T)⊙g 21 In the formula, x m For target splicing features; g 11 g 22 g 12 and g 21For different linear combination characteristics; ⊙ is principal component multiplication; R c R is a preset parameter. c ∈R W×H×C M is an initialization matrix with the same size as the linear combination feature size; R s R is a preset parameter. s ∈R W×H×C And R s =r s M,r s The scalar value used to control the intensity of the second type of feature; T is a preset parameter, T∈R W×H×C And max(0,R) c +R s -M)≤T≤min(R c ,R s ).

[0024] According to certain embodiments of the first aspect of this application, obtaining a first attention score based on the first pixel group and obtaining a second attention score based on the second pixel group includes:

[0025] The first key is obtained based on the first pixel group;

[0026] The second key is obtained based on the second pixel group;

[0027] A fixed query is obtained based on the color image corresponding to the target stitching feature, a learning query is obtained based on the grayscale image corresponding to the target stitching feature, and a total query is obtained based on the fixed query and the learning query.

[0028] The first attention score is obtained based on the first key and the total query.

[0029] The second attention score is obtained based on the second key and the total query.

[0030] According to certain embodiments of the first aspect of this application, the attention weights of the first pixel group and the second pixel group are both related to the number of valid keys in the query interaction with the attention mechanism.

[0031] According to certain embodiments of the first aspect of this application, the number of valid keys in the query interaction with the attention mechanism is expressed by the following formula: N (i，j) =||ρ (i，j) ||+||σ (X) ||-||μ (i，j) ||;In the formula, N (i，j) ρ is the number of valid keys that interact with the query in the attention mechanism. (i，j) For the first pixel group, σ (X) For the second pixel group, μ (i，j)The mask segmentation is performed on the pixel in the i-th row and j-th column of the industrial image to be detected.

[0032] According to certain embodiments of the first aspect of this application, the attention value is expressed by the following formula: In the formula, X is the industrial image to be detected, X (i,j) Let A(X) be the pixel in the i-th row and j-th column of the industrial image to be detected. (i,j) Let ρ(i,j) be the attention value corresponding to the pixel in the i-th row and j-th column of the industrial image to be detected, and let A be the first pixel group. (i,j)～ρ(i,j) The attention weights for the first pixel group. V is the attention score obtained by multiplying the learning key by the total query. ρ(i,j) Let σ(X) be the value obtained from the target stitching features through a sliding window, and A be the value obtained from the second pixel group. (i,j)～ρ(X) V represents the attention weight for the second pixel group. σ(X) The value obtained from the target concatenation feature through pooling is T, which is a matrix used to adjust the dynamic positional deviation of the total query.

[0033] According to certain embodiments of the first aspect of this application, obtaining mask features by masking based on the attention value includes:

[0034] The attention value is processed by linear projection and divided into a first sub-attention value and a second sub-attention value.

[0035] The first sub-attention value is sequentially processed through convolution and activation to generate gating information;

[0036] The gating information and the second sub-attention value are multiplied element by element to retain a portion of the elements in the second sub-attention value and suppress another portion of the elements in the sub-attention value, resulting in a multiplication result;

[0037] The result of the multiplication is added to the attention value to obtain the mask feature.

[0038] According to a second aspect of this application, an electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the deep learning chip-based industrial defect detection method described in the first aspect of this application.

[0039] According to a third aspect of this application, a computer-readable storage medium stores a computer program, characterized in that, when the computer program is executed by a processor, it implements the deep learning chip-based industrial defect detection method described in the first aspect of this application.

[0040] The above scheme has at least the following beneficial effects: It obtains target stitched features by stitching together features from different industrial images to be detected; it obtains a first pixel group from the target stitched features through a sliding window and a second pixel group from the target stitched features through pooling; it obtains a first attention score based on the first pixel group and a second attention score based on the second pixel group; it sets a first parameter matrix based on the first pixel group and a second parameter matrix based on the second pixel group; it determines the spatial relative coordinates between the second parameter matrix and the query of the attention mechanism, and determines the logarithmic interval continuous positional deviation of the spatial relative coordinates; it obtains the attention weights of the first pixel group and the second pixel group based on the stitching result of the first parameter matrix and the logarithmic interval continuous positional deviation of the spatial relative coordinates, as well as the stitching result of the first attention score and the second attention score; it obtains attention values ​​based on the attention weights of the first and second pixel groups; it obtains mask features by masking based on the attention values; it performs detection based on the mask features to obtain industrial defect detection results; and by combining different types of image features extracted by different methods with the corresponding attention features, it can achieve high-precision detection of small or hidden defects. Attached Figure Description

[0041] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0042] Figure 1 This is a flowchart illustrating the steps of an industrial defect detection method based on a deep learning chip provided in an embodiment of this application.

[0043] Figure 2 This is a diagram of the sub-steps of step S200 provided in the embodiments of this application;

[0044] Figure 3 This is a sub-step diagram of step S400 provided in the embodiments of this application;

[0045] Figure 4 This is a schematic diagram illustrating the determination of the first splicing sub-feature, the second splicing sub-feature, the third splicing sub-feature, and the fourth splicing sub-feature provided in an embodiment of this application;

[0046] Figure 5 This is a schematic diagram of a feature map for determining attention values ​​from a color image and a grayscale image, provided in an embodiment of this application.

[0047] Figure 6 This is a schematic diagram of step S700 provided in the embodiments of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0050] The embodiments of this application provide an industrial defect detection method, device, and medium based on deep learning chips, which can achieve high-precision detection of small or hidden defects.

[0051] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0052] Reference Figure 1 An industrial defect detection method based on deep learning chips includes, but is not limited to, the following steps:

[0053] Step S100: Obtain the industrial image to be inspected;

[0054] Step S200: The features of different industrial images to be detected are stitched together to obtain the target stitched features;

[0055] Step S300: Obtain the first pixel group from the target stitching features through a sliding window, and obtain the second pixel group from the target stitching features through pooling;

[0056] Step S400: Obtain the first attention score based on the first pixel group, and obtain the second attention score based on the second pixel group;

[0057] Step S500: Set the first parameter matrix according to the first pixel group, and set the second parameter matrix according to the second pixel group;

[0058] Step S600: Determine the spatial relative coordinates between the second parameter matrix and the query of the attention mechanism, and determine the logarithmic interval continuous positional deviation of the spatial relative coordinates;

[0059] Step S700: Based on the splicing result of the logarithmic interval continuous positional deviation between the first parameter matrix and the spatial relative coordinates, and the splicing result of the first attention score and the second attention score, the attention weights of the first pixel group and the second pixel group are obtained.

[0060] Step S800: Obtain the attention value based on the attention weights of the first pixel group and the second pixel group;

[0061] Step S900: Obtain mask features by masking based on attention values;

[0062] Step S1000: Detection is performed based on the mask features to obtain the industrial defect detection results.

[0063] This industrial defect detection method is compatible with the following systems: using the HarmonyOS operating system and the open-source deep learning framework Cambricon PyTorch, and employing domestically produced Hygon x86 and Cambricon MLU370-S4 / S8 chips for deep learning training and inference.

[0064] The system is set up as follows: Install Cambricon MLU370-S4 / S8 graphics cards and Hygon x86 CPUs. Create a Docker container using the Docker image with pre-installed software stack provided by Cambricon. Install Cambricon PyTorch, CNToolkit, MagicMind, and MagicMind Python in this environment to provide a deep learning environment for subsequent algorithm training. Finally, transfer the runtime environment to the HarmonyOS system for global operation.

[0065] For step S100, multiple industrial images to be detected are captured by an industrial camera, and these images are combined into a dataset. During the training phase, labeling software, such as LabLE LME, is used to label the dataset. The labeled dataset is then divided into a training set and a test set, for example, the ratio of training set to test set data can be 3:1.

[0066] Reference Figure 2 For step S200, the features of different industrial images to be detected are stitched together to obtain the target stitched features, including the following steps:

[0067] Step S210: Extract the first type of features and the second type of features from the industrial image to be detected;

[0068] Step S220: The first type of features of different industrial images to be detected are stitched together to obtain the first stitched sub-features;

[0069] Step S230: The second type features of different industrial images to be detected are stitched together to obtain the second stitched sub-features;

[0070] Step S240: Adaptive instance normalization is performed on the first and second splicing sub-features to obtain the third and fourth splicing sub-features.

[0071] Step S250: Linearly combine the industrial image to be detected, the third stitching sub-feature, and the fourth stitching sub-feature to obtain multiple linear combination features;

[0072] Step S260: Perform linear interpolation on multiple linear combination features based on the bounding box to obtain the target stitched features.

[0073] Reference Figure 4 In step S210, the industrial image to be detected is passed through encoder f to extract the content and style of the industrial image to be detected, forming a first type of feature and a second type of feature. The first type of feature and the second type of feature are features of different types. For example, the first type of feature is a content feature and the second type of feature is a style feature.

[0074] Here, encoder f is the layer from the beginning of the VGG-19 model to the ReLU4_1 layer. Its main function is to extract representative content features and style features from the input feature map.

[0075] In step S220, the first type features of different industrial images to be detected are stitched together to obtain a first stitched sub-feature. For example, the first type features of x1 and x2 of two industrial images to be detected are stitched together to obtain the first stitched sub-feature f. 11 .

[0076] For step S230, the second-type features of different industrial images to be detected are stitched together to obtain a second stitched sub-feature. For example, the second-type features of x1 and x2 of two industrial images to be detected are stitched together to obtain the second stitched sub-feature f. 22 .

[0077] For step S240, adaptive instance normalization is performed on the first and second splicing sub-features to obtain the third and fourth splicing sub-features. For example, the first splicing sub-feature f 11 Second splice feature f 22 Adaptive instance normalization is performed to obtain the third concatenated sub-feature f. 12 and the fourth splice feature f 21 The purpose of adaptive instance normalization is to align the mean and variance of the content features of the two combinations with the mean and variance of the style features.

[0078] For step S250, the industrial image to be detected, the third stitching sub-feature, and the fourth stitching sub-feature are linearly combined to obtain multiple linearly combined features. For example, the third stitching sub-feature f 12 and the fourth splice feature f 21New text is generated by decoding using decoder g, and then adaptively adjusted according to the selected style. The adjusted result is then linearly combined with the industrial image to be detected to obtain the linear combination feature g. 12 and g 21 It can be expressed by the formula: g 12 =γx1+(1-γ)g(f 12 ), g 21 =γx² + (1-γ)g(f 21 ).

[0079] Linear interpolation can solve the problem where the feature image is much smaller than the size of the image frame used to piece together images x1 and x2. 12 The image contains content x1 and a style blend of images x1 and x2 with a ratio of (γ, 1-γ). Therefore, the new parameter γ adjusts the degree of blending between the style and content images in the linearly combined feature image. Furthermore, since the two input images are randomly selected training images, there is a possibility of excessive style differences. Setting a higher γ value suppresses the degree of blending between images to prevent poor performance of the blended image.

[0080] For step S260, linear interpolation is performed on multiple linear combination features based on the bounding box to obtain the target stitched features.

[0081] Define a bounding box with coordinates B = (r x ,r y ,r w ,r h The bounding box is sampled, a region is cropped from image x1 and pasted into image x2. Linear interpolation g is then used. 11 g 22 g 12 g 21 Create a blended image x with target stitching features m It can be expressed by the formula: x m =T⊙g 11 +(MR c -R s +T)⊙g 22 +(R c -T)⊙g 12 +(R s -T)⊙g 21 In the formula, x m For target splicing features; g 11 g 22 g 12 and g 21 For different linear combination characteristics; ⊙ is principal component multiplication; R c R is a preset parameter. c ∈RW×H×C M is an initialization matrix with the same size as the linear combination feature size; R s R is a preset parameter. s ∈R W×H×C And R s =r s M,r s The scalar value used to control the intensity of the second type of feature; T is a preset parameter, T∈R W×H×C And max(0,R) c +R s -M)≤T≤min(R c ,R s ).

[0082] From x m The formula shows that the content of the bounding box consists of the content of x1 inside the box and the content of x2 outside the box. The style of the box is the sum of the styles of x1 and x2 inside the box, which is 1 - (1 - r). s (1-γ) and (1-r) s The ratio is (1-γ), while outside the frame it is r. s (1-γ) and 1-r s (1-γ).

[0083] Assuming the area of ​​the bounding box is λ, the ratio of x1 to x2 in the blended image is λ and 1-λ, and the pattern ratio is γλ+(1-γ)r. s and 1-(γλ+(1-γ)r s Based on this, the label y m By content tag y c and style tag y s Define y c and y s It is a linear combination of labels y1 and y2. The loss function is the mean squared error loss L. s for: In the formula, n is the total number of data sets, and y is the label mask image.

[0084] For step S300, the first pixel group is obtained from the target stitching features by sliding window, and the second pixel group is obtained from the target stitching features by pooling.

[0085] For example, a set of pixels centered at the pixel at (i, j) within the sliding window is defined as ρ(i, j). The set of pixels obtained by pooling the feature map is defined as σ(X).

[0086] Reference Figure 3 For step S400, obtaining a first attention score based on the first pixel group and a second attention score based on the second pixel group includes the following steps:

[0087] Step S410: Obtain the first key based on the first pixel group;

[0088] Step S420: Obtain the second key based on the second pixel group;

[0089] Step S430: Obtain a fixed query based on the color image corresponding to the target stitching features, obtain a learning query based on the grayscale image corresponding to the target stitching features, and obtain a total query based on the fixed query and the learning query;

[0090] Step S440: Obtain the first attention score based on the first key and the total query;

[0091] Step S450: Obtain the second attention score based on the second key and the total query.

[0092] Reference Figure 5 For example, a fixed query is obtained based on the color image corresponding to the target stitching features, such as an RGB image. A learning query is generated centered on the pixels of the grayscale image corresponding to the target stitching features. The fixed query and the learning query are added together to obtain the total query Q. (i，j) Total queries Q (i，j) Features and location information of color and grayscale images corresponding to the target stitching features.

[0093] The key to the target stitching features, i.e., the first key, is obtained through a k×k sliding window based on the first pixel group. Typically, a minimal form of a 3x3 sliding window is used to capture features near the visual focus, expanding the model's receptive field. Therefore, ||ρ(i,j)||=k 2 It is understandable that the target stitching feature value can also be obtained by using a sliding window based on the first pixel group.

[0094] Learn the first key, and multiply the learned first key by the total query Q. (i，j) The first attention score S was obtained. (i,j)～ρ(i,j) Then, a padding mask is used to pad the pixels at the edges of the obtained feature map to prevent the similarity calculation between the pixels at the edge of the feature map and the zero padding outside the boundary from affecting the Softmax operation.

[0095] A parameterless adaptive average pooling method is used for downsampling in the spatial dimension. The downsampling operator is LayerNorm(AvgPool(GELU(Linear(X)))). A single-layer neural network is used for projection and activation before feature map pooling to prevent information loss by the average pooling operator, compressing and extracting useful information in advance, thereby improving the information compression ratio after downsampling. The given pooling size is H. p ×W p Then we have ||σ(X)||=H p W pAfter pooling, layer normalization is used to normalize the output to ensure consistency in the variances of X and σ(X). This yields the key to the target stitching feature, i.e., the second key. It can be understood that the value of the target stitching feature can also be obtained from the second pixel group through pooling.

[0096] Learn the second key, and multiply the learned second key by the total query Q. (i，j) Receive the second attention score S (i,j)～σ(X) .

[0097] For step S500, the first parameter matrix is ​​set according to the first pixel group, and the second parameter matrix is ​​set according to the second pixel group.

[0098] Based on the characteristics of the first pixel group, a learnable first parameter matrix B is set. (i,j)～ρ(i,j) It can generalize the relative positional deviation of the sliding window.

[0099] The second pixel group is input into a multilayer perceptron with two hidden layers to obtain the second parameter matrix B. (i，j)～σ(X) The activation function of the hidden layer is ReLU. The multilayer perceptron helps the network learn complex data patterns and features.

[0100] For step S600, determine the spatial relative coordinates between the second parameter matrix and the query of the attention mechanism, and determine the logarithmic interval continuous positional deviation of the spatial relative coordinates.

[0101] For example, the second parameter matrix B is calculated. (i，j)～σ(X) Total Query Q (i，j) Spatial relative coordinates Δ (i，j)～σ(X) Then use the logarithmic interval continuous position deviation log-CPB = log(x) i )-log(x i-1 For spatial relative coordinates Δ (i，j)～σ(X) Calculate log-CPB(Δ) (i，j)～σ(X) ), where x i Indicates the current position, x i-1 This indicates the position preceding the current position. The introduction of logarithmic interval continuous positional bias serves to generalize the relative positional bias of the feature.

[0102] For step S700, the attention weights of the first pixel group and the second pixel group are obtained based on the splicing result of the logarithmic interval continuous positional deviation between the first parameter matrix and the spatial relative coordinates, and the splicing result of the first attention score and the second attention score.

[0103] Reference Figure 6 , the first parameter matrix B (i,j)～ρ(i,j)Log-CPB(Δ) continuous positional deviation from spatial relative coordinates (i，j)～σ(x) The first attention score S is then concatenated. (i,j)～ρ(i,j) With the second attention score S (i，j)～σ(X) The concatenation process involves adding the concatenation results of the first parameter matrix and the logarithmically spaced continuous positional deviations of the spatial relative coordinates, along with the concatenation results of the first attention score and the second attention score. The sum is then adjusted according to adjustable parameters, followed by Softmax normalization and partitioning to obtain the attention weight A for the first pixel group. (i,j)～ρ(i,j) Attention weights A for the second pixel group (i,j)～σ(X) .

[0104] It should be noted that the attention weights of the first pixel group and the second pixel group are both related to the number of valid keys in the query interaction with the attention mechanism.

[0105] To better integrate learnable keys and the total query, a scaled cosine attention mechanism is introduced. This mechanism incorporates additional learnable coefficients multiplied by the cosine similarity result of the total query and learnable keys, allowing the attention mechanism to effectively ignore unimportant tokens. The confidence of the attention output decreases as the input sequence length increases. Therefore, the attention mechanism is designed to be entropy invariant to better generalize to unknown lengths. When the query and key are approximately of size... The entropy estimate of the scaled dot product of a vector sequence of length n is:

[0106] τlogN is an adjustable parameter, where N is the number of valid keys interacting with the attention mechanism's query. N does not include the number of masked tokens; specifically, when applied to an attention decoder, tokens masked by the mask should not be counted in N. Under pixel attention, N is calculated as N... (i，j) =||ρ (i，j) ||+||σ (X) ||-||μ (i，j) ||, where μ (i，j) Let τ represent the set of padding mask tokens located at position (i, j). τ is a learnable variable.

[0107] For step S800, the feature map of the attention value is obtained based on the attention weight of the first pixel group and the attention weight of the second pixel group.

[0108] The attention value is expressed by the following formula: In the formula, X is the industrial image to be detected, X (i,j) Let A(X) be the pixel in the i-th row and j-th column of the industrial image to be detected. (i,j)Let ρ(i,j) be the attention value corresponding to the pixel in the i-th row and j-th column of the industrial image to be detected, and let A be the first pixel group. (i,j)～ρ(i,j) The attention weights for the first pixel group. V is the attention score obtained by multiplying the learning key by the total query. ρ(i,j) Let σ(X) be the value obtained from the target stitching features through a sliding window, and A be the value obtained from the second pixel group. (i,j)～σ(X) V represents the attention weight for the second pixel group. σ(X) T represents the value obtained from the target concatenation feature through pooling, and T is a matrix used to adjust the dynamic positional bias of the total query.

[0109] Steps S700 and S800 can be implemented by an encoder and a decoder.

[0110] For step S900, mask features are obtained by masking based on the attention value.

[0111] In gated channel attention, the gated branch has more decision-making power than the value branch and ultimately determines whether the corresponding output element is set to zero. Gated channel attention includes a gated linear unit (GLU), a channel mixer. A minimal form of a 3×3 depthwise convolution is added before the activation function of the GLU. The introduction of depthwise convolution helps the model learn richer and more abstract representations, transforming it into a gated channel attention mechanism based on nearest-neighbor features. First, the input feature map X is divided into two parts, X1 and X2, by a linear projection unit. X1 is then subjected to a 3×3 depthwise convolution and fed into a sigmoid activation function to generate gating information. This gating information determines whether each element in X2 is preserved or suppressed. Then, the gating information is multiplied element-wise with X2, and the result is added to the input feature map X by the linear projection unit to obtain a mask map containing mask features. Each token in the convolutional gated linear unit has a unique gating signal based on its nearest fine-grained feature, which addresses the overly coarse-grained nature of global average pooling in other mechanisms.

[0112] For step S1000, detection is performed based on the mask features to obtain the industrial defect detection results.

[0113] To better distinguish the types of industrial defect detection, a segmentation head outputs the category and probability. Specifically, a deep convolutional layer is added before the segmentation head to extract and integrate feature information. The feature extraction map X, containing mask features, is integrated into a k×m×m shape, where k is the number of industrial defect categories and m is the length and width of the feature map. The m×m size allows each layer to maintain an explicit m×m object spatial layout without collapsing into a vector representation lacking spatial dimensions, truly preserving the explicit spatial correspondence of each pixel. The extracted feature map is then fed into the segmentation head, which has two branches: one predicts the category of the industrial defect, and the other predicts the probability of the corresponding category. To connect the two paths and achieve end-to-end detection in the industrial defect detection model, cross-entropy loss is used. Cross-entropy loss allows the network to generate a mask for each class without competition between classes. A dedicated classification head is used to predict the class label used to select the output mask. Finally, the network parameters are optimized by minimizing the cross-entropy loss, with the loss function L... e Defined as: Where N represents the total number of samples, C represents the total number of categories, and y i,j Indicates whether the true label of the i-th sample is category j. This represents the model's prediction of whether the i-th sample belongs to category j.

[0114] The inspected images are visualized, allowing staff to further assess the accuracy of the inspection results. The industrial defect detection results, obtained from images captured by an automated optical inspection camera and processed using industrial defect detection methods, are continuously and automatically updated and displayed through the HarmonyOS's built-in UI. This real-time display enables staff to more accurately determine the correctness of the defect detection results.

[0115] An embodiment of this application provides an electronic device. The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the industrial defect detection method based on a deep learning chip as described above.

[0116] This electronic device can be any smart terminal, including computers.

[0117] In general, for the hardware structure of electronic devices, the processor can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, to execute relevant programs and implement the technical solutions provided in the embodiments of this application.

[0118] The memory can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and is called and executed by the processor.

[0119] Input / output interfaces are used to implement information input and output.

[0120] The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0121] The bus transmits information between various components of a device, such as the processor, memory, input / output interfaces, and communication interfaces. The processor, memory, input / output interfaces, and communication interfaces communicate with each other within the device via the bus.

[0122] An embodiment of this application provides a computer storage medium. The computer storage medium stores computer-executable instructions for executing the deep learning chip-based industrial defect detection method described above.

[0123] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0124] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0125] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0127] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms. Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0129] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A deep learning chip-based industrial defect detection method, characterized in that, The method comprises the following steps: acquiring an industrial image to be detected; splicing features of different industrial images to be detected to obtain target spliced features; acquiring a first pixel group from the target spliced features through a sliding window and acquiring a second pixel group from the target spliced features through pooling; obtaining a first attention score according to the first pixel group and a second attention score according to the second pixel group; setting a first parameter matrix according to the first pixel group and a second parameter matrix according to the second pixel group; determining spatial relative coordinates between the second parameter matrix and a query of an attention mechanism and a logarithmic interval continuous position deviation of the spatial relative coordinates; obtaining attention weights of the first pixel group and attention weights of the second pixel group according to a splicing result of the first parameter matrix and the logarithmic interval continuous position deviation of the spatial relative coordinates and a splicing result of the first attention score and the second attention score; obtaining an attention value according to the attention weights of the first pixel group and the attention weights of the second pixel group; performing masking according to the attention value to obtain a masked feature; performing detection according to the masked feature to obtain an industrial defect detection result; wherein the obtaining of the first attention score according to the first pixel group and the second attention score according to the second pixel group comprises: obtaining a first key according to the first pixel group; obtaining a second key according to the second pixel group; obtaining a fixed query according to a color image corresponding to the target spliced features and obtaining a learning query according to a grayscale image corresponding to the target spliced features, and obtaining a total query according to the fixed query and the learning query; obtaining the first attention score according to the first key and the total query; obtaining the second attention score according to the second key and the total query; The attention value is expressed by the following formula: ; in which, is an industrial image to be detected, is a pixel of the i-th row and j-th column of the industrial image to be detected, is an attention value corresponding to the pixel of the i-th row and j-th column of the industrial image to be detected, is a first pixel group, is an attention weight of the first pixel group, is an attention score obtained by multiplying the learning key and the total query, is a value obtained from the target concatenation feature by a sliding window, is a second pixel group, is an attention weight of the second pixel group, is a value obtained from the target concatenation feature by pooling, is a matrix for adjusting the dynamic position bias of the total query. 2.The deep learning chip-based industrial defect detection method according to claim 1, wherein, the splicing of the features of different industrial images to be detected to obtain the target spliced features comprises: extracting a first type feature and a second type feature of the industrial image to be detected, the first type feature and the second type feature being different in type; splicing the first type features of different industrial images to be detected to obtain a first spliced sub-feature; splicing the second type features of different industrial images to be detected to obtain a second spliced sub-feature; performing adaptive instance normalization on the first spliced sub-feature and the second spliced sub-feature to obtain a third spliced sub-feature and a fourth spliced sub-feature; performing linear combination on the industrial image to be detected, the third spliced sub-feature and the fourth spliced sub-feature to obtain a plurality of linear combination features; performing linear interpolation on the plurality of linear combination features according to a boundary box to obtain the target spliced features. 3.The deep learning chip-based industrial defect detection method according to claim 2, wherein, The target stitching feature is expressed by the following formula: ; in the formula, is a target stitching feature; , , and are different linear combination features; is a main element multiplication; is a preset parameter, ; is an initialization matrix with the same size as the linear combination feature; is a preset parameter, and = , is a scalar value for controlling the strength of the second type feature; is a preset parameter, and max(0, )≤T≤min( , ). 4.The deep learning chip-based industrial defect detection method of claim 1, wherein, The attention weights of the first pixel group and the attention weights of the second pixel group are both related to the number of effective keys interacting with the query of the attention mechanism. 5.The deep learning chip-based industrial defect detection method according to claim 4, wherein, The number of valid keys interacting with the query of the attention mechanism is expressed by the following formula: ; in which, is the number of valid keys interacting with the query of the attention mechanism, is the first pixel group, is the second pixel group, is the mask word segmentation corresponding to the pixel of the i-th row and the j-th column of the industrial image to be detected. 6.The deep learning chip-based industrial defect detection method according to claim 1, wherein, the masking according to the attention value to obtain the masked feature comprises: linearly projecting the attention value into a first sub-attention value and a second sub-attention value; generating gating information by sequentially performing convolution processing and activation processing on the first sub-attention value; The gating information and the second sub-attention value are multiplied element by element to retain a part of elements in the second sub-attention value and suppress another part of elements in the sub-attention value, to obtain a multiplication result; The multiplication result is added to the attention value to obtain a mask feature.

7. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the deep learning chip-based industrial defect detection method in any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the deep learning chip-based industrial defect detection method in any one of claims 1 to 6.

Images