Surface defect recognition method based on multi-scale mixed kernel and structure reparameterization

Abstract

The present application belongs to the technical field of defect identification, and particularly relates to a surface defect identification method and system based on a multi-scale mixed kernel and structure reparameterization, and an electronic device, aiming to solve the problem that the existing surface defect identification method cannot identify non-processing defects, and cannot balance accuracy and speed during identification, resulting in poor robustness of identification. The method comprises the following steps: obtaining an image of an industrial part to be identified for surface defects as an input image; obtaining a surface defect identification result corresponding to the input image through M 2 Rep-Net, wherein M 2 Rep-Net is a deep network based on a multi-scale mixed kernel and structure reparameterization. The present application can identify non-processing defects in the image of the industrial part, and achieve a good balance between accuracy and speed.

Description

Technical Field

[0001] This invention belongs to the field of defect identification technology, specifically relating to a surface defect identification method, system, and electronic device based on multi-scale hybrid kernel and structural reparameterization. Background Technology

[0002] Surface defect detection plays a crucial role in the quality inspection of industrial components. To avoid the serious impact of surface defects on the performance and health of industrial components during use, it is essential to quickly and efficiently identify surface defects before these components are put into use, thus rapidly eliminating defective parts. The key to this lies in the Industrial Surface Defect Classification (ISDC) algorithm. Compared to manual inspection using specialized instruments, machine vision-based ISDC methods have gained widespread attention due to their higher efficiency and reliability. Traditional ISDC algorithms primarily rely on manual features to identify some typical defects. However, industrial surface defects are inherently complex and subtle. Their complexity lies in the scale variations and irregular shapes of defects within and between classes, while their subtlety stems from the ultra-shallow depth at the micrometer / submicrometer level, resulting in extremely low contrast between defects and the background in the image. Traditional manual features struggle to distinguish these subtle yet complex defects. In recent years, deep learning technologies, represented by Convolutional Neural Networks (CNNs), have gradually become a research focus in industrial surface defect detection due to their advantages in automatic feature extraction and end-to-end architecture, achieving considerable progress in both accuracy and efficiency.

[0003] However, existing deep learning-based ISDC algorithms still have two problems: First, most existing algorithms only target a few types of manufacturing defects such as scratches, pits, and bubbles, ignoring non-manufacturing defects such as residue and fingerprints. It's important to note that non-manufacturing defects are easily removed through further cleaning, allowing parts to be used directly without reprocessing; therefore, they should be distinguished from manufacturing defects. Second, existing ISDC algorithms struggle to balance accuracy and speed when dealing with subtle and complex surface defects, both of which are crucial for quickly identifying defective parts.

[0004] To address the above problems, this invention proposes a surface defect identification method based on multi-scale hybrid kernels and structural reparameterization. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, namely, the inability of existing surface defect identification methods to identify non-processed defects and the inability to balance accuracy and speed during identification, resulting in poor robustness, this invention proposes a surface defect identification method based on multi-scale hybrid kernels and structural reparameterization. This method includes:

[0006] S100: Acquire an image of the industrial component to be identified for surface defect recognition, as the input image;

[0007] S200, via M 2 Rep-Net obtains the surface defect identification results corresponding to the input image;

[0008] Wherein, the M 2 Rep-Net is a deep network based on multi-scale hybrid kernels and structural reparameterization. Its construction method is as follows:

[0009] Construct the initial M 2 Rep-Net; the initial M2Rep-Net is constructed based on sequentially connected MMK modules, a first max pooling layer, four cascaded AMK modules, a second max pooling layer, and a fully connected layer; the MMK module is a module constructed based on convolutional layers of multiple sizes; the AMK module is a module constructed based on asymmetric convolutional layers;

[0010] Construct a training dataset, and apply the initial M based on the training dataset. 2 Train Rep-Net;

[0011] For the initial M after training 2 Rep-Net performs structural reparameterization, and the initial M after structural reparameterization 2 Rep-Net is the final constructed M 2 Rep-Net.

[0012] In some preferred embodiments, the MMK module includes four parallel-connected convolutional layers, four batch normalization layers, one summation layer, and one ReLU function layer;

[0013] The four parallel convolutional layers have sizes of 1×1, 3×3, 5×5 and 7×7, and each convolutional layer is followed by a batch normalization layer;

[0014] The summation layer is used to sum the outputs of the four batch normalization layers and input them into the ReLU function layer.

[0015] In some preferred embodiments, the AMK module includes a first convolutional layer, a second convolutional layer, a summation layer, a first ReLU function layer, a third convolutional layer, a batch normalization layer, and a second ReLU function layer connected in sequence.

[0016] Both the first convolutional layer and the third convolutional layer are 1×1 convolutional layers;

[0017] The second convolutional layer includes four parallel convolutional layers with sizes of 1×1, 3×1, 1×3 and 3×3, and each convolutional layer is followed by a batch normalization layer;

[0018] The summation layer is used to sum the outputs of the four batch normalization layers.

[0019] In some preferred embodiments, the initial M after training 2 Rep-Net performs structural reparameterization using the following method:

[0020] The initial M 2 The convolutional layers and batch normalization layers in Rep-Net are fused together;

[0021] The MMK module is then structurally reparameterized.

[0022] The convolutional layers with sizes of 1×1, 3×3, and 5×5 in the MMK module are all padded with zeros to expand them into 7×7 convolutional layers. Then, the four 7×7 convolutional layers are stacked together as a new convolutional layer.

[0023] The AMK module is then structurally reparameterized.

[0024] The convolutional layers with dimensions of 1×1, 3×1, and 1×3 in the second convolutional layer are all padded with zeros to expand them into 3×3 convolutional layers. Then, the four 3×3 convolutional layers are stacked together as a new convolutional layer.

[0025] In some preferred embodiments, the initial M 2 In Rep-Net, convolutional layers and batch normalization layers are fused together, and the calculation method for the fused layer is as follows:

[0026]

[0027] Where k and b are the weights and biases of the convolutional layer, respectively; γ and β are the scaling and translation variables of the batch normalization layer, respectively; μ and σ are the mean and standard deviation of each batch of samples during training; x represents the input of the fused layer; and y BN(conv) This represents the output of the merged layer.

[0028] In some preferred embodiments, the new convolutional layer obtained after stacking in the MMK module outputs a feature map o. MMK Represented as:

[0029]

[0030]

[0031]

[0032] Where, k rep-MMK b represents the weights of the new convolutional layer obtained after stacking in the MMK module. rep-MMKPad represents the bias of the new convolutional layer obtained after stacking in the MMK module. i×i () indicates a convolutional layer padded with zeros to an i×i value, where I MMK This is the input feature map of the new convolutional layer obtained after stacking in the MMK module. Represents convolution operation. Represents the initial M 2 The weights and biases of the layer after fusing convolutional layers and batch normalization layers in Rep-Net.

[0033] In some preferred embodiments, the second convolutional layer, after being stacked, produces a new convolutional layer whose output feature map is o. AMK Represented as:

[0034]

[0035]

[0036]

[0037] Where k′ and b′ represent the weights and biases of the first convolutional layer, respectively, and k″ and b″ represent the weights and biases of the third convolutional layer, respectively. AMK k represents the input feature map of the new convolutional layer obtained after stacking the second convolutional layer. rep-AMK b rep-AMK This represents the weights and biases of the new convolutional layer obtained by stacking the second convolutional layer.

[0038] In a second aspect, the present invention proposes a surface defect recognition system based on multi-scale hybrid kernel and structural reparameterization, the system comprising: an input image acquisition module and a surface defect recognition module;

[0039] The input image acquisition module is configured to acquire an image of an industrial component to be identified for surface defect recognition, as the input image;

[0040] The surface defect identification module is configured to use M 2 Rep-Net obtains the surface defect identification results corresponding to the input image;

[0041] Wherein, the M 2 Rep-Net is a deep network based on multi-scale hybrid kernels and structural reparameterization. Its construction method is as follows:

[0042] Construct the initial M 2Rep-Net; the initial M2Rep-Net is constructed based on sequentially connected MMK modules, a first max pooling layer, four cascaded AMK modules, a second max pooling layer, and a fully connected layer; the MMK module is a module constructed based on convolutional layers of multiple sizes; the AMK module is a module constructed based on asymmetric convolutional layers;

[0043] Construct a training dataset, and apply the initial M based on the training dataset. 2 Train Rep-Net;

[0044] For the initial M after training 2 Rep-Net performs structural reparameterization, and the initial M after structural reparameterization 2 Rep-Net is the final constructed M 2 Rep-Net.

[0045] A third aspect of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor for implementing the aforementioned surface defect identification method based on multi-scale hybrid kernels and structural reparameterization.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions for execution by the computer to implement the above-described surface defect identification method based on multi-scale hybrid kernel and structural reparameterization.

[0047] The beneficial effects of this invention are:

[0048] This invention can identify non-processing defects in images of industrial parts and achieves a good balance between accuracy and speed.

[0049] 1) This invention proposes a deep network M based on multi-scale hybrid kernels and structural reparameterization. 2 Rep-Net, used to identify industrial surface defects, including both machined and unmachined defects, M 2 Rep-Net includes Multi-Scale Hybrid Convolutional Kernel (MMK) and Asymmetric Hybrid Convolutional Kernel (AMK). MMK extracts and fuses rich and sufficient multi-scale feature information in the shallow layers of the network to better characterize various surface defects with large differences in size. AMK obtains rotation-robust mid- and deep feature information by mixing four types of convolutional kernels: square, point, horizontal, and vertical, to better characterize various surface defects with irregular shapes.

[0050] 2) This invention introduces a structural reparameterization strategy, which modifies M... 2Rep-Net decouples training and inference, enabling the model to acquire strong representational capabilities through multi-branch learning during the training phase, and to be equivalently transformed into a single-branch model during the inference phase through reparameterization, thus enabling M... 2 Rep-Net achieves ultra-high inference speed while maintaining a lightweight model size and satisfactory recognition accuracy without sacrificing any performance. Experimental results show that M... 2 With only 5.23M parameters, Rep-Net achieves a recognition accuracy of 97.39% and an ultra-fast inference speed of 201.76 frames per second, realizing a good balance between accuracy and speed, and meeting the actual needs of industrial surface defect recognition tasks. Attached Figure Description

[0051] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0052] Figure 1 This is a flowchart illustrating a surface defect identification method based on multi-scale hybrid kernel and structural reparameterization according to an embodiment of the present invention.

[0053] Figure 2 This is a schematic diagram of the framework of a surface defect identification system based on multi-scale hybrid kernel and structural reparameterization according to an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of a typical industrial surface defect according to an embodiment of the present invention;

[0055] Figure 4 M is an embodiment of the present invention 2 A detailed diagram illustrating the Rep-Net architecture construction process;

[0056] Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing electronic devices according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0059] A surface defect identification method based on multi-scale hybrid kernel and structural reparameterization according to the first embodiment of the present invention, such as... Figure 1 As shown, the method includes:

[0060] S100: Acquire an image of the industrial component to be identified for surface defect recognition, as the input image;

[0061] S200, via M 2 Rep-Net obtains the surface defect identification results corresponding to the input image;

[0062] Wherein, the M 2 Rep-Net is a deep network based on multi-scale hybrid kernels and structural reparameterization. Its construction method is as follows:

[0063] Construct the initial M 2 Rep-Net; the initial M2Rep-Net is constructed based on sequentially connected MMK modules, a first max pooling layer, four cascaded AMK modules, a second max pooling layer, and a fully connected layer; the MMK module is a module constructed based on convolutional layers of multiple sizes; the AMK module is a module constructed based on asymmetric convolutional layers;

[0064] Construct a training dataset, and apply the initial M based on the training dataset. 2 Train Rep-Net;

[0065] For the initial M after training 2 Rep-Net performs structural reparameterization, and the initial M after structural reparameterization 2 Rep-Net is the final constructed M 2 Rep-Net.

[0066] To more clearly illustrate the surface defect identification method based on multi-scale hybrid kernel and structural reparameterization of the present invention, the steps of one embodiment of the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0067] In the following embodiments, M is first... 2 The construction process of Rep-Net is described in detail, and the process of obtaining the surface defect recognition results corresponding to the images of industrial parts through the surface defect recognition method based on multi-scale hybrid kernel and structural reparameterization is described in detail.

[0068] 1. M 2 The construction process of Rep-Net

[0069] This invention proposes a deep network based on multi-scale mixed kernels and structural re-parameterization (M). 2 Rep-Net achieves both ultra-high inference speed and a lightweight model size with satisfactory recognition accuracy. Specifically, this invention first designs a multi-size mixed convolutional kernel (MMK), i.e., convolutional layers of multiple sizes, to extract and fuse rich and sufficient multi-scale feature information in the shallow layers of the network, better representing various surface defects with large differences in size; secondly, it designs an asymmetric mixed convolutional kernel (AMK), i.e., an asymmetric convolutional layer, which obtains rotation-robust mid- and deep feature information by mixing four types of convolutional kernels: square, dot, horizontal, and vertical, to better represent various surface defects with irregular shapes; finally, it introduces a structural reparameterization strategy to further refine the MK kernel. 2 Rep-Net decouples training and inference, enabling the model to acquire strong representational capabilities based on multi-branch learning during the training phase, and to be equivalently transformed into a single-branch model through reparameterization during the inference phase, resulting in a lightweight structure and ultra-high inference speed.

[0070] Among them, M 2 The construction process of Rep-Net is as follows:

[0071] Construct the initial M 2 Rep-Net; the initial M 2 Rep-Net is based on sequentially connected MMK modules and a first max pooling layer ( Figure 4 The maximum pooling layer (MP) with a medium size of 112×112, four cascaded AMK modules, and a second maximum pooling layer (MP). Figure 4 The system is constructed using a 7×7 max-pooling layer and a fully connected layer (FC); the MMK module is a module constructed based on convolutional layers of multiple sizes; the AMK module is a module constructed based on asymmetric convolutional layers.

[0072] In this embodiment, as Figure 4 As shown, the MMK module is a module built based on convolutional layers of multiple sizes. The MMK module includes four parallel-connected convolutional layers (Conv), four batch normalization layers, one summation layer, and one ReLU function layer.

[0073] The four parallel convolutional layers have sizes of 1×1, 3×3, 5×5, and 7×7 to fully capture shallow features at different scales. Each convolutional layer is followed by a batch normalization layer; that is, the MMK module expands the network and enriches the M... 2 The receptive field at the input of Rep-Net is used to obtain multi-scale shallow features containing rich original defect information.

[0074] The summation layer is used to sum the outputs of the four batch normalization layers and input them into the ReLU function layer.

[0075] The AMK module is a module built on asymmetric convolutional layers, which parallelize square, horizontal, vertical, and point convolutions along with an identity connection. Figure 4 As shown, the AMK module includes a first convolutional layer, a second convolutional layer, a summation layer, a first ReLU function layer, a third convolutional layer, a batch normalization layer, and a second ReLU function layer connected in sequence.

[0076] Both the first convolutional layer and the third convolutional layer are 1×1 convolutional layers;

[0077] The second convolutional layer includes four parallel convolutional layers with sizes of 1×1, 3×1, 1×3 and 3×3, and each convolutional layer is followed by a batch normalization layer;

[0078] The summation layer is used to sum the outputs of the four batch normalization layers.

[0079] First, a 1×1 convolutional layer with a stride of 2 (s=2) is executed to downsampling resolution and control dimensionality. Then, four convolutions (3×3, 3×1, 1×3, and 1×1 respectively) and an identity connection are executed to extract multi-branch features, which are then fused through a subsequent 1×1 convolutional layer (the third convolutional layer). In M... 2 In Rep-Net, four AMK modules are cascaded to mine mid-to-deep defect features with rotational robustness, which can better characterize irregularly shaped defects.

[0080] Construct a training dataset, and apply the initial M based on the training dataset. 2 Train Rep-Net;

[0081] In this embodiment, an industrial surface defect dataset containing 8500 images (224×224) was established and named OD8500. Figure 3As shown, the OD8500 dataset includes seven categories: no defects (3088 images), scratches (731 images), broken edges (251 images), defects (2102 images), fingerprints (1205 images), pits (365 images), and bubbles (758 images). In the experiments, 75% and 25% of the samples from the OD8500 dataset were used as training and testing data, respectively.

[0082] Based on the training data and its corresponding ground truth labels, a training dataset is constructed.

[0083] The training images are input into the MMK module to extract multi-scale shallow features, and then passed through a series of AMK modules to obtain rotationally robust mid-to-deep features. Finally, the network outputs prediction results (i.e., surface defect identification results) from seven categories, including scratches, bubbles, pits, broken edges, marks, fingerprints, and no defects.

[0084] Based on the prediction results and the ground truth labels of the identification results, the loss value is calculated, and the initial M is updated. 2 Rep-Net network parameters;

[0085] Iteration on the initial M 2 Rep-Net training continues until a well-trained initial M is obtained. 2 Rep-Net. The preferred learning rate is 0.0001, the optimizer algorithm is Adam, the preferred momentum term is (0.9, 0.999), and the total number of iterations is 350.

[0086] For the initial M after training 2 Rep-Net performs structural reparameterization, and the initial M after structural reparameterization 2 Rep-Net is the final constructed M 2 Rep-Net.

[0087] In this embodiment, a structural reparameterization strategy is introduced to modify M. 2 Rep-Net decouples the inference and training phases. After training, it converts the multi-branch structures of MMK and AMK into equivalent single-branch structures, thus retaining the strong representational capabilities gained during training while simplifying model deployment and improving inference speed. 2 Rep-Net's structural reparameterization can be divided into three parts: the fusion of convolution and BN, the structural reparameterization of MMK, and the structural reparameterization of AMK. Details are as follows:

[0088] 1) The initial M 2In Rep-Net, convolutional layers and batch normalization layers are fused; that is, the four parallel-connected convolutional layers and their connected batch normalization layers in the MMK and AMK modules are fused together. Specifically:

[0089] The operations of convolutional layers and batch normalization (BN) layers are as follows:

[0090] y conv =k·x+b (1)

[0091]

[0092] Where k and b are the weights and biases of the convolutional layer, respectively; γ and β are the scaling and translation variables of the batch normalization layer, respectively; μ and σ are the mean and standard deviation of each batch (i.e., each batch of samples) during training; and x... i Let represent the input of the i-th BN layer, x represent the input of the convolutional layer, and also represent the input of the fused layer (which can be equivalently represented as a single convolutional layer), y represent the input of the convolutional layer. conv y BN These represent the outputs of the convolutional layer and the batch normalization (BN) layer, respectively.

[0093] Based on formulas (1) and (2), the convolutional layer is fused with the BN layer, that is, the fused layer can be equivalently represented as a single convolutional layer, as shown in formula (3).

[0094]

[0095] in, Indicates the initial M 2 The weights and biases of the layer after fusing convolutional and batch normalization layers in Rep-Net, y BN(conv) This represents the output of the merged layer. Therefore, through the above transformation, the BN layer is integrated into the convolutional layer.

[0096] 2) Perform structural reparameterization on the MMK module, specifically as follows:

[0097] The MMK module contains four parallel convolutional layers: 1×1, 3×3, 5×5, and 7×7, all with consistent input and output channels. First, the 1×1, 3×3, and 5×5 convolutional layers are padded with zeros to form 7×7 parameter matrices, effectively making them equivalent to special 7×7 convolutional layers (with many zero parameters in the kernel). Second, based on the additivity of convolution, the four 7×7 convolutional layers are stacked into a new 7×7 convolutional layer with the following weights and biases:

[0098]

[0099]

[0100] Where, krep-MMK b represents the weights of the new convolutional layer obtained after stacking in the MMK module. rep-MMK Pad represents the bias of the new convolutional layer obtained after stacking in the MMK module. i×i () indicates a convolutional layer padded with zeros to an i×i shape. This represents the weights and biases of a 1×1 convolutional layer fused with a connected BN layer; the rest are similar and will not be explained further here. Therefore, through structural reparameterization, the multi-branch MMK is equivalently transformed into a single-branch structure. For the input feature map I... MMK The output feature map after MMK reparameterization. MMK It can be represented as:

[0101]

[0102] in, This represents the convolution operation.

[0103] 3) Perform structural reparameterization on the AMK module, specifically as follows:

[0104] The second convolutional layer in the AMK module consists of four parallel convolutional layers: 3×3, 1×3, 3×1, and 1×1, with consistent input and output channels. Similar to reparameterized MMK, the 1×3, 3×1, and 1×1 convolutional layers are zero-padded to become 3×3 convolutional layers. Then, the four 3×3 convolutional layers are stacked and merged into a new 3×3 convolutional layer with the following weights and biases:

[0105]

[0106]

[0107] Where, k rep-AMK b rep-AMK This represents the weights and biases of the new convolutional layer obtained by stacking the second convolutional layer.

[0108] Through structural reparameterization, the multi-branch hybrid convolutional part of AMK can be equivalently transformed into a single convolution, that is, AMK is reparameterized into a single-branch structure. Therefore, for the input feature map I AMK The output feature map after AMK reparameterization. AMK It can be represented as:

[0109]

[0110] Where k′ and b′ represent the weights and biases of the first convolutional layer, respectively, and k″ and b″ represent the weights and biases of the third convolutional layer, respectively.

[0111] In summary, through reparameterization, M 2While retaining the strong representational power (weights) during the training phase, Rep-Net features a lightweight single-branch structure during the inference phase, thus achieving lower memory usage and higher actual inference speed.

[0112] 2. A surface defect identification method based on multi-scale hybrid kernel and structural reparameterization

[0113] S100: Acquire an image of the industrial component to be identified for surface defect recognition, as the input image;

[0114] In this embodiment, images of industrial parts whose surface defects need to be identified are first acquired.

[0115] S200, via M 2 Rep-Net obtains the surface defect recognition results corresponding to the input image.

[0116] In this embodiment, the image of the industrial component whose surface defects are to be identified is input into the pre-constructed M. 2 Rep-Net (i.e.) Figure 4 M in the middle reasoning stage (single branch) 2 Rep-Net was used to obtain the surface defect recognition results corresponding to the images of industrial parts.

[0117] To further verify the effectiveness of the present invention, the M proposed in this invention will be used... 2 The Rep-Net model was compared with well-known classification networks such as MobileNetV2, ShuffleNet V2, SqueezeNet, ResNet34 / 50 / 101, ResNeXt, RepVGG, and ConvNeXt on the OD8500 dataset, and the results are shown in Table 1. The Rep-Net model achieved a high average accuracy and average F1 score on the OD8500 dataset, demonstrating excellent recognition performance. Regarding accuracy, the M... 2 Rep-Net achieves the highest A* cc The accuracy (97.389%) and F1 score (97.386%) are 5.768% and 7.816% higher than the second-place ConvNeXt, respectively. In terms of efficiency, M2Rep-Net achieves the fastest inference speed.

[0118] Table 1

[0119] Accuracy represents the proportion of correctly classified samples out of the total number of samples. Precision represents the proportion of correctly predicted positive samples out of all samples predicted as positive. Recall represents the proportion of correctly predicted positive samples out of the actual number of positive samples. The F1 score is the harmonic mean of precision and recall, representing a comprehensive evaluation metric. Higher accuracy, precision, recall, and F1 scores indicate better model performance. The formulas for each metric are as follows:

[0120]

[0121]

[0122]

[0123]

[0124] Among them, C TP C TN C FP and C FN They are respectively true yang, true yin, false yang, and false yin, A CC P, R, and P represent accuracy, precision, and recall, respectively. Furthermore, this invention uses a multi-class confusion matrix evaluation model to evaluate the identification results of various defects. In summary, M... 2 Rep-Net, as a lightweight and efficient ISDC algorithm, can meet the practical needs of rapid screening of defective products in the industrial field.

[0125] A surface defect identification system based on multi-scale hybrid kernels and structural reparameterization according to a second embodiment of the present invention, such as... Figure 2 As shown, the system includes: an input image acquisition module 100 and a surface defect recognition module 200;

[0126] The input image acquisition module 100 is configured to acquire an image of an industrial component to be identified for surface defect recognition, as the input image;

[0127] The surface defect identification module 200 is configured to use M 2 Rep-Net obtains the surface defect identification results corresponding to the input image;

[0128] Wherein, the M 2 Rep-Net is a deep network based on multi-scale hybrid kernels and structural reparameterization. Its construction method is as follows:

[0129] Construct the initial M 2Rep-Net; the initial M2Rep-Net is constructed based on sequentially connected MMK modules, a first max pooling layer, four cascaded AMK modules, a second max pooling layer, and a fully connected layer; the MMK module is a module constructed based on convolutional layers of multiple sizes; the AMK module is a module constructed based on asymmetric convolutional layers;

[0130] Construct a training dataset, and apply the initial M based on the training dataset. 2 Train Rep-Net;

[0131] For the initial M after training 2 Rep-Net performs structural reparameterization, and the initial M after structural reparameterization 2 Rep-Net is the final constructed M 2 Rep-Net.

[0132] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the system described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0133] It should be noted that the surface defect identification system based on multi-scale hybrid kernels and structural reparameterization provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the various modules or steps and are not considered as an improper limitation of the present invention.

[0134] A third embodiment of the present invention provides an electronic device comprising at least one processor and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor to implement the aforementioned surface defect identification method based on multi-scale hybrid kernel and structural reparameterization.

[0135] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions, which are executed by the computer to implement the above-described surface defect identification method based on multi-scale hybrid kernel and structural reparameterization.

[0136] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the electronic devices and computer-readable storage media described above can be referred to the corresponding processes in the foregoing method examples, and will not be repeated here.

[0137] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system suitable for implementing the methods and apparatus embodiments of this application. Figure 5 The server shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0138] like Figure 5 As shown, the computer system includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in Read Only Memory (ROM) 502 or programs loaded from storage section 508 into Random Access Memory (RAM) 503. RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0139] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0140] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by the central processing unit (CPU501), it performs the functions defined in the method of this application. It should be noted that the computer-readable medium mentioned above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this application... In this context, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0141] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0143] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0144] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0145] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A surface defect identification method based on multi-scale hybrid kernel and structural reparameterization, characterized in that, The method includes: S100: Acquire an image of the industrial component to be identified for surface defect recognition, as the input image; S200, via M 2 Rep-Net obtains the surface defect identification results corresponding to the input image; Wherein, the M 2 Rep-Net is a deep network based on multi-scale hybrid kernels and structural reparameterization. Its construction method is as follows: Construct the initial M 2 Rep-Net; the initial M 2 Rep-Net is constructed based on sequentially connected MMK modules, a first max pooling layer, four cascaded AMK modules, a second max pooling layer, and a fully connected layer; the MMK modules are modules constructed based on convolutional layers of multiple sizes; the AMK modules are modules constructed based on asymmetric convolutional layers. Construct a training dataset, and apply the initial M based on the training dataset. 2 Train Rep-Net; For the initial M after training 2 Rep-Net performs structural reparameterization, and the initial M after structural reparameterization 2 Rep-Net is the final constructed M 2 Rep-Net; The MMK module includes four parallel-connected convolutional layers, four batch normalization layers, one summation layer, and one ReLU function layer; The four parallel convolutional layers have sizes of 1×1, 3×3, 5×5 and 7×7, and each convolutional layer is followed by a batch normalization layer; The summation layer is used to sum the outputs of the four batch normalization layers and input them into the ReLU function layer; The AMK module includes a first convolutional layer, a second convolutional layer, a summation layer, a first ReLU function layer, a third convolutional layer, a batch normalization layer, and a second ReLU function layer connected in sequence. Both the first convolutional layer and the third convolutional layer are 1×1 convolutional layers; The second convolutional layer includes four parallel convolutional layers with sizes of 1×1, 3×1, 1×3 and 3×3, and each convolutional layer is followed by a batch normalization layer; The summation layer is used to sum the outputs of the four batch normalization layers.

2. The surface defect identification method based on multi-scale hybrid kernel and structural reparameterization according to claim 1, characterized in that, For the initial M after training 2 Rep-Net performs structural reparameterization using the following method: The initial M 2 The convolutional layers and batch normalization layers in Rep-Net are fused together; The MMK module is then structurally reparameterized. The convolutional layers with sizes of 1×1, 3×3, and 5×5 in the MMK module are all padded with zeros to expand them into 7×7 convolutional layers. Then, the four 7×7 convolutional layers are stacked together as a new convolutional layer. The AMK module is then structurally reparameterized. The convolutional layers with dimensions of 1×1, 3×1, and 1×3 in the second convolutional layer are all padded with zeros to expand them into 3×3 convolutional layers. Then, the four 3×3 convolutional layers are stacked together as a new convolutional layer.

3. The surface defect identification method based on multi-scale hybrid kernel and structural reparameterization according to claim 2, characterized in that, The initial M 2 In Rep-Net, convolutional layers and batch normalization layers are fused together, and the calculation method for the fused layer is as follows: ； in, k and b These represent the weights and biases of the convolutional layer, respectively. γ and β These are the scaling and translation variables for the batch normalization layer, respectively. μ and σ For each batch of samples, the mean and standard deviation during the training process are given. This represents the input of the merged layer. This represents the output of the merged layer.

4. The surface defect identification method based on multi-scale hybrid kernel and structural reparameterization according to claim 3, characterized in that, The new convolutional layer obtained by stacking in the MMK module outputs a feature map. Represented as: ； ； ； in, This represents the weights of the new convolutional layer obtained after stacking in the MMK module. Pad represents the bias of the new convolutional layer obtained after stacking in the MMK module. i×i ( ) indicates a convolutional layer padded with zeros to an i×i shape. This is the input feature map of the new convolutional layer obtained after stacking in the MMK module. Represents convolution operation. , , , Represents the initial M 2 The weights and biases of the layer after fusing convolutional layers and batch normalization layers in Rep-Net; The initial M are respectively 2 In Rep-Net, the bias of the layer obtained by fusing the convolutional layers of the 1×1 convolutional branch, 3×3 convolutional branch, 5×5 convolutional branch, and 7×7 convolutional branch with the batch normalization layer; The zero-padding operator is used to pad the input convolutional kernel with zeros to expand it into a 7×7 convolutional layer; The initial M are respectively 2 In Rep-Net, the weights of the layers are obtained by fusing the convolutional layers of the 1×1, 3×3, 5×5, and 7×7 convolutional branches with the batch normalization layer.

5. The surface defect identification method based on multi-scale hybrid kernel and structural reparameterization according to claim 4, characterized in that, The second convolutional layer, when stacked, produces a new convolutional layer whose output feature map... Represented as: ； ； ； in, k , b That is to say These represent the weights and biases of the first convolutional layer, respectively. k , b That is to say These represent the weights and biases of the third convolutional layer, respectively. This represents the input feature map of the new convolutional layer obtained after stacking the second convolutional layer. , This represents the weights and biases of the new convolutional layer obtained by stacking the second convolutional layer; The zero-padding operator is used to pad the input convolutional kernel with zeros to expand it into a 3×3 convolutional layer; The initial M are respectively 2 In Rep-Net, the weights of the layers are obtained by fusing the convolutional layers of 1×3 convolutional branches, 3×1 convolutional branches, 1×1 convolutional branches, and 3×3 convolutional branches with the batch normalization layer. The initial M are respectively 2 In Rep-Net, the bias of the layer is obtained by fusing the convolutional layers of 3×3 convolutional branches, 1×3 convolutional branches, 3×1 convolutional branches, and 1×1 convolutional branches with the batch normalization layer.

6. A surface defect identification system based on multi-scale hybrid kernel and structural reparameterization, characterized in that, The system includes: an input image acquisition module and a surface defect recognition module; The input image acquisition module is configured to acquire an image of an industrial component to be identified for surface defect recognition, as the input image; The surface defect identification module is configured to use M 2 Rep-Net obtains the surface defect identification results corresponding to the input image; Wherein, the M 2 Rep-Net is a deep network based on multi-scale hybrid kernels and structural reparameterization. Its construction method is as follows: Construct the initial M 2 Rep-Net; the initial M 2 Rep-Net is constructed based on sequentially connected MMK modules, a first max pooling layer, four cascaded AMK modules, a second max pooling layer, and a fully connected layer; the MMK modules are modules constructed based on convolutional layers of multiple sizes; the AMK modules are modules constructed based on asymmetric convolutional layers. Construct a training dataset, and apply the initial M based on the training dataset. 2 Train Rep-Net; For the initial M after training 2 Rep-Net performs structural reparameterization, and the initial M after structural reparameterization 2 Rep-Net is the final constructed M 2 Rep-Net.

7. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to at least one of the processors; The memory stores instructions that can be executed by the processor to implement the surface defect identification method based on multi-scale hybrid kernel and structural reparameterization as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are executed by the computer to implement the surface defect identification method based on multi-scale hybrid kernel and structure reparameterization as described in any one of claims 1-5.

Images