Deep learning-based industrial part defect detection method and system, and electronic device

Abstract

The application provides an industrial part defect detection method and system based on deep learning and an electronic device; the method comprises the following steps: acquiring a data set; constructing a deep neural network for detecting surface defects of a target industrial part; training the deep neural network by using the data set, acquiring a trained target neural network, and performing surface defect detection on the target industrial part based on the target neural network; the application provides an industrial part defect detection method based on deep learning, which realizes the detection of surface defects of an industrial part based on a convolutional neural network and a Transformer, combines the advantages of the convolutional neural network and the Transformer, improves the segmentation accuracy, and adopts a parallel branch design, thereby ensuring the convergence speed during the training of the deep neural network and the time requirement during inference testing.

Description

Technical Field

[0001] This invention relates to the field of physics, and more particularly to surface defect detection technology for industrial parts, especially a method, system, and electronic device for detecting defects in industrial parts based on deep learning. Background Technology

[0002] In industrial production, limitations of existing technology and working conditions can severely reduce the quality of finished products. Among these, surface defects are a typical manifestation of reduced product quality. Therefore, in order to ensure the pass rate and reliable quality, it is necessary to conduct surface defect detection on products.

[0003] "Defect" can generally be understood as a missing part, defect, or area compared to a normal sample. Surface defect detection refers to detecting scratches, defects, foreign object obstruction, color contamination, holes, and other defects on the sample surface to obtain a series of relevant information such as the type, outline, location, and size of the defects on the tested sample surface. Manual defect detection used to be the mainstream method, with workers trained to identify complex surface defects, but this method is inefficient. The detection results are easily affected by subjective human factors and cannot meet the requirements of real-time detection. Therefore, realizing the automation of defect detection is a significant and challenging task.

[0004] Traditional machine vision methods require manual feature extraction to adapt to specific domains, and then make decisions according to manually defined rules or learnable classifiers (such as SVM, decision trees, etc.). This approach relies heavily on human experience and has a long development cycle, making it difficult to keep up with the pace of product iteration. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and electronic device for detecting defects in industrial parts based on deep learning, in order to solve the aforementioned problems existing in current product surface defect detection technologies.

[0006] To achieve the above and other related objectives, this invention provides a deep learning-based method for detecting defects in industrial parts, comprising the following steps: acquiring a dataset; the dataset including images of target surface defects in industrial parts; constructing a deep neural network for detecting surface defects in target industrial parts; the deep neural network including a fusion module, a Transformer branch, a CNN branch, and a decoder; the Transformer branch, the CNN branch, and the decoder are all connected to the fusion module; wherein, the fusion module is used to fuse a first result output by the Transformer branch and a second result output by the CNN branch, the decoder is used to decode a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network; training the deep neural network using the dataset to obtain a trained target neural network, and then using the target neural network to detect surface defects in the target industrial parts.

[0007] In one embodiment of the present invention, the acquisition of the dataset includes the following steps: acquiring the original surface defect image of the industrial part; preprocessing the original surface defect image to acquire the target surface defect image.

[0008] In one embodiment of the present invention, the fusion module is further configured to enhance the fourth result generated by fusing the first result and the second result to generate a fifth result, and to splice the fifth result, the first result and the second result together.

[0009] In one embodiment of the present invention, the number of fusion modules is three; the decoder includes: a first attention module, a second attention module, a first convolutional layer, a second convolutional layer, and a segmentation head. The process of the decoder decoding the third result includes: inputting the third result output by one of the fusion modules and the third result output by another fusion module into the first attention module; inputting the sixth result output by the first attention module into the first convolutional layer to obtain a seventh result; inputting the seventh result and the third result output by yet another fusion module into the second attention module; inputting the eighth result output by the second attention module into the second convolutional layer to obtain a ninth result; inputting the ninth result into the segmentation head; and the output of the segmentation head is used as the output of the decoder.

[0010] In one embodiment of the present invention, the segmentation head includes a third convolutional layer and a bilinear interpolation layer; wherein, in the process of inputting the ninth result to the segmentation head, the ninth result is first input to the third convolutional layer, and then the output of the third convolutional layer is restored to the original resolution through the bilinear interpolation layer.

[0011] In one embodiment of the present invention, training the deep neural network using the dataset includes the following steps: selecting training images from the dataset; inputting the training images into the deep neural network to train the deep neural network; and training the deep neural network by minimizing a loss function during the training process.

[0012] In one embodiment of the present invention, during the training of the deep neural network, the deep neural network is trained by iterative training; the training of the deep neural network using the dataset further includes the following steps: evaluating the deep neural network using the intersection-union score and / or the Dice score.

[0013] This invention provides a deep learning-based industrial part defect detection system, comprising: an acquisition module for acquiring a dataset including target surface defect images of industrial parts; a construction module for constructing a deep neural network for detecting surface defects of target industrial parts; the deep neural network including a fusion module, a Transformer branch, a CNN branch, and a decoder; the Transformer branch, the CNN branch, and the decoder are all connected to the fusion module; wherein the fusion module is used to fuse a first result output by the Transformer branch and a second result output by the CNN branch, the decoder is used to decode a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network; and a training module for training the deep neural network using the dataset to obtain a trained target neural network for detecting surface defects of the target industrial parts based on the target neural network.

[0014] The present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described deep learning-based industrial part defect detection method.

[0015] The present invention provides an electronic device, comprising: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the electronic device performs the above-described deep learning-based industrial part defect detection method.

[0016] As described above, the deep learning-based industrial part defect detection method, system, and electronic device of the present invention have the following beneficial effects:

[0017] (1) Compared with the prior art, the purpose of this invention is to solve the problem that it is difficult to capture long-distance dependency information when using only convolutional neural networks. It proposes a deep learning-based industrial part defect detection method, which introduces Transformer and attention module to improve segmentation accuracy.

[0018] (2) This invention provides a method for detecting defects in industrial parts based on deep learning. It detects surface defects in industrial parts by using convolutional neural networks and Transformers as theoretical foundations. It combines the advantages of convolutional neural networks and Transformers to improve segmentation accuracy. At the same time, it adopts a parallel branch design to ensure the convergence speed during training of deep neural networks and the time requirements during inference testing. Attached Figure Description

[0019] Figure 1 The flowchart shown is a flowchart of one embodiment of the deep learning-based industrial part defect detection method of the present invention.

[0020] Figure 2 The diagram shown is a framework diagram of the deep neural network of the present invention in one embodiment.

[0021] Figure 3 The diagram shown is a structural schematic of the fusion module of the present invention in one embodiment.

[0022] Figure 4 The diagram shown is a structural schematic of the AttentionGate module of the present invention in one embodiment.

[0023] Figure 5 The diagram shown is a structural schematic of the SCSE module of the present invention in one embodiment.

[0024] Figure 6 The diagram shown is a structural schematic of the ViT of the present invention in one embodiment.

[0025] Figure 7 The diagram shown is a structural schematic of ResNet34 in one embodiment of the present invention.

[0026] Figure 8 The flowchart shown is a flowchart of another embodiment of the deep learning-based industrial part defect detection method of the present invention.

[0027] Figure 9 The diagram shown is a structural schematic of an embodiment of the deep learning-based industrial part defect detection system of the present invention. Detailed Implementation

[0028] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0029] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0030] In the past decade, with the development of massive data analysis and learning technologies, deep neural networks have been applied to many visual recognition tasks. Compared with classic machine vision methods, deep learning can directly learn high-level features from data, thus having a higher ability to represent complex structures. This has enabled the automatic learning process to replace the manual engineering of features.

[0031] Convolution operations possess translation invariance, making them naturally suitable for image processing. However, their locality limits their focus to a specific region, making it difficult to capture long-range dependencies. With the recent success of Transformers in natural language processing, their advantage of effectively capturing global dependencies through global attention mechanisms has also been applied to computer vision. This invention complements the advantages of both methods and combines them with an attention module to construct a complete network structure for defect detection in industrial applications.

[0032] The following will explain the deep learning-based industrial part defect detection method of the present invention with reference to specific embodiments and accompanying drawings.

[0033] like Figure 1 As shown, the deep learning-based industrial part defect detection method of the present invention is applied to the detection of defects on the surface of ceramic tiles; specifically, the deep learning-based industrial part defect detection method includes the following steps:

[0034] Step S1: Use a camera to acquire images of surface defects in the tiles.

[0035] Step S2: Preprocess the surface defect image to obtain the dataset.

[0036] Specifically, all surface defect images were resized to 256×256×3, and random data augmentation operations such as rotation, translation, and scaling were performed on the images to obtain a dataset.

[0037] Simultaneously, the above operations are applied to the segmentation labels of the images to obtain the dataset.

[0038] Then, the dataset was randomly divided into 385 images for training and 31 images for inference testing.

[0039] Step S3: During training, each time an image I of dimension H×W×C is obtained from the dataset. img As input.

[0040] Where H and W represent image I img The height and width, C represents the image I. img The number of channels.

[0041] Specifically, during training, a pair of images is randomly selected from the dataset each time: one is a surface defect image, and the other is the corresponding segmentation label.

[0042] Step S4: Construct a deep neural network and then process the image I. img As input, train a deep neural network.

[0043] Reference Figure 2 During training, image I img Input into the network.

[0044] In one embodiment, the deep neural network includes: a Transformer branch, a CNN branch, a fusion module, and a decoder; wherein the Transformer branch adopts ViT (Vision Transformer, see details below) Figure 6 As shown in the figure, the CNN branch uses a residual structure (ResNet) as its backbone network.

[0045] It should be noted that, in order to combine the advantages of Transformer and CNN, namely that Transformer can capture global dependencies well through global attention mechanism, while CNN is better at capturing local details, feature maps with the same resolution in Transformer branch and CNN branch are input into the corresponding fusion module, and then the output of the fusion module is input into the decoder, and the output of the deep neural network is obtained by upsampling in sequence.

[0046] like Figure 7 As shown, in one embodiment, the CNN branch uses ResNet34 as the backbone network.

[0047] It should be noted that the Transformer branch uses ViT as the backbone network, which divides the image into several 16×16×3 parts and resizes each part into a one-dimensional vector. Each one-dimensional vector is then linearly transformed to obtain a one-dimensional vector of length 384, which is then input into ViT. The several vectors output by ViT are then concatenated into a three-dimensional feature map. The resolution of the feature map is improved and the number of channels is reduced by upsampling twice, resulting in outputs with different resolutions.

[0048] For the CNN branch, ResNet is used as the backbone network. Specifically, after the image is input into ResNet, it first goes through a convolutional layer with a kernel size of 7 and a stride of 2, followed by a ReLU activation function, downsampling using max pooling, and then through three residual blocks, obtaining outputs of different resolutions corresponding to each residual block.

[0049] The feature information of the image is captured by the Transformer branch and the CNN branch. Then, the feature information is fused by the fusion module, and the decoder decodes the output from the fusion module.

[0050] The whole process is as follows Figure 2 As shown, this embodiment takes a two-dimensional image as an example. The input image size is 256×256×3. For the Transformer branch, the Transformer output vector is reshaped to 16×16×384. Then, through two bilinear interpolations and deconvolution, output feature maps with sizes of 32×32×128 and 64×64×64 are obtained respectively.

[0051] For the CNN branch, the first convolutional block has 64 convolutional kernels with a kernel size of 7 and a stride of 2, followed by a ReLU activation function and max pooling. After the first convolutional block, a feature map of size 64×64×64 is obtained. Then there are three consecutive residual blocks, each with a convolutional kernel size of 3. Except for the first residual block, which has a stride of 1, the stride of the others is 2. The number of convolutional kernels are 64, 128, and 256, respectively. The three residual blocks produce output feature maps of sizes 64×64×64, 32×32×128, and 16×16×256, respectively.

[0052] After obtaining the feature maps output from the CNN branch and the Transformer branch, a pair of feature maps with the same resolution are input into the fusion module. The structure of the fusion module is as follows: Figure 3As shown, the feature maps output by the CNN branch and the Transformer branch are respectively passed through convolutional layers with a kernel size of 1 to align the number of channels, then added together, and then passed through SCSEBlock (SpatialSqueeze and Channel Excitation Block). The output is then concatenated with the feature maps output by the CNN branch and the Transformer branch. Finally, the concatenated result is input into the Residual block to obtain the final output of the fusion module.

[0053] The SCSEBlock structure is as follows: Figure 5 As shown.

[0054] SCSEBlock comprises two branches: the SSE branch and the CSE branch. For the SSE branch, the input feature map passes through a single convolutional layer with a kernel size of 1, resulting in a 2D feature map. This 2D feature map is then multiplied by the input feature map along its spatial dimension to obtain the output of the SSE branch. For the CSE branch, the input feature map passes through an average pooling layer to obtain a 1D vector. This vector is then passed through two convolutional layers with a kernel size of 1, reducing its dimension before restoring it to a 1D vector with the same dimension as the number of channels. This CSE vector is then multiplied by the input feature map along its channel dimension to obtain the output of the CSE branch. Finally, the outputs of the SSE and CSE branches are added together to obtain the final output of SCSEBlock.

[0055] It should be noted that, in order to further improve the capture of global dependencies by the Transformer branch and the capture of local details by the CNN branch, this invention further enhances the features captured by the Transformer branch and the CNN branch by using an additional attention module in the fusion module. Specifically, the SCSE Block is used to enhance the result after fusing the outputs of the Transformer branch and the CNN branch. Finally, the enhanced feature maps are concatenated and passed through the Residual Block to obtain the output of the fusion module, which is then input into the final decoder.

[0056] In the decoding process of the decoder of this invention, the feature map output by the i-th fusion module is denoted as f. i Then the feature map output by the (i+1)th layer of the decoder is and Where Conv is a convolutional layer, Up is upsampling, and AG is an attention module.

[0057] In one embodiment, the AttentionGate module (AG module) is used in the decoding process of the present invention to further improve the final segmentation result. The deep feature map has a larger receptive field for each pixel, which can better focus on more global information. The module uses the information extracted from the deep feature map as an attention mechanism to apply to the shallow feature map, so that the shallow feature map can focus on more global information.

[0058] Let the feature map output by the i-th decoder be... The feature map output by the (i+1)th fusion module is f i+1 Then the output of the AG module is and Wherein, Conv is a convolutional layer with a kernel size of 1 and a stride of 1, and Up is a bilinear interpolation upsampling layer.

[0059] Specifically, the decoder consists of two AttentionGate modules, two convolutional layers, and a segmentation head. The structure of the AttentionGate module is as follows: Figure 4 As shown, the shallow feature map first passes through a convolutional layer with a kernel size of 1 to align the number of channels with the deep feature map. Then, it is downsampled to align the resolution with the deep feature map. The result is then added to the deep feature map, passed through a ReLU layer, and then through a convolutional layer with a kernel size of 1 to restore the number of channels to the original number. After passing through a sigmoid layer, the result is upsampled back to the resolution of the shallow feature map. The result is the attention feature map. Multiplying the attention feature map with the original shallow feature map gives the output of the AttentionGate module.

[0060] The decoder will output the fusion module's output. The f1 input is fed into the AttentionGate module, and the output is then passed through a convolutional layer to obtain... Then The input to f2 is another AttentionGate module, and the output is then passed through a convolutional layer to obtain... Finally The final segmentation result is obtained after passing through the segmentation head.

[0061] The segmentation head consists of convolutional layers and bilinear interpolation. Bilinear interpolation restores the output of the convolutional layers to the original resolution and obtains the final segmentation result.

[0062] In one embodiment, a deep neural network is trained by minimizing a loss function.

[0063] It should be noted that the loss function minimized is derived from the energy function of the traditional registration method:

[0064]

[0065] Among them, L total Deep supervision is used to improve the final segmentation accuracy. α, β, and γ are variable hyperparameters, G is the segmentation label, and head is the prediction head. The input is transformed into the segmentation result, t. i This is the result of the Transformer branch's output after i upsampling.

[0066] L = L IoU +L bce ;

[0067] in,

[0068]

[0069] Where y is the delimiter label. This is the output of the deep neural network.

[0070] In this embodiment, the number of epochs is set to 200, and the Adam optimizer is used to drive network optimization. total In the model, α, β, and γ are 0.5, 0.3, and 0.2, respectively. After completing the required number of iterations, the final model is obtained.

[0071] In one embodiment, the deep neural network is trained iteratively.

[0072] In one embodiment, for a trained deep neural network, the IOU (Intersection Over Union) score and the Dice score are used as indicators of defect segmentation performance.

[0073] It should be noted that the inference test is performed using a pre-trained deep neural network; specifically, during the test, one image is selected sequentially from the test set as input, and the corresponding segmentation label for each image is also input.

[0074] In this embodiment, the 2D image has 31 segmentation labels. The test network outputs the segmentation results and segmentation evaluation metrics. The evaluation metric is expressed as the IOU score, which is: Dice rating, expressed as: Where y is the delimiter label. For network output.

[0075] It should be noted that the IoU coefficient and the Dice coefficient (the Dice coefficient is a set similarity measure function, usually used to calculate the similarity between two samples) are set similarity metrics used to calculate the similarity between two samples. The values ​​range from [0,1]. The better the segmentation effect, the closer the IoU value and the Dice value are to 1.

[0076] This invention discloses a deep learning-based method for detecting defects in industrial parts. First, images of the industrial part's surface are acquired using a camera and subjected to data augmentation operations such as rotation, translation, and scaling to obtain preprocessed images. Then, the preprocessed images are input into a Transformer network and a Convolutional Neural Network (CNN) for feature extraction and fusion. Next, feature maps are transformed back to their original size using feature upsampling technology and input into a prediction layer to obtain the final semantic segmentation result. The training process is supervised, using IOU and Dice loss functions for iterative training and parameter optimization until the model parameters converge, and the model parameter file is saved. During testing, an industrial part defect image with a size of 224×224×3 is input, and the trained deep neural network is tested on it, ultimately obtaining a segmentation result with a size of 224×224×1. This invention combines the advantages of Transformer and CNN to achieve more accurate defect segmentation on industrial part datasets, improving the accuracy of the final segmentation result.

[0077] Existing solutions typically employ only convolutional neural networks. This invention, in addition to using convolutional neural networks, also introduces a Transformer and adds attention modules to the fusion module and decoder. This increases the weight of useful information, suppresses the influence of noise, and improves segmentation accuracy.

[0078] This invention relates to a deep learning-based method for detecting defects in industrial parts, and is an implementation method based on convolutional neural networks and Transformers. This invention combines the advantages of convolutional neural networks and Transformers to improve segmentation accuracy, while employing a parallel branching design to ensure fast convergence during training and meet the time requirements during inference and testing.

[0079] like Figure 8 As shown, in one embodiment, the deep learning-based industrial part defect detection method of the present invention includes the following steps:

[0080] Step H1: Obtain the dataset.

[0081] It should be noted that the dataset includes images of target surface defects of industrial parts.

[0082] In one embodiment, obtaining the dataset includes the following steps:

[0083] Step (11): Obtain the original surface defect image of the industrial part.

[0084] Step (12): Preprocess the original surface defect image to obtain the target surface defect image.

[0085] In one embodiment, the preprocessing in step (12) includes image enhancement processing; wherein the image enhancement processing includes at least, but is not limited to, any one of the following processing methods: rotation, translation, scaling.

[0086] Step H2: Construct a deep neural network for detecting surface defects of target industrial parts.

[0087] In this embodiment, the deep neural network includes: a fusion module, a Transformer branch, a CNN branch, and a decoder; the Transformer branch, the CNN branch, and the decoder are all connected to the fusion module; wherein, the fusion module is used to fuse the first result output by the Transformer branch and the second result output by the CNN branch, the decoder is used to decode the third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network.

[0088] In one embodiment, the fusion module is further configured to enhance the fourth result generated by fusing the first result and the second result to generate a fifth result, and to splice the fifth result, the first result, and the second result together.

[0089] In one embodiment, the number of fusion modules is three; the decoder includes: a first attention module, a second attention module, a first convolutional layer, a second convolutional layer, and a segmentation head, and the process of the decoder decoding the third result includes:

[0090] Step (21): The third result output by the fusion module. The third result f1, output by the other fusion module, is input to the first attention module.

[0091] Step (22): Input the sixth result output by the first attention module into the first convolutional layer to obtain the seventh result.

[0092] Step (23): The seventh result The third result f2, output by the fusion module described above, is input to the second attention module.

[0093] Step (24): Input the eighth result output by the second attention module into the second convolutional layer to obtain the ninth result.

[0094] Step (25): The ninth result The input is fed into the segmentation head; the output of the segmentation head is used as the output of the decoder.

[0095] In one embodiment, the segmentation head includes a third convolutional layer and a bilinear interpolation layer; wherein, during the process of inputting the ninth result into the segmentation head, the ninth result is first input into the third convolutional layer, and then the output of the third convolutional layer is restored to the original resolution via the bilinear interpolation layer.

[0096] Step H3: Use the dataset to train the deep neural network and obtain the trained target neural network to perform surface defect detection on the target industrial part based on the target neural network.

[0097] In one embodiment, training the deep neural network using the dataset includes the following steps:

[0098] Step (31): Select training images from the dataset.

[0099] Step (32): Input the training image into the deep neural network to train the deep neural network.

[0100] In one embodiment, the deep neural network is trained by minimizing a loss function during the training process.

[0101] In one embodiment, the deep neural network is trained through iterative training during the training process.

[0102] In one embodiment, training the deep neural network using the dataset further includes the step of evaluating the deep neural network using an intersection-union score and / or a Dice score.

[0103] In one embodiment, surface defect detection of the target industrial part based on the target neural network includes: inputting a target surface defect image of the target industrial part into the target neural network to realize the detection of surface defects of the target industrial part by the target neural network; the output of the target neural network is the result of its detection of surface defects of the target industrial part.

[0104] It should be noted that the working principle of the deep learning-based industrial part defect detection method provided in this embodiment can be referred to the introduction of the deep learning-based industrial part defect detection method in the above specific embodiments, and will not be repeated in detail here.

[0105] It should be noted that the scope of protection of the deep learning-based industrial part defect detection method described in this invention is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0106] The storage medium of this invention stores a computer program, which, when executed by a processor, implements the aforementioned deep learning-based industrial part defect detection method. The storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disk, USB flash drive, memory card, or optical disk.

[0107] One or more storage media may be used in any combination. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by or in connection with an instruction execution system, apparatus, or device.

[0108] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0109] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0110] Computer program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0111] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, these computer program instructions create means for implementing the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams.

[0112] These computer program instructions may also be stored in a computer-readable medium, which causes a computer, other programmable data processing apparatus, or other device to operate in a particular manner, thereby producing an article of manufacture that includes instructions for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0113] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus provide a process for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0114] The electronic device of the present invention includes a processor and a memory.

[0115] The memory is used to store computer programs; preferably, the memory includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card or optical disk.

[0116] The processor is connected to the memory and is used to execute the computer program stored in the memory, so that the electronic device performs the above-described deep learning-based industrial part defect detection method.

[0117] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0118] like Figure 9 As shown, in one embodiment, the deep learning-based industrial part defect detection system of the present invention includes:

[0119] Acquisition module 91 is used to acquire a dataset; the dataset includes images of target surface defects of industrial parts.

[0120] Module 92 is used to construct a deep neural network for detecting surface defects of target industrial parts; the deep neural network includes: a fusion module, a Transformer branch, a CNN branch, and a decoder; the Transformer branch, the CNN branch, and the decoder are all connected to the fusion module; wherein, the fusion module is used to fuse the first result output by the Transformer branch and the second result output by the CNN branch, the decoder is used to decode the third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network.

[0121] Training module 93 is used to train the deep neural network using the dataset, obtain the trained target neural network, and perform surface defect detection on the target industrial part based on the target neural network.

[0122] It should be noted that the structure and principle of the acquisition module 91, the construction module 92 and the training module 93 correspond one-to-one with the steps (steps H1 to H3) in the above-mentioned deep learning-based industrial part defect detection method, so they will not be repeated here.

[0123] It should be noted that the division of the various modules in the above system is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, module x can be a separate processing element, or it can be integrated into a chip within the system. Alternatively, it can be stored as program code in the system's memory, and its function can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0124] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-On-a-Chip (SOC).

[0125] It should be noted that the deep learning-based industrial part defect detection system of the present invention can implement the deep learning-based industrial part defect detection method of the present invention. However, the implementation device of the deep learning-based industrial part defect detection method of the present invention includes, but is not limited to, the structure of the deep learning-based industrial part defect detection system listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of the present invention are included within the protection scope of the present invention.

[0126] In summary, the present invention, a deep learning-based method, system, and electronic device for detecting defects in industrial parts, addresses the problem of capturing long-distance dependency information when using only convolutional neural networks. It proposes a deep learning-based method for detecting defects in industrial parts, introducing a Transformer and an attention module to improve segmentation accuracy. This invention provides a deep learning-based method for detecting defects in industrial parts, using convolutional neural networks and Transformers as theoretical foundations to detect surface defects in industrial parts. It combines the advantages of both convolutional neural networks and Transformers, improving segmentation accuracy. Furthermore, the use of parallel branching ensures fast convergence during deep neural network training and meets the time requirements for inference testing. Therefore, this invention effectively overcomes the various shortcomings of existing technologies and has high industrial applicability.

[0127] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A deep learning-based industrial part defect detection method, characterized in that, The method comprises the following steps: obtaining a data set; the data set comprises target surface defect images of industrial parts; constructing a deep neural network for detecting surface defects of target industrial parts; the deep neural network comprises a fusion module, a Transformer branch, a CNN branch and a decoder; the Transformer branch, the CNN branch and the decoder are all connected with the fusion module; wherein the fusion module is used for fusing a first result output by the Transformer branch and a second result output by the CNN branch, and the decoder is used for decoding a third result output by the fusion module, and an output of the decoder is used as an output of the deep neural network; the number of the fusion modules is three; the number of the first results, the number of the second results and the number of the third results are all three; the three first results are feature maps t0, t1 and t2 respectively; the three second results are feature maps g0, g1 and g2 respectively; the three third results are feature maps f0, f1 and f2 respectively; one fusion module is used for fusing the feature map t0 and the feature map g0; another fusion module is used for fusing the feature map t1 and the feature map g1; and the other fusion module is used for fusing the feature map t2 and the feature map g2; the decoder comprises a first attention module, a second attention module, a first convolutional layer, a second convolutional layer and a segmentation head, and a decoding process of the decoder on the third results comprises: inputting the third result output by one fusion module and the third result output by another fusion module into the first attention module; the third result output by one fusion module is the feature map f0; the third result output by another fusion module is the feature map f1; inputting a sixth result output by the first attention module into the first convolutional layer to obtain a seventh result; inputting the seventh result and the third result output by the other fusion module into the second attention module; the third result output by the other fusion module is the feature map f2; inputting an eighth result output by the second attention module into the second convolutional layer to obtain a ninth result; inputting the ninth result into the segmentation head; an output of the segmentation head is used as an output of the decoder; training the deep neural network by using the data set to obtain a trained target neural network, so as to detect surface defects of the target industrial parts based on the target neural network. 2.The deep learning-based industrial part defect detection method of claim 1, wherein, The step of obtaining the data set comprises the following steps: obtaining original surface defect images of industrial parts; preprocessing the original surface defect images to obtain target surface defect images. 3.The deep learning-based industrial part defect detection method of claim 1, wherein, The fusion module is further used for enhancing a fourth result generated after the fusion of the first result and the second result to generate a fifth result, and for splicing the fifth result, the first result and the second result. 4.The deep learning-based industrial part defect detection method of claim 1, wherein, The segmentation head comprises a third convolutional layer and a bilinear interpolation layer; wherein, in the process of inputting the ninth result into the segmentation head, the ninth result is first input into the third convolutional layer, and then the output of the third convolutional layer is restored to the original resolution through the bilinear interpolation layer. 5.The deep learning based industrial part defect detection method of claim 1, wherein, The training of the deep neural network using the data set comprises the following steps: selecting a training image from the data set; inputting the training image into the deep neural network to train the deep neural network; in the process of training the deep neural network, the deep neural network is trained by minimizing a loss function. 6.The deep learning-based industrial part defect detection method of claim 1, wherein, in the process of training the deep neural network, the deep neural network is trained by iterative training. The training of the deep neural network using the data set further comprises the following steps: evaluating the deep neural network using a Jaccard index score and / or a Dice score.

7. A deep learning based industrial part defect detection system implemented using the method of claim 1, wherein, comprising: an acquisition module configured to acquire a data set; the data set comprising target surface defect images of industrial parts; a construction module configured to construct a deep neural network for detecting surface defects of target industrial parts; the deep neural network comprises a fusion module, a Transformer branch, a CNN branch and a decoder; the Transformer branch, the CNN branch and the decoder are all connected with the fusion module; wherein, the fusion module is configured to fuse a first result output by the Transformer branch and a second result output by the CNN branch, the decoder is configured to decode a third result output by the fusion module, and the output of the decoder is taken as the output of the deep neural network; a training module configured to train the deep neural network using the data set, acquire a trained target neural network, and detect surface defects of the target industrial parts based on the target neural network.

8. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by a processor to implement the deep learning-based industrial part defect detection method of any one of claims 1-6.

9. An electronic device, comprising: comprising: a processor and a memory; the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory, so that the electronic device executes the deep learning-based industrial part defect detection method of any one of claims 1-6.

Images