Surface defect detection method and device for industrial products
By using defect-free image training in neural networks, map the feature images of the product image to be detected to multi-dimensional space, solving the problem of low recognition efficiency and accuracy in surface defect detection of industrial products, and achieving more efficient and accurate defect detection.
Patent Information
- Application Number
- CN202111266611.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-28
- Publication Date
- 2025-03-25
- Estimated Expiration
- 2041-10-28
AI Technical Summary
In the detection of surface defects of industrial products, the prior art results in poor identification efficiency and accuracy due to the scarcity of defect samples and large characteristics.
By inputting defect-free images into neural networks for training, the minimum supersphere of the branch corresponding to each residual block is obtained in advance, and then the product image to be detected is input to the neural network, the feature image is obtained and mapped to the multidimensional space where the minimum supersphere is located, the defect score is determined based on the distance, and the defect detection result is finally determined.
It improves the recognition efficiency and accuracy of surface defects of industrial products, avoids the difficulty of requiring a large number of defect samples to train neural networks, alleviates the problem of gradient diffusion, and improves the reliability of detection through multi-angle recognition.
Smart Images
Figure CN114140388B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to a method and device for detecting surface defects of industrial products. Background Art
[0002] When industrial products, such as nanomaterials, mobile phone screens, fabrics, castings, etc., have defects of various shapes on their surfaces, it will affect the economic value of the products and even cause safety hazards. Therefore, defect detection of industrial products is a crucial part of industrial product quality inspection.
[0003] Since the surface defects of industrial products may be irregular texture surface defects, and such defects have characteristics such as anisotropy, nonlinearity, non-uniformity, and easy confusion, they are difficult to detect. Therefore, in the related art, a large number of defect samples of industrial products are used for training through a deep learning method, so that the surface defects of industrial products can be identified through a deep learning neural network obtained after training based on a large number of defect samples of industrial products.
[0004] However, due to the scarcity of defect samples of most industrial products and the large differences in features between defect samples, the efficiency and accuracy of defect recognition are poor when using related technologies to identify surface defects of industrial products. Summary of the invention
[0005] The embodiments of the present application provide a method and device for detecting surface defects of industrial products, which improve the efficiency and accuracy of identifying surface defects of industrial products.
[0006] In a first aspect, an embodiment of the present application provides a method for detecting surface defects of an industrial product, comprising:
[0007] Inputting the image of the product to be inspected into the neural network, and obtaining each feature image corresponding to each residual block in the neural network;
[0008] Determining branches of each of the feature images according to the residual blocks corresponding to each of the feature images;
[0009] Mapping the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determining a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space;
[0010] Determining a defect detection result of the product image to be detected according to the defect score of each of the feature images;
[0011] The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network.
[0012] In one embodiment, it further includes:
[0013] Inputting each of the defect-free images into a neural network to obtain each output image set corresponding to each of the residual blocks;
[0014] Determining a branch of each output image set according to the residual block corresponding to each output image set;
[0015] Inputting the output image set into the neural network for training, and obtaining network parameters and hypersphere centers corresponding to branches of the output image set;
[0016] Determining a minimum hypersphere corresponding to a branch of the output image set according to the output image set, the network parameters and the center of the hypersphere;
[0017] The output image set includes output images corresponding one-to-one to each of the defect-free images.
[0018] In one embodiment, each of the defect-free images is input into a neural network to obtain each output image set corresponding to each of the residual blocks, including:
[0019] Inputting each of the defect-free images into a neural network, and obtaining each target image set corresponding to each residual block through each residual block of the neural network;
[0020] Extracting each first feature information of any target image in the target image set, and acquiring a first adjustment matrix of a residual block corresponding to the target image according to the cosine similarity between each first feature information;
[0021] Acquire the output image according to the first adjustment matrix and the defect-free image;
[0022] The output images corresponding to the same residual block are combined into the output image set to obtain output image sets corresponding to the residual blocks one by one.
[0023] In one embodiment, the image of the product to be detected is input into a neural network, and each feature image corresponding to each residual block in the neural network is obtained, including:
[0024] Inputting the image of the product to be inspected into the neural network, and obtaining initial images corresponding to the residual blocks through the residual blocks of the neural network;
[0025] Extracting each second feature information in the initial image, and acquiring a second adjustment matrix of the residual block corresponding to the initial image according to the cosine similarity between each second feature information;
[0026] The feature image is acquired according to the second adjustment matrix and the image of the product to be detected.
[0027] In one embodiment, the image of the product to be inspected is input into a neural network, and each initial image corresponding to each residual block is obtained through each residual block of the neural network, including:
[0028] Inputting the image of the product to be detected into the neural network for convolution operation and maximum pooling operation, and adjusting the image of the product to be detected;
[0029] The adjusted product image to be inspected is passed through the residual blocks of the neural network to obtain initial images corresponding to the residual blocks.
[0030] In one embodiment, determining the defect score of the feature image according to the distance between the feature image and the minimum hypersphere in the multidimensional space includes:
[0031] Obtaining a distance difference between the characteristic image and the center of the minimum hypersphere;
[0032] A defect score of the feature image is determined according to the distance difference.
[0033] In one embodiment, determining the defect detection result of the product image to be detected according to the defect score of each of the feature images includes:
[0034] According to the preset weights of the branches of each of the feature images, the defect scores of each of the feature images are weighted to obtain a target score;
[0035] A defect detection result of the product image to be inspected is determined according to the target score.
[0036] In a second aspect, an embodiment of the present application provides a surface defect detection device for an industrial product, comprising:
[0037] A feature image acquisition module, used for inputting the image of the product to be detected into the neural network, and acquiring each feature image corresponding to each residual block in the neural network;
[0038] A feature branch determination module, used to determine the branch of each feature image according to the residual block corresponding to each feature image;
[0039] a defect score determination module, used to map the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determine a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space;
[0040] An image defect detection module, used to determine a defect detection result of the image of the product to be detected according to the defect score of each of the feature images;
[0041] The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network.
[0042] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory storing a computer program, wherein when the processor executes the program, the steps of the surface defect detection method for industrial products described in the first aspect are implemented.
[0043] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the surface defect detection method for industrial products described in the first aspect.
[0044] The surface defect detection method and device of industrial products provided in the embodiment of the present application, by inputting a defect-free image into a neural network for training, obtaining in advance the minimum hypersphere of the branch corresponding to each residual block, inputting the image of the product to be detected into the neural network, obtaining the feature image of the branch corresponding to each residual block of the image of the product to be detected, and mapping the feature image to the multidimensional space where the minimum hypersphere corresponding to the branch of the feature image is located, and determining the defect score of the feature image according to the mapping result, so as to determine the defect detection result of the image of the product to be detected according to the defect score of each feature image. Thus, the surface defect recognition of industrial products is performed by the neural network obtained by training with a large number of defect-free samples of industrial products that are easily obtained, without obtaining all types of defects, overcoming the problem of requiring a large number of defect samples to train the neural network, resulting in poor training efficiency and defect recognition accuracy, and improving the recognition efficiency and accuracy of surface defects of industrial products. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0046] Figure 1 It is a schematic flow chart of a surface defect detection method for industrial products provided in an embodiment of the present application;
[0047] Figure 2 It is a schematic diagram of the structure of a surface defect detection device for industrial products provided in an embodiment of the present application;
[0048] Figure 3is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application; DETAILED DESCRIPTION
[0049] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0050] The embodiments of the present application are described in detail below with reference to the accompanying drawings.
[0051] See also Figure 1 , is one of the flow charts of the surface defect detection method of industrial products provided by an embodiment of the present invention, the method is applied in a server and is used to detect defects on the surface of industrial products. Figure 1 As shown, the surface defect detection method of an industrial product provided in this embodiment includes:
[0052] Step 101, inputting the product image to be detected into the neural network, and obtaining each feature image corresponding to each residual block in the neural network;
[0053] Step 102, determining a branch of each of the feature images according to the residual block corresponding to each of the feature images;
[0054] Step 103, mapping the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determining a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space;
[0055] Step 104, determining a defect detection result of the product image to be detected according to the defect score of each of the feature images;
[0056] The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network.
[0057] By inputting a defect-free image into a neural network for training, the minimum hypersphere of the branch corresponding to each residual block is obtained in advance, and then the image of the product to be detected is input into the neural network to obtain the feature image of the branch corresponding to each residual block of the product image to be detected, and the feature image is mapped to the multidimensional space where the minimum hypersphere corresponding to the branch of the feature image is located, and the defect score of the feature image is determined according to the mapping result, so as to determine the defect detection result of the product image to be detected according to the defect score of each feature image. Thus, the surface defect recognition of industrial products is performed by the neural network obtained by training with a large number of defect-free samples of industrial products that are easily obtained, without obtaining all types of defects, overcoming the problem of requiring a large number of defect samples to train the neural network, resulting in poor training efficiency and defect recognition accuracy, and improving the recognition efficiency and accuracy of surface defects of industrial products.
[0058] Moreover, the defect scores are determined by the branches corresponding to the multiple residual blocks, and then the defect detection results of the product image to be inspected are determined by using the defect scores of the branches corresponding to each residual block, so that when performing defect detection, the problem of gradient diffusion in the neural network can be alleviated. At the same time, since the depth of the neural network is increased by using multiple residual blocks, feature images of multiple branches can be generated, and multi-angle defect recognition can be performed on the product image to be inspected, which can further improve the recognition efficiency and accuracy of surface defects of industrial products.
[0059] In one embodiment, before step 101, the method further includes:
[0060] Inputting each of the defect-free images into a neural network to obtain each output image set corresponding to each of the residual blocks;
[0061] Determining a branch of each output image set according to the residual block corresponding to each output image set;
[0062] Inputting the output image set into the neural network for training, and obtaining network parameters and hypersphere centers corresponding to branches of the output image set;
[0063] Determining a minimum hypersphere corresponding to a branch of the output image set according to the output image set, the network parameters and the center of the hypersphere;
[0064] The output image set includes output images corresponding one-to-one to each of the defect-free images.
[0065] In one embodiment, a large number of defect-free images, such as 200 defect-free images, are obtained in advance as training sets and input into the neural network for training, and each defect-free image has the same size. For any defect-free image, after being input into the neural network, a convolution operation may be performed in sequence, and then a maximum pooling operation may be performed in sequence, and then the defect-free image that has undergone the convolution operation and the maximum pooling operation may be input into each residual block, and features of the defect-free image may be extracted to obtain an output image corresponding to each residual block.
[0066] The neural network may be a resnet-50 neural network, and the number of residual blocks in the neural network may be 4.
[0067] For example, the size of the defect-free image input to the neural network is 3x1024x1024. Before the residual block operation, a convolution operation is performed, the convolution kernel size is 64x7x7, and the output size is 64x1024x1024. Then, a maximum pooling operation is performed, the pooling kernel size is 3x3, and the output size is 64x512x512. Then, the output of the defect-free image after the first residual block is an output image of size 256x512x512, the output after the second residual block is an output image of size 512x512x512, the output after the third residual block is an output image of size 1024x512x512, and the output after the fourth residual block is an output image of size 2048x512x512.
[0068] After obtaining the output images of the four residual blocks, the outputs of the four residual blocks serve as inputs of four branches corresponding one-to-one to the four residual blocks.
[0069] In one embodiment, after obtaining the output image of the defect-free image on each branch, for the output image set composed of the output images on any branch, the neural network is iteratively trained by the stochastic gradient descent method to obtain the network parameters and the center of the hypersphere of the branch for the purpose of optimizing the objective function. The objective function is:
[0070]
[0071] where x i For the output image, FEM(.;w a ) is the network weight parameter w of branch a a The feature representation of , ca is the center of the hypersphere of the branch, α is the attenuation regularization weight of the hypersphere, and the value here is 1.
[0072] After obtaining the network parameters and the center of the hypersphere of the branch, the network parameters, the center of the hypersphere and the output image set consisting of the output images input into the branch can be substituted into the optimized objective function to calculate the minimum hypersphere corresponding to the branch of the output image set.
[0073] Considering that when training is performed directly through the output image obtained by the residual block, the feature information of the output image may not be obvious enough, that is, there may be noise between the feature information obtained from the output image, which affects the accuracy of subsequent training. To this end, in one embodiment, each of the defect-free images is input into a neural network to obtain each output image set corresponding to each of the residual blocks, including:
[0074] Inputting each of the defect-free images into a neural network, and obtaining each target image set corresponding to each residual block through each residual block of the neural network;
[0075] Extracting each first feature information of any target image in the target image set, and acquiring a first adjustment matrix of a residual block corresponding to the target image according to the cosine similarity between each first feature information;
[0076] Acquire the output image according to the first adjustment matrix and the defect-free image;
[0077] The output images corresponding to the same residual block are combined into the output image set to obtain output image sets corresponding to the residual blocks one by one.
[0078] In one embodiment, for any defect-free image, after being input into the neural network, each residual block is first used to output each target image corresponding to each residual block. For example, four target images of defect-free images are output through four residual blocks. Then, for any target image, the target image is resized and converted into a two-dimensional image. For example, the image size of the defect-free image is CWH, and the target image F output by the residual block a is in The size of the target image F is CaWaHa. ina After the size transformation, it becomes a two-dimensional image F i ' na , whose size is (Ca / la)×(WaHa), where the value of la is 2. After converting the target image into a two-dimensional image, extract the first feature information of the two-dimensional image, and calculate the cosine similarity between the i-th and j-th feature information. The cosine similarity calculation formula between the i-th and j-th feature information is:
[0079]
[0080] Thus, the similarity matrix between the feature information in the target image can be obtained, that is, the similarity matrix M corresponding to the residual block a is:
[0081]
[0082] At this time, the first adjustment matrix corresponding to the residual block a is M a =(1-M) / 2
[0083] After obtaining the first adjustment matrix M a After that, according to the first adjustment matrix M a and defect-free image F in , we can get the output image F corresponding to the residual block a out :
[0084] F out =F in +λF in M a
[0085] Here, λ is a learnable parameter and its value is 0.5.
[0086] After getting the output image F out After that, all the defect-free images are used to obtain all the output images F using the residual block a out The output image set that constitutes the corresponding residual block a.
[0087] By utilizing the similarity between each feature information of the image, a first adjustment matrix is obtained, so that the features of the output image are enhanced through the first adjustment matrix, thereby improving the accuracy of subsequent training.
[0088] In one embodiment, after a large number of defect-free images are used to complete the training of the neural network, the image of the product to be inspected can be input into the neural network, and each residual block in the neural network outputs each feature image corresponding to each residual block, and the branch of each feature image is determined according to the residual block corresponding to each feature image. Then, for any feature image, it is mapped to the multidimensional space where the minimum hypersphere corresponding to the branch is located, and the defect score of the feature image is determined according to whether the feature image is mapped into the minimum hypersphere.
[0089] Considering that when defect recognition is performed directly through the feature image obtained by the residual block, there may be noise between the feature information obtained from the feature image, which affects the accuracy of defect recognition. To this end, in one embodiment, the image of the product to be detected is input into the neural network, and each feature image corresponding to each residual block in the neural network is obtained, including:
[0090] Inputting the image of the product to be inspected into the neural network, and obtaining initial images corresponding to the residual blocks through the residual blocks of the neural network;
[0091] Extracting each second feature information in the initial image, and acquiring a second adjustment matrix of the residual block corresponding to the initial image according to the cosine similarity between each second feature information;
[0092] The feature image is acquired according to the second adjustment matrix and the image of the product to be detected.
[0093] In one embodiment, after the image of the product to be inspected is input into the neural network, each residual block is first used to output each initial image corresponding to each residual block.
[0094] In one embodiment, when acquiring the initial image, the image of the product to be detected may be input into a neural network to perform convolution operations and maximum pooling operations to adjust the image of the product to be detected;
[0095] The adjusted product image to be inspected is passed through the residual blocks of the neural network to obtain initial images corresponding to the residual blocks.
[0096] Exemplarily, the image of the product to be inspected is subjected to sequential convolution operations and then sequential maximum pooling operations to obtain an adjusted image of the inspected product. Then, each residual block of the neural network is passed to obtain each initial image corresponding to each residual block.
[0097] After obtaining each initial image, for any initial image, the initial image is resized and converted into a two-dimensional image. For example, the image size of the product image to be detected is CWH, and the initial image F output by the residual block a is in The size of ′ is CaWaHa, and the initial image F in ′ is transformed into a two-dimensional image F after size transformation i ' na ′, whose size is (Ca / la)×(WaHa), where the value of la is 2. After converting the initial image into a two-dimensional image, extract each second feature information of the two-dimensional image, and calculate the cosine similarity between the i-th and j-th feature information. The cosine similarity calculation formula between the i-th and j-th feature information is:
[0098]
[0099] Thus, the similarity matrix between the feature information in the target image can be obtained, that is, the similarity matrix M′ corresponding to the residual block a is:
[0100]
[0101] At this time, the second adjustment matrix corresponding to the residual block a is M a ′=(1-M′) / 2
[0102] After obtaining the second adjustment matrix Ma ', according to the second adjustment matrix M a ′ and the image to be detected F in ′, we can get the feature image F corresponding to the residual block a out ′:
[0103] F out ′=F in ′+λF in ′M a '
[0104] By utilizing the similarity between each feature information of the image, a second adjustment matrix is obtained, so that the feature image is feature enhanced by the second adjustment matrix, thereby improving the accuracy of subsequent defect recognition.
[0105] After obtaining the feature image through the initial image, the feature image F out ' is input into the DDM (defection detection module) of the neural network for defect detection. In one embodiment, after mapping the feature image to the high-dimensional space, the defect score of the feature image is determined according to the distance between the feature image and the minimum hypersphere in the multi-dimensional space, including:
[0106] Obtaining a distance difference between the characteristic image and the center of the minimum hypersphere;
[0107] A defect score of the feature image is determined according to the distance difference.
[0108] Among them, the feature image x in the a-th branch i ′’s defect score S a (x i The calculation formula of ′) is:
[0109] S a (x i ′)=UP(‖FEM(x i ′; w a ′)-c′ a ‖ F )
[0110] Among them, c′ a is the minimum hypersphere center of branch a obtained after training the neural network based on the defect-free image input, w a ′ is the network weight parameter of the ath branch obtained after the neural network is trained, and UP(·) is the upsampling operation.
[0111] In one embodiment, after the defect scores of the feature images in the ath branch are obtained, the defect detection result of the image of the product to be detected is determined according to the defect scores of each of the feature images, including:
[0112] According to the preset weights of the branches of each of the feature images, the defect scores of each of the feature images are weighted to obtain a target score;
[0113] A defect detection result of the product image to be inspected is determined according to the target score.
[0114] For example, the defect scores of the defect images of each branch are accumulated to obtain the target score S(x i ′), specifically:
[0115]
[0116] Where A is the number of residual blocks, α a is the preset weight of the defect score of the ath branch, which is used to adjust the defect scores between different branches and can be set according to actual conditions. Preferably, its value can be 0.5.
[0117] When S(x i ′) is greater than a preset value, it can be determined that there are defects in the image of the product to be inspected.
[0118] The defect scores are determined by the branches corresponding to multiple residual blocks, and the defect detection results of the product image to be inspected are determined by using the defect scores of the branches corresponding to each residual block, so that the problem of gradient diffusion in the neural network can be alleviated during defect detection. It can also perform multi-angle defect recognition on the product image to be inspected, thereby further improving the recognition efficiency and accuracy of surface defects of industrial products.
[0119] The surface defect detection device for industrial products provided in an embodiment of the present application is described below. The surface defect detection device for industrial products described below and the surface defect detection method for industrial products described above can be referenced to each other.
[0120] In one embodiment, if Figure 3 As shown, a surface defect detection device for industrial products is provided, comprising:
[0121] The feature image acquisition module 210 is used to input the product image to be detected into the neural network and obtain each feature image corresponding to each residual block in the neural network;
[0122] A feature branch determination module 220, configured to determine the branch of each feature image according to the residual block corresponding to each feature image;
[0123] A defect score determination module 230 is used to map the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determine a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space;
[0124] An image defect detection module 240 is used to determine a defect detection result of the product image to be detected according to the defect score of each of the feature images;
[0125] The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network.
[0126] In one embodiment, a neural network training module is further included, and the neural network training module is used to:
[0127] Inputting each of the defect-free images into a neural network to obtain each output image set corresponding to each of the residual blocks;
[0128] Determining a branch of each output image set according to the residual block corresponding to each output image set;
[0129] Inputting the output image set into the neural network for training, and obtaining network parameters and hypersphere centers corresponding to branches of the output image set;
[0130] Determining a minimum hypersphere corresponding to a branch of the output image set according to the output image set, the network parameters and the center of the hypersphere;
[0131] The output image set includes output images corresponding one-to-one to each of the defect-free images.
[0132] In one embodiment, the neural network training module is specifically used to:
[0133] Inputting each of the defect-free images into a neural network, and obtaining each target image set corresponding to each residual block through each residual block of the neural network;
[0134] Extracting each first feature information of any target image in the target image set, and acquiring a first adjustment matrix of a residual block corresponding to the target image according to the cosine similarity between each first feature information;
[0135] Acquire the output image according to the first adjustment matrix and the defect-free image;
[0136] The output images corresponding to the same residual block are combined into the output image set to obtain output image sets corresponding to the residual blocks one by one.
[0137] In one embodiment, the neural network training module is specifically used to:
[0138] Inputting each of the defect-free images into a neural network, and obtaining each target image set corresponding to each residual block through each residual block of the neural network;
[0139] Extracting each first feature information of any target image in the target image set, and acquiring a first adjustment matrix of a residual block corresponding to the target image according to the cosine similarity between each first feature information;
[0140] Acquire the output image according to the first adjustment matrix and the defect-free image;
[0141] The output images corresponding to the same residual block are combined into the output image set to obtain output image sets corresponding to the residual blocks one by one.
[0142] In one embodiment, the feature image acquisition module 210 is specifically used for:
[0143] Inputting the image of the product to be inspected into the neural network, and obtaining initial images corresponding to the residual blocks through the residual blocks of the neural network;
[0144] Extracting each second feature information in the initial image, and acquiring a second adjustment matrix of the residual block corresponding to the initial image according to the cosine similarity between each second feature information;
[0145] The feature image is acquired according to the second adjustment matrix and the image of the product to be detected.
[0146] In one embodiment, the feature image acquisition module 210 is specifically used for:
[0147] Inputting the image of the product to be detected into the neural network for convolution operation and maximum pooling operation, and adjusting the image of the product to be detected;
[0148] The adjusted product image to be inspected is passed through the residual blocks of the neural network to obtain initial images corresponding to the residual blocks.
[0149] In one embodiment, the defect score determination module 230 is specifically configured to:
[0150] Obtaining a distance difference between the characteristic image and the center of the minimum hypersphere;
[0151] A defect score of the feature image is determined according to the distance difference.
[0152] In one embodiment, the image defect detection module 240 is specifically used for:
[0153] According to the preset weights of the branches of each of the feature images, the defect scores of each of the feature images are weighted to obtain a target score;
[0154] A defect detection result of the product image to be inspected is determined according to the target score.
[0155] By inputting a defect-free image into a neural network for training, the minimum hypersphere of the branch corresponding to each residual block is obtained in advance, and then the image of the product to be detected is input into the neural network to obtain the feature image of the branch corresponding to each residual block of the product image to be detected, and the feature image is mapped to the multidimensional space where the minimum hypersphere corresponding to the branch of the feature image is located, and the defect score of the feature image is determined according to the mapping result, so as to determine the defect detection result of the product image to be detected according to the defect score of each feature image. Thus, the surface defect recognition of industrial products is performed by the neural network obtained by training with a large number of defect-free samples of industrial products that are easily obtained, without obtaining all types of defects, overcoming the problem of requiring a large number of defect samples to train the neural network, resulting in poor training efficiency and defect recognition accuracy, and improving the recognition efficiency and accuracy of surface defects of industrial products.
[0156] Moreover, the defect scores are determined by the branches corresponding to the multiple residual blocks, and then the defect detection results of the product image to be inspected are determined by using the defect scores of the branches corresponding to each residual block, so that when performing defect detection, the problem of gradient diffusion in the neural network can be alleviated. At the same time, since the depth of the neural network is increased by using multiple residual blocks, feature images of multiple branches can be generated, and multi-angle defect recognition can be performed on the product image to be inspected, which can further improve the recognition efficiency and accuracy of surface defects of industrial products.
[0157] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call a computer program in the memory 830 to execute the steps of the surface defect detection method of an industrial product, for example including:
[0158] Inputting the image of the product to be inspected into the neural network, and obtaining each feature image corresponding to each residual block in the neural network;
[0159] Determining branches of each of the feature images according to the residual blocks corresponding to each of the feature images;
[0160] Mapping the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determining a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space;
[0161] Determining a defect detection result of the product image to be detected according to the defect score of each of the feature images;
[0162] The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network.
[0163] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.
[0164] On the other hand, an embodiment of the present application further provides a computer program product, the computer program product including a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the steps of the surface defect detection method of the industrial product provided in the above embodiments, for example, including:
[0165] Inputting the image of the product to be inspected into the neural network, and obtaining each feature image corresponding to each residual block in the neural network;
[0166] Determining branches of each of the feature images according to the residual blocks corresponding to each of the feature images;
[0167] Mapping the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determining a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space;
[0168] Determining a defect detection result of the product image to be detected according to the defect score of each of the feature images;
[0169] The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network.
[0170] On the other hand, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, wherein the computer program is used to enable the processor to execute the steps of the methods provided in the above embodiments, for example, including:
[0171] Inputting the image of the product to be inspected into the neural network, and obtaining each feature image corresponding to each residual block in the neural network;
[0172] Determining branches of each of the feature images according to the residual blocks corresponding to each of the feature images;
[0173] Mapping the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determining a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space;
[0174] Determining a defect detection result of the product image to be detected according to the defect score of each of the feature images;
[0175] The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network.
[0176] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.
[0177] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for detecting surface defects of industrial products, characterized in that: include: Inputting the image of the product to be inspected into the neural network, and obtaining each feature image corresponding to each residual block in the neural network; Determining branches of each of the feature images according to the residual blocks corresponding to each of the feature images; Mapping the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determining a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space; Determining a defect detection result of the product image to be detected according to the defect score of each of the feature images; The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network; The method further comprises: Inputting each of the defect-free images into a neural network to obtain each output image set corresponding to each of the residual blocks; Determining a branch of each output image set according to the residual block corresponding to each output image set; Inputting the output image set into the neural network for training, and obtaining network parameters and hypersphere centers corresponding to branches of the output image set; Determining a minimum hypersphere corresponding to a branch of the output image set according to the output image set, the network parameters and the center of the hypersphere; The output image set includes output images corresponding one-to-one to each of the defect-free images.
2. The surface defect detection method of industrial products according to claim 1, characterized in that: The step of inputting each of the defect-free images into a neural network to obtain each output image set corresponding to each of the residual blocks comprises: Inputting each of the defect-free images into a neural network, and obtaining each target image set corresponding to each residual block through each residual block of the neural network; Extracting each first feature information of any target image in the target image set, and acquiring a first adjustment matrix of a residual block corresponding to the target image according to the cosine similarity between each first feature information; Acquire the output image according to the first adjustment matrix and the defect-free image; The output images corresponding to the same residual block are combined into the output image set to obtain output image sets corresponding to the residual blocks one by one.
3. The surface defect detection method of industrial products according to claim 1 or 2, characterized in that: The step of inputting the image of the product to be detected into the neural network and obtaining each feature image corresponding to each residual block in the neural network comprises: Inputting the image of the product to be inspected into the neural network, and obtaining initial images corresponding to the residual blocks through the residual blocks of the neural network; Extracting each second feature information in the initial image, and acquiring a second adjustment matrix of the residual block corresponding to the initial image according to the cosine similarity between each second feature information; The feature image is acquired according to the second adjustment matrix and the image of the product to be detected.
4. The surface defect detection method of industrial products according to claim 3, characterized in that: The step of inputting the image of the product to be detected into the neural network and obtaining initial images corresponding to the residual blocks through the residual blocks of the neural network comprises: Inputting the image of the product to be detected into the neural network for convolution operation and maximum pooling operation, and adjusting the image of the product to be detected; The adjusted product image to be inspected is passed through the residual blocks of the neural network to obtain initial images corresponding to the residual blocks.
5. The surface defect detection method of industrial products according to claim 1 or 2, characterized in that: Determining the defect score of the feature image according to the distance between the feature image and the minimum hypersphere in the multidimensional space includes: Obtaining a distance difference between the characteristic image and the center of the minimum hypersphere; A defect score of the feature image is determined according to the distance difference.
6. The surface defect detection method of industrial products according to claim 1 or 2, characterized in that: The step of determining the defect detection result of the product image to be detected according to the defect score of each of the feature images comprises: According to the preset weights of the branches of each of the feature images, the defect scores of each of the feature images are weighted to obtain a target score; A defect detection result of the product image to be inspected is determined according to the target score.
7. A surface defect detection device for industrial products, characterized in that: include: A feature image acquisition module, used to input the image of the product to be detected into the neural network, and obtain each feature image corresponding to each residual block in the neural network; A feature branch determination module, used to determine the branch of each feature image according to the residual block corresponding to each feature image; a defect score determination module, used to map the feature image to a multidimensional space where a minimum hypersphere corresponding to a branch of the feature image is located, and determine a defect score of the feature image according to a distance between the feature image and the minimum hypersphere in the multidimensional space; An image defect detection module, used to determine a defect detection result of the image of the product to be detected according to the defect score of each of the feature images; The minimum hypersphere corresponding to each branch is obtained by training a plurality of defect-free images through the neural network; Also included is a neural network training module, the neural network training module is used to: Inputting each of the defect-free images into a neural network to obtain each output image set corresponding to each of the residual blocks; Determining a branch of each output image set according to the residual block corresponding to each output image set; Inputting the output image set into the neural network for training, and obtaining network parameters and hypersphere centers corresponding to branches of the output image set; Determining a minimum hypersphere corresponding to a branch of the output image set according to the output image set, the network parameters and the center of the hypersphere; The output image set includes output images corresponding one-to-one to each of the defect-free images.
8. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of the surface defect detection method for industrial products according to any one of claims 1 to 6 are implemented.
9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the surface defect detection method for industrial products according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Batch process fault monitoring method based on multi-stage ICA-SVDD
CN107272655A
Low-quality video face recognition method based on SICNN
CN112580502A