Image defect detection method, device, electronic equipment and medium

Abstract

The present application relates to image analysis technology, and the present application provides an image defect detection method, device, electronic device and medium. The method can obtain an image to be detected, determine the field to which the image to be detected belongs, and based on the field, determine a target convolution layer from a pre-built convolutional neural network, use the target convolution layer to extract the features to be detected of the image to be detected, determine the target score of the image to be detected based on the features to be detected, determine a score threshold corresponding to the field, and when the target score is less than the score threshold, determine the image to be detected as a defective image. The present application can improve the accuracy of defect detection.

Description

Technical Field

[0001] The present application relates to the field of image analysis technology, and in particular to an image defect detection method, device, electronic equipment and medium. Background Art

[0002] In the field of industrial inspection, the inspection of defects such as damage and scratches in product images can ensure the high quality of products. The traditional method is to determine whether there are defects in product images through supervised learning classification models. However, since it is difficult to obtain defect data, there is insufficient training data, resulting in the inability of the trained supervised learning classification model to accurately identify whether there are defects in product images, thereby reducing the accuracy of defect detection. Summary of the invention

[0003] In view of the above, it is necessary to provide an image defect detection method, device, electronic device and medium that can improve the accuracy of defect detection.

[0004] A first aspect of the present application provides an image defect detection method, the image defect detection method comprising:

[0005] Acquire the image to be detected;

[0006] Determine the domain to which the image to be detected belongs, and based on the domain, determine a target convolutional layer from a pre-built convolutional neural network;

[0007] Extracting the features to be detected of the image to be detected by using the target convolution layer;

[0008] Determining a target score of the image to be detected according to the feature to be detected;

[0009] determining a score threshold corresponding to the domain;

[0010] When the target score is less than the score threshold, the image to be detected is determined as a defective image.

[0011] According to a preferred embodiment of the present application, the acquisition of the image to be detected includes a combination of one or more of the following methods:

[0012] Determine a detection object, and photograph the detection object using a camera device to obtain the image to be detected; and / or

[0013] Obtaining an image with a preset mark from a configuration library as the image to be detected; and / or

[0014] The image to be detected is crawled from a preset web page using a web crawler tool.

[0015] According to a preferred embodiment of the present application, determining a target convolutional layer from a pre-built convolutional neural network based on the field includes:

[0016] Acquire a plurality of training images on the domain;

[0017] Extracting a convolution kernel from any convolution layer of the convolutional neural network, wherein the convolutional neural network includes a plurality of convolution layers;

[0018] Use each extracted convolution kernel to perform convolution operation on each training image to obtain multiple feature images on each convolution layer;

[0019] Perform pooling on each feature image on each convolution layer to obtain multiple low-dimensional vectors on each convolution layer;

[0020] Use the T-SNE algorithm to visualize each low-dimensional vector and obtain the distribution images of multiple training images on each convolutional layer;

[0021] The target convolution layer is determined according to the distribution image of each convolution layer.

[0022] According to a preferred embodiment of the present application, the plurality of training images include defective images and normal images, and the step of determining the score threshold corresponding to the field includes:

[0023] Inputting the defect image and the normal image into the target convolution layer for convolution processing to obtain a first feature map of the defect image and a second feature map of the normal image;

[0024] Determining a target pooling layer corresponding to the target convolutional layer from the convolutional neural network;

[0025] Using the target pooling layer to perform pooling processing on the first feature map and the second feature map to obtain a first vector of the defective image and a second vector of the normal image;

[0026] Inputting the first vector and the second vector into a pre-trained Gaussian mixture model to obtain a first score for the defective image and a second score for the normal image;

[0027] determining a mean score of the first score and the second score, and determining a standard deviation of the first score and the second score;

[0028] The standard deviation is multiplied by a preset value to obtain a calculation result, and the mean score is subtracted from the calculation result to obtain the score threshold.

[0029] According to a preferred embodiment of the present application, determining the target score of the image to be detected according to the feature to be detected includes:

[0030] Performing mean pooling processing on the feature to be detected to obtain a target vector;

[0031] The target vector is input into the Gaussian mixture model to obtain the target score.

[0032] According to a preferred embodiment of the present application, after determining that the image to be detected is a defective image, the image defect detection method further includes:

[0033] determining a target number of said defective images;

[0034] When the target number is greater than a preset number, generating a warning message according to the defective image;

[0035] Encrypting the warning information using a symmetric encryption algorithm to obtain a ciphertext;

[0036] Determining an alarm level of the ciphertext according to the target quantity;

[0037] Determine an alarm mode according to the alarm level;

[0038] The ciphertext is sent in the alarm mode.

[0039] According to a preferred embodiment of the present application, the image defect detection method further includes:

[0040] When the target score is greater than or equal to the score threshold, the image to be detected is determined to be a flawless image.

[0041] A second aspect of the present application provides an image defect detection device, the image defect detection device comprising:

[0042] An acquisition unit, used for acquiring an image to be detected;

[0043] A determination unit, used to determine the field to which the image to be detected belongs, and based on the field, determine a target convolution layer from a pre-built convolutional neural network;

[0044] An extraction unit, used for extracting the features to be detected of the image to be detected by using the target convolution layer;

[0045] The determining unit is further used to determine the target score of the image to be detected according to the feature to be detected;

[0046] The determining unit is further configured to determine a score threshold corresponding to the field;

[0047] The determining unit is further configured to determine the image to be detected as a defective image when the target score is less than the score threshold.

[0048] According to a preferred embodiment of the present application, the acquisition unit acquires the image to be detected by one or more of the following methods:

[0049] Determine a detection object, and photograph the detection object using a camera device to obtain the image to be detected; and / or

[0050] Obtaining an image with a preset mark from a configuration library as the image to be detected; and / or

[0051] The image to be detected is crawled from a preset web page using a web crawler tool.

[0052] According to a preferred embodiment of the present application, the determining unit determines the target convolution layer from a pre-built convolutional neural network based on the field, including:

[0053] Acquire a plurality of training images on the domain;

[0054] Extracting a convolution kernel from any convolution layer of the convolutional neural network, wherein the convolutional neural network includes a plurality of convolution layers;

[0055] Use each extracted convolution kernel to perform convolution operation on each training image to obtain multiple feature images on each convolution layer;

[0056] Perform pooling on each feature image on each convolution layer to obtain multiple low-dimensional vectors on each convolution layer;

[0057] Use the T-SNE algorithm to visualize each low-dimensional vector and obtain the distribution images of multiple training images on each convolutional layer;

[0058] The target convolution layer is determined according to the distribution image of each convolution layer.

[0059] According to a preferred embodiment of the present application, the plurality of training images include defective images and normal images, and the determination unit determines the score threshold corresponding to the field including:

[0060] Inputting the defect image and the normal image into the target convolution layer for convolution processing to obtain a first feature map of the defect image and a second feature map of the normal image;

[0061] Determining a target pooling layer corresponding to the target convolutional layer from the convolutional neural network;

[0062] Using the target pooling layer to perform pooling processing on the first feature map and the second feature map to obtain a first vector of the defective image and a second vector of the normal image;

[0063] Inputting the first vector and the second vector into a pre-trained Gaussian mixture model to obtain a first score for the defective image and a second score for the normal image;

[0064] determining a mean score of the first score and the second score, and determining a standard deviation of the first score and the second score;

[0065] The standard deviation is multiplied by a preset value to obtain a calculation result, and the mean score is subtracted from the calculation result to obtain the score threshold.

[0066] According to a preferred embodiment of the present application, the determining unit determines the target score of the image to be detected according to the feature to be detected, including:

[0067] Performing mean pooling processing on the feature to be detected to obtain a target vector;

[0068] The target vector is input into the Gaussian mixture model to obtain the target score.

[0069] According to a preferred embodiment of the present application, the determining unit is further configured to determine a target number of defective images after determining the image to be detected as a defective image;

[0070] The image defect detection device also includes:

[0071] a generating unit, configured to generate warning information according to the defective image when the target number is greater than a preset number;

[0072] An encryption unit, used for encrypting the warning information by using a symmetric encryption algorithm to obtain a ciphertext;

[0073] The determining unit is further used to determine the warning level of the ciphertext according to the target quantity;

[0074] The determining unit is further used to determine an alarm mode according to the alarm level;

[0075] A sending unit is used to send the ciphertext in the alarm mode.

[0076] According to a preferred embodiment of the present application, the determining unit is further configured to determine the image to be detected as a flawless image when the target score is greater than or equal to the score threshold.

[0077] A third aspect of the present application provides an electronic device, the electronic device comprising:

[0078] a memory storing at least one instruction; and

[0079] The processor obtains the instruction stored in the memory to implement the image defect detection method.

[0080] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and the at least one instruction is acquired by a processor in an electronic device to implement the image defect detection method.

[0081] It can be seen from the above technical solutions that the present application determines the target convolution layer with the best feature extraction performance so as to better extract the features of the image to be detected, thereby enabling the present application to improve the accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 It is a flow chart of a preferred embodiment of the image defect detection method of the present application.

[0083] Figure 2 It is a functional module diagram of a preferred embodiment of the image defect detection device of the present application.

[0084] Figure 3 It is a schematic diagram of the structure of an electronic device of a preferred embodiment of the image defect detection method of the present application. DETAILED DESCRIPTION

[0085] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0086] like Figure 1 FIG. 1 is a flowchart of a preferred embodiment of the image defect detection method of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0087] The image defect detection method is applied to one or more electronic devices 1, and the electronic device 1 is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital processors (DSP), embedded devices, etc.

[0088] The electronic device 1 can be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0089] The electronic device 1 may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0090] The network where the electronic device 1 is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0091] S10, obtaining an image to be detected.

[0092] In at least one embodiment of the present application, the image to be detected can be obtained from a camera device, from a configuration library, or crawled from a website.

[0093] Furthermore, the image to be detected may include a detection object.

[0094] In at least one embodiment of the present application, the electronic device acquires the image to be detected by one or more of the following methods:

[0095] (1) The electronic device determines the detection object and uses the camera device to photograph the detection object to obtain the image to be detected.

[0096] The detection object may be any product or any surface of any product.

[0097] Furthermore, the imaging device may be a camera, and the imaging device may be installed directly in front of the detection object.

[0098] Through the above implementation, it is possible to quickly obtain an image to be inspected for defects.

[0099] (2) The electronic device obtains an image with a preset mark from the configuration library as the image to be detected.

[0100] The configuration library stores a plurality of images and identifications of the plurality of images.

[0101] Furthermore, the preset mark may be “not detected” or any other mark. The preset mark may be set arbitrarily by the user, and this application does not impose any limitation on this.

[0102] By means of the mapping relationship between the preset identifier and the image to be detected, the image to be detected can be accurately acquired.

[0103] (3) The electronic device uses a web crawler tool to crawl the image to be detected from a preset web page.

[0104] The preset web page may be a corporate website of the enterprise where the user works.

[0105] Furthermore, the web crawler tool may be any crawler tool, which will not be described in detail in this application.

[0106] By crawling the image to be detected from the specified web page, it can be ensured that the image finally presented on the specified web page is a flawless image.

[0107] S11, determining the field to which the image to be detected belongs, and based on the field, determining a target convolutional layer from a pre-built convolutional neural network.

[0108] In at least one embodiment of the present application, the field is the field where the object to be detected in the image to be detected is located. For example, the field may be sheet metal.

[0109] Furthermore, the pre-built convolutional neural network may be VGG-16, and the specific convolutional neural network is not limited in this application.

[0110] Furthermore, the target convolutional layer refers to the convolutional layer that extracts the best image features in the convolutional neural network.

[0111] In at least one embodiment of the present application, the electronic device determines the field to which the image to be detected belongs, including:

[0112] The electronic device determines the object to be detected of the image to be detected. Further, the electronic device determines the field according to the object to be detected.

[0113] Through the above implementation, the field to which the image to be detected belongs can be determined quickly and accurately.

[0114] In at least one embodiment of the present application, the electronic device determines the target convolution layer from a pre-built convolutional neural network based on the field, including:

[0115] The electronic device obtains multiple training images in the field, and the electronic device extracts a convolution kernel from any convolution layer of the convolutional neural network, wherein the convolutional neural network includes multiple convolution layers. Further, the electronic device uses each extracted convolution kernel to perform a convolution operation on each training image to obtain multiple feature images on each convolution layer. Further, the electronic device performs pooling processing on each feature image on each convolution layer to obtain multiple low-dimensional vectors on each convolution layer. Further, the electronic device uses a T-SNE algorithm to perform visualization processing on each low-dimensional vector to obtain a distribution image of the multiple training images on each convolution layer. Further, the electronic device determines the target convolution layer according to the distribution image of each convolution layer.

[0116] By analyzing multiple training images in the field, it can be ensured that the target convolution layer is suitable for feature extraction of the image to be detected in the field. At the same time, by analyzing normal images and defective images, the target convolution layer can not only extract features of normal images, but also extract features of defective images.

[0117] Specifically, the electronic device determines the target convolution layer according to the distribution image of each convolution layer, including:

[0118] The electronic device traverses each distribution image and determines the distribution image with the best distribution as the target distribution image. Further, the electronic device determines the convolution layer corresponding to the target distribution image as the target convolution layer.

[0119] S12, using the target convolutional layer to extract the features to be detected of the image to be detected.

[0120] In at least one embodiment of the present application, the feature to be detected is obtained by extracting features of the image to be detected using a convolution kernel in the target convolution layer.

[0121] In at least one embodiment of the present application, the electronic device extracting the feature to be detected of the image to be detected by using the target convolution layer includes:

[0122] The electronic device obtains a target convolution kernel in the target convolution layer. Further, the electronic device extracts features of the image to be detected using the target convolution kernel to obtain the features to be detected.

[0123] Among them, the target convolution kernel can be a 2*2 matrix or a 3*3 matrix, and this application does not impose any restrictions on this.

[0124] S13, determining a target score of the image to be detected according to the feature to be detected.

[0125] In at least one embodiment of the present application, the electronic device determining the target score of the image to be detected according to the feature to be detected includes:

[0126] The electronic device performs mean pooling processing on the feature to be detected to obtain a target vector. Further, the electronic device inputs the target vector into a pre-trained Gaussian mixture model to obtain the target score.

[0127] The Gaussian Mixture Model (GMM) can accurately quantify the score corresponding to the target vector using a Gaussian probability density function (normal distribution image curve).

[0128] In at least one embodiment of the present application, before inputting the target vector into a pre-trained Gaussian mixture model to obtain the target score, the image defect detection method further includes:

[0129] The electronic device divides the multiple low-dimensional vectors into a training set, a test set and a validation set. Furthermore, the electronic device iteratively trains the multiple low-dimensional vectors in the training set based on a maximum expectation algorithm to obtain a learner. Furthermore, the electronic device uses the multiple low-dimensional vectors in the test set to test the learner to obtain a test result. When the test result is less than a configuration value, the electronic device uses the multiple low-dimensional vectors in the validation set to adjust parameters in the learner to obtain the Gaussian mixture model.

[0130] Through the above implementation, the generated Gaussian mixture model can be made more accurate.

[0131] In at least one embodiment of the present application, before dividing the multiple low-dimensional vectors into a training set, a test set, and a validation set, the method further includes:

[0132] The electronic device calculates the number of the multiple low-dimensional vectors, and when the number is less than a preset number, the electronic device increases the number of the multiple low-dimensional vectors by using a data enhancement algorithm.

[0133] Through the above implementation, it is possible to avoid the poor generalization ability of the scores generated by the trained Gaussian mixture model due to the insufficient number of multiple low-dimensional vectors.

[0134] S14, determining a score threshold corresponding to the field.

[0135] In at least one embodiment of the present application, the plurality of training images include defective images and normal images, and the electronic device determines the score threshold corresponding to the field including:

[0136] The electronic device inputs the defect image and the normal image into the target convolution layer for convolution processing to obtain a first feature map of the defect image and a second feature map of the normal image. Further, the electronic device determines a target pooling layer corresponding to the target convolution layer from the convolution neural network. Further, the electronic device performs pooling processing on the first feature map and the second feature map using the target pooling layer to obtain a first vector of the defect image and a second vector of the normal image. The electronic device inputs the first vector and the second vector into a pre-trained Gaussian mixture model to obtain a first score of the defect image and a second score of the normal image. Further, the electronic device determines a mean score of the first score and the second score, and determines a standard deviation of the first score and the second score. Further, the electronic device multiplies the standard deviation by a preset value to obtain a calculation result, and subtracts the mean score from the calculation result to obtain the score threshold.

[0137] By calculating the mean scores of the normal image and the defective image and calculating the standard deviation scores of the normal image and the defective image, a score threshold suitable for the field can be determined.

[0138] S15: When the target score is less than the score threshold, the image to be detected is determined as a defective image.

[0139] In at least one embodiment of the present application, after determining that the image to be detected is a defective image, the image defect detection method further includes:

[0140] The electronic device determines a target number of defective images, and when the target number is greater than a preset number, the electronic device generates an alarm message based on the defective images. Further, the electronic device encrypts the alarm message using a symmetric encryption algorithm to obtain a ciphertext. Further, the electronic device determines an alarm level of the ciphertext based on the target number. The electronic device determines an alarm method based on the alarm level. Further, the electronic device sends the ciphertext in the alarm method.

[0141] The preset number may be calculated based on the tolerance rate of the image to be detected, and the present application does not impose any restriction on the value of the preset number.

[0142] Furthermore, the alarm levels include: level one, level two, etc.

[0143] Furthermore, the alarm method includes: an alarm sound from a speaker, an email, a telephone call, etc.

[0144] Through the above implementation, an alarm message can be issued when the target number of defective images is greater than a preset number. In addition, by encrypting the alarm message, it is possible to prevent the alarm message from being tampered with, thereby improving the security of the alarm message. At the same time, the alarm method is determined according to the alarm level, and the alarm message can be sent in an appropriate alarm method, making the sending of the alarm message more humane.

[0145] In at least one embodiment of the present application, the image defect detection method further includes:

[0146] When the target score is greater than or equal to the score threshold, the electronic device determines the image to be detected as a flawless image.

[0147] It can be seen from the above technical solutions that the present application determines the target convolution layer with the best feature extraction performance so as to better extract the features of the image to be detected, thereby enabling the present application to improve the accuracy of defect detection.

[0148] like Figure 2 , which is a functional module diagram of a preferred embodiment of the image defect detection device of the present application. The image defect detection device 11 includes an acquisition unit 110, a determination unit 111, a generation unit 112, an encryption unit 113, a sending unit 114, a division unit 115, a training unit 116, a testing unit 117, an adjustment unit 118, a calculation unit 119 and an extraction unit 120. The module / unit referred to in the present application refers to a series of computer program segments that can be acquired by the processor 13 and can complete fixed functions, which are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0149] The acquisition unit 110 acquires the image to be detected.

[0150] In at least one embodiment of the present application, the image to be detected can be obtained from a camera device, from a configuration library, or crawled from a website.

[0151] Furthermore, the image to be detected may include a detection object.

[0152] In at least one embodiment of the present application, the acquiring unit 110 acquires the image to be detected by one or more of the following methods:

[0153] (1) The acquisition unit 110 determines the detection object and uses the camera to photograph the detection object to obtain the image to be detected.

[0154] The detection object may be any product or any surface of any product.

[0155] Furthermore, the imaging device may be a camera, and the imaging device may be installed directly in front of the detection object.

[0156] Through the above implementation, it is possible to quickly obtain an image to be inspected for defects.

[0157] (2) The acquisition unit 110 acquires an image with a preset identifier from the configuration library as the image to be detected.

[0158] The configuration library stores a plurality of images and identifications of the plurality of images.

[0159] Furthermore, the preset mark may be “not detected” or any other mark. The preset mark may be set arbitrarily by the user, and this application does not impose any limitation on this.

[0160] By means of the mapping relationship between the preset identifier and the image to be detected, the image to be detected can be accurately acquired.

[0161] (3) The acquisition unit 110 uses a web crawler tool to crawl the image to be detected from a preset web page.

[0162] The preset web page may be a corporate website of the enterprise where the user works.

[0163] Furthermore, the web crawler tool may be any crawler tool, which will not be described in detail in this application.

[0164] By crawling the image to be detected from the specified web page, it can be ensured that the image finally presented on the specified web page is a flawless image.

[0165] The determination unit 111 determines the field to which the image to be detected belongs, and based on the field, determines a target convolutional layer from a pre-constructed convolutional neural network.

[0166] In at least one embodiment of the present application, the field is the field where the object to be detected in the image to be detected is located. For example, the field may be sheet metal.

[0167] Furthermore, the pre-built convolutional neural network may be VGG-16, and the specific convolutional neural network is not limited in this application.

[0168] Furthermore, the target convolutional layer refers to the convolutional layer that extracts the best image features in the convolutional neural network.

[0169] In at least one embodiment of the present application, the determining unit 111 determines the field to which the image to be detected belongs, including:

[0170] The determining unit 111 determines the object to be detected of the image to be detected. Further, the determining unit 111 determines the field according to the object to be detected.

[0171] Through the above implementation, the field to which the image to be detected belongs can be determined quickly and accurately.

[0172] In at least one embodiment of the present application, the determining unit 111 determines the target convolution layer from the pre-built convolutional neural network based on the field, including:

[0173] The determination unit 111 obtains multiple training images on the field, and the determination unit 111 extracts a convolution kernel from any convolution layer of the convolutional neural network, and the convolutional neural network includes multiple convolution layers. Further, the determination unit 111 uses each extracted convolution kernel to perform a convolution operation on each training image to obtain multiple feature images on each convolution layer. Further, the determination unit 111 performs pooling processing on each feature image on each convolution layer to obtain multiple low-dimensional vectors on each convolution layer. Further, the determination unit 111 uses the T-SNE algorithm to visualize each low-dimensional vector to obtain a distribution image of the multiple training images on each convolution layer. Further, the determination unit 111 determines the target convolution layer according to the distribution image of each convolution layer.

[0174] By analyzing multiple training images in the field, it can be ensured that the target convolution layer is suitable for feature extraction of the image to be detected in the field. At the same time, by analyzing normal images and defective images, the target convolution layer can not only extract features of normal images, but also extract features of defective images.

[0175] Specifically, the determining unit 111 determines the target convolution layer according to the distribution image of each convolution layer, including:

[0176] The determining unit 111 traverses each distribution image, and determines the distribution image with the best distribution as the target distribution image. Further, the determining unit 111 determines the convolution layer corresponding to the target distribution image as the target convolution layer.

[0177] The extraction unit 120 uses the target convolution layer to extract the features to be detected of the image to be detected.

[0178] In at least one embodiment of the present application, the feature to be detected is obtained by extracting features of the image to be detected using a convolution kernel in the target convolution layer.

[0179] In at least one embodiment of the present application, the extracting unit 120 extracts the feature to be detected of the image to be detected by using the target convolution layer, including:

[0180] The extraction unit 120 obtains the target convolution kernel in the target convolution layer. Further, the extraction unit 120 extracts the features of the image to be detected using the target convolution kernel to obtain the features to be detected.

[0181] Among them, the target convolution kernel can be a 2*2 matrix or a 3*3 matrix, and this application does not impose any restrictions on this.

[0182] The determining unit 111 determines the object score of the image to be detected according to the feature to be detected.

[0183] In at least one embodiment of the present application, the determining unit 111 determines the target score of the image to be detected according to the feature to be detected, including:

[0184] The determining unit 111 performs mean pooling processing on the feature to be detected to obtain a target vector. Further, the determining unit 111 inputs the target vector into a pre-trained Gaussian mixture model to obtain the target score.

[0185] The Gaussian Mixture Model (GMM) can accurately quantify the score corresponding to the target vector using a Gaussian probability density function (normal distribution image curve).

[0186] In at least one embodiment of the present application, before inputting the target vector into a pre-trained Gaussian mixture model to obtain the target score, the division unit 115 divides the multiple low-dimensional vectors into a training set, a test set and a validation set. Furthermore, the training unit 116 iteratively trains the multiple low-dimensional vectors in the training set based on the maximum expectation algorithm to obtain a learner. Furthermore, the testing unit 117 uses the multiple low-dimensional vectors in the test set to test the learner to obtain a test result. When the test result is less than the configuration value, the adjustment unit 118 uses the multiple low-dimensional vectors in the validation set to adjust the parameters in the learner to obtain the Gaussian mixture model.

[0187] Through the above implementation, the generated Gaussian mixture model can be made more accurate.

[0188] In at least one embodiment of the present application, before dividing the multiple low-dimensional vectors into a training set, a test set, and a validation set, the computing unit 119 calculates the number of the multiple low-dimensional vectors. When the number is less than a preset number, the computing unit 119 uses a data enhancement algorithm to increase the number of the multiple low-dimensional vectors.

[0189] Through the above implementation, it is possible to avoid the poor generalization ability of the scores generated by the trained Gaussian mixture model due to the insufficient number of multiple low-dimensional vectors.

[0190] The determining unit 111 determines a score threshold corresponding to the field.

[0191] In at least one embodiment of the present application, the plurality of training images include defective images and normal images, and the determination unit 111 determines the score threshold corresponding to the field including:

[0192] The determining unit 111 inputs the defect image and the normal image into the target convolution layer for convolution processing to obtain a first feature map of the defect image and a second feature map of the normal image. Further, the determining unit 111 determines a target pooling layer corresponding to the target convolution layer from the convolution neural network. Further, the determining unit 111 performs pooling processing on the first feature map and the second feature map using the target pooling layer to obtain a first vector of the defect image and a second vector of the normal image. The determining unit 111 inputs the first vector and the second vector into a pre-trained Gaussian mixture model to obtain a first score of the defect image and a second score of the normal image. Further, the determining unit 111 determines a mean score of the first score and the second score, and determines a standard deviation of the first score and the second score. Further, the determining unit 111 multiplies the standard deviation by a preset value to obtain a calculation result, and subtracts the mean score from the calculation result to obtain the score threshold.

[0193] By calculating the mean scores of the normal image and the defective image and calculating the standard deviation scores of the normal image and the defective image, a score threshold suitable for the field can be determined.

[0194] When the target score is less than the score threshold, the determining unit 111 determines the image to be detected as a defective image.

[0195] In at least one embodiment of the present application, after the image to be detected is determined as a defective image, the determination unit 111 determines a target number of the defective images, and when the target number is greater than a preset number, the generation unit 112 generates an alarm message based on the defective image. Further, the encryption unit 113 encrypts the alarm message using a symmetric encryption algorithm to obtain a ciphertext. Further, the determination unit 111 determines an alarm level of the ciphertext based on the target number. The determination unit 111 determines an alarm mode based on the alarm level. Further, the sending unit 114 sends the ciphertext in the alarm mode.

[0196] The preset number may be calculated based on the tolerance rate of the image to be detected, and the present application does not impose any restriction on the value of the preset number.

[0197] Furthermore, the alarm levels include: level one, level two, etc.

[0198] Furthermore, the alarm method includes: an alarm sound from a speaker, an email, a telephone call, etc.

[0199] Through the above implementation, an alarm message can be issued when the target number of defective images is greater than a preset number. In addition, by encrypting the alarm message, it is possible to prevent the alarm message from being tampered with, thereby improving the security of the alarm message. At the same time, the alarm method is determined according to the alarm level, and the alarm message can be sent in an appropriate alarm method, making the sending of the alarm message more humane.

[0200] In at least one embodiment of the present application, when the target score is greater than or equal to the score threshold, the determination unit 111 determines the image to be detected as a flawless image.

[0201] It can be seen from the above technical solutions that the present application determines the target convolution layer with the best feature extraction performance so as to better extract the features of the image to be detected, thereby enabling the present application to improve the accuracy of defect detection.

[0202] like Figure 3 , is a schematic diagram of the structure of an electronic device of a preferred embodiment of the image defect detection method of the present application.

[0203] In one embodiment of the present application, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and a computer program stored in the memory 12 and executable on the processor 13, such as an image defect detection program.

[0204] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the electronic device 1 may also include input and output devices, network access devices, buses, etc.

[0205] The processor 13 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 13 is the computing core and control center of the electronic device 1, and uses various interfaces and lines to connect various parts of the entire electronic device 1, and obtain the operating system of the electronic device 1 and various installed applications, program codes, etc.

[0206] The processor 13 obtains the operating system of the electronic device 1 and various installed applications. The processor 13 obtains the applications to implement the steps in the above-mentioned various image defect detection method embodiments, for example Figure 1 Steps shown.

[0207] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and acquired by the processor 13 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the acquisition process of the computer program in the electronic device 1. For example, the computer program may be divided into an acquisition unit 110, a determination unit 111, a generation unit 112, an encryption unit 113, a sending unit 114, a division unit 115, a training unit 116, a testing unit 117, an adjustment unit 118, a calculation unit 119, and an extraction unit 120.

[0208] The memory 12 can be used to store the computer program and / or module, and the processor 13 implements various functions of the electronic device 1 by running or acquiring the computer program and / or module stored in the memory 12, and calling the data stored in the memory 12. The memory 12 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 12 can include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0209] The memory 12 may be an external memory and / or an internal memory of the electronic device 1. Furthermore, the memory 12 may be a memory in a physical form, such as a memory stick, a TF card (Trans-flash Card), and the like.

[0210] If the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is obtained by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0211] The computer program includes computer program code, which may be in source code form, object code form, an accessible file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0212] Combination Figure 1The memory 12 in the electronic device 1 stores a plurality of instructions to implement an image defect detection method, and the processor 13 can obtain the plurality of instructions to implement: obtaining an image to be detected; determining the field to which the image to be detected belongs, and based on the field, determining a target convolution layer from a pre-built convolutional neural network; using the target convolution layer to extract features to be detected of the image to be detected; determining a target score of the image to be detected based on the features to be detected; determining a score threshold corresponding to the field; and determining the image to be detected as a defective image when the target score is less than the score threshold.

[0213] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0215] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0216] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0217] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present application is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0218] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0219] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A method for detecting image defects, characterized in that: The image defect detection method comprises: Acquire the image to be detected; Determine the domain to which the image to be detected belongs, and based on the domain, determine a target convolutional layer from a pre-built convolutional neural network; Extracting the features to be detected of the image to be detected by using the target convolution layer; Determining a target score of the image to be detected according to the feature to be detected; Acquire a plurality of training images in the field, wherein the plurality of training images include defective images and normal images; Determining a score threshold corresponding to the field includes: inputting the defect image and the normal image into the target convolution layer for convolution processing to obtain a first feature map of the defect image and a second feature map of the normal image; determining a target pooling layer corresponding to the target convolution layer from the convolution neural network; performing pooling processing on the first feature map and the second feature map using the target pooling layer to obtain a first vector of the defect image and a second vector of the normal image; inputting the first vector and the second vector into a pre-trained Gaussian mixture model to obtain a first score of the defect image and a second score of the normal image; determining a mean score of the first score and the second score, and determining a standard deviation of the first score and the second score; multiplying the standard deviation by a preset value to obtain a calculation result, and subtracting the mean score from the calculation result to obtain the score threshold; When the target score is less than the score threshold, the image to be detected is determined as a defective image.

2. The image defect detection method according to claim 1, wherein: The method of acquiring the image to be detected includes one or more of the following methods: Determine a detection object, and photograph the detection object using a camera device to obtain the image to be detected; and / or Obtaining an image with a preset mark from a configuration library as the image to be detected; and / or The image to be detected is crawled from a preset web page using a web crawler tool.

3. The image defect detection method according to claim 1, characterized in that: Determining a target convolutional layer from a pre-built convolutional neural network based on the field includes: Extracting a convolution kernel from any convolution layer of the convolutional neural network, wherein the convolutional neural network includes a plurality of convolution layers; Use each extracted convolution kernel to perform convolution operation on each training image to obtain multiple feature images on each convolution layer; Perform pooling on each feature image on each convolution layer to obtain multiple low-dimensional vectors on each convolution layer; Use the T-SNE algorithm to visualize each low-dimensional vector and obtain the distribution images of multiple training images on each convolutional layer; The target convolution layer is determined according to the distribution image of each convolution layer.

4. The image defect detection method according to claim 1, characterized in that: Determining the target score of the image to be detected according to the feature to be detected includes: Performing mean pooling processing on the feature to be detected to obtain a target vector; The target vector is input into the Gaussian mixture model to obtain the target score.

5. The image defect detection method according to claim 1, characterized in that: After determining that the image to be detected is a defective image, the image defect detection method further includes: determining a target number of said defective images; When the target number is greater than a preset number, generating a warning message according to the defective image; Encrypting the warning information using a symmetric encryption algorithm to obtain a ciphertext; Determining an alarm level of the ciphertext according to the target quantity; Determine an alarm mode according to the alarm level; The ciphertext is sent in the alarm mode.

6. The image defect detection method according to claim 1, characterized in that: The image defect detection method further comprises: When the target score is greater than or equal to the score threshold, the image to be detected is determined to be a flawless image.

7. An image defect detection device, characterized in that: The image defect detection device comprises: An acquisition unit, used for acquiring an image to be detected; A determination unit, used to determine the field to which the image to be detected belongs, and based on the field, determine a target convolution layer from a pre-built convolutional neural network; An extraction unit, used for extracting the features to be detected of the image to be detected by using the target convolution layer; The determining unit is further used to determine the target score of the image to be detected according to the feature to be detected; The acquisition unit is further used to acquire a plurality of training images in the field, wherein the plurality of training images include defective images and normal images; The determination unit is also used to determine the score threshold corresponding to the field, including: inputting the defect image and the normal image into the target convolution layer for convolution processing to obtain a first feature map of the defect image and a second feature map of the normal image; determining a target pooling layer corresponding to the target convolution layer from the convolution neural network; performing pooling processing on the first feature map and the second feature map using the target pooling layer to obtain a first vector of the defect image and a second vector of the normal image; inputting the first vector and the second vector into a pre-trained Gaussian mixture model to obtain a first score of the defect image and a second score of the normal image; determining a mean score of the first score and the second score, and determining a standard deviation of the first score and the second score; multiplying the standard deviation by a preset value to obtain a calculation result, and subtracting the mean score from the calculation result to obtain the score threshold; The determining unit is further configured to determine the image to be detected as a defective image when the target score is less than the score threshold.

8. An electronic device, characterized in that: The electronic device comprises: a memory storing at least one instruction; and A processor is configured to obtain instructions stored in the memory to implement the image defect detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is acquired by a processor in an electronic device to implement the image defect detection method according to any one of claims 1 to 6.

Images