Machine learning based defect detection method, apparatus, and electronic device

Abstract

The application provides a machine learning-based defect detection method, device and electronic equipment, and relates to the field of machine learning; the method comprises the following steps: acquiring an industrial product surface normal image and a surface defect image, and performing pretreatment to obtain an industrial product surface image dataset; wherein, the pretreatment process is image trimming on the acquired image according to a target square template, including image cropping and black pixel filling; a machine learning-based defect detection model is trained according to the surface image dataset, the model is a convolutional neural network model, and the architecture of the model comprises a convolutional layer, an activation function and a plurality of pooling layers; an industrial product surface image photographed in real time is received and pretreated to obtain a surface image to be detected; the surface image to be detected is input into the machine learning-based defect detection model, and a defect detection result is output. The application improves the model detection accuracy by limited expansion of industrial product sample data and pixel size limitation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a defect detection method and device based on machine learning and electronic equipment. BACKGROUND

[0002] Deep learning algorithms have made great breakthroughs in image recognition. Convolutional neural networks, as representatives of deep learning algorithms, have achieved excellent results in image classification, object detection, and image segmentation tasks. Through multiple convolution and pooling operations, convolutional neural networks can effectively extract feature representations of images, enabling automatic understanding and recognition of image content. For example, in face recognition and object detection, deep learning algorithms have surpassed human level, providing more convenient and secure technical support for society.

[0003] AI defect detection, as a key technology of intelligent manufacturing, is leading the development of industrial production towards intelligent manufacturing and becoming a new trend of quality control in the intelligent era. Currently, machine vision defect technology based on deep learning algorithms is widely used in quality detection processes in industrial production. The existing AI defect detection method based on deep learning algorithms has the problems of few training samples and image distortion when detecting defects on industrial products. Specifically, there are few industrial products with defects in production or high recognition difficulty, which cannot obtain a large number of training samples. Secondly, based on the different shapes and shooting angles of industrial products, the product pictures are different. When training the model, the product pictures are first scaled to a standard square template size to obtain training samples. The training samples enter the graphics processor for feature recognition by the square calculation matrix. Because the target shape is obtained by scaling the picture, there is an image distortion phenomenon in application, which leads to low accuracy of the model in subsequent defect detection application. SUMMARY

[0004] The present application aims to provide a defect detection method and device based on machine learning and electronic equipment, which aims to solve the problem of poor precision when machine learning with small sample data in industrial production. On the one hand, by obtaining data of industrial products under different lighting conditions and different angles, the training samples are increased. On the other hand, the target shape image is not obtained directly by scaling, which effectively avoids image distortion and improves the detection precision of the model.

[0005] To achieve the above purpose, the present application proposes the following technical solutions:

[0006] In a first aspect, a defect detection method based on machine learning is proposed, comprising the following steps:

[0007] An image dataset of an industrial product surface is obtained by acquiring normal images and defect images of the industrial product surface and performing preprocessing on the images. The preprocessing process includes image trimming according to a target square template, which includes image cropping and black pixel filling.

[0008] A machine learning-based defect detection model is trained according to the image dataset of the industrial product surface. The machine learning-based defect detection model is a convolutional neural network model, and its architecture includes convolutional layers, activation functions, and multiple pooling layers.

[0009] A real-time captured image of the industrial product surface is received, and the image is preprocessed to obtain a surface image to be detected.

[0010] The surface image to be detected is input into the machine learning-based defect detection model, and a defect detection result is output.

[0011] Further, the process of acquiring the defect images of the industrial product surface includes:

[0012] An image of the industrial product surface is acquired, and a target detection algorithm is used to identify defects in the image. The image of the industrial product surface includes images of the same industrial product under different lighting conditions.

[0013] For any defect identified by the target detection algorithm, a target classification algorithm is used to classify the defect and attach a class label.

[0014] The target detection algorithm identifies defects in the image of the industrial product surface, including determining the number and location of the defects. The identification process includes sequentially performing image filtering, threshold filtering and binarization, and morphological transformation noise reduction on each image under different lighting conditions to obtain defects of the industrial product under each lighting condition. The industrial product image under each lighting condition is cropped to obtain several defect images containing at least one defect. The defect images are combined under the same pixel conditions to obtain an industrial product defect image. The standard template box is used to identify and frame each defect in the industrial product defect image.

[0015] Further, the process of preprocessing the normal images and defect images of the industrial product surface includes:

[0016] According to the pixel size of the image to be processed, a target square template is selected from a first square template and a second square template. The pixel size of the first square template is larger than that of the second square template.

[0017] Placing the image in the pixel size of the target square template, image cropping the part of the image beyond the pixel size of the target square template and black pixel filling the part of the image missing in the pixel size of the target square template until the pixel size of the image is equal to the pixel size of the target square template.

[0018] Further, the process of receiving the real-time captured industrial product surface image, pre-processing the industrial product surface image to obtain the surface image to be detected is:

[0019] The industrial product surface image is sequentially processed by image filtering algorithm, threshold filtering binarization processing and morphological transformation method noise reduction processing to obtain a processed image; wherein, for any industrial product, at least real-time capture the industrial product surface image under two illumination conditions;

[0020] According to the pixel size of the surface image to be detected required by the model, the pixel size of the processed image is image trimmed.

[0021] Further, the process of identifying and framing each defect by using a standard template frame is:

[0022] For any two adjacent defects, calculate the minimum distance L between the two defects;

[0023] Determine the relationship between the minimum distance and the first distance threshold L1 and the second distance threshold L2, and determine the framing method of the defect according to the relationship; wherein, L1>L2;

[0024] When L≥L1, one of the defects is identified and framed by using the standard template frame;

[0025] When L1>L≥L2, two defects are respectively framed by using irregular curves, and the two framed defects do not overlap;

[0026] When L2>L, two defects are clustered and framed as one defect class by using the standard template frame.

[0027] Further, when the target square template is the first square template, the pre-processed image is reduced to the pixel size of the second square template.

[0028] Secondly, a defect detection device based on machine learning is provided, comprising:

[0029] The acquisition module is used to acquire normal images and defective images of industrial product surfaces and perform preprocessing to obtain an industrial product surface image dataset; wherein, the preprocessing process is to perform image trimming on the acquired images according to the target square template, and the image trimming includes image cropping and black pixel filling;

[0030] The training module is used to train a machine learning-based defect detection model based on the industrial product surface image dataset. The machine learning-based defect detection model is a convolutional neural network model, whose architecture includes convolutional layers, activation functions, and multiple pooling layers.

[0031] The receiving module is used to receive real-time captured images of the surface of industrial products, preprocess the images of the industrial product surface, and obtain an image of the surface to be inspected.

[0032] The defect detection module is used to input the image of the surface to be detected into the machine learning-based defect detection model and output the defect detection result.

[0033] Furthermore, the execution unit for acquiring the surface defect image of the industrial product by the acquisition module includes:

[0034] An acquisition and recognition unit is used to acquire surface images of industrial products and to identify defects in the surface images of industrial products using a target detection algorithm. The surface images of industrial products include surface images of the same industrial product under different lighting conditions.

[0035] The defect classification unit is used to classify any defect identified by the target detection algorithm and attach a class label to the defect using the target classification algorithm.

[0036] The acquisition and recognition unit uses a target detection algorithm to identify defects in the surface images of the industrial products, including determining the number and location of the defects. The recognition process is as follows: each surface image of the industrial product under different lighting conditions is sequentially processed by an image filtering algorithm, threshold filtering binarization, and morphological transformation to reduce noise, thereby obtaining the defects of the industrial product under each lighting condition; the processed industrial product images under each lighting condition are cropped to obtain several defect images containing at least one defect; the several defect images are stitched together under the same pixel conditions to obtain an industrial product defect image; and each defect is identified and selected using a standard template box in the industrial product defect image.

[0037] Furthermore, the process of identifying and selecting each defect using a standard template frame in the acquisition and recognition unit is as follows:

[0038] For any two adjacent defects, calculate the minimum distance L between the two defects;

[0039] Judge the relationship between the minimum distance and the first distance threshold L1, the second distance threshold L2, and determine the box selection mode of the defect according to the relationship; wherein, L1>L2;

[0040] When L is greater than or equal to L1, the standard template box is used to identify and box one of the defects respectively;

[0041] When L1 is greater than L and less than or equal to L2, two of the defects are respectively boxed by irregular curves, and the two defects after being boxed do not overlap;

[0042] When L2 is greater than L, the two defects are clustered into one defect class by using the standard template box.

[0043] In a third aspect, an electronic device is provided, including a computer program stored in a computer readable storage medium; when a processor of the electronic device reads the computer program from the computer readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of the above method.

[0044] From the above technical solutions, the technical solutions of the present application have the following beneficial effects:

[0045] The defect detection method, device and electronic device based on machine learning disclosed in the present application include: obtaining an industrial product surface normal image and an industrial product surface defect image and preprocessing to obtain an industrial product surface image dataset; wherein, the preprocessing process is image modification according to a target square template, including image cropping and black pixel filling, which can effectively ensure the morphological characteristics of the defect and the size of the target square template will not be distorted due to size scaling; training a defect detection model based on machine learning according to the industrial product surface image dataset, which is a convolutional neural network model, and its architecture includes convolutional layers, activation functions and multiple pooling layers; receiving a real-time captured industrial product surface image and preprocessing to obtain a surface image to be detected; inputting the surface image to be detected into the defect detection model based on machine learning to output a defect detection result. The present application increases defect image sample data by limited expansion of industrial product images, and ensures the authenticity of training samples by using pixel size limited method, thereby improving the accuracy of defect detection from two aspects of improving the accuracy of training model and providing more product images.

[0046] Specifically, increasing the defect sample data on the one hand processes more defects that cannot be detected under white light conditions by obtaining pictures of industrial products under different light conditions, and on the other hand increases the defect sample by separately cropping and identifying different defects, so as to increase the training sample during training. For the image size, the target square template detected by the model is used to limit the size of the sample image and the image to be detected from the pixel size, and the target size image is obtained in the way of cropping or black pixel filling, avoiding the phenomenon of image distortion caused by direct size scaling; using more accurate model training samples and more real defect images can significantly improve the precision of the present application in detecting defects of industrial products; at the same time, since the image is processed into a regular square target size at the beginning of training, in actual production, the target size image can be directly scaled down in length and width without affecting image feature recognition, so that the image of small size and real defect characteristics can be input to reduce the pressure on the graphics card, and the enterprise does not need to purchase expensive graphics cards with high computing power, thereby realizing the control of enterprise production cost.

[0047] It should be understood that all combinations of the foregoing concepts and additional concepts described in greater detail below can be seen as being part of the inventive subject matter of the present disclosure provided such concepts are not mutually inconsistent.

[0048] The foregoing and other aspects, embodiments and features of the present teachings can be better understood from the following description of the present teachings taken in conjunction with the accompanying drawings. Additional aspects, embodiments and features of the present teachings will be apparent from the following description of the exemplary embodiments, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings are not necessarily drawn to scale. In the drawings, each identical, or nearly identical, component that is illustrated in various figures is represented with a like numeral. For purposes of clarity, not every component is called out in every drawing. There now will be described, by way of example, embodiments of various aspects of the application. The drawings are schematic representations for purposes of the explanation of the present teachings.

[0050] Figure 1 Flow chart of the machine learning-based defect detection method disclosed in the present embodiment;

[0051] Figure 2 Flow chart of obtaining the surface defect image of the industrial product disclosed in the present embodiment;

[0052] Figure 3 Flow chart of recognizing and framing each defect using a standard template frame disclosed in the present embodiment;

[0053] Figure 4 Flow chart of pre-processing the normal and defect images of the surface of the industrial product disclosed in the present embodiment;

[0054] Figure 5 A flowchart for obtaining a surface image to be detected according to the embodiment;

[0055] Figure 6 A structural block diagram of a defect detection device based on machine learning according to the embodiment;

[0056] Figure 7 A structural block diagram of an electronic device according to the embodiment. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the described embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort belong to the scope of protection of the present application. Unless otherwise defined, the technical terms or scientific terms used herein should have the usual meanings understood by those of ordinary skill in the art.

[0058] The terms "first", "second", and similar terms used in the patent application specification and claims of the present application do not represent any order, number or importance, but are only used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular forms "a", "an" or "the" and the like do not represent a quantity limitation, but represent the existence of at least one. The terms "include" or "contain" and the like mean that the elements or objects appearing before "include" or "contain" cover the features, integers, steps, operations, elements and / or components listed after "include" or "contain", and do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or sets thereof.

[0059] Based on the existing AI defect detection method based on deep learning algorithm, there are problems of few training samples and picture distortion when performing defect detection on industrial products, resulting in low detection accuracy of the finally trained defect detection model; the present application aims to solve the above problems, and proposes a defect detection method, device and electronic equipment based on machine learning, which improves the accuracy of the training detection model by increasing the training samples and improving the picture accuracy.

[0060] The defect detection method, device and electronic equipment based on machine learning disclosed by the present application will be further specifically introduced below with reference to the specific embodiments shown in the drawings.

[0061] In combination with Figure 1As shown, the embodiment discloses a machine learning-based defect detection method, comprising the following steps:

[0062] In step S102, the normal image of the industrial product surface and the defect image of the industrial product surface are acquired and preprocessed to obtain the industrial product surface image dataset; wherein the preprocessing process is to perform image trimming on the acquired image according to a target square template, and the image trimming includes image cropping and black pixel filling;

[0063] The industrial product surface image dataset is used as a sample database for model training. The purpose of preprocessing the acquired normal image and defect image of the industrial product surface is to retain the learning degree to the greatest extent. In order to solve the problem of image distortion caused by directly scaling the image to obtain the target size to meet the application requirements of the image processor, the scheme uses the long-cut-short-fill method to trim the image to be processed to the target size, thereby solving the image distortion problem of the image sent to the image processor.

[0064] Specifically, the process of acquiring the defect image of the industrial product surface is as shown in the steps, comprising: Figure 2 As shown in the steps, comprising: in step S202, acquiring an industrial product surface image, using a target detection algorithm to identify defects in the industrial product surface image, and the industrial product surface image includes industrial product surface images of the same industrial product under different light conditions; Because the causes of defects are different, some defects cannot be completely detected under white light conditions, therefore, the scheme detects various defects by shooting images of the industrial product under different light conditions; different light conditions such as white light, binary stripe light, color light, ripple light, and Gray code structured light.

[0065] The target detection algorithm identifies the defects in the industrial product surface image, including determining the number and position of the defects, and the identification process is: sequentially performing filtering processing, threshold filtering binarization processing and morphological transformation method noise reduction processing on each industrial product surface image under different illumination conditions to obtain the defects of the industrial product under each illumination condition; the filtering processing is performed by using a high-pass filtering algorithm to determine the position of the defects in the image, and the main purpose of the noise reduction processing performed by using the morphological transformation method is to remove the background noise of the image; each industrial product image under each illumination condition after the shearing processing obtains a plurality of defect pictures containing at least one defect; the plurality of defect pictures are spliced under the same pixel condition to obtain an industrial product defect image, that is, the industrial product image is segmented according to the positions of the defects detected under different illumination conditions, and then assembled on the same picture for defect summarization; the standard template frame is used to identify and frame each defect in the industrial product defect image, and the defect sample data is further increased; in step S204, any defect identified by the target detection algorithm is classified by using a target classification algorithm and is attached with a class label, so that the model training can accurately learn the common features of each type of defect. Optionally, when the different illumination conditions include the illumination condition of the Gray code structured light, the image decoding is performed before the filtering of the captured image.

[0066] In the embodiment, in order to facilitate the acquisition of defect samples and more accurately classify the defects, the process of identifying and framing each defect by using the standard template frame is implemented as shown in Figure 3 The process of identifying and framing each defect by using the standard template frame is implemented as shown in

[0067] In step S104, a defect detection model based on machine learning is trained according to the industrial product surface image dataset, and the defect detection model based on machine learning is a convolutional neural network model, and the architecture of the model includes a convolutional layer, an activation function and a plurality of pooling layers.

[0068] Step S106, receiving the real-time captured industrial product surface image, pre-processing the industrial product surface image to obtain a to-be-detected surface image;

[0069] The pre-processing of the industrial product surface image is different from the process of constructing the industrial product surface image dataset. The pre-processing is aimed to obtain a to-be-detected surface image that is better for defect recognition and meets the model requirements, and to improve the model detection speed. In combination with the above Figure 4 As shown in the figure, the process of pre-processing the industrial product surface image to obtain a to-be-detected surface image includes: step S402, sequentially performing filtering processing, threshold filtering and binarization processing, and morphological transformation method noise reduction processing on the industrial product surface image to obtain a processed image; wherein, for any industrial product, at least two industrial product surface images under two illumination conditions are captured in real time; step S404, performing pixel size image modification on the processed image according to the pixel size of the to-be-detected surface image required by the model; the image modification process is the same as the process of obtaining the industrial product surface image dataset, mainly including image cropping and black pixel filling.

[0070] Step S108, inputting the to-be-detected surface image into the machine learning-based defect detection model to output a defect detection result.

[0071] As an optional embodiment, as Figure 5 As shown in the figure, the process of pre-processing the industrial product surface normal image and the industrial product surface defect image includes: step S502, selecting one of a first square template and a second square template as a target square template according to the pixel size of the to-be-processed image; wherein, the pixel size of the first square template is larger than the pixel size of the second square template, for example, the pixel size of the first square template is 1024x1024, and the pixel size of the second square template is 512x512; step S504, placing the image in the pixel size of the target square template, performing image cropping on the part of the image that is outside the pixel size of the target square template and filling black pixels for the part of the image that is missing in the pixel size of the target square template, until the pixel size of the image is equal to the pixel size of the target square template.

[0072] In order to retain the learning degree of the defect, a first square template with a large pixel size is used for the maximum retention of the defect with a large area. Therefore, in order to facilitate model training, when the target square template is the first square template, the preprocessed image is reduced to the pixel size of the second square template. Optionally, when the defect size exceeds the maximum square template, a square selection box is constructed around the edge of the defect. When the model is trained, the square selection box is reduced to the pixel size of the second square template, that is, the pixel size required for training. In this way, the scheme can fully retain the morphological characteristics of the defect without changing the authenticity of the image.

[0073] Since the image is processed into a regular square target size at the beginning of training, in actual production, the image of the target size can be directly scaled down in length and width without affecting image feature recognition. Therefore, by inputting a small size image that truly reflects the characteristics of the defect, the pressure on the graphics card during model training and application can be reduced. Enterprises do not need to purchase expensive graphics cards with high computing power to quickly identify defects, thereby controlling the production cost and time cost of enterprises.

[0074] At the same time, when the defect detection model based on machine learning outputs the defect detection result, a square template that can completely surround the defect is selected according to the pixel size of the defect to identify the defect position and indicate the defect size.

[0075] As an optional implementation, when the defect detection model based on machine learning is applied to identify defects, the defects are first screened, and defects that do not affect the application of industrial products are filtered out, and only defects that affect the application of industrial products are identified. The specific screening method is as follows: all defects identified by the model are identified by a square template, and the pixel size of the defects in each square template is counted. The ratio of the pixel size of the defects in each square template to the pixel size of the standard defect template is calculated. The ratio is compared with the standard defect threshold value, and the defect is identified when the ratio is not less than the standard defect threshold value. The standard defect threshold value is the ratio of the pixel size of the smallest defect affecting the application of industrial products to the pixel size of the standard defect template, such as 0.2. Through the above method, smaller defects can be effectively filtered out to ensure product yield in industrial product quality supervision.

[0076] The defect detection method based on machine learning disclosed in the above embodiments can increase defect training samples and improve defect detection rate after preprocessing by obtaining data of industrial products under different lighting conditions and at different angles. On the other hand, the method of shortening the long and complementing the short is used to realize the target shape image size of the model, avoiding direct scaling to obtain the target shape image, avoiding photo distortion caused by image scaling, not only improving the training accuracy of the model when training, but also being able to output more accurate detection results due to the input of more real detection images.

[0077] In embodiments of this application, an electronic device is also provided, including a computer program stored in a computer-readable storage medium. When the processor of the electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of the aforementioned machine learning-based defect detection method. Taking an electronic device running on a computer as an example, such as... Figure 7 As shown, the electronic device may include one or more (only one is shown in the figure) processors (processors may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory for storing data, and a transmission device for communication functions. Those skilled in the art will understand that... Figure 7 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device.

[0078] The aforementioned programs may run in a processor or be stored in memory, i.e., in a computer-readable medium. Computer-readable media include both permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media do not include temporary computer-readable media such as modulated data signals and carrier waves. These computer programs may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus provide for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented through different modules, corresponding to different method steps.

[0079] In this embodiment, such a device or system is provided, which can be called a machine learning-based defect detection device, such as... Figure 6As shown, comprising: an acquisition module, configured to acquire and pre-process normal images of industrial product surfaces and defect images of industrial product surfaces to obtain an industrial product surface image dataset; wherein the pre-processing process is image trimming according to a target square template, and the image trimming includes image cropping and black pixel filling; a training module, configured to train a machine learning-based defect detection model according to the industrial product surface image dataset, wherein the machine learning-based defect detection model is a convolutional neural network model, and the architecture of the convolutional neural network model includes a convolutional layer, an activation function, and a plurality of pooling layers; a receiving module, configured to receive a real-time captured industrial product surface image, pre-process the industrial product surface image to obtain a surface image to be detected; and a defect detection module, configured to input the surface image to be detected into the machine learning-based defect detection model to output a defect detection result.

[0080] The device is used to implement the steps of the machine learning-based defect detection method disclosed in the above embodiments, which have been described and will not be repeated here.

[0081] For example, the execution unit of the acquisition module for acquiring the defect images of the industrial product surfaces includes: an acquisition and recognition unit, configured to acquire industrial product surface images, recognize defects in the industrial product surface images by using a target detection algorithm, and the industrial product surface images include industrial product surface images of the same industrial product under different illumination conditions; and a defect classification unit, configured to classify any defect recognized by the target detection algorithm by using a target classification algorithm and attach a class label to the defect; wherein the acquisition and recognition unit recognizes the defects in the industrial product surface images by using the target detection algorithm, including determining the number and position of the defects, and the recognition process includes: sequentially performing filtering processing, threshold filtering binarization processing, and morphological transformation method noise reduction processing on each of the industrial product surface images under different illumination conditions by using an image filtering algorithm to obtain defects of the industrial product under each illumination condition; performing cutting processing on the industrial product images under each illumination condition to obtain a plurality of defect pictures each containing at least one defect; combining the plurality of defect pictures under the same pixel condition to obtain an industrial product defect image; and recognizing and framing each defect in the industrial product defect image by using a standard template frame.

[0082] For another example, the standard template box recognition and framing of each defect in the acquisition recognition unit is as follows: for any two adjacent defects, the minimum distance L between the two defects is calculated; the relationship between the minimum distance and the first distance threshold L1 and the second distance threshold L2 is judged, and the framing mode of the defects is determined according to the relationship; wherein L1>L2; when L≥L1, the standard template box is used to recognize and frame one of the defects; when L1>L≥L2, two irregular curves are used to frame the two defects respectively, and the two defects after framing do not overlap; when L2>L, the two defects are clustered and framed as one defect class by using the standard template box.

[0083] For another example, the execution unit of the acquisition module for pre-processing the normal image of the industrial product surface and the industrial product surface defect image includes: a selection unit configured to select one of a first square template and a second square template as a target square template according to the pixel size of the image to be processed; wherein the pixel size of the first square template is greater than the pixel size of the second square template; a first image trimming unit configured to place the image in the pixel size of the target square template, perform image cropping on the part of the image that is outside the pixel size of the target square template, and fill black pixels for the part of the image that is missing in the pixel size of the target square template, until the pixel size of the image is equal to the pixel size of the target square template. Furthermore, when the selection unit selects the first square template as the target square template, the image trimming unit is further configured to reduce the pre-processed image to the pixel size of the second square template to meet the image size requirement of model training.

[0084] For another example, the execution unit of the receiving module for receiving the real-time captured industrial product surface image, pre-processing the industrial product surface image, and obtaining the image to be detected includes: an image processing unit configured to sequentially perform filtering processing, threshold filtering binarization processing, and morphological transformation method noise reduction processing on the industrial product surface image to obtain a processed image; wherein at least two industrial product surface images of any industrial product under two kinds of illumination conditions are captured in real time; a second image trimming unit configured to perform image trimming of the pixel size of the processed image according to the pixel size of the image to be detected required by the model.

[0085] The embodiment of the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is used to execute the above method.

[0086] The machine learning-based defect detection method, device and electronic equipment disclosed by the present application can increase defect sample data by adjusting different illumination conditions of industrial products and segmenting defects, ensure the authenticity of the defect sample data by limiting the pixel size, obtain an industrial product surface image data set that can accurately reflect the defects of the industrial product, train a more accurate defect detection model by using the limited but accurate defect sample data in the data set, and finally improve the overall detection rate of defect detection. Compared with the defect detection rate of 60%-70% in the prior art, the defect detection rate of the present application can be improved to more than 80%, and while significantly improving the defect detection rate, the present application can also stabilize the yield of the industrial product and control the production cost of the enterprise.

[0087] The phenomenon of image distortion caused by direct size scaling is avoided; more accurate model training samples and more real defect images can significantly improve the accuracy of the present application in detecting defects of industrial products, and do not increase the application cost.

[0088] Although the present application has been disclosed above with reference to the preferred embodiments, it is not intended to limit the present application. Those skilled in the art can make various modifications and improvements without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application shall be subject to the scope defined by the claims.

Claims

1. A machine learning based defect detection method, characterized by, The method comprises the following steps: obtaining an industrial product surface normal image and an industrial product surface defect image and preprocessing to obtain an industrial product surface image dataset; obtaining the industrial product surface defect image, specifically comprising: obtaining the industrial product surface images of the same industrial product under different lighting conditions; sequentially filtering the industrial product surface images of the same industrial product using an image filtering algorithm, threshold filtering binarization processing and morphological transformation method noise reduction processing to obtain the defects of the industrial product under each lighting condition; shearing the industrial product surface image to obtain several defect pictures containing at least one defect; assembling the defect pictures according to the detected defect positions on the same picture for defect summary, calculating the minimum distance L between any two adjacent defects; judging the relationship between the minimum distance and the first distance threshold L1 and the second distance threshold L2, and determining the framing mode of the defects according to the relationship; wherein L1>L2; when L≥L1, a standard template frame is used to identify and frame one of the defects; when L1>L≥L2, two defects are framed using an irregular curve, and the two framed defects do not overlap; when L2>L, the two defects are clustered and framed as one defect class using the standard template frame; and adding a class label to the defects; preprocessing the industrial product surface normal image and the industrial product surface defect image, specifically comprising: selecting one of a first square template and a second square template as a target square template according to the pixel size of the image to be processed, wherein the pixel size of the first square template is larger than that of the second square template, and the pixel size of the second square template is the training required pixel size; placing the industrial product surface normal image and the industrial product surface defect image within the pixel size of the target square template, and performing image cropping on the part of the industrial product surface normal image and the industrial product surface defect image that exceeds the pixel size of the target square template and filling the missing part of the image with black pixels until the pixel size of the industrial product surface normal image and the industrial product surface defect image is equal to that of the target square template; when the target square template is the first square template, reducing the preprocessed image to the pixel size of the second square template; when the defect size exceeds the maximum square template, constructing a square selection frame around the edge of the defect, and then reducing the square selection frame to the pixel size of the second square template; training a machine learning-based defect detection model according to the industrial product surface image dataset, wherein the machine learning-based defect detection model is a convolutional neural network model, and the architecture of the model comprises a convolutional layer, an activation function and multiple pooling layers; receiving a real-time captured industrial product surface image, preprocessing the industrial product surface image to obtain a surface image to be detected; The surface image to be detected is input into the machine learning-based defect detection model, and a defect detection result is output; all defects identified by the defect detection model are respectively marked with a square template, and the pixel size of the defects in each square template is counted; the ratio of the pixel size of the defects in each square template to the pixel size of a standard defect template is calculated; the ratio is compared with a standard defect threshold value, and the defect is marked when the ratio is not less than the standard defect threshold value; wherein the standard defect threshold value is the ratio of the pixel size of the smallest defect affecting the application of an industrial product to the pixel size of the standard defect template.

2. A machine learning based defect detection apparatus, characterized by, Comprise: The acquisition module is used for acquiring and preprocessing industrial product surface normal images and industrial product surface defect images to obtain an industrial product surface image dataset; wherein, the industrial product surface defect images are acquired, specifically including: acquiring industrial product surface images of the same industrial product under different lighting conditions; sequentially performing filtering processing, threshold filtering binarization processing and morphological transformation method noise reduction processing on the industrial product surface images of the same industrial product to obtain defects of the industrial product under each lighting condition; performing shearing processing on the industrial product surface images to obtain a plurality of defect pictures containing at least one defect; assembling the defect pictures according to the detected defect positions on the same picture for defect summarization, calculating the minimum distance L between any two adjacent defects; judging the relationship between the minimum distance and the first distance threshold L1 and the second distance threshold L2, and determining the bounding mode of the defects according to the relationship; wherein, L1>L2; when L≥L1, a standard template frame is used to recognize and bound one defect; when L1>L≥L2, an irregular curve is used to bound two defects, and the two defects after bounding do not overlap; when L2>L, the standard template frame is used to cluster and bound two defects as one defect class; a class label is added to the defects; the industrial product surface normal images and the industrial product surface defect images are preprocessed, specifically including: selecting one of a first square template and a second square template as a target square template according to the pixel size of the image to be processed, wherein the pixel size of the first square template is larger than that of the second square template, and the pixel size of the second square template is the training required pixel size; placing the industrial product surface normal images and the industrial product surface defect images within the pixel size of the target square template, performing image cropping on the part of the industrial product surface normal images and the industrial product surface defect images that exceeds the pixel size of the target square template, and supplementing black pixels to the part of the image that lacks the pixel size of the target square template, until the pixel size of the industrial product surface normal images and the industrial product surface defect images is equal to that of the target square template; when the target square template is the first square template, the preprocessed image is reduced to the pixel size of the second square template; when the defect size exceeds the maximum square template, a square selection frame is constructed around the edge of the defect, and then the square selection frame is reduced to the pixel size of the second square template; The training module is used for training a machine learning-based defect detection model according to the industrial product surface image dataset, wherein the machine learning-based defect detection model is a convolutional neural network model, and the architecture of the model includes a convolutional layer, an activation function and a plurality of pooling layers; The receiving module is used for receiving real-time captured industrial product surface images, preprocessing the industrial product surface images to obtain a surface image to be detected. The defect detection module is used for inputting the surface image to be detected into the machine learning-based defect detection model, outputting a defect detection result, marking all defects identified by the defect detection model with square templates respectively, counting pixel sizes of the defects in each square template, calculating a ratio of the pixel sizes of the defects in each square template to a standard defect template pixel size, judging a size of the ratio and the standard defect threshold value, and marking the defects when the ratio is not less than the standard defect threshold value, wherein the standard defect threshold value is a ratio of a minimum defect pixel size affecting industrial product application to the standard defect template pixel size.

3. An electronic device, comprising: The computer program is stored in a computer readable storage medium, and when a processor of an electronic device reads the computer program from the computer readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of the method in claim 1.

Images