Defect detection model training method and defect detection method, device and equipment

Abstract

The present application discloses a defect detection model training method. The defect detection model training method includes obtaining a first reference image, a second reference image and a test image, calibrating the first reference image, the second reference image and the test image to obtain a calibration image, synthesizing the calibration image and the test image to obtain an input image, and passing the input image into a neural network for defect detection model training. If a certain position of one of the first reference image, the second reference image and the test image has a defect, it can be clearly highlighted, so that the neural network can train a more robust model, thereby improving the accuracy of defect detection.

Description

Technical Field

[0001] The present disclosure relates to the field of defect detection, and in particular to a defect detection model training method and a defect detection method, device and equipment. Background Art

[0002] Defect detection is a very important application in industry. Due to the variety of defects, it is difficult for traditional machine vision algorithms to fully model and migrate defect features. They are not very reusable and require different working conditions, which will waste a lot of manpower costs. In order to reduce the waste of human resources, the rapidly developing machine vision and artificial intelligence are introduced into the field of defect detection. The AI ​​(artificial intelligence) defect detection algorithm is to continuously train and feedback-adjust the built intelligent neural network by inputting a large number of images, so as to obtain a relatively stable network. The traditional input image for defect detection is that the image scanning device scans the same position of multiple identical test pieces and stores them as images. Using this image as the input image to train the neural network also brings many challenges: when the image scanning device scans the same position of multiple identical test pieces, the scanned image will be translated and rotated due to the rotation and translation of the device. Passing such an image into the neural network for training and learning will interfere with the neural network and reduce the accuracy of the neural network's defect detection. Summary of the invention

[0003] In view of this, the present disclosure proposes a defect detection model training method, comprising:

[0004] Acquire a first reference image, a second reference image, and a test image;

[0005] Calibrate the first reference image, the second reference image and the test image to obtain a calibration image;

[0006] Synthesizing the calibration image and the test image to obtain an input image;

[0007] The input image is passed into a neural network for defect detection model training.

[0008] In a possible implementation manner, the first reference image, the second reference image, and the test image are the same position of different images.

[0009] In a possible implementation manner, the method further includes a step of preprocessing the first reference image, the second reference image, and the test image;

[0010] Preprocessing the first reference image, the second reference image, and the test image includes:

[0011] A filtering operation is performed on the first reference image, the second reference image, and the test image.

[0012] In a possible implementation manner, calibrating the first reference image, the second reference image, and the test image to obtain a calibration image includes:

[0013] Calibrate the first reference image and the test image to obtain a first calibration image;

[0014] The second reference image is calibrated with the test image to obtain a second calibration image.

[0015] In a possible implementation manner, synthesizing the calibration image and the test image to obtain an input image includes:

[0016] synthesizing the first calibration image, the second calibration image, and the test image into an RGB image;

[0017] Wherein, each channel in the RGB image corresponds one-to-one to the first calibration image, the second calibration image and the test image respectively.

[0018] In a possible implementation, the G channel of the RGB image stores the grayscale value of the first calibration image;

[0019] The B channel of the RGB image stores the grayscale value of the second calibration image;

[0020] The R channel of the RGB image stores the grayscale value of the test image.

[0021] In a possible implementation, the neural network is a convolutional neural network.

[0022] According to another aspect of the present disclosure, there is provided a defect detection method, comprising:

[0023] Acquire a first image, a second image, and a first test image;

[0024] Calibrate the first image, the second image and the first test image to obtain a calibration reference image;

[0025] Synthesizing the calibration reference image and the first test image to obtain an image to be detected;

[0026] The image to be detected is input into a defect detection model obtained by any of the methods described above to perform defect detection.

[0027] According to another aspect of the present disclosure, there is provided a defect detection model training device, characterized in that it includes a data acquisition module, an image calibration module, an image synthesis module and a neural network module;

[0028] The data acquisition module is configured to acquire a first reference image, a second reference image and a test image;

[0029] The image calibration module is configured to calibrate the first reference image, the second reference image and the test image to obtain a calibration image;

[0030] The image synthesis module is configured to synthesize the calibration image and the test image to obtain an input image;

[0031] The neural network module is configured to pass the input image into the neural network for defect detection model training.

[0032] According to another aspect of the present disclosure, a defect detection model training device is provided, characterized in that it includes:

[0033] processor;

[0034] a memory for storing processor-executable instructions;

[0035] Wherein, the processor is configured to implement any of the above-mentioned methods when executing the executable instructions.

[0036] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, characterized in that when the computer program instructions are executed by a processor, any of the above-mentioned methods is implemented.

[0037] By acquiring a first reference image, a second reference image and a test image, calibrating the first reference image, the second reference image and the test image to obtain a calibration image, synthesizing the calibration image and the test image to obtain an input image, and passing the input image into the neural network for defect detection model training. If there is a defect at a certain position of one of the first reference image, the second reference image and the test image, it can be clearly highlighted, so that the neural network can train a more robust model, thereby improving the accuracy of defect detection.

[0038] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0040] Figure 1 A flow chart showing a defect detection model training method according to an embodiment of the present disclosure;

[0041] Figure 2 A schematic diagram showing a defect detection model training method according to an embodiment of the present disclosure;

[0042] Figure 3 A block diagram showing a defect detection model training device according to an embodiment of the present disclosure;

[0043] Figure 4 A block diagram of a defect detection model training device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0044] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0045] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0046] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0047] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0048] Figure 1 A flow chart of a defect detection model training method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the defect detection model training method includes:

[0049] In step S100, a first reference image, a second reference image and a test image are obtained. In step S200, a calibration is performed based on the first reference image, the second reference image and the test image to obtain a calibration image. In step S300, the calibration image and the test image are synthesized to obtain an input image. In step S400, the input image is passed into a neural network for defect detection model training.

[0050] By acquiring a first reference image, a second reference image and a test image, calibrating the first reference image, the second reference image and the test image to obtain a calibration image, synthesizing the calibration image and the test image to obtain an input image, and passing the input image into the neural network for defect detection model training. If there is a defect at a certain position of one of the first reference image, the second reference image and the test image, it can be clearly highlighted, so that the neural network can train a more robust model, thereby improving the accuracy of defect detection.

[0051] For details, see Figure 1 , execute step S100 to obtain a first reference image, a second reference image and a test image.

[0052] In a possible implementation, the first reference image, the second reference image, and the test image are all the first reference image, the second reference image, and the test image, and the first reference image, the second reference image, and the test image are the same positions of different images. For example, an image scanning device scans the same specific position of three test pieces to generate three grayscale images, namely, the first reference image, the second reference image, and the test image, and the three grayscale images are obtained.

[0053] Furthermore, in a possible implementation, after acquiring the first reference image, the second reference image and the test image, the step of preprocessing the first reference image, the second reference image and the test image is also included, and the preprocessing of the first reference image, the second reference image and the test image includes: filtering the first reference image, the second reference image and the test image. For example, a filter is used to filter the first reference image, the second reference image and the test image to remove noise. Among them, the filter includes any one of a Gaussian filter, a mean filter and a median filter. Exemplarily, the image scanning device scans at the same specific position of three specimens to generate three grayscale images, namely the first reference image, the second reference image and the test image, and the first reference image, the second reference image and the test image are filtered by Gaussian filters respectively, that is, denoising. Removing noise from an image is a conventional technical means in the art and will not be described in detail here.

[0054] For further information, see Figure 1, execute step S200, calibrate according to the first reference image, the second reference image and the test image to obtain a calibration image.

[0055] In a possible implementation, calibrating the first reference image, the second reference image, and the test image to obtain the calibration image includes: calibrating the first reference image with the test image to obtain the first calibration image, and calibrating the second reference image with the test image to obtain the second calibration image. For example, an image scanning device is used to scan the same specific position of three test pieces to generate three grayscale images, namely, the first reference image, the second reference image, and the test image. After acquiring the first reference image, the second reference image, and the test image, the first reference image, the second reference image, and the test image are filtered by Gaussian filters, respectively, and then the first reference image is calibrated with the test image to obtain the first calibration image, and the second reference image is calibrated with the test image to obtain the second calibration image. In this way, the image calibration is completed.

[0056] For further information, see Figure 1 After the image calibration is completed, step S300 is executed to synthesize the calibration image and the test image to obtain an input image.

[0057] In a possible implementation, synthesizing the calibration image and the test image to obtain the input image includes: synthesizing the first calibration image, the second calibration image and the test image into an RGB image, wherein each channel in the RGB image corresponds to the first calibration image, the second calibration image and the test image one by one. For example, an image scanning device is used to scan the same specific position of three test pieces to generate three grayscale images, namely, the first reference image, the second reference image and the test image. After the first reference image, the second reference image and the test image are obtained, the first reference image, the second reference image and the test image are filtered by a Gaussian filter respectively, and then the first reference image is calibrated with the test image to obtain the first calibration image, and the second reference image is calibrated with the test image to obtain the second calibration image, and then the first calibration image, the second calibration image and the test image are synthesized into an RGB image. The specific synthesis method is to put the first calibration image, the second calibration image and the test image into a channel of the RGB image respectively. For example, the G channel of the RGB image stores the grayscale value of the first calibration image, the B channel of the RGB image stores the grayscale value of the second calibration image, and the R channel of the RGB image stores the grayscale value of the test image. In this way, the image synthesis is completed.

[0058] For further information, see Figure 1 After obtaining the synthesized image, step S400 can be executed to pass the input image into the neural network for defect detection model training.

[0059] In a possible implementation, the neural network is a convolutional neural network. For example, after obtaining a first reference image, a second reference image, and a test image, the first reference image, the second reference image, and the test image are filtered by Gaussian filters respectively, and then the first reference image and the test image are calibrated to obtain a first calibration image, and the second reference image and the test image are calibrated to obtain a second calibration image, and then the first calibration image, the second calibration image, and the test image are synthesized into an RGB image, the G channel of the RGB image stores the grayscale value of the first calibration image, the B channel of the RGB image stores the grayscale value of the second calibration image, and the R channel of the RGB image stores the grayscale value of the test image. See Figure 2 , input this RGB image into the convolutional neural network for training, the convolutional neural network includes an input layer, a first convolutional layer, a first sampling layer, a second convolutional layer, a second sampling layer, a fully connected layer and an output layer, wherein the fully connected layer can be one or more. Exemplarily, the LeNet-5 model can be used. After training, the trained model is used for defect detection. Since three calibrated grayscale images (first reference image, second reference image and test image) are synthesized into an RGB image as the input image of the neural network, the RGB image has rich image information. If there is a defect in a certain position of one of the images, it can be clearly highlighted in the RGB image, so that the neural network can train a more robust model, thereby improving the accuracy of defect detection.

[0060] It should be noted that although the above embodiments are used as examples to introduce the defect detection model training method of the present disclosure, those skilled in the art will understand that the present disclosure should not be limited thereto. In fact, the user can flexibly set the defect detection model training method according to personal preferences and / or actual application scenarios, as long as the required functions are achieved.

[0061] In this way, by acquiring the first reference image, the second reference image and the test image, calibrating according to the first reference image, the second reference image and the test image to obtain a calibration image, synthesizing the calibration image and the test image to obtain an input image, and passing the input image into the neural network for defect detection model training. If there is a defect at a certain position of one of the first reference image, the second reference image and the test image, it can be clearly highlighted, so that the neural network can train a more robust model, thereby improving the accuracy of defect detection.

[0062] According to another aspect of the present disclosure, a defect detection method is provided, comprising: acquiring a first image, a second image, and a first test image, calibrating the first image, the second image, and the first test image to obtain a calibration reference image, synthesizing the calibration reference image and the first test image to obtain an image to be detected, and passing the image to be detected into a defect detection model obtained by any of the above methods to perform defect detection. Thus, the accuracy of defect detection is improved.

[0063] In one possible implementation, an image scanning device is used to scan the same specific position of three test pieces to generate three grayscale images, namely a first image, a second image and a first test image. After obtaining the first image, the second image and the first test image, the first image, the second image and the first test image are filtered by Gaussian filters respectively, and then the first image is calibrated with the first test image to obtain a first calibration reference image, and the second image is calibrated with the first test image to obtain a second calibration reference image. Then, the first calibration reference image, the second calibration reference image and the first test image are synthesized into an RGB image, which is an image to be detected, wherein the G channel of the RGB image stores the grayscale value of the first calibration reference image, the B channel of the RGB image stores the grayscale value of the second calibration reference image, and the R channel of the RGB image stores the grayscale value of the first test image. This image to be detected is input into a defect detection model obtained by the defect detection model training method to perform defect detection.

[0064] Further, according to another aspect of the present disclosure, a defect detection model training device 100 is also provided. Since the working principle of the defect detection model training device 100 of the present disclosure embodiment is the same as or similar to the principle of the defect detection model training method of the present disclosure embodiment, the repeated parts will not be repeated. Figure 3 , the defect detection model training device 100 of the embodiment of the present disclosure includes a data acquisition module 110, an image calibration module 120, an image synthesis module 130 and a neural network module 140;

[0065] The data acquisition module 110 is configured to acquire a first reference image, a second reference image and a test image;

[0066] An image calibration module 120 is configured to calibrate the first reference image, the second reference image and the test image to obtain a calibration image;

[0067] An image synthesis module 130 is configured to synthesize the calibration image and the test image to obtain an input image;

[0068] The neural network module 140 is configured to pass the input image into the neural network for defect detection model training.

[0069] Furthermore, according to another aspect of the present disclosure, a defect detection model training device 200 is also provided. Figure 4 The defect detection model training device 200 of the embodiment of the present disclosure includes a processor 210 and a memory 220 for storing executable instructions of the processor 210. The processor 210 is configured to implement any of the above-mentioned defect detection model training methods when executing the executable instructions.

[0070] Here, it should be noted that the number of processors 210 can be one or more. At the same time, in the defect detection model training device 200 of the embodiment of the present disclosure, an input device 230 and an output device 240 may also be included. Among them, the processor 210, the memory 220, the input device 230 and the output device 240 may be connected through a bus or in other ways, which are not specifically limited here.

[0071] The memory 220 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and various modules, such as the program or module corresponding to the defect detection model training method of the embodiment of the present disclosure. The processor 210 executes various functional applications and data processing of the defect detection model training device 200 by running the software program or module stored in the memory 220.

[0072] The input device 230 may be used to receive input numbers or signals. The signals may be key signals related to user settings and function control of the device / terminal / server. The output device 240 may include display devices such as display screens.

[0073] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by the processor 210, any of the defect detection model training methods described above is implemented.

[0074] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A defect detection model training method, characterized in that: include: Acquire a first reference image, a second reference image, and a test image; The first reference image, the second reference image and the test image are images of the same position of different test pieces respectively; Calibrate the first reference image, the second reference image and the test image to obtain a calibration image, including: calibrating the first reference image with the test image to obtain a first calibration image, and calibrating the second reference image with the test image to obtain a second calibration image; Synthesizing the calibration image and the test image to obtain an input image; The input image is passed into a neural network for defect detection model training.

2. The method according to claim 1, characterized in that The method further comprises the steps of preprocessing the first reference image, the second reference image and the test image; Preprocessing the first reference image, the second reference image, and the test image includes: A filtering operation is performed on the first reference image, the second reference image, and the test image.

3. The method according to claim 1, characterized in that The step of synthesizing the calibration image and the test image to obtain an input image comprises: synthesizing the first calibration image, the second calibration image, and the test image into an RGB image; Wherein, each channel in the RGB image corresponds one-to-one to the first calibration image, the second calibration image and the test image respectively.

4. The method according to claim 3, characterized in that The G channel of the RGB image stores the grayscale value of the first calibration image; The B channel of the RGB image stores the grayscale value of the second calibration image; The R channel of the RGB image stores the grayscale value of the test image.

5. The method according to claim 1, characterized in that The neural network is a convolutional neural network.

6. A defect detection method, characterized in that: include: Acquire a first image, a second image, and a first test image; The first image, the second image and the first test image are images of the same position of different test pieces respectively; Calibrate the first image, the second image and the first test image to obtain a calibration reference image, including: calibrating the first image with the first test image to obtain a first calibration reference image, and calibrating the second image with the first test image to obtain a second calibration reference image; Synthesizing the calibration reference image and the first test image to obtain an image to be detected; The image to be detected is input into the defect detection model obtained by the method described in any one of claims 1 to 5 for defect detection.

7. A defect detection model training device, characterized in that: It includes a data acquisition module, an image calibration module, an image synthesis module and a neural network module; The data acquisition module is configured to acquire a first reference image, a second reference image, and a test image; the first reference image, the second reference image, and the test image are images of the same position of different test pieces respectively; The image calibration module is configured to calibrate the first reference image, the second reference image and the test image to obtain a calibration image, including: calibrating the first reference image with the test image to obtain a first calibration image, and calibrating the second reference image with the test image to obtain a second calibration image; The image synthesis module is configured to synthesize the calibration image and the test image to obtain an input image; The neural network module is configured to pass the input image into the neural network for defect detection model training.

8. A defect detection model training device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 5 when executing the executable instructions.

9. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images