A method and device for online detection of socket panel defects based on generative adversarial model

Abstract

A method and device for online detection of socket panel defects based on a generative adversarial model, comprising the following steps: obtaining a first panel image and / or a second panel image; the first panel is a flat surface of the socket panel, and the second panel is a curved surface of the socket panel; repairing the first panel image and / or the second panel image by using a pre-trained generative adversarial model to generate a first repaired image and / or a second repaired image; comparing the first repaired image with the first panel image to obtain a first fusion result; and / or comparing the second repaired image with the second repaired image to obtain a second fusion result; obtaining a final defect detection result according to the first fusion result and / or the second fusion result; the present invention can repair the panel image to be detected by using a generative adversarial model to repair it into a defect-free standard panel, and then compare it with the original image to evaluate the defects in the original image, thereby realizing defect detection.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and more specifically to an online detection method and device for socket panel defects based on a generative adversarial model. Background Art

[0002] The transformation and upgrading of the traditional manufacturing industry is an inevitable trend for the development of new quality productivity. Socket panels are essential accessories for power products and have a large market demand. However, the current socket panel manufacturing field has long been affected by reliance on labor-intensive production, low automation, low production efficiency, and difficulty in detecting defective products, resulting in a large bottleneck in production.

[0003] During the production process of socket panels, there are usually a certain proportion of panel surface defects due to factors such as spraying process, improper human operation, and external force damage. These surface defects may affect the appearance of the panel surface at the least, or the safety of downstream electrical equipment at the most. Therefore, in the panel production process, timely detection of panel defects is a very important and critical production line quality inspection process.

[0004] At present, there are mainly the following difficulties in the automated and intelligent detection of socket panel defects:

[0005] First of all, the light source has a great influence on image-based appearance detection. Because the surface of the panel is smooth and has a large mirror reflection, the accuracy is seriously affected by the traditional image detection method.

[0006] Secondly, although some deep learning methods have higher accuracy than traditional detection methods, it is difficult to directly use deep learning-based detection methods when the initial defect samples are relatively small.

[0007] Therefore, how to design an effective intelligent detection method for the socket panel is an urgent problem that technicians in this field need to solve. Summary of the invention

[0008] In view of this, the present invention provides a method and device for online detection of socket panel defects based on a generative adversarial model, which can achieve more intelligent and effective detection of socket panel defects.

[0009] In order to achieve the above object, the present invention adopts the following technical solution:

[0010] A socket panel defect online detection method based on a generative adversarial model comprises the following steps:

[0011] Acquire a first panel image and / or a second panel image; the first panel is a flat surface of the socket panel, and the second panel is a curved surface of the socket panel.

[0012] The first panel image and / or the second panel image are repaired by using a pre-trained generative adversarial model, including: repairing the first panel image to generate a first repaired image; and / or repairing the second panel image to generate a second repaired image.

[0013] Detecting defects according to the restoration result of the generative adversarial model includes: comparing the first restoration image with the first panel image to obtain a first fusion result; and / or comparing the second restoration image with the second restoration image to obtain a second fusion result. A final defect detection result is obtained according to the first fusion result and / or the second fusion result.

[0014] Preferably, comparing the first repaired image with the first panel image specifically includes: generating a first pixel difference image and a first edge feature difference image according to the first repaired image and the first panel image; and fusing the first pixel difference image with the first edge feature difference image to obtain a first fusion result.

[0015] Preferably, the second repaired image and the second repaired image are compared to obtain a second fusion result, which specifically includes: generating a second pixel difference image and a second edge feature difference image according to the second repaired image and the second panel image; and fusing the second pixel difference image and the second edge feature difference image to obtain a second fusion result.

[0016] Preferably, the defect detection result is obtained by combining the first fusion result and the second fusion result, specifically including:

[0017] A pixel threshold is set to obtain pixel positions where the first fusion result and the second fusion result exceed the pixel threshold, that is, defect positions.

[0018] Preferably, the training step of the generative adversarial network model includes: constructing a training data set to pair defect samples with standard samples; inputting the defect samples into the generation module of the generative adversarial network model to generate an intermediate repaired image; inputting the repaired image and the corresponding standard sample into the discrimination module of the generative adversarial network model to calculate the image loss, and optimizing the generation module according to the image loss.

[0019] Preferably, the training step further comprises: expanding and constructing the defect samples, specifically: using the intermediate repaired images generated during the training process as defect samples.

[0020] Preferably, the generative adversarial model is composed of a plurality of cyclic adversarial generative models, each of which is obtained through independent training and is used to detect different socket panels.

[0021] A socket panel defect online detection device based on a generative adversarial model, comprising:

[0022] The image acquisition module is used to obtain a first panel image and / or a second panel image of the socket panel; the first panel image is the flat surface of the socket panel, and the second panel image represents the curved surface of the socket panel.

[0023] The image processing module is used to perform restoration processing on the first panel image and / or the second panel image by using a pre-trained generative adversarial model to obtain a first restored image and / or a second restored image.

[0024] A detection output module is used to detect defects according to the repair result of the generative adversarial model, compare the first repaired image with the first panel image to obtain a first fusion result; and / or compare the second repaired image with the second repaired image to obtain a second fusion result; and to obtain a final defect detection result according to the first fusion result and / or the second fusion result.

[0025] An image acquisition device suitable for socket panel defect detection, comprising:

[0026] A shell, a light source, a first camera arranged on the top of the shell, a plurality of second cameras arranged around the first camera and a dimming device; the light source is arranged on the top of the shell; the lens of the first camera is vertically downward to obtain the front image of the socket panel; the plurality of second cameras are symmetrically distributed, and the lenses are inclined at a certain angle to obtain the side and transition surface images of the socket panel; the dimming device is arranged on the side wall of the shell to adjust the light inside the shell so that the target panel surface is evenly illuminated.

[0027] Preferably, the dimming device includes a first dimming component and a second dimming component; the first dimming component is composed of a plurality of first dimming bands arranged on the upper side of the shell, each of the first dimming bands is symmetrically distributed, and the light output surface is inclined at a certain angle, so as to fill in the light of the transition surface of the socket panel; the second dimming component is composed of a plurality of second dimming bands arranged on the upper side of the shell, each of the second dimming bands is symmetrically distributed, so as to fill in the light of the side of the socket panel.

[0028] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method and device for online detection of socket panel defects based on a generative adversarial model. First, the method can repair the panel image to be detected by generating an adversarial model to make it a defect-free standard panel, and then compare it with the original image to evaluate the defects in the original image, thereby realizing defect detection. By generating a repair image using a generative adversarial model, defect samples can be generated autonomously during the training stage, so the number of existing defect samples required is lower; secondly, the present invention trains a detection model for each panel surface of the socket panel separately, and can specifically detect the flat surface and the curved surface with a transition section separately, thereby improving the detection accuracy; in addition, the present invention also provides an image acquisition device based on the characteristics of the existing socket panel, which is combined with a dimming device to realize three-dimensional lighting. Even for a smooth surface, it will not be affected by mirror reflection, thereby acquiring a higher quality panel image, thereby fundamentally improving the accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0030] Figure 1 A schematic diagram of an online detection method for socket panel defects based on a generative adversarial model provided by the present invention.

[0031] Figure 2 Schematic diagram of the online training process in an embodiment of the present invention.

[0032] Figure 3 Schematic diagram of a method for defect detection and positioning in an embodiment of the present invention.

[0033] Figure 4 A schematic diagram of the structure of an image acquisition device suitable for socket panel defect detection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example 1

[0036] like Figure 1 The embodiment of the present invention discloses a method for online detection of socket panel defects based on a generative adversarial model, comprising the following steps:

[0037] S1: Acquire an image of the board to be tested.

[0038] Specifically, a first panel image and / or a second panel image is obtained, wherein the first panel image is a flat surface of the socket panel, and the second panel image is a curved surface of the socket panel.

[0039] S2: performing restoration processing on the first panel image and / or the second panel image by using a pre-trained generative adversarial model, including:

[0040] S21: Repair the first panel image to generate a first repaired image.

[0041] S22: and / or repairing the second panel image to generate a second repaired image.

[0042] S3: Detect defects based on the repair results of the generative adversarial model, including:

[0043] S31: Compare the first repaired image and the first panel image to obtain a first fusion result.

[0044] S32: and / or compare the first restored image with the second restored image to obtain a second fusion result.

[0045] S33: Obtain a final defect detection result according to the first fusion result and / or the second fusion result.

[0046] In this embodiment, a cyclic generative adversarial model CycleGAN can be selected as a generative adversarial network. The present invention repairs the panel image to be detected by generating an adversarial model to restore it to a flawless standard panel, and then compares it with the original image to evaluate the defects in the original image, thereby realizing defect detection.

[0047] like Figure 2 In order to further implement the above technical solution, S31 specifically includes: generating a first pixel difference image and a first edge feature difference image according to the first repaired image and the first panel image; fusing the first pixel difference image and the first edge feature difference image to obtain a first fusion result.

[0048] S32 specifically includes: generating a second pixel difference image and a second edge feature difference image according to the second repaired image and the second panel image; and fusing the second pixel difference image and the second edge feature difference image to obtain a second fusion result.

[0049] In order to further implement the above technical solution, S33 specifically includes setting a pixel threshold to obtain the pixel position where the first fusion result and the second fusion result exceed the pixel threshold, that is, the defect position.

[0050] To further implement the above technical solution, the training steps of the generative adversarial network model include:

[0051] Construct a training data set and pair defective samples with standard samples; specifically, construct a paired data set of defective and intact panels, that is, a defective panel image is paired with a corresponding intact socket panel image of the same model. When constructing the data set, due to the small number of defective samples, the data set is constructed by two methods: data enhancement and artificial defect creation.

[0052] The defective sample is input into the generation module of the generative adversarial network model to generate an intermediate repaired image; the repaired image and the corresponding standard sample are input into the discriminant module of the generative adversarial network model to calculate the image loss, and the generation module is optimized according to the image loss. In the iterative optimization process, the repair image generated by the generator gradually approaches the standard until the preset convergence condition is met.

[0053] Furthermore, during training, the intermediate repair images generated by the generation module are used as defect samples to expand the sample library. For the detection method based on deep learning, a large number of defect samples need to be labeled to obtain a relatively high detection rate. However, it is difficult to find samples in actual industrial products. The present invention provides an online detection and positioning technology for socket panel defects based on a generative adversarial model. First, online training and learning are performed through artificial synthesis and simulation samples. By continuously accumulating learning samples, the detection capability of the model is continuously improved.

[0054] Specific as Figure 2 As shown in the figure, each surface of the socket panel is defined, the A-level surface refers to the front of the socket, and the B-level surfaces 1-4 refer to the four sides of the socket panel with transition parts; each surface corresponds to a cyclic generative adversarial model for online training.

[0055] Defect detection and location Figure 3 As shown in the figure, after the repair image is generated by the generative adversarial model, the defective panel image and the repair image are subtracted to obtain the difference image. Then, Sobel edge features are extracted from the input image and the repaired image respectively, and the edge features on the two images are subtracted. Then, the edge feature difference and the difference feature on the difference image are weighted fused, and finally, panel defect detection and positioning are achieved.

[0056] Furthermore, when it is necessary to detect both the flat surface and the curved surface of the socket panel, the generative adversarial model is composed of multiple cyclic adversarial generative models to detect different socket panel surfaces. At the same time, in the training stage of the model, each cyclic adversarial generative model is obtained through independent training.

[0057] Example 2

[0058] Based on the same inventive concept, an embodiment of the present invention discloses an online detection device for socket panel defects based on a generative adversarial model, comprising an image acquisition module, an image processing module and a detection output module; the image acquisition module is respectively connected to the image processing module and the detection output module in communication; the image processing module is connected to the detection output module in communication;

[0059] The image acquisition module is used to obtain the image of the surface to be inspected of the socket panel and send the acquired image to the image processing module and the detection output module; wherein the image acquisition module can be a camera or a smart phone, and a communication channel is established between the image acquisition module and the image processing module and the detection data module through the Internet.

[0060] The image processing module is used to perform restoration processing on the first panel image and / or the second panel image by using a pre-trained generative adversarial model to obtain a first restored image and / or a second restored image.

[0061] The detection output module is used to detect defects according to the repair result of the generative adversarial model, compare the first repaired image with the first panel image to obtain a first fusion result; and / or compare the second repaired image with the second repaired image to obtain a second fusion result; and to obtain a final defect detection result according to the first fusion result and / or the second fusion result.

[0062] Example 3

[0063] like Figure 4 The light source has a huge impact on image acquisition. When capturing images of the curved surface of the socket panel, it is often affected by the light source, resulting in large mirror reflections. The quality of image acquisition is low, which seriously affects the accuracy of defect detection at the root.

[0064] In this regard, based on the inventive concepts of Example 1 and Example 2, an embodiment of the present invention discloses an image acquisition device suitable for socket panel defect detection, including a shell, a light source 1, a first camera 2 arranged on the top of the shell, a plurality of second cameras 3 arranged around the first camera 2, and a dimming device; the light source 1 is arranged on the top of the shell; the lens of the first camera 2 is vertically downward, for obtaining a front image of the socket panel; the plurality of second cameras 3 are symmetrically distributed, and the lenses are inclined at a certain angle, for obtaining side and transition surface images of the socket panel; the dimming device is arranged on the side wall of the shell to adjust the light inside the shell so that the target panel surface is evenly illuminated.

[0065] In this embodiment, the role of the housing is to provide connection support for each component so that each component can maintain a relative position relationship. The housing can also be replaced by a bracket, or the components can be fixed by hoisting.

[0066] In order to further implement the above technical scheme, the dimming device includes a first dimming component and a second dimming component; the first dimming component is composed of a plurality of first dimming bands 4 arranged on the upper part of the side surface of the shell, each first dimming band 4 is symmetrically distributed, and the light output surface is inclined at a certain angle, which is used to fill the transition surface of the socket panel; the second dimming component is composed of a plurality of second dimming bands 5 arranged on the upper part of the side surface of the shell, each second dimming band 5 is symmetrically distributed, and is used to fill the side surface of the socket panel.

[0067] In this embodiment, the top light source 5, the first dimming component diagonally above and the second dimming component on the side can constitute a three-layer light source system, thereby achieving uniform lighting of each panel surface of the socket and improving the quality of image acquisition.

[0068] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0069] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A socket panel defect online detection method based on a generative adversarial model, characterized in that: The following steps are involved: Acquire a first panel image and a second panel image; the first panel is a flat surface of the socket panel, and the second panel is a curved surface of the socket panel; The first panel image and the second panel image are repaired by using a pre-trained generative adversarial model, including: Repairing the first panel image to generate a first repaired image; Repairing the second panel image to generate a second repaired image; Detect defects based on the repair results of the generative adversarial model, including: generating a first pixel difference image and a first edge feature difference image according to the first repaired image and the first panel image; fusing the first pixel difference image and the first edge feature difference image to obtain a first fusion result; generating a second pixel difference image and a second edge feature difference image according to the second repaired image and the second panel image; fusing the second pixel difference image and the second edge feature difference image to obtain a second fusion result; Obtaining a final defect detection result according to the first fusion result and / or the second fusion result; The generative adversarial model is composed of a plurality of cyclic adversarial generative models, each of which is obtained through independent training and is used to detect different socket panels; The training steps of the generative adversarial network model include: Construct a training data set to pair defective samples with standard samples; Inputting the defect sample into the generation module of the generative adversarial network model to generate an intermediate repaired image; Inputting the repaired image and the corresponding standard sample into the discriminant module of the generative adversarial network model, calculating the image loss, and optimizing the generation module according to the image loss; The defect samples are expanded and constructed, specifically: the intermediate repaired images generated during the training process are used as defect samples.

2. According to claim 1, a method for online detection of socket panel defects based on a generative adversarial model is characterized in that: The defect detection result is obtained by combining the first fusion result and the second fusion result, which specifically includes: A pixel threshold is set to obtain pixel positions where the first fusion result and the second fusion result exceed the pixel threshold, that is, defect positions.

3. A socket panel defect online detection device based on a generative adversarial model, characterized in that: It adopts the online detection method described in claim 1 or 2, comprising: An image acquisition module, used to acquire a first panel image and / or a second panel image of the socket panel; the first panel image is a flat surface of the socket panel, and the second panel image is a curved surface of the socket panel; An image processing module, used for performing restoration processing on the first panel image and / or the second panel image by using a pre-trained generative adversarial model to obtain a first restored image and / or a second restored image; A detection output module is used to detect defects according to the repair result of the generative adversarial model, compare the first repaired image with the first panel image to obtain a first fusion result; and / or compare the second repaired image with the second repaired image to obtain a second fusion result; and to obtain a final defect detection result according to the first fusion result and / or the second fusion result.

4. The device for online detection of socket panel defects based on a generative adversarial model according to claim 3, characterized in that: The image acquisition module comprises: A housing, a light source, a first camera arranged on the top of the housing, a plurality of second cameras arranged around the first camera, and a dimming device; The light source is arranged on the top of the housing; The lens of the first camera is vertically downward, and is used to obtain a front image of the socket panel; The plurality of the second cameras are symmetrically distributed, and the lenses are tilted at a certain angle, so as to obtain images of the side and transition surfaces of the socket panel; The dimming device is arranged on the side wall of the housing to adjust the light inside the housing so that the target panel is evenly illuminated; The dimming device comprises a first dimming component and a second dimming component; The first dimming component is composed of a plurality of first dimming strips arranged on the upper side of the housing, each of the first dimming strips is symmetrically distributed, and the light output surface is inclined at a certain angle, so as to provide supplementary light to the transition surface of the socket panel; The second dimming component is composed of a plurality of second dimming strips arranged on the upper side of the shell, and each of the second dimming strips is symmetrically distributed and is used to fill in the light on the side of the socket panel.

Images