Defect detection method, apparatus and electronic device

Abstract

Embodiments of the present application provide a defect detection method and device and electronic equipment. In the embodiments, a reference mask image generated based on a defect image corresponding to a specified defect object and a first specified standard image are input into a trained target generator network, so that the target generator network fuses the reference mask image and the first specified standard image into a defect sample image, so as to train a network model for defect detection based on the generated defect sample image, to automatically detect defects. Furthermore, the network model for defect detection trained based on the generated defect sample image detects defects, which saves labor costs and improves detection efficiency compared with common manual defect detection.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a defect detection method, apparatus and electronic device. Background Technology

[0002] In current industrial production, in order to ensure product quality, it is often necessary to perform defect detection on designated objects in production. For example, in vehicle production, it is often necessary to detect defects in designated objects (vehicles). Here, vehicle defects may include, for example, scratches or dents on the vehicle's surface.

[0003] Currently, the most common defect detection method is manual inspection. While manual inspection can detect defects in a specified object, it requires high manpower and has low efficiency. Summary of the Invention

[0004] In view of this, embodiments of this application provide a defect detection method, apparatus, and electronic device to achieve automatic detection of defects in a specified object.

[0005] According to a first aspect of the embodiments of this application, a defect detection method is provided, the method being applied to an electronic device, the method comprising:

[0006] Obtain a reference mask image; the reference mask image is generated based on the defect image corresponding to the specified defect object;

[0007] The reference mask image and the first specified standard image corresponding to the obtained specified object are input into the pre-trained target generator network, so that the target generator network fuses the reference mask image and the first specified standard image into a defect sample image; wherein, the defect sample image is used to train at least a network model for defect detection.

[0008] According to a second aspect of the embodiments of this application, a mask image apparatus is provided, the apparatus being applied to an electronic device, the apparatus comprising:

[0009] A reference mask image acquisition module is used to acquire a reference mask image; the reference mask image is generated based on the defect image corresponding to a specified defect object;

[0010] The defect sample image generation module is used to input the reference mask image and the first specified standard image corresponding to the obtained specified object into the trained target generator network, so that the target generator network fuses the reference mask image and the first specified standard image into a defect sample image; wherein, the defect sample image is used to train at least a network model for defect detection.

[0011] According to a third aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor and a memory;

[0012] The memory is used to store machine-executable instructions;

[0013] The processor is configured to read and execute machine-executable instructions stored in the memory to implement the method as described in the first aspect.

[0014] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0015] In this embodiment of the application, a reference mask image and a first specified standard image generated based on the defect image corresponding to the specified defect object are input into a pre-trained target generator network. The target generator network then fuses the reference mask image and the first specified standard image into a defect sample image, so as to train a network model for defect detection based on the generated defect sample image and realize automatic defect detection.

[0016] Furthermore, a network model for defect detection is trained based on the generated defect sample images. Compared with the commonly used manual defect detection, this saves labor costs and improves detection efficiency. Attached Figure Description

[0017] Figure 1 This is a flowchart of a defect detection method provided in an embodiment of this application.

[0018] Figure 2 This is a defect mask image provided in an embodiment of this application.

[0019] Figure 3 This is a flowchart of a target generator network training method provided in an embodiment of this application.

[0020] Figure 4 This is a block diagram of the defect detection device provided in the embodiments of this application.

[0021] Figure 5 This is a hardware structure diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0025] The embodiments of this application will now be described in detail.

[0026] See Figure 1 , Figure 1 This is a flowchart illustrating a method provided in an embodiment of this application. The method is applied to an electronic device; as an example, the electronic device may be a terminal, a server, etc., but this embodiment is not specifically limited to this.

[0027] like Figure 1 As shown, the process may include the following steps:

[0028] S110: Obtain the reference mask image.

[0029] For example, in this embodiment, the reference mask image refers to an RGB image, which is generated based on the defect image corresponding to the specified defect object. Here, the specified defect object is a possible defect of an object in an application scenario. For example, when the application scenario is vehicle defect detection, the specified defect object can be scratches on the vehicle, stickers on the vehicle, etc.

[0030] Figure 2 For a specific example of a reference mask image, in Figure 2 In the image, both curves and surfaces are reference mask images.

[0031] In this embodiment, there are many ways to obtain the defect image corresponding to the specified defect object. For example, it can be extracted from real images taken in a specified application scenario. Here, the specified application scenario is the application scenario in which the specified object is located, and the specified object is the object containing the specified defect object. Taking vehicle defect detection as an example, the specified object is a vehicle.

[0032] For example, the defect image corresponding to the specified defect object can also be extracted from real images taken in non-specified application scenarios. Here, non-specified application scenarios are scenarios different from the specified application scenarios mentioned above. For example, when the specified application scenario is vehicle defect detection, the non-specified application scenario can be home appliance defect detection, robot defect detection, etc.

[0033] For example, the defect image corresponding to the specified defect object can also be a drawn simulation image. Here, the drawn simulation image can be drawn by a technician on the operating interface of an electronic device based on the defects that may occur in the specified application scenario where the specified defect object is located.

[0034] For details on how to obtain the reference mask image, please refer to the examples described in the following embodiments, which will not be repeated here.

[0035] S120: Input the reference mask image and the first specified standard image corresponding to the obtained specified object into the trained target generator network, so that the target generator network fuses the reference mask image and the first specified standard image into a defect sample image; wherein, the defect sample image is used to train at least a network model for defect detection.

[0036] For example, in this embodiment, the specified object in the first specified standard image is defect-free. Here, the first specified standard image is a real industrial image captured by an industrial camera. The industrial camera is a key component of a machine vision system, and its function is to convert light signals into ordered electrical signals. The number of the first specified standard images can be one, but in order to ensure the diversity of defect sample images, the number of the first specified standard images can also be multiple.

[0037] The training process for the target generator network is as follows: Figure 3 The illustrated embodiments are described by way of example and will not be repeated here.

[0038] In this embodiment, in step S120, the reference mask image and the first specified standard image corresponding to the obtained specified object are input into the trained target generator network so that the target generator network can perform image encoding on the reference mask image and the first specified standard image. The encoded reference mask image and the encoded first specified standard image are then fused, and the fused image is decoded to obtain the defect sample image.

[0039] In this embodiment, after obtaining the defect sample image, the defect sample image is scaled according to the size information of the reference mask image to obtain a defect sample image corresponding to the size information of the reference mask image. Once the annotation information of the defect image sample is monitored, a network model for a specified task can be trained based on the annotated defect sample image. Here, the annotation information may include the defect bounding box and the defect type. Corresponding to the annotation information, the network model for the specified task can be a network model for defect detection or a network model for defect classification; this embodiment is not specifically limited to these two types.

[0040] This concludes the process. Figure 1 The process is shown below.

[0041] pass Figure 1 As can be seen from the process shown, in this embodiment of the application, a reference mask image and a first specified standard image generated based on the defect image corresponding to the specified defect object are input into the pre-trained target generator network, so that the target generator network fuses the reference mask image and the first specified standard image into a defect sample image, so as to train a network model for defect detection based on the generated defect sample image, so as to realize automatic defect detection.

[0042] Furthermore, a network model for defect detection is trained based on the generated defect sample images. Compared with the commonly used manual defect detection, this saves labor costs and improves detection efficiency.

[0043] The process of obtaining the reference mask image is described below:

[0044] (1) When the defect image corresponding to the specified defect object is a real image taken in the specified application scenario, the process of obtaining the reference mask image is as follows:

[0045] First, generate an initial blank image;

[0046] Secondly, defect candidate regions are obtained from the real image and then fused onto the initial blank image to obtain a reference mask image.

[0047] For example, in this embodiment, the initial blank image can be generated by code or created in software. This application does not specifically limit the method for generating the initial blank image; those skilled in the art can determine the method based on the actual situation.

[0048] In this embodiment, the defect candidate region can be obtained from the real image using conventional computer vision techniques. After obtaining the defect candidate region, it is pasted onto the initial blank image to obtain a reference mask image.

[0049] As an example, after obtaining the defect candidate region, the defect candidate region is colored and filled, that is, the defect candidate region is filled with a specified color, and the colored and filled image is pasted onto the initial blank image to obtain a reference mask image.

[0050] As for how to fill with a specified color, the following examples illustrate this, and will not be repeated here.

[0051] (2) When the defect image corresponding to the specified defect object is a real image taken in a non-specified application scenario, the process of obtaining the reference mask image is as follows:

[0052] First, generate an initial blank image;

[0053] Secondly, defect mask templates from non-specified application scenarios are fused onto the initial blank image to obtain a reference mask image.

[0054] The process of generating the initial blank image described above is the same as that of generating the initial blank image in (1), and will not be repeated here.

[0055] In this embodiment, as described above, the non-specified application scenario can be a home appliance defect detection scenario or a robot defect detection scenario, etc. As an example, the number of non-specified application scenarios can be one or more, and this application embodiment is not specifically limited.

[0056] In this embodiment of the application, when a defect mask template for a non-specified application scenario is obtained, it can be scaled and rotated before being pasted onto the initial blank image to obtain a reference mask image.

[0057] This application embodiment enriches the defect types of the specified application scenario by using defect mask images from non-specified application scenarios as defect mask images for the specified application scenario, thus ensuring the diversity of the generated defect sample images.

[0058] (3) When the defect image corresponding to the specified defect object is a drawn simulation image, the process of obtaining the reference mask image is as follows:

[0059] First, generate an initial blank image.

[0060] Secondly, determine the defect parameter information and defect drawing method corresponding to the specified application scenario. The specified application scenario is the scene in which the specified object is located, and the specified defect object refers to the defect that may occur in the specified object under the specified application scenario. The defect parameter information includes at least defect location information and defect size information. Based on the defect parameter information and defect drawing method, draw a reference mask image on the initial blank image.

[0061] The process of generating the initial blank image described above is the same as that of generating the initial blank image in (1), and will not be repeated here.

[0062] For example, in this embodiment, the above drawing methods may include: line drawing, shape drawing, etc. Line drawing may include straight line drawing, curve drawing, etc. Here, curve drawing can be performed using conventional functions, user-defined functions, etc. This application embodiment does not specifically limit the drawing method, line drawing content, or curve drawing method; these can be determined according to actual circumstances.

[0063] In this embodiment, shape drawing may include drawing circles, ellipses, and polygons, etc. This application is not specifically limited to these.

[0064] For example, in this embodiment, the step of drawing a reference mask image on the initial blank image based on defect parameter information and defect drawing method specifically involves: drawing on the initial blank image according to the defect location information and defect size information using the corresponding defect drawing method, and the resulting simulated image is the reference mask image.

[0065] As an example, after drawing the simulated image, the simulated image is colored and filled, and the colored and filled image is used as the reference mask image mentioned above.

[0066] It should be noted that in this embodiment, there can be multiple reference mask images, and these multiple reference mask images can be generated based on at least one of the above-described generation methods. This embodiment of the application is not specifically limited to these methods.

[0067] See Figure 3 , Figure 3 This is a flowchart illustrating the training process of the target generator network provided in an embodiment of this application. Figure 3 As shown, the process includes the following steps:

[0068] S310: Obtain candidate mask images for training the target generator network; the candidate mask images are generated based on real defect images corresponding to the specified objects.

[0069] For example, in this embodiment, the candidate mask image is an RGB image, which is generated based on the real defect image corresponding to the specified object. Here, the real defect image is a real industrial image captured by an industrial camera.

[0070] For an example description of how to generate a candidate mask image based on the real defect image corresponding to a specified object, please refer to the following embodiment, which will not be elaborated here.

[0071] S320: Input the candidate mask image and the obtained second specified standard image corresponding to the specified object into the initial generator network, so that the initial generator network can fuse the candidate mask image and the second specified standard image into a simulated defect image.

[0072] For example, in this embodiment, the specified object in the second specified standard image is defect-free. Here, the second specified standard image is a real industrial image captured by an industrial camera. This second specified standard image may be the same as or different from the first specified standard image. This application embodiment is not specifically limited.

[0073] In this embodiment, the initial generator network can be the generator in a generative adversarial network (GAN). After obtaining multiple candidate mask images and multiple second specified standard images, each candidate mask image is matched with each second specified standard image to obtain multiple one-to-one paired candidate mask images and second specified standard images. The one-to-one paired candidate mask images and second specified standard images are input into the initial generator network, and the initial generator fuses the candidate mask images and second specified standard images into a simulated defect image.

[0074] In this embodiment, in step S320, the initial generator network fuses the candidate mask image and the second specified standard image into a simulated defect image, which can specifically be as follows:

[0075] First, the second specified standard image is encoded through the initial generator network to obtain the first feature vector, and the candidate mask image is encoded to obtain the second feature vector.

[0076] Secondly, the first feature vector and the second feature vector are fused;

[0077] Next, the fused feature vectors are decoded to obtain a simulated defect image.

[0078] For example, in this embodiment, the feature vector may include texture feature vectors, distribution feature vectors, etc., and this application embodiment is not specifically limited. That is, the first feature vector and the second feature vector mentioned above may both include texture feature vectors and distribution feature vectors.

[0079] The first feature vector is obtained by encoding the second specified standard image through the initial generator network, and the second feature vector is obtained by encoding the candidate mask image. The first feature vector and the second feature vector are fused, which can be done by adding the first feature vector and the second feature vector. The fused feature vector is then decoded to obtain the simulated defect image.

[0080] S330: Input the simulated defect image and the candidate mask image into the trained discriminator network to obtain the first true / false judgment result of the simulated defect image, and input the real defect image and the candidate mask image into the trained discriminator network to obtain the second true / false judgment result of the real defect image.

[0081] For example, in this embodiment, the trained discriminator network can be a binary classification network, whose output true / false judgment result can be true or false. This true / false judgment result can be represented by numbers, for example, 1 represents true and 0 represents false. Therefore, in this embodiment, both the first true / false judgment result and the second true / false judgment result can be 1 or 0.

[0082] The discriminator network described above can be determined through the following steps: acquiring image training samples, which include simulated defect image negative samples, real defect image positive samples, and defect mask images; and using the defect mask images to assist in judgment. Both the simulated defect image negative samples and the real defect image positive samples carry label information, for example, simulated defect image - false, real defect image - true. The image training samples are then input into the initial discriminator network for supervised training.

[0083] Specifically, simulated defect image negative samples are superimposed with defect mask images, and real defect image positive samples are superimposed with defect mask images. The initial discriminator network continuously learns the features of the superimposed negative samples and the features of the superimposed positive samples until the training count reaches the preset number, thus obtaining the discriminator network.

[0084] In this embodiment, in step S330, the simulated defect image and the candidate mask image are input into the trained discriminator network to obtain a first true / false judgment result of the simulated defect image, and the real defect image and the candidate mask image are input into the trained discriminator network to obtain a second true / false judgment result of the real defect image. Specifically, this can be:

[0085] The simulated defect image is superimposed with the candidate mask image to obtain the first superimposed image. The feature vector of the first superimposed image is extracted by the discriminator network, and the first true or false judgment result of the simulated defect image is output based on the feature vector of the first superimposed image.

[0086] The real defect image and the candidate mask image are superimposed to obtain a second superimposed image. The feature vector of the second superimposed image is extracted by a discriminator network, and the second true or false judgment result of the real defect image is output based on the feature vector of the second superimposed image.

[0087] For example, in this embodiment, the simulated defect image and the candidate mask image are the same size and are both 3-channel images. The overlay process of the simulated defect image and the candidate mask image can specifically involve superimposing the two 3-channel images to obtain a 6-channel image (the first overlay image). The discriminator network extracts the feature vector of this 6-channel image and outputs the first true / false judgment result of the simulated defect image based on the feature vector. The processing procedure for the second overlay image is the same as that for the first overlay image, and will not be repeated here.

[0088] S340: Based on the first true / false judgment result and the second true / false judgment result, the network parameters of the initial generator network are adjusted in reverse. If the adjusted generator network meets the preset conditions, the adjusted generator network is used as the target generator network. If the adjusted generator network does not meet the preset conditions, the adjusted generator network is used as the initial generator network, and the step of inputting the candidate mask image and the obtained second specified standard image corresponding to the specified object into the initial generator network is continued.

[0089] For example, in this embodiment, the network parameters can be the weights of each layer in the initial generator network. Adjusting the network parameters of the initial generator network based on the first and second true / false judgment results can specifically involve: calculating a loss value based on the first and second true / false judgment results, and then using a conventional gradient backpropagation method to adjust the network parameters of the initial generator network based on this loss value. Here, the loss value can be the cross-entropy loss value, but this embodiment is not specifically limited to this.

[0090] For example, in this embodiment, the preset condition can be that the number of iterations reaches a preset number (e.g., 500 times), or that the initial generator network converges, for example, the loss value is less than a preset loss value (e.g., 0.08). This application embodiment does not specifically limit the preset condition, and those skilled in the art can determine it according to the actual situation.

[0091] As an optional implementation of this application, the simulated defect image includes a defect area and a background area. The above-mentioned preset conditions may also include: the difference between the pixel value of each pixel in the defect area of ​​the simulated defect image and the pixel value of the corresponding pixel in the candidate defect mask image is less than a first preset threshold, and the difference between the pixel value of each pixel in the background area of ​​the simulated defect image and the pixel value of the corresponding pixel in the second specified standard image is less than a second preset threshold.

[0092] For example, the first preset threshold can be 0.1, and the second preset threshold can also be 0.1. This application embodiment does not specifically limit the first preset threshold and the second preset threshold; those skilled in the art can determine them according to actual circumstances.

[0093] This concludes the process. Figure 3 The process is shown below.

[0094] pass Figure 3 The process shown yielded the target generator network.

[0095] The following is an example of how to obtain candidate mask images:

[0096] First, for each defect type in the real defect image, the defect area corresponding to that defect type is marked in the real defect image according to the marking method corresponding to that defect type.

[0097] For example, in this embodiment, the above-mentioned defect types can be determined according to a specified application scenario. Taking vehicle inspection as an example, the defect types can include: scratches, stickers, etc.

[0098] For example, in this embodiment, different identification methods correspond to different defect types. Here, the identification method can be color identification, numerical identification, etc., and this embodiment of the application is not specifically limited to these methods.

[0099] For each real-world defect image, which contains at least one defect type, taking a specific application scenario of vehicle inspection and color-coding as an example, for each defect type, the defect area corresponding to that defect type in the real-world defect image can be marked according to the marking method for that defect type: scratches can be marked in red, and stickers can be marked in green.

[0100] Secondly, candidate mask images are determined based on real defect images that identify the defect regions corresponding to each defect type.

[0101] For example, in this embodiment, after obtaining the real defect image that identifies the defect region corresponding to each defect type, the defect region that identifies each defect type is extracted and used as the above-mentioned candidate mask image.

[0102] This application embodiment, through neural network training, can learn the function of generating defective image samples by combining a real, defect-free image with a mask image. Through neural network encoding and decoding, the features of the generated defective portion are fused with the features of the defect-free image. In the generated defective image sample, the appearance shape of the defective portion is similar to that of the real defective portion, and the appearance shape of the defect-free portion is similar to that of the real, defect-free image.

[0103] Corresponding to the embodiments of the aforementioned methods, embodiments of the present application also provide embodiments of the apparatus and the terminal to which it is applied.

[0104] like Figure 4 As shown, Figure 4This is a block diagram of a defect detection device provided in an embodiment of this application. The defect detection device includes:

[0105] The reference mask image acquisition module is used to obtain a reference mask image; the reference mask image is generated based on the defect image corresponding to the specified defect object.

[0106] The defect sample image generation module is used to input a reference mask image and a first specified standard image corresponding to a specified object into a pre-trained target generator network, so that the target generator network can fuse the reference mask image and the first specified standard image into a defect sample image; wherein, the defect sample image is used to train at least a network model for defect detection.

[0107] As an optional implementation of this application, the defect detection device further includes:

[0108] The initial blank image generation module is used to generate an initial blank image;

[0109] The drawing module is used to determine the defect parameter information and defect drawing method corresponding to a specified application scenario. The specified application scenario refers to the scene in which the specified object exists, and the specified defect object refers to the defect that may occur in the specified object under the specified application scenario. The defect parameter information includes at least defect location information and defect size information. Based on the defect parameter information and defect drawing method, a reference mask image is drawn on the initial blank image; or...

[0110] The fusion module is used to fuse defect mask templates from non-specified application scenarios onto an initial blank image to obtain a reference mask image.

[0111] As an optional implementation of this application, the defect detection device further includes:

[0112] The candidate mask image acquisition module is used to obtain candidate mask images for training the target generator network; the candidate mask images are generated based on the real defect images corresponding to the specified objects.

[0113] The simulated defect image generation module is used to input the candidate mask image and the obtained second specified standard image corresponding to the specified object into the initial generator network, so that the initial generator network can fuse the candidate mask image and the second specified standard image into a simulated defect image.

[0114] The discrimination module is used to input the simulated defect image and the candidate mask image into the trained discriminator network to obtain the first true / false judgment result of the simulated defect image, and to input the real defect image and the candidate mask image into the trained discriminator network to obtain the second true / false judgment result of the real defect image.

[0115] The target generator network determination module is used to adjust the network parameters of the initial generator network in reverse based on the first true / false judgment result and the second true / false judgment result. If the adjusted generator network meets the preset conditions, the adjusted generator network is used as the target generator network. If the adjusted generator network does not meet the preset conditions, the adjusted generator network is used as the initial generator network, and the step of inputting the candidate mask image and the obtained second specified standard image corresponding to the specified object into the initial generator network continues.

[0116] As an optional implementation of this application, the candidate mask image acquisition module is specifically used for:

[0117] For each defect type in the real defect image, the defect area corresponding to that defect type is marked in the real defect image according to the marking method corresponding to that defect type.

[0118] Candidate mask images are determined based on real defect images that identify the defect regions corresponding to each defect type.

[0119] As an optional implementation of this application, the simulated defect image generation module is specifically used for:

[0120] The first feature vector is obtained by encoding the second specified standard image through the initial generator network, and the second feature vector is obtained by encoding the candidate mask image.

[0121] The first feature vector and the second feature vector are fused;

[0122] The fused feature vectors are then image-decoded to obtain a simulated defect image.

[0123] As an optional implementation of this application, the simulated defect image includes a defect region and a background region, and the preset conditions include: the difference between the pixel value of each pixel in the defect region of the simulated defect image and the pixel value of the corresponding pixel in the candidate defect mask image is less than a first preset threshold, and the difference between the pixel value of each pixel in the background region of the simulated defect image and the pixel value of the corresponding pixel in the second specified standard image is less than a second preset threshold.

[0124] As an optional implementation of this application, the above-mentioned discrimination module is specifically used for:

[0125] The simulated defect image is superimposed with the candidate mask image to obtain the first superimposed image. The feature vector of the first superimposed image is extracted by the discriminator network, and the first true or false judgment result of the simulated defect image is output based on the feature vector of the first superimposed image.

[0126] The real defect image and the candidate mask image are superimposed to obtain a second superimposed image. The feature vector of the second superimposed image is extracted by a discriminator network, and the second true or false judgment result of the real defect image is output based on the feature vector of the second superimposed image.

[0127] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0128] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0129] Correspondingly, embodiments of this application also provide Figure 4 The hardware structure diagram of the device shown is as follows: Figure 5 As shown, the electronic device can be a device implementing the above-described method. Figure 5 As shown, the hardware architecture includes a processor and memory.

[0130] The memory is used to store machine-executable instructions;

[0131] The processor is used to read and execute the machine-executable instructions stored in the memory to implement the corresponding defect detection method embodiment shown above.

[0132] As one embodiment, the memory can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, the memory can be volatile memory, non-volatile memory, or similar storage media. Specifically, the memory can be RAM (Random Access Memory), flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0133] This concludes the process. Figure 5 Description of the electronic device shown.

[0134] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0135] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention filed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0136] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0137] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A defect detection method, characterized in that, This method is applied to electronic devices, and the method includes: A reference mask image is obtained; the reference mask image is generated based on the defect image corresponding to the specified defect object; the reference mask image and the first specified standard image corresponding to the specified object are input into a pre-trained target generator network, so that the target generator network fuses the reference mask image and the first specified standard image into a defect sample image; wherein, the defect sample image is used to train at least a network model for defect detection; The target generator network is trained through the following steps: obtaining candidate mask images for training the target generator network; the candidate mask images are generated based on real defect images corresponding to the specified object; inputting the candidate mask images and the obtained second specified standard image corresponding to the specified object into an initial generator network, so that the initial generator network fuses the candidate mask images and the second specified standard image into a simulated defect image; inputting the simulated defect image and the candidate mask images into a trained discriminator network to obtain a first true / false judgment result of the simulated defect image, and inputting the real... The defect image and the candidate mask image are input into the trained discriminator network to obtain a second true / false judgment result of the real defect image; based on the first true / false judgment result and the second true / false judgment result, the network parameters of the initial generator network are adjusted in reverse. If the adjusted generator network meets the preset conditions, the adjusted generator network is used as the target generator network; if the adjusted generator network does not meet the preset conditions, the adjusted generator network is used as the initial generator network, and the step of inputting the candidate mask image and the obtained second specified standard image corresponding to the specified object into the initial generator network continues. Specifically, when adjusting the network parameters of the initial generator network in reverse based on the first true / false judgment result and the second true / false judgment result, a loss value is calculated based on the first true / false judgment result and the second true / false judgment result, and the network parameters of the initial generator network are adjusted in reverse based on the loss value.

2. The method according to claim 1, characterized in that, The reference mask image is obtained through the following steps: Generate an initial blank image; Determine the defect parameter information and defect rendering method corresponding to a specified application scenario, wherein the specified application scenario is the scenario in which the specified object is located, and the specified defect object refers to the defect that the specified object may have under the specified application scenario; the defect parameter information includes at least defect location information and defect size information; draw the reference mask image on the initial blank image based on the defect parameter information and the defect rendering method; or... The defect mask template of a non-specified application scenario is fused onto the initial blank image to obtain the reference mask image.

3. The method according to claim 1, characterized in that, The candidate mask image is determined through the following steps: For each defect type in the real defect image, the defect area corresponding to that defect type is marked in the real defect image according to the marking method corresponding to that defect type. The candidate mask image is determined based on the real defect image that identifies the defect region corresponding to each defect type.

4. The method according to claim 1, characterized in that, The step of inputting the candidate mask image and the obtained second specified standard image corresponding to the specified object into the initial generator network, so that the initial generator network fuses the candidate mask image and the second specified standard image into a simulated defect image, includes: The first feature vector is obtained by encoding the second specified standard image through the initial generator network, and the second feature vector is obtained by encoding the candidate mask image. The first feature vector and the second feature vector are fused together; The fused feature vectors are then image-decoded to obtain a simulated defect image.

5. The method according to claim 1, characterized in that, The simulated defect image includes a defect region and a background region. The preset conditions include: the difference between the pixel value of each pixel in the defect region of the simulated defect image and the pixel value of the corresponding pixel in the candidate mask image is less than a first preset threshold; and the difference between the pixel value of each pixel in the background region of the simulated defect image and the pixel value of the corresponding pixel in the second specified standard image is less than a second preset threshold.

6. The method according to claim 1, characterized in that, The step of inputting the simulated defect image and the candidate mask image into a trained discriminator network to obtain a first true / false judgment result for the simulated defect image, and inputting the real defect image and the candidate mask image into the trained discriminator network to obtain a second true / false judgment result for the real defect image, includes: The simulated defect image is superimposed with the candidate mask image to obtain a first superimposed image. The feature vector of the first superimposed image is extracted by the discriminator network, and the first true or false judgment result of the simulated defect image is output based on the feature vector of the first superimposed image. The real defect image and the candidate mask image are superimposed to obtain a second superimposed image. The feature vector of the second superimposed image is extracted by the discriminator network, and the second true / false judgment result of the real defect image is output based on the feature vector of the second superimposed image.

7. A defect detection device, characterized in that, This device is used in electronic devices and includes: A reference mask image acquisition module is used to acquire a reference mask image; the reference mask image is generated based on the defect image corresponding to a specified defect object; A defect sample image generation module is used to input the reference mask image and the first specified standard image corresponding to the obtained specified object into a pre-trained target generator network, so that the target generator network fuses the reference mask image and the first specified standard image into a defect sample image; wherein, the defect sample image is used to train at least a network model for defect detection; The candidate mask image acquisition module is used to obtain candidate mask images for training the target generator network; the candidate mask images are generated based on the real defect images corresponding to the specified objects. The simulated defect image generation module is used to input the candidate mask image and the obtained second specified standard image corresponding to the specified object into the initial generator network, so that the initial generator network can fuse the candidate mask image and the second specified standard image into a simulated defect image. The discrimination module is used to input the simulated defect image and the candidate mask image into the trained discriminator network to obtain the first true / false judgment result of the simulated defect image, and to input the real defect image and the candidate mask image into the trained discriminator network to obtain the second true / false judgment result of the real defect image. The target generator network determination module is used to reverse-adjust the network parameters of the initial generator network based on the first and second true / false judgment results. If the adjusted generator network meets the preset conditions, it is used as the target generator network. If the adjusted generator network does not meet the preset conditions, it is used as the initial generator network, and the step of inputting the candidate mask image and the obtained second specified standard image corresponding to the specified object into the initial generator network continues. Specifically, when reverse-adjusting the network parameters of the initial generator network based on the first and second true / false judgment results, the target generator network determination module calculates a loss value based on the first and second true / false judgment results and reverse-adjusts the network parameters of the initial generator network based on the loss value.

8. The apparatus according to claim 7, characterized in that, The device further includes: The initial blank image generation module is used to generate an initial blank image; A drawing module is used to determine the defect parameter information and defect drawing method corresponding to a specified application scenario, wherein the specified application scenario is the scene in which the specified object is located, and the specified defect object refers to the defect that the specified object may have in the specified application scenario; the defect parameter information includes at least defect location information and defect size information; based on the defect parameter information and the defect drawing method, the reference mask image is drawn on the initial blank image; or... The fusion module is used to fuse defect mask templates from non-specified application scenarios onto the initial blank image to obtain the reference mask image.

9. An electronic device, characterized in that, Electronic devices include: processors and memory; The memory is used to store machine-executable instructions; The processor is configured to read and execute machine-executable instructions stored in the memory to implement the method as claimed in any one of claims 1 to 6.

Images