Defect detection, repair method, device and storage medium

Abstract

The disclosed embodiment provides a defect detection and repair method, device and storage medium, wherein the defect detection method comprises: obtaining an image of the current product as an image to be detected, performing feature analysis on the image to be detected, and obtaining a feature atlas containing defect objects in the image to be detected; inputting the feature atlas into a pre-trained convolutional neural network, and obtaining a defect category matrix and defect position score of the defects in the image through the convolutional neural network; and calculating the score of the defect according to the defect category matrix and defect position score. The product yield can be guaranteed and unnecessary waste of production costs can be reduced.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, the field of display technology, and specifically relate to a defect detection and repair method, device, and storage medium. Background Art

[0002] Array defects are various defects caused by the production environment and production conditions during the production process of the array substrate, such as foreign matter residue, eardrum, membrane rupture, etc. The occurrence of some fatal defects will directly affect the quality of the display panel and even cause the final sample to be scrapped. Summary of the invention

[0003] The embodiments of the present disclosure provide a defect detection and repair method, device and storage medium. The detection scheme can detect and evaluate defects in advance, and the repair scheme can repair the detected repairable defects to improve product yield.

[0004] In a first aspect, an embodiment of the present disclosure provides a defect detection method, comprising:

[0005] Acquire an image of the current product as the image to be detected, perform feature analysis on the image to be detected, and obtain a feature atlas containing defect objects in the image to be detected;

[0006] Inputting the feature atlas into a pre-trained convolutional neural network, and obtaining a defect category matrix and defect position scores of defects in the image through the convolutional neural network;

[0007] The scores of the defects are calculated based on the defect category matrix and the defect location scores.

[0008] In a second aspect, an embodiment of the present disclosure provides a defect repair method, including:

[0009] After determining that the defect existing in the current product image is a repairable defect based on the aforementioned defect detection method, determining a defect-free image at the defect location;

[0010] Comparing the original image at the defect location with the non-defective image to obtain a defect repair template;

[0011] Defect repair is performed according to the defect repair template.

[0012] In a third aspect, an embodiment of the present disclosure further provides a defect detection device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the defect detection method described above when executing the program.

[0013] In a fourth aspect, an embodiment of the present disclosure further provides a defect repair device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the defect repair method as described above are implemented.

[0014] In a fifth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to implement the above-mentioned defect detection method and / or defect repair method.

[0015] The defect detection method of the disclosed embodiment achieves a grade assessment of product defects by extracting and scoring defect features, thereby preventing products with unqualified defect grade scores from continuing to be produced. For repairable product defects, the defect repair method of the disclosed embodiment can automatically enter the defect repair link, achieve intelligent repair, ensure product yield, reduce labor costs, and improve repair efficiency.

[0016] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. Other advantages of the present disclosure can be realized and obtained by the schemes described in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0018] Figure 1 This is a flow chart of a defect detection method according to an embodiment of the present disclosure;

[0019] Figure 2 It is a block diagram of the intelligent detection solution for array production according to the embodiment of the present disclosure;

[0020] Figure 3 This is a visualization result diagram of defect feature extraction in the embodiment of the present disclosure;

[0021] Figure 4 This is a flow chart of the defect repair method according to the embodiment of the present disclosure;

[0022] Figure 5 This is a flow chart of the intelligent repair solution for array defects according to an embodiment of the present disclosure;

[0023] Figure 6 A network architecture diagram of an array defect repair network according to an embodiment of the present disclosure;

[0024] Figure 7a It is an original defect characteristic diagram in the short-circuit defect repair process of the embodiment of the present disclosure;

[0025] Figure 7b A defect-free feature map generated by a GAN network in the short-circuit defect repair process of an embodiment of the present disclosure;

[0026] Figure 7c It is a feature map after denoising the defect-free feature map generated by the GAN network during the short-circuit defect repair process of the disclosed embodiment;

[0027] Figure 7d A defect repair template diagram in the short circuit defect repair process of the embodiment of the present disclosure;

[0028] Figure 8a The original image in the process of repairing the circuit breaker defect in the embodiment of the present disclosure;

[0029] Figure 8b A defect location diagram during the repair process of a circuit breaker defect in an embodiment of the present disclosure;

[0030] Figure 8c This is a diagram of network repair results during the circuit breaker defect repair process in the embodiment of the present disclosure;

[0031] Figure 8d It is a defect repair template in the process of repairing the circuit breaker defect in the embodiment of the present disclosure;

[0032] Fig. 9 It is a schematic diagram of the structure of the defect detection device / defect repair device in the embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0034] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.

[0035] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.

[0036] Unless otherwise defined, the technical terms or scientific terms disclosed in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0037] Array defects are defects caused by the production environment and conditions during the production of the array substrate, such as foreign matter residue, eardrum, membrane rupture, etc. Severe cases may cause the membrane to fall off, the membrane to be missing, and even cause disconnection and short circuit, which directly affect the display quality of the panel. The occurrence of some fatal defects will also cause the final product to be scrapped. If defects can be detected and evaluated in advance, and samples with fatal defects can be prevented from continuing to be produced, a lot of production costs can be saved.

[0038] Figure 1The defect detection method provided in the embodiment of the present disclosure includes steps 11 and 12.

[0039] Step 11, obtaining an image of the current product as an image to be detected, performing feature analysis on the image to be detected, and obtaining a feature atlas containing defect objects in the image to be detected;

[0040] The image to be tested may be an image of a product (such as a display panel) on a production line captured in real time, such as a picture of the display panel to be tested acquired by an image acquisition device. For example, in a standard test environment, the display picture of the panel to be tested is captured by one or more cameras.

[0041] For example, VGG-19 can be used to perform feature analysis on the input image to be detected to generate one or more sets of feature atlases, each set of feature atlases (or each feature atlas set group) corresponding to a defect object. A set of feature atlases contains one or more feature maps, that is, a feature atlas includes one or more feature maps.

[0042] Step 12, inputting the feature atlas into a pre-trained convolutional neural network, and obtaining a defect category matrix and defect position scores of defects in the image through the convolutional neural network;

[0043] The convolutional neural network is a type of feed-forward artificial neural network. The connection pattern between neurons in this neural network is inspired by the visual cortex of animals. A single neuron responds to stimuli in a limited area of ​​space (receptive field). The receptive fields of different neurons partially overlap each other, so that they are arranged to form the entire visual field. The response of a single neuron to stimuli within its receptive field can be mathematically approximated by a convolution operation. Therefore, convolutional neural networks have a wide range of applications in the fields of image and video recognition, as well as natural language processing.

[0044] In an exemplary embodiment, the convolutional neural network includes a defect category branch for predicting defect categories and a defect location branch for predicting defect locations. The defect category branch outputs a defect category matrix, which is used to represent the probability that the defect belongs to one or more defect categories. The defect location branch outputs a defect location score, which is used to represent the severity of the defect location. The defect category includes one or more of the following information: ignorable defects, general defects, repairable defects, and fatal defects.

[0045] When training and testing the defect category branch of the CNN network, the center loss function and the cross entropy loss function can be used as the target optimization function. The center loss function can reduce the difference within the class. The cross entropy loss function can measure the difference between two different probability distributions in the same random variable. In machine learning, it is expressed as the difference between the true probability distribution and the predicted probability distribution. The smaller the cross entropy value, the better the model prediction effect. The root mean square error can be used as the loss function when training and testing the defect location branch of the CNN network.

[0046] Step 13, calculating the score of the defect according to the defect category matrix and the defect location score.

[0047] In an exemplary embodiment, the defect category matrix, the defect location score, and the defect category influencing factor can be used to calculate the score of the defect. For example, the product of the above three is used as the score of the defect. In other embodiments, other calculation methods can also be used to calculate the score of the defect.

[0048] In an exemplary embodiment, step 14 may be included after step 13, and whether the production of the current product needs to be stopped is determined based on the score of the defect. For example, if the score of the defect is too high (higher than a first preset threshold), the production of the current product is stopped, for example, by an alarm indicating that the product is unqualified and production needs to be stopped. For another example, the average score of multiple defects in the image to be detected of the current product is calculated as the average defect score of the current product, and whether the production of the current product needs to be stopped is determined based on the average defect score. For example, if the average defect score is higher than a second preset threshold, the production of the current product is stopped.

[0049] Taking the product of panel as an example, a method for judging array defects on the panel is described through an embodiment, which can realize intelligent judgment of processes such as "continue", "repair" and "stop" in the array production process, thereby preventing panels with defect level scores higher than the threshold from continuing to be produced, and judging whether the panel defects can be repaired. If so, the defect repair link is entered. If not, the panel production is stopped to ensure product yield and reduce unnecessary waste of production costs.

[0050] The intelligent detection process of array production based on convolutional neural network is as follows Figure 2 As shown, the following steps are included:

[0051] The original image is input into the feature extraction layer, and the CNN network is used to extract the features of the original array defect image to obtain a feature map. The feature extraction layer is mainly used to separate the background of the image to be detected from the defect, ignore the defect background, and highlight the defect. The defect feature extraction layer may include a convolution layer, an activation layer, and a pooling layer, wherein the convolution layer is generally a collection of small neurons that partially overlap each other (which are also called convolution kernels in a mathematical sense); the activation layer is used to add nonlinear factors; the pooling layer is mainly used to downsample the output data of the previous layer while maintaining the features, reducing the amount of calculation and preventing overfitting. Figure 3 This is the visualization result after a multi-layer network extracts features from array defects. The inverse convolution method can be used to achieve visualization.

[0052] The features after background removal are input into the defect category branch and defect location branch of the CNN network respectively, and then the defect category matrix and defect location score are obtained.

[0053] Among them, in the defect category branch, center loss and cross entropy loss are used as objective functions. The morphology of array defects themselves is diverse and not fixed, such as residues, particles, etc., which are amorphous. Therefore, in order to obtain the "common" features of similar defects, this embodiment uses center loss and cross entropy loss as the target optimization function to extract the common features of defects.

[0054] Among them, in the defect location branch, the root mean square error loss function can be used as the target optimization function.

[0055] The final score C = μ*A*B of the defect is calculated based on the defect category matrix A and the defect location score B, where μ is the defect category influencing factor. For example, according to the impact of the array defect on the panel and the impact on the subsequent process, the defects can be divided into: negligible defects, general defects, repairable defects and fatal defects, and the corresponding influencing factor μ of each defect can be pre-defined according to the actual situation. The types of defects here are only examples, and this disclosure does not limit this.

[0056] For example, suppose there are five types of defects, and the defect category matrix output by the CNN network is A = [0.1, 0.1, 0.2, 0.3, 0.9], which represents the probability matrix of the current defect belonging to each category; the defect location score output by the CNN network is B = 0.4, which represents the location score of the current defect. Then the final score of the current defect is:

[0057] C=μ*A*B=0.4*[μ1,μ2,μ3,μ4,μ5]*[0.1,0.1,0.2,0.3,0.9] T .

[0058] After calculating the score of a defect, the above steps can be repeated to obtain the scores of other defects on the panel, and then the defect grade score of the defect on the entire panel can be calculated. For example, the formula Calculate the average defect score of the entire panel, where C i represents the score of the i-th defect, and n is the total number of defects.

[0059] A defect score threshold is pre-set, and if the average defect score of a panel is detected to be higher than the defect score threshold, the panel can be prevented from continuing production.

[0060] In an exemplary embodiment, the following steps may also be included: judging whether the defect can be repaired according to the defect category, and if so, entering the defect repair phase, and if not, stopping production to ensure product yield and reduce unnecessary waste of production costs;

[0061] For example, whether the current defect is repairable can be determined based on the probability that the current defect is a repairable defect in the defect category matrix A. For example, the probability can be compared with a preset probability threshold. If the probability is greater than the preset probability threshold, the current defect is considered to be a repairable defect. In other embodiments, the determination can also be made in combination with the defect location or other parameters, which is not limited in the present disclosure.

[0062] The array production intelligent decision-making solution based on convolutional neural network in this embodiment realizes the level evaluation of display panel defects through defect feature extraction and scoring, so as to prevent panels with defect level scores higher than the threshold from continuing to be produced, ensure product yield, and reduce unnecessary waste of production costs. Compared with the method of using AOI (Automatic OpticInspect) automatic optical inspection equipment to detect defects, CNN-based defect detection can obtain classification results, can detect non-periodic units, and the detection results are less affected by noise such as light.

[0063] As mentioned above, the presence of array defects will have a great impact on the performance of the product and even affect the display of the entire panel. Through defect repair, some defects can be repaired or even eliminated, thereby improving the yield of the product. At present, array defect repair in factories mainly relies on manual repair by operators. The repair effect is closely related to the operator's experience, professional skills, and even mood. This results in uneven repair results, which directly affects the performance of the product and even causes the product to be directly scrapped. Even experienced old employees need a month to fully master a defect repair solution.

[0064] To this end, the present disclosure also provides a defect repair method, such as Figure 4 As shown, the following steps are included:

[0065] Step 21, after determining that the defect existing in the current product image is a repairable defect according to the aforementioned defect detection method, a defect-free image at the defect location is determined;

[0066] Step 22, comparing the original image at the defect location with the non-defective image to obtain a defect repair template;

[0067] In an exemplary embodiment, determining the defect-free image at the defect location includes: using a generative adversarial network (GAN) to generate the defect-free image at the defect location.

[0068] In an exemplary embodiment, the defect pattern is determined by comparing the original image of the defect position with the defect-free image, and the defect pattern is used as a defect repair template. The defect repair template includes defect contour coordinates.

[0069] Step 23: Repair the defect according to the defect repair template.

[0070] For example, when the defect is a short circuit defect, the defect is removed by laser according to the defect contour. For another example, when the defect is an open circuit defect, the defect contour is repaired by punching holes at both ends.

[0071] The defect repair method described in the embodiment of the present disclosure can realize intelligent repair of product defects, assist or even replace manual repair, reduce labor costs, improve defect repair efficiency and repair effect, and promote the intelligent upgrade of factories.

[0072] Taking the panel as an example, an embodiment of a method for repairing array defects on a panel based on a convolutional neural network is described below. The method can realize intelligent repair of array defects, assist or even replace manual repair, reduce labor costs, improve array defect repair efficiency and repair effect, and promote the intelligent upgrade of the factory.

[0073] The array defect repair process based on convolutional neural network is as follows Figure 5 As shown, after using a defect detection network (or defect recognition network) to identify the position and category of the array defect and determining that the defect is a repairable defect, a defect template is first generated (step 31). The defect template can be a solid color picture that can cover the defect, such as a whiteboard; then the defect cover template is used to replace the array defect position, that is, to cover the defect (step 32); a generative adversarial network (GAN) is used to generate a repaired image after the defect is removed, that is, a defect-free image (step 33), which is used as a reference for array defect repair; the original defect image is compared with the repaired image to obtain an array defect repair template (step 34); finally, a program is used to control the repair device according to the repair template to realize intelligent repair of array defects (step 35).

[0074] The specific implementation process is as follows:

[0075] First, the defect detection network is used to detect array defects and output the category matrix and position score of array defects. The main purpose of this step is to obtain the category and location information of array defects, filter out defects that cannot be repaired or do not need to be repaired, and array defects that can be repaired can be screened out for intelligent repair.

[0076] According to the purpose of defect repair, array defects can be divided into two categories: repairable defects and non-repairable defects. Among them, repairable defects can be subdivided into metal line open, metal line short, film layer open, film layer short, etc., where metal line or film layer open can be repaired by punching connection, and metal line or film layer short can be repaired by cutting.

[0077] For repairable array defects, a generative adversarial network (GAN) is used to generate a repair image after removing the defect, that is, the array defect repair result, which is used as a reference for array defect repair. The original defect image is compared with the array defect repair result to obtain an image containing only the array defect morphology as an array defect repair template. Finally, the program is used to control the repair equipment according to the repair template to realize intelligent repair of array defects.

[0078] The overall architecture of the neural network used for defect repair is as follows Figure 6 As shown. The defect repair network includes a method for obtaining contour features ( Figure 6 The first network of feature 1) is used to obtain color features ( Figure 6 The invention relates to a second network for fusing the features output by the first network and the second network. The first network includes a first encoder (encoder 1) and a first decoder (decoder 1), and the second network includes a second encoder (encoder 2) and a second decoder (decoder 2). The third network includes a third decoder for fusing the contour features output by the first network and the color features output by the second network, and decoding the fused features.

[0079] In an exemplary embodiment, when in use, the original image after the occlusion defect can be input into the first network and the second network respectively, and the input image is convolved with a series of trained filters through encoding (i.e. compression), and it is downsampled to obtain a "minimum representation", wherein the first network obtains feature 1, i.e., the contour feature of the original image, through encoder 1, and the second network obtains feature 2, i.e., the color feature of the original image, through encoder 2. Feature 1 and feature 2 are fused and then upsampled and decoded to construct a new image, i.e., the original image after the defect is removed, and the new image retains the contour feature and color feature in the original image.

[0080] The training of the defect repair network can be divided into 3 parts:

[0081] The first part is the training of the first network, that is, the training of the defect image contour repair autoencoder network, which is used to repair the contour information of the image. The contour repair autoencoder network includes a first encoder and a first decoder. The first encoder is used to compress the image to extract the features in the image, and the first decoder is used to decompress the image and restore the image. The input is a defective image. The first encoder extracts features from the image, and the first decoder repairs the defective image according to the extracted image features to obtain the repaired contour map of the input image. When the network training is completed, when using the network, only the first encoder is required to compress the image to obtain the features. Figure 1 , that is, the defect contour feature.

[0082] The second part is the training of the second network, that is, the training of the defect image color filling autoencoding network. This network can realize the color filling of the contour image. The specific process is similar to the first part. The color filling autoencoding network includes a second encoder and a second decoder. The input is a defect image. The second encoder extracts the color of the image. The second decoder repairs the defect image according to the extracted color features to obtain the color features of the defective part. When the network training is completed, when using the network, only the second encoder is required to compress the image to obtain the features. Figure 2 , that is, the defect color feature.

[0083] The third part is the training of the third network, that is, the training of the decoding network after the fusion of the defect contour repair feature map and the defect contour color filling feature map. The fused features include the features of defect contour repair and the features of defect contour color filling, which can realize defect contour repair and color filling.

[0084] Taking short circuit defect repair as an example, the above repair scheme is explained. The defect image to be repaired ( Figure 7a ) Input the GAN-based defect repair network to generate defect-free images ( Figure 7b ), the final restoration result image can be obtained through denoising ( Figure 7c ), compare the repair result with the original defect image (subtract the repair result from the original defect image 7a Figure 7c ), and get the defect repair template diagram ( Figure 7d ), the defect repair template only contains the defective part, and the defective part is repaired by the repair machine, for example, the defect repair template Figure 7d The defective parts are removed by laser, and finally a repaired and defect-free product is obtained.

[0085] According to the flow chart of the circuit breaker defect repair solution, the defect image to be repaired ( Figure 8a ) is input into the CNN-based defect detection network to obtain the defect detection result, that is, the defect image ( Figure 8b ), input the defect image into the GAN-based defect repair network to obtain the defect repair result image ( Figure 8c ), by taking the defect picture to be repaired (8a) or the defect image Figure 8b Comparing with the defect repair result diagram (8c), a defect repair template (8d) is obtained. The defective part is repaired using a repair machine, for example, the two ends of the broken wire in the defect repair template can be connected by punching and evaporation processes.

[0086] In an exemplary embodiment, the present disclosure further provides a defect detection device and a defect repair device. The defect detection device may include a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the defect detection method in any of the above embodiments of the present disclosure when executing the computer program. The defect repair device may include a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the defect repair method in any of the above embodiments of the present disclosure when executing the computer program.

[0087] In an exemplary embodiment, Fig. 9 FIG. 1 is a schematic diagram of the structure of the defect detection device / defect repair device in the embodiment of the present disclosure. When necessary, the two devices can be combined together. Fig. 9 As shown, the device 90 includes: at least one processor 901; and at least one memory 902 and a bus 903 connected to the processor 901; wherein the processor 901 and the memory 902 communicate with each other through the bus 903; the processor 901 is used to call the program instructions in the memory 902 to execute the steps of the defect detection method and / or defect repair method in any of the above-mentioned embodiments.

[0088] The processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a transistor logic device, etc., and the present disclosure does not limit this.

[0089] The memory may include a read-only memory (ROM) and a random access memory (RAM), and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0090] In addition to the data bus, the bus may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig. 9 In the text file, various buses are marked as buses.

[0091] In the implementation process, the processing performed by the processing device can be completed by the hardware integrated logic circuit in the processor or the instructions in the form of software. That is, the method steps of the embodiment of the present disclosure can be embodied as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0092] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0093] Although the embodiments disclosed in the present invention are as above, the contents described are only embodiments adopted to facilitate understanding of the present invention and are not intended to limit the present invention. Any technician in the field to which the present invention belongs can make any modifications and changes in the form and details of implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A defect detection method, characterized in that: include: Acquire an image of the current product as the image to be detected, perform feature analysis on the image to be detected, and obtain a feature atlas containing defect objects in the image to be detected; Inputting the feature atlas into a pre-trained convolutional neural network, and obtaining a defect category matrix and defect position scores of defects in the image through the convolutional neural network; The score of the defect is calculated according to the defect category matrix and the defect location score; wherein the score C of the defect is calculated using the following formula: C=μ*A*B Among them, μ is the defect category influencing factor, which is determined by the defect category to which the defect belongs, A is the defect category matrix, and B is the defect location score.

2. The defect detection method according to claim 1, characterized in that: in: The convolutional neural network includes a defect category branch for predicting a defect category and a defect location branch for predicting a defect location.

3. The defect detection method according to claim 1 or 2, characterized in that: Also includes: Whether the production of the current product needs to be stopped is determined based on the score of the defect.

4. The defect detection method according to claim 3, characterized in that: The determining whether it is necessary to stop the production of the current product based on the defect score includes: calculating an average defect score of the current product based on the scores of all defects in the image to be detected, and stopping the production of the current product when the average defect score is higher than a preset defect score threshold.

5. The defect detection method according to claim 1, 2 or 4, characterized in that: The method further includes: if the defect can be repaired according to the defect category, then entering the defect repair phase, wherein: the defect category to which the defect belongs includes one or more of the following categories: ignorable defect, general defect, repairable defect, and fatal defect.

6. A defect repair method, characterized in that: include: After determining that the defect existing in the current product image is a repairable defect based on the defect detection method according to any one of claims 1 to 5, determining a defect-free image at the defect location; Comparing the original image at the defect location with the non-defective image to obtain a defect repair template; Defect repair is performed according to the defect repair template.

7. The defect repair method according to claim 6, characterized in that: in: The determining of the defect-free image at the defect position includes: using a generative adversarial network (GAN) to generate the defect-free image at the defect position.

8. The defect repair method according to claim 7, characterized in that: in: The generative adversarial network includes a first network for obtaining contour features, a second network for obtaining color features, and a third network for fusing features output by the first network and the second network, wherein the first network includes a first encoder and a first decoder, the second network includes a second encoder and a second decoder, the third network includes a third decoder, and the third network is used to fuse the contour features output by the first network and the color features output by the second network, and decode the fused features.

9. The defect repair method according to claim 7 or 8, characterized in that: in: The step of comparing the original image of the defect position with the defect-free image to obtain a defect repair template includes: determining a defect pattern by comparing the original image of the defect position with the defect-free image, and using the defect pattern as a defect repair template, wherein the defect repair template includes defect contour coordinates.

10. The defect repair method according to claim 9, characterized in that: in: The defect repairing according to the defect repair template includes: when the defect is a short circuit defect, using laser to cut the defect according to the defect contour; when the defect is an open circuit defect, repairing by punching holes at both ends of the defect contour.

11. A defect detection device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the defect detection method according to any one of claims 1 to 5 are implemented.

12. A defect repairing device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the defect repair method according to any one of claims 6 to 10 are implemented.

13. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method of any one of claims 1-5 or 5-10.

Images