Panel defect detection method, device, electronic equipment and storage medium

Abstract

The present invention relates to the field of image detection technology, and discloses a panel defect detection method, including: receiving a panel short circuit image, performing defect detection on the panel short circuit image according to a pre-trained defect detection model to obtain panel defect coordinates, performing cropping on the panel short circuit image according to the panel defect coordinates to obtain a panel defect image including defects, locating the metal line to which the defect belongs in the panel defect image, and finding the upper narrow line and the lower narrow line of the metal line based on digital image processing technology, judging whether the upper and lower narrow lines intersect with the defect in the panel defect image, and when the upper and lower narrow lines intersect with the defect in the panel defect image, judging whether the panel defect image has a broken line, and obtaining a broken line detection result of the panel defect image. The present invention also proposes a panel defect detection device, an electronic device, and a storage medium. The present invention can solve the problem of low defect detection rate of short circuit panels.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a panel defect detection method, device, electronic equipment and storage medium. Background Art

[0002] In the industrial manufacturing process of circuit boards, various forms of circuit board defects will occur due to factors such as process fluctuations and machine differences. Among them, the most serious circuit board defects include short circuit defects.

[0003] At present, the monitoring of short-circuit defects mainly relies on human identification, but human identification consumes too many resources. In the era of Industry 2.0, more and more circuit board manufacturers are beginning to use artificial intelligence ADC (automatic defect classification system) to replace human defect identification. At present, the detection model of artificial intelligence ADC still cannot achieve a high accuracy rate in short-circuit defect detection. Summary of the invention

[0004] The present invention provides a panel defect detection method, device, electronic equipment and storage medium, the main purpose of which is to solve the problem of low short-circuit panel defect detection rate.

[0005] To achieve the above object, the present invention provides a panel defect detection method, comprising:

[0006] Receiving a panel short-circuit image, and performing defect detection on the panel short-circuit image according to a pre-trained defect detection model to obtain panel defect coordinates;

[0007] Cutting the panel short circuit map according to the panel defect coordinates to obtain a panel defect map including the defects;

[0008] Locating the metal line to which the defect belongs in the panel defect image, and finding the upper narrow line and the lower narrow line of the metal line based on digital image processing technology;

[0009] Determine whether the upper narrow line and the lower narrow line intersect with the defects in the panel defect map. When the upper narrow line and the lower narrow line intersect with the defects in the panel defect map, determine whether there is a broken line in the panel defect map to obtain the broken line detection result of the panel defect map.

[0010] Based on the problem that the current detection model described in the background technology cannot achieve a high accuracy rate in short-circuit defect detection, the embodiment of the present invention first receives a panel short-circuit map, and performs defect detection on the panel short-circuit map according to the pre-trained defect detection model to obtain panel defect coordinates, because the panel defect coordinates may include a panel with a broken line, which provides a pre-condition for subsequent broken line detection, and then the panel short-circuit map is cropped according to the panel defect coordinates to obtain a panel defect map including defects, and further locates the metal line to which the defect belongs in the panel defect map, and searches for the upper narrow line and the lower narrow line of the metal line based on digital image processing technology, because the upper narrow line and the lower narrow line do not intersect each other in principle, if they intersect, a short circuit may be caused, so the upper and lower narrow lines intersect with the defects in the panel defect map, and determine whether the panel defect map has a broken line, and obtain the broken line detection result of the panel defect map. It can be seen that the panel defect detection of the present invention includes coordinate detection, narrow line detection and broken line detection, and the accuracy of broken line detection can be improved through careful multiple detections. Therefore, the panel defect detection method, device, electronic device and calculation proposed by the present invention can solve the problem of low detection rate of short-circuit panel defects.

[0011] Optionally, the clipping the panel short circuit map according to the panel defect coordinates to obtain a panel defect map includes:

[0012] Generate a defect rectangular frame according to the panel defect coordinates, and locate the frame center of the defect rectangular frame;

[0013] According to the preset length and width cropping ratio, cropping is performed with the center of the frame as the cropping center to obtain the panel defect map.

[0014] In a preferred embodiment of the present invention, a defect rectangular frame is generated with the panel defect coordinates as the center, and the defect rectangular frame further determines the cropping area, which can reduce the detection area of ​​the defective panel and save resource consumption for subsequent panel short circuit detection.

[0015] Optionally, locating the metal line to which the defect belongs in the panel defect image, and finding the upper narrow line and the lower narrow line of the metal line based on digital image processing technology, includes:

[0016] receiving the metal line identified by a user in the panel defect map;

[0017] Binarizing the panel defect map to obtain a binary defect map;

[0018] Creating a convolution kernel of the binary defect map, and performing an opening operation on the binary defect map using the convolution kernel to obtain an optimized defect map;

[0019] Based on the Hough line detection method, edge detection is performed on the metal line to obtain an upper narrow line and a lower narrow line of the metal line.

[0020] The embodiment of the present invention first pre-processes the panel defect image through digital image processing methods including binarization and convolution operations to remove some noise interference, and at the same time performs edge detection on the metal line based on the Hough line detection method to obtain the upper narrow line and the lower narrow line of the metal line. Since the upper narrow line and the lower narrow line are key information for judging panel defects, the defect position of the panel can be further accurately determined to improve the defect accuracy rate.

[0021] Optionally, creating a convolution kernel of the binary defect map includes:

[0022] The convolution kernel of the binary defect map is created according to the following calculation formula:

[0023]

[0024] Among them, kernerSize vertical represents the convolution kernel, h is the height of the binary defect map, and w is the width of the binary defect map.

[0025] The embodiment of the present invention can remove some panel defect image noise through a specific convolution kernel, thereby preventing subsequent panel defect detection from reducing detection accuracy due to noise interference.

[0026] Optionally, the determining whether there is a line break in the panel defect graph includes:

[0027] Performing threshold segmentation and denoising processing on the panel defect image to obtain a defect contour image to be extracted;

[0028] Using a contour detection method, performing contour extraction on the defect contour to be extracted image to obtain one or more groups of contour lines;

[0029] Calculate the area of ​​the contour region where each group of contour lines is located in turn, and extract the contour lines whose contour area area is greater than a preset threshold, to obtain one or more groups of broken line confirmation lines;

[0030] Determine whether the line to be confirmed has a breakpoint. If the line to be confirmed has a breakpoint, the panel defect map has a breakpoint. If the line to be confirmed does not have a breakpoint, the panel defect map does not have a breakpoint.

[0031] In an embodiment of the present invention, in order to prevent false detection of panel short circuits and further improve detection efficiency, threshold segmentation and denoising processing are first performed on the panel defect map to obtain a defect contour map to be extracted, and then contour lines whose contour area is greater than a preset threshold are extracted based on the defect contour map to be extracted to obtain one or more groups of broken wires to be confirmed. It should be explained that the broken wires to be confirmed include broken wires and non-broken wires, but most solutions do not further distinguish between broken wires and non-broken wires. Therefore, in order to further improve the detection accuracy of panel broken wires, an embodiment of the present invention first determines whether the broken wire to be confirmed line has a breakpoint. If the broken wire to be confirmed line has a breakpoint, then the panel defect map has a broken wire. If the broken wire to be confirmed line does not have a breakpoint, then the panel defect map does not have a broken wire, thereby improving the detection accuracy of panel broken wires.

[0032] Optionally, sequentially calculating the area of ​​the contour region where each group of contour lines is located includes:

[0033] The following calculation method is used to calculate the area of ​​the contour region where each group of contour lines is located:

[0034]

[0035] Among them, P(x,y) and Q(x,y) are any points within the contour line. and represents the first-order partial derivative.

[0036] In the embodiment of the present invention, the calculation accuracy of the contour area can be improved by using a specified contour area calculation formula.

[0037] Optionally, the pre-trained defect detection model includes:

[0038] Receive the number of convolutional layers input by the user, generate the same number of convolutional layers as the number of convolutional layers, and construct a defect detection model to be trained according to the convolutional layers;

[0039] A panel image training set is received, and the defect detection model to be trained is trained using the panel image training set to obtain the pre-trained defect detection model.

[0040] It has been proven that convolutional neural networks have excellent detection capabilities in the field of images. Therefore, the embodiment of the present invention is based on the number of convolutional layers and uses convolutional layers to construct a defect detection model. In order to improve the detection accuracy of the defect detection model, the panel image training set is used to optimize the model parameters of the defect detection model to be trained, so that the defect detection model can achieve the best detection effect on panel defects.

[0041] Optionally, generating the same number of convolutional layers as the number of the convolutional layers, and constructing a defect detection model to be trained according to the convolutional layers includes:

[0042] Generate the same number of convolutional layers to be adjusted as the number of the convolutional layers;

[0043] According to the preset convolution kernel size, the convolution kernel size of the convolution layer to be adjusted is adjusted, and the convolution step size of the convolution layer to be adjusted is set to the user's requirement;

[0044] A fully connected neural network and an activation function are added after the convolution layer with adjusted convolution kernel size and convolution step size to obtain the defect detection model to be trained.

[0045] The detection effect of the convolutional neural network is greatly affected by the convolution kernel and the convolution step size. Therefore, in order to improve the detection effect of the defect detection model, the embodiment of the present invention adjusts the convolution kernel size of the convolution layer to be adjusted according to the pre-set convolution kernel size, and sets the convolution step size of the convolution layer to be adjusted to the user's requirements, so that the defect detection model has excellent performance in the application of panel defect detection.

[0046] Optionally, the using the panel image training set to train the defect detection model to be trained to obtain the pre-trained defect detection model further includes:

[0047] Extracting a label of each panel image from the panel image training set, wherein the label of each panel image includes a normal panel or a defective panel;

[0048] The panel defect coordinates input by the user according to the label of the defective panel are received, and a panel image label set corresponding to the panel image training set is obtained.

[0049] The training set has a crucial impact on the model performance of the neural network. An excellent training set can improve the robustness of the neural network. Therefore, in an embodiment of the present invention, in order to further improve the accuracy of the defect detection model, a panel image training set is first received, and then a panel image label set corresponding to the panel image training set is constructed through user calibration. The model parameter optimization process is improved by combining the panel image training set with the panel image label set.

[0050] Optionally, the using the panel image training set to train the defect detection model to be trained to obtain the pre-trained defect detection model includes:

[0051] Receiving a training learning rate and a batch size input by a user according to the defect detection model to be trained, and inputting the panel image training set into the defect detection model to be trained;

[0052] According to the training learning rate and the batch size, using the convolution layer to perform feature extraction on the panel image training set to obtain a panel image feature set;

[0053] Inputting the panel image feature set into the fully connected neural network to perform dimensional transformation to obtain a single-dimensional image feature set;

[0054] Activating the one-dimensional image feature set using the activation function to obtain a predicted panel label set;

[0055] Calculating the error value between the predicted panel label set and the panel image training set;

[0056] Determine whether the error value is greater than a specified threshold value, if the error value is greater than the specified threshold value, adjust the training learning rate, the batch size and the internal parameters of the defect detection model to be trained, and return to the above-mentioned convolutional layer feature extraction step;

[0057] Until the error value is less than or equal to the specified threshold, the pre-trained defect detection model is obtained.

[0058] The neural network training process is a process of user intervention and adjustment. The present invention is implemented to improve the training effect of model training, receive the training learning rate and batch size input by the user according to the defect detection model to be trained, and optimize the internal parameters of the defect detection model in turn through the training learning rate and batch size, thereby improving the training process and improving the detection accuracy of the subsequent defect detection model for the defective panel.

[0059] In order to solve the above problems, the present invention further provides a panel defect detection device, the device comprising:

[0060] A defect coordinate detection module is used to receive a panel short circuit image, perform defect detection on the panel short circuit image according to a pre-trained defect detection model, and obtain panel defect coordinates;

[0061] A cropping module, used for cropping the panel short circuit map according to the panel defect coordinates to obtain a panel defect map including defects;

[0062] A narrow line search module, used to locate the metal line to which the defect belongs in the panel defect image, and to find the upper narrow line and the lower narrow line of the metal line based on digital image processing technology;

[0063] A narrow line intersection judgment module, used to judge whether the upper narrow line and the lower narrow line intersect with the defects in the panel defect map;

[0064] The broken line detection module is used to determine whether there is a broken line in the panel defect map when the upper narrow line and the lower narrow line intersect with the defect in the panel defect map, and obtain the broken line detection result of the panel defect map.

[0065] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0066] a memory storing at least one instruction; and

[0067] The processor executes the instructions stored in the memory to implement the above-mentioned panel defect detection method.

[0068] In order to solve the above problem, the present invention further provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned panel defect detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of a flow chart of a panel defect detection method provided by an embodiment of the present invention;

[0070] Figure 2 for Figure 1 The defective panel with broken and unbroken wires in one of the steps;

[0071] Figure 3 for Figure 1 A broken wire detection diagram of a defective panel in another step;

[0072] Figure 4 A functional module diagram of a panel defect detection device provided by an embodiment of the present invention;

[0073] Figure 5 A schematic diagram of the structure of an electronic device for implementing the panel defect detection method provided by an embodiment of the present invention.

[0074] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0075] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0076] The embodiment of the present application provides a panel defect detection method. The execution subject of the panel defect detection method includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the panel defect detection method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0077] Reference Figure 1 FIG. 1 is a flow chart of a panel defect detection method provided by an embodiment of the present invention. In this embodiment, the panel defect detection method includes:

[0078] S1. Receive a panel short-circuit image, and perform defect detection on the panel short-circuit image according to a pre-trained defect detection model to obtain panel defect coordinates.

[0079] It should be explained that the circuit panel is the basis of all power equipment, so it is extremely important to detect whether the panel is abnormal. The embodiment of the present invention uses a pre-trained defect detection model to first perform defect detection on the panel short circuit diagram to obtain the panel defect coordinates.

[0080] In detail, the pre-trained defect detection model includes:

[0081] Receive the number of convolutional layers input by the user, generate the same number of convolutional layers as the number of convolutional layers, and construct a defect detection model to be trained according to the convolutional layers;

[0082] A panel image training set is received, and the defect detection model to be trained is trained using the panel image training set to obtain the pre-trained defect detection model.

[0083] It should be understood that the convolution layer is the basic building layer of the deep neural network, and the defect detection model of the present invention can be constructed by connecting the convolution layers with each other.

[0084] In detail, generating the same number of convolutional layers as the number of convolutional layers, and constructing a defect detection model to be trained according to the convolutional layers includes:

[0085] Generate the same number of convolutional layers to be adjusted as the number of the convolutional layers;

[0086] According to the preset convolution kernel size, the convolution kernel size of the convolution layer to be adjusted is adjusted, and the convolution step size of the convolution layer to be adjusted is set to the user's requirement;

[0087] A fully connected neural network and an activation function are added after the convolution layer with adjusted convolution kernel size and convolution step size to obtain the defect detection model to be trained.

[0088] Exemplarily, the present invention can generate 9 convolutional layers to be adjusted, and sequentially design the convolution kernel size of the first 3 layers to be 3*3, the convolution kernel size of the middle 3 layers to be 4*4, and the convolution layer size of the last 3 layers to be 5*5, and the step size of each convolution layer is designed to be 1.

[0089] Furthermore, the panel image training set includes panels that can work normally, defective panels and the coordinates of the defective panels. The normal working panels and defective panels can help the defect detection model to iteratively train the internal model parameters to achieve the optimal value. The defective panels include disconnected defective panels and undisconnected defective panels. For details, please refer to Figure 2 As shown, Figure 2 The broken line defect panel (a) and the broken line defect panel (b) are two example cases of the broken line defect panel, and the non-broken line defect panel (c) and the non-broken line defect panel (d) are two example cases of the non-broken line defect panel.

[0090] Therefore, the panel image training set is used to train the defect detection model to be trained to obtain the pre-trained defect detection model, which also includes:

[0091] Extracting a label of each panel image from the panel image training set, wherein the label of each panel image includes a normal panel or a defective panel;

[0092] The panel defect coordinates input by the user according to the label of the defective panel are received, and a panel image label set corresponding to the panel image training set is obtained.

[0093] Further, after obtaining the panel image training set and the defect detection model to be trained, using the panel image training set to train the defect detection model to be trained to obtain the pre-trained defect detection model includes:

[0094] Receiving a training learning rate and a batch size input by a user according to the defect detection model to be trained, and inputting the panel image training set into the defect detection model to be trained;

[0095] According to the training learning rate and the batch size, using the convolution layer to perform feature extraction on the panel image training set to obtain a panel image feature set;

[0096] Inputting the panel image feature set into the fully connected neural network to perform dimensional transformation to obtain a single-dimensional image feature set;

[0097] Activating the one-dimensional image feature set using the activation function to obtain a predicted panel label set;

[0098] Calculating the error value between the predicted panel label set and the panel image training set;

[0099] Determine whether the error value is greater than a specified threshold value, if the error value is greater than the specified threshold value, adjust the training learning rate, the batch size and the internal parameters of the defect detection model to be trained, and return to the above-mentioned convolutional layer feature extraction step;

[0100] Until the error value is less than or equal to the specified threshold, the pre-trained defect detection model is obtained.

[0101] In an embodiment of the present invention, the training learning rate and the batch size can be set to 0.001 and 32 respectively, and the activation function can be set to the relu function until the training requirements are met. After the defect detection model is obtained, the defect detection model is used to detect the panel short circuit map. When the panel short circuit map is a defective panel, the defect coordinates of the output panel will correspond.

[0102] S2. Cutting the panel short circuit map according to the panel defect coordinates to obtain a panel defect map including defects.

[0103] It should be explained that since the defect coordinate range obtained by the above defect detection model when detecting the panel short-circuit map is too large, it is necessary to further perform cropping based on the panel defect coordinates. In detail, the panel short-circuit map is cropped according to the panel defect coordinates to obtain the panel defect map, including:

[0104] Generate a defect rectangular frame according to the panel defect coordinates, and locate the frame center of the defect rectangular frame;

[0105] According to the preset length and width cropping ratio, cropping is performed with the center of the frame as the cropping center to obtain the panel defect map.

[0106] In the embodiment of the present invention, the cropping ratio is set to 10%, based on cropping 10% up, down, left, and right of the center of the defective rectangular frame.

[0107] S3. Locate the metal line to which the defect belongs in the panel defect image, and find the upper narrow line and the lower narrow line of the metal line based on digital image processing technology.

[0108] In detail, the method of locating the metal line to which the defect belongs in the panel defect image and finding the upper narrow line and the lower narrow line of the metal line based on digital image processing technology includes:

[0109] receiving the metal line identified by a user in the panel defect map;

[0110] Binarizing the panel defect map to obtain a binary defect map;

[0111] Creating a convolution kernel of the binary defect map, and performing an opening operation on the binary defect map using the convolution kernel to obtain an optimized defect map;

[0112] Based on the Hough line detection method, edge detection is performed on the metal line to obtain an upper narrow line and a lower narrow line of the metal line.

[0113] Furthermore, the convolution kernel for creating the binary defect map includes:

[0114] The convolution kernel of the binary defect map is created according to the following calculation formula:

[0115]

[0116] Among them, kernerSize vertical represents the convolution kernel, h is the height of the binary defect map, and w is the width of the binary defect map.

[0117] S4. Determine whether the upper and lower narrow lines intersect with defects in the panel defect map. When the upper and lower narrow lines intersect with defects in the panel defect map, determine whether there is a line break in the panel defect map to obtain a line break detection result of the panel defect map.

[0118] In detail, the determining whether there is a disconnection in the panel defect map includes:

[0119] Performing threshold segmentation and denoising processing on the panel defect image to obtain a defect contour image to be extracted;

[0120] Using a contour detection method, performing contour extraction on the defect contour to be extracted image to obtain one or more groups of contour lines;

[0121] Calculate the area of ​​the contour region where each group of contour lines is located in turn, and extract the contour lines whose contour area area is greater than a preset threshold, to obtain one or more groups of broken line confirmation lines;

[0122] Determine whether the line to be confirmed has a breakpoint. If the line to be confirmed has a breakpoint, the panel defect map has a breakpoint. If the line to be confirmed does not have a breakpoint, the panel defect map does not have a breakpoint.

[0123] Furthermore, the step of sequentially calculating the area of ​​the contour region where each group of contour lines is located includes:

[0124] The following calculation method is used to calculate the area of ​​the contour region where each group of contour lines is located:

[0125]

[0126] Among them, P(x,y) and Q(x,y) are any points within the contour line. and It can be seen that after the contour area is calculated, it is judged whether the contour area is greater than the preset threshold, and the contour line with the contour area greater than the preset threshold is extracted to obtain the above-mentioned broken line to be confirmed line. Furthermore, when the broken line to be confirmed line has a breakpoint, it indicates that there is a broken line in the panel defect map, and the broken line detection result is obtained. For the broken line detection result, please refer to Figure 3 shown.

[0127] Based on the problem that the current detection model described in the background technology cannot achieve a high accuracy rate in short-circuit defect detection, the embodiment of the present invention first receives a panel short-circuit map, and performs defect detection on the panel short-circuit map according to the pre-trained defect detection model to obtain panel defect coordinates, because the panel defect coordinates may include a panel with a broken line, which provides a pre-condition for subsequent broken line detection, and then the panel short-circuit map is cropped according to the panel defect coordinates to obtain a panel defect map including defects, and further locates the metal line to which the defect belongs in the panel defect map, and searches for the upper narrow line and the lower narrow line of the metal line based on digital image processing technology, because the upper narrow line and the lower narrow line do not intersect each other in principle, if they intersect, a short circuit may be caused, so the upper and lower narrow lines intersect with the defects in the panel defect map, and determine whether the panel defect map has a broken line, and obtain the broken line detection result of the panel defect map. It can be seen that the panel defect detection of the present invention includes coordinate detection, narrow line detection and broken line detection, and the accuracy of broken line detection can be improved through careful multiple detections. Therefore, the panel defect detection method, device, electronic device and calculation proposed by the present invention can solve the problem of low detection rate of short-circuit panel defects.

[0128] like Figure 4 , is a functional module diagram of a panel defect detection device provided by an embodiment of the present invention.

[0129] The panel defect detection device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the panel defect detection device 100 may include a defect coordinate detection module 101, a cropping module 102, a narrow line search module 103, a narrow line intersection judgment module 104 and a broken line detection module 105. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0130] The defect coordinate detection module 101 is used to receive a panel short circuit image, perform defect detection on the panel short circuit image according to a pre-trained defect detection model, and obtain panel defect coordinates;

[0131] The clipping module 102 is used to clip the panel short circuit map according to the panel defect coordinates to obtain a panel defect map including defects;

[0132] The narrow line search module 103 is used to locate the metal line to which the defect belongs in the panel defect image, and find the upper narrow line and the lower narrow line of the metal line based on the digital image processing technology;

[0133] The narrow line intersection judgment module 104 is used to judge whether the upper and lower narrow lines intersect with the defects in the panel defect map;

[0134] The broken line detection module 105 is used to determine whether there is a broken line in the panel defect map when the upper and lower narrow lines intersect with defects in the panel defect map, and obtain a broken line detection result of the panel defect map.

[0135] In detail, each module in the panel defect detection device 100 in the embodiment of the present invention is used in the same manner as described above. Figure 1 The panel defect detection method described in the invention has the same technical means and can produce the same technical effects, so it will not be repeated here.

[0136] like Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a panel defect detection method provided by an embodiment of the present invention.

[0137] The electronic device 1 may include a processor 10 , a memory 11 , and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10 , such as a panel defect detection program 12 .

[0138] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the panel defect detection program 12, etc., but also be used to temporarily store data that has been output or is to be output.

[0139] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules (such as panel defect detection programs, etc.) stored in the memory 11, and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

[0140] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0141] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0142] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0143] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0144] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0145] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0146] The panel defect detection program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0147] Receiving a panel short-circuit image, and performing defect detection on the panel short-circuit image according to a pre-trained defect detection model to obtain panel defect coordinates;

[0148] Cutting the panel short circuit map according to the panel defect coordinates to obtain a panel defect map including the defects;

[0149] Locating the metal line to which the defect belongs in the panel defect image, and finding the upper narrow line and the lower narrow line of the metal line based on digital image processing technology;

[0150] Determine whether the upper and lower narrow lines intersect with the defects in the panel defect map. When the upper and lower narrow lines intersect with the defects in the panel defect map, determine whether there is a broken line in the panel defect map to obtain a broken line detection result of the panel defect map.

[0151] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 5 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0152] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0153] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0154] Receiving a panel short-circuit image, and performing defect detection on the panel short-circuit image according to a pre-trained defect detection model to obtain panel defect coordinates;

[0155] Cutting the panel short circuit map according to the panel defect coordinates to obtain a panel defect map including the defects;

[0156] Locating the metal line to which the defect belongs in the panel defect image, and finding the upper narrow line and the lower narrow line of the metal line based on digital image processing technology;

[0157] Determine whether the upper and lower narrow lines intersect with the defects in the panel defect map. When the upper and lower narrow lines intersect with the defects in the panel defect map, determine whether there is a broken line in the panel defect map to obtain a broken line detection result of the panel defect map.

[0158] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0159] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0161] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0162] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0163] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0164] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A panel defect detection method, characterized in that: The method comprises: Receiving a panel short-circuit image, performing defect detection on the panel short-circuit image according to a pre-trained defect detection model, and obtaining panel defect coordinates when the panel short-circuit image is a defective panel; Cutting the panel short circuit map according to the panel defect coordinates to obtain a panel defect map including the defects; Locating the metal line to which the defect belongs in the panel defect image, and finding the upper narrow line and the lower narrow line of the metal line based on digital image processing technology; Determine whether the upper narrow line and the lower narrow line intersect with the defects in the panel defect map; when the upper narrow line and the lower narrow line intersect with the defects in the panel defect map, determine whether there is a line break in the panel defect map, and obtain a line break detection result of the panel defect map; The determining whether the panel defect map has a broken line includes: Performing threshold segmentation and denoising processing on the panel defect image to obtain a defect contour image to be extracted; Using a contour detection method, performing contour extraction on the defect contour to be extracted image to obtain one or more groups of contour lines; Calculate the area of ​​the contour region where each group of contour lines is located in turn, and extract the contour lines whose contour area area is greater than a preset threshold, to obtain one or more groups of broken line to be confirmed; Determine whether the line to be confirmed has a breakpoint. If the line to be confirmed has a breakpoint, the panel defect map has a breakpoint. If the line to be confirmed does not have a breakpoint, the panel defect map does not have a breakpoint. The step of sequentially calculating the area of ​​the contour region where each group of contour lines is located includes: The following calculation method is used to calculate the area of ​​the contour region where each group of contour lines is located: Among them, P and Q are any points within the contour line. and represents the first-order partial derivative.

2. The panel defect detection method according to claim 1, characterized in that: The step of clipping the panel short circuit map according to the panel defect coordinates to obtain a panel defect map includes: Generate a defect rectangular frame according to the panel defect coordinates, and locate the frame center of the defect rectangular frame; According to the preset length and width cropping ratio, cropping is performed with the center of the frame as the cropping center to obtain the panel defect map.

3. The panel defect detection method according to claim 2, characterized in that: The method of locating the metal line to which the defect belongs in the panel defect image and finding the upper narrow line and the lower narrow line of the metal line based on the digital image processing technology includes: receiving the metal line identified by a user in the panel defect map; Binarizing the panel defect map to obtain a binary defect map; Creating a convolution kernel of the binary defect map, and performing an opening operation on the binary defect map using the convolution kernel to obtain an optimized defect map; Based on the Hough line detection method, edge detection is performed on the metal line to obtain an upper narrow line and a lower narrow line of the metal line.

4. The panel defect detection method according to claim 3, characterized in that: The convolution kernel for creating the binary defect map comprises: The convolution kernel of the binary defect map is created according to the following calculation formula: Among them, kernerSize vertical represents the convolution kernel, h is the height of the binary defect map, and w is the width of the binary defect map.

5. The panel defect detection method according to claim 1, characterized in that: The pre-trained defect detection model includes: Receive the number of convolutional layers input by the user, generate the same number of convolutional layers as the number of convolutional layers, and construct a defect detection model to be trained according to the convolutional layers; A panel image training set is received, and the defect detection model to be trained is trained using the panel image training set to obtain the pre-trained defect detection model.

6. The panel defect detection method according to claim 5, characterized in that: The generating of the same number of convolutional layers as the number of the convolutional layers, and constructing a defect detection model to be trained according to the convolutional layers, comprises: Generate the same number of convolutional layers to be adjusted as the number of convolutional layers; According to the preset convolution kernel size, the convolution kernel size of the convolution layer to be adjusted is adjusted, and the convolution step size of the convolution layer to be adjusted is set to the user's requirement; A fully connected neural network and an activation function are added after the convolution layer with adjusted convolution kernel size and convolution step size to obtain the defect detection model to be trained.

7. The panel defect detection method according to claim 6, characterized in that: The method of using the panel image training set to train the defect detection model to be trained to obtain the pre-trained defect detection model also includes: Extracting a label of each panel image from the panel image training set, wherein the label of each panel image includes a normal panel or a defective panel; The panel defect coordinates input by the user according to the label of the defective panel are received, and a panel image label set corresponding to the panel image training set is obtained.

8. The panel defect detection method according to claim 7, characterized in that: The method of training the defect detection model to be trained by using the panel image training set to obtain the pre-trained defect detection model includes: Receiving a training learning rate and a batch size input by a user according to the defect detection model to be trained, and inputting the panel image training set into the defect detection model to be trained; According to the training learning rate and the batch size, using the convolution layer to perform feature extraction on the panel image training set to obtain a panel image feature set; Inputting the panel image feature set into the fully connected neural network to perform dimensional transformation to obtain a single-dimensional image feature set; Activating the one-dimensional image feature set using the activation function to obtain a predicted panel label set; Calculating the error value between the predicted panel label set and the panel image training set; Determine whether the error value is greater than a specified threshold value, if the error value is greater than the specified threshold value, adjust the training learning rate, the batch size and the internal parameters of the defect detection model to be trained, and return to the above-mentioned convolutional layer feature extraction step; Until the error value is less than or equal to the specified threshold, the pre-trained defect detection model is obtained.

9. A panel defect detection device, characterized in that: The device comprises: A defect coordinate detection module is used to receive a panel short-circuit image, perform defect detection on the panel short-circuit image according to a pre-trained defect detection model, and obtain panel defect coordinates when the panel short-circuit image is a defective panel; A cropping module, used for cropping the panel short circuit map according to the panel defect coordinates to obtain a panel defect map including defects; A narrow line search module, used to locate the metal line to which the defect belongs in the panel defect image, and to find the upper narrow line and the lower narrow line of the metal line based on digital image processing technology; A narrow line intersection judgment module, used to judge whether the upper narrow line and the lower narrow line intersect with the defects in the panel defect map; A line break detection module, used for determining whether there is a line break in the panel defect map when the upper narrow line and the lower narrow line intersect with the defect in the panel defect map, and obtaining a line break detection result of the panel defect map; The determining whether the panel defect map has a broken line includes: Performing threshold segmentation and denoising processing on the panel defect image to obtain a defect contour image to be extracted; Using a contour detection method, performing contour extraction on the defect contour to be extracted image to obtain one or more groups of contour lines; Calculate the area of ​​the contour region where each group of contour lines is located in turn, and extract the contour lines whose contour area area is greater than a preset threshold, to obtain one or more groups of broken line to be confirmed; Determine whether the line to be confirmed has a breakpoint. If the line to be confirmed has a breakpoint, the panel defect map has a breakpoint. If the line to be confirmed does not have a breakpoint, the panel defect map does not have a breakpoint. The step of sequentially calculating the area of ​​the contour region where each group of contour lines is located includes: The following calculation method is used to calculate the area of ​​the contour region where each group of contour lines is located: Among them, P and Q are any points within the contour line. and represents the first-order partial derivative.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the panel defect detection method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the panel defect detection method according to any one of claims 1 to 8 is implemented.

Images