Defect detection method and device and electronic equipment
By extracting and partitioning the brightness feature of the photovoltaic glass white space area, the error detection and missed detection of defect detection in the photovoltaic glass white space area in the prior art is solved, and the accurate positioning of defects in the gradient type white space area is achieved.
Patent Information
- Application Number
- CN202411973202.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-25
AI Technical Summary
When detecting defects in the white area of photovoltaic glass, the prior art is affected by light and background factors, resulting in the binarization method being mis-detection and missed detection.
The brightness type is determined by extracting the brightness feature of the blank area in the photovoltaic glass image. If the brightness type is a gradient type, it is partitioned and divided into multiple target sub-regions with a brightness type that is non-gradient, and then defect extraction is performed through the target algorithm.
It realizes accurate and comprehensive positioning of defects in the blank area where the brightness type is gradient type, and improves the accuracy and applicability of detection.
Smart Images
Figure CN119887707A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular relates to a defect detection method, device and electronic equipment. Background Art
[0002] Photovoltaic glass is widely used in solar cell modules and photovoltaic systems. After screen printing, photovoltaic glass has screen printing areas and blank areas. Defects in the blank area (such as cracks, stains, etc.) will increase the impedance of the current transmission path, which will reduce the current efficiency of the photovoltaic module and reduce the reliability of the photovoltaic module in long-term use. Therefore, it is very important to detect defects in the blank area.
[0003] The related technology detects defects in the blank area directly through the binarization method. However, due to the influence of factors such as lighting and background, there are large differences in the grayscale of different photovoltaic glass products. Some blank areas have a gradual brightness change. In this case, the brightness of the defect may be higher than the background. Directly using the binarization method may result in problems such as false detection and missed detection. Summary of the invention
[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a defect detection method, device and electronic device, which can accurately and comprehensively locate the position information of defects in the blank area with a gradient brightness type by dividing the blank area with a gradient brightness type into multiple target sub-areas with non-gradient brightness types.
[0005] In a first aspect, the present application provides a defect detection method, the method comprising:
[0006] When it is determined that there are defects in the blank area of the photovoltaic glass image, brightness feature extraction processing is performed on the blank area to obtain the brightness type of the blank area; the brightness type includes a gradient type and a non-gradient type;
[0007] When it is determined that the brightness type of the blank area is a gradient type, partitioning the blank area at least once to obtain a plurality of target sub-areas, where the brightness type of the target sub-areas is a non-gradient type;
[0008] Perform defect extraction processing on each target sub-region through a target algorithm to obtain a defect extraction result of each target sub-region; the target algorithm includes at least one of a binarization algorithm and a connected area detection algorithm;
[0009] According to the defect extraction results of each target sub-region, the position information of the defects in the blank area is obtained.
[0010] According to the defect detection method of the present application, by partitioning the blank area at least once when determining that the brightness type of the blank area is a gradient type, a plurality of target sub-areas are obtained, and the brightness type of the target sub-areas is a non-gradient type, the blank area with a gradient brightness type is divided into a plurality of target sub-areas with a non-gradient brightness type, and defect extraction processing can be performed on each target sub-area through a target algorithm to obtain a defect extraction result for each target sub-area; based on the defect extraction result of each target sub-area, the location information of the defects in the blank area is obtained, so as to accurately and comprehensively locate the location information of the defects in the blank area with a gradient brightness type.
[0011] According to one embodiment of the present application, the target sub-region includes a first target sub-region and a second target sub-region;
[0012] Partition the blank area at least once to obtain multiple target sub-areas, including:
[0013] Traverse each sub-region of this iteration; each sub-region of the first iteration is obtained by partitioning the blank region;
[0014] When it is determined that the sub-region currently traversed has a defect, a brightness feature extraction process is performed on the sub-region currently traversed to obtain a brightness type of the sub-region currently traversed;
[0015] In the case where it is determined that the brightness type of the sub-region currently traversed is a non-gradient type, determining the sub-region currently traversed as a first target sub-region;
[0016] When it is determined that the currently traversed sub-region has no defects, the currently traversed sub-region is determined to be the second target sub-region.
[0017] According to one embodiment of the present application, after obtaining the brightness type of the sub-region currently traversed, the method further includes:
[0018] When it is determined that the brightness type of the sub-region currently traversed is a gradient type, partitioning the sub-region currently traversed according to the grayscale of each column of pixels in the sub-region currently traversed and the target number, to obtain the target number of sub-regions;
[0019] The target number of sub-regions belong to the sub-regions of the next iteration.
[0020] According to an embodiment of the present application, defect extraction processing is performed on each target sub-region through a target algorithm to obtain a defect extraction result of each target sub-region, including:
[0021] Performing defect extraction processing on each first target sub-region by using a target algorithm to obtain a defect extraction result of each first target sub-region;
[0022] According to the defect extraction results of each target sub-area, the location information of the defects in the blank area is obtained, including:
[0023] According to the defect extraction results of each first target sub-region, the position information of the defects in the blank region is obtained.
[0024] According to an embodiment of the present application, for any area in the blank area or sub-area, brightness feature extraction processing is performed on the area to obtain the brightness type of the area, including:
[0025] Performing histogram extraction processing and vertical projection extraction processing on the region to obtain the histogram features and vertical projection features of the region;
[0026] Perform feature fusion on the histogram features and vertical projection features of the region to obtain fused features;
[0027] Input the fused features into the pre-trained first classification network model to obtain the brightness type of the area output by the first classification network model;
[0028] The first classification network model uses the fusion features of the histogram features and the vertical projection features of the sample image as training samples, and the brightness type of the sample image as training labels.
[0029] According to an embodiment of the present application, when it is determined that there is a defect in the blank area in the photovoltaic glass image, before performing brightness feature extraction processing on the blank area, the method further includes:
[0030] Extract the image of the blank area;
[0031] Input the image of the blank area into the trained second classification network model to obtain the classification category of the blank area output by the second classification network model;
[0032] The second classification network model is a first image in which the brightness type of the blank area is a non-gradient type and contains defects, a second image in which the brightness type of the blank area is a gradient type and contains defects, a third image in which the brightness type of the blank area is a non-gradient type and does not contain defects, and a fourth image in which the brightness type of the blank area is a gradient type and does not contain defects as training samples, and is trained using the classification category of the blank area as training samples; the classification category is the presence of defects or the absence of defects.
[0033] According to an embodiment of the present application, after obtaining the brightness type of the blank area, the method includes:
[0034] When it is determined that the brightness type of the blank area is a non-gradient type, a defect extraction process is performed on the blank area through a target algorithm to obtain position information of defects in the blank area.
[0035] In a second aspect, the present application provides a defect detection device, the device comprising:
[0036] The first processing module is used to extract brightness features of the blank area in the photovoltaic glass image to obtain the brightness type of the blank area when it is determined that there is a defect in the blank area in the photovoltaic glass image; the brightness type includes a gradient type and a non-gradient type;
[0037] A second processing module is used to partition the blank area at least once to obtain a plurality of target sub-areas when it is determined that the brightness type of the blank area is a gradient type, and the brightness type of the target sub-areas is a non-gradient type;
[0038] The third processing module is used to perform defect extraction processing on each target sub-region through a target algorithm to obtain a defect extraction result of each target sub-region; the target algorithm includes at least one of a binarization algorithm and a connected area detection algorithm;
[0039] The fourth processing module is used to obtain the position information of the defects in the blank area according to the defect extraction results of each target sub-area.
[0040] According to the defect detection device of the present application, by partitioning the blank area at least once when determining that the brightness type of the blank area is a gradient type, a plurality of target sub-areas are obtained, and the brightness type of the target sub-areas is a non-gradient type; the blank area with a gradient brightness type is divided into a plurality of target sub-areas with a non-gradient brightness type, and defect extraction processing can be performed on each target sub-area through a target algorithm to obtain a defect extraction result for each target sub-area; based on the defect extraction result of each target sub-area, the location information of the defects in the blank area is obtained, so as to accurately and comprehensively locate the location information of the defects in the blank area with a gradient brightness type.
[0041] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the defect detection method provided in the first aspect described above is implemented.
[0042] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the defect detection method provided in the first aspect described above is implemented.
[0043] In a fifth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the defect detection method provided in the first aspect.
[0044] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the defect detection method provided in the first aspect above.
[0045] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:
[0046] When it is determined that there are defects in the blank area of the photovoltaic glass image, brightness feature extraction is performed on the blank area to obtain the brightness type of the blank area; the brightness type includes a gradient type and a non-gradient type; when it is determined that the brightness type of the blank area is a gradient type, the blank area is partitioned at least once to obtain a plurality of target sub-areas, and the brightness type of the target sub-areas is a non-gradient type; the blank area with a gradient brightness type is divided into a plurality of target sub-areas with a non-gradient brightness type, and defect extraction processing can be performed on each target sub-area through a target algorithm to obtain a defect extraction result for each target sub-area; based on the defect extraction results of each target sub-area, the location information of the defects in the blank area is obtained, so as to accurately and comprehensively locate the location information of the defects in the blank area with a gradient brightness type.
[0047] In addition, when it is determined that the brightness type of the blank area is a gradient type, the blank area is divided into multiple target sub-areas with non-gradient brightness types, and each target sub-area can be directly subjected to defect extraction processing through the target algorithm to obtain the location information of the defects in the blank area; when it is determined that the brightness type of the blank area is a non-gradient type, there is no need to partition the blank area, and the blank area can be directly subjected to defect extraction processing through the target algorithm to obtain the location information of the defects in the blank area. This method is suitable for defect detection in blank areas of different models and under different imaging conditions, and has strong applicability and versatility.
[0048] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0050] Figure 1 This is one of the flow charts of the defect detection method provided in the embodiment of the present application;
[0051] Figure 2a-2b is a schematic diagram of a photovoltaic glass image provided in an embodiment of the present application;
[0052] Figure 3a-3bis a schematic diagram of a blank area with a brightness type of a gradient type and a brightness type of a non-gradient type provided in an embodiment of the present application;
[0053] Figure 4 is a schematic diagram of partitioning a sub-region of this iteration provided by an embodiment of the present application;
[0054] Figure 5a-5b It is a schematic diagram of the histogram distribution and the vertical projection distribution of a blank area of a non-gradient brightness type provided in an embodiment of the present application;
[0055] Figure 6a-6b It is a schematic diagram of the histogram distribution and the vertical projection distribution of a blank area of a certain brightness type of a gradient type provided in an embodiment of the present application;
[0056] Figure 7 This is the second flow chart of the defect detection method provided in the embodiment of the present application;
[0057] Figure 8 is a schematic diagram of the structure of a defect detection device provided in an embodiment of the present application;
[0058] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0059] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0060] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0061] The defect detection method, defect detection device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0062] The defect detection method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.
[0063] The defect detection method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the defect detection method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The defect detection method provided in the embodiment of the present application is described below using the electronic device as an example of the execution subject.
[0064] like Figure 1 As shown, an embodiment of the present application provides a flow chart of a defect detection method, and the defect detection method includes: step 110, step 120, step 130 and step 140.
[0065] Step 110: When it is determined that there are defects in the blank area in the photovoltaic glass image, brightness feature extraction processing is performed on the blank area to obtain the brightness type of the blank area; the brightness type includes a gradient type and a non-gradient type.
[0066] Photovoltaic glass is widely used in solar cell modules and photovoltaic systems. In its production process, screen printing is a key process, and its quality directly affects the performance and reliability of photovoltaic products. In the screen printing process, the glass surface is printed to form a circuit pattern, and the accuracy and integrity of the screen printing edge are critical to the overall function.
[0067] The photovoltaic glass image in the embodiment of the present application can be an image of the photovoltaic glass that has been screen-printed. The photovoltaic glass image is usually a grayscale image. Of course, in some solutions, it can also be a color image, which is not limited.
[0068] After the photovoltaic glass is screen-printed, there are screen-printed areas and blank areas.
[0069] The screen-printed area of photovoltaic glass mainly refers to the electrode pattern area formed by the screen printing process. These patterns are used to conduct electricity and collect the current generated by photovoltaic cells.
[0070] The blank area of photovoltaic glass refers to the area on the glass surface that is not covered by coating, pattern or other materials. The blank area is usually not used for the functional structural part of the photovoltaic cell.
[0071] Photovoltaic glass usually has multiple blank areas, which can be of any size and shape.
[0072] like Figure 2a-2b As shown, the embodiment of the present application provides a schematic diagram of a photovoltaic glass image, Figure 2aThe image shown is a screen-printed photovoltaic glass image, where the black part is the screen-printed area and the grid area that is not screen-printed is the blank area. Figure 2b The image shown is Figure 2a The enlarged partial image in the upper left corner shows, from outside to inside, the glass edge, the silk-screen area, and the blank area.
[0073] For each blank area, the blank area can be roughly identified first to determine whether the corresponding blank area has defects. Specifically, the blank area can be determined by a deep learning neural network. Specifically, an image of the blank area can be extracted; the image of the blank area is preprocessed; the image of the blank area is input into a trained second classification network model to obtain a classification category of the blank area output by the second classification network model, and the classification category is either defective or non-defective.
[0074] The second classification network model can be a first image in which the brightness type of the blank area is a non-gradient type and contains defects, a second image in which the brightness type of the blank area is a gradient type and contains defects, a third image in which the brightness type of the blank area is a non-gradient type and does not contain defects, and a fourth image in which the brightness type of the blank area is a gradient type and does not contain defects as training samples, and is trained with the classification category of the blank area as the training sample, and the classification category is the presence of defects or the absence of defects.
[0075] The second classification network model belongs to a CNN (Convolutional Neural Network) network, and specifically, may be a classification model such as Resnet18 and FCNNet.
[0076] In practical applications, the ratio of images containing defects in the blank area to images containing no defects in the blank area can be between 3:7 and 7:3, that is, the ratio of the first total number between the first image and the second image to the second total number between the third image and the fourth image can be between 3:7 and 7:3.
[0077] In addition, the ratio of the image with a non-gradient brightness type in the blank area to the image with a gradient brightness type in the blank area can be between 8:2 and 6:4, that is, the ratio of the third total number between the first image and the third image to the second total number between the second image and the fourth image can be between 8:2 and 6:4.
[0078] Preprocessing includes but is not limited to rotating, scaling, translating, and cropping the image.
[0079] In the embodiment, when it is determined that there are defects in the blank area, brightness feature extraction processing is performed on the blank area to obtain the brightness type of the blank area. Specifically, the brightness and texture features of the blank area can be extracted through histogram distribution extraction, vertical projection distribution extraction, etc., to obtain the brightness change trend of the blank area, and then the brightness type of the blank area can be obtained.
[0080] The brightness type includes a gradient type and a non-gradient type.
[0081] The brightness type is a gradient type, indicating that the brightness value of the entire blank area or a sub-area of the blank area has a changing trend of gradually decreasing or increasing, presenting a visual effect from light to dark or from dark to light.
[0082] The brightness type is a non-gradient type, indicating that the brightness value of the entire blank area or the sub-area of the blank area does not change gradually or suddenly, and the brightness value is relatively balanced. The non-gradient type can be regarded as a normal type.
[0083] like Figure 3a-Figure 3b As shown, the embodiment of the present application provides a schematic diagram of a blank area with a brightness type of a gradient type and a brightness type of a non-gradient type. Figure 3a The brightness type of the blank area in the middle is non-gradient, and the brightness value is relatively balanced; Figure 3b The brightness type of the blank area in the middle is a gradient type, and its brightness value gradually decreases, presenting a visual effect from light to dark.
[0084] Step 120 : When it is determined that the brightness type of the blank area is a gradient type, partition the blank area at least once to obtain a plurality of target sub-areas, and the brightness type of the target sub-areas is a non-gradient type.
[0085] When the brightness type of the blank area is determined to be a gradient type, the brightness of the defect may be higher than that of the background, and the defect in the blank area may not be accurately and comprehensively detected directly by the binarization method.
[0086] To avoid the above situation, in the embodiment of the present application, when determining that the brightness type of the blank area is a gradient type, the blank area is partitioned at least once to obtain multiple target sub-areas, and the brightness type of the target sub-areas is a non-gradient type. That is, the blank area is divided into multiple target sub-areas whose brightness type is a non-gradient type, and the size of each target sub-area can be the same or different.
[0087] Step 130: performing defect extraction processing on each target sub-region by using a target algorithm to obtain a defect extraction result of each target sub-region; the target algorithm includes at least one of a binarization algorithm and a connected area detection algorithm;
[0088] Since the brightness type of the target sub-region is a non-gradient type, each target sub-region can be directly subjected to defect extraction processing by a target algorithm to obtain a defect extraction result of each target sub-region. The target algorithm here includes at least one of a binarization algorithm and a connected area detection algorithm.
[0089] The binarization algorithm may specifically be the Otsu algorithm, and the connected area detection algorithm may specifically be the Blob algorithm.
[0090] The defect extraction result may be that the target sub-region has no defects, or the location where defects exist in the target sub-region.
[0091] Step 140: Obtain location information of defects in the blank area according to the defect extraction results of each target sub-area.
[0092] After the defect extraction results of all target sub-regions are obtained, the defect extraction results of each target sub-region may be combined and mapped to the blank region, thereby obtaining the position information of the defects in the blank region.
[0093] In the embodiment of the present application, when it is determined that there are defects in the blank area in the photovoltaic glass image, brightness feature extraction processing is performed on the blank area to obtain the brightness type of the blank area; the brightness type includes a gradient type and a non-gradient type; when it is determined that the brightness type of the blank area is a gradient type, the blank area is partitioned at least once to obtain multiple target sub-areas, and the brightness type of the target sub-areas is a non-gradient type; the blank area with a gradient brightness type is divided into multiple target sub-areas with a non-gradient brightness type, and defect extraction processing can be performed on each target sub-area through a target algorithm to obtain a defect extraction result for each target sub-area; based on the defect extraction results of each target sub-area, the location information of the defects in the blank area is obtained, so as to accurately and comprehensively locate the location information of the defects in the blank area with a gradient brightness type.
[0094] In some embodiments, after obtaining the brightness type of the blank area, the method includes:
[0095] When it is determined that the brightness type of the blank area is a non-gradient type, a defect extraction process is performed on the blank area through a target algorithm to obtain position information of defects in the blank area.
[0096] In the embodiment of the present application, when it is determined that the brightness type of the blank area is a gradient type, the blank area is divided into multiple target sub-areas with non-gradient brightness types, and defect extraction processing can be directly performed on each target sub-area through a target algorithm to obtain the location information of defects in the blank area; when it is determined that the brightness type of the blank area is a non-gradient type, there is no need to partition the blank area, and defect extraction processing can be directly performed on the blank area through a target algorithm to obtain the location information of defects in the blank area. The invention is suitable for defect detection in blank areas of different models and under different imaging conditions, and has strong applicability and versatility.
[0097] In some embodiments, the target sub-region includes a first target sub-region and a second target sub-region;
[0098] Partition the blank area at least once to obtain multiple target sub-areas, including:
[0099] Traverse each sub-region of this iteration; each sub-region of the first iteration is obtained by partitioning the blank region;
[0100] When it is determined that the sub-region currently traversed has a defect, a brightness feature extraction process is performed on the sub-region currently traversed to obtain a brightness type of the sub-region currently traversed;
[0101] In the case where it is determined that the brightness type of the sub-region currently traversed is a non-gradient type, determining the sub-region currently traversed as a first target sub-region;
[0102] When it is determined that the currently traversed sub-region has no defects, the currently traversed sub-region is determined to be the second target sub-region.
[0103] In some embodiments, after obtaining the brightness type of the sub-region currently traversed, the method further includes:
[0104] When it is determined that the brightness type of the sub-region currently traversed is a gradient type, partitioning the sub-region currently traversed according to the grayscale of each column of pixels in the sub-region currently traversed and the target number, to obtain the target number of sub-regions;
[0105] The target number of sub-regions belong to the sub-regions of the next iteration.
[0106] The process of partitioning the blank area at least once in the embodiment of the present application is an iterative process, and the iteration can be started after the blank area is partitioned for the first time. Since the changes in the blank area are consistent in the vertical direction, the sub-areas can be partitioned according to the grayscale of each column of pixels in the blank area and the target number to obtain the target number of sub-areas; the target number can be the number of clusters, that is, the pixels in each column are clustered according to the grayscale of each column of pixels and the number of clusters to obtain the target number of cluster categories, and each cluster category corresponds to a sub-area.
[0107] For each subsequent iteration, each sub-region of this iteration can be traversed. For the sub-region currently traversed, it can be determined whether the sub-region currently traversed has defects. When it is determined that the sub-region currently traversed does not have defects, the sub-region currently traversed is determined to be the second target sub-region. The second target sub-region does not need to be partitioned again. In addition, there is no need to perform defect extraction processing on the second target sub-region subsequently.
[0108] When it is determined that the sub-region currently traversed has defects, brightness feature extraction processing is performed on the sub-region currently traversed to obtain the brightness type of the sub-region currently traversed. The brightness feature extraction processing includes but is not limited to histogram distribution extraction, vertical projection distribution extraction, etc., to extract the brightness and texture features of the sub-region currently traversed, obtain the brightness change trend of the sub-region currently traversed, and then obtain the brightness type of the sub-region currently traversed.
[0109] When it is determined that the brightness type of the currently traversed sub-region is a non-gradient type, the currently traversed sub-region is determined to be a first target sub-region, and the first target sub-region does not need to be partitioned again, but a defect extraction process needs to be performed on the first target sub-region subsequently.
[0110] When it is determined that the brightness type of the currently traversed sub-region is a gradient type, it is necessary to further partition the currently traversed sub-region. The main goal of partitioning is to divide areas with similar grayscale distribution into the same sub-region. Specifically, the currently traversed sub-region can be partitioned according to the grayscale of each column of pixels in the currently traversed sub-region and the target number, and the obtained target number of sub-regions belong to the sub-regions of the next round of iteration.
[0111] After it is determined that there are no sub-regions that need to be divided, that is, when all sub-regions are either the first target sub-regions or the second target sub-regions, it is determined that the iteration stops.
[0112] In practical applications, in order to reduce the impact of defects, the sub-region vertical projection grayscale analysis can be performed, and a clustering method, such as Kmeans, etc., can be used (the target number can be used as the number of clustering categories, and there is no limit on the value of the target number, such as 3, 4, 5, etc.), and the distance of the grayscale in the region is used as a similarity measurement method, such as but not limited to Euclidean distance, cosine similarity, etc., and the sub-region with a gradient brightness type in this iteration is divided into a target number of sub-regions, and the target number of sub-regions belong to the sub-regions of the next round of iteration.
[0113] like Figure 4 As shown, the embodiment of the present application provides a schematic diagram of partitioning a sub-region of this iteration, and horizontally, the sub-region of this iteration is further divided into 4 regions.
[0114] In some embodiments, performing defect extraction processing on each target sub-region to obtain a defect extraction result for each target sub-region includes:
[0115] Performing defect extraction processing on each first target sub-region by using a target algorithm to obtain a defect extraction result of each first target sub-region;
[0116] According to the defect extraction results of each target sub-area, the location information of the defects in the blank area is obtained, including:
[0117] According to the defect extraction results of each first target sub-region, the position information of the defects in the blank region is obtained.
[0118] The above-mentioned embodiment has explained that the first target sub-region is a region with a non-gradient brightness type and contains defects, and the second target sub-region is a region with a gradient brightness type and does not contain defects. Therefore, when performing defect extraction, the defect extraction process can be directly performed on each first target sub-region through the target algorithm to obtain the defect extraction result of each first target sub-region. The defect extraction result can be that the first target sub-region has no defects, or the location of the defect in the first target sub-region.
[0119] After obtaining the defect extraction results of all first target sub-areas, the defect extraction results of each first target sub-area can be combined and mapped to the blank area to obtain the location information of the defects in the blank area. The location information can be represented by pixel coordinates and / or sub-pixel coordinates, without limitation.
[0120] In some embodiments, for any area in the blank area or the sub-area, a brightness feature extraction process is performed on the area to obtain a brightness type of the area, including:
[0121] Performing histogram extraction processing and vertical projection extraction processing on the region to obtain the histogram features and vertical projection features of the region;
[0122] Perform feature fusion on the histogram features and vertical projection features of the region to obtain fused features;
[0123] Input the fused features into the pre-trained first classification network model to obtain the brightness type of the area output by the first classification network model;
[0124] The first classification network model uses the fusion features of the histogram features and the vertical projection features of the sample image as training samples, and the brightness type of the sample image as training labels.
[0125] For the blank area or sub-area whose brightness type is a gradient type, brightness feature extraction processing can be performed in the same way.
[0126] Specifically, for any area in the blank area or sub-area, the area can be subjected to histogram extraction processing and vertical projection extraction processing to obtain the histogram features (also called histogram distribution) and vertical projection features (also called vertical projection distribution) of the area. Of course, in some other embodiments, other feature extraction algorithms that can extract the grayscale distribution and texture features of the area can also be used, without limitation.
[0127] In fact, the histogram distribution and vertical projection distribution of the blank area with a gradient brightness type and the blank area with a non-gradient brightness type are different, and there is a relatively obvious difference between the two.
[0128] like Figure 5a-5b As shown, the embodiment of the present application provides a schematic diagram of the histogram distribution and a schematic diagram of the vertical projection distribution obtained after the histogram extraction processing and the vertical projection extraction processing are performed on the blank area of the brightness type that is not a gradient type, Figure 5a for Figure 3a Schematic diagram of the histogram distribution of the blank area of the medium brightness type and non-gradient type. Figure 5b for Figure 3a Schematic diagram of the vertical projection distribution of the blank area of the medium brightness type and non-gradient type.
[0129] like Figure 6a-6b As shown, the embodiment of the present application provides a schematic diagram of the histogram distribution and a schematic diagram of the vertical projection distribution obtained after the histogram extraction processing and the vertical projection extraction processing are performed on the blank area of the brightness type of the gradient type, Figure 6a for Figure 3b Schematic diagram of the histogram distribution of the blank area of the medium brightness type and non-gradient type; Figure 6b for Figure 3b Schematic diagram of the vertical projection distribution of the white space area with a medium brightness type of gradient type.
[0130] in Figure 5aand Figure 6a The horizontal axis of the histogram distribution in represents the grayscale level, each grayscale level has a corresponding grayscale interval, and the vertical axis represents the number of pixels or pixel columns falling into the grayscale interval; Figure 5a and Figure 6a It can be found that the grayscale levels of the pixels in each column of the blank area whose brightness type is non-gradient type are relatively concentrated, and the grayscale levels of the pixels in each column of the blank area whose brightness type is gradient type are relatively dispersed.
[0131] Figure 5b and Figure 6b The horizontal axis of the vertical projection distribution represents the pixel column position, that is, the pixels in the i-th column, and the vertical axis represents the grayscale of the pixels in the i-th column. Figure 5b and Figure 6b It can be found that the grayscale fluctuation of each column of pixels in the blank area with non-gradient brightness type is small and generally tends to be horizontal; the grayscale fluctuation of each column of pixels in the blank area with gradient brightness type is large, and generally shows a trend of first increasing and then decreasing.
[0132] After obtaining the histogram features and vertical projection features of the region, the histogram features and vertical projection features of the region may be fused to obtain fused features. Specifically, the features may be fused in series or by other methods.
[0133] Next, the fusion features are input into the pre-trained first classification network model to obtain the brightness type of the area output by the first classification network model. In practical applications, the first classification network model includes but is not limited to any one of a multilayer perceptron (MLP) network model and a convolutional neural network (CNN) with a specific layer.
[0134] The first classification network model uses the fusion features of the histogram features and vertical projection features of the sample image as training samples, and the brightness type of the sample image as training labels for training. For example, the label is represented by 1 and 0, 1 represents the brightness type as a gradient type, and 0 represents the brightness type as a non-gradient type. During the training process, the network weights are adjusted through the back propagation algorithm to minimize the classification error.
[0135] In some embodiments, when it is determined that there is a defect in the blank area in the photovoltaic glass image, before performing brightness feature extraction processing on the blank area, the method further includes:
[0136] Extract the image of the blank area;
[0137] Input the image of the blank area into the trained second classification network model to obtain the classification category of the blank area output by the second classification network model;
[0138] The second classification network model is a first image in which the brightness type of the blank area is a non-gradient type and contains defects, a second image in which the brightness type of the blank area is a gradient type and contains defects, a third image in which the brightness type of the blank area is a non-gradient type and does not contain defects, and a fourth image in which the brightness type of the blank area is a gradient type and does not contain defects as training samples, and is trained using the classification category of the blank area as training samples; the classification category is the presence of defects or the absence of defects.
[0139] The second classification model has been explained in the above-mentioned embodiment and will not be described in detail here.
[0140] like Figure 7 As shown, the present application embodiment provides a second flow chart of a defect detection method, comprising the following steps:
[0141] Step 701, obtaining an image of the blank area;
[0142] Step 702, determine whether there is a defect in the blank area; if yes, execute step 703; if no, end;
[0143] Step 703, determining whether the brightness type of the blank area is a gradient type; if so, executing step 704; if not, executing step 717;
[0144] Step 704, partitioning the blank area according to the grayscale of each column of pixels in the blank area and the target number to obtain the target number of sub-areas;
[0145] Step 705, determining the sub-region currently traversed from each sub-region of this iteration; each sub-region of the first iteration is obtained by partitioning the blank region;
[0146] Step 706, determine whether the sub-region currently traversed has defects; if so, execute step 707; if not, execute step 710;
[0147] Step 707, determine whether the brightness type of the sub-region currently traversed is a gradient type; if so, execute step 708; if not, execute step 709;
[0148] Step 708, partitioning the currently traversed sub-region according to the grayscale of each column of pixels in the currently traversed sub-region and the target number to obtain the target number of sub-regions; the target number of sub-regions belong to the sub-regions of the next round of iteration; executing step 711;
[0149] Step 709, determine that the sub-region currently traversed is the first target sub-region; execute step 711;
[0150] Step 710, determining that the sub-region currently traversed is the second target sub-region; executing step 711;
[0151] Step 711, determine whether all sub-areas of this iteration have been traversed, if so, execute step 712; if not, execute step 705;
[0152] Step 712, determine whether there are sub-regions for the next iteration; if yes, execute step 713; if no, execute step 714;
[0153] Step 713, updating each sub-region of this iteration to each sub-region of the next iteration; executing step 705;
[0154] Step 714, determine that the iteration stops; execute step 715;
[0155] Step 715, performing defect extraction processing on each first target sub-region by using a target algorithm to obtain a defect extraction result of each first target sub-region; executing step 716;
[0156] Step 716, obtaining the position information of the defects in the blank area according to the defect extraction results of each first target sub-area; end;
[0157] Step 717, perform defect extraction processing on the blank area through the target algorithm to obtain the position information of the defects in the blank area; end.
[0158] The detailed implementation process of the above steps 701 to 717 is shown in the above embodiments and will not be described again here.
[0159] The defect detection method provided in the embodiment of the present application can be executed by a defect detection device. In the embodiment of the present application, the defect detection device provided in the embodiment of the present application is described by taking the defect detection method executed by the defect detection device as an example.
[0160] The embodiment of the present application also provides a defect detection device.
[0161] like Figure 8 As shown, the defect detection device includes: a first processing module 810, a second processing module 820, a third processing module 830 and a fourth processing module 840.
[0162] The first processing module 810 is used to extract brightness features of the blank area in the photovoltaic glass image to obtain the brightness type of the blank area when it is determined that there is a defect in the blank area in the photovoltaic glass image; the brightness type includes a gradient type and a non-gradient type;
[0163] The second processing module 820 is used to partition the blank area at least once to obtain a plurality of target sub-areas when it is determined that the brightness type of the blank area is a gradient type, and the brightness type of the target sub-areas is a non-gradient type;
[0164] The third processing module 830 is used to perform defect extraction processing on each target sub-region through a target algorithm to obtain a defect extraction result of each target sub-region; the target algorithm includes at least one of a binarization algorithm and a connected area detection algorithm;
[0165] The fourth processing module 840 is used to obtain the position information of the defects in the blank area according to the defect extraction results of each target sub-area.
[0166] According to the defect detection device provided in the embodiment of the present application, by partitioning the blank area at least once when determining that the brightness type of the blank area is a gradient type, a plurality of target sub-areas are obtained, and the brightness type of the target sub-areas is a non-gradient type; the blank area with a gradient brightness type is divided into a plurality of target sub-areas with a non-gradient brightness type, and defect extraction processing can be performed on each target sub-area through a target algorithm to obtain a defect extraction result for each target sub-area; based on the defect extraction result of each target sub-area, the location information of the defects in the blank area is obtained, so as to accurately and comprehensively locate the location information of the defects in the blank area with a gradient brightness type.
[0167] In some embodiments, the target sub-region includes a first target sub-region and a second target sub-region;
[0168] The second processing module 820 includes:
[0169] The first processing submodule is used to traverse each sub-region of this iteration; each sub-region of the first iteration is obtained by partitioning the blank region;
[0170] The second processing submodule is used to perform brightness feature extraction processing on the subregion currently traversed to obtain the brightness type of the subregion currently traversed when it is determined that the subregion currently traversed has defects;
[0171] A third processing submodule is used to determine that the currently traversed subregion is a first target subregion when it is determined that the brightness type of the currently traversed subregion is a non-gradient type;
[0172] The fourth processing submodule is used to determine that the subregion currently traversed is a second target subregion when it is determined that the subregion currently traversed has no defects.
[0173] In some embodiments, the second processing module 820 further includes:
[0174] A fifth processing submodule is used to partition the currently traversed subregion according to the grayscale of each column of pixels in the currently traversed subregion and the target number to obtain the target number of subregions when it is determined that the brightness type of the currently traversed subregion is a gradient type;
[0175] The target number of sub-regions belong to the sub-regions of the next iteration.
[0176] In some embodiments, the third processing module is specifically used to perform defect extraction processing on each first target sub-region through a target algorithm to obtain a defect extraction result of each first target sub-region;
[0177] The fourth processing module is specifically configured to obtain location information of defects in the blank area according to the defect extraction results of each first target sub-area.
[0178] In some embodiments, for any area in the blank area or the sub-area, a brightness feature extraction process is performed on the area to obtain a brightness type of the area, including:
[0179] Performing histogram extraction processing and vertical projection extraction processing on the region to obtain the histogram features and vertical projection features of the region;
[0180] Perform feature fusion on the histogram features and vertical projection features of the region to obtain fused features;
[0181] Input the fused features into the pre-trained first classification network model to obtain the brightness type of the area output by the first classification network model;
[0182] The first classification network model uses the fusion features of the histogram features and the vertical projection features of the sample image as training samples, and the brightness type of the sample image as training labels.
[0183] In some embodiments, the defect detection apparatus further comprises:
[0184] The fifth processing module is used for:
[0185] Extract the image of the blank area;
[0186] Input the image of the blank area into the trained second classification network model to obtain the classification category of the blank area output by the second classification network model;
[0187] The second classification network model is a first image in which the brightness type of the blank area is a non-gradient type and contains defects, a second image in which the brightness type of the blank area is a gradient type and contains defects, a third image in which the brightness type of the blank area is a non-gradient type and does not contain defects, and a fourth image in which the brightness type of the blank area is a gradient type and does not contain defects as training samples, and is trained using the classification category of the blank area as training samples; the classification category is the presence of defects or the absence of defects.
[0188] In some embodiments, the defect detection apparatus further comprises:
[0189] The sixth processing module is used to perform defect extraction processing on the blank area through a target algorithm to obtain position information of defects in the blank area when it is determined that the brightness type of the blank area is a non-gradient type.
[0190] The defect detection device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0191] The defect detection device in the embodiment of the present application may be a device having an operating system. The operating system may be a Microsoft (Windows) operating system, an Android (Android) operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0192] The defect detection device provided in the embodiment of the present application can achieve Figures 1 to 7 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0193] In some embodiments, Fig. 9As shown, an embodiment of the present application also provides an electronic device 900, including a processor 901, a memory 902, and a computer program stored in the memory 902 and executable on the processor 901. When the program is executed by the processor 901, each process of the above-mentioned defect detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0194] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0195] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned defect detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0196] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0197] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned defect detection method when executed by a processor.
[0198] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0199] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned defect detection method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0200] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0201] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0202] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0203] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
[0204] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0205] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. A defect detection method, characterized in that: include: When it is determined that there is a defect in the blank area in the photovoltaic glass image, a brightness feature extraction process is performed on the blank area to obtain a brightness type of the blank area; the brightness type includes a gradient type and a non-gradient type; When it is determined that the brightness type of the blank area is a gradient type, partitioning the blank area at least once to obtain a plurality of target sub-areas, wherein the brightness type of the target sub-areas is a non-gradient type; Perform defect extraction processing on each target sub-region through a target algorithm to obtain a defect extraction result of each target sub-region; the target algorithm includes at least one of a binarization algorithm and a connected area detection algorithm; According to the defect extraction results of each target sub-region, the position information of the defects in the blank region is obtained.
2. The defect detection method according to claim 1, characterized in that: The target sub-region includes a first target sub-region and a second target sub-region; The blank area is partitioned at least once to obtain a plurality of target sub-areas, including: Traverse each sub-region of this iteration; each sub-region of the first iteration is obtained by partitioning the blank region; In the case where it is determined that the sub-region currently traversed has a defect, performing brightness feature extraction processing on the sub-region currently traversed to obtain a brightness type of the sub-region currently traversed; In the case where it is determined that the brightness type of the currently traversed sub-region is a non-gradient type, determining the currently traversed sub-region as a first target sub-region; When it is determined that the currently traversed sub-region has no defects, the currently traversed sub-region is determined to be a second target sub-region.
3. The defect detection method according to claim 2, characterized in that: After obtaining the brightness type of the currently traversed sub-area, the method further includes: When it is determined that the brightness type of the sub-region currently traversed is a gradient type, partitioning the sub-region currently traversed according to the grayscale of each column of pixels in the sub-region currently traversed and the target number, to obtain the target number of sub-regions; The target number of sub-regions belong to the sub-regions of the next iteration.
4. The defect detection method according to claim 2, characterized in that: The defect extraction process is performed on each target sub-region by using a target algorithm to obtain a defect extraction result of each target sub-region, including: Performing defect extraction processing on each of the first target sub-regions by using a target algorithm to obtain a defect extraction result of each of the first target sub-regions; The step of obtaining the position information of the defects in the blank area according to the defect extraction results of each target sub-area includes: According to the defect extraction results of each first target sub-region, the position information of the defects in the blank region is obtained.
5. The defect detection method according to claim 2, characterized in that: For any one of the blank area or the sub-area, performing brightness feature extraction processing on the area to obtain the brightness type of the area includes: Performing histogram extraction processing and vertical projection extraction processing on the region to obtain histogram features and vertical projection features of the region; Performing feature fusion on the histogram feature of the region and the vertical projection feature to obtain a fused feature; Inputting the fused features into a pre-trained first classification network model to obtain the brightness type of the area output by the first classification network model; The first classification network model uses the histogram features of the sample image and the fusion features of the vertical projection features as training samples, and uses the brightness type of the sample image as a training label.
6. The defect detection method according to claim 1, characterized in that: In the case where it is determined that there are defects in the blank area in the photovoltaic glass image, before performing brightness feature extraction processing on the blank area, the method further includes: Extracting an image of the blank area; Inputting the image of the blank area into a trained second classification network model to obtain a classification category of the blank area output by the second classification network model; The second classification network model is a first image in which the brightness type of the blank area is a non-gradient type and contains defects, a second image in which the brightness type of the blank area is a gradient type and contains defects, a third image in which the brightness type of the blank area is a non-gradient type and does not contain defects, and a fourth image in which the brightness type of the blank area is a gradient type and does not contain defects as training samples, and is trained using the classification category of the blank area as training samples; the classification category is the presence of defects or the absence of defects.
7. The defect detection method according to any one of claims 1 to 6, characterized in that: After obtaining the brightness type of the blank area, the method includes: When it is determined that the brightness type of the blank area is a non-gradient type, defect extraction processing is performed on the blank area using the target algorithm to obtain position information of defects in the blank area.
8. A defect detection device, characterized in that: include: A first processing module is used for, when it is determined that there is a defect in the blank area in the photovoltaic glass image, performing brightness feature extraction processing on the blank area to obtain the brightness type of the blank area; the brightness type includes a gradient type and a non-gradient type; A second processing module is used for partitioning the blank area at least once to obtain a plurality of target sub-areas when it is determined that the brightness type of the blank area is a gradient type, and the brightness type of the target sub-areas is a non-gradient type; A third processing module is used to perform defect extraction processing on each target sub-region through a target algorithm to obtain a defect extraction result of each target sub-region; the target algorithm includes at least one of a binarization algorithm and a connected area detection algorithm; The fourth processing module is used to obtain the position information of the defects in the blank area according to the defect extraction results of each target sub-area.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the defect detection method according to any one of claims 1 to 7 is implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the defect detection method according to any one of claims 1 to 7 is implemented.