A method and system for visually detecting surface defects of optoelectronic devices
By analyzing the grayscale image of the photovoltaic panel surface, combining local and overall grayscale characteristics, edge roughness and grayscale change parameters, the defect area is gradually corrected and expanded, and the problem of inaccurate identification of photovoltaic panel surface defects in the prior art is solved. Improves the accuracy and credibility of the detection.
Patent Information
- Application Number
- CN202411002115.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-25
AI Technical Summary
Existing image segmentation technology is difficult to accurately identify photovoltaic panel surface defects, especially because the defect area is similar to the grayscale of the reflective area of the metal line, resulting in inaccurate identification results.
By obtaining the grayscale image of the photovoltaic panel surface, segmenting the initial defect area, analyzing the local grayscale differences and overall grayscale characteristics of each pixel point, filtering out the suspected defect area, and correcting and expanding according to the edge roughness and grayscale change parameters. , and finally obtain the accurate defect area on the surface of the photovoltaic panel.
It improves the accuracy and credibility of photovoltaic panel surface defect detection, eliminates interference from pseudo-defect areas, and ensures the reliability of the detection results.
Smart Images

Figure CN118521591B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image segmentation processing, and in particular to a method and system for visually detecting surface defects of optoelectronic devices. Background Art
[0002] Photovoltaic panel surface defects refer to scratches, stains and cracks on the surface of photovoltaic panels, which may have a negative impact on sunlight absorption and electron transmission, thereby reducing the power generation efficiency of photovoltaic panels. In actual application, it is found that stains and cracks on the surface of photovoltaic panels are relatively common. If they are not maintained and handled in time, the power generation efficiency of photovoltaic panels may be reduced, greatly affecting the power generation income of photovoltaic panels.
[0003] In the process of segmenting and obtaining surface defects of photovoltaic panels through image segmentation technology, there is a disadvantage that the grayscale of the defect area and the reflective area of the metal wire are similar and difficult to distinguish. In addition, unreasonable setting of the segmentation threshold can easily cause the pixels in the edge area of the defect area to be identified as normal pixels, resulting in inaccurate surface defect recognition and detection results. Summary of the invention
[0004] In order to solve the technical problem that the existing image segmentation technology is inaccurate in identifying and detecting surface defects of optoelectronic devices, the purpose of the present invention is to provide a method and system for visually detecting surface defects of optoelectronic devices. The technical solutions adopted are as follows:
[0005] A method for visually detecting surface defects of an optoelectronic device, the method comprising:
[0006] Acquire a grayscale image of the surface of the optoelectronic device; segment the grayscale image to obtain an initial defect area;
[0007] According to the local grayscale difference of each pixel point in the initial defect area, combined with the difference of the overall grayscale characteristics of different initial defect areas, the first defect possibility of each initial defect area is obtained; and the suspected defect area is screened out based on the first defect possibility;
[0008] According to the edge roughness of the suspected defect area, obtaining a second defect possibility of each of the suspected defect areas; and screening out defect areas to be analyzed based on the second defect possibility;
[0009] According to the grayscale change characteristics of the pixels on the line connecting the center pixel and the edge pixel of the defect area to be analyzed, combined with the grayscale change difference between adjacent edge pixel points, the grayscale change parameter of each edge pixel of the defect area to be analyzed is obtained;
[0010] Along the direction of the line connecting the center pixel point and the edge pixel point of the defect area to be analyzed, pixel points are gradually added to form a corrected defect area; according to the difference in the second defect possibility between the corrected defect area and the corresponding defect area to be analyzed, combined with the difference between the grayscale change parameters, a third defect possibility is obtained; based on the third defect possibility, the defect area to be analyzed is expanded to obtain a final defect area on the surface of the optoelectronic device.
[0011] Furthermore, the method for obtaining the first defect possibility includes:
[0012] In each of the initial defect areas, a preset neighborhood window is established with each pixel point as the center; according to the gray value difference between all the window center pixels and the corresponding window non-center pixels in each of the initial defect areas, the first initial defect parameter of each of the initial defect areas is obtained; wherein the pixel point at the center of the preset neighborhood window is called the window center pixel point, and the other pixels are called the window non-center pixels; the gray value difference between the window center pixel point and the corresponding window non-center pixel point is positively correlated with the first initial defect parameter;
[0013] Acquire a second initial defect parameter of each initial defect area according to a fluctuation characteristic of the grayscale value of a pixel point in each initial defect area;
[0014] The first defect possibility is obtained according to the first initial defect parameter and the second initial defect parameter of each initial defect area; the first initial defect parameter and the second initial defect parameter are both positively correlated with the first defect possibility.
[0015] Furthermore, the method for obtaining the second defect possibility includes:
[0016] The distance from each edge pixel point to the center pixel point of each suspected defect area is obtained as the boundary distance, and the possibility of the second defect is obtained according to the degree of fluctuation of the boundary distance of each suspected defect area; the degree of fluctuation of the boundary distance is positively correlated with the possibility of the second defect.
[0017] Furthermore, the method for obtaining the grayscale change parameter includes:
[0018] On the line connecting the center pixel point and the edge pixel point of the defect area to be analyzed, a grayscale change sub-parameter is obtained according to the fluctuation degree of the grayscale value change between adjacent pixel points; the fluctuation degree of the grayscale value change between adjacent pixel points is positively correlated with the grayscale change sub-parameter;
[0019] The grayscale change parameter of each edge pixel is obtained according to the difference between the grayscale change sub-parameter of each edge pixel and the adjacent edge pixel; the difference between the grayscale change sub-parameter of the edge pixel and the adjacent edge pixel is positively correlated with the grayscale change parameter.
[0020] Furthermore, the method for obtaining the corrected defect area includes:
[0021] The line connecting the center pixel point and the edge pixel point of the defect area to be analyzed is selected in sequence as the target line, and an adjacent pixel point is selected on the straight line of the target line away from the center pixel point of the area to extend the target line to obtain the corrected defect area; wherein the corrected defect area is updated once each pixel point is added.
[0022] Furthermore, each time the corrected defect area is updated, the third defect possibility is also updated simultaneously; the difference in the second defect possibility between the corrected defect area and the corresponding defect area to be analyzed is positively correlated with the third defect possibility; the difference between the grayscale change parameters is negatively correlated with the third defect possibility; and the third defect possibility is also normalized.
[0023] Furthermore, the method for obtaining the final defective area includes:
[0024] When the third defect possibility is less than or equal to the third preset threshold, remove the most recently selected pixel point and exclude the corresponding target line; when all the target lines are excluded, take the last updated corrected defect area of each defect area to be analyzed as the final defect area of the defect area to be analyzed.
[0025] Furthermore, the method for obtaining the suspected defect area includes:
[0026] The first defect possibility is also normalized; when the first defect possibility is greater than a first preset threshold, the corresponding initial defect area is marked as the suspected defect area.
[0027] Furthermore, the method for obtaining the defect area to be analyzed includes:
[0028] The second defect possibility is further normalized; when the second defect possibility is greater than a second preset threshold, the corresponding suspected defect area is marked as the defect area to be analyzed.
[0029] The present invention also proposes a system for visual inspection of surface defects of optoelectronic devices, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the steps of a method for visual inspection of surface defects of optoelectronic devices when executing the computer program.
[0030] The present invention has the following beneficial effects:
[0031] The present invention first obtains a grayscale image of the surface of an optoelectronic device and performs segmentation to obtain an initial defect area, thereby providing a basis for subsequent analysis and determining an analysis object; further analysis is performed from the perspective of grayscale differences in the initial defect area to obtain a first defect possibility for each initial defect area, and suspected defect areas are screened out based on the first defect possibility, eliminating interference from pseudo-defect areas formed by reflective areas with strong grayscale consistency; further analysis is performed from the perspective of edge roughness of the suspected defect area to obtain a second defect possibility for each suspected defect area, and defect areas to be analyzed are screened out based on the second defect possibility, eliminating interference from pseudo-defect areas formed by reflective areas with smooth edges, supplementing the screening based on the first defect possibility to determine a more accurate object to be analyzed; and further grayscale change parameters of each edge pixel point of the defect area to be analyzed are obtained to characterize the area where each edge pixel point is located. The grayscale change characteristics on the line between the central pixel and the edge pixel are used to prepare for the subsequent expansion operation; further, along the direction of the line between the central pixel and the edge pixel of the defect area to be analyzed, pixel points are gradually added to form a corrected defect area, and the pixel points of the defect area that may be misjudged as normal pixels due to threshold segmentation are supplemented, so as to finally obtain a more accurate defect area; further, according to the difference in the second defect possibility between the corrected defect area and the corresponding defect area to be analyzed, combined with the difference between the grayscale change parameters, the third defect possibility is obtained, and an evaluation basis is provided for each expansion operation, so as to facilitate the control of the expansion operation; finally, based on the third defect possibility, the defect area to be analyzed is expanded to obtain the final defect area on the surface of the optoelectronic device; wherein, the final defect area is also processed by a closed operation to make the final defect area smoother, and the expansion method is supplemented at the same time. The present invention eliminates the interference of the pseudo-defect area formed by the reflective area by analyzing the grayscale change characteristics and edge roughness of the initial defect area obtained by threshold segmentation, and finally obtains an accurate defect area by adaptively expanding the defect area to be analyzed, thereby improving the credibility of the visual detection results. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0033] Figure 1 A flow chart of a method for visually detecting surface defects of an optoelectronic device provided by one embodiment of the present invention;
[0034] Figure 2 A grayscale image of a photovoltaic panel surface provided by an embodiment of the present invention;
[0035] Figure 3 A binary image of a photovoltaic panel surface provided by an embodiment of the present invention;
[0036] Figure 4 A flowchart of a method for obtaining a first defect possibility provided by an embodiment of the present invention;
[0037] Figure 5 A binary image of a photovoltaic panel surface including a suspected defect area provided by an embodiment of the present invention;
[0038] Figure 6 A binary image of a photovoltaic panel surface including a defect area to be analyzed is provided by one embodiment of the present invention;
[0039] Figure 7 A defect marking schematic diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0040] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a method and system for visual inspection of surface defects of optoelectronic devices proposed by the present invention, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0041] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0042] The specific scheme of the method and system for visual inspection of surface defects of optoelectronic devices provided by the present invention is described in detail below with reference to the accompanying drawings.
[0043] See also Figure 1 , which shows a flow chart of a method for visually detecting surface defects of an optoelectronic device provided by an embodiment of the present invention, specifically comprising:
[0044] Step S1: Acquire a grayscale image of the surface of the optoelectronic device; segment the grayscale image to obtain an initial defect area.
[0045] In an embodiment of the present invention, a grayscale image of the surface of an optoelectronic device is first acquired to provide a basis for analysis; then the grayscale image is preliminarily segmented to obtain an initial defect area, obtain an area where defects may exist, and narrow the scope of the analysis object, in order to prepare for further analysis of the initial defect area, eliminate interference from non-defective areas, and obtain the final defect area.
[0046] As an example: take an image of the board surface with a camera and convert the image into a grayscale image; see Figure 2 , which shows a grayscale image of a photovoltaic panel surface provided by an embodiment of the present invention; using the OTSU algorithm to Figure 2 Perform threshold segmentation to obtain a binary image, see Figure 3 , which shows a binary image of a photovoltaic panel surface provided by an embodiment of the present invention; Figure 3 In , the area formed by the foreground pixels is the initial defect area.
[0047] It should be noted that when the grayscale change characteristics are subsequently analyzed, the analysis is performed in the grayscale image, and the binary image is used to determine the defect area and to facilitate the analysis of the morphological characteristics of the defect area. The OTSU algorithm and the acquisition of binary images are technical means well known to those skilled in the art. In other embodiments of the present invention, the implementer may also use other threshold segmentation methods for threshold segmentation, or other segmentation methods such as superpixel segmentation to obtain the initial defect area.
[0048] Step S2: According to the local grayscale difference of each pixel in the initial defect area, combined with the difference of the overall grayscale characteristics in different initial defect areas, the first defect possibility of each initial defect area is obtained; and the suspected defect area is screened out based on the first defect possibility.
[0049] Taking into account that the defective parts of the photovoltaic panel reflect light in different directions, resulting in uneven grayscale distribution inside the defective area, the first defect possibility of each initial defect area is obtained from the grayscale distribution angle of the pixels in the initial defect area, according to the local grayscale difference of each pixel in the initial defect area and the difference in overall grayscale characteristics in different initial defect areas, and the grayscale difference characteristics of the initial defect area are measured to facilitate subsequent screening to remove interference from normal areas with uniform grayscale distribution.
[0050] Preferably, in one embodiment of the present invention, the method for obtaining the first defect possibility includes:
[0051] See also Figure 4 , which shows a flow chart of a method for obtaining a first defect possibility provided by an embodiment of the present invention, specifically comprising:
[0052] Step S401: In each initial defect area, a preset neighborhood window is established with each pixel as the center; and the first initial defect parameter of each initial defect area is obtained according to the gray value difference between all the window center pixels and the corresponding window non-center pixels in each initial defect area.
[0053] Among them, the pixel point at the center of the preset neighborhood window is called the window center pixel point, and the other pixel points are called the window non-center pixel points; the grayscale value difference between the window center pixel point and the corresponding window non-center pixel point is positively correlated with the first initial defect parameter.
[0054] Considering that establishing a preset neighborhood window is more conducive to analyzing the local grayscale distribution characteristics of pixel points, it can more accurately analyze the grayscale unevenness characteristics of the initial defect area. Therefore, a preset neighborhood window is established with each pixel point as the center. In the preset neighborhood window, the grayscale value difference between the non-central pixel point of the window and the central pixel point of the window is analyzed. The larger the grayscale value difference, the more uneven the grayscale distribution in the local preset neighborhood window. By combining the grayscale value difference characteristics of all pixels in the initial defect area in their preset neighborhood windows, the first initial defect parameter characterizing the entire initial defect area can be obtained.
[0055] As an example, the size of the preset neighborhood window is , in each initial defect area, the maximum absolute value of the gray value difference between all window center pixels and the corresponding window non-center pixels is averaged as the first initial defect parameter. The calculation formula is: ;in, is the serial number of the initial defect area; Indicates A first initial defect parameter of an initial defect region; Indicates The number of pixels in the initial defect area; Indicates the sequence number of the pixel points in the initial defect area; Indicates In the initial defect area, The maximum absolute value of the grayscale value difference between the center pixel of the window and the non-center pixel of the window in the preset neighborhood window with a pixel as the center. In the formula, considering that the maximum difference between the center pixel of the window and the non-center pixel of the window in each preset neighborhood window can better highlight the uneven characteristics of the grayscale, the first initial defect parameter is obtained by the maximum absolute value of the grayscale value difference between the center pixel of the window and the corresponding non-center pixel of the window; The larger it is, the more uneven the grayscale distribution in the local preset neighborhood window is, the more likely it is a defective area, and the larger the first initial defect parameter is.
[0056] As another example, the size of the preset neighborhood window is , in each initial defect area, the average of the absolute values of the grayscale value differences between all window center pixels and the corresponding window non-center pixels is taken as the first initial defect parameter.
[0057] In other embodiments of the present invention, the implementer may also use other methods such as The preset neighborhood window size is not described here.
[0058] Step S402: acquiring a second initial defect parameter of each initial defect region according to the fluctuation characteristics of the grayscale values of the pixels in each initial defect region.
[0059] As an example, considering that when the grayscale distribution of the defect area is uneven, the fluctuation range of the grayscale value of the pixel point is large, and the larger the fluctuation range, the larger the grayscale value variation range of the pixel point in the defect area, the more uneven the grayscale distribution, and the more obvious the defect characteristics, so according to the fluctuation range of the grayscale value of the pixel point in each initial defect area, the second initial defect parameter of each initial defect area is obtained; the fluctuation range of the grayscale value of the pixel point is positively correlated with the second initial defect parameter. Therefore, in one embodiment of the present invention, the extreme difference of the grayscale value of the pixel point in each initial defect area is used as the second initial defect parameter.
[0060] As another example, considering that the variance can reflect the discrete characteristics of data fluctuations, the larger the variance, the more discrete the data fluctuations and the more uneven the distribution, so the variance of the grayscale values of pixels in the initial defect area is obtained as the second initial defect parameter.
[0061] Step S403: obtaining a first defect possibility according to a first initial defect parameter and a second initial defect parameter of each initial defect region; the first initial defect parameter and the second initial defect parameter are both positively correlated with the first defect possibility.
[0062] The first initial defect parameter analyzes the grayscale distribution characteristics of the initial defect area from the perspective of local grayscale distribution, and the second initial defect parameter analyzes the grayscale distribution characteristics of the initial defect area from the perspective of overall grayscale distribution. Therefore, the two analysis perspectives of local and overall are further integrated to obtain a more representative first defect possibility.
[0063] As an example, the calculation formula for the first defect probability includes: ;in, is the serial number of the initial defect area; Indicates The first defect probability of the initial defect area; Indicates A first initial defect parameter of an initial defect region; represents the standard normalization function; represents the average value of the first initial defect parameters of all initial defect regions; Indicates a second initial defect parameter of an initial defect region; represents the maximum value of the second initial defect parameter of all initial defect regions; where, The larger the value is, the larger the first initial defect parameter of the current initial defect area is compared with the first initial defect parameter of all initial defect areas, reflecting that the grayscale difference of the current defect area is greater, the grayscale distribution is more uneven, the possibility of it being a defect area is higher, and the possibility of the first defect is greater; at the same time, in order to prevent Negative, impact The correlation with the first defect probability is Perform normalization processing; The larger the value, the more discrete the grayscale distribution of the pixels in the current initial defect area is, the more uneven the grayscale distribution is, and the greater the possibility of the first defect is. Keeping the same order of magnitude, also using right Perform normalization.
[0064] As another example, the implementer may directly use the product of the first initial defect parameter and the second initial defect parameter as the first defect probability.
[0065] The present invention also provides an embodiment: considering that the grayscale value inside the normal reflective area is uniform and the grayscale value is large, the proportion of pixels with smaller grayscale values inside the normal reflective area is smaller, while the proportion of pixels with smaller grayscale values inside the defective area is larger, so according to the proportion of pixels with grayscale values less than the average grayscale value in each initial defective area, the third initial defect parameter is obtained, and the first initial defect parameter, the second initial defect parameter and the third initial defect parameter are integrated to obtain the first defect possibility, wherein the third initial parameter is positively correlated with the first defect possibility. As an example, the ratio of the number of pixels with grayscale values less than the average grayscale value in the initial defective area to the number of all pixels in the initial defective area is used as the third initial parameter, and the product of the first initial defect parameter, the second initial defect parameter and the third initial defect parameter is used as the first defect possibility.
[0066] The first defect possibility measures the grayscale difference characteristics within the initial defect area. The larger the first defect possibility, the more likely it is a defective area. Therefore, based on the first defect possibility, suspected defective areas are screened out to exclude interference from normal areas.
[0067] Preferably, in one embodiment of the present invention, the first defect possibility is further normalized; when the first defect possibility is greater than a first preset threshold, the corresponding initial defect area is marked as a suspected defect area.
[0068] As an example, the first preset threshold is 0.5. In other embodiments of the present invention, implementers can adjust it by themselves.
[0069] See also Figure 5 , which shows a binary image of a photovoltaic panel surface including a suspected defect area provided by an embodiment of the present invention.
[0070] Step S3: obtaining a second defect possibility of each suspected defect area according to the edge roughness of the suspected defect area; and selecting the defect area to be analyzed based on the second defect possibility.
[0071] In the embodiment of the present invention, it is also taken into consideration that the defective area is damaged randomly due to the cause. For example, if the photovoltaic panel is damaged, the strength and direction of the external force attack are random, so the edge of the defective area is rougher, while the light plate in the reflective area is relatively regular in shape of a circle or an ellipse with a smoother edge. Therefore, the possibility of a second defect in each suspected defective area is obtained by further measuring the edge roughness of the suspected defective area for further screening.
[0072] Preferably, in one embodiment of the present invention, considering that the greater the difference in distance from the center point to the edge point of the defect area and the more discrete the distance, the less smooth the edge of the suspected defect area and the greater the edge roughness, the more likely it is a defect area and the greater the possibility of the second defect; based on this, the distance from each edge pixel point to the center pixel point of each suspected defect area is obtained as the boundary distance, and the possibility of the second defect is obtained according to the degree of fluctuation of the boundary distance of each suspected defect area; the degree of fluctuation of the boundary distance is positively correlated with the possibility of the second defect.
[0073] As an example, in any suspected defect area, the second defect possibility is obtained based on the variance and range of the boundary distance, combined with the difference between each boundary distance and the average boundary distance, and the difference between the average boundary distance and the average boundary distance of all suspected defect areas. The calculation formula of the second defect possibility includes:
[0074] ;in, is the serial number of the suspected defect area; Indicates The second defect possibility of the suspected defect area; Indicates The maximum value of the boundary distance of the suspected defect area; Indicates The minimum value of the boundary distance of the suspected defect area; Shidi The average value of the boundary distances of the suspected defect areas; It represents the average value of the boundary distance of all suspected defect areas; Indicates The number of edge pixels in the suspected defect area; Indicates the serial number of edge pixels; Indicates The suspected defect area The boundary distance corresponding to the edge pixels; Indicates The variance of all boundary distances of the suspected defect area; represents the standard normalization function; Indicates the zero division parameter to prevent the denominator from being 0. .
[0075] In the calculation formula of the second defect probability, The larger the value is, the more discrete the boundary distance from the center pixel to the edge pixel of the area is, and the more drastic the fluctuation is, which reflects that the edge of the suspected defect area is less smooth and rougher, and the more likely it is a defect area, and the greater the possibility of the second defect is; similarly, The larger it is, the less concentrated the boundary distance is. The larger it is, the larger the fluctuation range of the boundary distance is. It also means that the more discrete the boundary distance is, the greater the fluctuation is, and the greater the possibility of the second defect is. The larger the value is, the larger the overall boundary distance of the current suspected defect area is compared with the overall boundary distance of all suspected defect areas. According to prior experience, the reflective area is generally larger than the defect area, and the average boundary distance from the center point to the edge point of the defect area is smaller than that of the reflective area. At the same time, the denominator is prevented from being negative and affecting the correlation of other parameters. Therefore, Normalize it, The smaller it is, the more likely it is a defective area, and the greater the possibility of the second defect. In one embodiment of the present invention, .
[0076] The calculation formula for the second defect possibility integrates the variance and range, as well as the difference between each boundary distance and the average boundary distance, measures the severity of the fluctuation of the boundary distance from multiple angles, and obtains a more convincing second defect possibility; in other embodiments of the present invention, the implementer may also select any one or more analysis angles of the variance and range, and the difference between each boundary distance and the average boundary distance, to measure the severity of the fluctuation of the boundary distance and obtain the second defect possibility.
[0077] It should be noted that, in one embodiment of the present invention, the center of the suspected defect area is determined by the centroid formula. The implementer can also determine the center pixel of the area by establishing a coordinate system and selecting the pixel point corresponding to the average coordinate value of the suspected defect area; or select a pixel point in the defect area, iteratively obtain the minimum sum of the vectors connecting the selected pixel point and all edge pixels, and determine the center pixel point of the area; the edge pixel point is the outermost circle of pixels in the area, which will not be described in detail here.
[0078] It should be noted that the center pixel of the defect area must be inside the defect area. If the center pixel of the region determined by the centroid formula is not inside the defect area, other methods must be used to determine the center pixel of the region.
[0079] It should be noted that, in another embodiment of the present invention, the implementer can also analyze the edge roughness of the suspected defect area from the stability of the angle change on the edge line of the suspected defect area; as an example, select any suspected edge area, obtain the angle of the line between adjacent edge pixels, take the absolute value of the difference between the angles of the adjacent lines as the angle change value, and take the variance of the angle change value as the second defect possibility; for example, there is currently a sequence number of For three adjacent edge pixels, the connecting angle between edge pixel points numbered 30 and 31 is 15 degrees, and the connecting angle between edge pixel points numbered 31 and 32 is 10 degrees, so an angle change value of 5 degrees can be obtained.
[0080] After analyzing the possibility of the second defect of the suspected defect area from the perspective of the edge roughness of the suspected defect area, the suspected defect area can be further screened to obtain a more accurate defect area to be analyzed.
[0081] Preferably, in one embodiment of the present invention, the second defect possibility is further normalized; when the second defect possibility is greater than a second preset threshold, the corresponding suspected defect area is marked as a defect area to be analyzed.
[0082] As an example: the second preset threshold is 0.5. In other embodiments of the present invention, implementers can adjust it by themselves.
[0083] See also Figure 6 , which shows a binary image of a photovoltaic panel surface including a defect area to be analyzed provided by an embodiment of the present invention.
[0084] Step S4: According to the grayscale change characteristics of the pixels on the line connecting the center pixel and the edge pixel of the defect area to be analyzed, combined with the corresponding grayscale change differences between adjacent edge pixels, the grayscale change parameters of each edge pixel of the defect area to be analyzed are obtained; the final defect area is also processed by a closed operation.
[0085] In an embodiment of the present invention, considering that the segmentation of an image by threshold segmentation may cause some pixels on the edge of a defect to be excluded from the defect area to be analyzed, and the edge shape of the defect area to be analyzed is often similar to the edge shape of the actual defect area, when the defect area to be analyzed is subsequently expanded, pixels are gradually added along the direction of the line connecting the center pixel of the region and the edge pixel of the defect area to be analyzed to form a corrected defect area; but it is necessary to control the expansion process so that the expansion operation is terminated in time. Since the expansion is performed along the direction of the line connecting the center pixel of the region and the edge pixel of the defect area to be analyzed, the grayscale change characteristics of the pixels on the line connecting the center pixel of the region and the edge pixel are analyzed, and the grayscale change parameters of each edge pixel of the defect area to be analyzed are obtained in combination with the corresponding grayscale change differences between adjacent edge pixels. Based on this, the newly added pixels should not change the grayscale change characteristics before the expansion, so as to facilitate the subsequent control of the expansion operation.
[0086] Preferably, in one embodiment of the present invention, considering that the fluctuation degree of grayscale value change between adjacent pixels on the line connecting the regional center pixel point and the edge pixel point of the defect area to be analyzed reflects the grayscale change characteristics on this line, a grayscale change sub-parameter is obtained according to the fluctuation degree of grayscale value change between adjacent pixels on the line connecting the regional center pixel point and the edge pixel point of the defect area to be analyzed, so as to characterize the grayscale change characteristics on the line; the fluctuation degree of grayscale value change between adjacent pixels is positively correlated with the grayscale change sub-parameter;
[0087] As an example, select any edge pixel point of any defect area to be analyzed, obtain the line connecting the center pixel point of the area and the edge pixel point, and on the line, obtain the absolute value of the grayscale value difference between each pixel point and the adjacent pixel point in the direction away from the center pixel point of the area as the local change parameter of each pixel point, and analyze the fluctuation degree of the grayscale value change between adjacent pixels through the absolute value of the difference of the local change parameters of all adjacent pixels, and take the sum of the absolute value of the difference of the local change parameters of all adjacent pixels as the grayscale change sub-parameter.
[0088] In other embodiments of the present invention, the implementer may also obtain other parameters that can represent discrete features, such as the variance of the local change parameter, measure the fluctuation degree of the grayscale value change between adjacent pixels, and obtain the grayscale change sub-parameter, which will not be elaborated here.
[0089] Taking into account the large degree of texture change in the defect area, that is, the grayscale change sub-parameters corresponding to adjacent edge pixels have certain differences, the grayscale change parameters of each edge pixel are obtained according to the difference between the grayscale change sub-parameters of each edge pixel and the adjacent edge pixel, so as to characterize the grayscale change characteristics corresponding to each edge pixel and provide a basis for subsequent expansion operations; the difference between the grayscale change sub-parameters of the edge pixel and the adjacent edge pixel is positively correlated with the grayscale change parameter.
[0090] As an example: the absolute values of the differences between the grayscale change sub-parameter corresponding to the current edge pixel and the grayscale change sub-parameters of two edge pixels with adjacent serial numbers are calculated respectively, and the sum of the two absolute values of the differences is taken as the grayscale change parameter.
[0091] It should be noted that, since the edge pixels of the defect area form a closed figure, each edge pixel has two adjacent edge pixels, and the edge pixel with the largest serial number is adjacent to the edge pixel with the smallest serial number, which is similar to 360 degrees being adjacent to 1 degree.
[0092] Step S5: gradually add pixels along the direction of the line connecting the center pixel point and the edge pixel point of the defect area to be analyzed to form a corrected defect area; obtain the third defect possibility based on the difference in the second defect possibility between the corrected defect area and the corresponding defect area to be analyzed, combined with the difference in grayscale change parameters; expand the defect area to be analyzed based on the third defect possibility to obtain the final defect area on the surface of the optoelectronic device.
[0093] After obtaining the grayscale change parameters, the defect area to be analyzed can be gradually expanded. First, the expansion method is determined, which has been selected in step S4, that is, along the direction of the line connecting the center pixel point and the edge pixel point of the defect area to be analyzed, pixel points are gradually added to form a corrected defect area.
[0094] Preferably, in one embodiment of the present invention, in order to accurately expand the defect area to be analyzed, the corrected defect area is updated once after each pixel point is expanded, so as to calculate the possibility of the third defect, determine the expansion effect, and obtain a more accurate final defect area, specifically: the line connecting the center pixel point and the edge pixel point of the defect area to be analyzed is selected in turn as the target line, and an adjacent pixel point is selected on the straight line of the target line away from the center pixel point of the area to extend the target line to obtain the corrected defect area; wherein, the corrected defect area is updated once after each pixel point is added.
[0095] It should be noted that, in one embodiment of the present invention, in the process of expanding the defect area to be analyzed, a method of gradually expanding in all directions is adopted, which is analogous to a circle expanding outward from 1 degree to 360 degrees one by one, and then expanding from 1 degree to 360 degrees one by one.
[0096] After each expansion to obtain the corrected defect area, it is necessary to evaluate the expansion result of the current corrected defect area, taking into account that the addition of pixels will change the edge shape of the corrected defect area and the corresponding defect area to be analyzed, that is, the second defect possibility changes, and the grayscale change parameter will also change. Therefore, the third defect possibility is obtained based on the difference in the second defect possibility between the corrected defect area and the corresponding defect area to be analyzed, combined with the difference in grayscale change parameters.
[0097] Preferably, in one embodiment of the present invention, each time the corrected defect area is updated, the third defect possibility is also updated simultaneously; considering that when the line between the center pixel point of the area and the edge pixel point is extended in the direction away from the center pixel point of the area, the roughness of the edge of the defect area to be analyzed will increase, and the greater the second defect degree, the more it conforms to the characteristics of the defect area, so the difference in the second defect possibility between the corrected defect area and the corresponding defect area to be analyzed is positively correlated with the third defect possibility; at the same time, considering that when adding pixels to extend the line, it is necessary to conform to the grayscale change characteristics of the defect area to be analyzed, the difference between the grayscale change parameters is negatively correlated with the third defect possibility; and in order to facilitate the evaluation of the expansion effect, the third defect possibility is also normalized.
[0098] As an example: The calculation formula for the third defect probability includes: ;in, Indicates the serial number of the defect area to be analyzed; Indicates The third defect probability of the defect area to be analyzed corresponding to the current defect area to be corrected; Indicates The second defect possibility of the defect area to be analyzed; Indicates The second defect possibility of the defect area to be analyzed corresponding to the current defect area to be corrected; Indicates the serial number of the edge pixel point of the defect area to be analyzed, which is also the serial number of the edge pixel point corresponding to the line extended by the last added pixel point; Indicates The first defect area to be analyzed Grayscale change parameter corresponding to edge pixels; Indicates The first defect area to be analyzed The grayscale change parameter of the edge pixel points corresponding to the edge pixel points of the current defect correction area; Indicates the zero division parameter to prevent the denominator from being 0. .
[0099] In the calculation formula of the third defect probability, The larger the value, the rougher the edge of the extended corrected defect area is, the more it conforms to the characteristics of the defect area, and the greater the possibility of the third defect; The smaller it is, the more similar the grayscale change characteristics of the extended line are to those before and after the extension, and the added pixel points are more consistent with the grayscale change characteristics of the defective area to be analyzed. The more likely it is that the added pixel points are defective pixels that are misjudged as normal pixels during threshold segmentation. The better the extension effect, the greater the possibility of the third defect.
[0100] It should be noted that in other embodiments of the present invention, implementers may also use other negative correlation mapping methods, such as right Negative correlation mapping.
[0101] It should be noted that when calculating the second defect probability corresponding to a certain edge pixel point in the defect correction area, It represents the average value of the boundary distances from all edge pixels to the corresponding center pixels of the region before the defect area to be analyzed is expanded.
[0102] After obtaining the evaluation basis of each expansion, i.e., the third defect possibility, the defect area to be analyzed can be expanded based on the third defect possibility, the expansion process can be controlled, and the final defect area on the surface of the optoelectronic device can be obtained;
[0103] Preferably, in one embodiment of the present invention, when the third defect possibility is less than or equal to the third preset threshold, it is considered that the grayscale change characteristics of the most recently selected pixel point and the defect area to be analyzed are too different, the most recently selected pixel point is removed, and the corresponding target line is excluded, and the subsequent expansion operation no longer considers expanding this target line; when all target lines are excluded, the most recently updated corrected defect area of each defect area to be analyzed is used as the final defect area of the defect area to be analyzed.
[0104] As an example, the third preset threshold is 0.5. In other embodiments of the present invention, implementers can adjust it by themselves.
[0105] In one embodiment of the present invention, considering that after the expansion operation is completed, the final corrected defect area appears as a hedgehog-like area with spike features. In order to make the final defect area smoother, there are some pixels between the spikes that may belong to the defect area. Due to the limitation of the expansion method, they are not included in the final defect area. Therefore, the final defect area still needs to be closed to fill the gaps. The closing operation is a technical means well known to those skilled in the art and will not be described in detail here.
[0106] The embodiment of the present invention may also include a marking operation for the surface defects of the optoelectronic device, specifically including: marking the area selected in each step by multiple mask operations, see Figure 7 , which shows a defect marking schematic diagram provided by an embodiment of the present invention: the first mask operation marks all suspected defect areas and draws the minimum circumscribed rectangle of each suspected defect area for marking; the second mask operation marks all defect areas to be analyzed; the third mask operation marks the final defect area; the mask operation is a technical means well known to those skilled in the art and will not be described in detail here.
[0107] The marked grayscale image can be presented to relevant personnel through a visual display device, such as a monitor, to facilitate management by relevant personnel.
[0108] An embodiment of the present invention also provides a system for visual inspection of surface defects of optoelectronic devices, the system comprising a memory, a processor and a computer program, wherein the memory is used to store a corresponding computer program, the processor is used to run the corresponding computer program, and when the computer program runs in the processor, it can implement a method for visual inspection of surface defects of optoelectronic devices described in steps S1-S5.
[0109] In summary: In response to the technical problem that the existing image segmentation technology is inaccurate in identifying and detecting surface defects of optoelectronic devices, the present invention proposes a method and system for visual detection of surface defects of optoelectronic devices. The present invention first segments the grayscale image of the surface of the optoelectronic device to obtain the initial defect area; further obtains the first defect possibility of each initial defect area and filters out the suspected defect area; further obtains the second defect possibility of each suspected defect area and filters out the defect area to be analyzed; further obtains the grayscale change parameters of each edge pixel point of the defect area to be analyzed; further gradually adds pixels to form a corrected defect area; further obtains the third defect possibility, and finally expands the defect area to be analyzed based on the third defect possibility to obtain the final defect area on the surface of the optoelectronic device. The present invention analyzes the grayscale change characteristics and edge roughness of the initial defect area obtained by threshold segmentation, eliminates interference from non-defective areas, and adaptively expands the defect area to be analyzed to finally obtain an accurate defect area, thereby improving the credibility of the visual detection results.
[0110] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0111] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
Claims
1. A method for visually detecting surface defects of optoelectronic devices. It is characterized in that The method comprises: Acquire a grayscale image of the surface of the optoelectronic device; segment the grayscale image to obtain an initial defect area; According to the local grayscale difference of each pixel point in the initial defect area, combined with the difference of the overall grayscale characteristics of different initial defect areas, the first defect possibility of each initial defect area is obtained; based on the first defect possibility, the suspected defect area is screened out; According to the edge roughness of the suspected defect area, obtaining a second defect possibility of each of the suspected defect areas; and screening out defect areas to be analyzed based on the second defect possibility; According to the grayscale change characteristics of the pixels on the line connecting the center pixel and the edge pixel of the defect area to be analyzed, combined with the grayscale change difference between adjacent edge pixel points, the grayscale change parameter of each edge pixel of the defect area to be analyzed is obtained; Along the direction of the line connecting the center pixel point and the edge pixel point of the defect area to be analyzed, pixel points are gradually added to form a corrected defect area; according to the difference between the second defect possibility between the corrected defect area and the corresponding defect area to be analyzed, combined with the difference between the grayscale change parameters, a third defect possibility is obtained; based on the third defect possibility, the defect area to be analyzed is expanded to obtain a final defect area on the surface of the optoelectronic device; The method for obtaining the first defect possibility includes: In each of the initial defect areas, a preset neighborhood window is established with each pixel point as the center; according to the gray value difference between all the window center pixels and the corresponding window non-center pixels in each of the initial defect areas, the first initial defect parameter of each of the initial defect areas is obtained; wherein the pixel point at the center of the preset neighborhood window is called the window center pixel point, and the other pixels are called the window non-center pixels; the gray value difference between the window center pixel point and the corresponding window non-center pixel point is positively correlated with the first initial defect parameter; Acquire a second initial defect parameter of each initial defect area according to the fluctuation characteristics of the grayscale value of the pixel point in each initial defect area; Obtaining the first defect possibility according to the first initial defect parameter and the second initial defect parameter of each initial defect area; the first initial defect parameter and the second initial defect parameter are both positively correlated with the first defect possibility; The method for obtaining the second defect possibility includes: Obtaining the distance from each edge pixel point to the center pixel point of each suspected defect area as the boundary distance, and obtaining the second defect possibility according to the fluctuation intensity of the boundary distance of each suspected defect area; the fluctuation intensity of the boundary distance is positively correlated with the second defect possibility; The method for obtaining the corrected defect area comprises: Selecting the line connecting the center pixel point and the edge pixel point of the defect area to be analyzed as the target line in turn, selecting an adjacent pixel point on the straight line of the target line away from the center pixel point of the area to extend the target line, and obtaining the corrected defect area; wherein the corrected defect area is updated once each pixel point is added; Each time the corrected defect area is updated, the third defect possibility is updated simultaneously; the difference between the second defect possibility between the corrected defect area and the corresponding defect area to be analyzed is positively correlated with the third defect possibility; the difference between the grayscale change parameters is negatively correlated with the third defect possibility; the third defect possibility is also normalized; The method for obtaining the final defect area includes: When the third defect possibility is less than or equal to the third preset threshold, remove the most recently selected pixel point and exclude the corresponding target line; when all the target lines are excluded, take the last updated corrected defect area of each defect area to be analyzed as the final defect area of the defect area to be analyzed.
2. A method for visually detecting surface defects of an optoelectronic device according to claim 1, It is characterized in that The method for obtaining the grayscale change parameter includes: On the line connecting the center pixel point and the edge pixel point of the defect area to be analyzed, a grayscale change sub-parameter is obtained according to the fluctuation degree of the grayscale value change between adjacent pixel points; the fluctuation degree of the grayscale value change between adjacent pixel points is positively correlated with the grayscale change sub-parameter; The grayscale change parameter of each edge pixel is obtained according to the difference between the grayscale change sub-parameter of each edge pixel and the adjacent edge pixel; the difference between the grayscale change sub-parameter of the edge pixel and the adjacent edge pixel is positively correlated with the grayscale change parameter.
3. A method for visually detecting surface defects of an optoelectronic device according to claim 1, It is characterized in that The method for obtaining the suspected defect area includes: The first defect possibility is also normalized; when the first defect possibility is greater than a first preset threshold, the corresponding initial defect area is marked as the suspected defect area.
4. A method for visually detecting surface defects of an optoelectronic device according to claim 1, It is characterized in that The method for obtaining the defect area to be analyzed includes: The second defect possibility is further normalized; when the second defect possibility is greater than a second preset threshold, the corresponding suspected defect area is marked as the defect area to be analyzed.
5. A system for visually inspecting surface defects of an optoelectronic device, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method for visually detecting surface defects of an optoelectronic device as described in any one of claims 1 to 4 are implemented.
Citation Information
Cited By
Electronic device detection system based on computer vision
CN119784754A
