Lamp bead defect detection method, electronic device and storage medium

Abstract

The present application discloses a method for detecting lamp bead defects, an electronic device and a storage medium. The method includes: obtaining a lamp bead lighting image of a panel to be detected; determining a binary image of the lit lamp bead based on the lamp bead lighting image; determining the lamp bead pixel coordinates of each lamp bead based on the binary image of the lit lamp bead; merging adjacent RGB lamp beads in the lamp bead lighting image into each lamp bead combination grid according to the lamp bead pixel coordinates of each lamp bead; determining the lamp bead feature information corresponding to each lamp bead combination grid; determining the lamp bead defect type corresponding to each lamp bead combination grid based on the lamp bead feature information corresponding to each lamp bead combination grid. Utilizing the technical solution of the present application, the detection accuracy of lamp bead defects can be improved, the lamp bead defect conditions can be effectively classified, and the production quality of MiniLED panels can be improved.

Description

Technical Field

[0001] The present application generally relates to the field of image processing technology. More specifically, the present application relates to a lamp bead defect detection method, an electronic device and a storage medium. Background Art

[0002] MiniLED panel is a new display technology, which is an upgraded version of the traditional LCD screen. It improves the display effect by using smaller LED backlight beads. MiniLED backlight technology increases the number of LED beads from 50 to 10,000 to 20,000, greatly improving the density of backlight beads per unit area and achieving a more delicate display effect.

[0003] However, various defects may occur in the production process of MiniLED panels, mainly defects that exist when the MiniLED panel lamp beads are lit, which may include defects such as lamp beads being too bright, lamp beads being too dark, lamp beads being extinguished, and lamp beads being lit at the same time. In order to accurately detect the above lamp bead defects, the lamp beads need to be accurately positioned to output the row and column coordinates of the abnormal lamp beads, and the positions of the RGB lamp beads that are lit separately need to be merged, which is conducive to classifying the lamp bead defects.

[0004] In view of this, there is an urgent need to provide a lamp bead defect detection method in order to improve the detection accuracy of lamp bead defects, effectively classify the lamp bead defects, and improve the production quality of MiniLED panels. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a lamp bead defect detection method, an electronic device and a storage medium in multiple aspects. The lamp bead defect detection method can improve the detection accuracy of lamp bead defects, effectively classify lamp bead defects, and improve the production quality of MiniLED panels.

[0006] In the first aspect, the present application provides a lamp bead defect detection method, including: obtaining a lamp bead lighting image of a panel to be inspected; determining a binary image of the lit lamp beads based on the lamp bead lighting image; determining the lamp bead pixel coordinates of each lamp bead based on the lit lamp bead binary image; merging adjacent RGB lamp beads in the lamp bead lighting image into each lamp bead combination grid according to the lamp bead pixel coordinates of each lamp bead; determining the lamp bead feature information corresponding to each lamp bead combination grid; determining the lamp bead defect type corresponding to each lamp bead combination grid based on the lamp bead feature information corresponding to each lamp bead combination grid.

[0007] In some embodiments, determining a binary image of lit lamp beads based on a lamp bead lighting image includes: determining a lamp bead response image based on the lamp bead lighting image; and determining a binary image of lit lamp beads based on the lamp bead response image.

[0008] In some embodiments, determining a lamp bead response map based on a lamp bead lighting image includes: performing threshold segmentation on the lamp bead lighting image according to a first segmentation threshold to obtain a lamp bead region of interest image; performing angle correction processing on the lamp bead region of interest image to obtain a detection correction image; and processing the detection correction image through a morphological top hat to obtain a lamp bead response map.

[0009] In some embodiments, determining a binary image of lit lamp beads based on a lamp bead response image includes: performing threshold segmentation on the lamp bead response image according to a second segmentation threshold to obtain a binary image; and filtering out bright spots in the binary image according to a preset lamp bead length and a preset lamp bead area to obtain a binary image of lit lamp beads.

[0010] In some embodiments, determining the pixel coordinates of each lamp bead based on the binary image of lit lamp beads includes: searching for closed areas in the binary image of lit lamp beads; determining the center point coordinates corresponding to each closed area, and determining the center point coordinates corresponding to each closed area as the pixel coordinates of each lit lamp bead; performing a morphological opening operation on the binary image of lit lamp beads to obtain an opening operation image and determine the edge distance of the lamp bead column and the edge distance of the lamp bead row based on the opening operation image; determining the pixel coordinates of each unlit lamp bead according to the lamp bead pixel coordinates of each lit lamp bead, the edge distance of the lamp bead column, the edge distance of the lamp bead row, the preset lamp bead column spacing and the preset lamp bead row spacing; determining the pixel coordinates of each lamp bead based on the lamp bead pixel coordinates of each lit lamp bead and the lamp bead pixel coordinates of each unlit lamp bead.

[0011] In some embodiments, respectively determining the lamp bead characteristic information corresponding to each lamp bead combination grid includes: respectively determining the average grayscale value of the lamp beads corresponding to each lamp bead combination grid; and respectively determining the number of closed areas corresponding to each lamp bead combination grid.

[0012] In some embodiments, determining the lamp bead defect type corresponding to each lamp bead combination grid based on the lamp bead characteristic information corresponding to each lamp bead combination grid includes: determining the lamp bead defect type corresponding to each lamp bead combination grid according to the average grayscale value of the lamp bead corresponding to each lamp bead combination grid and the number of closed areas corresponding to each lamp bead combination grid.

[0013] In some embodiments, determining the lamp bead defect type corresponding to each lamp bead combination grid based on the average grayscale value of the lamp beads corresponding to each lamp bead combination grid and the number of closed areas corresponding to each lamp bead combination grid includes: if the number of closed areas corresponding to the lamp bead combination grid is 0, then determining the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp beads are off; if the number of closed areas corresponding to the lamp bead combination grid is 1, then comparing the closed area size corresponding to the lamp bead combination grid with the upper limit value of the lamp bead size, and if the closed area size corresponding to the lamp bead combination grid is greater than the upper limit value of the lamp bead size, then determining the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp beads are lit in parallel; if the number of closed areas corresponding to the lamp bead combination grid is greater than 1, then determining the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp beads are lit in parallel; if the average grayscale value of the lamp beads corresponding to the lamp bead combination grid is less than the lower limit value of the grayscale of the lamp beads, then determining the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp beads are too dark; if the average grayscale value of the lamp beads corresponding to the lamp bead combination grid is greater than the upper limit value of the grayscale of the lamp beads, then determining the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp beads are too bright.

[0014] In a second aspect, the present application provides an electronic device, comprising: a processor; and a memory, on which is stored a program code for lamp bead defect detection, and when the program code is executed by the processor, the electronic device implements the method as described above.

[0015] In a third aspect, the present application provides a non-transitory machine-readable storage medium having stored thereon a program code for lamp bead defect detection, and when the program code is executed by a processor, the method as described above can be implemented.

[0016] The technical solution provided by this application may have the following beneficial effects:

[0017] The lamp bead defect detection method, electronic device and storage medium provided in the present application obtain the lamp bead lighting image of the panel to be inspected, and then determine the lit lamp bead binary image based on the lamp bead lighting image, and then determine the lamp bead pixel coordinates of each lamp bead based on the lit lamp bead binary image, thereby achieving accurate positioning of each lamp bead.

[0018] Furthermore, the present application can merge adjacent RGB lamp beads in the lamp bead lighting image into each lamp bead combination grid according to the lamp bead pixel coordinates of each lamp bead, and then determine the lamp bead feature information corresponding to each lamp bead combination grid, and provide effective feature information for the lamp bead defect classification of each lamp bead combination grid. Then, based on the lamp bead feature information corresponding to each lamp bead combination grid, the lamp bead defect type corresponding to each lamp bead combination grid is determined, so that the lamp bead defect type at each position on the panel to be detected can be accurately determined, which is conducive to quickly providing effective defect solutions for the corresponding lamp bead defect type.

[0019] In general, this application can improve the detection accuracy of lamp bead defects, effectively classify lamp bead defects, and improve the production quality of MiniLED panels. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0021] Figure 1 An exemplary flow chart of a lamp bead defect detection method according to some embodiments of the present application is shown;

[0022] Figure 2 An exemplary flow chart showing a method for detecting lamp bead defects according to other embodiments of the present application is shown;

[0023] Figure 3 An exemplary flow chart of a lamp bead defect detection method according to some other embodiments of the present application is shown;

[0024] Figure 4 A macroscopic schematic diagram of a lamp bead lighting image in a lamp bead defect detection method according to an embodiment of the present application is shown;

[0025] Figure 5 A microscopic schematic diagram of a lamp bead lighting image in a lamp bead defect detection method according to an embodiment of the present application is shown;

[0026] Figure 6 A schematic diagram showing a lamp bead response diagram in a lamp bead defect detection method according to an embodiment of the present application is shown;

[0027] Figure 7 A schematic diagram of a binary image of a lit lamp bead in a lamp bead defect detection method according to an embodiment of the present application is shown;

[0028] Figure 8 A partial schematic diagram of a binary image of a lit lamp bead before the on operation in the lamp bead defect detection method of an embodiment of the present application is shown;

[0029] Fig. 9 A partial schematic diagram of an on-operation image formed by a binary image of a lit lamp bead after an on-operation in a lamp bead defect detection method according to an embodiment of the present application is shown;

[0030] Fig.10 A schematic diagram showing the recording of the pixel coordinates of each lamp bead of the panel to be inspected in the lamp bead defect detection method of the embodiment of the present application is shown;

[0031] Fig.11A microscopic schematic diagram of the lamp bead defect detection method in an embodiment of the present application after adjacent RGB lamp beads in a lamp bead lighting image are merged into each lamp bead combination grid;

[0032] Fig.12 A schematic diagram showing the detection result when the lamp bead defect type corresponding to the lamp bead combination grid is lamp bead extinguishing in the lamp bead defect detection method of an embodiment of the present application;

[0033] Fig.13 A schematic diagram showing the detection results when the number of closed areas corresponding to the lamp bead combination grid is 1 and the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp bead is not lit in the lamp bead defect detection method of the embodiment of the present application;

[0034] Fig.14 A schematic diagram showing the detection result when the number of closed areas corresponding to the lamp bead combination grid is greater than 1 and the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp bead is not lit in the lamp bead defect detection method of the embodiment of the present application;

[0035] Fig.15 A schematic diagram showing the detection result when the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp bead is too dark in the lamp bead defect detection method of the embodiment of the present application;

[0036] Fig.16 A schematic diagram showing the detection result when the lamp bead defect type corresponding to the lamp bead combination grid is the lamp bead being too bright in the lamp bead defect detection method of the embodiment of the present application;

[0037] Fig.17 A schematic diagram of the structure of an electronic device shown in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. For the simplicity and clarity of the description, the figure marks may be repeated in the drawings to indicate corresponding or similar elements when deemed appropriate. In addition, the present application sets forth many specific details in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other cases, well-known methods, processes, and components are not described in detail to avoid blurring the embodiments described herein. Moreover, the description should not be regarded as limiting the scope of the embodiments described herein. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0039] It should be understood that the possible terms "first" or "second" etc. in the claims, specifications and drawings disclosed in this application are used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of this application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0040] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0041] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0042] MiniLED panels may have various defects during the production process, mainly defects that exist when the MiniLED panel lamp beads are lit, including defects such as lamp beads being too bright, lamp beads being too dark, lamp beads being extinguished, and lamp beads being lit at the same time. In order to accurately detect the above lamp bead defects, the lamp beads need to be accurately positioned to output the row and column coordinates of the abnormal lamp beads, and the positions of the RGB lamp beads that are lit separately need to be merged, which is conducive to classifying the lamp bead defects.

[0043] In view of this, there is an urgent need to provide a lamp bead defect detection method in order to improve the detection accuracy of lamp bead defects, effectively classify the lamp bead defects, and improve the production quality of MiniLED panels.

[0044] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0045] Figure 1 An exemplary flow chart 100 showing a method for detecting lamp bead defects in some embodiments of the present application is shown in FIG. Figure 1 , the lamp bead defect detection method shown in the embodiment of the present application may include:

[0046] In step S101, an image of the lamp beads of the panel to be detected is obtained. In the embodiment of the present application, the panel to be detected may refer to the MiniLED panel to be detected. MiniLED panels use smaller LED lamp beads as backlight sources. The number of lamp beads is usually between thousands and tens of thousands. These lamp beads can be divided into multiple independent backlight partitions, and the brightness of each partition can be controlled separately, thereby achieving a more refined display effect.

[0047] like Figure 4 As shown in FIG. 1 , the aforementioned lamp bead lighting image refers to an image of the panel to be inspected that is captured by an imaging device such as an industrial camera after all the lamp beads on the panel to be inspected are powered on and lit. The lamp bead lighting image is magnified for microscopic observation, such as Figure 5 As shown, it can be found that lamp beads can be divided into red light beads, green light beads and blue light beads, so that various colors can be displayed by adjusting the light intensity and proportion of these three colors of lamp beads.

[0048] In step S102, a binary image of the lit lamp beads is determined based on the lamp bead lit image. In the embodiment of the present application, since the positioning of the lamp beads does not require color information, the lamp bead lit image can be threshold segmented and binarized to generate the following: Figure 7 The binary image of the lit lamp beads is shown. The bright spots in the binary image of the lit lamp beads are used to determine the position of each lamp bead.

[0049] In step S103, the pixel coordinates of each lamp bead are determined based on the binary image of the lit lamp beads. In the embodiment of the present application, the bright spot in the binary image of the lit lamp beads can be used to locate each lamp bead, and the center point coordinates of the bright spot can be used as the pixel coordinates of each lamp bead. Fig.10 As shown in the figure, after the pixel coordinates of each lamp bead are confirmed, the following Fig.10 The recording schematic diagram is used to record the pixel coordinates of each lamp bead.

[0050] In step S104, adjacent RGB lamp beads in the lamp bead lighting image are merged into each lamp bead combination grid according to the lamp bead pixel coordinates of each lamp bead. In the embodiment of the present application, RGB lamp beads refer to a lamp bead combination of red lamp beads, green lamp beads and blue lamp beads. Figure 5 As shown in the microscopic diagram of the lamp bead lighting image, the RGB lamp beads are arranged in sequence horizontally. Therefore, the adjacent RGB lamp beads in the lamp bead lighting image can be searched out by using the lamp bead pixel coordinates of each lamp bead, so as to Fig.11 As shown, the RGB lamp beads obtained by the search are merged into each lamp bead combination grid.

[0051] In step S105, the lamp bead feature information corresponding to each lamp bead combination grid is determined respectively. In the embodiment of the present application, the aforementioned lamp bead feature information may include but is not limited to the average grayscale value of the lamp bead, the number of closed areas, and the area of ​​the closed area. Among them, the aforementioned closed area refers to the closed contour of the continuous area in the image, and the closed area can be searched by the findContours function in OpenCV to obtain the number of closed areas. And the number of pixels in the closed area can be used as the closed area area corresponding to the closed area.

[0052] In step S106, the defect type of the lamp beads corresponding to each lamp bead combination grid is determined based on the lamp bead characteristic information corresponding to each lamp bead combination grid. It can be understood that the average grayscale value of the lamp beads can be used to determine whether the lamp beads in the lamp bead combination grid are too bright or too dark. In addition, the area occupied by each lamp bead can be regarded as a closed area, so the number of closed areas and the area of ​​the closed areas can be used to determine whether the lamp beads in the lamp bead combination grid are lit or extinguished.

[0053] The embodiment of the present application obtains the lamp bead lighting image of the panel to be inspected, and then determines the binary image of the lit lamp beads based on the lamp bead lighting image, and then determines the lamp bead pixel coordinates of each lamp bead based on the binary image of the lit lamp beads, thereby accurately positioning each lamp bead. Furthermore, the present application can merge adjacent RGB lamp beads in the lamp bead lighting image into each lamp bead combination grid according to the lamp bead pixel coordinates of each lamp bead, and then determine the lamp bead feature information corresponding to each lamp bead combination grid, and provide effective feature information for the lamp bead defect classification of each lamp bead combination grid. Then, based on the lamp bead feature information corresponding to each lamp bead combination grid, the lamp bead defect type corresponding to each lamp bead combination grid is determined, so that the lamp bead defect type at each position on the panel to be inspected can be accurately determined, which is conducive to quickly providing effective defect solutions for the corresponding lamp bead defect type. In general, the present application can improve the detection accuracy of lamp bead defects, effectively classify lamp bead defect conditions, and improve the production quality of MiniLED panels.

[0054] In some embodiments, the process of determining the pixel coordinates of each lamp bead can be further designed. Figure 2 The process of determining the pixel coordinates of each lamp bead is described in detail. Figure 2 An exemplary flow chart 200 showing a method for detecting lamp bead defects in other embodiments of the present application is shown. Figure 2 , the lamp bead defect detection method shown in the embodiment of the present application may include:

[0055] In step S201, an image of the lamp beads of the panel to be detected is obtained. In the embodiment of the present application, the content of step S201 is substantially the same as that of step S101, and will not be repeated here.

[0056] In step S202, the lamp bead response map is determined based on the lamp bead lighting image. In the embodiment of the present application, the lamp bead lighting image can be threshold segmented according to the first segmentation threshold to obtain the lamp bead area of ​​interest image. It is understandable that the area where the lamp bead is lit will be brighter than other edge areas of the panel to be detected, so the first segmentation threshold can be set to a relatively high grayscale value, such as between 120 and 150, so as to remove the pixels below the first segmentation threshold to obtain the lamp bead area of ​​interest image.

[0057] Furthermore, since the image of the lamp bead lighting may have an angle deviation during the imaging process, in order to improve the detection accuracy, the image of the lamp bead region of interest obtained after threshold segmentation can be angle-corrected to obtain a detection correction image. Specifically, the four edges of the lamp bead region of interest image can be first extracted through an edge detection algorithm (such as Canny edge detection); then the pixel points of the four edges can be line-fitted, for example, the Hough Transform can be used to detect straight lines; then the intersection of the four straight lines obtained by fitting can be calculated based on the parameters of the fitted lines to obtain four corner points; finally, the four corner points can be used to calculate the perspective transformation matrix using OpenCV's cv2.getPerspectiveTransform() function to complete the angle correction and obtain a detection correction image.

[0058] Furthermore, the detection correction image can be processed by morphological top-hat to obtain the lamp bead response map. Morphological top-hat transformation is an image processing operation, which is defined as the result of subtracting the opening operation from the original image. The opening operation is a process of corrosion followed by expansion, which can eliminate the smaller bright areas in the image. Therefore, morphological top-hat can enhance the brighter details in the image and eliminate the unevenness in the background, thus obtaining the following: Figure 6 The lamp bead response diagram is shown.

[0059] In step S203, a binary image of the lit lamp beads is determined based on the lamp bead response image. In the embodiment of the present application, the lamp bead response image can be threshold segmented according to the second segmentation threshold to obtain a binary image, wherein the second segmentation threshold can be set to a grayscale value suitable for binarizing the lamp bead response image, for example, between 120 and 150, so as to highlight the bright spots corresponding to the lamp beads. Furthermore, the bright spots in the binary image can be screened out according to the preset lamp bead length and the preset lamp bead area, so as to screen out some interfering bright spots that are not lamp beads, thereby obtaining the following: Figure 7 The binary image of the lighted lamp bead is shown. It can be understood that Figure 7 The binary image of the lit lamp bead is shown in Figure 6 Based on the lamp bead response diagram shown in the figure, the image is obtained after further screening out the lamp beads whose length and area do not meet the requirements. Since the overall size of the lamp beads is small, Figure 6 and Figure 7 The changes are not obvious, but there are still differences between the two figures.

[0060] In step S204, the pixel coordinates of each lamp bead are determined based on the binary image of the lit lamp beads. In the embodiment of the present application, the binary image of the lit lamp beads can be used to locate the position of the lit lamp beads. Specifically, the closed area in the binary image of the lit lamp beads can be searched. For example, the findContours function in OpenCV can be used to search for all closed areas in the binary image of the lit lamp beads, such as Figure 8 As shown, Figure 8 The white area in the figure represents the closed area of ​​the lamp bead. Furthermore, each closed area can be traversed to determine the center point coordinates corresponding to each closed area through the boundingRect operation in OpenCV, and the center point coordinates corresponding to each closed area are determined as the pixel coordinates of each lit lamp bead.

[0061] After determining the position of the lit lamp bead, the position of the lit lamp bead can be used to infer the position of the extinguished lamp bead. The morphological opening operation can be performed on the binary image of the lit lamp bead to remove the burrs around the closed area and remove the interference of the burrs on the distance reasoning calculation, and the following is obtained: Fig. 9 The on operation image shown is displayed, and the edge distance of the lamp bead column and the edge distance of the lamp bead row are determined based on the on operation image. In the embodiment of the present application, the aforementioned edge distance of the lamp bead column refers to the distance between the first column of lamp beads on the left side of the lamp bead array and the left edge of the lamp bead area of ​​interest, or the distance between the first column of lamp beads on the right side of the lamp bead array and the right edge of the lamp bead area of ​​interest. The aforementioned edge distance of the lamp bead row refers to the distance between the first row of lamp beads on the top of the lamp bead array and the top edge of the lamp bead area of ​​interest, or the distance between the first row of lamp beads on the bottom of the lamp bead array and the bottom edge of the lamp bead area of ​​interest.

[0062] Further, the pixel coordinates of each extinguished lamp bead are determined according to the lamp bead pixel coordinates of each lit lamp bead, the lamp bead column edge distance, the lamp bead row edge distance, the preset lamp bead column spacing and the preset lamp bead row spacing. Exemplarily, if the lamp bead column edge distance is the distance between the left first column of lamp beads in the lamp bead array and the left edge of the lamp bead area of ​​interest, and the lamp bead row edge distance is the distance between the top first row of lamp beads in the lamp bead array and the top edge of the lamp bead area of ​​interest, then a fixed step length search can be performed along the diagonal from the lamp bead in the upper left corner to the lower right corner according to the preset lamp bead column spacing and the preset lamp bead row spacing. Specifically, the search range in the X direction can be determined based on the distance between the edges of the lamp beads and the length of the lamp bead array, and the search range in the Y direction can be determined based on the distance between the edges of the lamp beads and the width of the lamp bead array. Then, the search is performed along the diagonal line based on the preset lamp bead column spacing (i.e., the spacing between adjacent lamp beads in the X direction) and the preset lamp bead row spacing (i.e., the spacing between adjacent lamp beads in the Y direction) until the lamp bead in the lower right corner is found, so that the lamp bead pixel coordinates of each extinguished lamp bead can be determined by inference. Finally, the lamp bead pixel coordinates of each lamp bead are determined based on the lamp bead pixel coordinates of each lit lamp bead and the lamp bead pixel coordinates of each extinguished lamp bead, and the following can be generated: Fig.10 Schematic diagram of coordinate recording shown.

[0063] In some embodiments, the defect type of the lamp beads corresponding to each lamp bead combination grid can be determined based on the lamp bead feature information corresponding to each lamp bead combination grid. Figure 3 The process of determining the lamp bead defect type corresponding to each lamp bead combination grid is described in detail. Figure 3 An exemplary flow chart showing a method for detecting lamp bead defects in some other embodiments of the present application is shown in FIG. Figure 3 , the lamp bead defect detection method shown in the embodiment of the present application may include:

[0064] In step S301, adjacent RGB lamp beads in the lamp bead lighting image are merged into each lamp bead combination grid according to the lamp bead pixel coordinates of each lamp bead. In the embodiment of the present application, the content of step S301 is substantially the same as that of step S104, and will not be repeated here.

[0065] In step S302, the average grayscale value of the lamp beads corresponding to each lamp bead combination grid is determined respectively, and the number of closed areas corresponding to each lamp bead combination grid is determined respectively. First, the lamp bead area in the lamp bead combination grid can be segmented out by image processing methods such as threshold segmentation or edge detection. Then, a mask is created based on the segmentation result. The mask is a binary image of the same size as the original image (segmentation result), in which the lamp bead area is white (value is 255) and the background is black (value is 0). Then, based on the mask and the segmentation results, the mean function can be used to determine the average grayscale value of the lamp beads corresponding to the lamp bead combination grid. In addition, the findContours function in OpenCV can be used to search for closed areas to obtain the number of closed areas.

[0066] In step S303, the defect type of the lamp beads corresponding to each lamp bead combination grid is determined according to the average grayscale value of the lamp beads corresponding to each lamp bead combination grid and the number of closed areas corresponding to each lamp bead combination grid. Fig.12 As shown in , if the number of closed areas corresponding to the lamp bead combination grid is 0, it is determined that the lamp bead defect type corresponding to the lamp bead combination grid is lamp bead extinguishing. Fig.13 As shown in the figure, if the number of closed areas corresponding to the lamp bead combination grid is 1, the size of the closed area corresponding to the lamp bead combination grid is compared with the upper limit of the lamp bead size, and if the size of the closed area corresponding to the lamp bead combination grid is greater than the upper limit of the lamp bead size, it means that the current closed area is composed of more than one lamp bead, and the lamp bead defect type corresponding to the lamp bead combination grid is determined to be lamp bead and light. Fig.14 As shown in , if the number of closed areas corresponding to the lamp bead combination grid is greater than 1, it is determined that the lamp bead defect type corresponding to the lamp bead combination grid is the lamp bead and light. Fig.15 As shown in , if the average grayscale value of the lamp beads corresponding to the lamp bead combination grid is less than the lower limit of the lamp bead grayscale, it is determined that the defect type of the lamp beads corresponding to the lamp bead combination grid is that the lamp beads are too dark. Fig.16 As shown, if the average grayscale value of the lamp beads corresponding to the lamp bead combination grid is greater than the upper limit of the lamp bead grayscale, it is determined that the defect type of the lamp beads corresponding to the lamp bead combination grid is that the lamp beads are too bright.

[0067] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides an electronic device for executing a lamp bead defect detection method and a corresponding embodiment.

[0068] Fig.17 FIG. 1 is a block diagram showing the hardware configuration of an electronic device 1700 that can implement the lamp bead defect detection method of the embodiment of the present application. Fig.17 As shown, the electronic device 1700 may include a processor 1710 and a memory 1720. Fig.17In the electronic device 1700, only the components related to this embodiment are shown. Therefore, it is obvious to those skilled in the art that the electronic device 1700 may also include Fig.17 The components shown in the figure are different from the common components. For example: fixed-point arithmetic units.

[0069] The electronic device 1700 may correspond to a computing device having various processing functions, for example, a function for generating a neural network, training or learning a neural network, quantizing a floating-point neural network to a fixed-point neural network, or retraining a neural network. For example, the electronic device 1700 may be implemented as various types of devices, such as a personal computer (PC), a server device, a mobile device, etc.

[0070] The processor 1710 controls all functions of the electronic device 1700. For example, the processor 1710 controls all functions of the electronic device 1700 by executing a program stored in the memory 1720 on the electronic device 1700. The processor 1710 may be implemented by a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), an artificial intelligence processor chip (IPU), etc. provided in the electronic device 1700. However, the present application is not limited thereto.

[0071] In some embodiments, the processor 1710 may include an input / output (I / O) unit 1711 and a computing unit 1712. The I / O unit 1711 may be used to receive various data, such as a lamp bead lighting image. Exemplarily, the computing unit 1712 may be used to determine a lit lamp bead binary image based on the lamp bead lighting image received via the I / O unit 1711; determine the lamp bead pixel coordinates of each lamp bead based on the lit lamp bead binary image; merge adjacent RGB lamp beads in the lamp bead lighting image into each lamp bead combination grid according to the lamp bead pixel coordinates of each lamp bead; determine the lamp bead feature information corresponding to each lamp bead combination grid; determine the lamp bead defect type corresponding to each lamp bead combination grid based on the lamp bead feature information corresponding to each lamp bead combination grid. This lamp bead defect type may be output by the I / O unit 1711, for example. The output data may be provided to the memory 1720 for reading and use by other devices (not shown), or may be directly provided to other devices for use.

[0072] The memory 1720 is hardware for storing various data processed in the electronic device 1700. For example, the memory 1720 can store processed data and data to be processed in the electronic device 1700. The memory 1720 can store data sets involved in the process of the lamp bead defect detection method that has been processed or to be processed by the processor 1710, such as lamp bead lighting images, etc. In addition, the memory 1720 can store applications, drivers, etc. to be driven by the electronic device 1700. For example: the memory 1720 can store various programs related to the lamp bead defect detection method to be executed by the processor 1710. The memory 1720 can be a DRAM, but the present application is not limited thereto. The memory 1720 can include at least one of a volatile memory or a non-volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a phase change RAM (PRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), a ferroelectric RAM (FRAM), and the like. The volatile memory may include dynamic RAM (DRAM), static RAM (SRAM), synchronous DRAM (SDRAM), PRAM, MRAM, RRAM, ferroelectric RAM (FeRAM), etc. In an embodiment, the memory 1720 may include at least one of a hard disk drive (HDD), a solid state drive (SSD), a high-density flash memory (CF), a secure digital (SD) card, a micro secure digital (Micro-SD) card, a mini secure digital (Mini-SD) card, an extreme digital (xD) card, caches, or a memory stick.

[0073] In summary, the specific functions implemented by the memory 1720 and the processor 1710 of the electronic device 1700 provided in the implementation mode of this specification can be explained in comparison with the aforementioned implementation modes in this specification, and can achieve the technical effects of the aforementioned implementation modes, and will not be repeated here.

[0074] In this embodiment, the processor 1710 may be implemented in any suitable manner. For example, the processor 1710 may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc.

[0075] It should also be understood that any module, unit, component, server, computer, terminal or device that executes instructions exemplified herein may include or otherwise access computer-readable media, such as storage media, computer storage media or data storage devices (removable and / or non-removable) such as disks, optical disks or tapes. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data.

[0076] Although multiple embodiments of the present application have been shown and described herein, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art can think of many changes, modifications and alternatives without departing from the thought and spirit of the present application. It should be understood that in the process of practicing the present application, various alternatives to the embodiments of the present application described herein can be adopted. The attached claims are intended to limit the scope of protection of the present application, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for detecting lamp bead defects, characterized in that: include: Obtain the image of the lamp beads lighting up the panel to be detected; Determine a binary image of the lit lamp beads based on the lamp bead lighting image; Determine the pixel coordinates of each lamp bead based on the binary image of the lit lamp beads; According to the pixel coordinates of each lamp bead, adjacent RGB lamp beads in the lamp bead lighting image are merged into each lamp bead combination grid; Determine the lamp bead characteristic information corresponding to each lamp bead combination grid respectively; Wherein, the step of respectively determining the lamp bead characteristic information corresponding to each lamp bead combination grid includes: Determine the average grayscale value of the lamp beads corresponding to each lamp bead combination grid; and Determine the number of closed areas corresponding to each lamp bead combination grid; Determine the lamp bead defect type corresponding to each lamp bead combination grid based on the lamp bead feature information corresponding to each lamp bead combination grid; Wherein, determining the lamp bead defect type corresponding to each lamp bead combination grid based on the lamp bead feature information corresponding to each lamp bead combination grid includes: Determine the lamp bead defect type corresponding to each lamp bead combination grid according to the average grayscale value of the lamp bead corresponding to each lamp bead combination grid and the number of closed areas corresponding to each lamp bead combination grid; Wherein, determining the lamp bead defect type corresponding to each lamp bead combination grid according to the average gray value of the lamp bead corresponding to each lamp bead combination grid and the number of closed areas corresponding to each lamp bead combination grid includes: If the number of closed areas corresponding to the lamp bead combination grid is 0, then the lamp bead defect type corresponding to the lamp bead combination grid is determined to be lamp bead extinguishing; If the number of closed areas corresponding to the lamp bead combination grid is 1, the size of the closed area corresponding to the lamp bead combination grid is compared with the upper limit of the lamp bead size, and if the size of the closed area corresponding to the lamp bead combination grid is greater than the upper limit of the lamp bead size, it is determined that the lamp bead defect type corresponding to the lamp bead combination grid is the lamp bead and light; If the number of closed areas corresponding to the lamp bead combination grid is greater than 1, it is determined that the lamp bead defect type corresponding to the lamp bead combination grid is that the lamp bead is not lit; If the average grayscale value of the lamp beads corresponding to the lamp bead combination grid is less than the lower limit of the lamp bead grayscale, it is determined that the defect type of the lamp beads corresponding to the lamp bead combination grid is that the lamp beads are too dark; If the average grayscale value of the lamp beads corresponding to the lamp bead combination grid is greater than the upper limit of the lamp bead grayscale, it is determined that the defect type of the lamp beads corresponding to the lamp bead combination grid is that the lamp beads are too bright.

2. The lamp bead defect detection method according to claim 1, characterized in that: Determining the binary image of the lit lamp beads based on the lamp bead lighting image includes: Determine a lamp bead response diagram based on the lamp bead lighting image; The lit lamp bead binary map is determined based on the lamp bead response map.

3. The lamp bead defect detection method according to claim 2, characterized in that: Determining the lamp bead response diagram based on the lamp bead lighting image includes: Perform threshold segmentation on the lamp bead lighting image according to a first segmentation threshold to obtain a lamp bead region of interest image; Performing angle correction processing on the image of the region of interest of the lamp bead to obtain a detection correction image; The detection correction image is processed by morphological top hat to obtain the lamp bead response map.

4. The lamp bead defect detection method according to claim 2, characterized in that: Determining the lit lamp bead binary image based on the lamp bead response image includes: Performing threshold segmentation on the lamp bead response image according to a second segmentation threshold to obtain a binary image; The bright spots in the binary image are screened out according to the preset lamp bead length and the preset lamp bead area to obtain the binary image of the lit lamp beads.

5. The lamp bead defect detection method according to claim 1, characterized in that: The step of determining the pixel coordinates of each lamp bead based on the lit lamp bead binary image comprises: Searching for a closed area in the binary image of the lit lamp beads; Determine the center point coordinates corresponding to each closed area, and determine the center point coordinates corresponding to each closed area as the pixel coordinates of each lighted lamp bead; Performing a morphological opening operation on the binary image of the lit lamp beads to obtain an opening operation image and determining the edge distance of the lamp bead column and the edge distance of the lamp bead row based on the opening operation image; Determine the pixel coordinates of each extinguished lamp bead according to the lamp bead pixel coordinates of each lit lamp bead, the lamp bead column edge distance, the lamp bead row edge distance, the preset lamp bead column spacing and the preset lamp bead row spacing; The pixel coordinates of each lamp bead are determined based on the pixel coordinates of each lit lamp bead and the pixel coordinates of each extinguished lamp bead.

6. An electronic device, characterized in that: include: processor; as well as A memory having program codes for lamp bead defect detection stored thereon, wherein when the program codes are executed by the processor, the electronic device implements the method as described in any one of claims 1-5.

7. A non-transitory machine-readable storage medium having stored thereon a program code for lamp bead defect detection, wherein when the program code is executed by a processor, the method as claimed in any one of claims 1 to 5 is implemented.

Images