Detection method of display device
By using machine vision-based automatic pattern detection methods to generate feature maps based on the positional mapping relationship between driving maps and captured images, the problem of insufficient accuracy in display device defect detection is solved. This achieves full-point coverage and accurate identification of defect points, improving the robustness and efficiency of detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU CHINA STAR OPTOELECTRONICS TECH CO LTD
- Filing Date
- 2026-01-28
- Publication Date
- 2026-05-15
AI Technical Summary
In existing technologies, defect detection of display devices relies on lighting conditions, resulting in insufficient detection accuracy. Furthermore, deep learning models require significant resource investment in the detection of large-size display panels, and it is difficult to balance detection time with detection capability.
An automatic pattern detection method based on machine vision is adopted. By acquiring a driving image and a captured image with matching resolution, coarse and fine localization is performed to establish a position mapping relationship and generate a feature map for defect detection, thereby reducing the dependence on hardware conditions and the need for sample training.
It improves the accuracy and robustness of defect detection in display devices, reduces dependence on lighting conditions, lowers hardware and resource requirements, and achieves full-point coverage and accurate identification of defect points.
Smart Images
Figure CN122048852A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of equipment testing technology, and in particular to a testing method for a display device. Background Technology
[0002] To ensure the quality of display devices, it is necessary to test whether the light-emitting points on the display device can emit light normally before the display device leaves the factory, which means it is necessary to test whether the display device has any defects.
[0003] In related technologies, the main method for detecting defects in display devices is to identify and locate defects in images of the display device taken when the screen is on, based on visual inspection methods. However, this method is highly dependent on the lighting conditions during shooting, and cannot guarantee the accuracy of defect identification. Summary of the Invention
[0004] This application provides a method for detecting a display device, which improves the accuracy of defect detection in the display device and at least partially solves the above-mentioned technical problems.
[0005] To achieve the above objectives, according to a first aspect of this application, a method for detecting a display device is provided, comprising: acquiring a driving map that matches the resolution of the display device to be detected, and acquiring images captured by a plurality of imaging devices when the display device displays the driving map, thereby obtaining an image captured by the driving map. Coarse positioning is performed based on the driving diagram and the corresponding captured image to obtain a coarse positioning result; wherein, the coarse positioning result includes the coarse positioning of the light-emitting points of the display device; Based on the coarse positioning result, the driving diagram and the corresponding photographic image of the driving diagram, fine positioning is performed to obtain the fine positioning of the light-emitting point of the display device. Establish a positional mapping relationship between the display device and the captured image based on the precise positioning of the light-emitting points; Based on the position mapping relationship and the detection screen of the display device, a feature map corresponding to the display device is generated; Defect detection is performed based on the feature map to obtain the defect detection result of the display device.
[0006] In some embodiments, the driving graph includes a first driving graph and a second driving graph; the first driving graph is represented as a grid dot matrix graph, and the second driving graph is represented as a stripe graph; The step of performing coarse localization based on the driving map and the corresponding captured image to obtain a coarse localization result includes: Coarse positioning is performed based on the first driving map and the first captured image corresponding to the first driving map to obtain the coarse positioning result; The step of performing fine positioning based on the coarse positioning result, the driving diagram, and the corresponding photographic image of the driving diagram to obtain the fine positioning of the light-emitting point of the display device includes: Based on the coarse positioning result, the second driving map, and the second image corresponding to the second driving map, fine positioning is performed to obtain the fine positioning of the light-emitting point of the display device.
[0007] In some embodiments, the step of performing coarse positioning based on the first driving map and the first captured image corresponding to the first driving map to obtain the coarse positioning result includes: Based on the correspondence between the first driving image and the first captured image, the affine transformation relationship between the first driving image and the first captured image is determined. The scaling ratio between the first driving image and the first captured image is determined based on the area ratio between the first driving image and the first captured image. Based on the affine transformation relationship and the scaling ratio, and combined with the preset dot matrix spacing of the second driving image, the coordinate mapping relationship between the second driving image and the second captured image is determined; Based on the coordinate mapping relationship, the coarse positioning of each light-emitting point on the display device is determined.
[0008] In some embodiments, the method further includes: Based on the scaling ratio and the pixel spacing between the edge of the first driving image and the boundary of the display device, the spacing between the boundary formed by the outermost dot matrix in the first captured image and the actual boundary of the display device is determined. Based on the position of the boundary formed by the outermost dot matrix in the first captured image and the spacing, the position of the actual boundary of the display device is determined; Based on the actual boundary position of the display device and the first driving map, the effective detection area of the shooting device corresponding to the first driving map is determined. Determining the coordinate mapping relationship between the second driving image and the second captured image based on the affine transformation relationship and the scaling ratio includes: Based on the effective detection area of the shooting device, the effective detection area in the second driving graph corresponding to the shooting device is determined. Based on the affine transformation relationship and the scaling ratio, the coordinate mapping relationship between the effective detection area in the second driving map and the second shooting map corresponding to the second driving map is determined.
[0009] In some embodiments, before determining the scaling ratio between the first driving image and the first captured image based on the area ratio between the first driving image and the first captured image, the method further includes: The product of the number of dots in the first driving graph and the area of a single dot is calculated, and combined with the area occupied by the dot spacing in the first driving graph, the dot coverage area of the first driving graph is obtained; wherein, the area occupied by the dot spacing in the first driving graph is determined according to the preset dot spacing of the first driving graph. Calculate the dot matrix coverage area of the first driving image and the coverage area of the first captured image corresponding to the first driving image to obtain the area ratio between the first driving image and the first captured image.
[0010] In some embodiments, the step of performing fine positioning based on the coarse positioning result, the second driving map, and the second captured image corresponding to the second driving map to obtain the fine positioning of the light-emitting point of the display device includes: Based on the imaging features of the second image, the position of the actual light-emitting point in the second image is determined; Based on the coarse positioning of the light-emitting points in the display device, a search area is defined for each light-emitting point in the second captured image; Using the coarse location of each luminous point as the center, find the brightest point within the search area of the luminous point; If the position of the brightest point of the light-emitting point matches the position of the actual light-emitting point, the position of the brightest point is taken as the precise positioning of the light-emitting point.
[0011] In some embodiments, after determining the position of the brightest point as the precise location of the light-emitting point, the method further includes: Based on the preset dot matrix spacing of the second driving diagram, and combined with the precise positioning of the light-emitting points, the remaining light-emitting points in the second driving diagram are supplemented to obtain the precise positioning of each light-emitting point in the display device.
[0012] In some embodiments, the coarse positioning result also includes the effective detection area corresponding to the imaging device; The process of filling in the remaining light-emitting points in the second driving map based on the preset dot matrix spacing of the second driving map and the precise positioning of the light-emitting points includes: Based on the effective detection area corresponding to the shooting device, the effective detection area in the second driving diagram is determined; Based on the preset dot matrix spacing of the second driving map, and combined with the precise positioning of the light-emitting points, row and column recursion is performed to complete the remaining light-emitting points in the effective detection area of the second driving map.
[0013] In some embodiments, the combined field of view of the plurality of shooting devices can cover the entire display device.
[0014] In some embodiments, generating a feature map corresponding to the display device based on the position mapping relationship and the detection screen of the display device includes: For each detection mode, based on the position mapping relationship, the pixel value of the light-emitting point in the display device is found from the detection screen, and a feature map matching the resolution of the display device is generated.
[0015] According to a second aspect of this application, a detection device for a display device is provided, comprising: An image acquisition module is used to acquire a driving map that matches the resolution of the display device to be detected, and to acquire images captured by multiple imaging devices when the driving map is displayed on the display device, so as to obtain an image corresponding to the driving map; A coarse positioning module is used to perform coarse positioning based on the driving diagram and the corresponding captured image to obtain a coarse positioning result; wherein, the coarse positioning result includes the coarse positioning of the light-emitting points of the display device; The fine positioning module is used to perform fine positioning based on the coarse positioning result, the driving diagram and the corresponding photographic image of the driving diagram, so as to obtain the fine positioning of the light-emitting point of the display device. The position mapping module is used to establish a position mapping relationship between the display device and the captured image based on the precise positioning of the light-emitting point; The feature map generation module is used to generate a feature map corresponding to the display device based on the position mapping relationship and the detection screen of the display device; The defect detection module is used to perform defect detection based on the feature map and obtain the defect detection result of the display device.
[0016] According to a third aspect of this application, a detection system for a display device is provided, comprising: The display device to be tested is used to display a driving graph that matches the resolution of the display device; A shooting device is used to capture an image when the driving diagram is displayed on the display device, and to obtain a captured image corresponding to the driving diagram; A controller, connected to the imaging device, is used to execute the detection method for the display device as described above.
[0017] According to a fourth aspect of this application, an electronic device is provided, including a memory storing a plurality of instructions; a processor loading instructions from the memory to execute steps in a detection method for any of the display devices provided in the embodiments of this application.
[0018] According to a fifth aspect of this application, a computer-readable storage medium is provided, which stores a plurality of instructions adapted for loading by a processor to perform the steps in any of the detection methods for display devices provided in this application.
[0019] According to a sixth aspect of this application, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implement the steps in the detection method of any of the display devices provided in this application.
[0020] The display device detection method provided in this application obtains a driving image matching the resolution of the display device to be detected and a corresponding photographic image. Then, based on the driving image and the photographic image, coarse positioning is performed to determine the approximate position of each light-emitting point on the display device, achieving full coverage of the display panel and avoiding the loss of defect points due to only locating actual light-emitting points. Next, based on the approximate position of the display device, the precise position of all light-emitting points on the display device (including actual light-emitting points and non-light-emitting points, i.e., normal points and defect points) is further determined, achieving precise full coverage of normal points and defect points on the display device. Then, using the precise positioning of all light-emitting points on the display device, the positional mapping relationship between the display device and the photographic image is determined. This positional mapping relationship, along with the detection image of the display device, is used to generate a feature map corresponding to the display device. This feature map clearly presents the features of the detection image, ensuring the presentation quality of each light-emitting point feature, thereby improving the clarity of defect point presentation and ultimately improving the accuracy of identifying defects on the display device using the feature map.
[0021] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description
[0022] Figure 1 This is a schematic flowchart of a detection method for a display device provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a driving image capture provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a light-emitting dot matrix of a display device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a first driving diagram provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a second driving diagram provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of coarse positioning provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a boundary provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of a precise positioning method provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of a detection screen and a corresponding feature map provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of a detection device for a display device provided in an embodiment of the present invention; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0024] In related technologies, defect detection of display devices (such as light-emitting diodes (LEDs)) is generally based on traditional visual inspection methods, or uses deep learning AI models to identify and locate defects.
[0025] Traditional visual inspection methods generally require the use of various auxiliary images and lighting conditions to extract defects and filter interference. They are highly dependent on hardware structure and limited by the original imaging conditions of the display device, which cannot guarantee the stability of defect detection on the display device, and thus cannot guarantee the accuracy of defect detection.
[0026] Deep learning AI modeling methods require significant resource investment in sample collection, annotation, and model training. The detection capability of AI models is greatly affected by the comprehensiveness of defect samples. Furthermore, in the inspection of large-size LED display panels, there is a negative correlation between the detection capability of AI models and the required test time (TT), and achieving optimal performance in both areas is quite challenging.
[0027] Therefore, to address the aforementioned issues, and considering the relationship between the original image of the display device to be inspected and the precisely located information of each luminous point, a feature map generation method for automatic pattern inspection (API) of display devices based on machine vision is proposed. This feature map is generated based on panel features, and can comprehensively, accurately, and clearly acquire the imaging information of each luminous point on the display device. It maximizes the elimination of interference from non-detection areas and filters out complex backgrounds without affecting defect information. Thus, feature map-based defect detection of display devices can ensure defect detection accuracy, eliminates the need for multiple auxiliary images, has low hardware dependence, and does not require a large number of samples to train the AI model.
[0028] The feature map generation method described above, which is also the detection method for display devices, will be described in detail below. This method can be executed by an electronic device. For example... Figure 1 As shown, the detection method for the display device may include: S201. Obtain a driving diagram that matches the resolution of the display device to be tested, and obtain images captured by multiple imaging devices when the driving diagram is displayed on the display device, to obtain an image corresponding to the driving diagram.
[0029] In this embodiment of the application, the display device can display a driving map with the same resolution. Simply put, the light-emitting points on the display device emit light (e.g., Figure 3 (The diagram shows a schematic of the light-emitting dot matrix of the display device). Multiple shooting devices are triggered simultaneously to capture images of different areas of the display device, obtain clear information about the illuminated screen of the display device, and thus obtain the captured image corresponding to the driving image.
[0030] The resolution of a display device is the number of pixels in the horizontal and vertical directions. Resolution, also known as resolution or image resolution, determines the fineness of the details in a bitmap image and is usually used as a measure of the detail and clarity displayed in images, videos, or display devices.
[0031] In some embodiments, the display device is placed below multiple shooting devices, with the screens of the shooting devices and the display device plane being parallel at the same height (they remain parallel in the Z-axis direction). Taking a camera as an example, and the number of shooting devices being four, and the display device being a display panel, the positional relationship between the camera and the display panel is as follows: Figure 2 As shown.
[0032] Optionally, such as Figure 2 As shown, the cameras do not communicate with each other. After each camera acquires an image (such as a photograph), it can transmit the image to the device connected to that camera. Correspondingly, electronic devices can obtain images acquired by different cameras from various devices.
[0033] In some embodiments, in order to achieve comprehensive defect detection of the display device, the combined field of view (FOV) of the above-mentioned multiple shooting devices can cover the entire display device, so that the combined images captured by all shooting devices can cover the driving image displayed by the display device.
[0034] For example, the aforementioned multiple imaging devices include four cameras. The display device is placed on the detection carrier and transmitted to the field of view of the cameras. The FOV of the four cameras can fully cover the detection area of the display device, and the FOVs of the four cameras cover the upper left, lower left, upper right, and lower right areas of the panel, respectively. Among them, the FOVs of each camera partially overlap in the boundary area to ensure that the detection area of the display device is fully covered without omission. Each camera captures information of approximately 1 / 4 of the display device area.
[0035] In some embodiments, the driving graph may include a first driving graph (or Grid driving graph, Grid dot matrix driving graph) and a second driving graph (or Stripe driving graph); the first driving graph is represented as a grid dot matrix graph, and the second driving graph is represented as a stripe graph.
[0036] Grid matrix driven graph (e.g.) Figure 4 The grid dot matrix driver pattern (as shown) is automatically generated based on the panel resolution. It is a non-uniformly spaced dot matrix pattern that is symmetrical in the X and Y directions and centrally symmetrical. The grid dot matrix driver pattern consists of N rectangular blocks arranged in rows and columns in the horizontal and vertical directions, maintaining a fixed distance (non-close contact) from the boundary of the display device. The number of dots in the horizontal and vertical directions of the grid dot matrix driver pattern is fixed, while the spacing is dynamically adjusted according to the resolution. Optionally, N > 200 and N < 300. Additionally, the grid dot matrix driver pattern can be used to achieve coarse positioning of the light-emitting points.
[0037] Optionally, the electronic device can drive the Grid dot matrix to light up according to the resolution of the display device. Based on the horizontal and vertical resolution of the display device, the device is driven to display an image with a fixed number of Grid dots in the horizontal and vertical directions and an adjustable spacing according to the resolution.
[0038] Stripe driving graph (e.g.) Figure 5 The image shown is automatically generated based on the panel resolution. It is a green dot matrix pattern with rows and columns spaced proportionally (with a central luminous point with a certain row spacing). In addition, the Stripe driving map can be used to achieve precise positioning of the luminous points.
[0039] Optionally, the electronic device can drive the Stripe driver map to light up according to the resolution of the display device. Based on the horizontal and vertical resolution of the display device, the device is driven to display a green pixel map with proportionally spaced rows and columns.
[0040] It should be noted that the aforementioned display device can be a regular rectangle, without rounded corners or curved surfaces, and is a standard flat product. The display device can include large-size IT or TV panels, with resolutions typically ranging from 1920x1080, 2560x1440, or 2560x1080, among other common resolutions.
[0041] S202. Perform coarse positioning based on the driving diagram and the corresponding captured image to obtain the coarse positioning result.
[0042] The coarse positioning result includes the coarse positioning of the light-emitting point of the display device (LocateInfo1), which can represent the approximate position of the light-emitting point in the pixel coordinates of the captured image. It is used to narrow down the range, that is, to narrow down the search range of the subsequent fine positioning and eliminate invalid interference.
[0043] In this embodiment, the electronic device can use the driving diagram and the corresponding photographic image to initially locate the approximate position of each light-emitting point on the display device. These light-emitting points can include normal light-emitting points and abnormal light-emitting points, i.e., defect points, on the display device.
[0044] In some embodiments, as described above, the electronic device can perform coarse positioning based on the first driving image and the corresponding first captured image to obtain a coarse positioning result. The electronic device completes coarse dot matrix positioning based on the Grid dot matrix image and the corresponding captured image to determine the approximate position of each luminous point.
[0045] Optionally, the above coarse localization results may also include the effective detection area (ROI) corresponding to the imaging device.
[0046] In some embodiments, the above coarse positioning process may include: determining the affine transformation relationship between the first driving image and the first captured image based on the correspondence between the first driving image and the first captured image.
[0047] The scaling ratio between the first driving image and the first captured image is determined based on the area ratio between the first driving image and the first captured image. Based on the affine transformation relationship and scaling ratio, and combined with the preset dot matrix spacing of the second driving image, the coordinate mapping relationship between the second driving image and the second captured image is determined. Based on the coordinate mapping relationship, the coarse positioning of each light-emitting point on the display device is determined.
[0048] In this embodiment, the affine transformation relationship β (including translation, scaling, and rotation parameters) between the dot matrix rules (number of rows and columns, symmetry relationship) of the Grid driving graph and the extracted features of the first image corresponding to the Grid driving graph is compared to correct geometric deviations during the imaging process. In general, the affine transformation relationship represents the linear mapping relationship between the dot matrix coordinates (u, v) on the Grid driving graph and the dot matrix coordinates (x, y) on the corresponding first image. The formula used is: x = a×u + b×v + c; y = d×u + e×v + f; where (a, b, c, d, e, f) are the affine transformation parameters, which together constitute the transformation relationship β. Specifically, three non-collinear Grid points on the Grid driving graph are selected as reference points. In the image corresponding to the Grid driving graph, the imaging coordinates corresponding to these three reference points are found (by matching the shape and brightness features of the Grid light-emitting blocks through image recognition). These three sets of (u, v) and (x, y) are substituted into the affine transformation formula to obtain the parameters.
[0049] The electronic device can calculate the scaling ratio based on k = aqrt(Area_GridA / Area_origGrid); where Area_GridA represents the area of the first captured image corresponding to the first driving image, and Area_origGrid represents the area of the first driving image. The scaling ratio (or scaling factor) reflects the imaging scaling ratio.
[0050] The electronic device determines the actual spacing (StripeDist) of the dots on the Stripe driving map based on the scaling ratio and the preset dot spacing of the second driving map (i.e., the row and column spacing of the bright spots on the theoretical Stripe driving map), which is the product of the two. This converts the theoretical spacing of the Stripe driving map into the actual imaging spacing of the second image corresponding to the Stripe driving map.
[0051] Subsequently, the electronic device can establish a mapping relationship between the Stripe driving map and the corresponding captured image. For any luminous point (i.e., the theoretical luminous point) on the Stripe driving map, its theoretical mapped coordinates on the captured image corresponding to the Stripe driving map are calculated using the affine transformation relationship β and the adapted actual imaging spacing.
[0052] Then, the electronic device can use the theoretical mapped coordinates of the light-emitting point as the coarse location of the light-emitting point (e.g., Figure 6 (As shown).
[0053] Optionally, the process of determining the area ratio between the first driving image and the first captured image may include: The product of the number of dots in the first driving graph and the area of a single dot is calculated, and combined with the area occupied by the dot spacing within the first driving graph, the dot coverage area of the first driving graph is obtained; wherein, the area occupied by the dot spacing within the first driving graph is determined according to the preset dot spacing (GridDist) of the first driving graph.
[0054] Calculate the dot matrix coverage area of the first driving image and the coverage area of the first captured image corresponding to the first driving image to obtain the area ratio between the first driving image and the first captured image.
[0055] Among them, GridDist is the original design spacing between adjacent points on the Grid driving graph; The StripeDist above represents the row and column spacing of the highlights on the Stripe-driven graph.
[0056] In this embodiment, coarse localization provides an initial position reference for subsequent fine localization based on the Stripe-driven graph, avoiding inefficiency or positioning deviations caused by an excessively large search range during fine localization. Furthermore, the affine transformation relationship β and scaling factor calculations involved in coarse localization correct for geometric distortion and scaling errors during the imaging process, ensuring positioning accuracy.
[0057] In some embodiments, the light-emitting points involved in the coarse positioning described above can be light-emitting points within the effective detection area. Based on the effective detection area of the imaging device, the effective detection area in the second driving image corresponding to the imaging device is determined. Based on the affine transformation relationship and scaling ratio, the coordinate mapping relationship between the effective detection area in the second driving image and the second imaging image corresponding to the second driving image is determined.
[0058] In this context, the effective detection area in the second driving diagram corresponding to the shooting device is consistent with the effective detection area of the shooting device.
[0059] Optionally, the process of determining the effective detection area may include: Based on the scaling ratio and the preset pixel spacing between the edge of the first driving image and the boundary of the display device, the spacing between the boundary formed by the outermost dot matrix in the first captured image and the actual boundary of the display device is determined. The preset pixel spacing between the edge of the first driving image (e.g., the edge of the first driving image) is... Figure 7 The pixel pitch between the boundary ① shown and the boundary of the display device represents the pixel spacing between the dot matrix on the Grid driving map projected onto the display device and the actual edge of the panel (such as...). Figure 7The fixed spacing of the boundary shown in ②) is, for example, 5 pixels. Specifically, the electronic device can use expandDist = 5*aqrt(Area_GridA / Area_origGrid) to determine the spacing between the boundary formed by the outermost dot matrix in the first captured image and the actual boundary of the display device. Here, expandDist represents the spacing between the boundary formed by the outermost dot matrix in the first captured image and the actual boundary of the display device, and 5 represents the preset pixel spacing between the edge of the first driving image and the boundary of the display device.
[0060] Then, based on the position and spacing of the boundary formed by the outermost dot matrix in the first image, the position of the actual boundary of the display device is determined.
[0061] Subsequently, based on the actual boundary position of the display device and the first driving map, the effective detection area of the shooting device corresponding to the first driving map is determined.
[0062] In this embodiment of the application, by determining the ROI, interference from non-detection areas in the FOV of the shooting device that extend beyond the boundary of the display device is directly eliminated, and the data processing range is narrowed.
[0063] S203. Based on the coarse positioning results, the driving diagram and the corresponding photographic image, perform fine positioning to obtain the fine positioning of the light-emitting point of the display device.
[0064] In this embodiment, after coarse positioning, fine positioning can be performed using the coarse positioning result, the driving diagram, and the corresponding photographic image to fine-tune the coarse positioning of the light-emitting point, thereby determining the precise position information of the light-emitting point (such as an LED bead), which is to obtain the fine positioning of the light-emitting point (such as an LED bead). Figure 8 (As shown).
[0065] In some embodiments, as described above, the electronic device can perform fine positioning based on the coarse positioning result, the second driving map, and the second captured image corresponding to the second driving map to obtain the fine positioning of the light-emitting point of the display device.
[0066] Among them, the precise positioning of the light-emitting point of the display device refers to the accurate position of the light-emitting point on the captured image, that is, the position of the light-emitting point.
[0067] In this embodiment of the application, the electronic device can use the imaging of the Stripe driving map of the display device, that is, the captured image (i.e. the second captured image) corresponding to the Stripe driving map, to correct the coarse positioning of the light-emitting point in order to obtain the fine positioning of the light-emitting point.
[0068] In some embodiments, the process of implementing the above-mentioned precise positioning may include: Based on the imaging features of the second image, the position of the actual luminous point in the second image is determined; Based on the coarse positioning of the light-emitting points in the display device, a search area is defined for each light-emitting point in the second image; Using the coarse location of each luminous point as the center, find the brightest point within the search area of the luminous points; If the brightest point of the luminous point matches the actual position of the luminous point, the brightest point is used as the precise location of the luminous point.
[0069] In this embodiment, the electronic device extracts the pixel coordinates of all actual luminous points in the second captured image using algorithms such as contour detection and grayscale comparison, thereby obtaining the actual luminous point positions (StripeLocate) in the second captured image and forming a StripeLocate set. The StripeLocate of the luminous points serves to locate the actual pixel position of the display device.
[0070] Next, for each luminous point's coarse localization, that is, each coarse localization point (x1, y1) in the LocateInfo1 set, a search region (or pixel neighborhood) centered on (x1, y1) is defined in the second image, such as a 5x5 pixel neighborhood (i.e., horizontally from x1-2 to x1+2, and vertically from y1-2 to y1+2, for a total of 25 pixels). All pixels within the 5x5 neighborhood corresponding to the coarse localization point are traversed, the grayscale value of each pixel is read, and the pixel with the highest grayscale value, i.e., the brightest point, is selected and its position (X2, Y2) is recorded.
[0071] Based on the principle of proximity, the brightest point (X2, Y2) is matched with the position of the actual light-emitting point in the StripeLocate set. If the position of the brightest point matches the position of an actual light-emitting point (meaning the distance between the actual light-emitting point and the highest point is less than or equal to a preset distance), the electronic device can use the position of the brightest point (LocateInfo2) as the precise position of the light-emitting point. Alternatively, the electronic device can also use the position of the actual light-emitting point that matches the position of the brightest point as the precise position of that light-emitting point.
[0072] If the position of the brightest point does not match the position of the actual light-emitting point, the electronic device will discard the light-emitting point, meaning that the coarse positioning point is an invalid positioning point.
[0073] Optionally, the coarse positioning based on the light-emitting points in the display device can be the light-emitting points within the ROI, excluding invalid positioning points.
[0074] The light-emitting points corresponding to the brightest points identified above are only a portion of the light-emitting points on the display device, not all of them. This excludes defective points; therefore, the electronic device needs to complete the remaining light-emitting points based on the preset dot matrix spacing of the second driving diagram and the precise positioning of the light-emitting points.
[0075] For example, an electronic device can use the coordinates of the brightest point in the LocateInfo2 set as the anchor point to extract the preset row and column spacing of the brightest point in the Stripe driving graph, which is the preset dot matrix spacing of the second driving graph, such as the horizontal spacing Sx and the vertical spacing Sy. Then, the electronic device can perform recursive row and column padding. For a given anchor point (x2, y2), the positions (i.e. coordinates) of the other emitting points in the same row are recursively calculated according to the horizontal spacing Sx, such as (x2+Sx, y2), (x2+2Sx, y2)...(x2-nSx, y2), until the horizontal boundary of the ROI is reached. Fill in the same column, and recursively advance the position of the light-emitting points in the same column according to the vertical spacing Sy, such as (x2,y2+Sy), (x2,y2+2Sy)...(x2,y2-mSy), until the vertical boundary of the ROI is reached.
[0076] Additionally, it is understandable that all points in the recursion must fall within the ROI; points outside the ROI are discarded. The remaining luminous points mentioned above, along with LocateInfo2, can be considered as theoretical luminous points (i.e., LocateInfo3).
[0077] In this embodiment, the remaining light-emitting points that are recursively supplemented include actual light-emitting points that were not selected as LocateInfo2 in StripeLocate and defective points that did not emit light, ensuring that all theoretical light-emitting points on the display device (such as ROI) have corresponding positions, thus achieving full coverage of all points.
[0078] S204. Establish the positional mapping relationship between the display device and the captured image based on the precise positioning of the light-emitting point.
[0079] The position mapping relationship refers to the binding relationship between the physical coordinates of each light-emitting point (or theoretical light-emitting point) in the display device and the pixel coordinates in the captured image with the same resolution as the display device, i.e., the correspondence relationship.
[0080] In this embodiment of the application, for each light-emitting point in the display device, the electronic device can establish a position mapping relationship between the coordinates (i.e., physical coordinates) of the light-emitting point on the display device and the precise positioning of the light-emitting point, so as to establish a correspondence between the light-emitting point on the display device and the pixel point on the captured image.
[0081] S205. Based on the position mapping relationship and the detection screen of the display device, generate a feature map corresponding to the display device.
[0082] In this embodiment, the detection screen of the display device is matched with the resolution of the display device. Simply put, the pixel coordinates of the detection screen are consistent with those of the aforementioned captured image. The electronic device can utilize the position mapping relationship and the detection screen of the display device to determine the pixel points of the detection screen that match the light-emitting points in the display device, thereby determining the pixel values corresponding to each light-emitting point in the display device, obtaining effective information reflecting the detection screen, and maximizing the elimination of interference in the feature map.
[0083] In some embodiments, the above-mentioned detection screen can be one or more. For each detection mode, the corresponding detection screen (e.g.) Figure 9 As shown in (a), based on the position mapping relationship, the pixel values of the light-emitting points in the display device are found from the detection screen corresponding to the detection mode, and a feature map matching the resolution of the display device is generated (e.g., ...). Figure 9 (As shown in (b)). Specifically, for each pixel in the detection image, the position mapping relationship is used to determine the light-emitting point on the display device corresponding to the position of the pixel, and the pixel value of the pixel is used as the pixel value of the corresponding light-emitting point, which is the pixel value of the pixel on the feature map.
[0084] In this embodiment of the application, by utilizing the detection images corresponding to different detection modes, feature maps corresponding to different detection modes can be generated. Thus, defect detection can be performed using the feature maps corresponding to different detection modes to obtain defects under different detection modes, ensuring the comprehensiveness of defect detection.
[0085] S206. Perform defect detection based on feature maps to obtain the defect detection results of the display device.
[0086] In this embodiment, after obtaining the feature map, the electronic device can directly compare the feature map with the standard template (i.e., the standard feature map corresponding to the display device) to determine defects on the feature map, such as brightness defects and positional defects. Compared with directly detecting the original captured image of the display device, defect detection through the feature map eliminates the influence of background interference and geometric distortion, significantly improving the accuracy and robustness of defect detection.
[0087] For example, an electronic device can compare the grayscale values of pixels on a feature map with the grayscale values of pixels on a standard template to determine whether the grayscale values of pixels on the feature map are normal, that is, whether the light-emitting point corresponding to the pixel on the display device can emit light normally, thereby realizing brightness defect detection.
[0088] This application also provides a detection device for a display device, which is used to perform any of the aforementioned detection methods for a display device.
[0089] For example, such as Figure 10 As shown, the detection device 600 of the display device may include: The image acquisition module 610 is used to acquire a driving map that matches the resolution of the display device to be detected, and to acquire images captured by multiple shooting devices when the driving map is displayed on the display device, so as to obtain a shooting map corresponding to the driving map; The coarse positioning module 620 is used to perform coarse positioning based on the driving diagram and the corresponding captured image to obtain a coarse positioning result; wherein, the coarse positioning result includes the coarse positioning of the light-emitting points of the display device; The fine positioning module 630 is used to perform fine positioning based on the coarse positioning result, the driving diagram and the corresponding photographic image of the driving diagram, so as to obtain the fine positioning of the light-emitting point of the display device. The position mapping module 640 is used to establish a position mapping relationship between the display device and the captured image based on the precise positioning of the light-emitting point; The feature map generation module 650 is used to generate a feature map corresponding to the display device based on the position mapping relationship and the detection screen of the display device; The defect detection module 660 is used to perform defect detection based on the feature map to obtain the defect detection result of the display device.
[0090] Figure 11 This is a schematic diagram of the structure of an electronic device provided in some embodiments of this application. Figure 11 The dashed line in the text indicates that the unit or module is optional. Figure 11 The electronic device 700 can be used to implement the methods described in the above method embodiments. The electronic device 700 can be a device with data processing capabilities, such as a chip, terminal device, or server.
[0091] Electronic device 700 may include one or more processors 710. The processor 710 can support the electronic device 700 in implementing the methods described in the preceding method embodiments. The processor 710 can be a general-purpose processor or a special-purpose processor. For example, the processor can be a Central Processing Unit (CPU). Alternatively, the processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0092] The electronic device 700 may also include one or more memories 720. Computer programs are stored on the memories 720. The memories 720 may be independent of the processor 710 or integrated into the processor 710.
[0093] Electronic device 700 may also include transceiver 730. Processor 710 can communicate with other devices (such as industrial cameras) or chips via transceiver 730. For example, processor 710 can send and receive data with other devices or chips via transceiver 730.
[0094] The computer program in memory 720 can be executed by processor 710, causing processor 710 to perform the following steps: A driving diagram matching the resolution of the display device to be detected is obtained, and images captured by multiple imaging devices when the driving diagram is displayed on the display device are obtained, to obtain an image corresponding to the driving diagram; Coarse positioning is performed based on the driving diagram and the corresponding captured image to obtain a coarse positioning result; wherein, the coarse positioning result includes the coarse positioning of the light-emitting points of the display device; Based on the coarse positioning result, the driving diagram and the corresponding photographic image, fine positioning is performed to obtain the fine positioning of the light-emitting point of the display device. Establish a positional mapping relationship between the display device and the captured image based on the precise positioning of the light-emitting points; Based on the position mapping relationship and the detection screen of the display device, a feature map corresponding to the display device is generated; Defect detection is performed based on the feature map to obtain the defect detection result of the display device.
[0095] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0096] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which is loaded by a processor to execute the steps described in the above-described method embodiments of this application. For example, the computer program loaded by the processor can execute the display device detection method as described above.
[0097] For details on the implementation of each of the above operations / steps, please refer to the previous examples, which will not be repeated here.
[0098] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0099] Since the computer program stored in the computer-readable storage medium can execute the steps in any of the above method embodiments provided in the embodiments of this application, the beneficial effects that the methods described in any of the above method embodiments can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0100] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above embodiments.
[0101] This application embodiment also provides a detection system for a display device, including: The display device to be tested; A shooting device is used to capture an image when the driving diagram is displayed on the display device, and to obtain a captured image corresponding to the driving diagram; A controller, connected to the imaging device, is used to execute the detection method for the display device as described above.
[0102] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0103] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0104] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.
[0105] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.
Claims
1. A method for detecting a display device, characterized in that, include: A driving diagram matching the resolution of the display device to be detected is obtained, and images captured by multiple imaging devices when the driving diagram is displayed on the display device are obtained, to obtain an image corresponding to the driving diagram; Coarse positioning is performed based on the driving diagram and the corresponding captured image to obtain a coarse positioning result; wherein, the coarse positioning result includes the coarse positioning of the light-emitting points of the display device; Based on the coarse positioning result, the driving diagram, and the corresponding photographic image of the driving diagram, fine positioning is performed to obtain the fine positioning of the light-emitting point of the display device. Establish a positional mapping relationship between the display device and the captured image based on the precise positioning of the light-emitting points; Based on the position mapping relationship and the detection screen of the display device, a feature map corresponding to the display device is generated; Defect detection is performed based on the feature map to obtain the defect detection result of the display device.
2. The method according to claim 1, characterized in that, The driving graph includes a first driving graph and a second driving graph; the first driving graph is represented as a grid dot matrix graph, and the second driving graph is represented as a stripe graph. The step of performing coarse localization based on the driving map and the corresponding captured image to obtain a coarse localization result includes: Coarse positioning is performed based on the first driving map and the first captured image corresponding to the first driving map to obtain the coarse positioning result; The step of performing fine positioning based on the coarse positioning result, the driving diagram, and the corresponding photographic image of the driving diagram to obtain the fine positioning of the light-emitting point of the display device includes: Based on the coarse positioning result, the second driving map, and the second shooting image corresponding to the second driving map, fine positioning is performed to obtain the fine positioning of the light-emitting point of the display device.
3. The method according to claim 2, characterized in that, The step of performing coarse localization based on the first driving map and the first captured image corresponding to the first driving map to obtain the coarse localization result includes: Based on the correspondence between the first driving image and the first captured image, the affine transformation relationship between the first driving image and the first captured image is determined. The scaling ratio between the first driving image and the first captured image is determined based on the area ratio between the first driving image and the first captured image. Based on the affine transformation relationship and the scaling ratio, and combined with the preset dot matrix spacing of the second driving image, the coordinate mapping relationship between the second driving image and the second captured image is determined; Based on the coordinate mapping relationship, the coarse positioning of each light-emitting point on the display device is determined.
4. The method according to claim 3, characterized in that, The method further includes: Based on the scaling ratio and the pixel spacing between the edge of the first driving image and the boundary of the display device, the spacing between the boundary formed by the outermost dot matrix in the first captured image and the actual boundary of the display device is determined. Based on the position of the boundary formed by the outermost dot matrix in the first captured image and the spacing, the position of the actual boundary of the display device is determined; Based on the actual boundary position of the display device and the first driving map, the effective detection area of the shooting device corresponding to the first driving map is determined. Determining the coordinate mapping relationship between the second driving image and the second captured image based on the affine transformation relationship and the scaling ratio includes: Based on the effective detection area of the shooting device, the effective detection area in the second driving graph corresponding to the shooting device is determined. Based on the affine transformation relationship and the scaling ratio, the coordinate mapping relationship between the effective detection area in the second driving map and the second shooting map corresponding to the second driving map is determined.
5. The method according to claim 3 or 4, characterized in that, Before determining the scaling ratio between the first driving image and the first captured image based on the area ratio between the first driving image and the first captured image, the method further includes: The product of the number of dots in the first driving graph and the area of a single dot is calculated, and combined with the area occupied by the dot spacing in the first driving graph, the dot coverage area of the first driving graph is obtained; wherein, the area occupied by the dot spacing in the first driving graph is determined according to the preset dot spacing of the first driving graph. Calculate the dot matrix coverage area of the first driving image and the coverage area of the first captured image corresponding to the first driving image to obtain the area ratio between the first driving image and the first captured image.
6. The method according to claim 2, characterized in that, The step of performing fine positioning based on the coarse positioning result, the second driving map, and the second captured image corresponding to the second driving map to obtain the fine positioning of the light-emitting point of the display device includes: Based on the imaging features of the second image, the position of the actual light-emitting point in the second image is determined; Based on the coarse positioning of the light-emitting points in the display device, a search area is defined for each light-emitting point in the second captured image; Using the coarse location of each luminous point as the center, find the brightest point within the search area of the luminous point; If the position of the brightest point of the light-emitting point matches the position of the actual light-emitting point, the position of the brightest point is taken as the precise positioning of the light-emitting point.
7. The method according to claim 6, characterized in that, After determining the position of the brightest point as the precise location of the luminous point, the method further includes: Based on the preset dot matrix spacing of the second driving diagram, and combined with the precise positioning of the light-emitting points, the remaining light-emitting points in the second driving diagram are supplemented to obtain the precise positioning of each light-emitting point in the display device.
8. The method according to claim 7, characterized in that, The coarse positioning result also includes the effective detection area corresponding to the imaging device; The process of filling in the remaining light-emitting points in the second driving map based on the preset dot matrix spacing of the second driving map and the precise positioning of the light-emitting points includes: Based on the effective detection area corresponding to the shooting device, the effective detection area in the second driving diagram is determined; Based on the preset dot matrix spacing of the second driving map, and combined with the precise positioning of the light-emitting points, row and column recursion is performed to complete the remaining light-emitting points in the effective detection area of the second driving map.
9. The method according to any one of claims 1-4, 6 or 7, characterized in that, The combined field of view of the multiple shooting devices can cover the entire display device.
10. The method according to any one of claims 1-4, 6 or 7, characterized in that, The step of generating a feature map corresponding to the display device based on the position mapping relationship and the detection screen of the display device includes: For each detection mode, based on the position mapping relationship, the pixel value of the light-emitting point in the display device is found from the detection screen, and a feature map matching the resolution of the display device is generated.