Optical detection method of liquid crystal display panel

Abstract

The invention belongs to the technical field of automatic optical detection, and particularly discloses an optical detection method of a liquid crystal display panel. The method comprises the following steps: firstly, constructing a recognition feature vertex from a current frame image of a liquid crystal display panel; deriving a transformation matrix based on the coordinates of the feature vertexes in the current frame of imaging and the coordinates of the standard feature vertexes; the jitter degree of the current frame imaging is represented by adopting the distance between the same feature vertexes in multiple frames of imaging including the current frame imaging and adjacent in time sequence; and finally, judging whether the jitter degree exceeds a threshold value or not, if so, correcting the defect pixel coordinates by using the transformation matrix and then outputting, otherwise, directly outputting the defect pixel coordinates. The jitter correction method is perfectly integrated into the optical detection system of the liquid crystal display panel, the calculation cost is low, and the application range is wide.

Description

Technical Field

[0001] This application belongs to the field of automatic optical inspection technology, and more specifically, relates to an optical inspection method for liquid crystal display panels. Background Technology

[0002] Automated Optical Inspection (AOI) is a technology that uses optical imaging and image processing techniques to automatically inspect products. It replaces the human eye, using high-precision cameras and powerful computer algorithms to quickly, stably, and objectively check the appearance quality and assembly correctness of products. In the manufacturing process of LCD panels, the use of AOI technology for defect detection is already widespread. In recent years, as customers have become increasingly stringent in their defect detection specifications for LCD panels, the requirements for camera imaging during the inspection process have become increasingly demanding. Whether the optical inspection equipment experiences jitter or exceeds the acceptable range has a significant impact on the image quality of the LCD panel, and further affects the reliability of defect detection in the subsequent product inspection stages.

[0003] To address the impact of optical inspection equipment jitter on imaging during the optical inspection of LCD panel products, common methods currently include: adding a vibration meter to monitor the inspection equipment in real time during the optical inspection process, such as... Figure 1 As shown, this method can assist in monitoring device jitter, but it requires additional equipment costs and cannot accurately correlate detected device jitter with imaging jitter. Training an image detection model based on a neural network model to detect images is computationally expensive, and its accuracy is difficult to guarantee when dealing with LCD panels of different sizes. Therefore, a simpler and more adaptable imaging jitter detection method is urgently needed. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the prior art, this application provides an optical inspection method for liquid crystal display panels, which aims to solve the technical problem of high cost of imaging jitter detection in the optical inspection process of liquid crystal display panels.

[0005] To achieve the above objectives, in a first aspect, this application provides an optical inspection method for a liquid crystal display panel, comprising:

[0006] Optical detection is performed on the current frame image of the liquid crystal display panel to obtain the coordinates of defective pixels; and a region of interest is obtained from the current frame image, and the edges of the region of interest are fitted with straight lines to obtain feature edges; the intersection of the feature edges obtains feature vertices; The transformation matrix is ​​derived based on the coordinates of the feature vertices and the standard feature vertices in the current frame imaging; and the distance between the same feature vertices in the temporally adjacent multi-frame imaging is used to characterize the jitter of the current frame imaging, wherein the multi-frame imaging includes the current frame imaging. If the degree of jitter exceeds a threshold, the defective pixel coordinates are corrected using the transformation matrix and then output; otherwise, the defective pixel coordinates are output directly.

[0007] Preferably, the transformation matrix is ​​obtained through the following method: Let the transformation matrix be a 3×3 homography matrix; The product of the coordinates of the feature vertex and the transformation matrix is ​​proportional to the coordinates of the standard feature vertex in the homogeneous coordinate system. The transformation matrix is ​​obtained by solving the proportional relationship using the least squares method.

[0008] Preferably, the standard feature vertices are obtained through the following method: When the optical inspection equipment is stationary, a standard image of the liquid crystal display panel is acquired; Obtain the standard region of interest from the standard image; Standard feature edges are obtained by fitting straight lines to the edges of the standard region of interest. The standard feature vertices are obtained by the intersection of the standard feature edges.

[0009] Preferably, the defective pixel coordinates are corrected using the transformation matrix, specifically by: mapping the defective pixel coordinates to a homogeneous coordinate system to obtain homogeneous defective pixel coordinates; obtaining the corrected homogeneous defective pixel coordinates by the product of the homogeneous defective pixel coordinates and the transformation matrix; and projecting the corrected homogeneous defective pixel coordinates onto a two-dimensional coordinate system to obtain the corrected defective pixel coordinates.

[0010] Preferably, the number of the feature vertices is one or more.

[0011] Preferably, the feature vertex is located outside and / or inside the region of interest.

[0012] Preferably, the jitter level of the current frame image is obtained through the following method: Obtain the coordinates of the first feature vertex in each of the multi-frame images, and use the maximum distance between the coordinates to characterize the jitter distance of the first feature vertex; Iteratively obtain the jitter distance of all feature vertices in the current frame image, and use the maximum jitter distance to characterize the jitter degree of the current frame image.

[0013] Preferably, the region of interest is obtained by performing grayscale conversion, binarization thresholding, morphological processing, and maximum contour filtering on the image sequentially.

[0014] Preferably, in the temporally adjacent multi-frame imaging, the acquisition interval of adjacent frames is the same.

[0015] In a second aspect, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0016] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) This application integrates a jitter detection and correction method into the optical inspection system of a liquid crystal display panel. The principle is to construct and identify feature vertices in the imaging process during optical inspection, and then perform jitter detection and correction based on the feature vertices. In this application, the identification of feature vertices does not require the construction of a complex image recognition model, resulting in lower computational overhead. Moreover, it does not require additional hardware equipment, making it simpler than existing methods. The technical solution of this application can be integrated simply by upgrading the software on the basis of the existing optical inspection equipment.

[0017] (2) In the jitter detection and correction technology of this application, the number and position of the feature vertices to be constructed and identified are not limited. The feature vertices can be inside the region of interest, outside the region of interest, or both outside and inside the region of interest at the same time. Therefore, it has great adaptability and is applicable to all liquid crystal display panels with intersecting straight edges, such as right-angle screens, rounded corner screens, notch screens, punch-hole screens, and even triangular screens, fan-shaped screens and other irregular liquid crystal display panels. It has a wide range of applications.

[0018] (3) This application perfectly integrates the jitter detection and correction method with the optical inspection of the liquid crystal display panel. It performs jitter detection and correction on the imaging that requires optical inspection at the same time. By introducing a homogeneous coordinate system and linearizing it through dimension increase, the complex nonlinear jitter correction problem is transformed into a linear algebra problem that can be solved stably, while maintaining the complete expressive ability of geometric transformation. This can better ensure the accuracy of product inspection and avoid production accidents. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of an existing optical inspection method for liquid crystal display panels.

[0020] Figure 2 This is a schematic flowchart of an optical inspection method for a liquid crystal display panel provided in an embodiment of this application.

[0021] Figure 3 This is one of the original camera images of the liquid crystal display panel provided in the embodiments of this application.

[0022] Figure 4 This is a schematic flowchart of a method for obtaining the ROI region of a liquid crystal display panel from raw camera images, provided in an embodiment of this application.

[0023] Figure 5 This is a schematic diagram of the ROI area of ​​the liquid crystal display panel provided in the embodiments of this application.

[0024] Figure 6 This is a schematic diagram of the four vertex positions of the ROI region provided in the embodiments of this application.

[0025] Figure 7 This is a schematic diagram showing multiple overlapping ROI regions provided in an embodiment of this application.

[0026] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first vertex" and "second vertex," etc., are used to distinguish different vertices, not to describe a specific order of vertices.

[0029] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0030] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0031] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple feature vertices means two or more feature vertices, multiple positions means two or more positions, etc.

[0032] First, the technical terms involved in the embodiments of this application will be introduced.

[0033] AOI (Automated Optical Inspection) is a technology that uses optical imaging and image processing techniques to automatically inspect products.

[0034] ROI (Region of Interest) refers to the key information area that is focused on in image processing technology.

[0035] The embodiments of this application are described below with reference to the accompanying drawings.

[0036] This application uses a rounded-corner punch-hole screen as the object of imaging jitter detection to further describe the technical solution of this application. It should be noted that the implementation object of this application method is not limited to rounded-corner punch-hole screens; this application method is applicable to right-angle screens, notch screens, and even irregularly shaped liquid crystal display panels such as triangular screens and fan-shaped screens. Figure 2 As shown, the embodiments of this application specifically include the following steps: (1) The production line transfers the rounded corner punch-hole screen to the product inspection table. The rounded corner punch-hole screen is lit up and displays a white image with maximum brightness. Selecting to display a white image here can maximize the differentiation between ROI and background in the image. Furthermore, the higher the grayscale, the more accurate the subsequent acquisition of ROI and the straight line fitting of ROI boundaries will be.

[0037] (2) Using the same time interval, images were taken of the rounded corner punch screen on the product testing platform. The imaging effect is as follows: Figure 3 As shown. In this embodiment, the imaging interval is 0.5 seconds.

[0038] Optical inspection is performed on the current frame image of the liquid crystal display panel to obtain the coordinates of defective pixels. .

[0039] (3) Identify the ROIs in the image, and follow the specific steps as follows: Figure 4 As shown: Each image is sequentially subjected to grayscale conversion, binarization thresholding, morphological processing, and maximum contour filtering. The Regions of Interest (ROIs) in each image are obtained, as shown below. Figure 5 As shown.

[0040] (4) Identify multiple feature vertices of the ROI in the image, specifically: The least squares method is used to fit straight lines to the edges of the ROI, such as... Figure 6 As shown, four straight lines are obtained. , , and The four straight edges intersect each other to form the four feature vertices of the ROI. , , and .

[0041] (5) The transformation matrix is ​​derived based on the coordinates of the feature vertices and the standard feature vertices in the current frame image, specifically: (51) First obtain the coordinates of the standard feature vertices: When the optical inspection equipment is stationary, a standard image of the liquid crystal display panel is acquired; a standard region of interest is obtained from the standard image; the edges of the standard region of interest are fitted with straight lines to obtain standard feature edges; the intersection of the standard feature edges yields standard feature vertices. .

[0042] (52) Let the transformation matrix be a 3×3 homography matrix. : ; The feature vertex The coordinates and the transformation matrix The product of the standard feature vertices The coordinates of the given coordinates satisfy a proportional relationship in the homogeneous coordinate system: ; in, It is a feature vertex Two-dimensional coordinates, Standard feature vertex Two-dimensional coordinates; Further information The linear constraint equations of the elements in the system, for each pair of feature vertices ( Contribute two linear equations: ; ; Will The 9 elements are arranged into a vector Construct homogeneous linear systems ,in for The coefficient matrix.

[0043] Finally, the solution is obtained using the least squares method. The transformation matrix is ​​obtained. .

[0044] (6) Overlay and display multiple continuously acquired images (including the current frame image). In this embodiment, jitter detection uses three consecutive images. After the three images are overlaid and displayed, as shown... Figure 7 As shown, the same feature vertices are displayed in different positions: The top-left feature vertex is displayed at the following location: , and ; The lower left feature vertex is displayed at the following location: , and ; The lower right feature vertex is displayed at the following location: , and ; The top right feature vertex is displayed at the following location: , and .

[0045] Calculate the distance between different positions of vertices with the same feature: (61) Calculate the distance between different positions of the top-left feature vertex: Location and The distance between them is: ; Location and The distance between them is: ; Location and The distance between them is: ; (62) Calculate the distance between different positions of the lower left feature vertex: Location and The distance between them is: ; Location and The distance between them is: ; Location and The distance between them is: ; (63) Calculate the distances between different positions of the upper right feature vertex: Location and The distance between them is: ; Location and The distance between them is: ; Location and The distance between them is: ; (64) Calculate the distances between different positions of the lower right feature vertex: Location and The distance between them is: ; Location and The distance between them is: ; Location and The distance between them is: .

[0046] In the formula, The index indicates the position of the feature vertex, with the numerical subscript representing the position number. Indicates the position Coordinates, subscript Indicates the position coordinate.

[0047] In some embodiments, five consecutive images are used to calculate the distance between different positions of the same vertex; in other embodiments, two consecutive images are used. The more images used, the higher the accuracy, but the greater the computational cost. The specific number of images is determined based on actual needs and computing power.

[0048] (7) The maximum value of the distance is used to characterize the degree of image jitter: (71) Using the maximum value function Calculate the maximum distance between different positions of the top-left feature vertex of the ROI. : ; (72) Calculate the maximum distance between different positions of the lower left feature vertex of the ROI. : ; (73) Calculate the maximum distance between different positions of the lower right feature vertex of the ROI. : ; (74) Calculate the maximum distance between different positions of the top right feature vertex of the ROI. : ; (75) Calculate the maximum value of all distances. : ; (76) Adopt It represents the degree of jitter in the current frame of imaging.

[0049] In this embodiment, the degree of jitter is measured by the distance between the same feature vertices in multiple images. This method is not only easy to implement, but also suitable for most liquid crystal display panels. It is applicable to any liquid crystal display panel with intersecting straight edges, such as right-angle screens, rounded corner screens, notch screens, punch-hole screens, etc., and has a wide range of applications.

[0050] (8) Comparison And the preset jitter threshold, if If the jitter threshold is greater than the threshold value, the transformation matrix is ​​used. Correct the defect pixel coordinates Output the defective pixel coordinates directly, otherwise output them directly. Specifically: Defect pixel coordinates Mapping to a homogeneous coordinate system yields the homogeneous coordinates of the defect pixels. ; The homogeneous coordinates of the defective pixels and the transformation matrix The product yields the corrected homogeneous coordinates of the defect pixel: ; The homogeneous coordinates of the corrected defect pixels Projecting onto a two-dimensional coordinate system yields the corrected defect pixel coordinates. .

[0051] Currently, the commonly used methods for jitter detection in the optical inspection process of LCD panels include: The method of obtaining the vibration level of the device by adding a vibration meter requires additional hardware, which is costly, and the vibration level of the device is difficult to align with the vibration level of the detection image.

[0052] Image detection models based on neural networks are trained to detect images, but their detection computation costs are high, and the detection accuracy is difficult to guarantee when dealing with LCD display panels of different specifications.

[0053] In this embodiment, the imaging position is located by constructing feature vertices, and the displacement of the feature vertices is detected to characterize the imaging jitter. This method has low computational overhead and is suitable for a wide variety of screens. It can directly obtain the degree of imaging jitter, so suitable images can be selected for subsequent product quality inspection based on the degree of imaging jitter. Therefore, the method of this application can be well integrated into the product quality inspection process of liquid crystal display panels, better ensure the accuracy of product quality inspection, and avoid the occurrence of production accidents.

[0054] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0055] Based on the methods in the above embodiments, this application provides an electronic device, such as... Figure 8 As shown, the electronic device may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions stored in the memory to execute the methods described in the above embodiments.

[0056] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0057] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0058] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0059] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0060] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0061] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0062] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0063] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An optical inspection method for a liquid crystal display panel, characterized in that, include: Optical inspection is performed on the current frame image of the liquid crystal display panel to obtain the coordinates of defective pixels; The region of interest is obtained from the current frame imaging, and the edge of the region of interest is fitted with a straight line to obtain the feature edge line; The intersection of the feature edges forms a feature vertex; The transformation matrix is ​​derived based on the coordinates of the feature vertices and the standard feature vertices in the current frame imaging; and the distance between the same feature vertices in the temporally adjacent multi-frame imaging is used to characterize the jitter of the current frame imaging, wherein the multi-frame imaging includes the current frame imaging. If the degree of jitter exceeds a threshold, the defective pixel coordinates are corrected using the transformation matrix and then output; otherwise, the defective pixel coordinates are output directly.

2. The optical detection method according to claim 1, characterized in that, The transformation matrix is ​​obtained specifically through the following method: Let the transformation matrix be a 3×3 homography matrix; The product of the coordinates of the feature vertex and the transformation matrix is ​​proportional to the coordinates of the standard feature vertex in the homogeneous coordinate system. The transformation matrix is ​​obtained by solving the proportional relationship using the least squares method.

3. The optical detection method according to claim 1 or 2, characterized in that, The standard feature vertices are obtained specifically through the following methods: When the optical inspection equipment is stationary, a standard image of the liquid crystal display panel is acquired; Obtain the standard region of interest from the standard image; Standard feature edges are obtained by fitting straight lines to the edges of the standard region of interest. The standard feature vertices are obtained by the intersection of the standard feature edges.

4. The optical detection method according to claim 1, characterized in that, The defect pixel coordinates are corrected using the transformation matrix, specifically by: mapping the defect pixel coordinates to a homogeneous coordinate system to obtain the homogeneous coordinates of the defect pixel; obtaining the corrected homogeneous coordinates of the defect pixel by the product of the homogeneous coordinates of the defect pixel and the transformation matrix; and projecting the corrected homogeneous coordinates of the defect pixel onto a two-dimensional coordinate system to obtain the corrected defect pixel coordinates.

5. The optical detection method according to claim 1, characterized in that, The number of the feature vertices is one or more.

6. The optical detection method according to claim 1, characterized in that, The feature vertex is located outside and / or inside the region of interest.

7. The optical detection method according to claim 1, characterized in that, The degree of jitter in the current frame image is obtained through the following method: Obtain the coordinates of the first feature vertex in each of the multi-frame images, and use the maximum distance between the coordinates to characterize the jitter distance of the first feature vertex; Iteratively obtain the jitter distance of all feature vertices in the current frame image, and use the maximum jitter distance to characterize the jitter degree of the current frame image.

8. The optical detection method according to claim 1, characterized in that, The region of interest is obtained by performing grayscale conversion, binarization thresholding, morphological processing, and maximum contour filtering on the image in sequence.

9. The optical detection method according to claim 1, characterized in that, In the temporally adjacent multi-frame imaging, the acquisition interval of adjacent frames is the same.

10. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-9.

Images