Defect analysis method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a defect analysis method and device, electronic equipment and a storage medium, and relates to the technical field of machine vision. The defect analysis method comprises the following steps: in response to generation of a first defect detection result of any detection object in a target process, aiming at each second defect detection result, determining each second defect and the position of each first defect mapped in a target defect distribution diagram under the condition that at least one second defect detection result exists; for each second defect detection result, performing repeated defect analysis processing on each first defect to obtain a repeated defect analysis result; and removing repeated defects in the first defects to obtain an optimized defect detection result of the detection object in the target process. By applying the scheme provided by the embodiment of the invention, the problems that the production line is shut down and the production efficiency is influenced when each defect indicated by the defect detection result of any process is only a repeated defect can be effectively solved.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a defect analysis method, device, electronic device, and storage medium. Background Art

[0002] During the production process, objects such as liquid crystal substrates and screens undergo multiple processing steps. After each process, the object is inspected for defects to obtain the defect detection results for that process. Defect detection can be performed by performing image analysis on captured target images of the object.

[0003] In related technologies, if the defect detection result indicates that the inspection object has at least one defect, the production line will be controlled to shut down; and after the shutdown, the inspection object with defects is re-analyzed to obtain the analysis result, and the inspection object with defects can be released or scrapped according to the analysis result. Specifically, the re-analysis can be for each defect of the inspection object, and whether it is a tolerable defect (i.e., a defect that does not affect the product function and can be ignored) or an intolerable defect (i.e., a defect that affects the product function and cannot be ignored) is analyzed manually or by a defect analysis algorithm; and if the inspection object only has tolerable defects, the inspection object can continue to flow into the next process; if the analysis shows that the inspection object has intolerable defects, the inspection object can be scrapped.

[0004] It is understood that, with the exception of the first process, any defect indicated by the defect detection results in any process could be a newly added defect in that process or a recurring defect, i.e., a tolerable defect caused by processing in a preceding process. Therefore, if the defects indicated by the defect detection results in any process are solely recurring defects, the relevant technology will cause production line downtime, which will undoubtedly affect production efficiency. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a defect analysis method, apparatus, electronic device, and storage medium to effectively address the problem of production line downtime and production efficiency being affected when the defects indicated by defect detection results in any process are only repeated defects. The specific technical solution is as follows:

[0006] In a first aspect, the present application provides a defect analysis method, the method comprising:

[0007] In response to generating a first defect detection result indicating the presence of a defect for any inspection object in a target process, determining whether at least one second defect detection result exists for the inspection object; wherein the second defect detection result is a defect detection result indicating the presence of a defect for the inspection object in a process preceding the target process;

[0008] If present, for each second defect detection result, determining the position of each second defect indicated by the second defect detection result mapped in a target defect distribution map, and determining the position of each first defect indicated by the first defect detection result mapped in the target defect distribution map; wherein the target defect distribution map is an image representing defect distribution in an object area of ​​the inspection object;

[0009] For each second defect detection result, based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, performing repeated defect analysis processing on each first defect to obtain a repeated defect analysis result;

[0010] The repeated defects indicated by the obtained repeated defect analysis results are removed from the first defects to obtain the optimized defect detection result of the inspection object in the target process.

[0011] In a second aspect, the present application provides a defect analysis device, comprising:

[0012] a first determining module configured to, in response to generating a first defect detection result indicating the presence of a defect for any inspection object in a target process, determine whether at least one second defect detection result exists for the inspection object; wherein the second defect detection result is a defect detection result indicating the presence of a defect for the inspection object in a process preceding the target process;

[0013] a second determining module configured to, for each second defect detection result, determine, if present, a position of each second defect mapped by the second defect detection result in a target defect distribution map, and to determine a position of each first defect mapped by the first defect detection result in the target defect distribution map; wherein the target defect distribution map is an image representing defect distribution in an object area of ​​the inspection object;

[0014] an analysis module configured to, for each second defect detection result, perform a repeated defect analysis process on each first defect based on a position of each first defect in the target defect distribution map and a position of each second defect indicated by the second defect detection result in the target defect distribution map, to obtain a repeated defect analysis result;

[0015] The deduplication module is used to remove the repeated defects indicated by the repeated defect analysis results from the first defects to obtain the optimized defect detection results of the detection object in the target process.

[0016] In a third aspect, the present application provides an electronic device, comprising:

[0017] Memory for storing computer programs;

[0018] The processor is configured to implement any of the above-mentioned defect analysis methods when executing a program stored in the memory.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the above-mentioned defect analysis methods.

[0020] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described defect analysis methods.

[0021] Beneficial effects of the embodiments of the present application:

[0022] The solution provided by the embodiment of the present application is to determine whether there is at least one second defect detection result for any detection object in the target process in response to generating a first defect detection result that characterizes the existence of defects. If there is, it means that there is a high probability of repeated defects in the first defect detection result. At this time, for each second defect detection result, the position of each second defect indicated by the second defect detection result mapped in the target defect distribution map can be determined, and the position of each first defect indicated by the first defect detection result mapped in the target defect distribution map can be determined; for each second defect detection result, based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, a repeated defect analysis process is performed on each first defect to obtain a repeated defect analysis result, and the repeated defects in each first defect are determined; the repeated defects in each first defect indicated by the repeated defect analysis result are removed, and the optimized defect detection result of the detection object in the target process can be obtained.

[0023] Since there are no repeated defects in the optimized defect detection results of the target process, when analyzing whether the production line needs to be shut down according to the optimized defect detection results, if the optimized defect detection results of the target process indicate that there are no defects, it means that the detection object has not caused defects in the target process and will not cause the production line to be shut down. If the optimized defect detection results of the target process indicate that there are defects, it will cause the production line to be shut down, but the reason for the production line shutdown at this time is that the target process has caused defects to the detection object. Therefore, by utilizing the optimized defect detection results of the target process, it is possible to effectively reduce the situation where the defects indicated by the defect detection results are only repeated defects, resulting in production line shutdowns. It can be seen that the solution of the present application can effectively solve the problem of production line shutdowns and production efficiency impact when the defects indicated by the defect detection results of any process are only repeated defects.

[0024] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0026] Figure 1 A schematic diagram of a defect analysis method provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of an exemplary production line provided in this application;

[0028] Figure 3 A schematic diagram of a defect detection result provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of a target defect distribution map provided in an embodiment of the present application;

[0030] Figure 5 Schematic diagram of target images of three different processes provided in the embodiments of the present application;

[0031] Figure 6 A schematic diagram of establishing a designated coordinate system for detection objects in target images of three different processes provided in an embodiment of the present application;

[0032] Figure 7 A schematic diagram of establishing a specified coordinate system provided in an embodiment of the present application;

[0033] Figure 8 The detection object provided in the embodiment of this application is Figure 2 A schematic diagram of target images captured at each production process of a production line;

[0034] Figure 9a A schematic diagram describing a specified coordinate system provided in an embodiment of the present application;

[0035] Figure 9b Another schematic diagram for describing a specified coordinate system provided in an embodiment of the present application;

[0036] Figure 10 Schematic diagram of defect distribution diagrams for three different processes provided in an embodiment of the present application;

[0037] Figure 11 Another schematic diagram of establishing a specified coordinate system provided in an embodiment of the present application;

[0038] Figure 12a A schematic diagram of determining the rotation angle of an image coordinate system provided in an embodiment of the present application;

[0039] Figure 12b A schematic diagram of another method for determining the rotation angle of an image coordinate system provided in an embodiment of the present application;

[0040] Figure 13 A schematic diagram of a defect analysis device provided in an embodiment of the present application;

[0041] Figure 14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0043] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing the manufacturer's production information are all carried out with the manufacturer's authorization.

[0044] The following are the professional terms involved in this application:

[0045] Liquid crystal substrate: The raw material used to manufacture liquid crystal display panels, which is processed into liquid crystal display panels after cleaning, coating, photolithography and other processes.

[0046] Missed inspection: NG (Non Good, referring to products that do not meet design specifications, process requirements or quality standards) parts are mistakenly inspected as OK parts (referring to products that meet design specifications, process requirements or quality standards). This is a serious quality accident and there is zero tolerance.

[0047] Overlay: Overlay shape thumbnails of the same size based on a fixed point.

[0048] Overlay image duplication judgment: After the same LCD substrate is inspected in different processes, the defect distribution thumbnails obtained are overlaid and the duplication judgment is performed based on the defect position and area overlap after overlay.

[0049] Defect distribution: By overlaying the defect thumbnails of multiple LCD substrates, a defect distribution heat map can be obtained, which can be used to analyze and study the defect distribution patterns and improve production line quality.

[0050] Heat map: Displays some characteristic areas and their proportions in a specially highlighted form. It can be used in industry to display defect distribution and study and analyze the factors causing defects.

[0051] The following is an example of a detailed introduction to defect detection in a production scenario where the production object is a liquid crystal substrate. The current solution for defect detection of liquid crystal substrates is as follows:

[0052] A defect detection system is installed on the LCD substrate production line to perform defect detection. For parts with NG test results, the production line is shut down. However, for parts with OK test results (referring to products that meet design specifications, process requirements or quality standards), the production line does not stop and directly enters the subsequent process. If the detection result is inaccurate and an OK part is mistakenly detected as an NG part, the production line will be shut down, which will affect production efficiency but will not cause quality problems. However, if an NG part is mistakenly detected as an OK part, that is, missed detection, it will be a serious quality problem. The production process of an LCD substrate involves many processes, and generally a defect detection system is deployed after each process. Some detected defects are manually or algorithmically confirmed as tolerable defects or features that are easily mistakenly detected as defects, and continue to flow into subsequent processes. However, the subsequent process is very likely to detect the defect again and determine it as an NG part, causing the production line to shut down and affecting production.

[0053] In the LCD substrate manufacturing industry, defect detection at the factory and during the processing phase is crucial. If some product defects (such as severe edge chipping and fragmentation) are not promptly detected and intercepted, they can break and scatter debris during transportation on the production line, making cleanup difficult and consuming significant manpower and resources. Furthermore, if defective products reach end customers, it can become a serious quality incident.

[0054] If an OK part is mistakenly detected as an NG part, the production line will be shut down, and it is very likely that the part will be mistakenly detected as an NG part again in the next process, causing the production line to shut down again. Although this will not cause quality problems, it will greatly affect production efficiency. In addition, the production line shutdown means that manual confirmation and restart are required, resulting in a large waste of human resources. For example, the defects detected on the liquid crystal substrate in process A are all tolerable defects after manual or algorithmic analysis. The liquid crystal substrate flows to process B, where defects can be detected. At this time, it is necessary to shut down the production line and conduct manual or algorithmic analysis on the defects again. It is understandable that even if the defects detected in process B are all repeated defects, it will cause a shutdown. It can be seen that if the defects indicated by the defect detection results of any process are only repeated defects, it will cause the production line to shut down, affecting production efficiency.

[0055] Furthermore, every process in the LCD substrate production line is subject to the possibility of mechanical aging, process problems, or other external factors causing defects in the LCD substrate during use. Without timely analysis and improvement of the process, the production line downtime rate will increase.

[0056] In order to effectively solve the problem that when the defects indicated by the defect detection results of any process are only repeated defects, the production line will be shut down, which will affect production efficiency. The embodiments of the present application provide a defect analysis method, device, electronic device and storage medium; a defect analysis method of the present application can be applied to electronic devices, which can be terminal devices, servers, etc. The terminal device can be a tablet computer, desktop computer and mobile phone, etc.; the present application does not limit the specific form of the electronic device. The method includes:

[0057] In response to generating a first defect detection result indicating the presence of a defect for any inspection object in a target process, determining whether at least one second defect detection result exists for the inspection object; wherein the second defect detection result is a defect detection result indicating the presence of a defect for the inspection object in a process preceding the target process;

[0058] If present, for each second defect detection result, determining the position of each second defect indicated by the second defect detection result mapped in a target defect distribution map, and determining the position of each first defect indicated by the first defect detection result mapped in the target defect distribution map; wherein the target defect distribution map is an image representing defect distribution in an object area of ​​the inspection object;

[0059] For each second defect detection result, based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, performing repeated defect analysis processing on each first defect to obtain a repeated defect analysis result;

[0060] The repeated defects indicated by the obtained repeated defect analysis results are removed from the first defects to obtain the optimized defect detection result of the inspection object in the target process.

[0061] The solution provided by the embodiment of the present application is to determine whether there is at least one second defect detection result for any detection object in the target process in response to generating a first defect detection result that characterizes the existence of defects. If there is, it means that there is a high probability of repeated defects in the first defect detection result. At this time, for each second defect detection result, the position of each second defect indicated by the second defect detection result mapped in the target defect distribution map can be determined, and the position of each first defect indicated by the first defect detection result mapped in the target defect distribution map can be determined; for each second defect detection result, based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, a repeated defect analysis process is performed on each first defect to obtain a repeated defect analysis result, and the repeated defects in each first defect are determined; the repeated defects in each first defect indicated by the repeated defect analysis result are removed, and the optimized defect detection result of the detection object in the target process can be obtained.

[0062] Since there are no repeated defects in the optimized defect detection results of the target process, when analyzing whether the production line needs to be shut down according to the optimized defect detection results, if the optimized defect detection results of the target process indicate that there are no defects, it means that the detection object has not caused defects in the target process and will not cause the production line to be shut down. If the optimized defect detection results of the target process indicate that there are defects, it will cause the production line to be shut down, but the reason for the production line shutdown at this time is that the target process has caused defects to the detection object. Therefore, by utilizing the optimized defect detection results of the target process, it is possible to effectively reduce the situation where the defects indicated by the defect detection results are only repeated defects, resulting in production line shutdowns. It can be seen that the solution of the present application can effectively solve the problem of production line shutdowns and production efficiency impact when the defects indicated by the defect detection results of any process are only repeated defects.

[0063] A defect analysis method provided in an embodiment of the present application is introduced below with reference to the accompanying drawings.

[0064] like Figure 1As shown, a defect analysis method provided in an embodiment of the present application may include the following steps:

[0065] S101, in response to generating a first defect detection result of any inspection object in a target process, indicating the existence of a defect, determining whether there is at least one second defect detection result for the inspection object; wherein the second defect detection result is a defect detection result of the inspection object in a preceding process of the target process, indicating the existence of a defect.

[0066] In this application, the inspection objects involved are the production objects of the production line, such as liquid crystal substrates, screens and other objects. These objects usually only need to be inspected for defects on the surface, and the production objects corresponding to different production lines may be different. This application does not limit the production objects. Since it is usually only necessary to inspect the surface for defects for the above-mentioned objects, the defect detection results of this application are usually the detection results obtained by performing defect detection on the surface of the inspection object. The defect detection results can indicate whether there are defects and the positions of each defect detected when there are defects in the target image. The target image is the image containing the inspection object based on which the defect detection is performed.

[0067] Optionally, the defect detection result may also include information such as defect type, which is not limited in this application.

[0068] For any production line, each process of the production line can be deployed with a defect detection system to determine the defect detection results. The defect detection system can include an acquisition device and a processing device. The processing device in the defect detection system of each process can be the same, that is, each process can share the same processing device. In an optional implementation, the electronic device that executes the defect analysis method of the present application can be the same device as the processing device in the above-mentioned defect detection system, or it can be an independent device.

[0069] For example, Figure 2 This is a schematic diagram of a production line provided in this application. This production line manufactures liquid crystal substrates. The production line processes may include cleaning, coating, and gluing. Each process has an image acquisition device to capture images of the product being processed. The image acquisition device in this production line is a line array camera. Within this production line, each acquisition device can be connected to a PC (Personal Computer) to upload the captured images to a server, which then executes the defect analysis method.

[0070] In the present application, the acquisition device used to acquire images of the detection object can be any type of camera, for example, it can be a single camera, an array camera, etc. This application does not limit the acquisition device.

[0071] The architecture of different production lines may be different, and thus, the execution subjects of the defect analysis method may also be different. Exemplarily, the architecture of a production line is as follows: for each process of the production line, there is a common electronic device, which can be connected and communicated with the defect detection system of each process, so that each process can share the electronic device, and the electronic device can execute the defect analysis method for each process; the architecture of a production line is as follows: each process of the production line is provided with an electronic device, and the electronic device of each process is connected and communicated with the defect detection system of the process, so that the electronic device can only execute the defect analysis method for the corresponding process. Of course, in this case, the electronic devices can communicate with each other, so that the electronic device corresponding to any process can obtain the data of each process, or the data of each process are stored in the same storage device, so that the electronic device corresponding to any process can obtain the data of each process from the storage device.

[0072] It is understandable that all defects that occur in the first process of the production line can be considered to be caused by the first process, and there is no need to repeat the defect analysis. Of course, when the target process is the first process, there must be no second defect detection result, and the subsequent process of the defect analysis method of this application cannot be executed. Therefore, the target process targeted by the solution of this application is any process except the first process. The first process of the production line is the first process of the production line. For example, Figure 2 For the production line in the machine, the cleaning process is the first process.

[0073] Typically, all production objects on a production line are of the same type, and thus have substantially the same appearance and size. However, each production object may have a corresponding identifier, such as an identification code, number, etc., to distinguish the individual production objects. Therefore, for any inspection object, after generating a first defect detection result indicating the presence of a defect for the inspection object in the target process, the inspection object's identifier can be used to obtain the inspection result for the inspection object in the preceding process, thereby determining whether the inspection object has at least one second defect detection result.

[0074] In addition, in the present application, the inspection of the inspection object and the acquisition of the defect detection result can be completed by performing image analysis on the target image of the inspection object obtained by the acquisition. Specifically, the corresponding defect analysis algorithm can be used to perform image analysis on the target image of the inspection object obtained by the acquisition. The defect analysis algorithm is not the invention point of the present application. Any defect analysis algorithm that can perform image analysis on the target image of the inspection object to obtain the defect detection result can be applied to the solution of the present application. For different production scenarios, a defect analysis algorithm that is compatible with the scenario can be selected to perform defect detection, and the defect analysis algorithm used in each process can be the same or different.

[0075] For example, the defect detection results described in this application can be presented in different forms such as images, tables, and texts. This application does not limit the specific presentation form of the defect detection results. For example, the defect detection results can be presented in the form of images. Figure 3 The figure shown is an example diagram of a defect detection result provided by this application. Figure 3 It is a schematic diagram showing the detected defects in the target image. In this application, for any process, the schematic diagram showing the defects detected in the process in the target image corresponding to the process can be called the defect distribution map of the process. The defect distribution map of each process uses the object area of ​​the detection object to represent the defect distribution, and the posture of the object area is the posture of the detection object in the process. Figure 3 A, B and C are the detected defects, 301 is the detection object, and 302 is the target image. The defect distribution map can also be called a defect thumbnail.

[0076] Moreover, since the defects detected in the process before the target process can usually include the defects of each preceding process, in one implementation, the second defect detection result can be the defect detection result of the inspection object in the process before the target process, which characterizes the existence of defects.

[0077] In each process, due to errors in the analysis algorithm, changes in the processing of the production object caused by each process, and other influencing factors, the defect detection results may contain errors. For example, a defect may be detected in process 1 but not in process 2. Therefore, in order to accurately analyze repeated defects, in one implementation, the second defect detection result can be a defect detection result of the test object in a specified predecessor process of the target process, which indicates the presence of a defect. The specified predecessor process can be any of the predetermined predecessor processes whose generated defect detection results are prone to errors.

[0078] In addition, in one implementation, the second defect detection result can be a defect detection result of the test object in any preceding process of the target process, indicating the presence of a defect. That is, in each process preceding the target process, if the defect detection result of the test object in that process indicates the presence of a defect, the defect detection result of that process will serve as the second defect detection result. Using the second defect detection results determined by the above two implementations as the second defect detection results used in the solution of this application can improve the accuracy of repeated defect analysis.

[0079] The present application determines whether the inspection object has at least one second defect detection result in order to make a preliminary judgment on whether the first defect detection result has repeated defects. Specifically, if the defect detection result of the inspection object in the preceding process indicates that there are no defects, then it means that the defects in the first defect detection result are all caused by the target process and there are no repeated defects, and there is no need to proceed to subsequent steps. If the defect detection result of the inspection object in the preceding process indicates that there are defects, then it means that the defects represented by the first defect detection result are likely to include repeated defects, and S102 can be executed.

[0080] S102, if present, for each second defect detection result, determine the position of each second defect map indicated by the second defect detection result in the target defect distribution map, and determine the position of each first defect map indicated by the first defect detection result in the target defect distribution map; wherein the target defect distribution map is: an image representing the defect distribution in the object area of ​​the detection object.

[0081] For the same type of inspection object, the target defect distribution map can usually be the same. The target defect distribution map can be a pre-set defect distribution map. For example, the object area of ​​the inspection object in a certain posture can be used as the target domain defect distribution map, and a certain point in the object area of ​​the inspection object in a certain posture can be set as the origin of the coordinate system, thereby establishing the coordinate system of the target defect distribution map. For example, Figure 4 FIG. 1 is a schematic diagram of a defect distribution diagram. Figure 4 401 is the object area of ​​the inspection object, the origin of the coordinate system of the defect distribution map is the vertex of the chamfer of the inspection object, 402 is the chamfer of the inspection object, and points A, B and C are three defects.

[0082] After establishing the coordinate system of the target defect distribution map, the positions of each first defect and each second defect mapped in the coordinate system of the target defect distribution map are determined, and the positions of each first defect and each second defect in the defect distribution map can be obtained.

[0083] In this application, the coordinate system can be established according to specific scenarios and requirements, and this application does not limit this.

[0084] The target defect distribution map may also be a defect distribution map of a specified process. For example, the defect distribution map of process A may be specified as the target defect distribution map.

[0085] In this application, determining the position of each second defect map indicated by the second defect detection result in the target defect distribution map, and determining the position of each first defect map indicated by the first defect detection result in the target defect distribution map can be understood as a process of overlaying images.

[0086] S103, for each second defect detection result, based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, repeat defect analysis processing is performed on each first defect to obtain a repeated defect analysis result.

[0087] It is understood that, for any first defect in the target defect distribution map, if there is a second defect at the same location as the first defect, then the first defect can be determined to be a repeating defect. Thus, in one implementation, for each first defect, it can be determined whether there is a second defect at the same location in the target defect distribution map as the first defect; if so, the first defect is a repeating defect.

[0088] Of course, since there will be a certain error between the calculated positions of the first defects in the target defect distribution map and the positions of the second defects indicated by the second defect detection result in the target defect distribution map, the coordinates of the first defect and the second defect corresponding to the repeated defect are not the same. In this case,

[0089] Optionally, in one implementation, the repeated defect analysis process includes:

[0090] For each first defect, determining a second defect closest to the first defect from among the second defects indicated by the second defect detection result;

[0091] If the distance between the determined second defect and the first defect is smaller than a predetermined threshold, the first defect is determined to be a repeated defect.

[0092] Optionally, in one implementation, the repeated defect analysis process includes:

[0093] For each first defect, each second defect is traversed to determine the distance between each second defect and the first defect. If the distance between the traversed second defect and the first defect is less than a predetermined threshold, the first defect is determined to be a repeated defect, and the analysis of the first defect is terminated.

[0094] In the above implementation, whether the first defect is a repeated defect is determined by the relationship between the distance between the second defect and the first defect and a predetermined threshold, which can effectively reduce the impact caused by errors.

[0095] Furthermore, since defects have an area, for a first defect and a second defect belonging to the same defect, the first defect and the second defect usually overlap in the target defect distribution map. Taking into account the existence of errors, the overlap of a first defect and a second defect belonging to the same defect in the target defect distribution map may not reach 100%. Therefore, a target overlap can be set according to actual conditions. For each first defect, if the distance between the first defect and the second defect is less than a predetermined threshold, and the overlap between the first defect and the second defect is greater than the target overlap, the first defect can be determined to be a repeated defect. Therefore, this embodiment can effectively determine whether a first defect is a repeated defect by using distance and overlap.

[0096] S104 , removing the repeated defects indicated by the obtained repeated defect analysis results from the first defects, and obtaining the optimized defect detection result of the inspection object in the target process.

[0097] Since there is at least one second defect detection result, the obtained repeated defect analysis result can also be at least one; therefore, for any first defect, if there is at least one repeated defect analysis result indicating that the first defect is a repeated defect, the first defect can be removed from each first defect, thereby obtaining the optimized defect detection result of the detection object in the target process.

[0098] It is understandable that there are no repeated defects in the optimized defect detection result, and the optimized defect detection result can be used as the final defect detection result of the target process to analyze whether to shut down the production line.

[0099] If the optimized defect detection result indicates that there are no defects, it means that the target process has not caused any defects to the inspection object, and there is no need to stop the machine for inspection.

[0100] If the optimized defect detection result indicates defects, the indicated defects are caused by the target process, and the process can be shut down. The optimized defect detection result can be used to further analyze the defects in the target process.

[0101] Furthermore, since the optimized defect detection results indicate defects caused by the target process, analysis using the optimized defect detection results can provide an effective basis for optimizing the target process.

[0102] In addition, the solution of the present application can be applied to products with the same aspect ratio, that is, products of the same type, after being converted into the same size, for overlay research. In addition, the solution of the present application can be applied to production inspection scenarios of other large-area products for multi-station combined analysis and single-station analysis of the impact of machine parameters, different processes, and production lines on defects.

[0103] The solution provided by the embodiment of the present application is to determine whether there is at least one second defect detection result for any detection object in the target process in response to generating a first defect detection result that characterizes the existence of defects. If there is, it means that there is a high probability of repeated defects in the first defect detection result. At this time, for each second defect detection result, the position of each second defect indicated by the second defect detection result mapped in the target defect distribution map can be determined, and the position of each first defect indicated by the first defect detection result mapped in the target defect distribution map can be determined; for each second defect detection result, based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, a repeated defect analysis process is performed on each first defect to obtain a repeated defect analysis result, and the repeated defects in each first defect are determined; the repeated defects in each first defect indicated by the repeated defect analysis result are removed, and the optimized defect detection result of the detection object in the target process can be obtained.

[0104] Since there are no repeated defects in the optimized defect detection results of the target process, when analyzing whether the production line needs to be shut down according to the optimized defect detection results, if the optimized defect detection results of the target process indicate that there are no defects, it means that the detection object has not caused defects in the target process and will not cause the production line to be shut down. If the optimized defect detection results of the target process indicate that there are defects, it will cause the production line to be shut down, but the reason for the production line shutdown at this time is that the target process has caused defects to the detection object. Therefore, by utilizing the optimized defect detection results of the target process, it is possible to effectively reduce the situation where the defects indicated by the defect detection results are only repeated defects, resulting in production line shutdowns. It can be seen that the solution of the present application can effectively solve the problem of production line shutdowns and production efficiency impact when the defects indicated by the defect detection results of any process are only repeated defects.

[0105] Optionally, determining the position of each first defect indicated by the first defect detection result mapped on the target defect distribution map includes steps A1 and A2;

[0106] Step A1: Determine the relative coordinates of each first defect relative to the first reference point using the coordinates of each first defect indicated by the first defect detection result in a target coordinate system and the coordinates of the first reference point in the target coordinate system; wherein the target coordinate system is an image coordinate system of a first target image, and the first target image is the target image based on which the first defect detection result is determined;

[0107] In the target images taken for the inspection object in different processes, the posture of the inspection object may be different, and the position of the defect detected in different processes is represented by the image coordinates of the target image of the process. Therefore, if the posture of the inspection object in the target images taken for the inspection object in different processes is different, then the coordinates of the same defect of the inspection object in the target images of different processes are not the same. For example, the posture of the inspection object in process A and process B are different, then for the same defect of the inspection object, the coordinates in the target image of process A are (4, 3), and the coordinates in the target image of process B are (7, 4). It can be seen that the coordinates in the target image of process A and the coordinates in the target image of process B are not the same.

[0108] Figure 5 In the figure, the target images of the same inspection object are collected in three different processes: the first process, the second process, and the third process in a production line. Figure 5 The three processes can be three processes with a sequence relationship. Figure 5 501 , 502 and 503 are detection objects. It can be seen that the postures of the detection objects in the target images of the three different processes are different.

[0109] The image coordinate system of each target image is the same. Usually, the upper left corner of the target image is used as the origin of the coordinate system, the horizontal direction is the x-axis, and the vertical direction is the y-axis. Of course, the image coordinate system of the target image can also be established according to the actual scene, and this application does not limit this.

[0110] The first reference point may be a pre-set point. For example, Figure 5 The point at the chamfer of the detection object can be used as the first reference point. As a point on the detection object, the position of the first reference point on the detection object is not affected by the posture of the detection object.

[0111] The method for determining the relative coordinates of each first defect relative to the first reference point can specifically be: subtracting the coordinates of each first defect in the target coordinate system from the coordinates of the first reference point in the target coordinate system to obtain the relative coordinates of each first defect relative to the first reference point.

[0112] Step A2: Based on the relative coordinates of each first defect, determine the position of each first defect mapping in a specified coordinate system to obtain the position of each first defect mapping in the target defect distribution map; wherein the specified coordinate system is the image coordinate system of the target defect distribution map.

[0113] The image coordinate system of the target defect distribution map may be a coordinate system with the first reference point as its origin.

[0114] If the object area of ​​the inspection object in the first target image has the same posture as the object area represented by the target defect distribution map, the relative coordinates of each first defect can be directly used as the position of each first defect mapped in the target defect distribution map.

[0115] Of course, the object region of the inspection object in the first target image and the object region represented by the target defect distribution map may not necessarily have the same pose. In this case, it is necessary to further calculate the position of each first defect to determine the position of the first defect map in the target defect distribution map. The following describes a specific method for determining the position of the first defect map in the target defect distribution map.

[0116] There may be multiple ways to implement the determination of the positions of the first defects indicated by the first defect detection results mapped on the target defect distribution map, and any one of them may be applied in the present application.

[0117] Optionally, in one implementation, the designated coordinate system is a coordinate system having a target reference point of the detection object as an origin and a designated side as a designated coordinate axis; the designated side is a side on which the target reference point is located; and the first reference point is a point in the first target image that represents the target reference point.

[0118] The target reference point of the detection object is a pre-set point. The side where the target reference point is located can be used as the x-axis or y-axis of the specified coordinate system, so that after determining one coordinate axis of the coordinate system, the other coordinate axis of the specified coordinate system can be determined.

[0119] In some production scenarios, the posture of the production object in each process is not fixed. Figure 5 Schematic diagram of target images of three different processes provided in the embodiments of the present application; Figure 5 The three processes can be three processes with a sequence relationship; Figure 5 As shown, 501 is the inspection object in the target image of the first process, 502 is the inspection object in the target image of the second process, and 503 is the inspection object in the target image of the third process. The angular difference between the postures of the inspection objects in each process is not a specific angle but can be any angle. For this scenario, the method of this embodiment can be used to determine the location of each first defect indicated by the first defect detection result mapped in the target defect distribution map.

[0120] Before determining the position of each first defect mapped to the designated coordinate system based on the relative coordinates of each first defect, the method further includes:

[0121] determining an angle between a specified coordinate axis of the target coordinate system and the specified edge of the detection object in the first target image;

[0122] In this embodiment, the detection object may be a polygon, and the designated side is a side of the detection object. Regardless of the position of the detection object, the angle is calculated using the designated side and the designated coordinate axis of the target coordinate system.

[0123] like Figure 6 In the target images of the first, second, and third steps shown, the positions of the detection objects are different, but the coordinate system is established by taking the point at the chamfer of the detection object as the target reference point and the designated side where the target reference point is located as the y-axis. Figure 6 The y-axis is used as the specified coordinate axis. Figure 6 The x-axis is determined based on the position of the y-axis after the y-axis is determined.

[0124] Determining an angle between a specified coordinate axis of the target coordinate system and the specified side of the detection object in the first target image includes:

[0125] Determining the coordinates of an auxiliary reference point in the first target image in the target coordinate system; wherein the auxiliary reference point represents any point on the designated edge except the target reference point;

[0126] Based on the coordinates of the auxiliary reference point in the target coordinate system and the coordinates of the first reference point in the target coordinate system, the angle between the unit vector of the specified coordinate axis of the target coordinate system and the direction vector of the specified side of the detection object in the first target image is calculated. Exemplarily, in one implementation, when the specified coordinate axis is the y-axis, the angle between the unit vector of the specified coordinate axis of the target coordinate system and the direction vector of the specified side of the detection object in the first target image can be calculated based on a first formula;

[0127] The first formula includes:

[0128] Where θ is the angle, X p Y is the horizontal coordinate of the first reference point in the target coordinate system, p is the vertical coordinate of the first reference point in the target coordinate system, X k Y is the horizontal coordinate of the auxiliary reference point in the target coordinate system, k is the ordinate of the auxiliary reference point in the target coordinate system, and the unit vector in the positive direction of the y-axis is (0, 1).

[0129] For example, Figure 7 A schematic diagram of establishing a specified coordinate system provided in an embodiment of the present application, Figure 7 The three images in the figure are for the same target image. Figure 7 As shown, point P is the first reference point and point k is the auxiliary reference point. In order to facilitate understanding of the angle between the specified coordinate axis of the image coordinate system and the specified edge of the detection object in the target image, the image coordinate system can be moved to the point P. It can be seen that the angle between the specified coordinate axis of the image coordinate system and the specified edge of the detection object is the angle between the edge where point P and point k are located and the y-axis. In this embodiment, the y-axis is the specified coordinate axis, and the edge where point P and point k are located is the specified edge. Figure 7 The coordinate system in the third figure is the specified coordinate system, which can be understood as being obtained by moving the image coordinate system to the first reference point and rotating it by an angle θ so that the specified coordinate axis of the image coordinate system coincides with the specified edge.

[0130] In addition, the coordinates of the auxiliary reference point in the target coordinate system and the coordinates of the first reference point in the target coordinate system can be used to calculate the slope corresponding to the designated side of the detection object, thereby using the slope to calculate the angle between the designated coordinate axis of the target coordinate system and the designated side of the detection object in the first target image. This application does not limit the method for calculating the angle between the designated coordinate axis of the target coordinate system and the designated side of the detection object in the first target image. Any calculation method that can calculate the angle between the designated coordinate axis of the target coordinate system and the designated side of the detection object in the first target image can be applied to the solution of this application.

[0131] The determining, based on the relative coordinates of each first defect, the position of each first defect mapping in the specified coordinate system to obtain the position of each first defect mapping in the target defect distribution map includes:

[0132] Based on the angle and the relative coordinates of each first defect, the coordinates of each first defect mapping in the specified coordinate system are calculated to obtain the position of each first defect mapping in the target defect distribution map.

[0133] If the angle is 0 degrees or 360 degrees, the relative coordinates of each first defect can be directly used as the position of each first defect mapped in the target defect distribution map.

[0134] When the angle is any angle, calculating the coordinates of each first defect mapped in the specified coordinate system based on the angle and the relative coordinates of each first defect may include:

[0135] Using the angle, according to a predetermined coordinate transformation relationship, the relative coordinates of each first defect are transformed into the specified coordinate system to obtain the coordinates of each first defect in the specified coordinate system; wherein, the predetermined coordinate transformation relationship is a transformation relationship regarding the angle, used to transform any relative coordinate into the specified coordinate.

[0136] Specifically, the predetermined coordinate transformation relationship can be represented by a second formula.

[0137] The second formula includes:

[0138] x A" =x A′ cosθ+y A′ sinθ;

[0139] y A" =y A′ cosθ-x A′ sinθ;

[0140] Among them, the x A" is the abscissa of any first defect in the specified coordinate system, the y A" is the vertical coordinate of any first defect in the specified coordinate system, the x A′ is the horizontal coordinate of the relative coordinate of any first defect, the y A′ is the ordinate of the relative coordinate of any first defect, and θ is the angle.

[0141] Exemplarily, the relative coordinates of any first defect are Figure 7 In the Figure 7 In the second figure, the image coordinate system is moved to the coordinates in the coordinate system at the point P, and Figure 7The specified coordinate system of the third figure is obtained by rotating the coordinate system of the second figure clockwise by the angle, so that the coordinates of any first defect in the specified coordinate system are equivalent to the relative coordinates of the first defect, which are obtained by rotating the relative coordinates of the first defect counterclockwise by the angle. Figure 7 In the above, the coordinates of any first defect in the specified coordinate system are the coordinates of the first defect in Figure 7 The coordinates in the coordinate system in the third figure.

[0142] That is, it can be understood that the coordinates of the first defect mapped in the specified coordinate system are obtained by rotating the relative coordinates of the first defect counterclockwise by the angle.

[0143] Any method that can calculate the coordinates after coordinate rotation can be applied to the solution of this application, and this application does not limit this.

[0144] The determining of the position of each second defect indicated by the second defect detection result mapped on the target defect distribution map includes:

[0145] The coordinates of each second defect mapping indicated by the second defect detection result in the specified coordinate system are obtained, and the positions of each second defect mapping indicated by the second defect detection result in the target defect distribution map are obtained.

[0146] In the process corresponding to the second defect detection result, the defect analysis method of the present application may also be executed on the inspection object. In this case, the coordinates of each second defect indicated by the second defect detection result in the specified coordinate system have already been calculated, and thus can be directly obtained. If the coordinates of each second defect indicated by the second defect detection result in the specified coordinate system have not been calculated, the coordinates of each second defect in the specified coordinate system can be calculated in the same manner as the coordinates of each first defect indicated by each first defect detection result in the specified coordinate system. No further details will be given here.

[0147] The solution of this embodiment can effectively determine the position of each first defect mapping in the defect distribution map when the angular difference between the postures of the inspection object in each process is any angle.

[0148] Optionally, in one implementation, the first reference point is a vertex of the inspection object at a specified position in the first target image; the specified coordinate system is a coordinate system with a second reference point as its origin, and the second reference point is a vertex of the inspection object at the specified position in the second target image; and the second target image is a target image on which the second defect detection result is determined.

[0149] In some production scenarios, production objects are transported to various processes by conveying devices. By pre-configuring the conveying devices, process positions, and image acquisition equipment positions, in the target images taken of the inspection objects in each process, the angular differences between the postures of the object areas of the inspection objects can be specific angles such as 90 degrees, 180 degrees, 270 degrees, or 360 degrees, for example. Figure 8 As shown, the position difference of the object area of ​​the detection object between the coating process and the cleaning process is 90 degrees, and the position difference of the object area of ​​the detection object between the gluing process and the coating process is 90 degrees. The solution of this embodiment is aimed at the scenario where the detection object of each process has the above-mentioned specific rotation angle. Figure 8 801a, 801b and 801c are target images; 802a, 802b and 802c are detection objects.

[0150] The posture difference of the detection object between the two processes is the rotation angle of the detection object in one process relative to the other process.

[0151] In this embodiment, the object region of the detection object represented by the target defect distribution map has the same posture as the object region of the detection object in the second target image.

[0152] The designated position can be set according to the actual situation. Figure 9a and Figure 9b Two schematic diagrams describing the specified coordinate system are provided in the embodiments of this application. Figure 9a and Figure 9b As shown, when the designated position is the upper left corner, you can Figure 9a and Figure 9b The vertex in the upper left corner of the target image is the coordinate origin, the horizontal direction is the x-axis, and the vertical direction is the y-axis. A coordinate system is established. Figure 9a The second target image A and Figure 9b The second target image B is two different second target images. 901 is the detection object in the second target image A, and 902 is the detection object in the second target image B. The postures of the detection objects in these two images are different, but the coordinate system is established with the vertex in the upper left corner of the detection object in the target image as the origin.

[0153] Before determining the position of each first defect mapping in a specified coordinate system based on the relative coordinates of each first defect to obtain the position of each first defect mapping in the target defect distribution map, the method further includes:

[0154] determining a rotation angle of the detection object in the first target image relative to the detection object in the second target image, and determining size information of the detection object, the size information including length and / or width;

[0155] In this embodiment, since the position of the inspection object in each process is fixed, the position difference of the inspection object between each process can also be determined in advance. Therefore, for each second defect detection result, the position difference of the inspection object between the process corresponding to the defect detection result and the target process can be obtained, and the rotation angle of the inspection object can be obtained. The rotation angle of the inspection object in the first target image relative to the inspection object in the second target image can be understood as: rotating the inspection object in the second target image by the rotation angle can obtain the position of the inspection object in the first target image.

[0156] The size information may also be predetermined, and the predetermined size information may be obtained. Of course, the detection object in the target image may also be identified to obtain the size information.

[0157] The determining, based on the relative coordinates of each first defect, the position of each first defect mapping in the specified coordinate system to obtain the position of each first defect mapping in the target defect distribution map includes:

[0158] Based on the rotation angle, the size information of the inspection object and the relative coordinates of each first defect, the coordinates of each first defect in the specified coordinate system are determined to obtain the position of each first defect mapped in the target defect distribution map.

[0159] In this embodiment, the rotation angles include four clockwise rotations of 90 degrees, 180 degrees, 270 degrees, and 360 degrees. It can be understood that a counterclockwise rotation of 90 degrees is the same as a clockwise rotation of 270 degrees, a counterclockwise rotation of 180 degrees is the same as a clockwise rotation of 180 degrees, and a counterclockwise rotation of 270 degrees is the same as a clockwise rotation of 90 degrees.

[0160] Assume that the relative coordinates of any first defect are (x, y), the size of the side of the detection object in the first target image in the horizontal direction is w, and the size of the side of the detection object in the first target image in the vertical direction is h.

[0161] When the rotation angle is 360 degrees clockwise, the posture of the inspection object is the same, and the relative coordinates of each first defect can be directly used as the position of each first defect mapped in the target defect distribution map.

[0162] When the rotation angle is 90 degrees clockwise, the first defect is mapped to the position (x b ,y b ) is calculated as follows: (x b ,y b)=(y,wx);

[0163] When the rotation angle is 180 degrees clockwise, the first defect is mapped to the position (x b ,y b ) is calculated as follows: (x b ,y b )=(wx,hy);

[0164] When the rotation angle is 270 degrees clockwise, the first defect is mapped at a position (x b ,y b ) is calculated as follows: (x b ,y b )=(hy,x).

[0165] Optionally, determining the position of each second defect indicated by the second defect detection result mapped in the target defect distribution map includes:

[0166] The relative coordinates of each second defect indicated by the second defect detection result relative to the second reference point are obtained to obtain the position of each second defect indicated by the second defect detection result mapped on the specified target defect distribution map.

[0167] Since the designated coordinate system is a coordinate system with the second reference point as its origin, the second reference point is the vertex of the inspection object at the designated position in the second target image; the relative coordinates of each second defect relative to the second reference point are the coordinates of the designated coordinate system, and thus, by obtaining the relative coordinates of each second defect indicated by the second defect detection result relative to the second reference point, the position of each second defect indicated by the second defect detection result mapped in the designated target defect distribution map can be obtained.

[0168] In the process corresponding to the second defect detection result, the defect analysis method of the present application may also be executed on the inspection object. In this case, the relative coordinates of each second defect indicated by the second defect detection result relative to the second reference point have already been calculated and can thus be directly obtained. If the relative coordinates of each second defect indicated by the second defect detection result relative to the second reference point have not been calculated, the relative coordinates of each second defect relative to the second reference point can be calculated in the same manner as the relative coordinates of each first defect relative to the first reference point described above. No further details will be given here.

[0169] Next, a specific example is used to introduce how to determine the position of each first defect map in the target defect distribution map through the implementation method. In this embodiment, the production line can be Figure 2The indicated production line.

[0170] like Figure 2 As shown, the liquid crystal substrate moves on the conveyor belt and always maintains a certain posture. Since the moving direction on the conveyor belt is different in each process, the image after being imaged by the line array camera shows different postures. Figure 10 Schematic diagram of defect distribution diagrams of three different processes provided in the embodiment of this application. Figure 10 As shown, the schematic diagram of the defect distribution map of three different processes is obtained by presenting the defects detected in each process on the target image of the process for the cleaning process, the coating imaging process and the gluing process. Figure 10 A, B, and C are three defects; defect A is caused by the cleaning process, defect B is caused by the coating process, and defect C is caused by the gluing process.

[0171] like Figure 10 As shown, assuming that a tolerable defect A is detected after the cleaning process, the liquid crystal substrate continues to enter the coating process along the conveyor belt, and defects A and B are detected in the coating process. Defect B is also a tolerable defect. Then the liquid crystal substrate continues to enter the gluing process along the conveyor belt, and defects A, B and C are detected in the coating process.

[0172] When the coating process is used as the target process, the steps for converting the defect A detected in the cleaning process into the defect distribution map of the cleaning process are as follows:

[0173] Step 1: Get the coordinates (X, Y) of the upper left corner of the liquid crystal substrate in the target image.

[0174] In this embodiment, the coating process is equivalent to the target process, and the cleaning process is a pre-process of the target process. The upper left corner vertex of the liquid crystal substrate in the target image of the cleaning process is the second reference point. The defect distribution map of the cleaning process is the target defect distribution map.

[0175] Step 2: According to the coordinates of point A (X A ,Y A ), and obtain the coordinates of defect A on the defect distribution map of the coating process (M A ,N A )=(X A -X,Y A -Y).

[0176] (M A ,N A )=(X A -X,Y A -Y) is the relative coordinate of the second defect A relative to the second reference point.

[0177] For the coating process, the steps to map the detected defect B to the defect distribution map of the cleaning process are as follows:

[0178] Step 1: Obtain the coordinates (X, Y) of the upper left corner of the liquid crystal substrate and the width and height dimensions W, H of the liquid crystal substrate in the target image.

[0179] The upper left corner of the liquid crystal substrate in the target image of the coating process is the first reference point. The width and height dimensions W and H of the liquid crystal substrate are the above-mentioned dimension information.

[0180] Step 2: According to the coordinates of point B (X B ,Y B ), and obtain the coordinates of defect B on the defect distribution map of the coating process (M' B ,N' B )=(X B -X,Y B -Y).

[0181] (M' B ,N' B )=(X B -X,Y B -Y) is the relative coordinate of the first defect B relative to the first reference point.

[0182] Step 3: The coating process image is rotated 90 degrees clockwise relative to the cleaning process image, so the coordinates of defect B on the defect distribution map of the cleaning process are obtained (M B ,N B )=(N' B ,W-M' B )=(Y B –Y,W–X B +X).

[0183] Step three can be calculated using the formula introduced in the above embodiment, and will not be described in detail here.

[0184] Similarly, the coordinates of the defect distribution map of the A' defect in the cleaning process (M A’ ,N A’ At this point, the mapping process from the defects detected in the coating process to the defect distribution map of the cleaning process is complete. Next, we can enter the deduplication process, traversing the mapping points of the defects detected in the coating process and calculating the distance between each defect in the cleaning process and its mapping point. If the distance is less than the preset threshold, it indicates a repeated defect. The defect has already been detected and ignored in the previous process, and can be ignored in the coating process as well.

[0185] When the gluing process is used as the target process, there are multiple preceding processes of the target process. The defects of the gluing process can be mapped to the defect distribution map of each preceding process, and the defect distribution map of each preceding process is the target defect distribution map.

[0186] For the gluing process, the steps to map the detected defect C to the defect distribution map of the cleaning process are as follows:

[0187] Get the coordinates (X, Y) of the upper left corner of the liquid crystal substrate and the width and height dimensions W, H of the liquid crystal substrate in the image.

[0188] According to the coordinates of point C (X C ,Y C ), and obtain the coordinates of defect C on its own defect distribution map (M” C ,N” C )=(X C -X,Y C -Y).

[0189] The imaging of the coating process is rotated 90 degrees clockwise relative to the imaging of the coating process, so the coordinates of the C defect on the defect distribution map of the coating process are obtained (M' B ,N' B )=(N” B ,W–M” B )=(Y C –Y,W–X C +X).

[0190] The specific calculation process can be performed using the formula introduced in the above embodiment, and will not be described in detail here.

[0191] Similarly, the defect distribution map of the coating process can be converted to the defect distribution map of the cleaning process, and (M B ,N B )=(N' B ,H–M' B )=(W–X C +X,H–Y C +Y).

[0192] The implementation method of defect mapping is the same as that of the above embodiment, and will not be described in detail here.

[0193] Similarly, the coordinates of defects A”, B’ in the defect distribution diagram of the cleaning process can be obtained (M A” ,N A” ), (M B’ ,N B’ ).

[0194] The solution of this embodiment can effectively determine the position of each first defect mapping in the defect distribution map when the angular difference between the postures of the inspection object between various processes is a specific angle.

[0195] The following uses a specific embodiment to introduce the implementation process of determining the positions of the first defects indicated by the first defect detection result mapped in the target defect distribution map using the implementation method of this embodiment.

[0196] Specifically, for scenarios where the inspection object is a liquid crystal substrate, the solution of this embodiment actually converts the coordinates of the defect in the target image's image coordinate system into the coordinates of the defect in the liquid crystal substrate coordinate system (the liquid crystal substrate coordinate system has the substrate chamfer as its origin, the short side of the substrate adjacent to the chamfer as the x-axis, and the long side of the substrate adjacent to the chamfer as the y-axis). Therefore, the entire transformation process can be described by the translation and rotation of the coordinate system. The target image's image coordinate system is the target coordinate system, and the liquid crystal substrate coordinate system is the designated coordinate system.

[0197] Figure 11 Another schematic diagram of establishing a specified coordinate system provided in an embodiment of the present application. For example, Figure 11 As shown, the imaging of the defect detection result of process N is taken as an example, where defect A(x A ,y A ) is detected, and the vertex P(x P ,y P ), which is the origin of the coordinate system in the liquid crystal substrate coordinate system. The origin O of the image coordinate system is in the upper left corner of the image. P is the target reference point, A is the defect, and K is the auxiliary reference point.

[0198] To determine the coordinates of defect A in the target defect distribution map, the following transformation steps can be performed:

[0199] Step 1: Translate the coordinate system: move the origin from O to P. Therefore, the coordinates of point A become A'(x A ',y A ')=(x A -x P ,y A -y P ).

[0200] Step 2: Rotate the coordinate system: Rotate the translated coordinate system clockwise by an angle of θ to obtain the liquid crystal substrate coordinate system, where θ is Figure 11 The coordinate system in the middle image is Figure 11 The clockwise rotation angle of the image coordinate system on the right is calculated as follows:

[0201] Figure 12a and Figure 12bA schematic diagram of determining the rotation angle of an image coordinate system provided in an embodiment of the present application; Figure 12a and Figure 12b As shown, Figure 12a and Figure 12b The posture of the detection object in the image is different. P is the target reference point, K is the auxiliary reference point, and each detection object takes the vertex P at the chamfer as the starting point and the direction of the long side of the liquid crystal substrate as the vector. This vector is located on the edge where points p and k are located, and the vertical direction is vector vector That is the unit vector of the y-axis of the image coordinate system, and The angle between them is θ, and the calculation formula of θ can be obtained from point K and point P:

[0202]

[0203] The rotation transformation matrix of the coordinate system rotating clockwise by angle θ is:

[0204]

[0205] Therefore, the coordinate A of defect A in the target defect distribution map (x A ”,y A ”) is calculated as follows:

[0206]

[0207] Right now:

[0208] x A" =x A′ cosθ+y A′ sinθ;

[0209] y A" =y A′ cosθ-x A′ sinθ;

[0210] Among them, the x A" is the horizontal coordinate of defect A in the specified coordinate system, the y A" is the ordinate of defect A in the specified coordinate system, the x A′ is the horizontal coordinate of the relative coordinate of defect A, the y A′ is the ordinate of the relative coordinate of defect A, θ is the angle, x A is the horizontal coordinate of defect A in the target coordinate system, y A is the ordinate of defect A in the target coordinate system, x P is the horizontal coordinate of point P in the target coordinate system, y P is the ordinate of point P in the target coordinate system.

[0211] Therefore, this formula can be used to map the defects detected in different processes into the target defect distribution map.

[0212] In this embodiment, since the short side of the liquid crystal substrate adjacent to the chamfer is perpendicular to the long side of the substrate adjacent to the chamfer, the short side of the substrate adjacent to the chamfer is the x-axis, and the long side of the substrate adjacent to the chamfer is the y-axis. If the detection object is any polygon, the solution in this embodiment only needs to determine one side of the detection object as the designated coordinate axis of the designated coordinate system, and the other coordinate axis does not need to be determined by using the side of the detection object. For example, Figure 11 In the figure, let the designated side be the long side of the substrate adjacent to the chamfer, and the designated coordinate axis be the y-axis. Rotate the coordinate system of the middle image clockwise by θ so that the y-axis of the coordinate system coincides with the long side of the substrate adjacent to the chamfer. At this time, the coordinate system in the image on the right can be obtained, and the obtained coordinate system is the designated coordinate system.

[0213] By using the solution of this embodiment, regardless of the difference in the posture of the inspection object between each process, the position of each first defect map in the target defect distribution map can be effectively determined. Optionally, the method further includes:

[0214] Obtaining optimized defect detection results of multiple inspection objects in the target process;

[0215] Based on the optimized defect detection results of the multiple inspection objects, a defect distribution heat map for the target process is generated.

[0216] The obtained defect detection results of the multiple inspection objects after optimization in the target process may be optimized defect detection results indicating defects.

[0217] By mapping each defect indicated by each optimized defect detection result onto the defect distribution map of the target process, a defect distribution heat map of the target process can be obtained.

[0218] The defect distribution map of the target process can be an image that represents the defect distribution in an object area with the same posture as the detection object of the target process, so that there is no need to convert coordinates. The relative coordinates of each defect indicated by each optimized defect detection result can be used as the position of each defect mapped in the defect distribution map of the target process.

[0219] Specifically, in one implementation, the relative coordinates of each defect indicated by each optimized defect detection result can be used to distribute the indicated defects in the defect distribution map of the target process to obtain a defect distribution heat map of the target process.

[0220] In one implementation, the defect distribution maps corresponding to the optimized defect detection results may be superimposed to obtain a defect distribution heat map of the target process.

[0221] In addition, when the defect detection result includes information such as defect type, information such as defect type of each defect indicated by each optimized defect detection result can also be marked in the defect distribution map of the target process.

[0222] The defect distribution heat map of the target process can effectively characterize information such as the location distribution of defects occurring in the target process, thereby providing an effective basis for optimizing the process.

[0223] Moreover, after optimizing the process using the defect distribution heat map of this embodiment, the occurrence of defects can be reduced. It can be seen that the solution of this embodiment can effectively solve the problem that the process of the production line causes defects to the production object due to mechanism aging, process problems or other external factors, resulting in an increase in the production line downtime rate.

[0224] Corresponding to the above-mentioned defect analysis method, the embodiment of the present application also provides a defect analysis device, such as Figure 13 As shown, the device includes:

[0225] A first determining module 1301 is configured to, in response to generating a first defect detection result indicating the presence of a defect for any inspection object in a target process, determine whether at least one second defect detection result exists for the inspection object; wherein the second defect detection result is a defect detection result indicating the presence of a defect for the inspection object in a process preceding the target process;

[0226] The second determining module 1302 is configured to, for each second defect detection result, determine, if present, a location of each second defect mapped by the second defect detection result in a target defect distribution map, and determine a location of each first defect mapped by the first defect detection result in the target defect distribution map; wherein the target defect distribution map is an image representing defect distribution in an object area of ​​the inspection object;

[0227] An analysis module 1303 is configured to perform repeated defect analysis processing on each second defect detection result based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, to obtain a repeated defect analysis result;

[0228] The deduplication module 1304 is configured to remove duplicate defects indicated by the obtained duplicate defect analysis results from the first defects, and obtain an optimized defect detection result of the inspection object in the target process.

[0229] The solution provided by the embodiment of the present application is to determine whether there is at least one second defect detection result for any detection object in the target process after generating a first defect detection result that characterizes the existence of a defect. If so, it means that there must be a repeated defect in the first defect detection result. At this time, the position of each second defect indicated by the second defect detection result mapped in the target defect distribution map can be determined by executing, for each second defect detection result, determining the position of each second defect indicated by the second defect detection result mapped in the target defect distribution map, and determining the position of each first defect indicated by the first defect detection result mapped in the target defect distribution map. For each second defect detection result, based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, a repeated defect analysis process is performed on each first defect to obtain a repeated defect analysis result step, and the repeated defects in each first defect are determined; the repeated defects indicated by the repeated defect analysis result in each first defect are removed, and the optimized defect detection result of the detection object in the target process can be obtained.

[0230] There are no repeated defects in the optimized defect detection results of the target process. If the optimized defect detection results of the target process indicate that there are no defects, it means that the detection object has not caused defects in the target process and will not cause the production line to stop. If the optimized defect detection results of the target process indicate that there are defects, it will cause the production line to stop, but the reason for the production line to stop is that the target process has caused defects to the detection object. Therefore, by using the optimized defect detection results of the target process, it is possible to effectively reduce the situation where the defects indicated by the defect detection results are only repeated defects, which leads to production line shutdown. It can be seen that the solution of the present application can effectively solve the problem of production line shutdown and production efficiency being affected when the defects indicated by the defect detection results of any process are only repeated defects.

[0231] Optionally, the repeated defect analysis process includes:

[0232] For each first defect, determining a second defect closest to the first defect from among the second defects indicated by the second defect detection result;

[0233] If the distance between the determined second defect and the first defect is smaller than a predetermined threshold, the first defect is determined to be a repeated defect.

[0234] Optionally, the second determining module includes:

[0235] a first determination submodule, configured to determine the relative coordinates of each first defect relative to the first reference point using the coordinates of each first defect indicated by the first defect detection result in a target coordinate system and the coordinates of the first reference point in the target coordinate system; wherein the target coordinate system is an image coordinate system of a first target image, and the first target image is the target image based on which the first defect detection result is determined;

[0236] The second determination submodule is used to determine the position of each first defect mapping in a specified coordinate system based on the relative coordinates of each first defect, so as to obtain the position of each first defect mapping in the target defect distribution map; wherein the specified coordinate system is the image coordinate system of the target defect distribution map.

[0237] Optionally, the designated coordinate system is a coordinate system with the target reference point of the detection object as the origin and the designated side as the designated coordinate axis; the designated side is the side on which the target reference point is located; the first reference point is a point in the first target image that represents the target reference point;

[0238] Before determining the position of each first defect mapped in the designated coordinate system based on the relative coordinates of each first defect, the second determining module further includes:

[0239] a third determining submodule, configured to determine an angle between a specified coordinate axis of the target coordinate system and the specified side of the detection object in the first target image;

[0240] The second determining submodule includes:

[0241] The first calculation unit is configured to calculate the coordinates of each first defect mapping in the specified coordinate system based on the angle and the relative coordinates of each first defect, so as to obtain a position of each first defect mapping in the target defect distribution map.

[0242] Optionally, the third determining submodule includes:

[0243] A first determining unit is configured to determine the coordinates of an auxiliary reference point in the first target image in the target coordinate system; wherein the auxiliary reference point represents any point on the designated edge except the target reference point;

[0244] a second calculation unit, configured to calculate an angle between a unit vector of a designated coordinate axis of the target coordinate system and a direction vector of the designated side of the detection object in the first target image based on the coordinates of the auxiliary reference point in the target coordinate system and the coordinates of the first reference point in the target coordinate system;

[0245] The computing unit comprises:

[0246] The conversion subunit is used to use the angle to convert the relative coordinates of each first defect to the specified coordinate system according to a predetermined coordinate conversion relationship, so as to obtain the coordinates of each first defect in the specified coordinate system; wherein the predetermined coordinate conversion relationship is a conversion relationship with respect to the angle, which is used to convert any relative coordinate to the specified coordinate.

[0247] Optionally, the second determining module includes:

[0248] The first acquisition submodule is configured to acquire the coordinates of each second defect mapping indicated by the second defect detection result in the specified coordinate system, and obtain the position of each second defect mapping indicated by the second defect detection result in the target defect distribution map.

[0249] Optionally, the device further includes:

[0250] An acquisition module is used to obtain the optimized defect detection results of multiple detection objects in the target process; based on the optimized defect detection results of the multiple detection objects, a defect distribution heat map of the target process is generated.

[0251] The present application also provides an electronic device, such as Figure 14 Shown, including:

[0252] Memory 1401, used for storing computer programs;

[0253] The processor 1402 is configured to implement any of the above-mentioned defect analysis methods when executing the program stored in the memory 1401 .

[0254] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 1402, the communication interface, and the memory 1401 communicate with each other via the communication bus.

[0255] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0256] The communication interface is used for communication between the above electronic device and other devices.

[0257] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0258] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0259] In another embodiment provided by the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of any of the above-mentioned defect analysis methods are implemented.

[0260] In another embodiment provided by the present application, a computer program product including instructions is further provided, which, when executed on a computer, enables the computer to execute the defect analysis method described in any one of the above embodiments.

[0261] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD).

[0262] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0263] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0264] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A defect analysis method, characterized in that: The method comprises: In response to generating a first defect detection result indicating the presence of a defect for any inspection object in a target process, determining whether at least one second defect detection result exists for the inspection object; wherein the second defect detection result is a defect detection result indicating the presence of a defect for the inspection object in a process preceding the target process; If present, for each second defect detection result, determining the position of each second defect indicated by the second defect detection result mapped in a target defect distribution map, and determining the position of each first defect indicated by the first defect detection result mapped in the target defect distribution map; wherein the target defect distribution map is an image representing defect distribution in an object area of ​​the inspection object; For each second defect detection result, based on the position of each first defect in the target defect distribution map and the position of each second defect indicated by the second defect detection result in the target defect distribution map, performing repeated defect analysis processing on each first defect to obtain a repeated defect analysis result; The repeated defects indicated by the obtained repeated defect analysis results are removed from the first defects to obtain the optimized defect detection result of the inspection object in the target process.

2. The method according to claim 1, characterized in that The repeated defect analysis process includes: For each first defect, determining a second defect closest to the first defect from among the second defects indicated by the second defect detection result; If the distance between the determined second defect and the first defect is smaller than a predetermined threshold, the first defect is determined to be a repeated defect.

3. The method according to claim 1 or 2, characterized in that Determining the location of each first defect indicated by the first defect detection result mapped on the target defect distribution map includes: Determining the relative coordinates of each first defect relative to the first reference point using the coordinates of each first defect indicated by the first defect detection result in the target coordinate system and the coordinates of the first reference point in the target coordinate system; wherein the target coordinate system is an image coordinate system of a first target image, and the first target image is the target image based on which the first defect detection result is determined; Based on the relative coordinates of each first defect, the position of each first defect map in a specified coordinate system is determined to obtain the position of each first defect map in the target defect distribution map; wherein the specified coordinate system is an image coordinate system of the target defect distribution map.

4. The method according to claim 3, characterized in that The designated coordinate system is a coordinate system having the target reference point of the detection object as its origin and a designated side as a designated coordinate axis; the designated side is the side on which the target reference point is located; and the first reference point is a point in the first target image that represents the target reference point. Before determining the position of each first defect mapped in the designated coordinate system based on the relative coordinates of each first defect, the method further includes: determining an angle between a specified coordinate axis of the target coordinate system and the specified edge of the detection object in the first target image; The determining, based on the relative coordinates of each first defect, the position of each first defect mapping in the specified coordinate system to obtain the position of each first defect mapping in the target defect distribution map includes: Based on the angle and the relative coordinates of each first defect, the coordinates of each first defect mapping in the specified coordinate system are calculated to obtain the position of each first defect mapping in the target defect distribution map.

5. The method according to claim 4, characterized in that Determining an angle between a specified coordinate axis of the target coordinate system and the specified side of the detection object in the first target image includes: Determining the coordinates of an auxiliary reference point in the first target image in the target coordinate system; wherein the auxiliary reference point represents any point on the designated edge except the target reference point; Calculating an angle between a unit vector of a designated coordinate axis of the target coordinate system and a direction vector of a designated side of the inspection object in the first target image based on the coordinates of the auxiliary reference point in the target coordinate system and the coordinates of the first reference point in the target coordinate system; and calculating the coordinates of each first defect mapped in the designated coordinate system based on the angle and the relative coordinates of each first defect, including: Using the angle, according to a predetermined coordinate transformation relationship, the relative coordinates of each first defect are transformed into the specified coordinate system to obtain the coordinates of each first defect in the specified coordinate system; wherein, the predetermined coordinate transformation relationship is a transformation relationship regarding the angle, used to transform any relative coordinate into the specified coordinate.

6. The method according to claim 4, characterized in that The determining of the position of each second defect indicated by the second defect detection result mapped on the target defect distribution map includes: The coordinates of each second defect mapping indicated by the second defect detection result in the specified coordinate system are obtained, and the positions of each second defect mapping indicated by the second defect detection result in the target defect distribution map are obtained.

7. The method according to claim 1 or 2, characterized in that The method further comprises: Obtaining optimized defect detection results of multiple detection objects in the target process; and generating a defect distribution heat map for the target process based on the optimized defect detection results of the multiple detection objects.

8. A defect analysis device, characterized in that: The device comprises: a first determining module configured to, in response to generating a first defect detection result indicating the presence of a defect for any inspection object in a target process, determine whether at least one second defect detection result exists for the inspection object; wherein the second defect detection result is a defect detection result indicating the presence of a defect for the inspection object in a process preceding the target process; a second determining module configured to, for each second defect detection result, determine, if present, a position of each second defect mapped by the second defect detection result in a target defect distribution map, and to determine a position of each first defect mapped by the first defect detection result in the target defect distribution map; wherein the target defect distribution map is an image representing defect distribution in an object area of ​​the inspection object; an analysis module configured to, for each second defect detection result, perform a repeated defect analysis process on each first defect based on a position of each first defect in the target defect distribution map and a position of each second defect indicated by the second defect detection result in the target defect distribution map, to obtain a repeated defect analysis result; The deduplication module is used to remove the repeated defects indicated by the repeated defect analysis results from the first defects to obtain the optimized defect detection results of the detection object in the target process.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images