Novel online intelligent jet printing regulation and control method for display jet printing manufacturing

Abstract

The invention belongs to the related technical field of printing display, and particularly relates to a novel online intelligent jet printing regulation and control method for display jet printing manufacturing, which comprises the following steps of: constructing a bipartite graph structure based on a jet printing defect picture set and a corresponding relation between pixel pits and spray holes, determining jet printing characteristics of each pixel pit by adopting a defect characteristic extraction model, assigning nodes to the pixel pits, and obtaining jet printing characteristics of each pixel pit; assigning the historical printing state data of the spray holes to the spray hole nodes; inputting each bipartite graph into a defect traceability model to obtain a classification result of each spray hole; inputting the historical printing state data of the spray holes and the classification result into a spray hole jet printing parameter regulation and control model to obtain parameter regulation and control values corresponding to the abnormal types of the spray holes; on the basis of the classification result of each spray hole, the to-be-regulated and to-be-controlled abnormal type of the spray hole is screened, so that an abnormal spray hole corresponding to each abnormal pixel pit is determined; and generating a parameter combination scheme based on the to-be-regulated abnormal type parameter regulation value of the spray hole and the normal spray hole parameter regulation value, and performing spray printing planning. The invention relates to an online high-robustness jet printing pattern planning parameter regulation and control method.

Description

Technical Field

[0001] The present invention belongs to the technical field related to printing display, and more specifically, relates to a novel online intelligent printing control method for display printing manufacturing. Background Art

[0002] The use of inkjet printing to manufacture organic light-emitting diodes (OLEDs) has the advantages of low environmental requirements, no need for masks, rapid development, and material savings. It has shown broad application prospects in the fields of smart products such as mobile phones and notebooks, and will become one of the core technologies for the transformation of new displays. However, there are still many problems to be overcome from technical implementation to production and manufacturing. How to achieve online high-robust printing pattern planning and control is an important problem that inkjet OLEDs must face in order to move towards mass production.

[0003] The main problems faced by online printing patterning are sudden printing defects and printing production rhythm. Due to long-term printing manufacturing, the state of the nozzle and the positioning deviation of the substrate will affect the landing accuracy and injection state of the printing, resulting in defects such as deviation, color mixing, and leakage. If the printing parameters are not adjusted in time, the defects will continue to exist, affecting the overall yield rate of the production line products; and in the online printing process, the printing of each substrate must be completed within the beat time. Temporary planning greatly hinders the online printing production rhythm. It is necessary to establish a parallel printing planning calculation module to continuously predict the printing planning parameters and back up multiple planning results, so as to provide effective planning documents in time when abnormalities occur again without affecting the production rhythm.

[0004] Therefore, the field needs an online intelligent printing control method to effectively realize the production line-scale pixel inkjet printing of new displays. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a new online intelligent printing control method for display printing manufacturing, which aims to provide an online highly robust printing pattern planning parameter control method.

[0006] To achieve the above object, according to one aspect of the present invention, a novel online intelligent printing control method for display printing manufacturing is provided, comprising:

[0007] Based on the current printing defect image set and the known correspondence between pixel pits and nozzles determined by the printing plan, all abnormal pixel pits are determined and a bipartite graph structure is constructed in which all abnormal pixel pits are associated and connected by nozzle nodes and pixel pit nodes; the trained pixel pit defect feature extraction model is used to determine the printing features of the image area corresponding to each pixel pit, and the features are assigned to the corresponding pixel pit nodes in the bipartite graph structure; the collected nozzle historical printing status data is assigned to the corresponding nozzle nodes in the bipartite graph structure, and all bipartite graphs corresponding to the current printing are obtained;

[0008] Input each bipartite graph into the trained defect tracing model to obtain the classification result of each nozzle node corresponding to the nozzle, and the classification result is composed of the probabilities of multiple preset abnormal types; input the historical printing status data and classification result of the nozzle corresponding to each nozzle node into the trained nozzle printing parameter control model to obtain the parameter control value corresponding to each abnormal type of the nozzle;

[0009] The probabilities of each abnormal type in the classification results of each nozzle node corresponding to each nozzle in each bipartite graph are sorted to screen the abnormal type of the nozzle to be regulated; according to the probability of the abnormal type to be regulated of all suspected nozzles corresponding to each abnormal pixel pit, the abnormal nozzle corresponding to the abnormal pixel pit is determined;

[0010] Inputting the historical printing state data of each normal nozzle hole and the normal classification data that are not involved in the construction of the bipartite graph structure into the nozzle hole printing parameter control model to obtain the parameter control value corresponding to the normal nozzle hole;

[0011] Based on the parameter control values ​​of the to-be-controlled abnormal type of all abnormal nozzles corresponding to each abnormal pixel pit, the parameter control values ​​of the to-be-controlled abnormal type of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values ​​of all normal nozzles not participating in the construction of the bipartite graph structure, a parameter combination scheme is generated as a new printing planning parameter for the next printing planning and printing.

[0012] Further, the historical printing status data of each nozzle in the nozzle history database includes: the minimum landing point deviation in the X direction, the average landing point deviation in the X direction, the maximum landing point deviation in the X direction, the standard deviation of the landing point deviation in the X direction, the minimum landing point deviation in the Y direction, the average landing point deviation in the Y direction, the maximum landing point deviation in the Y direction, the standard deviation of the landing point deviation in the Y direction, the minimum deviation of the injection volume, the maximum deviation of the injection volume, the average injection volume, the standard deviation of the injection volume deviation, and the number of times as a suspected nozzle.

[0013] Furthermore, the pixel pit defect feature extraction model is obtained by training with a VIT network structure.

[0014] Furthermore, the defect tracing model is obtained by training with a Graphormer network structure.

[0015] Furthermore, the nozzle printing parameter control model is obtained by training a neural network using a DQN reinforcement learning algorithm.

[0016] Furthermore, each bipartite graph structure is constructed as follows:

[0017] According to the current printing defect picture set, the number of each abnormal pixel pit is determined, and according to the known correspondence between the pixel pit and the nozzle hole determined by the printing plan, all nozzle holes corresponding to the number of the abnormal pixel pit are determined as suspected nozzle holes; all normal pixel pits involved in the printing of each suspected nozzle hole are taken as related pixel pits; and nozzle holes that participated in the printing of the related pixel pits and did not participate in the printing of the abnormal pixel pits are taken as related nozzle holes;

[0018] Each abnormal pixel pit and all its corresponding related pixel pits, all suspected nozzles, all related nozzles, and other abnormal pixel pits involved by all suspected nozzles and all their corresponding related pixel pits, all suspected nozzles, and all related nozzles are taken as a node in a bipartite graph structure, and each pixel pit node is connected to each nozzle node that prints it, so as to obtain a bipartite graph structure; wherein a connection weight is set on each connection line, and the weight value is the number of drops of the pixel pit on the connection line printed by the nozzle on the connection line, and there is no node connection between the two bipartite graph structures.

[0019] Furthermore, the abnormal nozzle corresponding to each abnormal pixel pit is determined as follows:

[0020] Based on the current printing defect image set and the prior relationship between image grayscale and the number of droplets, the number of droplets in the abnormal pixel pit is determined by the grayscale of each abnormal pixel pit image, and the number of droplet missing in the abnormal pixel pit is determined by combining it with the number of droplets required for normal injection. The abnormal pixel pits are sorted from large to small according to the line weights associated with the bipartite graph, and the number of droplet missing is subtracted from the line weights in sequence until it is reduced to 0 or below 0 in one cycle, and the total number of cycles is taken as the number n of abnormal nozzles corresponding to the abnormal pixel pit; according to the The classification results of all the suspected orifices of the abnormal pixel pit are removed, and the nozzles classified as normal are sorted according to the probability of the abnormal type to be regulated determined by the screening of the remaining suspected nozzles; if n is equal to the number of the remaining suspected nozzles, the remaining suspected nozzles are used as the abnormal nozzles corresponding to the abnormal pixel pit; if n is less than the number of the remaining suspected nozzles, n are determined from the remaining suspected nozzles by random permutation and combination as the abnormal nozzles corresponding to the abnormal pixel pit.

[0021] Furthermore, the method for screening and determining the abnormal type to be regulated for each nozzle in the bipartite graph is as follows:

[0022] Based on the ranking of the probabilities of each abnormal type of each nozzle, determine whether the second highest abnormal type probability of the nozzle is greater than half of the first highest abnormal type probability. If so, the abnormal type corresponding to the first two highest abnormal type probabilities of the nozzle is used as the abnormal type to be regulated for the nozzle; otherwise, the abnormal type corresponding to the highest abnormal type probability of the nozzle is used as the abnormal type to be regulated for the nozzle.

[0023] Furthermore, the new printing planning parameters are determined as follows:

[0024] The parameter control values ​​of the to-be-controlled abnormal type of all abnormal nozzles corresponding to each abnormal pixel pit, the parameter control values ​​of the to-be-controlled abnormal type of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values ​​of all normal nozzles not participating in the construction of the bipartite graph structure are combined to generate a variety of parameter combination schemes. According to the total abnormal type probability corresponding to each parameter combination scheme, the parameter combination scheme corresponding to the highest total abnormal type probability is selected as the new printing planning parameter.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the steps of the method described above.

[0026] In general, compared with the prior art, the technical solution conceived by the present invention has the following beneficial effects:

[0027] 1. The present invention proposes a novel online intelligent printing control method for display printing manufacturing. Based on offline training data training, a pixel pit defect feature extraction model, a defect tracing model and a nozzle printing parameter control model are obtained to perform online printing manufacturing parameter control. The three models can update parameters in real time through online data, and can ensure the control accuracy in real time. Among them, the pixel pit defect feature extraction model adopts image convolution, and the defect tracing model adopts a graph neural network. The combination of the two further enhances the reasoning ability of defect tracing and improves the accuracy of defect tracing. In addition, the present invention introduces a bipartite graph as the input of the defect tracing model. By describing the correspondence between the nozzle and the pixel pit as a bipartite graph structure, the topological structure of the connection between the nozzle and the pixel pit is greatly retained during training, so that the features between the nodes are transmitted through edge connections, making the expression ability of the graph neural network much higher than that of the ordinary neural network. Furthermore, by combining the parameter control values ​​of the abnormal type to be regulated of all abnormal nozzles corresponding to each abnormal pixel pit, the parameter control values ​​of the abnormal type to be regulated of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values ​​of all normal nozzles not participating in the construction of the bipartite graph structure, a variety of parameter combination schemes are generated for printing planning, which further enhances the robustness of online printing. Therefore, the present invention realizes the online high-robustness parameter control of printing pattern planning.

[0028] 2. The present invention further proposes a bipartite graph construction method, which describes the correspondence between the nozzles and the pixel pits as a bipartite graph structure. There is no node connection between the two bipartite graph structures, which makes the node relationship clearer. Therefore, when the defect tracing model is used for defect tracing, a more accurate classification result of the nozzle corresponding to each nozzle node is obtained. In addition, a connection weight is set on each connection line of the bipartite graph structure, and the weight value is the number of drops printed by the nozzle on the connection line for the pixel pit on the connection line. This is more in line with the flexible change of the number of drops printed by a nozzle on the target pixel pit in actual printing, thereby ensuring the generalization of the application of the method of the present invention.

[0029] 3. The present invention further proposes a method for determining abnormal nozzle holes, which determines the number of abnormal droplets according to the grayscale of pixels after printing is completed, and combines the connection structure of the bipartite graph to determine the number of abnormal nozzle holes as prior knowledge of the printing control scheme, accurately eliminating inaccurate control schemes and improving the accuracy of the control process.

[0030] 4. The present invention further proposes a method for determining the type of abnormality to be regulated, and selecting the first one or two types to be regulated by probability reasonably omits excessive and meaningless printing parameter combinations, saves computing resources of the printing process, and reduces interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of a novel online intelligent printing control method for display printing manufacturing provided by an embodiment of the present invention;

[0032] Figure 2 It is a schematic diagram of the structure of the nozzle historical printing status database provided by an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of an intelligent hybrid printing planning algorithm provided by an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of a printing defect data set provided by an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of a pixel pit defect feature extraction model provided in an embodiment of the present invention;

[0036] Figure 6 A schematic diagram of a defect tracing model provided by an embodiment of the present invention;

[0037] Figure 7 A schematic diagram of a nozzle printing parameter control model provided in an embodiment of the present invention;

[0038] Figure 8 A schematic diagram of a printing planning parameter sorting algorithm provided in an embodiment of the present invention;

[0039] Fig. 9 A diagram of a printing control framework provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0041] Embodiment 1

[0042] A new type of online intelligent printing control method for display printing manufacturing, such as Figure 1 As shown, including:

[0043] Collect all the pictures of printing defects after printing a printing plan online to form a printing defect picture set. Based on the current printing defect picture set and the known correspondence between pixel pits and nozzles determined by the printing plan, determine all abnormal pixel pits and construct a bipartite graph structure in which all abnormal pixel pits are associated and connected by nozzle nodes and pixel pit nodes. Use the trained pixel pit defect feature extraction model to determine the printing features of the image area corresponding to each pixel pit, assign them to the corresponding pixel pit nodes in the bipartite graph structure, assign the collected nozzle historical printing status data to the corresponding nozzle nodes in the bipartite graph structure, and obtain all the bipartite graphs corresponding to the current printing.

[0044] Input each bipartite graph into the trained defect tracing model to obtain the classification result of each nozzle node corresponding to the nozzle, and the classification result is composed of the probabilities of multiple preset abnormal types; input the historical printing status data and classification result of the nozzle corresponding to each nozzle node into the trained nozzle printing parameter control model to obtain the parameter control value corresponding to each abnormal type of the nozzle;

[0045] The probabilities of each abnormal type in the classification results of each nozzle node corresponding to each nozzle in each bipartite graph are sorted to screen the abnormal type of the nozzle to be regulated; according to the probability of the abnormal type to be regulated of all suspected nozzles corresponding to each abnormal pixel pit, the abnormal nozzle corresponding to the abnormal pixel pit is determined;

[0046] Inputting the historical printing state data of each normal nozzle hole and the normal classification data that are not involved in the construction of the bipartite graph structure into the nozzle hole printing parameter control model to obtain the parameter control value corresponding to the normal nozzle hole;

[0047] Based on the parameter control values ​​of the to-be-controlled abnormal type of all abnormal nozzles corresponding to each abnormal pixel pit, the parameter control values ​​of the to-be-controlled abnormal type of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values ​​of all normal nozzles not participating in the construction of the bipartite graph structure, a parameter combination scheme is generated as a new printing planning parameter for the next printing planning and printing.

[0048] During the online execution process, there will be a nozzle historical printing status database, which contains statistics of short-term printing data of all nozzles, including the volume, angle, speed and printing performance of the nozzle spray. It has two functions: a. As a parameter for printing planning, such as the different volumes and landing point errors of different nozzles, accurate measurement can improve printing accuracy; b. As a feature of the nozzle, it is used to locate abnormalities and predictions.

[0049] As a preferred example, Figure 2 As shown, the nozzle historical printing status database stores various printing indicators and status of all nozzles. In this database, the X-direction landing error, Y-direction landing error, injection volume and number of suspected times are mainly counted for each nozzle, among which the X-direction landing error, Y-direction landing error and injection volume all contain minimum value, mean value, maximum value and standard deviation as specific parameters, and the number of suspected nozzles means the number of times the defective pixel pit is involved in the printing after the printing defect occurs, which is a cumulative value in the entire online printing process. Based on the injection state performance of the nozzle, the state vector of each nozzle can be constructed:

[0050] F pn = <x min ,x mean ,x max ,x std ,ymin ,…,v std ,f sus >

[0051] That is, the historical printing status data of each nozzle in the nozzle history database includes: the minimum landing point deviation in the X direction, the average landing point deviation in the X direction, the maximum landing point deviation in the X direction, the standard deviation of the landing point deviation in the X direction, the minimum landing point deviation in the Y direction, the average landing point deviation in the Y direction, the maximum landing point deviation in the Y direction, the standard deviation of the landing point deviation in the Y direction, the minimum deviation of the injection volume, the maximum deviation of the injection volume, the average injection volume, the standard deviation of the injection volume deviation, and the number of times as a suspected nozzle.

[0052] In more detail, the data in the nozzle historical printing status database is collected through ink droplet observation, landing point detection and defect tracing. Ink droplet observation can observe parameters such as the injection volume, emission angle and speed of each nozzle in a static state; while landing point detection directly obtains the X, Y errors of the printing landing point during printing; after the printing is completed, the abnormal pixel pit number is traced back based on the defect location information, and then the associated suspected nozzle is obtained, and the number of suspected times is counted.

[0053] During the online execution process, printing planning issues are also involved in order to establish an integer programming model. One implementation method is to use the pixel pit volume as a constraint and the minimum number of prints as the optimization goal, and to reduce the pixel pit volume error by combining the volumes of different nozzles.

[0054] A specific implementation method is to set multiple printing stops on the printing path, generate the positional relationship between the nozzle and the pixel pit at each stop point, determine whether it can be sprayed, and build an integer programming model. The volume uniformity between all pixels is constrained, the minimization of the number of printing times is the optimization goal, and the number of nozzle printing sequences is the decision variable to build a planning model:

[0055]

[0056] In addition, the results of the printing planning include the nozzle's selection of stop points and the number of times each nozzle prints at each stop point, and then it can be concluded which nozzles spray each pixel pit. That is, the aforementioned known correspondence between pixel pits and nozzles determined by the printing planning can be used to construct a bipartite graph structure.

[0057] More specifically, intelligent hybrid printing planning algorithms, such as Figure 3 As shown, the modeling method is:

[0058] S1, set the number of nozzles to m, the number of pixels to n, set p equidistant printing stops on the printing path, and at each stop position, determine the positional relationship between all nozzles and all pixel pits, using the coefficient matrix A n×(m×p)Indicates whether the nozzle k at the jth stop can spray pixel i, and the coefficient is expressed as a i,k×j , when the position relationship allows injection a i,k×j Set to the volume value of nozzle k, otherwise set to 0;

[0059] S2. In order to constrain the volume value of each pixel pit, set the injection number variable X of all nozzles at each stop point. For the specific injection number variable of nozzle k at the jth stop point, use x k×j,1 Indicates that the range size is Only integers are taken, where V p is the standard volume of a pixel pit, V dn is the volume median of all nozzles, based on which the volume constraint of each pixel pit is established to be within the upper and lower limits of the volume [v lb ,v ub ];

[0060] S3. Under the premise of satisfying the volume, it is necessary to set the optimization goal of minimizing the number of printings to reduce the time required for printing. The nozzle with the largest number of injections at each stop point is the number of prints at that stop point, which is set as Y. The number of prints at stop point j is y. j , so the optimization goal is to minimize the number of printing times for all stops, and complete the establishment of the printing planning model:

[0061]

[0062] In addition, the results of the printing planning include the nozzle's choice of stop points and the number of times each nozzle prints at each stop point, and then it can be determined which nozzles spray each pixel pit, which serves as a necessary network relationship for subsequent tracing.

[0063] During the online execution process, it can be used as a specific implementation method to construct a printing defect data set, which collects unqualified printing defects on the pixel pit substrate after printing, such as scattered points, mis-spraying, etc. At the same time, combined with the correspondence between the nozzle and the pixel pit, a network relationship between the defective (abnormal) pixel pit and the suspected nozzle is constructed, that is, a bipartite graph, which is used for subsequent defect tracing.

[0064] Specifically, the printing defect dataset is a comprehensive data set that combines defect images, labeled nozzle nodes and pixel nodes, and nozzle control results obtained by the nozzle printing parameter control model. In order to enrich the defect dataset of printing and realize supervised training, known abnormal nozzles are used to participate in printing planning, a large number of printing defects are artificially created, defects are collected through AOI, and defect features are extracted through the pixel pit defect feature extraction model, and relevant location information is provided; the bipartite graph data includes: 1) a bipartite graph structure constructed by the connection relationship between the pixel pit and the nozzle output by the printing planning result and the abnormal pixel pit number, including nozzle nodes and pixel pit nodes and the connecting edges between them; 2) node features, the features of the nozzle node are composed of the historical printing data, and the features of the pixel pit node are obtained by processing the pixel pit defect image by the trained pixel pit defect feature extraction model, which is the training data of the intelligent defect tracing model; the nozzle control result is the parameter that needs to be regulated for each abnormal nozzle or the flag of whether it is enabled, which is the training data of the nozzle printing parameter control model.

[0065] More specifically, the printing defect dataset, such as Figure 4 As shown, specifically:

[0066] S1. In order to enrich the defect data set of printing and realize supervised training, known abnormal nozzles are used to participate in printing planning, a large number of printing defects are artificially created, and defects are collected through AOI and the composition 4) printing defect visual detection network realizes defect feature extraction and provides relevant location information;

[0067] S2, the bipartite graph structure connecting the nozzle node and the pixel node is generated by combining the printing planning result, which is the training data of the intelligent defect tracing network;

[0068] S3. The nozzle control result is the parameter that needs to be controlled for each abnormal nozzle or the flag of whether it is enabled, which is the training data of the nozzle printing parameter control model.

[0069] As a preferred implementation, each bipartite graph structure is constructed as follows:

[0070] According to the current printing defect picture set, the number of each abnormal pixel pit is determined, and according to the known correspondence between the pixel pit and the nozzle hole determined by the printing plan, all nozzle holes corresponding to the number of the abnormal pixel pit are determined as suspected nozzle holes; all normal pixel pits involved in the printing of each suspected nozzle hole are taken as related pixel pits; and nozzle holes that participated in the printing of the related pixel pits and did not participate in the printing of the abnormal pixel pits are taken as related nozzle holes;

[0071] Each abnormal pixel pit and all its corresponding related pixel pits, all suspected nozzles, all related nozzles, and other abnormal pixel pits involved by all suspected nozzles and all their corresponding related pixel pits, all suspected nozzles, and all related nozzles are taken as a node in a bipartite graph structure, and each pixel pit node is connected to each nozzle node that prints it, so as to obtain a bipartite graph structure; wherein a connection weight is set on each connection line, and the weight value is the number of drops of the pixel pit on the connection line printed by the nozzle on the connection line, and there is no node connection between the two bipartite graph structures.

[0072] The pixel pit defect feature extraction model is based on the defect image and uses an intelligent image processing algorithm to extract the defect features of the substrate after printing, complete defect identification, and feedback the location of the defect.

[0073] It can be used as a preferred embodiment, such as Figure 5 As shown in the figure, the pixel pit defect feature extraction model is the VIT image feature extraction network. VIT uses the self-attention mechanism to capture the global dependencies in the image. In this method, the collected printing defect images are divided into blocks for semantic extraction to avoid the storage pressure caused by large images in the training process. After segmentation, each image is given a position code. After regularization, a multi-head attention module is used to extract information. The output result is added with residual connection, and then regularized and finally connected to the fully connected layer to add residual connection output. The above process becomes an encoder, and the prediction result is output after cyclically connecting multiple encoders.

[0074] As a supplement, the images include scattered point defects, mixed color defects, missed defects, etc. The extracted feature codes are used for defect classification and as the features of the pixel nodes of the bipartite graph in the printing defect data set. The results of defect classification are combined and used as the features of the pixel nodes.

[0075] In the online execution process, the defect tracing model is also involved, which can be used as an optimal choice, such as Figure 6 As shown in the figure, it is a multi-head graph attention network. The bipartite graph structure connecting the labeled nozzle nodes and pixel nodes of the printing defect dataset is used as the training data of the defect tracing model. The bipartite graph specifically includes four types of labeled nodes, namely abnormal pixel pit nodes, suspected nozzle nodes, related pixel pit nodes and related nozzle nodes. The essence is to classify the nozzle nodes and trace the abnormal nozzle nodes. The network makes full use of the information of the graph. The initial features are added with the centrality features of the graph. After self-attention, the spatial structure features and edge connection features of the bipartite graph are added. In order to fully extract features during network training, an asynchronous information extraction mode is used:

[0076]

[0077] in, represents the encoding of the pixel pit node at the l+1th layer, and represents the code of the nozzle nodes at the lth and l+1th layers, a n,i,j and a p,i,j are the attention coefficients of the nozzle node and the pixel possible node, respectively, l is the weight of the lth layer.

[0078] The online execution process will also involve a nozzle printing parameter control model, which is used to control the printing planning parameters according to the classification results output by the defect tracing model and the nozzle historical printing status data. Preferably, the two networks are divided into a landing point control network and a shielding network, which respectively output the control values ​​of nozzles of different defect types and the corresponding probability distribution.

[0079] Specifically, Figure 7 As shown in the figure, the nozzle printing parameter control model includes two three-layer fully connected neural networks, which are used to predict the X and Y landing point control values ​​of each nozzle and whether the nozzle is shielded. The neural network is trained by the DQN reinforcement learning algorithm. The input of the network is the characteristic data of the nozzle history printing state database combined with the classification result of the intelligent defect tracing model. One of the two networks is selected for prediction, and the control result is the control deviation probability distribution of X, Y or X and Y and the binary classification probability of whether it is shielded.

[0080] The online execution process also involves a printing planning parameter sorting algorithm, which sorts the printing participation states of all nozzles according to the probability distribution of the defect tracing model prediction classification results and parameter regulation, and creates multiple sets of different plannable parameters to plan patterns online.

[0081] Printing planning parameter sorting algorithm, such as Figure 8 As shown in the figure, after the nozzle printing parameter control model predicts the control parameters of all nozzles, the control nodes are selected according to the following rules: A. Prioritize the nozzles with abnormalities and select them according to the probability of the selected abnormal types to be controlled; B. Predict the nozzles that may have printing defects according to the historical characteristics of the nozzles to obtain the control parameters. Sort the selected abnormal types to be controlled for each nozzle from large to small according to the probability value, and use the probability of the abnormal type to be controlled for each nozzle as the basis for judgment. Based on the sorting of the probabilities of each abnormal type of each nozzle, judge whether the second highest abnormal type probability of the nozzle is greater than half of the first highest abnormal type probability. If so, the abnormal type corresponding to the first two highest abnormal type probabilities of the nozzle is used as the abnormal type to be controlled for the nozzle. Otherwise, the abnormal type corresponding to the highest abnormal type probability of the nozzle is used as the abnormal type to be controlled for the nozzle. Multiple sets of printing planning parameters are formed, and all planning parameters are pre-planned according to the intelligent hybrid printing planning algorithm as a backup plan.

[0082] As a preferred implementation, the abnormal nozzle corresponding to each abnormal pixel pit is determined as follows:

[0083] Based on the current printing defect image set and the prior relationship between image grayscale and the number of droplets, the number of droplets in the abnormal pixel pit is determined by the grayscale of each abnormal pixel pit image, and the number of droplet missing in the abnormal pixel pit is determined by combining it with the number of droplets required for normal injection. The abnormal pixel pits are sorted from large to small according to the line weights associated with the bipartite graph, and the number of droplet missing is subtracted from the line weights in sequence until it is reduced to 0 or below 0 in one cycle, and the total number of cycles is taken as the number n of abnormal nozzles corresponding to the abnormal pixel pit; according to the The classification results of all the suspected orifices of the abnormal pixel pit are removed, and the nozzles classified as normal are sorted according to the probability of the abnormal type to be regulated determined by the screening of the remaining suspected nozzles; if n is equal to the number of the remaining suspected nozzles, the remaining suspected nozzles are used as the abnormal nozzles corresponding to the abnormal pixel pit; if n is less than the number of the remaining suspected nozzles, n are determined from the remaining suspected nozzles by random permutation and combination as the abnormal nozzles corresponding to the abnormal pixel pit.

[0084] As a preferred implementation, the method for screening and determining the abnormal type to be regulated for each nozzle in the bipartite graph is as follows:

[0085] Based on the ranking of the probabilities of each abnormal type of each nozzle, determine whether the second highest abnormal type probability of the nozzle is greater than half of the first highest abnormal type probability. If so, the abnormal type corresponding to the first two highest abnormal type probabilities of the nozzle is used as the abnormal type to be regulated for the nozzle; otherwise, the abnormal type corresponding to the highest abnormal type probability of the nozzle is used as the abnormal type to be regulated for the nozzle.

[0086] As a preferred implementation, the new printing planning parameters are determined as follows:

[0087] The parameter control values ​​of the to-be-controlled abnormal type of all abnormal nozzles corresponding to each abnormal pixel pit, the parameter control values ​​of the to-be-controlled abnormal type of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values ​​of all normal nozzles not participating in the construction of the bipartite graph structure are combined to generate a variety of parameter combination schemes. According to the total abnormal type probability corresponding to each parameter combination scheme, the parameter combination scheme corresponding to the highest total abnormal type probability is selected as the new printing planning parameter.

[0088] In order to facilitate understanding of the method of this embodiment, the overall framework can be seen Fig. 9 shown.

[0089] The online intelligent printing closed-loop operation method is as follows:

[0090] Step 1: Before formal production, first carry out processes such as ink droplet observation, landing point accuracy detection, and trial printing to complete the data collection of the current nozzles and build an offline nozzle history database and printing defect data set.

[0091] Step 2: Offline training of the printing defect visual inspection network, intelligent defect tracing network and printing planning parameter control network to improve the accuracy of network defect recognition, tracing and control.

[0092] Step 3: Sort the parameters obtained from the most recent printing defect results according to the printing planning parameter sorting algorithm, select the top ten printing planning parameters, and proceed with the printing planning process.

[0093] Step 4: Perform trial printing according to the sorting and planning results, and start online printing after no defects are found.

[0094] Step 5: During the online printing execution time, continuously simulate the nozzles that may have defects, generate new printing planning parameters, and cycle the printing planning process.

[0095] Step 6: After the defect occurs, the defect is collected and passed through the printing defect visual inspection network and the intelligent defect tracing network respectively, the nozzle history database and the printing defect data set are updated, new parameters are generated for planning, and the most similar shielding nozzle data is selected based on the existing planning to execute the printing process.

[0096] This method can be used in long-term inkjet display manufacturing to perform closed-loop feedback adjustment of planning strategies based on detected defects, effectively predict and suppress defect generation, thereby improving manufacturing yield and reducing the manpower and time costs of regulation.

[0097] Embodiment 2

[0098] The present application also relates to a computer-readable storage medium having a computer program stored thereon, and the steps of the above method are implemented when the computer program is executed by a processor.

[0099] Specifically, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0100] The relevant technical solutions are the same as above and will not be described again here.

[0101] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A novel online intelligent printing control method for display printing manufacturing, characterized in that: include: Based on the current printing defect image set and the known correspondence between pixel pits and nozzles determined by the printing plan, all abnormal pixel pits are determined and a bipartite graph structure is constructed in which all abnormal pixel pits are associated and connected by nozzle nodes and pixel pit nodes; the trained pixel pit defect feature extraction model is used to determine the printing features of the image area corresponding to each pixel pit, and the features are assigned to the corresponding pixel pit nodes in the bipartite graph structure; the collected nozzle historical printing status data is assigned to the corresponding nozzle nodes in the bipartite graph structure, and all bipartite graphs corresponding to the current printing are obtained; Input each bipartite graph into the trained defect tracing model to obtain the classification result of each nozzle node corresponding to the nozzle, and the classification result is composed of the probabilities of multiple preset abnormal types; input the historical printing status data and classification result of the nozzle corresponding to each nozzle node into the trained nozzle printing parameter control model to obtain the parameter control value corresponding to each abnormal type of the nozzle; The probabilities of each abnormal type in the classification results of each nozzle node corresponding to each nozzle in each bipartite graph are sorted to screen the abnormal type of the nozzle to be regulated; according to the probability of the abnormal type to be regulated of all suspected nozzles corresponding to each abnormal pixel pit, the abnormal nozzle corresponding to the abnormal pixel pit is determined; Inputting the historical printing state data of each normal nozzle hole and the normal classification data that are not involved in the construction of the bipartite graph structure into the nozzle hole printing parameter control model to obtain the parameter control value corresponding to the normal nozzle hole; Based on the parameter control values ​​of the to-be-controlled abnormal type of all abnormal nozzles corresponding to each abnormal pixel pit, the parameter control values ​​of the to-be-controlled abnormal type of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values ​​of all normal nozzles not participating in the construction of the bipartite graph structure, a parameter combination scheme is generated as a new printing planning parameter for the next printing planning and printing.

2. A novel online intelligent printing control method for display printing manufacturing as claimed in claim 1, characterized in that: The historical printing status data of each nozzle in the nozzle history database includes: the minimum landing point deviation in the X direction, the average landing point deviation in the X direction, the maximum landing point deviation in the X direction, the standard deviation of the landing point deviation in the X direction, the minimum landing point deviation in the Y direction, the average landing point deviation in the Y direction, the maximum landing point deviation in the Y direction, the standard deviation of the landing point deviation in the Y direction, the minimum deviation of the ejection volume, the maximum deviation of the ejection volume, the average ejection volume, the standard deviation of the ejection volume deviation, and the number of times as a suspected nozzle.

3. A novel online intelligent printing control method for display printing manufacturing as claimed in claim 1, characterized in that: The pixel pit defect feature extraction model is obtained by training with a VIT network structure.

4. A novel online intelligent printing control method for display printing manufacturing as claimed in claim 1, characterized in that: The defect tracing model is obtained by training with a Graphormer network structure.

5. A novel online intelligent printing control method for display printing manufacturing as claimed in claim 1, characterized in that: The nozzle printing parameter control model is obtained by training a neural network using a DQN reinforcement learning algorithm.

6. A novel online intelligent printing control method for display printing manufacturing as claimed in claim 1, characterized in that: Each bipartite graph structure is constructed as follows: According to the current printing defect picture set, the number of each abnormal pixel pit is determined, and according to the known correspondence between the pixel pit and the nozzle hole determined by the printing plan, all nozzle holes corresponding to the number of the abnormal pixel pit are determined as suspected nozzle holes; all normal pixel pits involved in the printing of each suspected nozzle hole are taken as related pixel pits; and nozzle holes that participated in the printing of the related pixel pits and did not participate in the printing of the abnormal pixel pits are taken as related nozzle holes; Each abnormal pixel pit and all its corresponding related pixel pits, all suspected nozzles, all related nozzles, and other abnormal pixel pits involved by all suspected nozzles and all their corresponding related pixel pits, all suspected nozzles, and all related nozzles are taken as a node in a bipartite graph structure, and each pixel pit node is connected to each nozzle node that prints it, so as to obtain a bipartite graph structure; wherein a connection weight is set on each connection line, and the weight value is the number of drops of the pixel pit on the connection line printed by the nozzle on the connection line, and there is no node connection between the two bipartite graph structures.

7. A novel online intelligent printing control method for display printing manufacturing as claimed in claim 6, characterized in that: The abnormal nozzle corresponding to each abnormal pixel pit is determined as follows: Based on the current printing defect image set and the prior relationship between image grayscale and the number of droplets, the number of droplets in the abnormal pixel pit is determined by the grayscale of each abnormal pixel pit image, and the number of droplet missing in the abnormal pixel pit is determined by combining it with the number of droplets required for normal injection. The abnormal pixel pits are sorted from large to small according to the line weights associated with the bipartite graph, and the number of droplet missing is subtracted from the line weights in sequence until it is reduced to 0 or below 0 in one cycle, and the total number of cycles is taken as the number n of abnormal nozzles corresponding to the abnormal pixel pit; according to the The classification results of all the suspected orifices of the abnormal pixel pit are removed, and the nozzles classified as normal are sorted according to the probability of the abnormal type to be regulated determined by the screening of the remaining suspected nozzles; if n is equal to the number of the remaining suspected nozzles, the remaining suspected nozzles are used as the abnormal nozzles corresponding to the abnormal pixel pit; if n is less than the number of the remaining suspected nozzles, n are determined from the remaining suspected nozzles by random permutation and combination as the abnormal nozzles corresponding to the abnormal pixel pit.

8. A novel online intelligent printing control method for display printing manufacturing as claimed in claim 1, characterized in that: The method of screening and determining the abnormal type to be regulated for each nozzle in the bipartite graph is: Based on the ranking of the probabilities of each abnormal type of each nozzle, determine whether the second highest abnormal type probability of the nozzle is greater than half of the first highest abnormal type probability. If so, the abnormal type corresponding to the first two highest abnormal type probabilities of the nozzle is used as the abnormal type to be regulated for the nozzle; otherwise, the abnormal type corresponding to the highest abnormal type probability of the nozzle is used as the abnormal type to be regulated for the nozzle.

9. A novel online intelligent printing control method for display printing manufacturing as claimed in claim 8, characterized in that: The new printing planning parameters are determined as follows: The parameter control values ​​of the to-be-controlled abnormal type of all abnormal nozzles corresponding to each abnormal pixel pit, the parameter control values ​​of the to-be-controlled abnormal type of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values ​​of all normal nozzles not participating in the construction of the bipartite graph structure are combined to generate a variety of parameter combination schemes. According to the total abnormal type probability corresponding to each parameter combination scheme, the parameter combination scheme corresponding to the highest total abnormal type probability is selected as the new printing planning parameter.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to perform the steps of the method according to any one of claims 1 to 9.

Images