An online intelligent jet printing regulation method for display jet printing manufacturing

Abstract

The present application belongs to the technical field of printing display, and particularly relates to an online intelligent jet printing regulation method for display jet printing manufacturing, comprising the following steps: constructing a two-part graph structure based on a jet printing defect picture set and a pixel pit and a nozzle hole corresponding relationship, determining jet printing features of each pixel pit by using a defect feature extraction model, assigning the pixel pit nodes, and assigning the nozzle hole historical printing state data to the nozzle hole nodes; inputting each two-part graph into a defect tracing model to obtain a classification result of each nozzle hole; inputting the nozzle hole historical printing state data and the classification result into a nozzle hole jet printing parameter regulation model to obtain parameter regulation values corresponding to each abnormal type of the nozzle hole; screening the abnormal types of the nozzle hole to be regulated based on the classification result of each nozzle hole to determine the abnormal nozzle hole corresponding to each abnormal pixel pit; generating a parameter combination scheme based on the parameter regulation values of the abnormal types of the nozzle hole to be regulated and the parameter regulation values of the normal nozzle hole to perform jet printing planning. The present application is an online high-robustness jet printing pattern planning parameter regulation method.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of printed display, and more particularly relates to an online intelligent jet printing regulation method for new display jet printing manufacturing. BACKGROUND

[0002] The organic light-emitting diode (OLED) manufactured by the jet printing method has the advantages of low environmental requirements, no need for a mask plate, fast development, and material saving, and thus has broad application prospects in the field of smart products such as mobile phones and notebooks, and will become one of the core technologies of new display transformation. However, many problems still need to be overcome from the technical implementation to the production manufacturing, and how to realize online high-robustness jet printing pattern planning and regulation is an important problem that must be faced by the jet printing OLED to go to mass production.

[0003] The main problems of online jet printing patterning are sudden jet printing defects and jet printing production rhythm. Due to long-time jet printing manufacturing, the state of the jet head and the positioning deviation of the substrate will affect the landing accuracy and jetting state of the jet printing, thereby causing defects such as landing deviation, color mixing, and missing jetting. If the jet printing parameters are not adjusted in time, the defects will continue to exist, affecting the overall yield of the product line. In the online jet printing process, each substrate needs to be completed within the rhythm time, and temporary planning greatly hinders the online jet printing production rhythm. It is necessary to establish a parallel jet printing planning calculation module to constantly predict jet printing planning parameters and backup multiple planning results, so as to provide effective planning files in time when an exception occurs without affecting the production rhythm.

[0004] Therefore, there is a need in the field for an online intelligent jet printing regulation method to effectively realize pixel jet printing of the production line scale of new display. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides an online intelligent jet printing regulation method for new display jet printing manufacturing, which aims to provide an online high-robustness jet printing pattern planning parameter regulation method.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, an online intelligent jet printing regulation method for new display jet printing manufacturing is provided, comprising:

[0007] Based on the current printing defect picture set and the known correspondence between pixel pits and nozzles determined by the printing plan, all abnormal pixel pits are determined and a bipartite graph structure is constructed, in which all abnormal pixel pits are associated and connected by nozzle nodes and pixel pit nodes; a trained pixel pit defect feature extraction model is used to determine the printing features of the corresponding image area of each pixel pit, which are assigned to the corresponding pixel pit nodes in the bipartite graph structure, and the collected historical printing state data of the nozzles are assigned to the corresponding nozzle nodes in the bipartite graph structure, to obtain all bipartite graphs corresponding to the current printing;

[0008] Each bipartite graph is input into a trained defect tracing model to obtain the classification results of the corresponding nozzles of each nozzle node, which consist of probabilities of multiple abnormal types; the historical printing state data and classification results of each nozzle node corresponding to the nozzles are input into a trained nozzle printing parameter regulation model to obtain parameter regulation values corresponding to each abnormal type of the nozzles;

[0009] The probabilities of each abnormal type in the classification results of each nozzle node corresponding to the nozzles in each bipartite graph are sorted to screen abnormal types to be regulated for the nozzles; and the abnormal nozzles corresponding to each abnormal pixel pit are determined according to the abnormal types to be regulated and the probability sizes of all suspected nozzles corresponding to the abnormal pixel pit;

[0010] The historical printing state data and normal classification data of each normal nozzle not participating in the construction of the bipartite graph structure are input into the nozzle printing parameter regulation model to obtain parameter regulation values corresponding to the normal nozzles;

[0011] Based on the abnormal type parameter regulation values of all abnormal nozzles corresponding to each abnormal pixel pit, the abnormal type parameter regulation values of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter regulation values of all normal nozzles not participating in the construction of the bipartite graph structure, a parameter combination scheme is generated as a new printing plan parameter for the next printing plan and printing.

[0012] Further, the historical printing state data of each nozzle in the nozzle historical database includes: X-direction minimum drop point deviation, X-direction drop point deviation average, X-direction maximum drop point deviation, X-direction drop point deviation standard deviation, Y-direction minimum drop point deviation, Y-direction drop point deviation average, Y-direction maximum drop point deviation, Y-direction drop point deviation standard deviation, minimum ejection volume deviation, maximum ejection volume deviation, average ejection volume, ejection volume deviation standard deviation, and the number of suspected nozzles.

[0013] Further, the pixel pit defect feature extraction model is trained by using a VIT network structure.

[0014] Further, the defect tracing model is trained by using a Graphormer network structure.

[0015] Further, the nozzle ejection parameter regulation model is obtained by training a neural network using a DQN reinforcement learning algorithm.

[0016] Further, each bipartite graph structure is constructed in the following manner:

[0017] According to the current set of ejection defect images, determine the number of each abnormal pixel pit, according to the known correspondence between the pixel pit and the nozzle determined by the ejection planning, determine all nozzles corresponding to the number of the abnormal pixel pit as suspect nozzles; all normal pixel pits printed by each suspect nozzle are regarded as related pixel pits; all nozzles that participate in the printing of related pixel pits and do not participate in the printing of abnormal pixel pits are regarded as related nozzles;

[0018] Each abnormal pixel pit and all its corresponding related pixel pits, all suspect nozzles, all related nozzles, and all other abnormal pixel pits and all their corresponding related pixel pits, all suspect nozzles, all related nozzles, and all suspect nozzles are regarded as a node in a bipartite graph structure, and each pixel pit node is connected to each nozzle node for ejection to obtain a bipartite graph structure; wherein each connection has a connection weight, and the weight is the number of drops of the nozzle on the connection for ejection of the pixel pit on the connection, and there is no node connection between two bipartite graph structures.

[0019] Further, the determination of the abnormal nozzle corresponding to each abnormal pixel pit is as follows:

[0020] Based on the current set of ejection defect images and the prior relationship between image gray scale and number of droplets, the number of droplets in each abnormal pixel pit is determined from the image gray scale of the abnormal pixel pit, which is combined with the number of droplets required for normal ejection to determine the number of missing droplets in the abnormal pixel pit; the connection weights associated with the abnormal pixel pit in the bipartite graph are sorted in descending order, and the number of missing droplets is sequentially subtracted from the connection weights until it is reduced to 0 or below in one cycle, and the total number of cycles is taken as the number n of abnormal nozzles corresponding to the abnormal pixel pit; according to the classification results of all suspect nozzles of the abnormal pixel pit, remove the nozzles with normal classification results, and sort the remaining suspect nozzles of the abnormal pixel pit according to the probability of the abnormal type determined by the screening; if n is equal to the number of remaining suspect nozzles, then the remaining suspect nozzles are regarded as the abnormal nozzles corresponding to the abnormal pixel pit, and if n is less than the number of remaining suspect nozzles, then n abnormal nozzles are determined from the remaining suspect nozzles by random permutation and combination as the abnormal nozzles corresponding to the abnormal pixel pit.

[0021] Further, the method for screening and determining the abnormal type to be regulated of each nozzle in the bipartite graph is as follows:

[0022] Based on the ranking of each injection hole in each abnormal type probability, it is judged whether the second high abnormal type probability of the injection hole is greater than one half of the first high abnormal type probability, if yes, the abnormal type corresponding to the first two high abnormal type probabilities of the injection hole is taken as the abnormal type to be regulated of the injection hole, otherwise, the abnormal type corresponding to the highest abnormal type probability of the injection hole is taken as the abnormal type to be regulated of the injection hole.

[0023] Further, the determination manner of the new printing planning parameter is specifically:

[0024] The parameter regulation values of the abnormal type to be regulated of all abnormal injection holes corresponding to each abnormal pixel pit, the parameter regulation values of the abnormal type to be regulated of all normal injection holes participating in the construction of the bipartite graph structure and the parameter regulation values of all normal injection holes not participating in the construction of the bipartite graph structure are combined to generate a plurality of parameter combination schemes, and the parameter combination scheme corresponding to the highest total abnormal type probability is selected as the new printing planning parameter according to the high and low of the total abnormal type probability corresponding to each parameter combination scheme.

[0025] According to another aspect of the present application, a computer readable storage medium is provided, which comprises a stored computer program, wherein when the computer program is run by a processor, the device where the storage medium is located is controlled to execute the steps of the method as described above.

[0026] Overall, compared with the prior art, the technical scheme provided by the present application mainly has the following beneficial effects:

[0027] 1.The present application proposes a new type of display jet printing manufacturing online intelligent jet printing regulation method, which trains pixel pit defect feature extraction model, defect tracing model and jet hole jet printing parameter regulation model based on offline training data to regulate online jet printing manufacturing parameters. The three models can update parameters in real time through online data, and can ensure real-time regulation accuracy. Among them, the pixel pit defect feature extraction model adopts image convolution, and the defect tracing model adopts graph neural network. The combination of the two further enhances the reasoning ability of defect tracing and improves the accuracy of defect tracing. In addition, the present application introduces a two-part graph as the input of the defect tracing model. By describing the corresponding relationship between the jet hole and the pixel pit as a two-part graph structure, the topological structure of the connection between the jet hole and the pixel pit is greatly preserved during training, so that the features between nodes are transmitted through edge connection, and the expression ability of the graph neural network is much higher than that of the ordinary neural network. Further, combining the abnormal type parameter regulation value of all abnormal jet holes corresponding to each abnormal pixel pit, the abnormal type parameter regulation value of all normal jet holes participating in the two-part graph structure construction, and the parameter regulation value of all normal jet holes not participating in the two-part graph structure construction, a variety of parameter combination schemes are generated for jet printing planning, further enhancing the robustness of online jet printing. Therefore, the present application realizes online high-robustness jet printing pattern planning parameter regulation.

[0028] 2.The present application further proposes a two-part graph construction method. By describing the corresponding relationship between the jet hole and the pixel pit as a two-part graph structure, there is no node connection between two two-part graph structures, so that the node relationship is more clear, thereby obtaining more accurate classification results of each jet hole node corresponding to the jet hole when using the defect tracing model for defect tracing. In addition, the two-part graph structure is provided with a link weight on each link, and the weight value is the drop number of the jet hole on the link printing the pixel pit on the link, which is more consistent with the flexible change of the printing drop number of one jet hole to the target pixel pit in actual printing, thereby ensuring the generalization of the application of the method.

[0029] 3.The present application further proposes an abnormal jet hole determination method. The abnormal drop number is determined according to the gray scale of the pixel after jet printing, and the number of abnormal jet holes is determined as prior knowledge of the jet printing regulation scheme in combination with the connection structure of the two-part graph, thereby accurately excluding inaccurate regulation schemes and improving the accuracy of the regulation process.

[0030] 4.The present application further proposes a determination method of the abnormal type to be regulated. The selection of the first or two types of abnormal types reasonably omits excessive and meaningless jet printing parameter combinations, saves the calculation resources of the jet printing process, and reduces the interference. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flow chart of a new type of display jet printing manufacturing online intelligent jet printing regulation method provided by the embodiment of the present application;

[0032] Figure 2 is a schematic diagram of a nozzle history printing state database structure provided by an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of an intelligent hybrid inkjet printing planning algorithm provided by an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of an inkjet printing defect data set provided by an embodiment of the present application;

[0035] Figure 5 is a schematic diagram of a pixel pit defect feature extraction model provided by an embodiment of the present application;

[0036] Figure 6 is a schematic diagram of a defect tracing model provided by an embodiment of the present application;

[0037] Figure 7 is a schematic diagram of a nozzle inkjet printing parameter regulation model provided by an embodiment of the present application;

[0038] Figure 8 is a schematic diagram of an inkjet printing planning parameter sorting algorithm provided by an embodiment of the present application;

[0039] Figure 9 is a schematic diagram of an inkjet printing regulation framework provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0041] Embodiment one

[0042] A novel display inkjet printing online intelligent inkjet printing regulation method, as shown in Figure 1 , includes:

[0043] Collecting pictures of all inkjet printing defects after one inkjet printing planning printing, constituting an inkjet printing defect picture set, determining all abnormal pixel pits based on the current inkjet printing defect picture set and the known corresponding relationship between pixel pits and nozzles determined by the inkjet printing planning, and building a two-part graph structure associated with all abnormal pixel pits and connected by nozzle nodes and pixel pit nodes; using a trained pixel pit defect feature extraction model, determining the inkjet printing features of the corresponding picture area of each pixel pit, assigning the corresponding pixel pit nodes in the two-part graph structure, and assigning the collected nozzle history printing state data to the corresponding nozzle nodes in the two-part graph structure to obtain all two-part graphs corresponding to the current inkjet printing.

[0044] inputting each bipartite graph into the trained defect tracing model to obtain a classification result of each nozzle node corresponding to the nozzle, the classification result being composed of probabilities of a plurality of preset abnormal types; inputting the historical printing state data of each nozzle node corresponding to the nozzle and the classification result into the trained nozzle printing parameter regulation model to obtain a parameter regulation value corresponding to each abnormal type of the nozzle;

[0045] sorting the probabilities of each abnormal type in the classification result of each nozzle node corresponding to the nozzle in each bipartite graph to screen abnormal types to be regulated of the nozzle; determining an abnormal nozzle corresponding to each abnormal pixel pit according to the abnormal types to be regulated of all suspect nozzles corresponding to the abnormal pixel pit;

[0046] inputting the historical printing state data of each normal nozzle not participating in the construction of the bipartite graph and normal classification data into the nozzle printing parameter regulation model to obtain a parameter regulation value corresponding to the normal nozzle;

[0047] generating a parameter combination scheme as a new printing planning parameter based on the abnormal types to be regulated parameter regulation values of all abnormal nozzles corresponding to each abnormal pixel pit, the abnormal types to be regulated parameter regulation values of all normal nozzles participating in the construction of the bipartite graph, and the parameter regulation values of all normal nozzles not participating in the construction of the bipartite graph, and using the parameter combination scheme to perform next printing planning and printing.

[0048] In the online execution process, there is a nozzle historical printing state database, which statistically has short-term printing data of all nozzles, including the volume, angle, speed of nozzle ejection, and printing performance, which has two functions: a, as a parameter of printing planning, such as different volumes and landing point errors of different nozzles, accurate measurement can improve the printing accuracy; b, as a feature of the nozzle, used for positioning abnormalities and prediction.

[0049] As preferred, as shown in Figure 2 The nozzle historical printing state database stores various printing indicators and states of all nozzles. The database mainly statistically has X-direction landing point error, Y-direction landing point error, ejection volume and suspect number for each nozzle, wherein the X-direction landing point error, Y-direction landing point error and ejection volume all contain minimum value, mean value, maximum value and standard deviation as specific parameters, and the suspect number of the suspect nozzle means the number of times of participating in printing of the defect pixel pit after the printing defect is generated, which is an accumulated value in the entire online printing process. Based on the ejection state performance of the nozzle, a state vector of each nozzle can be constructed:

[0050] F pn = <x min , x mean , x max , x std , ymin v std f sus

[0051] That is, the historical printing state data of each nozzle in the nozzle history database includes: the minimum X-direction landing point deviation, the average X-direction landing point deviation, the maximum X-direction landing point deviation, the X-direction landing point deviation standard deviation, the minimum Y-direction landing point deviation, the average Y-direction landing point deviation, the maximum Y-direction landing point deviation, the Y-direction landing point deviation standard deviation, the minimum ejection volume deviation, the maximum ejection volume deviation, the average ejection volume deviation, the ejection volume deviation standard deviation, and the number of times as a suspected nozzle.

[0052] In more detail, the data collection method in the nozzle historical printing state database is droplet observation, landing point detection, and defect tracing, wherein the droplet observation can observe the ejection volume, exit angle, and speed of each nozzle in a static state; the landing point detection directly obtains the X and Y errors of the printing landing point in the printing state; after the printing is completed, the abnormal pixel pit number is traced according to the position information of the defect, and then the associated suspected nozzle is obtained, and the number of times is counted.

[0053] The printing planning problem is also involved in the online execution process to establish an integer programming model. In an implementation, the pixel pit volume is used as a constraint, the minimum printing number is used as an optimization target, the volume combination of different nozzles is used to reduce the pixel pit volume error.

[0054] In a specific implementation, a plurality of printing stop points are set in the printing path, the position relationship between each stop point nozzle and pixel pit is generated, it is judged whether it can be ejected, an integer programming model is constructed, the volume uniformity between all pixels is used as a constraint, the minimum printing number is used as an optimization target, and the number of times of the nozzle printing timing is used as a decision variable to construct a programming model:

[0055]

[0056] In addition, the result of the printing planning includes the selection of the nozzle for the stop point and the printing number of each nozzle at each stop point, and then it is derived which nozzles eject each pixel pit, that is, the aforementioned known corresponding relationship between the pixel pit and the nozzle determined by the printing planning, which can be used for the construction of the bipartite graph structure.

[0057] More specifically, the intelligent hybrid printing planning algorithm, as shown in Figure 3 The modeling method is as follows:

[0058] S1, set the number of nozzles as m and the number of pixels as n, set p equidistant printing stop points in the printing path, at each stop point position, judge the position relationship between all nozzles and all pixel pits, use the coefficient matrix A n×(m×p) ​a represents whether the nozzle k at the jth docking point can spray the pixel i, and the coefficient is represented as a i,k×j a represents whether the nozzle k at the jth docking point can spray the pixel i, and the coefficient is represented as a i,k×j is set as the volume value of the nozzle k, otherwise 0;

[0059] S2, in order to constrain the volume value of each pixel pit, set the spraying times variable X of all nozzles at each docking point, and the spraying times variable of the nozzle k at the jth docking point is represented as x k×j,1 , the range size is only integer, wherein V p is the standard volume of the pixel pit, V dn is the volume median of all nozzles, and the volume constraint of each pixel pit is established within the upper and lower limits of the volume [v lb ,v ub ];

[0060] S3, under the premise of meeting the volume, an optimization goal of minimizing the printing times is needed to reduce the time required for printing, and the nozzle with the maximum spraying times at each docking point is the printing times of the docking point, which is set as Y, and the printing times of the docking point j is represented as y j Therefore, the optimization goal is to minimize the printing times of all docking points, and the printing planning model is established:

[0061]

[0062] As a supplement, the result of the printing planning includes the selection of the nozzle for the docking point and the printing times of each nozzle at each docking point, and then it is concluded which nozzles spray each pixel pit, which is a necessary network relationship for subsequent tracing.

[0063] In the online execution process, as a specific implementation, a printing defect data set is constructed, which collects unqualified printing defects on the pixel pit substrate after printing, such as scattered points and misprints, and at the same time, the corresponding relationship between the nozzle and the pixel pit is constructed, the network relationship between the defect (abnormal) pixel pit and the suspected nozzle, i.e. bipartite graph, is constructed, which is used for subsequent defect tracing.

[0064] Specifically, the jet printing defect data set is a comprehensive data set that combines defect pictures, a two-part graph structure of labeled jet orifice node and pixel node connection, and jet orifice regulation results obtained by the jet orifice jet printing parameter regulation model. In order to enrich the jet printing defect data set and realize supervised training, known abnormal jet orifices are used to participate in jet printing planning, a large number of jet printing defects are artificially manufactured, defect collection is performed through AOI, defect feature extraction is realized through a pixel pit defect feature extraction model, and relevant position information is provided; the two-part graph data includes: 1) a two-part graph structure constructed by the connection relationship between the pixel pits and the jet orifices output by the jet printing planning result and the abnormal pixel pit number, including the jet orifice node and the pixel pit node and the connection edge therebetween; 2) node features, the features of the jet orifice node are composed of jet history jet printing data, the features of the pixel pit node are obtained by processing the pixel pit defect image by the trained pixel pit defect feature extraction model, and are training data of the intelligent defect tracing model; the jet orifice regulation result is the parameter to be regulated or the flag whether to be enabled for each abnormal jet orifice, and is the training data of the jet orifice jet printing parameter regulation model.

[0065] More specifically, the jet printing defect data set, as shown in Figure 4 , specifically includes:

[0066] S1, in order to enrich the jet printing defect data set and realize supervised training, known abnormal jet orifices are used to participate in jet printing planning, a large number of jet printing defects are artificially manufactured, defect collection is performed through AOI, and defect feature extraction is realized through the jet printing defect visual inspection network, and relevant position information is provided;

[0067] S2, the two-part graph structure of the jet orifice node and the pixel node connection is combined with the jet printing planning result to generate, which is the training data of the intelligent defect tracing network;

[0068] S3, the jet orifice regulation result is the parameter to be regulated or the flag whether to be enabled for each abnormal jet orifice, and is the training data of the jet orifice jet printing parameter regulation model.

[0069] As a preferred embodiment, the construction method of each two-part graph structure is:

[0070] According to the current jet printing defect picture set, the number of each abnormal pixel pit is determined, according to the known corresponding relationship between the pixel pit and the jet orifice determined by the jet printing planning, all jet orifices corresponding to the number of the abnormal pixel pit are regarded as suspicious jet orifices; all normal pixel pits printed by each suspicious jet orifice are regarded as related pixel pits; the jet orifices participating in the printing of the related pixel pits and not participating in the printing of the abnormal pixel pits are regarded as related jet orifices;

[0071] Each abnormal pixel pit and all its corresponding related pixel pits, all its suspected ejection orifices, all its related ejection orifices, and all other abnormal pixel pits and all its corresponding related pixel pits, all its suspected ejection orifices, all its related ejection orifices participated by the suspected ejection orifices are taken as a node in a bipartite graph structure, and each pixel pit node is connected to each ejection orifice node for printing, to obtain a bipartite graph structure; wherein a connection weight is set on each connection, and the weight value is the drop number of the ejection orifice on the connection for printing the pixel pit on the connection, and there is no node connection between two bipartite graph structures.

[0072] The pixel pit defect feature extraction model is based on a defect image, and uses an intelligent image processing algorithm to realize defect feature extraction of a substrate after printing, complete defect identification, and feedback the position of the defect.

[0073] As a preferred implementation, as shown in Figure 5 The pixel pit defect feature extraction model is a VIT image feature extraction network. VIT uses a self-attention mechanism to capture global dependencies in images. In this method, the collected printed defect pictures are divided into blocks for semantic extraction to avoid storage pressure on the training process caused by large pictures. After cutting, each picture is given a position encoding. After regularization, the information is extracted using a multi-head attention module. The output result adds a residual connection, and then is regularized and connected to a fully connected layer to add a residual connection for output. The above process is an encoder. After connecting multiple encoders in a loop, the prediction result is output.

[0074] As a supplement, the pictures include printed scattered dot defects, color mixing defects, and missing printing defects. The extracted feature codes are used for defect classification, and at the same time, as the features of the pixel nodes of the bipartite graph in the printed defect data set. The results of defect classification are merged together as the features of the pixel nodes.

[0075] During online execution, a defect tracing model will also be involved. As a preferred implementation, as shown in Figure 6 It is a multi-head graph attention network. According to the bipartite graph structure of the labeled ejection orifice nodes and pixel nodes of the printed defect data set as the training data of the defect tracing model, the bipartite graph specifically includes four types of label nodes, namely abnormal pixel pit nodes, suspected ejection orifice nodes, related pixel pit nodes, and related ejection orifice nodes. The essence is to classify the ejection orifice nodes to realize the tracing of abnormal ejection orifice nodes. The network fully utilizes the information of the graph. The initial features add the centrality features of the graph. After self-attention, the spatial structure features and edge connection features of the bipartite graph are added. In order to fully extract the features during network training, an asynchronous information extraction mode is used:

[0076]

[0077] Among them, an encoding representing the l+1th layer pixel pit node, and an encoding representing the lth and l+1th layer jet node, a n,i,j and a p,i,j are attention coefficients of the jet node and the pixel possible node respectively, W l is the weight of the lth layer.

[0078] The jet printing parameter regulation model is also involved in the online execution process, which is used to regulate the jet printing planning parameters according to the classification results output by the intelligent defect tracing model and the historical printing state data of the jet, and as preferred, is divided into a landing point regulation network and a shielding network to output regulation values of different defect types of the jet, and output corresponding probability distributions.

[0079] Specifically, as shown in Figure 7 , the jet printing parameter regulation model includes two three-layer fully connected neural networks, which are respectively used to predict the X and Y landing point regulation values of each jet and whether the jet is shielded, and the neural networks are trained through the DQN reinforcement learning algorithm, and the input of the network is the feature data of the jet historical printing state database and the classification results of the intelligent defect tracing model, one of the two networks is selected for prediction, and the regulation results are the regulation deviation probability distribution of X and Y or X and Y and the binary classification probability of whether to shield.

[0080] The jet printing planning parameter sorting algorithm is also involved in the online execution process, which sorts the jet printing participation state of all jets according to the probability distribution of the classification results predicted by the defect tracing model and the parameter regulation, and creates multiple groups of different planning parameters to plan the pattern online.

[0081] The jet printing planning parameter sorting algorithm, as shown in Figure 8 , after the jet printing parameter regulation model predicts the regulation parameters of all jets, the regulation nodes are selected according to the following rules: A, preferentially processing abnormal jets, selecting according to the probability of the selected abnormal types to be regulated; B, predicting the jets that may appear printing defects according to the historical characteristics of the jets, and obtaining the regulation parameters. According to the probability value, sort the selected abnormal types to be regulated of each jet from large to small, take the abnormal type probability of each jet to be regulated as the judgment basis, based on the sorting of each abnormal type probability of each jet, judge whether the second high abnormal type probability of the jet is greater than half of the first high abnormal type probability, if yes, take the abnormal types corresponding to the top two high abnormal type probabilities of the jet as the abnormal types to be regulated of the jet, otherwise, take the abnormal type corresponding to the highest abnormal type probability of the jet as the abnormal type to be regulated of the jet, form multiple sets of jet printing planning parameters, and pre-plan all planning parameters according to the intelligent hybrid jet printing planning algorithm as a backup scheme.

[0082] As a preferred embodiment, the determination of the abnormal nozzle corresponding to each abnormal pixel pit is as follows:

[0083] Based on the current printing defect image set and the prior relationship between image gray scale and the number of droplets, the number of droplets in each abnormal pixel pit image gray scale is determined, which is combined with the number of droplets required for normal ejection to determine the number of missing droplets in the abnormal pixel pit; the abnormal pixel pit is sorted in descending order according to the weight of the associated line in the bipartite graph, and the number of missing droplets is sequentially subtracted from the line weight until it is reduced to 0 or below in one cycle, and the total number of cycles is taken as the number of abnormal nozzles n corresponding to the abnormal pixel pit; according to the classification results of all suspected nozzles of the abnormal pixel pit, remove the nozzles with normal classification results, and sort the remaining suspected nozzles of the abnormal pixel pit according to the probability of the abnormal type determined by the screening; if n is equal to the number of remaining suspected nozzles, the remaining suspected nozzles are taken as the abnormal nozzles corresponding to the abnormal pixel pit, and if n is less than the number of remaining suspected nozzles, n is determined from the remaining suspected nozzles by random permutation and combination as the abnormal nozzles corresponding to the abnormal pixel pit.

[0084] As a preferred embodiment, the determination of the abnormal nozzle corresponding to each abnormal pixel pit is as follows:

[0085] Based on the sorting of the probability of each abnormal type of each nozzle, it is judged whether the second highest abnormal type probability of the nozzle is greater than one half of the first highest abnormal type probability, if so, the abnormal types corresponding to the top two highest abnormal type probabilities of the nozzle are taken as the abnormal type to be controlled of the nozzle, otherwise, the abnormal type corresponding to the highest abnormal type probability of the nozzle is taken as the abnormal type to be controlled of the nozzle.

[0086] As a preferred embodiment, the determination of the new printing planning parameter is as follows:

[0087] The abnormal type parameter control values of all abnormal nozzles corresponding to each abnormal pixel pit, the abnormal type parameter control values of all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values of all normal nozzles not participating in the construction of the bipartite graph structure are combined to generate multiple parameter combination schemes, and the parameter combination scheme corresponding to the highest total abnormal type probability is selected as the new printing planning parameter according to the high and low of the total abnormal type probability corresponding to each parameter combination scheme.

[0088] For the convenience of understanding the method of the embodiment, the overall framework can be seen from Figure 9 .

[0089] The online intelligent printing closed-loop operation method has the following steps:

[0090] Step one: Before formal production, first carry out ink drop observation, drop point precision detection, test printing and other processes, complete the data collection of the current nozzle, and build an offline nozzle history database and a printing defect data set.

[0091] Step two: offline training of printing defect visual detection network, intelligent defect tracing network and printing planning parameter regulation network to improve the accuracy of network defect identification, tracing and regulation.

[0092] Step three: According to the printing planning parameter sorting algorithm, the parameters obtained from the last printing defect result are sorted, and the top ten printing planning parameters are selected for printing planning process.

[0093] Step four: According to the sorting selection planning result, test printing is executed, and after no defect, online printing is started.

[0094] Step five: During the online printing execution time, the nozzles that may produce defects are simulated constantly, new printing planning parameters are generated, and the printing planning process is recycled.

[0095] Step six: After the defect occurs, the defect is collected and respectively passed through the printing defect visual detection network and the intelligent defect tracing network, the nozzle history database and the printing defect data set are updated, new parameters are generated for planning, and the most similar shielding nozzle data is selected based on the existing planning to execute the printing process.

[0096] The method can be used in long-time printing display manufacturing, and the planning strategy is adjusted through closed-loop feedback according to the detected defects, which effectively predicts and suppresses the generation of defects, improves the manufacturing yield, and reduces the labor and time cost of regulation.

[0097] Embodiment two

[0098] The application also relates to a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the above method.

[0099] Specifically, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0100] The related technical solutions are the same as above, and will not be repeated here.

[0101] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for online intelligent inkjet printing control in display inkjet printing manufacturing, characterized in that, include: Based on the current set of printing defect images and the known correspondence between pixel pits and nozzles determined by the printing plan, all abnormal pixel pits are identified and a bipartite graph structure is constructed that associates all abnormal pixel pits and connects nozzle nodes with pixel pit nodes. Using a trained pixel pit defect feature extraction model, the printing features of the image region corresponding to each pixel pit are determined and assigned to the corresponding pixel pit node in the bipartite graph structure. The collected historical printing status data of nozzles is assigned to the corresponding nozzle node in the bipartite graph structure to obtain all bipartite graphs corresponding to the current printing. Each bipartite graph is input into the trained defect tracing model to obtain the classification result of the nozzle corresponding to each nozzle node. The classification result consists of the probabilities of multiple preset anomaly types. The historical printing status data and classification results of the nozzle corresponding to each nozzle node are input into the trained nozzle printing parameter control model to obtain the parameter control values ​​corresponding to each anomaly type of the nozzle. The probability of each abnormal type in the classification results of each nozzle node in each bipartite diagram is sorted to filter the abnormal type of the nozzle to be controlled; the abnormal nozzle corresponding to the abnormal pixel pit is determined according to the probability of the abnormal type of all suspected nozzles to be controlled for each abnormal pixel pit. Input the historical printing status data of each normal nozzle that did not participate in the construction of the bipartite graph structure and the normal classification data into the nozzle printing parameter control model to obtain the parameter control value of the corresponding normal nozzle. Based on the parameter adjustment values ​​of the abnormal type parameters to be adjusted for all abnormal nozzles corresponding to each abnormal pixel pit, the parameter adjustment values ​​of the abnormal type parameters to be adjusted for all normal nozzles participating in the construction of the bipartite graph structure, and the parameter adjustment values ​​of all normal nozzles not participating in the construction of the bipartite graph structure, a parameter combination scheme is generated as a new printing planning parameter for the next printing planning and printing.

2. The online intelligent inkjet printing control method for display inkjet printing manufacturing as described in claim 1, characterized in that, The historical printing status data for each nozzle in the nozzle history database includes: minimum X-axis landing point deviation, average X-axis landing point deviation, maximum X-axis landing point deviation, standard deviation of X-axis landing point deviation, minimum Y-axis landing point deviation, average Y-axis landing point deviation, maximum Y-axis landing point deviation, standard deviation of Y-axis landing point deviation, minimum spray volume deviation, maximum spray volume deviation, average spray volume, standard deviation of spray volume deviation, and the number of times it was identified as a suspected nozzle.

3. The online intelligent inkjet printing control method for display inkjet printing manufacturing as described in claim 1, characterized in that, The pixel pit defect feature extraction model is obtained by training a VIT network structure.

4. The online intelligent inkjet printing control method for display inkjet printing manufacturing as described in claim 1, characterized in that, The defect tracing model is obtained by training a Graphormer network structure.

5. The online intelligent inkjet printing control method for display inkjet printing manufacturing as described in claim 1, characterized in that, The nozzle printing parameter control model is obtained by training a neural network using the DQN reinforcement learning algorithm.

6. The online intelligent inkjet printing control method for display inkjet printing manufacturing as described in claim 1, characterized in that, Each bipartite graph structure is constructed as follows: Based on the current set of printing defect images, determine the number of each abnormal pixel pit. Based on the known correspondence between pixel pits and nozzles determined by the printing plan, determine all nozzles corresponding to the number of the abnormal pixel pit, and classify them as suspected nozzles. All normal pixel pits printed by each suspected nozzle are classified as related pixel pits. All nozzles that participated in the printing of related pixel pits but did not participate in the printing of abnormal pixel pits are classified as related nozzles. Each abnormal pixel pit and all its corresponding related pixel pits, all suspected nozzles, all related nozzles, and all other abnormal pixel pits and their corresponding related pixel pits, all suspected nozzles, and all related nozzles involved by all suspected nozzles are treated as nodes in a bipartite graph structure. Each pixel pit node is connected to each nozzle node that it prints, resulting in a bipartite graph structure. Each connection has a connection weight, which is the number of drops that the nozzle on that connection prints onto the pixel pit on that connection. There are no node connections between any two bipartite graph structures.

7. The online intelligent inkjet printing control method for display inkjet printing manufacturing as described in claim 6, characterized in that, The abnormal nozzle corresponding to each abnormal pixel pit is determined as follows: Based on the current set of printing defect images and the prior relationship between image grayscale and droplet count, the number of droplets in each abnormal pixel pit is determined by the image grayscale. This number is then combined with the number of droplets required for normal spraying to determine the number of missing droplets in that abnormal pixel pit. The abnormal pixel pits are sorted from largest to smallest according to the weights of the associated connections in the bipartite graph. The number of missing droplets is then subtracted from the connection weights sequentially until it reaches 0 or less in one loop. The total number of loops is taken as the number of abnormal nozzles (n) corresponding to that abnormal pixel pit. The classification results of all suspected nozzles of the abnormal pixel pit are used to remove nozzles that are classified as normal. The remaining suspected nozzles are then sorted according to the probability of the abnormal type to be controlled determined by the screening. If n is equal to the number of remaining suspected nozzles, the remaining suspected nozzles are taken as the abnormal nozzles corresponding to the abnormal pixel pit. If n is less than the number of remaining suspected nozzles, n are randomly selected from the remaining suspected nozzles as the abnormal nozzles corresponding to the abnormal pixel pit.

8. The online intelligent inkjet printing control method for display inkjet printing manufacturing as described in claim 1, characterized in that, The method for screening and determining the type of anomaly to be controlled for each nozzle in the two-part diagram is as follows: Based on the ranking of the probabilities of each abnormal type of each nozzle, it is determined whether the second highest abnormal type probability of the nozzle is greater than half of the first highest abnormal type probability. If so, the abnormal type corresponding to the first two highest abnormal type probabilities of the nozzle is taken as the abnormal type to be controlled for the nozzle. Otherwise, the abnormal type corresponding to the highest abnormal type probability of the nozzle is taken as the abnormal type to be controlled for the nozzle.

9. The online intelligent inkjet printing control method for display inkjet printing manufacturing as described in claim 8, characterized in that, The specific method for determining the new printing planning parameters is as follows: The parameter control values ​​of the abnormal type parameters to be controlled for all abnormal nozzles corresponding to each abnormal pixel pit, the parameter control values ​​of the abnormal type parameters to be controlled for all normal nozzles participating in the construction of the bipartite graph structure, and the parameter control values ​​of all normal nozzles not participating in the construction of the bipartite graph structure are combined to generate multiple parameter combination schemes. According to the total abnormal type probability corresponding to each parameter combination scheme, the parameter combination scheme with the highest total abnormal type probability is selected as the new printing planning parameter.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein the computer program, when executed by a processor, controls the device on which the storage medium is located to perform the steps of the method as described in any one of claims 1 to 9.

Images