A method for constructing an electrofluid printing mode classification model

Abstract

The application belongs to the field of electrohydrodynamic printing, and particularly relates to a method for constructing an electrohydrodynamic printing mode classification model, which comprises: each training sample comprises an electrohydrodynamic Taylor cone and jet video containing f frames of images and corresponding electrohydrodynamic printing modes; a neural network framework: an image feature extraction module is used for preliminary extraction of image features; an adjacent frame motion feature calculation module is used for calculation of feature level motion information between adjacent frames; a time flow feature extraction unit is connected with the image feature extraction module and used for further extraction of motion features; a space flow feature extraction unit is connected with the adjacent frame motion feature calculation module and used for further extraction of image features; and a classification module is connected with the time flow feature extraction unit and the space flow feature extraction unit for classification. The application takes video data as input and combines a time-space double flow attention mechanism, effectively making up for the problem of insufficient recognition accuracy of an image-based classification model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of electrohydrodynamic printing, and more particularly relates to a method for constructing an electrohydrodynamic printing pattern classification model. BACKGROUND

[0002] Electrohydrodynamic printing (EHDP) is a very promising emerging technology in the field of additive manufacturing. Due to its good compatibility (wide range of applicable materials and high viscosity liquids), low cost, simple structure, high resolution and other advantages, it has been considered as one of the most promising micro / nano-scale 3D printing technologies. Compared with traditional inkjet printing technology, the printing resolution of EHDP is improved by about two orders of magnitude, and it can print patterns with a size one order of magnitude lower than the structure of the nozzle. Therefore, there is increasing interest in using EHDP technology to manufacture custom devices, including various sensors, thin film transistors, solar cells, and light emitting devices. Its successful application covers various fields, including flexible electronics, biochips and screen manufacturing.

[0003] With the development of artificial intelligence technology, more and more people use machine learning or deep learning technology to monitor the electrohydrodynamic printing pattern. Although the existing method effectively realizes the classification of the printing pattern, it highly depends on manual extraction of image features, and has high requirements for the quality of the image and the written feature extraction algorithm. And since the Taylor cone is dynamically changing during the printing process, observing the image alone can easily misjudge. Therefore, a new electrohydrodynamic printing pattern monitoring method is needed to identify the Taylor cone pattern and achieve more accurate pattern recognition. SUMMARY

[0004] In view of the defects and improvement needs of the prior art, the present application provides a method for constructing an electrohydrodynamic printing pattern classification model, which aims to make up for the problem of insufficient recognition accuracy of the image-based classification model.

[0005] To achieve the above-mentioned purpose, according to one aspect of the present application, a method for constructing an electrohydrodynamic printing pattern classification model is provided, comprising:

[0006] constructing a training sample set, each training sample including an electrohydrodynamic Taylor cone and jet segment video containing f frames of images and its corresponding electrohydrodynamic printing pattern;

[0007] The neural network framework comprises an image feature extraction module, an adjacent frame motion feature calculation module, a space-time flow feature extraction module, and a classification module. The image feature extraction module is configured to perform initial image feature extraction on each frame of image in each input video. The adjacent frame motion feature calculation module is configured to subtract the initial image features corresponding to adjacent frames of image in each input video to obtain inter-frame image feature difference information of the input video as feature-level motion information. The space-time flow feature extraction module comprises a time flow feature extraction unit and a space flow feature extraction unit. The time flow feature extraction unit is configured to output f time flow feature vectors based on f initial image features of each input video. The space flow feature extraction unit is configured to output f-1 space flow feature vectors based on f-1 inter-frame image feature difference information of each input video. The classification module is configured to output a classification result of the electrohydrodynamic printing mode according to a splicing vector of the f time flow vectors and the f-1 space flow vectors.

[0008] The neural network framework is trained by using a training sample set to obtain an electrohydrodynamic printing mode classification model.

[0009] Further, the classification module comprises a multi-layer encoder unit and a multi-layer perception output unit.

[0010] The input of the multi-layer encoder unit is 2f vectors spliced by the splicing vector and a preset class label used for recording sample categories, and the output is a class label, which is the result of the input class label after training. The multi-layer perception output unit is configured to output a classification result L based on the class label output by the multi-layer encoder unit. output , L output is a vector composed of classification probabilities.

[0011] Further, the electrohydrodynamic printing mode labels in the training sample set comprise D class, P class, CJ class, TJ-S class, TJ-UnS class and MJ class.

[0012] Further, the implementation manner of training the neural network framework by using the training sample set is as follows:

[0013] establishing a time feature extraction model and a space feature extraction model for pre-training the neural network framework, wherein the first n layers of the space feature extraction model are the same as the image feature extraction module, the (n+1)th to (n+m)th layers are the same as the space stream feature extraction unit structure, the output of the (n+m)th layer, a first flattening operation and a first multi-layer perception are sequentially connected, the first multi-layer perception outputs a vector with a length x, and the size of x depends on the number of categories of classification; the first n layers of the time feature extraction model are the same as the image feature extraction module, the (n+1)th to (n+m)th layers are the same as the time stream feature extraction unit structure, the output of the (n+m)th layer, a second flattening operation and a second multi-layer perception are sequentially connected, and the second multi-layer perception outputs a vector with a length x;

[0014] training the space feature extraction model on the single-frame image classification sample set until the model converges, migrating the pre-trained space feature extraction model parameters of the first n layers to the image feature extraction module in the neural network framework, and migrating the parameters of the (n+1)th to (n+m)th layers to the space stream feature extraction unit in the neural network framework;

[0015] migrating the pre-trained space feature extraction model parameters of the first n layers to the first n layers of the time feature extraction model, training the time feature extraction model on the inter-frame difference image classification sample set after fixing the model parameters of the first n layers until the model converges, and migrating the pre-trained time feature extraction model parameters of the (n+1)th to (n+m)th layers to the time stream feature extraction unit in the neural network framework;

[0016] training the current neural network framework after parameter migration using the training sample set until the entire model converges to obtain a current fluid printing mode classification model, and completing the training;

[0017] wherein each frame image and its corresponding fluid printing mode in the training sample set are taken as a single-frame image classification sample to construct a single-frame image classification sample set; and the inter-frame difference image of each adjacent two frame images and its corresponding fluid printing mode in the training sample set are taken as an inter-frame difference image classification sample to construct an inter-frame difference image classification sample set.

[0018] Further, the space stream feature extraction unit and the time stream feature extraction unit have the same structure.

[0019] Further, the image feature extraction module has the same structure as the first n layers of ResNet, and the value of n is preferably selected according to the training condition.

[0020] The time stream feature extraction unit is composed of m residual blocks and one flattening layer, the m residual blocks have the same structure as the (n+1)th to (m+n)th layers of ResNet, and the value of m is preferably selected according to the training condition.

[0021] The application also provides a method for monitoring electro-fluid printing, comprising:

[0022] S1, moving the nozzle to an observation area to start printing with target printing parameters;

[0023] S2, after the ejection is stable, collecting f continuous Taylor cones and jet segments images of the current ejection hole by using a high-speed observation system; inputting the f continuous Taylor cones and jet segments images into an electro-fluid printing mode classification model to obtain an electro-fluid printing mode classification result; wherein the electro-fluid printing mode classification model is obtained by the construction method of an electro-fluid printing mode classification model as described above; repeating the step for the next ejection hole until all the f continuous Taylor cones and jet segments images of the measured ejection holes are collected and the corresponding electro-fluid printing mode classification results are calculated;

[0024] S3, judging whether the current printing meets the printing requirements according to the electro-fluid printing mode type and the printing stability of each ejection hole to realize printing monitoring.

[0025] The application also provides an electro-fluid printing monitoring device for executing the method for monitoring electro-fluid printing as described above, comprising a printing module, an observation module and a processor module; wherein the printing module is used for executing a printing operation, the observation module is used for executing a time sequence image collection operation, and the processor is used for executing an electro-fluid printing mode classification operation.

[0026] The application also provides a computer readable storage medium, characterized in that the computer readable storage medium comprises a stored computer program, wherein when the computer program is run by a processor, the device where the storage medium is located executes the construction method of an electro-fluid printing mode classification model as described above.

[0027] Overall, the above technical solutions conceived by the application can achieve the following beneficial effects:

[0028] (1) The application provides a mode classification model construction method for electrofluid printing, and specifically proposes electrofluid printing mode classification according to video data, and designs a mode classification model structure, which comprises an image feature extraction module, an adjacent frame motion feature calculation module, a space-time flow feature extraction module and a classification module; wherein the image feature extraction module is used for preliminary extraction of image features; the adjacent frame motion feature calculation module is used for calculation of feature-level motion information between adjacent frames, the space-time flow feature extraction module comprises a time flow feature extraction unit and a space flow feature extraction unit, and is used for further extraction of image features and motion features, the time flow feature extraction unit is connected with the image feature extraction module and used for further extraction of motion features, the space flow feature extraction unit is connected with the adjacent frame motion feature calculation module and used for further extraction of image features, and the classification module is connected with the time flow feature extraction unit and the space flow feature extraction unit, respectively, and classification is performed based on the output splicing result of the time flow feature extraction unit and the space flow feature extraction unit; this space-time dual-flow network can effectively extract video information, thereby ensuring the accuracy of mode classification based on the space-time dual-flow attention mechanism; in addition, compared with the traditional optical flow feature, the feature-level interframe difference information can effectively extract motion features while greatly simplifying the calculation process, reducing the calculation consumption and improving the calculation efficiency. In general, the application proposes video data as input and combines the space-time dual-flow attention mechanism, starts from the time sequence image, considers the dynamic change process of the Taylor cone shape compared with the traditional classification method based on the Taylor cone and the single image of the jet, can start from higher-dimensional information, effectively improves the classification accuracy, and effectively makes up for the problem of insufficient recognition accuracy of the image-based classification model.

[0029] (2) The application also proposes that the classification module comprises a multi-layer encoder unit and a multi-layer perception output unit, and the multi-head attention mechanism module is used for fusing space-time information, fully utilizing the parallelism of the attention mechanism network, and simultaneously paying attention to each time point and each feature, so as to accelerate the model training and reasoning speed. The input of the multi-head attention mechanism module is feature-level space-time information, which avoids the problems of high calculation complexity and model parameter quantity, and effectively reduces the overfitting risk.

[0030] (3) The application also proposes that the electrofluid printing mode classification labels in the training sample set comprise D class, P class, CJ class, TJ-S class, TJ-UnS class and MJ class, that is, the classification of abnormal modes other than the CJ class is more fine, which guarantees the accurate classification of the model, effectively makes up for the insufficient recognition accuracy of the image-based classification model to a certain extent, and is helpful for realizing higher-precision detection of the whole inkjet printing process.

[0031] (4) The application also proposes a pre-training-migration learning method for training the model, which gradually trains from a simple network to a target network in steps, improves the training efficiency and reduces the training consumption, and at the same time, the method uses a self-built data set for pre-training, avoiding the problem of unsatisfactory training effect caused by the obvious deviation between the Taylor cone image and the data in the public data set.

[0032] (5) The application also proposes a method for monitoring electrofluidic printing based on the electrofluidic printing mode classification model constructed by the aforementioned model construction method, which is reliable in monitoring results and has a guiding effect on the adjustment of process parameters due to the fine mode classification, thereby ensuring efficient and high-quality electrofluidic printing. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A flow chart of a method for constructing an electrofluidic printing mode classification model according to an embodiment of the application is shown in

[0034] Figure 2 A structure diagram of an electrofluidic printing mode classification model according to an embodiment of the application is shown in

[0035] Figure 3 A structure diagram of an image feature extraction module and a space-time flow feature extraction unit according to an embodiment of the application is shown in

[0036] Figure 4 A schematic diagram of a method for training an electrofluidic printing mode classification model according to an embodiment of the application is shown in

[0037] Figure 5 A schematic diagram of a Taylor cone time sequence image classification category according to an embodiment of the application is shown in DETAILED DESCRIPTION

[0038] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0039] Embodiment one

[0040] A method for constructing an electrofluidic printing mode classification model, as shown in Figure 1 , comprising:

[0041] Constructing a training sample set, each training sample including an electrofluidic Taylor cone and a jet segment video containing f frames of images and its corresponding electrofluidic printing mode;

[0042] The neural network framework is built, including: an image feature extraction module, an adjacent frame motion feature calculation module, a space-time flow feature extraction module, and a classification module; wherein the image feature extraction module is used for initial image feature extraction of each frame image in each input video; the adjacent frame motion feature calculation module is used for subtraction of initial image features corresponding to each adjacent frame image in each input video to obtain inter-frame image feature difference information of the input video as feature-level motion information; the space-time flow feature extraction module includes a time flow feature extraction unit and a space flow feature extraction unit, which are used for further extraction of image features and motion features; the time flow feature extraction unit is used for outputting f time flow feature vectors (i.e. motion features) based on f initial image features of each input video; the space flow feature extraction unit is used for outputting f-1 space flow feature vectors (i.e. image features) based on f-1 inter-frame image feature difference information of each input video; and the classification module is used for outputting a classification result of the electrohydrodynamic printing mode according to a splicing vector of the f time flow feature vectors and the f-1 space flow feature vectors.

[0043] The neural network framework is trained by using the training sample set to obtain an electrohydrodynamic printing mode classification model.

[0044] For the above model, as shown in Figure 2 , the specific description is as follows:

[0045] The image feature extraction module preliminarily extracts image features, as shown in Figure 3 , the module is spliced by an initial convolution layer, a pooling layer, and a plurality of residual blocks; the residual block generally includes a convolution layer for feature extraction, a batch normalization layer for accelerating training and improving model generalization, an activation function, and a short connection connecting the input and the output. Preferably, the image feature extraction module can be the same as the first n layers of resnet, and n is preferably selected according to the training condition. The input of the image feature extraction module is a video, that is, f matrices with a size of w x h x c, wherein w is the width of each input image, h is the height of the input image, and c is the number of input image channels. The output of the image feature extraction module is f matrices with a size of w1 x h1 x c1, wherein w1 is the output feature width, h1 is the output feature height, and c1 is the output feature channel number, i.e. the channel number of the last layer of the image feature extraction module.

[0046] The adjacent frame motion feature calculation module calculates the feature-level motion information between adjacent frames. The module subtracts the outputs of the image feature extraction module from the adjacent two frame images to obtain inter-frame image feature difference information as feature-level motion information. The input of the adjacent frame motion feature calculation module is the output of the image feature extraction module, i.e. f matrices with a size of w1 x h1 x c1, and the output is f-1 matrices with a size of w1 x h1 x c1.

[0047] The spatio-temporal flow feature extraction module further extracts image features and motion features, including two units, a temporal flow feature extraction unit and a spatial flow feature extraction unit; preferably, as shown in Figure 3 The temporal flow feature extraction unit is composed of m layers of residual blocks and one layer of flattening layers, and further preferably, the m layers of residual blocks can have the same structure as the n+1 to m+n layers of ResNet, and m is selected according to the training condition. Preferably, the structure of the spatial flow feature extraction unit is the same as that of the temporal flow feature extraction unit, but the parameters are different. The input of the temporal flow feature extraction unit is the output of the image feature extraction module, that is, f matrices with a size of w1xh1xc1, and the output is f vectors with a length of x, which is related to the output size of the m layers of residual blocks. The input of the spatial flow feature extraction unit is the output of the adjacent frame motion feature calculation module, that is, f-1 matrices with a size of w1xh1xc1, and the output is f-1 vectors with a length of x. Finally, the outputs of the temporal flow feature extraction unit and the spatial flow feature extraction unit are spliced into a (2f-1)x matrix, which is the output of the spatio-temporal flow feature extraction module.

[0048] As a preferred embodiment, the classification module includes a multi-layer encoder unit and a multi-layer perceptron output unit; wherein the input of the multi-layer encoder unit is 2f vectors spliced from the spliced vector and a class label. The multi-layer encoder unit is used to compress high-dimensional data into low-dimensional representation by introducing an attention mechanism, so as to better understand the internal structure and features of the data, and finally the multi-layer perceptron output unit outputs the classification result.

[0049] The input of the multi-layer encoder unit is a 2f x matrix spliced from the output of the spatio-temporal flow feature extraction module and a class label, wherein the class label is a trainable vector with a length of x, which is used to record the sample category in the attention network; the multi-layer encoder module only takes the class label vector as the input of the next module (multi-layer perceptron output unit), that is, outputs a vector with a length of x.

[0050] The multi-layer encoder unit is composed of an encoder stacked L times, wherein L is selected according to the actual situation, and is preferably 12; the encoder is composed of a layer normalization layer, a multi-head attention layer and a multi-layer perceptron block; the layer normalization layer performs layer normalization on each sample through the following formula:

[0051]

[0052] wherein x is the input vector of the layer normalization layer, μ is the mean of the vector x, σ is the standard deviation of the vector x, ∈ is a constant greater than zero with a small value, which is used to avoid calculation error caused by zero denominator, γ and β are stretching parameters and offset parameters respectively, which have the same shape as x and are parameters learned together with other model parameters, LN(x) is the output of the layer normalization layer.

[0053] The multi-head attention layer includes h attention aggregation outputs, each attention head h i The calculation method is:

[0054]

[0055] Among them, q i 、k i 、v i are the query, key, and value for the given i-th header, are the query, key, and value weights of the ith attention head, respectively, are learnable parameters, f is the attention aggregation function, and h i is the i-th attention head, i is a real number from 1 to h, h is the total number of attention heads, and d represents the vector k i length;

[0056] The output of the multi-head attention layer also needs to consider that different heads need to focus on different parts of the input, so a linear transformation is required before output to splice all heads:

[0057]

[0058] Among them, W O are learnable parameters, h1…h h They are the 1st to hth attention heads respectively;

[0059] The multi-layer perceptron block is composed of a fully connected layer, an activation function, and a random dropout layer.

[0060] The multi-layer perceptron output unit is used to output the classification results. It is composed of a multi-layer or single-layer perceptron. The unit input is the output of the multi-layer encoder unit, that is, a vector of length x, and the output is a vector of length L. output The vector, L output The size of depends on the number of categories being classified.

[0061] As a preferred embodiment, Figure 4 As shown in the figure, the implementation method of using the training sample set to train the neural network framework is as follows:

[0062] The time feature extraction model and the space feature extraction model for pre-training the neural network framework are established, wherein the first n layers of the space feature extraction model are the same as the image feature extraction module, the (n+1)th to (n+m)th layers are the same as the space flow feature extraction unit structure, the output of the (n+m)th layer, the first flattening operation and the first multilayer perceptron are sequentially connected, the first multilayer perceptron outputs a vector with a length of x, and the size of x depends on the number of categories of classification; the first n layers of the time feature extraction model are the same as the image feature extraction module, the (n+1)th to (n+m)th layers are the same as the time flow feature extraction unit structure, the output of the (n+m)th layer, the second flattening operation and the second multilayer perceptron are sequentially connected, and the second multilayer perceptron outputs a vector with a length of x;

[0063] The space feature extraction model is trained on the single-frame image classification sample set until the model converges, the pre-trained space feature extraction model is migrated to the image feature extraction module in the neural network framework, and the (n+1)th to (n+m)th layer parameters are migrated to the space flow feature extraction unit in the neural network framework.

[0064] The pre-trained space feature extraction model is migrated to the first n layers of the time feature extraction model, and the time feature extraction model is trained on the inter-frame difference image classification sample set after fixing the model parameters of the first n layers until the model converges, the pre-trained time feature extraction model is migrated to the time flow feature extraction unit in the neural network framework.

[0065] The current neural network framework after parameter migration is trained by using the training sample set until the whole model converges, and a current fluid printing mode classification model is obtained, and the training is completed.

[0066] Each frame image and the corresponding fluid printing mode classification in the training sample set are taken as a single-frame image classification sample, and a single-frame image classification sample set is constructed; the inter-frame difference image of each adjacent two frame images in the training sample set and the corresponding fluid printing mode classification are taken as an inter-frame difference image classification sample, and an inter-frame difference image classification sample set is constructed.

[0067] Specifically, the single-frame image classification sample set, the inter-frame difference image classification sample set and the time sequence image classification sample set (i.e. the aforementioned training sample set) can be constructed in the following directions:

[0068] A Taylor cone and a high-speed jet observation system are used to collect videos, and the categories of each video are recorded. During the data collection process, it is necessary to ensure that each type of data is balanced as much as possible.

[0069] The data set is arranged, the video is cropped into a short video of each f frame, and the data set size is expanded through center cropping, random occlusion and brightness adjustment. Finally, each video sample is a continuous image of f frames with a size of w x h, forming a time sequence image classification sample set. The image frames in the video are extracted to form a single-frame image classification sample set. The difference information between adjacent frames is calculated to form an inter-frame difference image classification sample set.

[0070] As a preferred embodiment, as shown in Figure 5 The current fluid printing mode labels in the training sample set include D, P, CJ, TJ-S, TJ-UnS and MJ.

[0071] That is, the category of each video sample is distinguished according to the following principles: no triangular Taylor cone is formed, and the droplets are dropped one by one, which is recorded as D (Dripping); a Taylor cone and a jet segment are initially formed, and the shape changes periodically, and the droplets are dropped at a fixed frequency, which is recorded as P (Pulsating); a stable and symmetrical Taylor cone and jet segment are formed, the shape of the Taylor cone is stable, the waist is concave, and the length of the jet segment hardly changes, which is recorded as CJ (Cone-Jet); a stable and asymmetrical Taylor cone is formed, the jet segment is inclined and the angle length hardly changes, which is recorded as TJ (Tiled-Jet); an unstable and asymmetrical Taylor cone is formed, the jet segment is inclined and the angle length fluctuates, which is recorded as TJ_Un (Unstable Tiled-Jet); an irregularly shaped Taylor cone and multi-segment jet are formed, which is recorded as MJ (Multi-Jet).

[0072] The embodiment sets six categories, which makes the model classification more fine and is beneficial to guide the efficient adjustment of subsequent process parameters.

[0073] The high-speed observation system described above can include a high-speed camera, a lens, and a light source. Preferably, the camera exposure time is 1-50us, the lens magnification is appropriate, the complete Taylor cone and jet segment can be captured, the light source is 180 degrees away from the camera axis, and the back light method is used to irradiate the image capturing target area. The f continuous Taylor cone and jet segment images are input into the pre-trained classification model to obtain the Taylor cone classification result.

[0074] Embodiment two

[0075] A current fluid printing monitoring method comprises the following steps:

[0076] S1, moving the nozzle to the observation area, and starting printing with the target printing parameters;

[0077] S2, after the jet is stable, a high-speed observation system is used to collect f continuous Taylor cone and jet segment images of the current nozzle; the f continuous Taylor cone and jet segment images are input into an electrofluid printing mode classification model to obtain an electrofluid printing mode classification result; wherein the electrofluid printing mode classification model is obtained by the construction method of an electrofluid printing mode classification model described in embodiment one; the next nozzle repeats the step until f continuous Taylor cone and jet segment images of all measured nozzles are collected and the corresponding electrofluid printing mode classification result is calculated;

[0078] S3, according to the electrofluid printing mode type and printing stability of each nozzle, it is judged whether the current printing meets the printing requirements, printing monitoring is realized, and process parameter adjustment is carried out.

[0079] The related technical solutions are the same as those of embodiment one, and will not be repeated here.

[0080] Embodiment three

[0081] An electrofluid printing monitoring device for executing an electrofluid printing monitoring method as described above, comprising a printing module, an observation module, and a processor module; wherein the printing module is used to execute a printing operation, the observation module is used to execute a time sequence image collection operation, and the processor is used to execute an electrofluid printing mode classification operation.

[0082] The related technical solutions are the same as those of embodiment two, and will not be repeated here.

[0083] Embodiment four

[0084] A computer readable storage medium comprising a stored computer program, wherein when the computer program is run by a processor, the device where the storage medium is located executes an electrofluid printing mode classification model construction method as described above.

[0085] The related technical solutions are the same as those of embodiment one, and will not be repeated here.

[0086] In general, in order to solve the problems of insufficient classification accuracy and categories in existing electrofluid printing mode monitoring, an electrofluid printing mode monitoring method based on video data is proposed. Starting from the time sequence image, compared with the traditional classification method based on single Taylor cone and jet image, the dynamic change process of Taylor cone shape is considered, which can start from higher dimensional information to effectively improve the classification accuracy.

[0087] Those skilled in the art will readily understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing an electrofluidic printing pattern classification model, characterized in that: include: Constructing a training sample set, each training sample includes an electrofluid Taylor cone containing f frames of images and a jet segment video and its corresponding electrofluid printing mode; A neural network framework is constructed, comprising: an image feature extraction module, an adjacent frame motion feature calculation module, a spatiotemporal flow feature extraction module, and a classification module; wherein the image feature extraction module is used to extract initial image features from each frame image in each input video; the adjacent frame motion feature calculation module is used to subtract the initial image features corresponding to each adjacent frame image in each input video to obtain image feature difference information between each frame of the input video; the spatiotemporal flow feature extraction module includes a temporal flow feature extraction unit and a spatial flow feature extraction unit, the temporal flow feature extraction unit is used to output f temporal flow feature vectors based on f initial image features of each input video; the spatial flow feature extraction unit is used to output f-1 spatial flow feature vectors based on f-1 inter-frame image feature difference information of each input video; the classification module is used to output a print mode classification result based on a concatenation vector of the f temporal flow feature vectors and the f-1 spatial flow feature vectors; The neural network framework is trained using the training sample set to obtain the electrofluidic printing pattern classification model; The classification module includes a multi-layer encoder unit and a multi-layer perceptron output unit; The input of the multi-layer encoder unit is a 2f vector composed of the spliced ​​vector and a preset category label for recording the sample category, and the output is a category label, which is the result of the input category label after training; the multi-layer perceptron output unit is used to output the classification result L based on the category label output by the multi-layer encoder unit. output , L output is a vector consisting of the probabilities of each classification; The electrofluid printing mode labels in the training sample set include D, P, CJ, TJ-S, TJ-UnS and MJ. Among them, the triangular Taylor cone is not formed, and the droplets drip one by one, which is recorded as D; the Taylor cone and jet segment are initially formed and its shape changes periodically, and the droplets drip at a fixed frequency, which is recorded as P; the formation of a stable, symmetrical Taylor cone and jet segment, the Taylor cone shape is stable and the waist is concave, and the length of the jet segment does not change, is recorded as CJ; the formation of a stable, asymmetric Taylor cone, the jet segment is tilted and the angle length does not change, is recorded as TJ-S; the formation of an unstable, asymmetric Taylor cone, the jet segment is tilted and the angle length fluctuates, is recorded as TJ-UnS; the formation of an irregular Taylor cone and multi-segment jets is recorded as MJ.

2. The construction method according to claim 1, characterized in that The implementation method of using the training sample set to train the neural network framework is as follows: Establishing a temporal feature extraction model and a spatial feature extraction model for pre-training the neural network framework, wherein the first n layers of the spatial feature extraction model are the same as the image feature extraction module, the n+1th to n+mth layers are the same as the spatial stream feature extraction unit structure, the output of the n+mth layer, the first flattening operation, and the first multilayer perceptron are connected in sequence, and the first multilayer perceptron outputs a vector of length x, where the size of x depends on the number of classification categories; the first n layers of the temporal feature extraction model are the same as the image feature extraction module, the n+1th to n+mth layers are the same as the temporal stream feature extraction unit structure, the output of the n+mth layer, the second flattening operation, and the second multilayer perceptron are connected in sequence, and the second multilayer perceptron outputs a vector of length x; Training the spatial feature extraction model on a single-frame image classification sample set until the model converges, migrating the first n layers of parameters of the pre-trained spatial feature extraction model to the image feature extraction module in the neural network framework, and migrating the n+1 to n+m layers of parameters to the spatial stream feature extraction unit in the neural network framework; Migrating the first n layers of the pre-trained spatial feature extraction model parameters to the first n layers of the temporal feature extraction model, and after fixing the first n layers of the model parameters, training the temporal feature extraction model on the inter-frame difference image classification sample set until the model converges, and migrating the n+1th to n+mth layer parameters of the pre-trained temporal feature extraction model to the temporal stream feature extraction unit in the neural network framework; The current neural network framework after the training parameter migration is trained using the training sample set until the entire model converges to obtain an electrofluidic printing pattern classification model, thereby completing the training; Among them, each frame image in the training sample set and its corresponding electrofluid printing mode are taken as a single-frame image classification sample to construct a single-frame image classification sample set; the inter-frame difference image of each two adjacent frames of the training sample set and its corresponding electrofluid printing mode are taken as an inter-frame difference image classification sample to construct an inter-frame difference image classification sample set.

3. The construction method according to claim 1 or 2, characterized in that The spatial stream feature extraction unit and the temporal stream feature extraction unit have the same structure.

4. The construction method according to claim 1 or 2, characterized in that The image feature extraction module has the same structure as the first n layers of ResNet, and the value of n is optimized according to the training situation; The temporal stream feature extraction unit consists of m layers of residual blocks and a flattening layer. The m layers of residual blocks have the same structure as the n+1 to m+n layers of ResNet, and the value of m is optimized according to the training situation.

5. A method for monitoring electrofluidic printing, characterized in that: include: S1. Move the nozzle to the observation area and start printing with the target printing parameters; S2. After the jetting is stabilized, using a high-speed observation system to collect f continuous Taylor cone and jet segment images of the current nozzle; inputting the f continuous Taylor cone and jet segment images into an electrofluidic printing mode classification model to obtain an electrofluidic printing mode classification result; wherein the electrofluidic printing mode classification model is constructed using the method for constructing an electrofluidic printing mode classification model according to any one of claims 1 to 4; repeating this step for the next nozzle until f continuous Taylor cone and jet segment images are collected for all tested nozzles and the corresponding electrofluidic printing mode classification results are calculated; S3. According to the electrofluidic printing mode type and printing stability of each nozzle, it is determined whether the current printing meets the printing requirements, thereby realizing printing monitoring.

6. An electrofluid printing monitoring device, characterized in that: Used to execute an electrofluid printing monitoring method as described in claim 5, comprising: a printing module, an observation module, and a processor module; wherein the printing module is used to perform a printing operation, the observation module is used to perform a time-series image acquisition operation, and the processor is used to perform an electrofluid printing mode classification operation.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute a method for constructing an electrofluid printing pattern classification model according to any one of claims 1 to 4.

Images