A real-time evaluation method and system for printing equipment based on simulation analysis

Abstract

The present invention discloses a real-time evaluation method and system for printing equipment based on simulation analysis. By collecting video data of the printing process, key frames are extracted to generate an image set, the printing area is identified, and solder paste printing points are counted, and N image areas are divided. The Sobel operator is used to extract contour features, and the color features are extracted through the color histogram to construct a multidimensional feature set. The GAN model is used to simulate features, and feature learning and dimension reduction are performed on the autoencoder according to the simulated features, and identification feature data is generated. The similarity of the identification feature data is analyzed based on the Euclidean distance, and the predicted defect area that deviates from the overall feature is screened out. By predicting the corresponding features of the defect area, the printing defects are identified and evaluated in combination with the comparison features, and the equipment printing defect evaluation information is generated. Through the present invention, it is possible to achieve rapid defect location and defect area prediction, optimize solder paste printing equipment parameters, and improve production efficiency.

Description

Technical Field

[0001] The present invention relates to the field of printing equipment data analysis, and more specifically, to a printing equipment real-time evaluation method and system based on simulation analysis. Background Art

[0002] Solder paste printing is one of the key steps in surface mount technology (SMT), and its printing quality directly affects the quality of subsequent welding and the reliability of the product. However, in the actual production process, due to factors such as equipment wear, improper parameter settings, and insufficient data analysis capabilities, solder paste printing equipment often has various printing defects, such as uneven solder paste thickness, blurred edges, and missing prints. Traditional solder paste printing equipment evaluation methods mainly rely on manual inspection and statistics, and have problems such as long evaluation cycles, low efficiency, and insufficient accuracy. In addition, traditional technologies lack effective prediction and real-time evaluation of defective areas, making it difficult to improve production efficiency. Summary of the invention

[0003] The present invention overcomes the defects of the prior art and proposes a real-time evaluation method and system for printing equipment based on simulation analysis.

[0004] A first aspect of the present invention provides a real-time evaluation method for printing equipment based on simulation analysis, comprising:

[0005] Collect video data of solder paste printing equipment during the printing process, extract key frames from the video data, and generate key image sets;

[0006] Identify the printing area of ​​the key image set, count the number of solder paste printing points based on the printing area, and divide the printing area into N image areas based on the statistical information;

[0007] Taking the image area as the analysis unit, the contour feature of the image area is extracted through the Sobel operator, and the color feature of the image area is extracted through the color histogram, and multi-dimensional contour features and color features are generated respectively;

[0008] Based on the GAN generation model, the contour features and color features are simulated and analyzed, and simulated data is generated to build an autoencoder, and the simulated data is imported into the autoencoder for model training;

[0009] Based on the autoencoder, feature learning and feature dimension reduction are performed on the contour features and color features, and recognition feature data of the image area is generated. Based on the Euclidean distance, data similarity analysis is performed on the recognition feature data of N image areas, and image areas that deviate from the overall printing area characteristics are screened out and marked as predicted defect areas;

[0010] All predicted defect areas are merged into the area to be tested, printing defects are identified based on the contour features and color features of the area to be tested, defects are evaluated based on the contrast features, and equipment printing defect evaluation information is generated.

[0011] In this solution, the video data of the solder paste printing device during the printing process is collected, key frames are extracted from the video data, and a key image set is generated, specifically:

[0012] Based on a printing production cycle, the video data of the solder paste printing equipment during the printing process is collected by a camera device;

[0013] Based on the video processing library, the video data is loaded and key frames are extracted to form a key image set;

[0014] Perform image smoothing and enhancement preprocessing on the image data in the key image set.

[0015] In this solution, the printing area is identified for the key image set, the number of solder paste printing points is counted based on the printing area, and the printing area is divided into N image areas according to the statistical information, specifically:

[0016] Load image data according to the key image set to obtain real-time image loading data;

[0017] Perform printing positioning and printing area recognition on real-time image loading data, and mark the printing area;

[0018] According to the printing area, the real-time image loading data is used to identify and count the printing points of the solder paste, and the statistical data of the printing points in the printing area are obtained;

[0019] The printing area is divided into regions by using the statistical data of printing points to form N image regions;

[0020] During the division process, it is ensured that each image area includes a preset number of solder paste printing dots.

[0021] In this solution, the image area is used as the analysis unit, the contour feature of the image area is extracted by the Sobel operator, the color feature of the image area is extracted by the color histogram, and multi-dimensional contour features and color features are generated respectively, specifically:

[0022] Taking the image area as the analysis unit, the pixel points in each image area are grayed out and the contour information is analyzed and extracted through the Sobel operator to generate contour features;

[0023] The color statistics of the image area are performed through the color histogram, and the statistical data are vectorized to generate color features;

[0024] The outline features and color features are both multi-dimensional feature data.

[0025] In this solution, the GAN-based generation model performs feature simulation analysis on the contour features and color features, generates simulated data, constructs an autoencoder, and imports the simulated data into the autoencoder for model training, specifically:

[0026] Build a GAN-based generative model, which includes a generator and a discriminator;

[0027] Integrate contour features and color features into high-dimensional image features, import the high-dimensional image features into the generator for feature learning and simulation generation, and obtain simulation features;

[0028] Import the simulated features into the discriminator to judge the authenticity of the data, and adjust the parameters of the generator and discriminator in real time based on the preset loss function;

[0029] Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium;

[0030] Generate real-time simulation features of a preset data volume through the trained generative model, segment the real-time simulation features into multiple data segments, perform feature similarity analysis on each data segment and the high-dimensional image features, calculate data similarity based on Euclidean distance, and select the expected data from multiple data segments through the preset similarity standard and mark them as simulation data;

[0031] Construct an autoencoder and initialize autoencoder parameters;

[0032] The simulated data is imported into the autoencoder as high-dimensional data for feature learning, and the simulated data is mapped to a low-dimensional representation through a multi-layer perceptron to obtain low-dimensional representation data;

[0033] Reconstruct the low-dimensional representation data to obtain high-dimensional output data;

[0034] The mean square error is used as the loss function to determine the deviation between the high-dimensional output data and the simulated data, and the iterative training is repeated until the loss function converges to a smaller preset value and the autoencoder training is completed.

[0035] In this solution, the self-encoder is used to perform feature learning and feature dimension reduction on the contour features and color features, and to generate recognition feature data of the image area. The recognition feature data of N image areas are analyzed for data similarity based on the Euclidean distance, and the image areas that deviate from the overall printing area features are screened out and marked as predicted defect areas, specifically:

[0036] Import contour features and color features into the autoencoder for feature learning and feature dimension reduction, and generate identification feature data for each image area;

[0037] Based on the dimension of the recognition feature data, a multi-dimensional space is constructed, the recognition feature data is mapped to the multi-dimensional space and multiple data points are generated. The distance value between each data point is calculated through the Euclidean distance, and the data similarity analysis of the recognition feature data is performed based on the distance value, and the spatial distribution analysis of all data points is performed;

[0038] In the multi-dimensional space, a data point is selected as the center point, the data range is defined based on the preset distance threshold, and the data points outside the data range are marked to obtain a set of deviated data points;

[0039] All data points are set as center points to perform deviation point analysis, and multiple groups of deviation data points are obtained;

[0040] According to multiple groups of deviation data points, the occurrence frequency of the deviation data points is counted, and the high-frequency data points are marked to obtain multiple defect data points;

[0041] Based on a plurality of defect data points, corresponding identification feature data and corresponding image regions are marked, and the corresponding image regions are used as predicted defect regions.

[0042] In this solution, the predicted defect area includes multiple ones.

[0043] In this solution, all predicted defect areas are merged into the area to be tested, printing defects are identified according to the contour features and color features of the area to be tested, defects are evaluated in combination with the contrast features, and equipment printing defect evaluation information is generated, specifically:

[0044] Merge all predicted defect areas into the area to be tested;

[0045] Obtain the contour features and color features of the area to be tested and mark them as current contour features and current color features;

[0046] Acquire the comparative printing defect features based on the system database, perform feature comparison and printing defect identification on the current contour features, the current color features and the comparative printing defect features, and obtain the defect identification result;

[0047] The real-time operating parameter information of the solder paste printing equipment during the printing process is obtained, and the printing defect evaluation information of the solder paste printing equipment is generated based on the defect recognition results.

[0048] The second aspect of the present invention further provides a printing device real-time evaluation system based on simulation analysis, the system comprising: a memory, a processor, the memory comprising a printing device real-time evaluation program based on simulation analysis, the printing device real-time evaluation program based on simulation analysis being executed by the processor to implement the following steps:

[0049] Collect video data of solder paste printing equipment during the printing process, extract key frames from the video data, and generate key image sets;

[0050] Identify the printing area of ​​the key image set, count the number of solder paste printing points based on the printing area, and divide the printing area into N image areas based on the statistical information;

[0051] Taking the image area as the analysis unit, the contour feature of the image area is extracted through the Sobel operator, and the color feature of the image area is extracted through the color histogram, and multi-dimensional contour features and color features are generated respectively;

[0052] Based on the GAN generation model, the contour features and color features are simulated and analyzed, and simulated data is generated to build an autoencoder, and the simulated data is imported into the autoencoder for model training;

[0053] Based on the autoencoder, feature learning and feature dimension reduction are performed on the contour features and color features, and recognition feature data of the image area is generated. Based on the Euclidean distance, data similarity analysis is performed on the recognition feature data of N image areas, and image areas that deviate from the overall printing area characteristics are screened out and marked as predicted defect areas;

[0054] All predicted defect areas are merged into the area to be tested, printing defects are identified based on the contour features and color features of the area to be tested, defects are evaluated based on the contrast features, and equipment printing defect evaluation information is generated.

[0055] The third aspect of the present invention also provides a computer-readable storage medium, which includes a real-time evaluation program for printing equipment based on simulation analysis. When the real-time evaluation program for printing equipment based on simulation analysis is executed by a processor, the steps of the real-time evaluation method for printing equipment based on simulation analysis as described in any one of the above are implemented.

[0056] The present invention discloses a real-time evaluation method and system for printing equipment based on simulation analysis. By collecting video data of the printing process, key frames are extracted to generate an image set, the printing area is identified, and solder paste printing points are counted, and N image areas are divided. The contour features are extracted by using the Sobel operator, and the color features are extracted by using the color histogram to construct a multidimensional feature set. The GAN model is used to simulate the features, and the autoencoder is subjected to feature learning and dimensionality reduction according to the simulated features, and identification feature data is generated. The similarity of the identification feature data is analyzed based on the Euclidean distance, and the predicted defect area that deviates from the overall feature is screened out. By predicting the corresponding features of the defect area, the printing defects are identified and evaluated in combination with the contrast features, and the equipment printing defect evaluation information is generated. Through the present invention, it is possible to realize the rapid positioning of defects and the prediction of defect areas, optimize the parameters of solder paste printing equipment, and improve production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flow chart showing a method for real-time evaluation of printing equipment based on simulation analysis according to the present invention is shown;

[0058] Figure 2 A block diagram of a printing equipment real-time evaluation system based on simulation analysis according to the present invention is shown. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0061] Figure 1 A flow chart of a method for real-time evaluation of printing equipment based on simulation analysis of the present invention is shown.

[0062] like Figure 1 As shown, the first aspect of the present invention provides a real-time evaluation method for printing equipment based on simulation analysis, comprising:

[0063] S102, collecting video data of the solder paste printing device during the printing process, extracting key frames from the video data, and generating a key image set;

[0064] S104, identifying the printing area of ​​the key image set, counting the number of solder paste printing points based on the printing area, and dividing the printing area into N image areas according to the statistical information;

[0065] S106, taking the image region as the analysis unit, extracting contour features of the image region by using the Sobel operator, extracting color features of the image region by using the color histogram, and generating multi-dimensional contour features and color features respectively;

[0066] S108, performing feature simulation analysis on the contour features and the color features based on the GAN generation model, generating simulation data, constructing an autoencoder, and importing the simulation data into the autoencoder for model training;

[0067] S110, based on the autoencoder, feature learning and feature dimension reduction are performed on the contour features and the color features, and recognition feature data of the image region is generated, and data similarity analysis is performed on the recognition feature data of the N image regions based on the Euclidean distance, and image regions that deviate from the overall printing region features are screened out and marked as predicted defect regions;

[0068] S112, merging all predicted defect areas into a test area, identifying printing defects based on contour features and color features of the test area, performing defect assessment based on comparison features, and generating equipment printing defect assessment information.

[0069] According to an embodiment of the present invention, the video data of the solder paste printing device during the printing process is collected, key frames are extracted from the video data, and a key image set is generated, specifically:

[0070] Based on a printing production cycle, the video data of the solder paste printing equipment during the printing process is collected by a camera device;

[0071] Based on the video processing library, the video data is loaded and key frames are extracted to form a key image set;

[0072] Perform image smoothing and enhancement preprocessing on the image data in the key image set.

[0073] It should be noted that the key image set includes PCB raw material image data before and after printing.

[0074] According to an embodiment of the present invention, the printing area is identified for the key image set, the number of solder paste printing points is counted based on the printing area, and the printing area is divided into N image areas according to the statistical information, specifically:

[0075] Load image data according to the key image set to obtain real-time image loading data;

[0076] Perform printing positioning and printing area recognition on real-time image loading data, and mark the printing area;

[0077] According to the printing area, the real-time image loading data is used to identify and count the printing points of the solder paste, and the statistical data of the printing points in the printing area are obtained;

[0078] The printing area is divided into regions by using the statistical data of printing points to form N image regions;

[0079] During the division process, it is ensured that each image area includes a preset number of solder paste printing dots.

[0080] It should be noted that the area division may be based on a grid division method or a manual division method.

[0081] According to an embodiment of the present invention, the image region is used as the analysis unit, contour features are extracted from the image region through the Sobel operator, color features are extracted from the image region through the color histogram, and multi-dimensional contour features and color features are generated respectively, specifically:

[0082] Taking the image area as the analysis unit, the pixel points in each image area are grayed out and the contour information is analyzed and extracted through the Sobel operator to generate contour features;

[0083] The color statistics of the image area are performed through the color histogram, and the statistical data are vectorized to generate color features;

[0084] The outline features and color features are both multi-dimensional feature data.

[0085] It should be noted that the contour features can analyze the contour features of printed solder paste dots, and the color features can effectively analyze the effect of solder paste printing, and then perform defect analysis later.

[0086] According to an embodiment of the present invention, the GAN-based generation model performs feature simulation analysis on contour features and color features, generates simulated data, constructs an autoencoder, and imports the simulated data into the autoencoder for model training, specifically:

[0087] Build a GAN-based generative model, which includes a generator and a discriminator;

[0088] Integrate contour features and color features into high-dimensional image features, import the high-dimensional image features into the generator for feature learning and simulation generation, and obtain simulation features;

[0089] Import the simulated features into the discriminator to judge the authenticity of the data, and adjust the parameters of the generator and discriminator in real time based on the preset loss function;

[0090] Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium;

[0091] Generate real-time simulation features of a preset data volume through the trained generative model, segment the real-time simulation features into multiple data segments, perform feature similarity analysis on each data segment and the high-dimensional image features, calculate data similarity based on Euclidean distance, and select the expected data from multiple data segments through the preset similarity standard and mark them as simulation data;

[0092] Construct an autoencoder and initialize autoencoder parameters;

[0093] The simulated data is imported into the autoencoder as high-dimensional data for feature learning, and the simulated data is mapped to a low-dimensional representation through a multi-layer perceptron to obtain low-dimensional representation data;

[0094] Reconstruct the low-dimensional representation data to obtain high-dimensional output data;

[0095] The mean square error is used as the loss function to determine the deviation between the high-dimensional output data and the simulated data, and the iterative training is repeated until the loss function converges to a smaller preset value and the autoencoder training is completed.

[0096] It should be noted that the autoencoder includes an encoder and a decoder. The encoder is used to convert high-dimensional data into low-dimensional data representation, and the decoder is used to reconstruct low-dimensional data back to high-dimensional data.

[0097] According to an embodiment of the present invention, based on the autoencoder, feature learning and feature dimensionality reduction are performed on the contour features and color features, and recognition feature data of the image area is generated, and data similarity analysis is performed on the recognition feature data of N image areas based on the Euclidean distance, and image areas that deviate from the overall printing area features are screened out and marked as predicted defect areas, specifically:

[0098] Import contour features and color features into the autoencoder for feature learning and feature dimension reduction, and generate identification feature data for each image area;

[0099] Based on the dimension of the recognition feature data, a multi-dimensional space is constructed, the recognition feature data is mapped to the multi-dimensional space and multiple data points are generated. The distance value between each data point is calculated through the Euclidean distance, and the data similarity analysis of the recognition feature data is performed based on the distance value, and the spatial distribution analysis of all data points is performed;

[0100] In the multi-dimensional space, a data point is selected as the center point, the data range is defined based on the preset distance threshold, and the data points outside the data range are marked to obtain a set of deviated data points;

[0101] All data points are set as center points to perform deviation point analysis, and multiple groups of deviation data points are obtained;

[0102] According to multiple groups of deviation data points, the occurrence frequency of the deviation data points is counted, and the high-frequency data points are marked to obtain multiple defect data points;

[0103] Based on a plurality of defect data points, corresponding identification feature data and corresponding image regions are marked, and the corresponding image regions are used as predicted defect regions.

[0104] It should be noted that, in the generation of multiple data points, each data point corresponds to the identification feature data of each image area. Through the above-mentioned spatial distribution analysis and data point distance analysis process, the data points based on the overall deviation can be effectively screened out, thereby screening out the predicted defective area. A data point is selected as the center point, the data range is delineated based on a preset distance threshold, and the data points outside the data range are marked, that is, the distance between the data points other than the center point and the center point is analyzed to determine whether the distance is greater than the preset distance threshold. If it is greater, it is determined to exceed the data range and is a deviated data point. Each group of deviated data points includes multiple data points. The image area corresponding to the multiple defective data points is the image area that deviates from the overall printing area characteristics.

[0105] It is worth mentioning that in traditional defect analysis technology, solder paste printing defects are often based on manual inspection and manual equipment control, which is time-consuming, labor-intensive and inefficient. Based on this, the present invention collects corresponding high-dimensional image features during the production process, learns the corresponding features through the GAN model and generates simulated data. The simulated data is used for feature learning of the autoencoder model, and through GAN simulation features, it can effectively generate high-similarity features for model learning, so as to predict defect areas in subsequent analysis; through the feature learning of the autoencoder, it can effectively perform feature learning and feature dimensionality reduction extraction from high-dimensional image feature data, and perform similarity analysis on multiple image areas in the printing area, and screen out image areas that deviate from the overall printing area features. The deviated area is the area with a certain probability of defects. The screening process has a high degree of informatization and automation, and can be effectively applied to the fast printing process. For large-scale printing production, it can realize fast defect positioning, especially in the fast assembly line printing process, to achieve accurate defect area positioning and defect tracking marking for multi-product printing production. In the future, corresponding defect tracking and equipment control are carried out based on the defect area, so as to reduce the defect rate of printing production.

[0106] According to an embodiment of the present invention, the predicted defect area includes multiple ones.

[0107] According to an embodiment of the present invention, all predicted defect areas are merged into a test area, printing defects are identified according to contour features and color features of the test area, defects are evaluated in combination with contrast features, and equipment printing defect evaluation information is generated, specifically:

[0108] Merge all predicted defect areas into the area to be tested;

[0109] Obtain the contour features and color features of the area to be tested and mark them as current contour features and current color features;

[0110] Acquire the comparative printing defect features based on the system database, perform feature comparison and printing defect identification on the current contour features, the current color features and the comparative printing defect features, and obtain the defect identification result;

[0111] The real-time operating parameter information of the solder paste printing equipment during the printing process is obtained, and the printing defect evaluation information of the solder paste printing equipment is generated based on the defect recognition results.

[0112] It should be noted that the equipment printing defect assessment information includes the type of printing defect, the time period corresponding to the printing defect, the real-time parameters corresponding to the printing defect, etc. The real-time operation parameter information includes parameters such as printing speed, printing pressure, scraper angle, solder paste viscosity, and solder paste temperature.

[0113] By using the equipment printing defect evaluation information, the performance of the printing equipment can be effectively adjusted, thereby optimizing production based on the defects.

[0114] According to an embodiment of the present invention, it also includes:

[0115] Based on M printing production cycles, defect detection is performed on the solder paste printed products, and the image areas where the defects appear are marked to obtain M groups of actual defect areas;

[0116] Taking each group of actual defect areas as the analysis unit, the deviation between a group of actual defect areas and the predicted defect areas under the corresponding printing production cycle is calculated;

[0117] The deviation degree is calculated specifically by calculating the distance from the center point of each actual defect area to the center point of the nearest predicted defect area, and obtaining multiple distance values, and averaging the multiple distance values ​​to obtain the deviation degree;

[0118] Based on M groups of actual defect areas, M deviations are obtained, and the corresponding Pearson coefficients are calculated using the M deviations as sample data. The Pearson coefficient is used to determine whether there is a linear correlation in the sample data.

[0119] If there is a linear correlation, it is determined that the defects have regional correlation, and the actual defect area and the predicted defect area in the last printing production cycle are marked as the positioning deviation area;

[0120] Solder paste printing is performed based on the positioning deviation area to evaluate positioning defects and correct the positioning of the solder paste printing equipment.

[0121] It should be noted that the solder paste printed product is a PCB printed product. M groups of actual defect areas correspond to M printing production cycles, and each group of actual defect areas includes at least one image area. In the determination of whether there is a linear correlation by the Pearson coefficient, it is specifically determined whether the Pearson coefficient is within a preset numerical range. If so, it means that there is a linear correlation in the deviation, that is, the actual defect area is correlated with the predicted defect area. At the same time, it means that there is a regional correlation in the occurrence of defects. At this time, it can be concluded that there is a high probability of deviation in PCB positioning, resulting in defect correlation in certain defect areas. It is necessary to conduct a positioning defect assessment on the PCB, and reduce the continuous occurrence of defects by adjusting the PCB positioning in time.

[0122] Figure 2 A block diagram of a printing equipment real-time evaluation system based on simulation analysis according to the present invention is shown.

[0123] The second aspect of the present invention further provides a printing device real-time evaluation system 2 based on simulation analysis, the system comprising: a memory 21, a processor 22, the memory 21 comprising a printing device real-time evaluation program based on simulation analysis, the printing device real-time evaluation program based on simulation analysis when executed by the processor 22 implements the following steps:

[0124] Collect video data of solder paste printing equipment during the printing process, extract key frames from the video data, and generate key image sets;

[0125] Identify the printing area of ​​the key image set, count the number of solder paste printing points based on the printing area, and divide the printing area into N image areas based on the statistical information;

[0126] Taking the image area as the analysis unit, the contour feature of the image area is extracted through the Sobel operator, and the color feature of the image area is extracted through the color histogram, and multi-dimensional contour features and color features are generated respectively;

[0127] Based on the GAN generation model, the contour features and color features are simulated and analyzed, and simulated data is generated to build an autoencoder, and the simulated data is imported into the autoencoder for model training;

[0128] Based on the autoencoder, feature learning and feature dimension reduction are performed on the contour features and color features, and recognition feature data of the image area is generated. Based on the Euclidean distance, data similarity analysis is performed on the recognition feature data of N image areas, and image areas that deviate from the overall printing area characteristics are screened out and marked as predicted defect areas;

[0129] All predicted defect areas are merged into the area to be tested, printing defects are identified based on the contour features and color features of the area to be tested, defects are evaluated based on the contrast features, and equipment printing defect evaluation information is generated.

[0130] According to an embodiment of the present invention, the video data of the solder paste printing device during the printing process is collected, key frames are extracted from the video data, and a key image set is generated, specifically:

[0131] Based on a printing production cycle, the video data of the solder paste printing equipment during the printing process is collected by a camera device;

[0132] Based on the video processing library, the video data is loaded and key frames are extracted to form a key image set;

[0133] Perform image smoothing and enhancement preprocessing on the image data in the key image set.

[0134] It should be noted that the key image set includes PCB raw material image data before and after printing.

[0135] According to an embodiment of the present invention, the printing area is identified for the key image set, the number of solder paste printing points is counted based on the printing area, and the printing area is divided into N image areas according to the statistical information, specifically:

[0136] Load image data according to the key image set to obtain real-time image loading data;

[0137] Perform printing positioning and printing area recognition on real-time image loading data, and mark the printing area;

[0138] According to the printing area, the real-time image loading data is used to identify and count the printing points of the solder paste, and the statistical data of the printing points in the printing area are obtained;

[0139] The printing area is divided into regions by using the statistical data of printing points to form N image regions;

[0140] During the division process, it is ensured that each image area includes a preset number of solder paste printing dots.

[0141] It should be noted that the area division may be based on a grid division method or a manual division method.

[0142] According to an embodiment of the present invention, the image region is used as the analysis unit, contour features are extracted from the image region through the Sobel operator, color features are extracted from the image region through the color histogram, and multi-dimensional contour features and color features are generated respectively, specifically:

[0143] Taking the image area as the analysis unit, the pixel points in each image area are grayed out and the contour information is analyzed and extracted through the Sobel operator to generate contour features;

[0144] The color statistics of the image area are performed through the color histogram, and the statistical data are vectorized to generate color features;

[0145] The outline features and color features are both multi-dimensional feature data.

[0146] It should be noted that the contour features can analyze the contour features of printed solder paste dots, and the color features can effectively analyze the effect of solder paste printing, and then perform defect analysis later.

[0147] According to an embodiment of the present invention, the GAN-based generation model performs feature simulation analysis on contour features and color features, generates simulated data, constructs an autoencoder, and imports the simulated data into the autoencoder for model training, specifically:

[0148] Build a GAN-based generative model, which includes a generator and a discriminator;

[0149] Integrate contour features and color features into high-dimensional image features, import the high-dimensional image features into the generator for feature learning and simulation generation, and obtain simulation features;

[0150] Import the simulated features into the discriminator to judge the authenticity of the data, and adjust the parameters of the generator and discriminator in real time based on the preset loss function;

[0151] Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium;

[0152] Generate real-time simulation features of a preset data volume through the trained generative model, segment the real-time simulation features into multiple data segments, perform feature similarity analysis on each data segment and the high-dimensional image features, calculate data similarity based on Euclidean distance, and select the expected data from multiple data segments through the preset similarity standard and mark them as simulation data;

[0153] Construct an autoencoder and initialize autoencoder parameters;

[0154] The simulated data is imported into the autoencoder as high-dimensional data for feature learning, and the simulated data is mapped to a low-dimensional representation through a multi-layer perceptron to obtain low-dimensional representation data;

[0155] Reconstruct the low-dimensional representation data to obtain high-dimensional output data;

[0156] The mean square error is used as the loss function to determine the deviation between the high-dimensional output data and the simulated data, and the iterative training is repeated until the loss function converges to a smaller preset value and the autoencoder training is completed.

[0157] It should be noted that the autoencoder includes an encoder and a decoder. The encoder is used to convert high-dimensional data into low-dimensional data representation, and the decoder is used to reconstruct low-dimensional data back to high-dimensional data.

[0158] According to an embodiment of the present invention, based on the autoencoder, feature learning and feature dimensionality reduction are performed on the contour features and color features, and recognition feature data of the image area is generated, and data similarity analysis is performed on the recognition feature data of N image areas based on the Euclidean distance, and image areas that deviate from the overall printing area features are screened out and marked as predicted defect areas, specifically:

[0159] Import contour features and color features into the autoencoder for feature learning and feature dimension reduction, and generate identification feature data for each image area;

[0160] Based on the dimension of the recognition feature data, a multi-dimensional space is constructed, the recognition feature data is mapped to the multi-dimensional space and multiple data points are generated. The distance value between each data point is calculated through the Euclidean distance, and the data similarity analysis of the recognition feature data is performed based on the distance value, and the spatial distribution analysis of all data points is performed;

[0161] In the multi-dimensional space, a data point is selected as the center point, the data range is defined based on the preset distance threshold, and the data points outside the data range are marked to obtain a set of deviated data points;

[0162] All data points are set as center points to perform deviation point analysis, and multiple groups of deviation data points are obtained;

[0163] According to multiple groups of deviation data points, the occurrence frequency of the deviation data points is counted, and the high-frequency data points are marked to obtain multiple defect data points;

[0164] Based on a plurality of defect data points, corresponding identification feature data and corresponding image regions are marked, and the corresponding image regions are used as predicted defect regions.

[0165] It should be noted that, in the generation of multiple data points, each data point corresponds to the identification feature data of each image area. Through the above-mentioned spatial distribution analysis and data point distance analysis process, the data points based on the overall deviation can be effectively screened out, thereby screening out the predicted defective area. A data point is selected as the center point, the data range is delineated based on a preset distance threshold, and the data points outside the data range are marked, that is, the distance between the data points other than the center point and the center point is analyzed to determine whether the distance is greater than the preset distance threshold. If it is greater, it is determined to exceed the data range and is a deviated data point. Each group of deviated data points includes multiple data points. The image area corresponding to the multiple defective data points is the image area that deviates from the overall printing area characteristics.

[0166] It is worth mentioning that in traditional defect analysis technology, solder paste printing defects are often based on manual inspection and manual equipment control, which is time-consuming, labor-intensive and inefficient. Based on this, the present invention collects corresponding high-dimensional image features during the production process, learns the corresponding features through the GAN model and generates simulated data. The simulated data is used for feature learning of the autoencoder model, and through GAN simulation features, it can effectively generate high-similarity features for model learning, so as to predict defect areas in subsequent analysis; through the feature learning of the autoencoder, it can effectively perform feature learning and feature dimensionality reduction extraction from high-dimensional image feature data, and perform similarity analysis on multiple image areas in the printing area, and screen out image areas that deviate from the overall printing area features. The deviated area is the area with a certain probability of defects. The screening process has a high degree of informatization and automation, and can be effectively applied to the fast printing process. For large-scale printing production, it can realize fast defect positioning, especially in the fast assembly line printing process, to achieve accurate defect area positioning and defect tracking marking for multi-product printing production. In the future, corresponding defect tracking and equipment control are carried out based on the defect area, so as to reduce the defect rate of printing production.

[0167] According to an embodiment of the present invention, the predicted defect area includes multiple ones.

[0168] According to an embodiment of the present invention, all predicted defect areas are merged into a test area, printing defects are identified according to contour features and color features of the test area, defects are evaluated in combination with contrast features, and equipment printing defect evaluation information is generated, specifically:

[0169] Merge all predicted defect areas into the area to be tested;

[0170] Obtain the contour features and color features of the area to be tested and mark them as current contour features and current color features;

[0171] Acquire the comparative printing defect features based on the system database, perform feature comparison and printing defect identification on the current contour features, the current color features and the comparative printing defect features, and obtain the defect identification result;

[0172] The real-time operating parameter information of the solder paste printing equipment during the printing process is obtained, and the printing defect evaluation information of the solder paste printing equipment is generated based on the defect recognition results.

[0173] It should be noted that the equipment printing defect assessment information includes the type of printing defect, the time period corresponding to the printing defect, the real-time parameters corresponding to the printing defect, etc. The real-time operation parameter information includes parameters such as printing speed, printing pressure, scraper angle, solder paste viscosity, and solder paste temperature.

[0174] By using the equipment printing defect evaluation information, the performance of the printing equipment can be effectively adjusted, thereby optimizing production based on the defects.

[0175] According to an embodiment of the present invention, it also includes:

[0176] Based on M printing production cycles, defect detection is performed on the solder paste printed products, and the image areas where the defects appear are marked to obtain M groups of actual defect areas;

[0177] Taking each group of actual defect areas as the analysis unit, the deviation between a group of actual defect areas and the predicted defect areas under the corresponding printing production cycle is calculated;

[0178] The deviation degree is calculated specifically by calculating the distance from the center point of each actual defect area to the center point of the nearest predicted defect area, and obtaining multiple distance values, and averaging the multiple distance values ​​to obtain the deviation degree;

[0179] Based on M groups of actual defect areas, M deviations are obtained, and the corresponding Pearson coefficients are calculated using the M deviations as sample data. The Pearson coefficient is used to determine whether there is a linear correlation in the sample data.

[0180] If there is a linear correlation, it is determined that the defects have regional correlation, and the actual defect area and the predicted defect area in the last printing production cycle are marked as the positioning deviation area;

[0181] Solder paste printing is performed based on the positioning deviation area to evaluate positioning defects and correct the positioning of the solder paste printing equipment.

[0182] It should be noted that the solder paste printed product is a PCB printed product. M groups of actual defect areas correspond to M printing production cycles, and each group of actual defect areas includes at least one image area. In the determination of whether there is a linear correlation by the Pearson coefficient, it is specifically determined whether the Pearson coefficient is within a preset numerical range. If so, it means that there is a linear correlation in the deviation, that is, the actual defect area is correlated with the predicted defect area. At the same time, it means that there is a regional correlation in the occurrence of defects. At this time, it can be concluded that there is a high probability of deviation in PCB positioning, resulting in defect correlation in certain defect areas. It is necessary to conduct a positioning defect assessment on the PCB, and reduce the continuous occurrence of defects by adjusting the PCB positioning in time.

[0183] The third aspect of the present invention also provides a computer-readable storage medium, which includes a real-time evaluation program for printing equipment based on simulation analysis. When the real-time evaluation program for printing equipment based on simulation analysis is executed by a processor, the steps of the real-time evaluation method for printing equipment based on simulation analysis as described in any one of the above are implemented.

[0184] The present invention discloses a real-time evaluation method and system for printing equipment based on simulation analysis. By collecting video data of the printing process, key frames are extracted to generate an image set, the printing area is identified, and solder paste printing points are counted, and N image areas are divided. The contour features are extracted by using the Sobel operator, and the color features are extracted by using the color histogram to construct a multidimensional feature set. The GAN model is used to simulate the features, and the autoencoder is subjected to feature learning and dimensionality reduction according to the simulated features, and identification feature data is generated. The similarity of the identification feature data is analyzed based on the Euclidean distance, and the predicted defect area that deviates from the overall feature is screened out. By predicting the corresponding features of the defect area, the printing defects are identified and evaluated in combination with the contrast features, and the equipment printing defect evaluation information is generated. Through the present invention, it is possible to realize the rapid positioning of defects and the prediction of defect areas, optimize the parameters of solder paste printing equipment, and improve production efficiency.

[0185] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0186] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0187] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0188] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0189] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0190] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A real-time evaluation method for printing equipment based on simulation analysis, characterized in that: include: Collect video data of solder paste printing equipment during the printing process, extract key frames from the video data, and generate key image sets; Identify the printing area of ​​the key image set, count the number of solder paste printing points based on the printing area, and divide the printing area into N image areas based on the statistical information; Taking the image area as the analysis unit, the contour feature of the image area is extracted through the Sobel operator, and the color feature of the image area is extracted through the color histogram, and multi-dimensional contour features and color features are generated respectively; Based on the GAN generation model, the contour features and color features are simulated and analyzed, and simulated data is generated to build an autoencoder, and the simulated data is imported into the autoencoder for model training; Based on the autoencoder, feature learning and feature dimension reduction are performed on the contour features and color features, and recognition feature data of the image area is generated. Based on the Euclidean distance, data similarity analysis is performed on the recognition feature data of N image areas, and image areas that deviate from the overall printing area characteristics are screened out and marked as predicted defect areas; All predicted defect areas are merged into the area to be tested, and printing defects are identified based on the contour features and color features of the area to be tested. Defect evaluation is performed based on the comparison features, and equipment printing defect evaluation information is generated; The image region is used as the analysis unit, and the contour feature of the image region is extracted by the Sobel operator, and the color feature of the image region is extracted by the color histogram, and multi-dimensional contour features and color features are generated respectively, specifically: Taking the image area as the analysis unit, the pixel points in each image area are grayed out and the contour information is analyzed and extracted through the Sobel operator to generate contour features; The color statistics of the image area are performed through the color histogram, and the statistical data are vectorized to generate color features; The outline features and color features are both multi-dimensional feature data; The GAN generation model is used to perform feature simulation analysis on contour features and color features, generate simulated data, construct an autoencoder, and import the simulated data into the autoencoder for model training, specifically: Build a GAN-based generative model, which includes a generator and a discriminator; Integrate contour features and color features into high-dimensional image features, import the high-dimensional image features into the generator for feature learning and simulation generation, and obtain simulation features; Import the simulated features into the discriminator to judge the authenticity of the data, and adjust the parameters of the generator and discriminator in real time based on the preset loss function; Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium; Generate real-time simulation features of a preset data volume through the trained generative model, segment the real-time simulation features into multiple data segments, perform feature similarity analysis on each data segment and the high-dimensional image features, calculate data similarity based on Euclidean distance, and select the expected data from multiple data segments through the preset similarity standard and mark them as simulation data; Construct an autoencoder and initialize autoencoder parameters; The simulated data is imported into the autoencoder as high-dimensional data for feature learning, and the simulated data is mapped to a low-dimensional representation through a multi-layer perceptron to obtain low-dimensional representation data; Reconstruct the low-dimensional representation data to obtain high-dimensional output data; The mean square error is used as the loss function to determine the deviation between the high-dimensional output data and the simulated data, and the iterative training is repeated until the loss function converges to a smaller preset value and the autoencoder training is completed.

2. The method for real-time evaluation of printing equipment based on simulation analysis according to claim 1, characterized in that: The method collects video data of the solder paste printing device during the printing process, extracts key frames from the video data, and generates a key image set, specifically: Based on a printing production cycle, the video data of the solder paste printing equipment during the printing process is collected by a camera device; Based on the video processing library, the video data is loaded and key frames are extracted to form a key image set; Perform image smoothing and enhancement preprocessing on the image data in the key image set.

3. The real-time evaluation method for printing equipment based on simulation analysis according to claim 1 is characterized in that: The printing area is identified for the key image set, the number of solder paste printing points is counted based on the printing area, and the printing area is divided into N image areas according to the statistical information, specifically: Load image data according to the key image set to obtain real-time image loading data; Perform printing positioning and printing area recognition on real-time image loading data, and mark the printing area; According to the printing area, the real-time image loading data is used to identify and count the printing points of the solder paste, and the statistical data of the printing points in the printing area are obtained; The printing area is divided into regions by using the statistical data of printing points to form N image regions; During the division process, it is ensured that each image area includes a preset number of solder paste printing dots.

4. The real-time evaluation method for printing equipment based on simulation analysis according to claim 1, characterized in that: Based on the autoencoder, feature learning and feature dimension reduction are performed on the contour features and color features, and recognition feature data of the image area is generated. Based on the Euclidean distance, the recognition feature data of N image areas are analyzed for data similarity, and image areas that deviate from the overall printing area features are screened out and marked as predicted defect areas, specifically: Import contour features and color features into the autoencoder for feature learning and feature dimension reduction, and generate identification feature data for each image area; Based on the dimension of the recognition feature data, a multi-dimensional space is constructed, the recognition feature data is mapped to the multi-dimensional space and multiple data points are generated. The distance value between each data point is calculated through the Euclidean distance, and the data similarity analysis of the recognition feature data is performed based on the distance value, and the spatial distribution analysis of all data points is performed; In the multi-dimensional space, a data point is selected as the center point, the data range is defined based on the preset distance threshold, and the data points outside the data range are marked to obtain a set of deviated data points; All data points are set as center points to perform deviation point analysis, and multiple groups of deviation data points are obtained; According to multiple groups of deviation data points, the occurrence frequency of the deviation data points is counted, and the high-frequency data points are marked to obtain multiple defect data points; Based on a plurality of defect data points, corresponding identification feature data and corresponding image regions are marked, and the corresponding image regions are used as predicted defect regions.

5. The method for real-time evaluation of printing equipment based on simulation analysis according to claim 4, characterized in that: The predicted defect areas include a plurality of areas.

6. A printing equipment real-time evaluation method based on simulation analysis according to claim 5, characterized in that: The method combines all predicted defect areas into a test area, identifies printing defects based on the contour features and color features of the test area, evaluates defects based on the comparison features, and generates equipment printing defect evaluation information, specifically: Merge all predicted defect areas into the area to be tested; Obtain the contour features and color features of the area to be tested and mark them as current contour features and current color features; Acquire the comparative printing defect features based on the system database, perform feature comparison and printing defect identification on the current contour features, the current color features and the comparative printing defect features, and obtain the defect identification result; The real-time operating parameter information of the solder paste printing equipment during the printing process is obtained, and the printing defect evaluation information of the solder paste printing equipment is generated based on the defect recognition results.

7. A real-time evaluation system for printing equipment based on simulation analysis, characterized in that: The system includes: a memory and a processor, wherein the memory includes a real-time evaluation program for printing equipment based on simulation analysis, and when the real-time evaluation program for printing equipment based on simulation analysis is executed by the processor, the steps of the real-time evaluation method for printing equipment based on simulation analysis as described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a real-time evaluation program for printing equipment based on simulation analysis. When the real-time evaluation program for printing equipment based on simulation analysis is executed by a processor, the steps of the real-time evaluation method for printing equipment based on simulation analysis as described in any one of claims 1 to 6 are implemented.

Images