Data processing method and device, terminal equipment and computer readable storage medium
By screening and reducing the characteristic data of LCD panel production data, training the neural network model for detection solves the problem of frequent interception and increase cost in LCD panel production, and achieving efficient quality prediction and production efficiency.
Patent Information
- Application Number
- CN202310724727.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-16
- Publication Date
- 2024-12-17
AI Technical Summary
During the production process of LCD panels, frequent product interception is required, which increases production costs and reduces production efficiency. Traditional methods rely on complex neural network models for quality prediction, resulting in too large model calculation parameters and increasing equipment burden.
By screening the data sets of the trained neural network, using less feature data to train the detection model, so that the model can only detect based on fewer data features during the application stage, reducing the model's operation parameters.
It reduces the computational burden of the model, improves production efficiency, reduces production costs, and ensures prediction accuracy.
Smart Images
Figure CN119149983A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, terminal device and computer-readable storage medium. Background Art
[0002] In the electronic information industry, the production process of LCD panels is complex and requires high precision. During the production process, multiple measurement stations need to be set up and product inspections need to be frequently carried out to ensure product quality, which increases production costs and reduces production efficiency.
[0003] In liquid crystal panels, electrical properties (or other measured values) are important indicators that affect the panel yield. The traditional method is to use special equipment to perform random inspections on the panels in production during the production process, extract unqualified products or perform secondary repairs to prevent unqualified products from flowing out; known technologies generally use prediction methods to replace random inspections, thereby achieving virtual full inspections to improve the efficiency of the production line. Alternatively, the quality parameters of liquid crystal panels are predicted through neural network models, and more complex neural network models are generally used. Summary of the invention
[0004] An embodiment of the present application provides a data processing method, which filters the data in the data set for training the neural network and trains the detection model with less feature data, so that the detection model can only perform detection based on fewer data features during the application stage, thereby reducing the model's operating parameters.
[0005] In a first aspect, the present application provides a data processing method, the method comprising:
[0006] Obtain sample production data for sample panels;
[0007] The sample production data is screened to obtain target sample production data for training a detection model; the detection model is used to detect the panel quality of the sample panel.
[0008] In a second aspect, the present application further provides a data processing device, the device comprising:
[0009] An acquisition module, used for acquiring sample production data of a sample panel;
[0010] The screening module is used to screen the sample production data to obtain target sample production data for training a detection model; the detection model is used to detect the panel quality of the sample panel.
[0011] In a third aspect, the present application also provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in any one of the data processing methods described.
[0012] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps in any one of the data processing methods described.
[0013] The data processing method provided by the present application obtains partial target sample production data by screening sample production data. The detection model for detecting display panels is trained with the target sample production data, so that when the model performs subsequent prediction tasks, the quality parameters of the display panel can be predicted based on only partial production data, which reduces the calculation parameters of the model and further reduces the burden on the computing device. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0015] Figure 1 is a scenario schematic diagram of a data processing system provided in an embodiment of the present application;
[0016] Figure 2 This is a flow chart of an embodiment of a data processing method in an embodiment of the present application;
[0017] Figure 3 is a functional module schematic diagram of a data processing device in an embodiment of the present application;
[0018] Figure 4 It is a schematic diagram of the structure of the terminal device in the embodiment of the present application. DETAILED DESCRIPTION
[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0020] In the description of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the feature. In the description of the present application, "plurality" means two or more, unless otherwise clearly and specifically defined.
[0021] In this application, the word "exemplary" is used to mean "used as an example, illustration or description". Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. At the same time, it is to be understood that in the specific implementation of this application, when user information, user data and other related data are involved, when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data shall comply with relevant laws, regulations and standards of relevant countries and regions.
[0022] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art will recognize that the present application can be implemented without using these specific details. In other examples, known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present application obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in the present application.
[0023] The present application provides a data processing method, apparatus, device and storage medium, which are described in detail below.
[0024] See also Figure 1 , Figure 1 Schematic diagram of a data processing system provided in an embodiment of the present application. The data processing system may include a terminal device 100 and a storage device 200. The storage device 200 may transmit data to the terminal device 100. Figure 1 The terminal device 100 in the embodiment can obtain the sample data of the sample panel stored in the storage device 200 to execute the data processing method in the present application.
[0025] In the embodiment of the present application, the terminal device 100 includes but is not limited to a desktop computer, a portable computer, a network server, a PDA (Personal Digital Assistant), a tablet computer, a wireless terminal device, an embedded device, etc.
[0026] In an embodiment of the present application, communication between the terminal device 100 and the storage device 200 can be achieved through any communication method, including but not limited to mobile communications based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and Worldwide Interoperability for Microwave Access (WiMAX), or computer network communications based on the TCP / IP Protocol Suite (TCP / IP) and User Datagram Protocol (UDP).
[0027] It should be noted that Figure 1 The scenario diagram of the data processing system shown is merely an example. The data processing system and scenario described in the embodiments of the present application are intended to more clearly illustrate the technical solution of the embodiments of the present application, and do not constitute a limitation on the technical solution provided in the embodiments of the present application. A person of ordinary skill in the art will appreciate that with the evolution of the data processing system and the emergence of new business scenarios, the technical solution provided in the embodiments of the present application is equally applicable to similar technical problems.
[0028] like Figure 2 As shown, Figure 2 This is a flow chart of an embodiment of a data processing method in an embodiment of the present application. The data processing method may include the following steps 201 to 202:
[0029] 201. Obtain sample production data of a sample panel.
[0030] The sample panel involved in the embodiment of the present application may be a liquid crystal panel, and the sample production data may represent the data generated when the liquid crystal panel is produced. For example, when the liquid crystal panel is produced, most of them will go through the following production processes: Substrate preparation: Processing and treating germanium glass, silicon glass or other substrate materials to form a flat, defect-free substrate surface. Coating: Coating a layer of transparent conductive material, such as indium tin oxide (ITO), on the surface of the substrate to form electrodes and ensure that the liquid crystal material can be evenly distributed on the surface of the substrate. Liquid crystal injection: Injecting the pre-modulated liquid crystal material between the two substrates through vacuum to form a liquid crystal layer, so that the surface of the substrate is evenly covered with liquid crystal material. Packaging: Bonding two substrates coated with liquid crystal together with an adhesive and packaging them in a packaging structure, and adding the required backlight source, polarizer and other components to form a complete liquid crystal screen. Therefore, the production data may at least include: substrate processing parameters, coating application parameters, liquid crystal material injection parameters, and packaging parameters, etc. Therefore, the processing parameters of the substrate, the coating parameters, the injection parameters of the liquid crystal material, and the packaging parameters can be collectively regarded as the production data of the liquid crystal panel; each of the processing parameters of the substrate, the coating parameters, the injection parameters of the liquid crystal material, and the packaging parameters are the sub-sample production data corresponding to the sample panel.
[0031] 202. Screen the sample production data to obtain target sample production data for training a detection model; the detection model is used to detect panel quality of the sample panel.
[0032] Therefore, according to the above steps, the production data can actually include multiple sub-production data. For the detection model used to predict panel quality parameters, the smaller the calculation parameters of the model, the smaller the operating burden of the hardware device equipped with the model. Therefore, in order to reduce the operating burden of the equipment, it is necessary to reduce the calculation amount of the model.
[0033] However, the amount of calculation of the model can be reduced by reducing the calculation parameters of the model. Specifically, if the model refers to fewer feature data when predicting the quality parameters of the panel, the amount of calculation of the model will be smaller. Therefore, in order to achieve this effect, during the training phase of the model, some data can be selected from a large amount of production data to train the model, rather than inputting all feature data into the model for training. In this way, during the actual application phase, the model only needs to consider some feature parameters used for training during the training phase to make predictions.
[0034] However, it is well known that if the types of feature data used in the training phase are fewer, the prediction accuracy of the model may not be guaranteed. Therefore, in order to reduce the amount of calculation of the model and at the same time ensure a certain accuracy, it is necessary to screen out feature data with obvious features or large feature weights from a large number of data and input them into the model for training. In the embodiment of the present application, many types of sub-production data can be randomly grouped and untrained detection models can be trained at the same time. For example: Assume that the production data corresponding to each sample panel includes 5 different types of sub-production data. At this time, 5 different types of sub-production data can be randomly captured, but the number of captured sub-production data needs to be less than 5, so that at least 1 type of production data can be eliminated. At this time, the captured sub-production data is input into the model for training. This step can be repeated many times until the results of random capture are exhausted. In this way, multiple detection models trained according to different training sets can be obtained. Afterwards, when verifying the prediction accuracy of multiple detection models, a detection model with the highest accuracy can be selected as the model that finally completes the training, and the sub-production data in the training set corresponding to the detection model with the highest accuracy is the target sample production parameter obtained after the screening. It should be noted that in the embodiment of the present application, the retained sub-production data are regarded as target sample production parameters, and the unreserved sub-production data and the retained sub-production data are collectively regarded as sample production data.
[0035] It should be noted that each sub-production data corresponds to a label, which can be a manually labeled label. Therefore, when the sub-production data is input into the model, the corresponding label is also input at the same time, and the model training method of the embodiment of the present application can adopt a supervised training method.
[0036] The data processing method provided by the present application obtains partial target sample production data by screening sample production data. The detection model for detecting display panels is trained with the target sample production data, so that when the model performs subsequent prediction tasks, the quality parameters of the display panel can be predicted based on only partial production data, which reduces the calculation parameters of the model and further reduces the burden on the computing device.
[0037] In order to better implement the embodiment of the present application, in one embodiment of the present application, the sample production data includes multiple sample production parameters, and the sample production data is screened to obtain target sample production data for training the detection model, including:
[0038] The sample production data are grouped to obtain a training sample data set and a test sample data set; the type distribution data of each sample production parameter in the training sample data set and the type distribution data of each sample production parameter in the test sample data set are determined through a preset classification model; based on the first type distribution data of each sample production parameter in the training sample data set and the second type distribution data of each sample production parameter in the test sample data set, the target sample production parameters for training the detection model are determined; based on the target sample production parameters, the target sample production data for training the detection model are obtained from the sample production data.
[0039] The above embodiment provides a random method for filtering sub-production data from production data. However, the efficiency of the random method is low. Therefore, in order to improve the efficiency of filtering data, the embodiment of the present application provides a non-random filtering method.
[0040] Specifically, when performing data screening, the embodiment of the present application can group the sample panels to determine the training data set and the test data set. And then, through a trained classification model, classify each sub-sample production data of each sample panel in the training data set to obtain the first predicted distribution of each sub-sample production data of each sample panel in the training set. At the same time, the same operation is performed on the sub-sample production data of each sample panel in the test set to obtain the second predicted distribution of each sub-sample production data of each sample panel in the test set. Afterwards, compare whether the distribution of each sub-sample production data in the two predicted distributions is consistent. If the distribution is the same, the sub-sample production data with the same distribution is retained; if the distribution is different, the sub-sample production data with different distributions is eliminated. Finally, according to the type of the sub-sample production data left, the sample production data with the same type as the retained sub-sample production data is obtained from all sample panels as the training data for training the detection model.
[0041] In order to better implement the embodiment of the present application, in one embodiment of the present application, according to the first type of distribution data of each sample production parameter in the training sample data set and the second type of distribution data of each sample production parameter in the test sample data set, the target sample production parameter for training the detection model is determined, including:
[0042] The first type distribution data of each sample production parameter and the second type distribution data of each sample production parameter are compared to determine the accuracy of each sample production parameter; the sample production parameters with an accuracy lower than a preset accuracy threshold are eliminated to obtain the target sample production parameters for training the detection model.
[0043] In the above embodiment, a scheme is provided for using sub-sample production data with the same distribution as data for training the detection model. However, in actual situations, there are situations where the distribution is not exactly the same. For example, the distribution of sub-sample production data A in the first prediction distribution is 90% the same as that of sub-sample production data A in the second prediction distribution. At this time, sub-sample production data A can still be regarded as data with relatively obvious characteristics for the prediction of the model. Therefore, removing sub-sample production data A at this time may affect subsequent model training. Therefore, an accuracy threshold can be set, and the accuracy threshold can be set according to actual conditions, such as 10%. If the prediction accuracy of the classification model for sub-sample production data A of the training sample data set is 90%, and the prediction accuracy of the classification model for sub-sample production data A of the test sample data set is 85%, the difference between the two accuracy rates is only 5%, and at this time 5% is less than 10%, so the sub-sample production data A is a reserved data type. By analogy, the retention of each sub-sample production data is determined, and then the target sample production parameters for training the detection model are screened out. It should be noted that the specific value of the accuracy threshold in the embodiment of the present application can be adjusted according to the situation, and the specific value of the accuracy threshold does not constitute a limitation on the embodiment of the present application.
[0044] In order to better implement the embodiment of the present application, in one embodiment of the present application, according to the target sample production parameters, target sample production data for training the detection model is obtained from the sample production data, including:
[0045] The target sample production parameters are input into a preset importance condition decision tree to obtain the importance parameters of each target sample production parameter; according to the importance parameters, each target sample production parameter is ranked in importance to obtain the target sample production data for training the detection model and including the importance ranking.
[0046] However, if the target sample production parameters, that is, the retained parameter types, are all used for model training, it may still cause the model to occupy more computing resources during calculation. Therefore, in order to further reduce the computing burden of the model, the embodiment of the present application also provides a data screening method.
[0047] Specifically, the importance of various types of parameters in the target sample production parameters can be evaluated based on the target sample production parameters to obtain the importance ranking of each sub-production data type in the target sample production parameters. Then, the sub-production data with lower rankings are eliminated, leaving the sub-production data with higher importance. At this point, the model is trained based on the sub-production data with higher importance, which can further reduce the amount of calculation of the model.
[0048] In an embodiment of the present application, the importance condition decision tree can use the decision tree of GridSearch to output the importance coefficients of each sub-production data in the target sample production parameters. Decision tree is a supervised learning algorithm based on a tree structure, which is used for classification and regression problems. In a decision tree, each internal node represents a feature or attribute, each branch represents a feature value or attribute condition, and each leaf node represents a category or target value. The decision tree gradually classifies the input samples into leaf nodes through a series of judgment conditions, thereby performing prediction or classification tasks. GridSearch is a method for systematically traversing a given parameter combination, which searches for the best model parameters by exhaustively enumerating all possible parameter combinations. When using GridSearch for decision tree model selection, we can define a set of decision tree-related hyperparameters in advance, such as maximum depth, minimum sample split number, etc., and then traverse all possible combinations of these parameters through GridSearch, evaluate model performance on the basis of cross-validation, and select the parameter combination with the best performance as the final decision tree model. For example: set importance conditions for each branch node in the GridSearch decision tree. At this time, each sub-production data in the target sample production parameter is judged according to the importance conditions in each node. The more sub-production data that pass the importance judgment conditions, the higher the importance coefficient. According to this method, the importance coefficient of each sub-production data in the target sample production parameter can be obtained. Then, according to the size of the coefficient, the importance of each sub-production data in the target sample production parameter is sorted from large to small, that is, the importance ranking of each sub-production data in the target sample production parameter can be obtained.
[0049] Afterwards, when training the model based on the importance ranking, you can remove the data with the lowest ranking, and then train the model based on the remaining data.
[0050] In order to better implement the embodiment of the present application, in one embodiment of the present application, each target sample production parameter is ranked in importance according to the importance parameter, and target sample production data including importance ranking for training the detection model is obtained, including:
[0051] According to the order of importance ranking, the top N target sample production parameters are input into the detection model to obtain the first accuracy of the top N target sample production parameters; according to the order of importance ranking, the top N+1 target sample production parameters are input into the detection model to obtain the second accuracy of the top N+1 target sample production parameters; according to the first accuracy and the second accuracy, a first accuracy change value is obtained; if it is determined that the first accuracy change value is less than a preset first accuracy change threshold, it is determined that the input target sample production parameters are the initial sample production parameter data set used to train the detection model.
[0052] The above embodiment provides a solution to remove the data with a low ranking and then train the model based on the retained data. However, this method has poor flexibility, and sometimes the prediction accuracy of the model trained according to this method is low. Therefore, in order to solve this problem, the embodiment of the present application provides a solution for inputting the sub-production data into the model for training in sequence according to the importance ranking, so as to gradually determine the change in the prediction accuracy of the model, and then determine the degree of influence of the newly added training data on the model prediction based on the change. If the degree of influence is low, it can be determined that the newly added data will cause the amount of calculation of the model to increase.
[0053] Specifically, in an embodiment of the present application, it is assumed that there are 10 sub-production data for the target sample production parameter, and the value of N is less than 10. For example: when the value of N is 5, the sub-production data ranked top 5 in importance are used as training data. For example: if there are 10,000 sample panels, the 10,000 sample panels can be grouped to obtain a training set and a test set. At this time, according to the importance ranking, the top 5 sub-production data corresponding to the sample panel in the training set are input into the model for training to obtain a detection model that has completed the training. At this time, when the test set is input into the detection model, the detection model can make predictions based on the top 5 sub-production data corresponding to the sample panel in the test set to obtain a prediction accuracy, which is the first accuracy.
[0054] And N+1 is 6, at this time, the sub-production data ranked in the top 6 in importance are used as training data. For example: according to the importance ranking, the sub-production data corresponding to the top 6 in the sample panel in the training set are input into the model for training to obtain a detection model that has completed the training. At this time, when the test set is input into the detection model, the detection model can make predictions based on the top 6 sub-production data corresponding to the sample panel in the test set to obtain a prediction accuracy, which is the second accuracy. Therefore, in the embodiment of the present application, the value of N is different, and the first accuracy and the second accuracy are different. At this time, according to the different values of N, a situation in which the difference between the first accuracy and the second accuracy is less than the first accuracy change value is determined, and then a value of N is determined. Assuming that when N is 5, the difference between the first accuracy and the second accuracy is less than the first accuracy change value, the target sample production parameters that have been input at this time are these 5 sub-production data. If the difference between the first accuracy and the second accuracy is greater than or equal to the first accuracy change value regardless of the value of N, then each sub-production data in the target sample production parameter is data for model training. These sub-production data for model training are the initial sample production parameter data set.
[0055] In order to better implement the embodiment of the present application, in one embodiment of the present application, after determining that the input target sample production parameter is an initial sample production parameter data set for training the detection model, the method further includes:
[0056] The target sample production parameters in the initial sample production parameter data set are randomly eliminated to obtain an intermediate sample production parameter data set for training the detection model; the sample production parameters in the intermediate sample production parameter data set are input into the detection model to obtain a third accuracy rate; a second accuracy rate change value is obtained according to the first accuracy rate and the third accuracy rate; until it is determined that the second accuracy rate change value is less than a preset second accuracy rate change threshold, the intermediate sample production parameter data set is determined to be the actual sample production parameter data set for training the detection model.
[0057] In order to further reduce the computational complexity of the model, after the training data in the input model is determined, some data can be randomly eliminated.
[0058] Specifically, if the example of the above embodiment is still used for example, if N is 5, the sub-production data ranked in the top 5 are regarded as the initial sample production parameter data set for model training. At this time, the sub-production data can be randomly eliminated, including eliminating multiple at a time, or randomly eliminating one by one. For example: randomly eliminate one sub-production data from 5 sub-production data to obtain 4 sub-production data. At this time, the 4 sub-production data are used for model training to obtain a trained model. Specifically, the same as the above embodiment, including: inputting the 4 sub-production data corresponding to the sample panel in the training set into the model for training to obtain a trained detection model. At this time, when the test set is input into the detection model, the detection model can predict according to the 4 sub-production data corresponding to the sample panel of the test set to obtain a prediction accuracy, which is the third accuracy. Afterwards, the difference between the third accuracy and the first accuracy is determined to obtain a second accuracy change value. It should be noted that if the change in the second accuracy change value is greater than or equal to the second accuracy change threshold, it means that the currently eliminated feature has a greater impact on the model. At this time, it is necessary to consider the sub-production data under other elimination schemes until a situation is selected in which the second accuracy change value is determined to be less than the preset second accuracy change threshold, and the sub-production data corresponding to this situation is the actual production parameter data set used for training. For example: if the change value of randomly eliminating sub-production data A is greater than the second accuracy change threshold, sub-production data A cannot be eliminated; if the change value of eliminating sub-production data B is less than the second accuracy change threshold, sub-production data B can be eliminated. Afterwards, on the basis of eliminating sub-production data B, other sub-production data are eliminated until the next sub-production data to be eliminated cannot be found. It should be noted that if the change value corresponding to each elimination scheme is greater than or equal to the second accuracy change threshold, each sub-production data in the initial sample production parameter data set cannot be eliminated. At this time, the initial sample production parameter data set can be directly used as the actual sample production parameter data set.
[0059] After the actual sample production parameter data set is determined, the model is trained according to each sub-production data in the actual sample production parameter data set, so that a detection model with low computing requirements can be obtained, which reduces the computational burden of the computing device equipped with the detection model.
[0060] In order to better implement the embodiment of the present application, in one embodiment of the present application, after screening the sample production data to obtain the target sample production data for training the detection model, the method further includes:
[0061] The sample production data of the sample panel is sampled multiple times to obtain multiple sampling groups and non-sampling groups, and one sampling group corresponds to one non-sampling group; the detection model is trained according to the target sample production data of each sampling group to obtain multiple trained detection models; the target sample production data of each non-sampling group is used to predict the detection model to obtain the prediction results corresponding to each sampling group; the average prediction result of each prediction result is determined, and the detection model corresponding to the average detection result is regarded as the target detection model that has completed the training.
[0062] In order to make the prediction results of the detection model more representative, multiple trainings can be performed on the basis of the above embodiments to obtain multiple trained detection models, and then a detection model with relatively average prediction performance can be selected from the multiple trained detection models.
[0063] Specifically, after obtaining the sample panel, the sample panel can be randomly sampled multiple times, and the randomly sampled sample panel can be used as a training set, so that the model training set has general applicability, and thus the trained model also has a certain general applicability. For example: assuming that there are 10,000 sample panels, 65% of them are randomly sampled at this time, and multiple sampling groups including 6,500 sample panels are obtained. At this time, due to different sampling, the remaining 3,500 sample panels are also different. Then, for the target sample production data included in the sample panels of different 6,500 sampling panels, that is, the target sample production data obtained in the above embodiment, the detection model is trained to obtain a detection model that has been trained. For example: the first sampling obtains sampling group A and non-sampling group A; the second sampling obtains sampling group B and non-sampling group B; ...; the Nth sampling obtains sampling group X and non-sampling group X. At this time, the target sample production data of the sample panel in sampling group A is input into the detection model for training, and the trained detection model A can be obtained. At this time, the prediction rate is verified by non-sampling group A to obtain a prediction result A; similarly, the target sample production data of the sample panel in sampling group B is input into the detection model for training, and the trained detection model B can be obtained. At this time, the prediction rate is verified by non-sampling group B to obtain a prediction result B; ...; the target sample production data of the sample panel in sampling group X is input into the detection model for training, and the trained detection model X can be obtained. At this time, the prediction rate is verified by non-sampling group X to obtain a prediction result X. Finally, multiple prediction results are obtained. At this time, several results with better prediction results can be selected, and then an intermediate result is selected from the several results with better prediction results as the final result. Assume that the detection model corresponding to the selected intermediate result is detection model B, then the detection model B is the target detection model that has finally completed training.
[0064] In summary, the overall solution of the embodiment of the present application can be as follows:
[0065] Step S1: Collect sample panels and production data corresponding to the sample panels, where production data is the production equipment parameter data plus the corresponding measurement value labels. Afterwards, the data can be cleaned, including: filtering out parameter data with more than 70% null values, filtering out abnormal values, etc.; and filling the remaining null values with Lagrange interpolation.
[0066] Step S2: Feature engineering: Based on the consideration of interpretability, multiple sub-production data are constructed for each sample panel parameter. For example, 22 sub-production data are counted. Then, the data after feature engineering are screened at two levels and a data importance table is constructed for subsequent model training. The specific implementation is as follows:
[0067] Step S21:
[0068] For the original training and test data constructed in S1, the following methods are used: filtering low-variance features, features with low correlation (linear correlation & nonlinear correlation) with measurement values (<0.15), features with high correlation between features [only one is retained], multicollinearity (VIF inflation factor > 10) deduplication, etc.; in this step, features with correlation and measurement value labels > 0.15 are saved.
[0069] Step S22:
[0070] The data retained in S21 are trained and the consistency of the predicted data is checked: the specific method is to use a decision tree based on GridSearch, and the classification model of the training set and the test set is distinguished by training data by data; for data with high classification accuracy, it is considered that the distribution inconsistency between the training set and the test set is high and should be removed; otherwise, it is retained; for the retained data, based on the feature importance coefficient output in the decision tree, a data importance table Dict_1 is maintained, and data with poor consistency is deleted; table Dict_1 is stored in the form of key-value pairs.
[0071] Step S3: This scheme adopts a random forest regression model for modeling. At the same time, in order to alleviate the large difference in training and test data performance caused by modeling and training data, 10 outer random sampling training sets (65%) with different seeds and inner forward and backward double-round feature screening based on cross-validation (on 35% training sets) are used to construct the model during the training process, which improves the model's generalization and noise resistance.
[0072] The specific steps are as follows:
[0073] a_begin: The i-th outer layer randomly samples 65% of the training set as the data D_train_i for training the model;
[0074] b_begin: Use the remaining 35% of the data set as the validation set D_val to perform forward and backward feature screening on the inner layer;
[0075] Forward feature addition filter, initial feature list Feat_List, Feat_List.append(Dict_1.keys[:N])
[0076] Loop_forward:
[0077] 1. Based on Dict_1, take the top N features in terms of feature importance + random 50% data in D_val to train the model, use the remaining 50% data in D_val for 50% prediction, and get the mean r2_score score r1;
[0078] 2. Based on 1, the model is trained using the top N+1 features in importance and 50% of the random data in D_val, and the remaining 50% of the data in D_val is used for 50% prediction to obtain the mean r2_score score r2, which can be regarded as the prediction score of the first prediction accuracy;
[0079] If r2–r1>=set threshold:
[0080] r1=r2,N=N+1, continue to execute Loop_forward; Feat_list.append(Dict_2.keys[N])
[0081] If r2–r1<set threshold: stop forward feature addition and screening, return to Feat_List, at which point the feature data in Feat_List is the initial sample production parameter data set;
[0082] Backward feature elimination screening, given Feat_List, randomly shuffle Feat_List; initialize Feat_List_Train = Feat_List
[0083] Loop_backward, traverse all the features Feat in Feat_List:
[0084] 3. Based on the features in Feat_List + random 50% data in D_val, the model is trained, and the remaining 50% data in D_val is used for 5-fold prediction to obtain the mean r2_score score r1;
[0085] 4. Eliminate the feature Feat in Feat_List, train the model based on the features in Feat_List + random 50% data in D_val, use the remaining 50% data in D_val for 50% prediction, and get the mean r2_score score r2;
[0086] If r2-r1>=set threshold: continue to execute Loop_backward, r1=r2;
[0087] If r2-r1<set threshold:Feat_List_Train.remove(Feat);
[0088] b_end: Return to Feat_List_Train, at which point Feat_List_Train is the final actual sample production parameter data set.
[0089] a_end: Based on D_train_i and Fear_List_Train obtained by forward and backward feature screening, traverse the optimal model parameters (similar to GridSearchCV) to build a random forest regression model RF_Regressor_i, and update the importance score Dict_1 of the training feature to Dict_1_i; save the top 7 models and feature scores of RF_Regressor_i with better effects in the validation set (35%) in 10 times;
[0090] Step S4: Prediction
[0091] 5. Based on the 7 training models and corresponding feature scores saved in step S3, make predictions on the training set, and take the average prediction results of the 7 times as the prediction result of the final measurement value;
[0092] 6. Based on the 7 feature importance score dictionaries saved in step S3, the factor variables that mainly affect the measurement value are counted and sorted and output;
[0093] Step S5: Model update: This solution takes into account the problem that the model effect deteriorates over time due to problems such as equipment drift during the production process; after the newly produced data accumulates to a certain amount, steps S1 to S3 will be used again to update the model and redeploy it for use to improve the generalization of the model.
[0094] In order to better implement the data processing method in the embodiment of the present application, in addition to the data processing method, the embodiment of the present application also provides a data processing device, such as Figure 3 As shown, the device 300 includes:
[0095] An acquisition module 301 is used to acquire sample production data of a sample panel;
[0096] The screening module 302 is used to screen the sample production data to obtain target sample production data for training the detection model; the detection model is used to detect the panel quality of the sample panel.
[0097] The data processing device provided by the present application obtains the sample production data of the sample panel through the acquisition module 301, and then screens the sample production data through the screening module 302 to obtain part of the target sample production data. The detection model used to detect the display panel is trained with the target sample production data, so that when the model performs the prediction task later, the quality parameters of the display panel can be predicted based on only part of the production data, which reduces the calculation parameters of the model and further reduces the burden on the computing device.
[0098] In some embodiments of the present application, the screening module 302 is specifically used to:
[0099] Grouping the sample production data to obtain a training sample data set and a test sample data set;
[0100] Determine the type distribution data of each sample production parameter in the training sample data set and the type distribution data of each sample production parameter in the test sample data set through a preset classification model;
[0101] Determine the target sample production parameter for training the detection model according to the first type distribution data of each sample production parameter in the training sample data set and the second type distribution data of each sample production parameter in the test sample data set;
[0102] According to the target sample production parameters, target sample production data for training the detection model is obtained from the sample production data.
[0103] In some embodiments of the present application, the screening module 302 is further used to:
[0104] Compare the first type distribution data of each sample production parameter with the second type distribution data of each sample production parameter to determine the accuracy of each sample production parameter;
[0105] Eliminate sample production parameters whose accuracy is lower than the preset accuracy threshold to obtain target sample production parameters for training the detection model.
[0106] In some embodiments of the present application, the screening module 302 is further used to:
[0107] Input the target sample production parameters into a preset importance condition decision tree to obtain importance parameters of each target sample production parameter;
[0108] According to the importance parameter, each target sample production parameter is ranked in importance, and target sample production data including importance rankings for training the detection model is obtained.
[0109] In some embodiments of the present application, the screening module 302 is further used to:
[0110] According to the order of importance ranking, the top N target sample production parameters are input into the detection model to obtain the first accuracy of the top N target sample production parameters;
[0111] According to the order of importance ranking, the production parameters of the top N+1 target samples are input into the detection model to obtain the second accuracy of the production parameters of the top N+1 target samples;
[0112] Obtaining a first accuracy change value according to the first accuracy and the second accuracy;
[0113] If it is determined that the first accuracy change value is less than a preset first accuracy change threshold, it is determined that the input target sample production parameters are an initial sample production parameter data set for training the detection model.
[0114] In some embodiments of the present application, the screening module 302 is further used to:
[0115] Randomly remove the target sample production parameters in the initial sample production parameter data set to obtain an intermediate sample production parameter data set for training the detection model;
[0116] Inputting the sample production parameters in the intermediate sample production parameter data set into the detection model to obtain a third accuracy rate;
[0117] Obtaining a second accuracy change value according to the first accuracy and the third accuracy;
[0118] Until it is determined that the second accuracy change value is less than a preset second accuracy change threshold, the intermediate sample production parameter data set is determined to be the actual sample production parameter data set for training the detection model.
[0119] In some embodiments of the present application, the apparatus 300 further includes a training module 303, and the training module 303 is specifically used for:
[0120] Sampling the sample production data of the sample panel multiple times to obtain multiple sampling groups and non-sampling groups, one sampling group corresponds to one non-sampling group;
[0121] The detection model is trained according to the target sample production data of each sampling group to obtain multiple trained detection models;
[0122] The target sample production data of each non-sampling group is used to predict the detection model to obtain the prediction results corresponding to each sampling group;
[0123] The average prediction result of each prediction result is determined, and the detection model corresponding to the average detection result is regarded as the target detection model that has completed training.
[0124] The present application also provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of any data processing method in the present application. The terminal device integrates any data processing method provided in the present application. Figure 4 As shown, it shows a schematic diagram of the structure of the terminal device involved in the embodiment of the present application, specifically:
[0125] The terminal device may include one or more processing core processors 401, one or more computer-readable storage media memories 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will appreciate that Figure 4 The terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:
[0126] The processor 401 is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 executes various functions of the terminal device and processes data, thereby monitoring the terminal device as a whole. Optionally, the processor 401 may include one or more processing cores; the processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface and the application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.
[0127] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.
[0128] The terminal device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption management and other functions through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.
[0129] The terminal device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
[0130] Although not shown, the terminal device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the terminal device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, such as:
[0131] Obtain sample production data for sample panels;
[0132] The sample production data is screened to obtain target sample production data for training a detection model; the detection model is used to detect the panel quality of the sample panel.
[0133] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0134] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any data processing method provided in the embodiment of the present application. For example, the computer program is loaded by a processor to execute the following steps:
[0135] Obtain sample production data for sample panels;
[0136] The sample production data is screened to obtain target sample production data for training a detection model; the detection model is used to detect the panel quality of the sample panel.
[0137] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.
[0138] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.
[0139] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0140] The above is a detailed introduction to a data processing method and device provided in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A data processing method, characterized in that: The method comprises: Obtain sample production data for sample panels; The sample production data is screened to obtain target sample production data for training a detection model; the detection model is used to detect the panel quality of the sample panel.
2. The data processing method according to claim 1, characterized in that: The sample production data includes a plurality of sample production parameters, and the screening of the sample production data to obtain target sample production data for training a detection model includes: Grouping the sample production data to obtain a training sample data set and a test sample data set; Determine the type distribution data of each of the sample production parameters in the training sample data set and the type distribution data of each of the sample production parameters in the test sample data set through a preset classification model; Determining target sample production parameters for training the detection model according to the first type distribution data of each sample production parameter in the training sample data set and the second type distribution data of each sample production parameter in the test sample data set; According to the target sample production parameters, target sample production data for training a detection model is obtained from the sample production data.
3. The data processing method according to claim 2, characterized in that: Determining the target sample production parameter for training the detection model according to the first type distribution data of each sample production parameter in the training sample data set and the second type distribution data of each sample production parameter in the test sample data set includes: Comparing the first type distribution data of each of the sample production parameters with the second type distribution data of each of the sample production parameters to determine the accuracy of each of the sample production parameters; Sample production parameters whose accuracy is lower than a preset accuracy threshold are eliminated to obtain target sample production parameters for training the detection model.
4. The data processing method according to claim 2, characterized in that: The step of obtaining target sample production data for training a detection model from the sample production data according to the target sample production parameters includes: Inputting the target sample production parameters into a preset importance condition decision tree to obtain importance parameters of each target sample production parameter; According to the importance parameters, each of the target sample production parameters is ranked in importance to obtain target sample production data for training the detection model and including the importance ranking.
5. The data processing method according to claim 4, characterized in that: The step of ranking the importance of each target sample production parameter according to the importance parameter to obtain target sample production data for training the detection model and including the importance ranking includes: According to the order of the importance ranking, the top N target sample production parameters are input into the detection model to obtain the first accuracy of the top N target sample production parameters; According to the order of the importance ranking, the top N+1 target sample production parameters are input into the detection model to obtain the second accuracy of the top N+1 target sample production parameters; Obtaining a first accuracy change value according to the first accuracy and the second accuracy; If it is determined that the first accuracy change value is less than a preset first accuracy change threshold, it is determined that the input target sample production parameters are an initial sample production parameter data set for training the detection model.
6. The data processing method according to claim 5, characterized in that: After determining that the input target sample production parameters are the initial sample production parameter data set for training the detection model, the method further includes: Randomly removing target sample production parameters from the initial sample production parameter data set to obtain an intermediate sample production parameter data set for training a detection model; Inputting the sample production parameters in the intermediate sample production parameter data set into the detection model to obtain a third accuracy rate; Obtaining a second accuracy change value according to the first accuracy and the third accuracy; Until it is determined that the second accuracy change value is less than a preset second accuracy change threshold, the intermediate sample production parameter data set is determined to be an actual sample production parameter data set for training the detection model.
7. The data processing method according to claim 1, characterized in that: After screening the sample production data to obtain target sample production data for training the detection model, the method further includes: Sampling the sample production data of the sample panel multiple times to obtain multiple sampling groups and non-sampling groups, one sampling group corresponds to one non-sampling group; The detection model is trained according to the target sample production data of each sampling group to obtain a plurality of trained detection models; Using the target sample production data of each non-sampling group to predict the detection model, and obtaining the prediction results corresponding to each sampling group; An average prediction result of each of the prediction results is determined, and the detection model corresponding to the average detection result is regarded as the target detection model that has completed training.
8. A data processing device, characterized in that: The device comprises: An acquisition module, used for acquiring sample production data of a sample panel; The screening module is used to screen the sample production data to obtain target sample production data for training a detection model; the detection model is used to detect the panel quality of the sample panel.
9. A terminal device, characterized in that: The terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps in the data processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in the data processing method according to any one of claims 1 to 7.