A data cleaning method, device, equipment and storage medium

Abstract

The present disclosure relates to a data cleaning method, device, equipment and storage medium. The data cleaning method comprises: collecting component picture data set, wherein the component picture data set comprises a plurality of component pictures, each component picture in the plurality of component pictures is labeled with a first label, and the first label is used to label whether the component is mounted on a printed circuit board correctly or has a mounting defect; determining the confidence of the first label, and based on the confidence of the first label, predicting the label of each component picture in the component picture data set to obtain a second label; and cleaning the first label based on the second label. The present disclosure can realize efficient, low-cost, high-accuracy, fast-discrimination, short-rejudgment-time and reusable data cleaning.

Description

Technical Field

[0001] This disclosure relates to the field of industrial digital intelligence, and in particular to a data cleaning method, apparatus, equipment and storage medium. Background Technology

[0002] In related technologies, defect detection of PCB components is an indispensable step in the production and soldering process of PCB (Printed Circuit Board) components during SMT (Surface Mount Technology) manufacturing.

[0003] In related technologies, the problem of incorrect image data labeling for PCB board components mainly relies on manual judgment. Multiple experts in the relevant field re-evaluate the labels provided by AOI (Automated Optical Inspection) equipment, synthesizing the results of multiple experts to identify erroneous label samples. After data cleaning, a high-quality dataset is obtained. This method has many problems: low judgment efficiency, significant susceptibility to human error in detection results, poor consistency in judgment, waste of significant manpower, and low detection accuracy. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a data cleaning method, apparatus, device and storage medium.

[0005] According to a first aspect of the present disclosure, a data cleaning method is provided, the method comprising:

[0006] A component image dataset is collected, comprising multiple component images, each labeled with a first label. The first label indicates whether the component is properly mounted on the printed circuit board or has mounting defects. The confidence level of the first label is determined, and based on the confidence level of the first label, a label for each component image in the component image dataset is predicted to obtain a second label. The first label is then cleaned based on the second label.

[0007] In one embodiment, determining the confidence level of the first label includes: inputting the component image dataset into a prediction model, classifying the component image dataset to obtain qualified component image data and defective component image data; performing cross-validation on the qualified component image data and defective component image data respectively, and determining the category probability of each component image data corresponding to its respective category, and determining the category probability as the confidence level of the first label.

[0008] In one embodiment, the prediction model is trained as follows: the component image data sample set is divided into k mutually exclusive subsets of similar size; based on the mutually exclusive subsets, k-1 subsets are selected as training sets, and the remaining subsets are selected as test sets; a neural network is built as a base model, and the base model is trained based on the training set and the test set to obtain the prediction model.

[0009] In one implementation, predicting the label of each component image in the component image dataset based on the confidence level of the first label to obtain a second label includes: if the confidence level of the first label of the target component image is greater than a confidence threshold, then determining the second label of the target component image as the first label; if the confidence level of the first label of the target component image is less than or equal to the confidence threshold, then determining the second label as the label of the component image corresponding to the highest category probability in the category to which the target component image belongs; wherein, the target component image refers to each component image in the component image dataset, and the confidence threshold is the average of the category probabilities of all components in the category to which the target component belongs.

[0010] In one embodiment, the step of cleaning the first label based on the second label includes: if the second label and the first label are different, then determining the label quality value of the second label; if the label quality value of the second label is greater than a threshold, then determining that the first label is accurately labeled and retaining the first label; if the label quality value of the second label is less than or equal to the threshold, then determining that the first label is incorrectly labeled and cleaning the first label.

[0011] According to a second aspect of the present disclosure, a data cleaning apparatus is provided, comprising:

[0012] The acquisition unit is used to acquire a component image dataset, which includes multiple component images. Each component image is labeled with a first label, wherein the first label is used to indicate whether the component is properly mounted on the printed circuit board or has mounting defects.

[0013] The prediction unit determines the confidence level of the first label and, based on the confidence level of the first label, predicts the label of each component image in the component image dataset to obtain the second label.

[0014] A cleaning unit is used to clean the first label based on the second label.

[0015] In one embodiment, the prediction unit determines the confidence level of the first label in the following manner: the component image dataset is input into the prediction model, and the component image dataset is classified to obtain qualified component image data and defective component image data; cross-validation is performed on the qualified component image data and the defective component image data respectively, and the category probability of the component image data corresponding to their respective categories is determined, and the category probability is determined as the confidence level of the first label.

[0016] In one embodiment, the prediction unit is trained to obtain the prediction model in the following manner: the component image data sample set is divided into k mutually exclusive subsets of similar size; based on the mutually exclusive subsets, k-1 subsets are selected as training sets, and the remaining subsets are used as test sets; a neural network is built as a base model, and the base model is trained based on the training set and the test set to obtain the prediction model.

[0017] In one implementation, the prediction unit predicts the label of each component image in the component image dataset based on the confidence level of the first label, to obtain a second label: if the confidence level of the first label of the target component image is greater than a confidence threshold, then the second label of the target component is determined to be the first label; if the confidence level of the first label of the target component image is less than or equal to the confidence threshold, then the second label is determined to be the label of the component image corresponding to the highest category probability in the category to which the target component image belongs; wherein, the target component image is each component image in the component image dataset, and the confidence threshold is the average of the category probabilities of all components in the category to which the target component belongs.

[0018] In one embodiment, the cleaning unit cleans the first label based on the second label in the following manner: if the second label and the first label are different, the label quality value of the second label is determined; if the label quality value of the second label is greater than a threshold, the first label is determined to be accurate and the first label is retained; if the label quality value of the second label is less than or equal to the threshold, the first label is determined to be incorrect and the first label is cleaned out.

[0019] According to a third aspect of the present disclosure, a data cleaning apparatus is provided, comprising:

[0020] Processor; memory used to store processor-executable instructions;

[0021] The processor is configured to execute the data cleaning method described in the first aspect or any embodiment of the first aspect.

[0022] According to a fourth aspect of the present disclosure, a storage medium is provided, characterized in that the storage medium stores instructions that, when executed by a processor of a terminal, enable the terminal to perform the data cleaning method described in the first aspect or any embodiment of the first aspect.

[0023] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: by calculating confidence levels and performing data cleaning on the first label based on the second label, a higher quality dataset can be obtained after data cleaning. This disclosure enables data cleaning that is efficient, low-cost, highly accurate, fast in judgment, short in re-judgment time, and can be widely reused.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0026] Figure 1 This is a flowchart illustrating a data cleaning method according to an exemplary embodiment.

[0027] Figure 2 This is a flowchart illustrating a method for determining the confidence level of a first label according to an exemplary embodiment.

[0028] Figure 3 This is a flowchart illustrating a method for training a prediction model according to an exemplary embodiment.

[0029] Figure 4 This is a flowchart illustrating a method for predicting labels according to an exemplary embodiment.

[0030] Figure 5 This is a flowchart illustrating a data cleaning method according to an exemplary embodiment.

[0031] Figure 6 This is a flowchart illustrating a data cleaning method according to an exemplary embodiment.

[0032] Figure 7 This is a block diagram illustrating a data cleaning apparatus according to an exemplary embodiment.

[0033] Figure 8 This is a block diagram illustrating a data cleaning apparatus according to an exemplary embodiment.

[0034] Figure 9 This is a block diagram illustrating a data cleaning apparatus according to an exemplary embodiment. Detailed Implementation

[0035] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0036] The method provided in this disclosure can be applied to PCB board data cleaning scenarios based on confidence learning during SMT production.

[0037] SMT is a technology and process in the electronics assembly industry. Soldering materials are printed onto the pads of a bare PCB board. Electronic components are then placed onto the pads of the PCB board using a pick-and-place machine. Finally, the PCB board is sent to a reflow oven for soldering. In short, SMT assembly involves several processes to mount electronic components onto a bare PCB board.

[0038] AOI (Automated Optical Inspection) equipment is an automated optical inspection device based on optical principles that automatically detects defects in the placement of PCB components during the SMT (Surface Mount Technology) production process. This equipment can be placed before (before) reflow soldering and after (after) reflow soldering, using captured component images to inspect the placement of PCB components.

[0039] PCB component data cleaning refers to the process of identifying the component placement status on the PCB board by using AOI equipment to collect component image data. The identified data labels may contain errors, so the data is re-identified, and the incorrectly identified image data labels are corrected to obtain a higher quality component image dataset.

[0040] There are currently two methods for cleaning PCB component data. One method involves using additional equipment to identify the collected PCB component image data. This method can be further divided into electrical inspection and optical inspection based on the detection principle. Electrical inspection requires corresponding pin-type machine tools, and different machine tools are needed for different circuit boards, which cannot meet the industry's demand for factories with rapid line changeover capabilities and rapid product updates. The other method is manual judgment. During the production process, the problem of incorrect labeling of PCB component image data is mainly solved by manual judgment. AOI equipment saves all inspected component image data in the cloud. Relevant technicians download the image data from the cloud to their local machines and then classify them according to different component types. For all image data of each type of component, experts review the data sequentially, classifying images that the experts deem correctly labeled as incorrectly labeled. In practice, multiple experts may need to review the same set of component image data. Combining the results of multiple experts is necessary to more accurately identify incorrectly labeled images and, after data cleaning, obtain a high-quality dataset. This method relies on experts in the relevant field. These experts use their professional knowledge to judge the PCB component image data collected by the AOI equipment. The judgment results are greatly affected by human intervention. This method is not only inefficient and costly, but also has low judgment accuracy and poor consistency.

[0041] In view of this, this disclosure provides a data cleaning method. An AOI intelligent re-judgment algorithm model is trained based on deep learning and confidence learning. This model is then used to clean PCB component image data, resulting in a higher-quality image dataset. Using an algorithm model to replace manual re-judgment not only improves efficiency and reduces labor costs, but also offers high accuracy, speed, consistency, and precision, making it well-suited to industry needs.

[0042] Figure 1 This is a flowchart illustrating a data cleaning method according to an exemplary embodiment, such as... Figure 1 As shown, the data cleaning method includes the following steps:

[0043] In step S11, a component image dataset is collected. The component image dataset includes multiple component images. Each component image is labeled with a first label, which is used to indicate whether the component is properly mounted on the printed circuit board or has mounting defects.

[0044] In this embodiment, an automated optical inspection device can capture and collect component image data to obtain a component image dataset. Multiple component images can be capacitor images, resistor images, etc. The first label can be a label obtained through manual review of the collected component image dataset. Corresponding to whether the mounting is qualified or defective, the first label can be divided into a qualified mounting label and a defective mounting label.

[0045] In this embodiment of the disclosure, the component image data is classified to facilitate separate processing of different categories, which can achieve data cleaning with high discrimination accuracy.

[0046] In step S12, the confidence level of the first label is determined, and based on the confidence level of the first label, the label of each component image in the component image dataset is predicted to obtain the second label.

[0047] In this embodiment of the disclosure, the confidence level of the first label reflects the output probability value of the first label, and the output probability value of the first label is the probability value of the first label belonging to the category with the highest probability.

[0048] In this embodiment of the disclosure, the second label is predicted by the confidence level of the first label, thereby judging the first label to achieve data cleaning with high judgment accuracy.

[0049] In step S13, the first label is cleaned based on the second label.

[0050] In this embodiment of the disclosure, the labels can be divided into different categories, such as qualified categories and defective categories. A second label is compared with a first label; if the second label belongs to a different category than the first label, the first label is cleaned based on the second label.

[0051] In this embodiment of the disclosure, the second label, which is predicted based on the confidence level of the first label, is used to determine whether a labeling error has occurred in the first label, thereby achieving data cleaning with high discrimination accuracy.

[0052] Figure 2 This is a flowchart illustrating a method for determining the confidence level of a first label according to an exemplary embodiment, such as... Figure 2 As shown, it includes the following steps:

[0053] In step S21, the component image dataset is input into the prediction model, and the component image dataset is classified to obtain qualified component image data and defective component image data.

[0054] In this embodiment of the disclosure, the prediction model can be trained using a deep learning method, based on a neural network as the base model, and classify the component image dataset according to the features learned by the neural network.

[0055] In this embodiment of the disclosure, a neural network is used as the base model and trained multiple times. During this process, the neural network learns to recognize and classify images through its self-learning function.

[0056] In this embodiment of the disclosure, the components are classified by a prediction model to obtain an accurately classified target dataset, which can achieve data cleaning with high discrimination accuracy.

[0057] In step S22, cross-validation is performed on the qualified component image data and the defective component image data respectively, and the category probability of each component image data to its respective category is determined. The category probability is then used as the confidence level of the first label.

[0058] In this embodiment of the disclosure, the first label confidence level reflects the category probability of the component image data to which it belongs. Cross-validation of qualified component image data yields the category probability of qualified, and cross-validation of defective component image data yields the category probability of defective.

[0059] In this embodiment of the disclosure, after classifying the component image data, cross-validation is performed to obtain cleaned component image data with good discrimination effect and high discrimination accuracy.

[0060] Figure 3 This is a flowchart illustrating a method for training a prediction model according to an exemplary embodiment, such as... Figure 3 As shown, it includes the following steps:

[0061] In step S31, the component image data sample set is divided into k mutually exclusive subsets of similar size.

[0062] In this embodiment of the disclosure, the k mutually exclusive subsets of similar size obtained by partitioning are further partitioned, resulting in more accurate model training results.

[0063] In step S32, based on mutually exclusive subsets, k-1 subsets are selected as the training set, and the remaining subsets are selected as the test set.

[0064] In this embodiment of the disclosure, the training set is known and consists of data samples used for model fitting, i.e., the training sample set. This set can be used to train the parameters in the neural network, adjust network weights, and perform gradient descent on the training error during training. The test set is unknown and is used to evaluate the generalization ability of the final model. It can be used to assess the model's performance and determine its actual predictive ability.

[0065] In this embodiment, the component image data sample set is divided into a training set and a test set. These are used to train the base model, resulting in a model with good classification performance, thus making the classification of component image data more accurate.

[0066] In step S33, a neural network is built as a base model, and the base model is trained based on the training set and the test set to obtain the prediction model.

[0067] In this embodiment of the disclosure, the neural network can classify images based on the features learned by the network, and the model trained by the base model can be used to classify the input component image dataset.

[0068] In this embodiment of the disclosure, the model built on the neural network may include weights and thresholds. The weights and thresholds can be obtained through training. When the samples change, the model can learn to adjust the weights to adapt to the new sample conditions.

[0069] In this embodiment of the disclosure, a neural network is used as the base model for training, which can yield a model with good learning and classification performance, making the model more accurate.

[0070] In this embodiment, the specific implementation process of the method for predicting the second label is described.

[0071] Figure 4 This is a flowchart illustrating a method for predicting labels according to an exemplary embodiment, such as... Figure 4 As shown, it includes the following steps:

[0072] In step S41, if the confidence level of the first label of the target component image is greater than the confidence level threshold, then the second label of the target component image is determined to be the first label.

[0073] In this embodiment of the disclosure, the confidence level of the first label is the probability of the category to which the target component image belongs, and the confidence level threshold is the average probability of the category to which the target component image belongs. If the probability of the category to which the target component image belongs is greater than the average probability of the category to which the target component image belongs, then the category to which the second label of the target component image belongs is the same as the category to which the first label belongs, that is, the second label of the target component image is the first label.

[0074] In this embodiment of the disclosure, the second label is determined by the confidence level of the first label being greater than the confidence level threshold, and the accuracy of the category to which the second label belongs is high.

[0075] In step S42, if the confidence level of the first label of the target component image is less than or equal to the confidence threshold, then the second label is determined to be the label of the component image corresponding to the highest category probability in the category to which the target component image belongs. Here, the target component image refers to all component images in the component image dataset, and the confidence threshold is the average of the category probabilities of all components in the category to which the target component belongs.

[0076] In this embodiment of the disclosure, if the confidence level of the first label of the target component image is less than or equal to the confidence level threshold, that is, the probability of the target component image belonging to a category is less than or equal to the average probability of the target component image belonging to a category, then the category to which the second label of the target component image belongs is the category with the highest probability value.

[0077] In this embodiment of the disclosure, if the confidence level of the first label is less than or equal to the confidence level threshold, the category to which the second label belongs is determined by the maximum probability category, and the obtained second label category has high accuracy and high reliability.

[0078] Figure 5 This is a flowchart illustrating a data cleaning method according to an exemplary embodiment, such as... Figure 5 As shown, it includes the following steps:

[0079] In step S51, if the second label is different from the first label, the label quality value of the second label is determined.

[0080] In this embodiment of the disclosure, if the second label and the first label are different, it is determined that the first label may be incorrect. It is necessary to further confirm whether the first label is incorrect by determining the label quality value of the second label.

[0081] In step S52, if the label quality value of the second label is greater than the threshold, then the first label is determined to be accurate and the first label is retained.

[0082] In this embodiment, the tag quality value of the second tag is located in the range [0,1], reflecting the tag quality status of the second tag. The threshold is the allowable threshold for tag quality services, which can be obtained through empirical or experimental values.

[0083] In this embodiment of the disclosure, if the tag quality value of the second tag is greater than the threshold, that is, the tag quality value is greater than the tag quality service allowable threshold, it indicates that the tag quality is good, it is determined that no tag error has occurred, and the first tag is retained.

[0084] In this embodiment of the disclosure, if the label quality value of the second label is greater than the threshold, the first label is further determined to be retained. The accuracy of the judgment is high by comparing the label quality value with the threshold.

[0085] In step S53, if the label quality value of the second label is less than or equal to the threshold, then the first label is determined to be incorrectly labeled, and the first label is removed.

[0086] In this embodiment of the disclosure, if the tag quality value of the second tag is less than or equal to the threshold, that is, the tag quality value is less than or equal to the tag quality service allowable threshold, it indicates that the tag quality is poor, and it is determined that a tag error has occurred, and the first tag is cleaned up.

[0087] In this embodiment of the disclosure, if the label quality value of the second label is less than or equal to a threshold, it is determined that the label quality is poor, and the first label is further determined to be cleaned up, which can achieve high discrimination accuracy and high discrimination precision.

[0088] Figure 6 This is a flowchart illustrating a data cleaning method according to an exemplary embodiment, such as... Figure 6 As shown, it includes the following steps:

[0089] In step S61, data acquisition is performed.

[0090] In this embodiment of the disclosure, data acquisition can be performed using AOI equipment to obtain component image data. When the AOI equipment performs automatic inspection, it automatically scans the PCB with a camera, acquires images, compares the tested solder joints with qualified parameters in the database, and after image processing, detects component surface mount defects on the PCB and marks the defects on a display or with an automatic marker.

[0091] In step S62, input the raw data.

[0092] In this embodiment of the disclosure, the original data may be data containing original labels obtained after manually reviewing the collected component image data. The original labels may contain errors in the component image data labels, requiring further confirmation to determine if such errors exist.

[0093] In step S63, the dataset is divided.

[0094] In this embodiment of the disclosure, the component image data sample set is divided into k mutually exclusive subsets of similar size.

[0095] In a mutually exclusive subset, select k-1 subsets as the training set and the remaining subset as the test set. The training set is used to train the parameters of the neural network. The test set is unknown and is used to evaluate the generalization ability of the final model, thus assessing the model's performance.

[0096] In step S64, the base model is trained.

[0097] In this embodiment, a neural network is used as the base model, which is trained to obtain a model with good classification performance. The neural network can self-learn; through training and learning, it can classify component image data based on learned features.

[0098] In step S65, the target dataset is input.

[0099] In this embodiment of the disclosure, the original dataset after manual review is used as the target dataset. The target dataset contains multiple types of components, each of which has two types: qualified and defective. The target dataset is classified using a model with good classification performance obtained by training the base model. The classification can be either qualified or defective.

[0100] In step S66, cross-validation is performed.

[0101] In this embodiment, cross-validation first requires cross-validating the dataset to calculate the probability P[i][j] of the i-th sample in the j-th class; then, the average probability t[j] of each manually labeled class j is calculated as the confidence threshold; finally, for sample i, its true label y* is the maximum probability argmax among the j classes. j P[i]P[j], and P[i][j]>t[j].

[0102] In step S67, the counting matrix is ​​calculated.

[0103] In this embodiment of the disclosure, the formula for calculating the counting matrix is ​​as follows: the rows of the confidence joint counting matrix represent the number of predicted label samples, and the columns represent the number of real label samples.

[0104] In step S68, the counting matrix is ​​calibrated.

[0105] In this embodiment of the disclosure, the purpose of the calibration technique matrix is ​​to make the total count the same as the total number of manually labeled samples.

[0106] In step S69, the joint distribution is estimated.

[0107] In this embodiment of the disclosure, the joint distribution of noise labels and true labels is obtained based on the original label distribution of the target dataset, and a joint confidence count matrix C is constructed. The formula for constructing the matrix is ​​as follows:

[0108]

[0109] In matrix C, each cell C[i][j] represents the number of samples with true label j and predicted label i. Represents the defined noise label, a discrete random variable. This refers to noisy labels. y* is the defined true label, a discrete random variable y*∈[m], representing an unknown, true, and undamaged label. m represents the total number of categories.

[0110]

[0111] in, Represents the construction matrix The corresponding determinant, This indicates that when the probability of the i-th sample in the j-th category is greater than the average probability t[j] in the j-th category, This represents the category with the highest probability among j categories as the true label.

[0112] In step S610, it is determined whether the value is greater than or equal to the threshold.

[0113] In this embodiment of the disclosure, it is determined whether the probability P[i][j] of the i-th sample in the j-th category is greater than or equal to the average probability t[j] of the manually labeled category j. The category to which the predicted label belongs is further determined by comparing the category probability with the category average probability.

[0114] In step S611, if the judgment result is yes, the predicted label is obtained.

[0115] In this embodiment of the disclosure, if the judgment result is yes, that is, the category probability is greater than or equal to the average category probability, then the category to which the predicted label belongs at this time is obtained, and the predicted label is obtained.

[0116] In step S612, if the judgment result is negative, the category with the highest probability value is predicted to belong to the label.

[0117] In this embodiment of the disclosure, if the judgment result is negative, that is, the category probability is less than the average category probability, then the predicted label belongs to the category with the highest probability value, and the predicted label is obtained.

[0118] In step S613, label correction is performed.

[0119] In this embodiment of the disclosure, if the predicted label belongs to the same category as the original label, it is determined that no labeling error has occurred, and the original label is retained.

[0120] In this embodiment, a predicted category for each sample can be obtained. If the predicted category label differs from the original label, a labeling error may occur, requiring further confirmation. For each sample's predicted label, confidence learning also outputs label quality data. The label quality value corresponding to this data is located within the range [0,1]. When the label quality value is lower than the business-allowed threshold, it is determined that the sample has a labeling error, and the original label is removed. After removing erroneous labeled samples, a dataset with higher data quality is obtained. The base model can be trained on the cleaned dataset to improve the model's classification accuracy. Confidence learning continues with the collected data.

[0121] The data cleaning method provided in this disclosure has been verified on a PCB board image dataset from an AOI device. This dataset contains 14 categories of components, each with two types of labels: qualified and defective. The dataset has low label quality, contains a lot of noisy data, and has many incorrectly labeled samples—that is, qualified samples are labeled as defective, and corresponding defects are labeled as qualified. Taking a surface-mount capacitor with a package size of 0402 as an example, for the AOI data of this type of component, this patented method can identify all samples with incorrectly labeled qualified / defective labels, achieving a 100% label correction accuracy. Compared with manual review methods, the time required by this disclosure to find all incorrectly labeled samples is less than that of manual label correction methods.

[0122] This disclosure utilizes algorithms for label correction, reducing the workload of expert review and eliminating the need for multiple technical experts, thereby lowering labor costs. Compared to related technologies that rely on the expertise and experience of multiple technical experts, this method is more efficient and has lower labor costs.

[0123] This disclosure provides an AOI intelligent re-judgment algorithm model based on confidence learning. This algorithm model replaces manual re-judgment and diagnosis, improving re-judgment efficiency, reducing labor costs, and offering high accuracy, speed, consistency, and precision. Furthermore, given the high false alarm rate of AOI equipment, this disclosure reduces re-judgment time and increases overall production capacity during production line operation.

[0124] This disclosed embodiment can also be conveniently applied in the production process. If a production line needs to be changed or new requirements arise, the algorithm model can be adjusted accordingly without spending excessive time, effectively adapting to industry needs. Furthermore, it provides a referential method for image data labeling and error correction, promoting its reuse and enhancing the core competitiveness of smart factories.

[0125] Based on the same concept, embodiments of this disclosure also provide a data cleaning apparatus.

[0126] It is understood that the data cleaning apparatus provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0127] Figure 7 This is a block diagram of a data cleaning apparatus according to an exemplary embodiment. (Refer to...) Figure 7 The device 100 includes a data acquisition unit 101, a prediction unit 102, and a cleaning unit 103.

[0128] The acquisition unit 101 is used to acquire a component image dataset. The component image dataset includes multiple component images. Each component image is labeled with a first label, which is used to indicate whether the component is properly mounted on the printed circuit board or has mounting defects.

[0129] In one embodiment, the acquisition unit 101 can be an automatic optical inspection device, capable of capturing component image data and automatically detecting surface mount defects of PCB components based on the captured component image data. The automatic optical inspection device can be placed before (before reflow soldering) and after (after reflow soldering) of the PCB components.

[0130] In one implementation, images captured by the furnace-front AOI equipment are used as the dataset for method verification; however, the images can also be applied after appropriate modifications to the furnace-back AOI equipment.

[0131] The prediction unit 102 is used to determine the confidence level of the first label and, based on the confidence level of the first label, predict the label of each component image in the component image dataset to obtain the second label.

[0132] In one implementation, the prediction unit 102 determines the confidence level of the first label as follows: The component image dataset is input into the prediction model, and the dataset is classified into qualified component image data and defective component image data. Cross-validation is performed on the qualified and defective component image data respectively, and the category probability of each component image data corresponding to its respective category is determined. This category probability is then used as the confidence level of the first label.

[0133] In one implementation, the prediction unit 102 trains the prediction model as follows: The component image data sample set is divided into k mutually exclusive subsets of similar size. Based on these mutually exclusive subsets, k-1 subsets are selected as the training set, and the remaining subsets are used as the test set, thus dividing the component image data sample set into a training set and a test set. A neural network is built as the base model, and the base model is trained based on the training set and the test set to obtain the prediction model.

[0134] In one implementation, the prediction unit 102 obtains a second label based on the confidence level of a first label and the label of each component image in the component image dataset: If the confidence level of the first label of the target component image is greater than a confidence threshold, then the second label of the target component is determined to be the first label. If the confidence level of the first label of the target component image is less than or equal to the confidence threshold, then the second label is determined to be the label of the component image corresponding to the highest category probability in the category to which the target component image belongs. Wherein, the target component image refers to each component image in the component image dataset, and the confidence threshold is the average of the category probabilities of all components in the category to which the target component belongs.

[0135] In one implementation, the first label can be a label obtained by manually reviewing the target dataset collected by the AOI device. This label may contain labels with errors, such as a qualified label being judged as a defective label, or a defective label being judged as a qualified label.

[0136] The cleaning unit 103 is used to clean the first label based on the second label.

[0137] In one embodiment, the cleaning unit 103 cleans the first label based on the second label in the following manner: if the second label and the first label are different, the label quality value of the second label is determined. If the label quality value of the second label is greater than a threshold, the first label is determined to be accurate and the first label is retained. If the label quality value of the second label is less than or equal to the threshold, the first label is determined to be incorrect and the first label is cleaned up.

[0138] Regarding the device in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0139] Figure 8 This is a block diagram illustrating an apparatus 200 for data cleaning according to an exemplary embodiment. For example, apparatus 200 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0140] Reference Figure 8 The device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.

[0141] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

[0142] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0143] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.

[0144] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0145] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

[0146] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0147] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0148] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0149] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0150] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0151] Figure 9 This is a block diagram illustrating an apparatus 300 for data cleaning according to an exemplary embodiment. For example, apparatus 300 may be provided as a server. (Refer to...) Figure 9 The device 300 includes a processing component 322, which further includes one or more processors, and memory resources represented by memory 332 for storing instructions, such as application programs, that can be executed by the processing component 322. The application programs stored in memory 332 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 322 is configured to execute instructions to perform the methods described above…

[0152] Device 300 may also include a power supply component 326 configured to perform power management of device 300, a wired or wireless network interface 350 configured to connect device 300 to a network, and an input / output (I / O) interface 352. Device 300 may operate on an operating system stored in memory 332, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0153] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0154] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0155] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0156] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0157] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A data cleaning method, characterized in that, include: A component image dataset is collected, which includes multiple component images. Each component image is labeled with a first label. The first label is used to indicate whether the component is properly mounted on the printed circuit board or has mounting defects. The first label is a label obtained by manually reviewing the component image dataset. Determine the confidence level of the first label, and based on the confidence level of the first label, predict the label of each component image in the component image dataset to obtain the second label; The first label is cleaned based on the second label; The cleaning of the first label based on the second label includes: If the second label is different from the first label, then the label quality value of the second label is determined. The label quality value of the second label is used to confirm whether the first label has a label error. If the label quality value of the second label is greater than the threshold, then the first label is determined to be accurate and the first label is retained. If the label quality value of the second label is less than or equal to the threshold, then the first label is determined to be incorrect and the first label is removed.

2. The data cleaning method according to claim 1, characterized in that, Determining the confidence level of the first label includes: The component image dataset is input into the prediction model, and the component image dataset is classified to obtain qualified component image data and defective component image data. Cross-validation is performed on the qualified component image data and the defective component image data respectively, and the category probability of each component image data corresponding to its respective category is determined. The category probability is determined as the confidence level of the first label.

3. The data cleaning method according to claim 2, characterized in that, The prediction model was trained in the following manner: The component image data sample set is divided into k mutually exclusive subsets of similar size; Based on the mutually exclusive subsets, select k-1 subsets as the training set, and the remaining subsets as the test set; A neural network is constructed as a base model, and the base model is trained based on the training set and the test set to obtain the prediction model.

4. The data cleaning method according to claim 2 or 3, characterized in that, The second label is obtained by predicting the label of each component image in the component image dataset based on the confidence score of the first label, including: If the confidence level of the first label of the target component image is greater than the confidence level threshold, then the second label of the target component image is determined to be the first label; If the confidence level of the first label of the target component image is less than or equal to the confidence level threshold, then the second label is determined to be the label of the component image corresponding to the highest category probability in the category to which the target component image belongs; Wherein, the target component image is each component image in the component image dataset, and the confidence threshold is the average probability of all components in the category to which the target component belongs.

5. A data cleaning apparatus, characterized in that, include: The acquisition unit is used to acquire a component image dataset, which includes multiple component images. Each component image is labeled with a first label. The first label is used to indicate whether the component is properly mounted on the printed circuit board or has mounting defects. The first label is a label obtained by manually reviewing the component image dataset. A prediction unit is used to determine the confidence level of the first label and, based on the confidence level of the first label, predict the label of each component image in the component image dataset to obtain a second label. A cleaning unit is used to clean the first label based on the second label; The cleaning unit cleans the first label based on the second label in the following manner: If the second label is different from the first label, then the label quality value of the second label is determined. The label quality value of the second label is used to confirm whether the first label has a label error. If the label quality value of the second label is greater than the threshold, then the first label is determined to be accurate and the first label is retained. If the label quality value of the second label is less than or equal to the threshold, then the first label is determined to be incorrect and the first label is removed.

6. The data cleaning apparatus according to claim 5, characterized in that, The prediction unit determines the confidence level of the first label in the following manner: The component image dataset is input into the prediction model, and the component image dataset is classified to obtain qualified component image data and defective component image data. Cross-validation is performed on the qualified component image data and the defective component image data respectively, and the category probability of each component image data corresponding to its respective category is determined. The category probability is determined as the confidence level of the first label.

7. The data cleaning apparatus according to claim 6, characterized in that, The prediction unit is trained to obtain the prediction model in the following manner: The component image data sample set is divided into k mutually exclusive subsets of similar size; Based on the mutually exclusive subsets, select k-1 subsets as the training set, and the remaining subsets as the test set; A neural network is constructed as a base model, and the base model is trained based on the training set and the test set to obtain the prediction model.

8. The data cleaning apparatus according to claim 6 or 7, characterized in that, The prediction unit predicts the label of each component image in the component image dataset based on the confidence level of the first label, thereby obtaining the second label: If the confidence level of the first label of the target component image is greater than the confidence level threshold, then the second label of the target component image is determined to be the first label; If the confidence level of the first label of the target component image is less than or equal to the confidence level threshold, then the second label is determined to be the label of the component image corresponding to the highest category probability in the category to which the target component image belongs; Wherein, the target component image is each component image in the component image dataset, and the confidence threshold is the average probability of all components in the category to which the target component belongs.

9. A data cleaning device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Perform the data cleaning method according to any one of claims 1-4.

10. A storage medium, characterized in that, The storage medium stores instructions that, when executed by the terminal's processor, enable the terminal to perform the data cleaning method according to any one of claims 1-4.

Images