Optimization method and apparatus for machine learning classification tasks for anomaly detection

Abstract

Provided are an optimization method and apparatus for a machine learning classification task for anomaly detection. The method according to one aspect of the present disclosure includes: collecting sample data for reflecting the status of a production process or product to form a sample data set, wherein each piece of sample data in the sample data set has multiple features; determining one or more candidate principal component sets for the sample data set; for each of the one or more candidate principal component sets, performing the following operations: projecting the sample data set onto the candidate principal component set to obtain a projected sample data set, using the projected sample data set to train each of a plurality of candidate machine learning classification models, and evaluating the performance score of each of the trained candidate machine learning classification models; selecting the candidate machine learning classification model with the highest performance score and the corresponding candidate principal component set as the optimal machine learning classification model and the optimal principal component set.

Description

Technical Field

[0001] The present disclosure relates generally to information processing, and more particularly to methods and apparatus for optimizing machine learning classification tasks for anomaly detection. Background Art

[0002] As a key branch of artificial intelligence, machine learning has experienced significant growth in recent years with the advancement of computing power, the emergence of new algorithms and models, and the availability of massive amounts of data. Classification is one of the most common application scenarios for machine learning technology, including in the industrial sector. For example, the use of machine learning classification models has significantly improved the efficiency and automation of discrete and process control operations, including predictive maintenance and diagnostics. Predictive diagnostics and maintenance not only provide early warnings and protective guidance, but also increase machine / equipment uptime and reduce maintenance costs, thereby minimizing or even preventing financial losses.

[0003] One of the challenges facing machine learning classification tasks lies in the selection and processing of input data. In many practical scenarios, although a variety of data is readily available as input to machine learning classification models to perform classification tasks, the high dimensionality or excessive number of variables in the data, that is, the excessive number of features or attributes, not only reduces computational performance but also introduces noise, thereby increasing analysis complexity and reducing analytical capabilities. While methods such as principal component analysis can be used to automatically reduce the dimensionality of high-dimensional data, the performance of machine learning classification tasks performed on such reduced data still needs to be improved. Summary of the Invention

[0004] This summary is provided to introduce some selected concepts in a simplified form, which will be further described in the detailed description below. This summary is not intended to identify any key features or essential features of the claimed subject matter, nor is it intended to be used to help determine the scope of the claimed subject matter.

[0005] According to one aspect of the present disclosure, there is provided an optimization method for a machine learning classification task for anomaly detection, comprising: collecting sample data for reflecting the status of a production process or a product to form a sample data set, wherein each piece of sample data in the sample data set has multiple features; determining one or more candidate principal component sets for the sample data set; for each of the one or more candidate principal component sets, performing the following operations: projecting the sample data set onto the candidate principal component set to obtain a projected sample data set, using the projected sample data set to train each of a plurality of candidate machine learning classification models, and evaluating the performance score of each of the trained candidate machine learning classification models; and selecting the candidate machine learning classification model and the corresponding candidate principal component set with the highest performance score as the optimal machine learning classification model and the optimal principal component set.

[0006] The above aspects of the present disclosure provide an efficient mechanism that helps optimize machine learning classification tasks for anomaly detection. Based on the ability to effectively reduce the dimensionality of original high-dimensional sample data to reduce the computational load and improve computing performance, it can fully explore the optimal match between different target dimension options (i.e., corresponding to one or more candidate principal component sets) and different machine learning classification models, so that the execution performance of the machine learning classification task can be optimized.

[0007] In addition, in an example of the aforementioned aspect, the method may also include using the optimal machine learning classification model to perform the machine learning classification task, which includes: projecting the collected sample data with multiple features onto the optimal principal component set; providing the projected sample data as input to the optimal machine learning classification model; and obtaining the prediction result output by the optimal machine learning classification model.

[0008] Furthermore, in an example of the aforementioned aspect, determining one or more candidate principal component sets for the sample data set may include: performing centralization processing on all sample data in the sample data set; calculating a covariance matrix of the sample data; performing eigendecomposition on the covariance matrix to obtain a plurality of eigenvalues ​​and corresponding eigenvectors; and calculating a cumulative variance contribution rate c of the largest M eigenvalues ​​among the plurality of eigenvalues. M , so that c MSatisfy a preset cumulative variance contribution rate threshold B; compare M with T, where T is a preset threshold for the number of principal components; if M>T, then for each m=T+1, T+2, …, M, determine m eigenvectors corresponding to the m largest eigenvalues among the multiple eigenvalues as a candidate principal component set, a total of (M - T) candidate principal component sets; if M<T, then for each m=T-1, T-2, …, M, determine m eigenvectors corresponding to the m largest eigenvalues among the multiple eigenvalues as a candidate principal component set, a total of (T - M) candidate principal component sets; and if M=T, then for each m=T - A, …, T-1, T, T+1, …, T+A, determine m eigenvectors corresponding to the m largest eigenvalues among the multiple eigenvalues as a candidate principal component set, a total of (2A + 1) candidate principal component sets, where A is a preset adjustment range for the number of principal components.

[0009] In addition, in an example of the foregoing aspect, if |c M - B| ≤ Δ, then the cumulative variance contribution rate c of the M largest eigenvalues among the multiple eigenvalues M satisfies the preset cumulative variance contribution rate threshold B, where Δ is a preset allowable deviation.

[0010] In addition, in an example of the foregoing aspect, performing eigen - decomposition on the covariance matrix to obtain multiple eigenvalues and corresponding eigenvectors may further include: recording the features in the multiple features of the sample data set corresponding to each of the multiple eigenvalues; and wherein the method may further include: based on the eigenvalues corresponding to the eigenvectors in the optimal principal component set, determining an optimal feature set according to the record, and optimizing sample data collection according to the determined optimal feature set.

[0011] In addition, in an example of the foregoing aspect, evaluating the performance scores of each of the trained candidate machine - learning classification models may include: calculating the accuracy of each candidate machine - learning classification model trained using the training set in the projected sample data set based on the test set in the projected sample data set.

[0012] In addition, in an example of the foregoing aspect, the method may further include controlling the production process or the product based on the prediction result.

[0013] According to another aspect of the present disclosure, an optimization device for a machine learning classification task for anomaly detection is provided, comprising: a collection module for collecting sample data reflecting the status of a production process or a product to form a sample data set, wherein each sample data in the sample data set has multiple features; a determination module for determining one or more candidate principal component sets for the sample data set; a processing module for performing the following operations for each of the one or more candidate principal component sets: projecting the sample data set onto the candidate principal component set to obtain a projected sample data set, using the projected sample data set to train each of a plurality of candidate machine learning classification models, and evaluating the performance score of each of the trained candidate machine learning classification models; and a selection module for selecting the candidate machine learning classification model and the corresponding candidate principal component set with the highest performance score as the optimal machine learning classification model and the optimal principal component set.

[0014] According to another aspect of the present disclosure, a computing device is provided, comprising: at least one processor; and a memory coupled to the at least one processor and configured to store instructions, wherein when the instructions are executed by the at least one processor, the at least one processor is configured to perform any one of the methods described herein.

[0015] According to yet another aspect of the present disclosure, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by at least one processor, the at least one processor is caused to perform any one of the methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Implementations of the present disclosure are illustrated by way of example and not limitation in the accompanying drawings in which like reference numerals designate identical or similar parts and in which:

[0017] Figure 1 shows an exemplary environment in which some implementations of the present disclosure may be implemented;

[0018] Figure 2 A flowchart illustrating an exemplary method according to some implementations of the present disclosure is shown;

[0019] Figure 3 A flowchart illustrating exemplary operations according to some implementations of the present disclosure is shown;

[0020] Figure 4 shows an architectural example of some implementations according to the present disclosure;

[0021] Figure 5 A block diagram illustrating an exemplary apparatus according to some implementations of the present disclosure; and

[0022] Figure 6 A block diagram of an exemplary computing device according to some implementations of the present disclosure is shown.

[0023] Reference Signs List

[0024] 110: Data source 120: Processing device 130: Network

[0025] 210-240: Steps 310-380: Steps

[0026] 510: Collection module 520: Determination module 530: Processing module

[0027] 540: Select module

[0028] 610: Processor 620: Memory DETAILED DESCRIPTION

[0029] In the following description, for the purpose of explanation, a large number of specific details are set forth. However, it should be understood that the implementation of the present disclosure can be implemented without these specific details. In other examples, well-known circuits, structures and technologies are not shown in detail so as not to affect the understanding of the description.

[0030] References throughout this specification to "an implementation," "implementations," "example implementations," "some implementations," "various implementations," etc., indicate that the implementations of the present disclosure described may include particular features, structures, or characteristics. However, it does not imply that every implementation must include these particular features, structures, or characteristics. Furthermore, some implementations may have some, all, or none of the features described for other implementations.

[0031] In a manner that is most helpful in understanding the claimed subject matter, various operations may be described as multiple discrete actions or operations in a sequential order. However, the order of description should not be interpreted as implying that these operations are necessarily order-dependent. In particular, these operations may not be performed in the order presented. In other implementations, various additional operations may be performed, and / or various operations already described may be omitted.

[0032] In the specification and claims, where the phrase "A and / or B" may appear, it is intended to mean one of the following: (A), (B), (A and B). Similarly, where the phrase "A, B and / or C" may appear, it is intended to mean one of the following: (A), (B), (C), (A and B), (A and C), (B and C), (A and B and C).

[0033] Throughout the specification and claims, the terms "coupled" and "connected," along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular implementations, "connected" is used to indicate that two or more components are in direct physical or electrical contact with each other, while "coupled" is used to indicate that two or more components cooperate or interact with each other, but they may or may not be in direct physical or electrical contact.

[0034] First refer to Figure 1 , which illustrates an exemplary operating environment 100 in which some implementations of the present disclosure may be implemented. Figure 1 As shown, in some implementations, operating environment 100 may include at least one data source 110 and processing device 120 , which may be communicatively coupled to each other via a network 130 .

[0035] The processing device 120 is configured to implement the various aspects described herein based, at least in part, on data from at least one data source 110. Taking the application scenario of production process diagnosis and maintenance as an example, the data source 110 is typically deployed at an appropriate location within the production process, and the data it provides can reflect the production status of the process. By way of example and not limitation, the data source 120 may include various types of sensors, such as speed sensors, temperature sensors, pressure sensors, humidity sensors, current sensors, voltage sensors, and the like. Each of the sensors 120 may be an Internet of Things (IoT) sensor, and together they may form part of the IoT. In some examples, these sensors 120 may be configured to collect data at a specified frequency or in response to an external indication signal. During each sampling operation, the values ​​collected by these sensors collectively constitute a piece of sample data for the processing device 120, which contains multiple features or dimensions, such as speed, temperature, pressure, humidity, current, voltage, and the like. It should be noted that the data sources described herein are not limited to the specific application scenarios and examples described above; other types of data sources capable of providing data for processing by the processing device 120 are also feasible. For example, in some examples, the data source 110 may also be a database, a storage device, or any other type of device for providing data.

[0036] Examples of processing device 120 may include, but are not limited to, a mobile device, a personal digital assistant (PDA), a wearable device, a smartphone, a cellular phone, a handheld device, a messaging device, a computer, a personal computer (PC), a desktop computer, a laptop computer, a notebook computer, a handheld computer, a tablet computer, a workstation, a minicomputer, a mainframe computer, a supercomputer, a network device, a web device, a processor-based system, a multi-processor system, a consumer electronic device, a programmable consumer electronic device, a television, a digital television, a set-top box, or any combination thereof. In some implementations, processing device 120 may be deployed at a location remote from data source 110. Furthermore, in some implementations, the functionality of processing device 120 may be implemented by an application running on it, although the present disclosure is not limited thereto.

[0037] In addition, despite Figure 1 While processing device 120 is shown as a single device, those skilled in the art will appreciate that processing device 120 may also be implemented as a group of devices. In some implementations, processing device 120 may be implemented as a single server, a server array, or a server cluster. Furthermore, in some implementations, processing device 120, or at least a portion thereof, may be deployed in a distributed computing environment. In some implementations, processing device 120, or at least a portion thereof, may be deployed in the cloud, utilizing cloud computing technology.

[0038] The network 130 may include any type of wired or wireless communication network, or a combination of wired and wireless communication networks. Examples of communication networks may include, but are not limited to, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a public telephone network (PSTN), the Internet, an intranet, an Internet of Things (IoT), an infrared (IR) network, a Bluetooth network, a near field communication (NFC) network, a ZigBee network, and the like. In addition, although Figure 1 Although a single network 130 is shown in FIG, the network 130 may be configured to include multiple networks. In some implementations, the processing device 120 and the at least one data source 110 may be directly communicatively coupled without going through the network 130.

[0039] For machine learning classification tasks, the higher the dimensionality of the sample data, or the more features it contains, the more noise and redundancy it inevitably contains. This not only increases computational complexity but can also reduce analytical capabilities. Therefore, it is necessary to reduce the dimensionality of the sample data. Principal component analysis (PCA) is one of the most common dimensionality reduction methods. It automatically projects high-dimensional data into a low-dimensional space represented by a specific set of principal components through a linear transformation.

[0040] Typically, for a machine learning classification task, high-dimensional sample data is reduced to a specific target dimension using PCA. A specific machine learning classification model then uses the reduced sample data as input and outputs predictions. However, this approach doesn't fully consider the match between the target dimension and the classification model, and can significantly impact the performance of the specific machine learning classification task.

[0041] Figure 2 A flow chart of an exemplary method 200 according to some implementations of the present disclosure is shown. The method 200 can optimize a machine learning classification task. The method 200 can be used, for example, in Figure 1 The embodiment may be implemented in the processing device 120 shown in FIG. 1 or any similar or related entity.

[0042] like Figure 2 As shown, method 200 begins at step 210, where sample data is collected to form a sample dataset. Each piece of sample data in the sample dataset has multiple features. In some implementations, the sample data reflects the status of a production process or product, and the machine learning classification task can be used to perform anomaly detection on the production process or product based on the sample data. The multiple features or dimensions of each piece of sample data are specific values ​​representing the status of the production process or product. By way of example and not limitation, the status of a production process can be indicated by operating parameters of the production process, such as speed, temperature, pressure, humidity, current, voltage, etc., as described above. The status of a product can include static properties of the product, such as its external dimensions, or operational properties, such as its speed, current, and voltage during operation. These values ​​can be acquired via various sensors, but the present disclosure is not limited thereto. The sample dataset is composed of a certain number of collected sample data pieces and can be constructed, for example, using a matrix. Taking d-dimensional sample data as an example, each piece of sample data contains d features. Each row of the matrix representing the sample dataset is a d-dimensional sample data piece, and the number of columns of the matrix corresponds to the number of samples.

[0043] Next, the method 200 proceeds to step 220, in which one or more candidate principal component sets for the sample data set are determined. Figure 3 , Figure 3 A flow chart illustrating example operations 300 according to some implementations of the present disclosure is shown, which may correspond to one implementation of step 220 in example method 200 .

[0044] like Figure 3As shown, operation 300 first proceeds to step 310, in which all sample data in the sample data set are centralized. Still taking d-dimensional sample data as an example, the centralization process makes the mean of all sample data in each of the d dimensions in the processed sample data set zero.

[0045] Next, in step 320, the covariance matrix of the sample data processed in step 310 is calculated. Continuing with the example of d-dimensional sample data, the covariance matrix obtained is a d×d matrix. The values ​​at positions on the main diagonal of the matrix (e.g., (1,1), (2,2)…(d,d)) represent the variance of the sample data set in the corresponding dimension, while the values ​​at other positions (e.g., (1,2), (2,3), (d,d-1), etc.) represent the covariance of the sample data set in the corresponding two dimensions.

[0046] Operation 300 proceeds to step 330, in which eigendecomposition is performed on the covariance matrix obtained in step 320 to obtain a plurality of eigenvalues ​​and corresponding eigenvectors. Continuing with the previous example, by performing eigendecomposition on the d×d covariance matrix, d eigenvalues ​​and an eigenvector corresponding to each eigenvalue can be obtained, where each eigenvector is a d×1 matrix.

[0047] Then, in step 340, the cumulative variance contribution rate c of the largest M eigenvalues ​​among the multiple eigenvalues ​​is calculated. M , so that c M Satisfy the preset cumulative variance contribution rate threshold B. The variance contribution rate of each eigenvalue can be determined by calculating the ratio of the eigenvalue to the sum of all eigenvalues. In step 340, all d eigenvalues ​​can be sorted from large to small, and then it is determined how many eigenvalues ​​in the front of the sequence need to be selected to make the sum of their contribution rates meet the preset threshold B. The number determined here to be selected is recorded as M. In some implementations, the preset cumulative variance contribution rate threshold B can be set based on experience or convention, but the present disclosure is not limited to this.

[0048] In some implementations, an allowable deviation Δ may be preset so that if the determination condition |c M -B|≤Δ, then the cumulative variance contribution rate c of the largest M eigenvalues ​​among the multiple eigenvalues ​​is considered M The preset cumulative variance contribution rate threshold B has been met. In some implementations, the allowable deviation Δ can also be set based on experience or according to convention, but the present disclosure is not limited to this. In addition, regarding c M It is also possible to satisfy other judgment conditions of B.

[0049] Next, in step 350, the number M determined in step 340 is compared with a preset threshold T of the number of principal components. If the comparison result indicates that M>T, then in step 360, for each m=T+1, T+2, …, M, the m eigenvectors corresponding to the m largest eigenvalues among the multiple eigenvalues are determined as a set of candidate principal components, a total of (M-T) sets of candidate principal components. It can be understood that the value of T is less than or equal to the dimensionality of the original sample data. As an example, assume that T=5 and M=8. Then, the 6 eigenvectors corresponding to the T+1=6 largest eigenvalues among the multiple eigenvalues are successively determined as the first set of candidate principal components, the 7 eigenvectors corresponding to the largest T+2=7 eigenvalues are determined as the second set of candidate principal components, and the 8 eigenvectors corresponding to the largest T+3=8 eigenvalues are determined as the third set of candidate principal components. In this way, a total of 8-5=3 sets of candidate principal components are determined.

[0050] On the other hand, if the comparison result of step 350 indicates that M<T, then in step 370, for each m=T-1, T-2, …, M, the m eigenvectors corresponding to the m largest eigenvalues among the multiple eigenvalues are determined as a set of candidate principal components, a total of (T-M) sets of candidate principal components. Still taking T=5 as an example, assume that M=3 is determined in step 340. Then, the 4 eigenvectors corresponding to the largest T-1=4 eigenvalues among the multiple eigenvalues are successively determined as the first set of candidate principal components, and the 3 eigenvectors corresponding to the largest T-2=3 eigenvalues are determined as the second set of candidate principal components. In this way, a total of 5-3=2 sets of candidate principal components are determined.

[0051] Returning to step 350, if the comparison result of this step indicates that M=T, then in step 380, for each m=T-A, …, T-1, T, T+1, …, T+A, the m eigenvectors corresponding to the m largest eigenvalues among the multiple eigenvalues are determined as a set of candidate principal components, a total of (2A+1) sets of candidate principal components, where A is a preset adjustment range of the number of principal components. In some implementations, the value of A can be set according to the requirements of the classification task, and A can take natural numbers. Still taking T=5 as an example, assume that A=1 and assume that M=5 is determined in step 340 and thus the comparison result of step 350 indicates that T and M are equal. Then, the 4 eigenvectors corresponding to the largest T-1=4 eigenvalues among the multiple eigenvalues are successively determined as the first set of candidate principal components, the 5 eigenvectors corresponding to the largest T=5 eigenvalues are determined as the second set of candidate principal components, and the 6 eigenvectors corresponding to the largest T+1=6 eigenvalues are determined as the third set of candidate principal components. In this way, a total of 2+1=3 sets of candidate principal components are determined.

[0052] By using the exemplary operation 300 described above, one or more candidate principal component sets can be determined for a sample data set having multiple features, each of which corresponds to a candidate target dimension, thereby enabling subsequent evaluation based on these candidates and selection of the best one.

[0053] It should be noted that the exemplary operations 300 discussed above are not Figure 2 In some alternative implementations, instead of the operations in steps 350-380, the cumulative variance contribution rate c of the largest M eigenvalues ​​among the multiple eigenvalues ​​can be calculated in step 340. M , so that c M After the preset cumulative variance contribution rate threshold B is met, step 350 is not performed, but rather partial operations similar to those described in step 380 are directly performed to determine one or more candidate principal component sets. Specifically, for each of m=MA′,…,M-1,M,M+1,…,M+A′, the m eigenvectors corresponding to the m largest eigenvalues ​​among the multiple eigenvalues ​​are determined as a candidate principal component set, for a total of (2A′+1) candidate principal component sets, where A′ is a preset adjustment range for the number of principal components. In some implementations, its value can be the same as the aforementioned A, but the present disclosure is not limited thereto. In addition, other methods for determining one or more candidate principal component sets for a sample data set with multiple features are also feasible.

[0054] Some implementations of step 220 are described above in conjunction with exemplary operation 300. Some operations involved in these operations, such as centering, covariance matrix calculation, and eigendecomposition to obtain eigenvalues ​​and corresponding eigenvectors, are calculated in a manner well known to those skilled in the art and will not be repeated here.

[0055] Back to Figure 2 After determining one or more candidate principal component sets for the sample data set in step 220, method 200 may proceed to step 230, in which the following operations are performed for each of the one or more candidate principal component sets.

[0056] Specifically, first, in step 232, the sample dataset is projected onto the candidate principal component set to obtain a projected sample dataset. As discussed previously, each candidate principal component set corresponds to a set of eigenvectors of varying numbers, with the number of eigenvectors in the set representing a candidate target dimension d′. This set of eigenvectors forms a projection matrix, with each column representing an eigenvector. Based on this projection matrix, the original d-dimensional sample dataset can be projected into a d′-dimensional space, thereby achieving dimensionality reduction of the sample dataset.

[0057] Then, in step 234, the projected sample data set is used to train each of the multiple candidate machine learning classification models. According to some implementations of the present disclosure, multiple machine learning classification models are provided for candidates, so that subsequent evaluation and selection of the best one from them become possible. According to some implementations of the present disclosure, the provided machine learning classification models are used to perform anomaly detection based on sample data used to reflect the status of the production process or product. In some implementations, the prediction results output by the machine learning classification model are used to indicate whether the production process or product has an anomaly, but the present disclosure is not limited to this. In some implementations, the multiple candidate machine learning classification models may include two or more of the following: logistic regression (LR), Softmax, artificial neural network, extreme learning machine (ELM), K nearest neighbor (KNN), support vector machine (SVM), decision tree (DT), naive Bayes, random forest, boosting, linear discriminant analysis (LDA). In addition, other types of classic machine learning classification models are also feasible, and the present disclosure is not limited to this.

[0058] After the training in step 234 is performed, in step 236, the performance score of each of the trained candidate machine learning classification models is evaluated. In some implementations, the sample data set is divided into a training set and a test set, wherein the training set is used to train the machine learning classification model, and the test set is used to test the trained model and evaluate the performance of the model. In some implementations, the accuracy of each candidate machine learning classification model can be calculated as the performance score of the model. In other implementations, other calculation methods of the performance score are also feasible, including but not limited to precision, recall, etc.

[0059] It can be understood that through the processing of step 230, x×y combinations can be constructed for x candidate principal component sets and y candidate machine learning classification models, and training and evaluation can be performed for each combination. In some implementations, training and evaluation are performed in parallel, respectively, thereby further improving the execution efficiency of the solution of the present disclosure. In addition, in some implementations, different parallel execution strategies can be adopted depending on constraints such as computing hardware conditions. For example, based on a candidate principal component set, y candidate machine learning classification models can be trained in parallel and then evaluated in parallel. In another example, based on a candidate machine learning classification model, x variants of the model corresponding to different candidate principal component sets can be trained in parallel and then evaluated in parallel. In yet another example, all x×y combinations can be trained in parallel and then evaluated in parallel. It can be understood by those skilled in the art that other parallel execution strategies are also feasible, and the present disclosure is not limited to one or more specific ones.

[0060] Then, method 200 proceeds to step 240, in which the candidate machine learning classification model and the corresponding candidate principal component set with the highest performance score are selected as the optimal machine learning classification model and the optimal principal component set. In other words, according to some implementations of the present disclosure, a combination with the highest performance score can be selected from the aforementioned x×y combinations. The machine learning classification model in this combination is the optimal machine learning classification model, and the principal component set is the optimal principal component set.

[0061] Combined with the above Figure 2 The described exemplary method 200 provides an efficient mechanism that helps optimize machine learning classification tasks. Based on the ability to effectively reduce the dimensionality of original high-dimensional sample data to reduce computing load and improve computing performance, it can fully explore the optimal match between different target dimension options and different machine learning classification models, so that the execution performance of the machine learning classification task can be optimized and the most accurate anomaly detection results can be obtained.

[0062] In addition, in some implementations, when the covariance matrix is ​​subjected to eigendecomposition in step 330 to obtain a plurality of eigenvalues ​​and corresponding eigenvectors, the features corresponding to each of the plurality of eigenvalues ​​obtained in the plurality of features of the sample data set may also be recorded, for example, the features, eigenvalues, and eigenvectors may be stored in association with each other. In some implementations, the exemplary method 200 may further include, after determining the optimal machine learning classification model and the optimal principal component set in step 240, determining the optimal feature set based on the eigenvalues ​​corresponding to the eigenvectors in the optimal principal component set according to the records and optimizing the sample data collection according to the determined optimal feature set. For example, the values ​​of other features outside the optimal feature set may no longer be collected, and even the hardware such as sensors related thereto may be omitted, thereby reducing the collection cost and computational load.

[0063] Furthermore, in some implementations, after determining the optimal machine learning classification model and the optimal principal component set in step 240, exemplary method 200 may further include using the determined optimal machine learning classification model to perform the machine learning classification task, which includes: projecting the collected sample data having multiple features onto the optimal principal component set; providing the projected sample data as input to the optimal machine learning classification model; and obtaining a prediction result output by the optimal machine learning classification model. The machine learning classification task performed in this manner fully utilizes the optimal match between the principal component set and the classification model, and further enjoys the highest predictive power provided by the optimal classification model based on successfully reducing the dimensionality of the original high-dimensional sample data to reduce computational overhead.

[0064] Furthermore, as previously mentioned, in some implementations, each piece of sample data in a sample data set having multiple features can be used to reflect the status of a production process or a product. Accordingly, a machine learning classification model is used to predict whether there is an abnormality in the production process or the product, or in some implementations, the classification capability of the classification model can be used to predict which specific type of abnormality exists. Therefore, the exemplary method 200 may also include: controlling the production process or the product based on the prediction results obtained above. For example, in response to the prediction result indicating that there is an abnormality in the production process, the operation of the production process may be suspended. For example, in response to the prediction result indicating that there is an abnormality or defect in the product, the product with the abnormality or defect may be disposed of, such as by specifically marking it or even discarding the product. For example, in response to the prediction result indicating that there is a specific type of abnormality in the production process or product, more refined processing for this type of abnormality may be triggered, and so on. Other control measures are also feasible.

[0065] Furthermore, in some implementations, the mechanisms described in this disclosure can also be used for fault diagnosis. For example, for potentially problematic equipment, data related to the equipment can be collected as sample data, and a machine learning classification model can be used to predict or diagnose that one or more components of the equipment are faulty. Based on these predictions or diagnostic results, subsequent actions can be triggered, including but not limited to replacing the faulty component.

[0066] Those skilled in the art will appreciate that the application scenarios described above are merely illustrative and non-restrictive, and the mechanisms described in this disclosure should not be limited to one or several applications.

[0067] Figure 4 Schematic examples of some implementations of the present disclosure are shown. Figure 4 As shown, according to some implementations of the present disclosure, there may be PCS1, PCS2, ... PCS R There are R candidate principal component sets, that is, there are R candidate target dimensions, which can be determined using step 220 of the exemplary method 200 discussed above. Using the sample data set projected onto each of the R candidate principal component sets, ML1, ML2, ...ML N A total of N candidate machine learning classification models are trained. In this way, a total of R×N different combinations can be constructed. In other words, for each of the N candidate machine learning classification models, since its input data has R different dimensions, it is equivalent to having R different variants of this model. For each specific combination, the performance score of the corresponding model is evaluated. Taking the first model ML1 as an example, the performance score of each of its R variants can be evaluated, which are denoted as S 11 ,S 12 ,…S 1R For the second model ML2 until the last model ML N The same applies to the processing of . Next, the machine learning classification model and principal component set corresponding to the maximum value among all R×N performance scores can be determined, which can correspond to step 240 in the exemplary method 200 discussed above. As an example, Figure 4 In the above example, it is determined that the largest of the R performance scores associated with the first model ML1 is the Hth, i.e., S 1H , the corresponding candidate principal component set is PCS H Similarly, determine the largest of the R performance scores associated with the second model ML2 as S 2E , the corresponding candidate principal component set is PCS E , and so on, until the Nth model ML is determined N The largest of the R related performance scores is S NK, the corresponding candidate principal component set is PCS K Then, we further find the largest one among the N values ​​determined above, which is S in this example. 2E , so the optimal machine learning classification model is ML2, and the optimal principal component set is PCS E The above method of determining the maximum value is only an example, and other processes are also feasible.

[0068] Reference below Figure 5 , which shows a block diagram of an exemplary apparatus 500 according to some implementations of the present disclosure. The apparatus 500 can optimize machine learning classification tasks for exception handling. The apparatus 500 can be used, for example, in Figure 1 The embodiment may be implemented in the processing device 120 shown in FIG. 1 or any similar or related entity.

[0069] like Figure 5 As shown, the device 500 may include a collection module 510, which is used to collect sample data for reflecting the status of a production process or product to form a sample data set, wherein each sample data in the sample data set has multiple features. The device 500 may also include a determination module 520, which is used to determine one or more candidate principal component sets for the sample data set. The device 500 may also include a processing module 530, which is used to perform the following operations for each of the one or more candidate principal component sets: projecting the sample data set onto the candidate principal component set to obtain a projected sample data set, using the projected sample data set to train each of a plurality of candidate machine learning classification models, and evaluating the performance score of each of the trained candidate machine learning classification models. In addition, the device 500 may also include a selection module 540, which is used to select the candidate machine learning classification model with the highest performance score and the corresponding candidate principal component set as the optimal machine learning classification model and the optimal principal component set.

[0070] In addition, in some implementations, one or more of the above modules of the apparatus 500 may further include further submodules, and / or the apparatus 500 may further include additional modules for performing other operations described in the specification, such as combining Figure 2 A flowchart of an exemplary method 200 of Figure 3300 is described with reference to the flowchart of the exemplary operation 300. In addition, in some implementations, the various modules of the apparatus 500 may be combined or split depending on actual needs. For example, the processing module 530 may be split into the following three modules: a projection module for projecting the sample data set onto the candidate principal component set to obtain a projected sample data set; a training module for training each of a plurality of candidate machine learning classification models using the projected sample data set; and an evaluation module for evaluating the performance score of each of the trained candidate machine learning classification models. The above and other variations all fall within the scope of the present disclosure.

[0071] Those skilled in the art will appreciate that the exemplary apparatus 500 may be implemented using software, hardware, firmware, or any combination thereof.

[0072] Figure 6 A block diagram of an exemplary computing device 600 according to some implementations of the present disclosure is shown. The computing device 600 can optimize machine learning classification tasks for exception handling. The computing device 600 can, for example, Figure 1 The embodiment may be implemented in the processing device 120 shown in FIG. 1 or any similar or related entity.

[0073] like Figure 6 As shown, the computing device 600 may include at least one processor 610. The processor 610 may include any type of general-purpose processing unit (e.g., CPU, GPU, etc.), a dedicated processing unit, a core, a circuit, a controller, etc. In addition, the computing device 600 may also include a memory 620 coupled to the processor 610. The memory 620 may include any type of medium that can be used to store data. In some implementations, the memory 620 is configured to store instructions that, when executed, cause the at least one processor 610 to perform the operations described in the present disclosure, for example, in conjunction with Figure 2 A flowchart of an exemplary method 200 of Figure 3 3. The exemplary operation 300 is described with reference to FIG.

[0074] Furthermore, in some implementations, the computing device 600 may be coupled to or equipped with one or more peripheral components, which may include, but are not limited to, a display, a speaker, a mouse, a keyboard, and the like. Furthermore, in some implementations, the computing device 600 may be equipped with a communication interface that may support various types of wired / wireless communication protocols for communicating with a communication network. Examples of communication networks may include, but are not limited to, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a public telephone network, the Internet, an intranet, the Internet of Things, an infrared network, a Bluetooth network, a near field communication (NFC) network, a ZigBee network, and the like.

[0075] Furthermore, in some implementations, the above-mentioned and other components may communicate with each other via one or more buses / interconnects that may support any suitable bus / interconnect protocol, including Peripheral Component Interconnect (PCI), PCI Express, Universal Serial Bus (USB), Serial Attached SCSI (SAS), Serial ATA (SATA), Fibre Channel (FC), System Management Bus (SMBus), or other suitable protocols.

[0076] Those skilled in the art will appreciate that the above description of the structure of the computing device 600 is merely exemplary and non-restrictive, and devices with other structures are also feasible as long as they can be used to implement the functions discussed in this disclosure.

[0077] Various implementations of the present disclosure may be implemented using hardware units, software units, or a combination thereof. Examples of hardware units may include devices, components, processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, application specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), memory cells, logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc. Examples of software units may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof. Determining whether an implementation is implemented using hardware elements and / or software elements can vary depending on a variety of factors, such as desired computational rate, power levels, thermal tolerances, processing cycle budget, input data rate, output data rate, memory resources, data bus speed, and other design or performance constraints as desired for a given implementation.

[0078] Some implementations of the present disclosure may include articles of manufacture. Articles of manufacture may include storage media for storing logic. Examples of storage media may include one or more types of computer-readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and the like. Examples of logic may include various software units, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, application program interfaces (APIs), instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof. In some implementations, for example, articles of manufacture may store executable computer program instructions that, when executed by a processor, cause the processor to perform the methods and / or operations described herein. Executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. Executable computer program instructions can be implemented according to a predefined computer language, method or syntax for commanding a computer to perform a specific function. The instructions can be implemented using any appropriate high-level, low-level, object-oriented, visual, compiled and / or interpreted programming language.

[0079] What has been described above includes examples of the disclosed architecture. It is, of course, not possible to describe every conceivable combination of components and / or methodologies, but those skilled in the art will appreciate that many other combinations and permutations are possible. Therefore, the novel architecture is intended to embrace all such alternatives, modifications, and variations that fall within the spirit and scope of the appended claims.

Claims

1. An optimization method for machine learning classification tasks for anomaly detection, comprising: Collecting sample data for reflecting the state of a production process or a product to form a sample data set, wherein each piece of sample data in the sample data set has multiple features; Determining one or more candidate principal component sets for the sample data set; For each of the one or more candidate principal component sets, performing the following operations: Projecting the sample data set onto the candidate principal component set to obtain a projected sample data set, Using the projected sample data set to train each of a plurality of candidate machine learning classification models, and Evaluating the performance scores of each of the trained candidate machine learning classification models; and selecting the candidate machine learning classification model with the highest performance score and the corresponding candidate principal component set as the optimal machine learning classification model and the optimal principal component set; Wherein the plurality of candidate machine learning classification models include two or more of the following: Logistic Regression (LR), Softmax, Artificial Neural Network, Extreme Learning Machine (ELM), K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Decision Tree (DT), Naive Bayes, Random Forest, Boosting Method, Linear Discriminant Analysis (LDA); Wherein determining one or more candidate principal component sets for the sample data set includes: Centering all sample data in the sample data set; Calculating the covariance matrix of the sample data; Performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues and corresponding eigenvectors; Calculate the cumulative variance contribution rate c of the largest M eigenvalues ​​among the multiple eigenvalues M , so that c M Satisfy the preset cumulative variance contribution rate threshold B; Comparing M with T, wherein T is a preset threshold for the number of principal components; If M>T, then for each m=T+1, T+2, …, M, determining the m eigenvectors corresponding to the m largest eigenvalues among the plurality of eigenvalues as a candidate principal component set, for a total of (M-T) candidate principal component sets; If M<T, then for each m=T-1, T-2, …, M, determining the m eigenvectors corresponding to the m largest eigenvalues among the plurality of eigenvalues as a candidate principal component set, for a total of (T-M) candidate principal component sets; and If M=T, then for each m=T-A, …, T-1, T, T+1, …, T+A, determining the m eigenvectors corresponding to the m largest eigenvalues among the plurality of eigenvalues as a candidate principal component set, for a total of (2a + 1) candidate principal component sets, wherein a is a preset adjustment range for the number of principal components. 2.The method according to claim 1, further comprising using the optimal machine learning classification model to perform the machine learning classification task, which includes: Projecting the collected sample data with multiple features onto the optimal principal component set; Providing the projected sample data as an input to the optimal machine learning classification model; And Obtaining the prediction result output by the optimal machine learning classification model.

3. The method according to claim 1, wherein If |c M -B|≤Δ, then the cumulative variance contribution rate c of the largest M eigenvalues ​​among the multiple eigenvalues M The preset cumulative variance contribution rate threshold B is met, where Δ is a preset allowable deviation.

4. The method according to claim 1, wherein Performing eigendecomposition on the covariance matrix to obtain a plurality of eigenvalues ​​and corresponding eigenvectors further comprises: recording a feature corresponding to each of the plurality of eigenvalues ​​in the plurality of features of the sample data set; and The method further comprises: determining an optimal feature set from the records based on eigenvalues ​​corresponding to eigenvectors in the optimal principal component set, and Optimize sample data collection based on the determined optimal feature set.

5. The method according to claim 1 or 2, wherein: The performance scores for evaluating each of the trained candidate machine learning classification models include: Based on the test set in the projected sample data set, the accuracy of each candidate machine learning classification model trained using the training set in the projected sample data set is calculated.

6. The method according to claim 2, further comprising: The production process or the product is controlled based on the prediction result.

7. An optimization device for a machine learning classification task for anomaly detection, comprising: a collection module, configured to collect sample data reflecting the status of a production process or a product to form a sample data set, wherein each piece of sample data in the sample data set has a plurality of features; A determination module, configured to determine one or more candidate principal component sets for the sample data set; a processing module configured to, for each of the one or more candidate principal component sets, perform the following operations: project the sample dataset onto the candidate principal component set to obtain a projected sample dataset, train each of a plurality of candidate machine learning classification models using the projected sample dataset, and evaluate a performance score of each of the trained candidate machine learning classification models; and A selection module, which is used to select the candidate machine learning classification model and the corresponding candidate principal component set with the highest performance score as the optimal machine learning classification model and the optimal principal component set; wherein the plurality of candidate machine learning classification models include two or more of the following: logistic regression (LR), Softmax, artificial neural network, extreme learning machine (ELM), K nearest neighbor (KNN), support vector machine (SVM), decision tree (DT), naive Bayes, random forest, boosting, linear discriminant analysis (LDA); Determining one or more candidate principal component sets for the sample data set includes: Centralizing all sample data in the sample data set; Calculating the covariance matrix of the sample data; Performing eigendecomposition on the covariance matrix to obtain a plurality of eigenvalues ​​and corresponding eigenvectors; Calculate the cumulative variance contribution rate c of the largest M eigenvalues ​​among the multiple eigenvalues M , so that c M Satisfy the preset cumulative variance contribution rate threshold B; Compare M with T, where T is the preset threshold of the number of principal components; If M>T, then for each of m=T+1, T+2, ..., M, m eigenvectors corresponding to the largest m eigenvalues ​​among the multiple eigenvalues ​​are determined as a candidate principal component set, for a total of (MT) candidate principal component sets; If M < T, for each m = T - 1, T - 2, …, M, determine m eigenvectors corresponding to the m largest eigenvalues among the plurality of eigenvalues as a candidate principal component set, for a total of (T - M) candidate principal component sets; and If m = T, for each m = T - A, …, T - 1, T, T + 1, …, T + A, determine m eigenvectors corresponding to the m largest eigenvalues among the plurality of eigenvalues as a candidate principal component set, for a total of (2A + 1) candidate principal component sets, where A is a preset range for adjusting the number of principal components.

8. A computing device, comprising: At least one processor; And A memory coupled to the at least one processor and configured to store instructions, wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform the method according to any one of claims 1 - 6.

9. A computer-readable storage medium having instructions stored thereon, which when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 - 6.

Images