Systems and methods for enhancing machine learning using hierarchical prediction and composite thresholding

Abstract

A computer device is programmed to receive a plurality of real-time data sets from one or more sensors associated with a tool to be analyzed, calibrate the plurality of real-time data sets, generate a time sliding window for each real-time data set in the plurality of real-time data sets, generate a random probability distribution curve, compare the random probability distribution curve to each time sliding window to determine whether the time sliding window contains anomalous data, and generate a prediction result based on the comparison.

Description

[0001] Cross-reference of related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 112,028, filed November 10, 2020, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] The technical field broadly involves augmented machine learning, and more specifically, machine learning using hierarchical prediction and composite thresholding for anomaly detection. Background Technology

[0004] In the production of integrated circuit (IC) chips, semiconductor wafers are typically used as substrates. Chip manufacturers require wafers with extremely flat and parallel surfaces to ensure that the maximum number of chips can be manufactured from each wafer. After slicing from the ingot, the wafers typically undergo grinding and polishing processes designed to improve specific surface features such as flatness and parallelism.

[0005] Simultaneous double-sided polishing operations are performed on both sides of the wafer to produce a wafer with a highly planarized surface. The polishing machine performing double-sided polishing uses a wafer clamping device to hold the semiconductor wafer during polishing. The clamping device typically includes a pair of hydrostatic pads and a pair of polishing wheels. The pads and wheels are oriented relative to each other to hold the wafer vertically between them. The hydrostatic pads advantageously create a fluid barrier between the respective pad and the wafer surface for holding the wafer during polishing, while the rigid pads do not physically contact the wafer. This reduces damage to the wafer that can be caused by physical clamping and allows the wafer to move tangentially (rotate) relative to the pad surface with less friction. While this polishing process can improve the flatness and / or parallelism of the polished wafer surface, it can cause a degradation in the topology of the wafer surface. Specifically, misalignment of the clamping planes of the hydrostatic pads and polishing wheels is known to cause this degradation. Furthermore, any degradation of any of the tools used in the process can allow hundreds of wafers to potentially be processed before the problem can be detected. Additionally, each individual production line and apparatus may have specific characteristics that can be varied between apparatuses. Therefore, a system is needed to detect and determine when a tool will degrade, require realignment, or otherwise cause problems in production lines. However, many systems used to train systems to identify when a tool is likely to fail are trained using only historical data and are susceptible to overfitting. Therefore, a training system is needed to train systems to identify when a tool will fail.

[0006] This "Background Art" section is intended to introduce the reader to various aspects of the art that may relate to the aspects of this disclosure described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the invention. Therefore, it should be understood that these statements should be read in this context, rather than as an admission of prior art. Summary of the Invention

[0007] On one hand, a computer device includes at least one processor communicating with at least one memory device. The at least one processor is programmed to receive multiple real-time datasets from one or more sensors associated with an tool to be analyzed. The at least one processor is also programmed to calibrate the multiple real-time datasets. The at least one processor is further programmed to generate a time sliding window for each of the multiple real-time datasets. Additionally, the at least one processor is programmed to generate a random probability distribution curve. Furthermore, the at least one processor is programmed to compare the random probability distribution curve with each time sliding window to determine whether the time sliding window contains anomalous data. Furthermore, the at least one processor is programmed to generate a prediction based on the comparison.

[0008] On the other hand, a method for analyzing an tool is implemented on a computer device including at least one processor communicating with at least one memory device. The method includes receiving multiple real-time datasets from one or more sensors associated with the tool to be analyzed. The method also includes calibrating the multiple real-time datasets. The method further includes generating a time sliding window for each of the multiple real-time datasets. Additionally, the method includes generating a random probability distribution curve. Furthermore, the method includes comparing the random probability distribution curve with each time sliding window to determine whether the time sliding window contains outlier data. Furthermore, the method includes generating a prediction result based on the comparison.

[0009] Various improvements exist regarding the features mentioned above. Further features may also be incorporated into the above aspects. These improvements and additional features may exist individually or in any combination. For example, various features discussed below with respect to any of the illustrated embodiments may be incorporated individually or in any combination into any of the aspects described above. Attached Figure Description

[0010] Figure 1 A block diagram illustrating a system for training and using machine learning including hierarchical prediction and composite thresholding, according to embodiments of the present disclosure.

[0011] Figure 2 This explains the generation used for training. Figure 1 The flowchart of the example program for the system's training dataset is shown in the image.

[0012] Figure 3 This is an instruction on how to use. Figure 1 The flowchart of the system example program shown in the document.

[0013] Figure 4 This is an explanation of using from Figure 1 The flowchart of an example program for predicting data from the system is shown in the image.

[0014] Figure 5 This diagram illustrates a system for training and using machine learning, which includes hierarchical prediction and composite thresholding.

[0015] Figure 6 This describes an example based on the present disclosure. Figure 1 The system shown in the document and Figure 5 The example configuration of the user computer device used in the system is shown in the figure.

[0016] Figure 7 This describes an example based on the present disclosure. Figure 1 The system shown in the document and Figure 5 The example configuration of the server computer device used in the system shown in the figure.

[0017] Figure 8 It is used to illustrate its use. Figure 1 The system shown in the article is through Figures 2 to 4 The example charts shown here illustrate the results of the analysis of the program execution.

[0018] Several views throughout the drawing are accompanied by reference symbols indicating the corresponding parts. Detailed Implementation

[0019] Figure 1 This illustration shows a block diagram of a system 100 for training and using machine learning, incorporating hierarchical prediction and composite thresholding, according to at least one embodiment of the present disclosure. In exemplary embodiments, system 100 is associated with one or more production lines for manufacturing products, wherein the products are processed by one or more tools. For example, the production line may be used to manufacture and polish silicon wafers, and the tools are wafer grinders or polishers. Although the systems and methods described herein are described using tools and production lines, those skilled in the art will understand that the systems and methods described herein can be used with other assembly lines, other tools, and other modeling situations where overfitting may occur, and are not limited to a production environment.

[0020] System 100 includes an intake section 102, a processing section 104, and an output section 106. The intake section 102 includes receiving, cleaning, and preprocessing data. The processing section 104 includes applying the data to a model and further processing the data to receive one or more predictions. The output section 106 includes formatting and displaying the data (including optionally providing alerts).

[0021] Intake unit 102 is capable of receiving raw dataset 108. Raw dataset 108 may contain historical data for training purposes. Raw dataset 108 may also contain real-time or real-time sensor data received from one or more sensors or measuring devices. Raw dataset 108 may contain data measuring the product before and after it has been processed by the tool discussed. Raw dataset 108 may be organized by data clustering component 110. Data clustering component 110 performs exploratory data analysis on raw dataset 108 to organize the data in raw dataset 108 into datasets by clustering or grouping based on similarity. In an example embodiment, data clustering component 110 is used to cluster raw dataset 108 received from sensors or measuring devices. In an example embodiment, training datasets have been organized into data clusters. Data clusters group similar data into several groups. Similarity may be related to time series, results, or other relationships.

[0022] Data calibration and alignment component 112 may receive raw datasets 108 (e.g., training datasets) that may have been clustered, or those already clustered by data clustering component 110 (e.g., sensor data). Data calibration and alignment component 112 prepares anomalous data and safe or clean data based on historical data. In some embodiments, data calibration and alignment component 112 sorts anomalous data and safe data into separate datasets. Anomalous data includes data where actual anomalies occur. Safe data is data without any anomalies and may contain false alarms. In some embodiments, the dataset is not cleanly fitted to either the anomalous dataset or the clean dataset. In these embodiments, dataset cleaning component 114 discards the dataset. Dataset cleaning component 114 removes noisy data, such as data that cannot be classified as clean or anomalous. After the data is cleaned, training dataset 116 can be sent to processing unit 104. In some embodiments, one or more characteristics of the noisy data are predefined by subject matter experts. Dataset cleaning component 114 compares one or more characteristics of the dataset with those of the noisy data to determine whether the dataset contains noisy data.

[0023] In processing section 104, training dataset 116 is sent to predictive modeling component 118. Predictive modeling component 118 acquires anomalous and safe data and uses a model to classify individual datasets as anomalous or safe data. Predictive modeling component 118 performs time series data prediction.

[0024] The sliding time window component 120 acquires categorized data and generates a time window that guides the information to be analyzed. For example, if the data is divided into one-minute time intervals, the sliding time window component 120 can combine the one-minute segment to be analyzed with the 29 minutes preceding said one-minute segment. This allows the system 100 to analyze the time leading to the one-minute segment. During training, this trains the system 100 to analyze the time preceding the anomaly to understand what caused the anomaly. This allows the system 100 to predict when another anomaly might occur by detecting similar lead-ups to the anomaly. For safe data, this allows the system 100 to learn what safe data looks like. If a change occurs in the data, but the lead is similar to the safe data from the training, then the system 100 can classify the analyzed data as safe. Although a 30-minute time window is described herein, other sizes of time windows may be analyzed depending on the circumstances and the items being analyzed. For example, in some embodiments, a degradation or change in the operation of the device may be more gradual. In these embodiments, the sliding time window component 120 may analyze 30-second data segments acquired over a two-week period, or optionally any other combination, to detect anomalies.

[0025] The random probability distribution curve component 122 analyzes a dataset with an upper bound. The random probability distribution curve component 122 uses random probabilities to generate an upper bound for detection purposes. When data in the dataset exceeds the generated upper bound, the system 100 marks the corresponding data as an anomaly. The random probability distribution curve component 122 determines the upper limit for each time point analyzed. This line is used as a composite threshold to determine whether the data is safe or contains anomalies. The random probability distribution curve defines the confidence interval of the random probability distribution as the anomaly warning boundary of the predicted curve generated for each time window. This boundary is a dynamic random probability distribution.

[0026] In at least one embodiment, the random probability curve is generated using parameters n and p as discrete probability distributions of the number of successes in a sequence of n independent experiments, each independent experiment asking a yes-no question, and each independent experiment having its own Boolean-valued result: success / yes / true / one (with probability p) or failure / no / false / zero (with probability q = 1-p). The random probability curve can be generated using a single success / failure experiment called a Bernoulli trial, where the sequence of results is a Bernoulli procedure.

[0027] During training, the feedback and experience learning component 124 compares the predictions of system 100 with actual data. For example, it checks whether the dataset contains anomalies or whether system 100 detects said anomalies, or whether system 100 predicts anomalies in the security data. The feedback and experience learning component 124 traces back to actual entity environmental factors in past predictions. This includes the correlation between the operational settings of organic parameters, prediction accuracy, correct prediction rate, and false alarm rate.

[0028] Output section 106 sorts the data and forecasts so that users can view them (e.g., via a webpage). Output section 106 may include visualization output component 126, alert component 128, and / or log file component 130. Visualization output component 126 organizes the data into viewable formats, such as displaying the data by creating dashboards or charts. An example of a chart generated by visualization output component 126 is shown in... Figure 8 As can be seen in the image. Alarm component 128 can trigger one or more alarms to inform the user of potential events. Log file component 130 can store data and predictive results of data analysis for future review.

[0029] Figure 2 This describes the generation of training system 100 (in...) Figure 1 The flowchart of example procedure 200 is shown in the example embodiment. In this example embodiment, the steps of procedure 200 are performed using a hierarchical prediction and composite thresholding (HPCT) computer device 510 (in...). Figure 5 (As shown in the diagram) to be executed. Program 200 includes steps for generating a training dataset for the model or enabling system 100 to classify data as to whether the data contains anomalies.

[0030] HPCT computer device 510 extracts 202 raw data. In an example embodiment, the raw data is a historical dataset, which contains a mixture of normal and abnormal datasets. For the purposes of this discussion, the abnormal dataset contains one or more anomalies, and the normal dataset does not contain any anomalies.

[0031] For each dataset, the HPCT computer device 510 classifies the corresponding dataset 204 as anomalous (containing anomalous data) or normal (not containing anomalous data). Next, the HPCT computer device 510 selects datasets based on whether the data is anomalous 206. If the dataset is not anomalous, it is sent to step 210. If the data is anomalous, the HPCT computer device 510 determines 208 whether the anomalous data originates from the analyzed tool. If not, the dataset is discarded. The anomalous data may originate from user errors or other non-tool-related problems exhibited in the dataset. Otherwise, if the anomalous data is confirmed to originate from the analyzed tool, the HPCT computer device 510 proceeds to step 210.

[0032] HPCT computer device 510 segments the data 210 according to a time series. In some embodiments, HPCT computer device 510 segments the dataset 210 into time segments of a predetermined amount of time (e.g., 10 seconds or one minute), depending on the situation. In some embodiments, HPCT computer device 510 also appends a certain amount of previous data from before the time of the dataset.

[0033] HPCT computing device 510 performs 212 dataset calibration and alignment to ensure that the size and format of the dataset match the size and format of other datasets. For example, HPCT computing device 510 can remove portions of a dataset to make it the same size as other datasets. Or HPCT computing device 510 can trim all datasets to match the shortest dataset with anomalies.

[0034] HPCT computer device 510 stores the dataset 214, for example, in database 520 (in...). Figure 5 (As shown in the diagram). HPCT computer device 510 performs exploratory data analysis (e.g., clustering) on ​​the dataset 216. Once clustering is complete, HPCT computer device 510 returns to step 212 and continues procedure 200. In the exploratory data analysis step 216, HPCT computer device 510 determines the type and structure of the provided data. HPCT computer device 510 examines the dataset to determine if any outliers or anomalies exist in the data. HPCT computer device 510 also determines any correlations or relationships between the data. Additionally, while in some embodiments the complete training data (220) may be fixed, the different tools being analyzed may have their own time series or metrics. Therefore, it may be necessary to retrain the predictive model for the different tools or devices being analyzed.

[0035] HPCT computer device 510 cleans 218 noisy data. Next, HPCT computer device 510 uses the cleaned dataset to generate a training dataset. In some embodiments, one or more characteristics of the noisy data are predefined by a subject matter expert. HPCT computer device 510 compares one or more characteristics of the dataset with those of the noisy data to determine whether the dataset contains noisy data. HPCT computer device 510 is programmed to search for potential noisy data based on the performance of the tool or device's historical behavior and previous anomalies.

[0036] The training dataset is used to train the model and / or system 100 to identify when real-time data exhibits anomalies. In some embodiments, the training dataset is generated by the HPCT computer device 510 and / or processing unit 104 (in... Figure 1 (As shown in the image) is for training purposes.

[0037] For example, HPCT computer device 510 receives multiple datasets for training a model. HPCT computer device 510 extracts 202 the raw data into datasets. For each dataset, HPCT computer device 510 attempts to classify the data 204 as normal or anomalous. If the dataset is anomalous, then HPCT computer device 510 determines 208 whether the anomalousness in the dataset originates from the tool being analyzed or from another source (e.g., but not limited to, human error or noise). If the source is not the tool, then the dataset is not further analyzed. If the dataset is considered normal or anomalous due to the tool, then HPCT computer device 510 segments the data 210 according to a time series, such that the dataset has a specific time size or a specific number of data points. HPCT computer device 510 calibrates 212 the datasets to ensure that the dataset size and format match the size and format of other datasets. HPCT computer device 510 stores 214 the data. In some embodiments, HPCT computer device 510 performs cluster analysis on the datasets. For example, after calibration 212 and alignment of multiple datasets, HPCT computer device 510 performs 216 exploratory data analysis (which includes data clusters) on the multiple datasets. HPCT computer device 510 cleans up 218 noisy data and then generates more than 220 training datasets based on the stored dataset.

[0038] Figure 3 This indicates the use of system 100 (in Figure 1 The flowchart of example procedure 300 (shown in the example embodiment). In the example embodiment, the steps of procedure 300 are performed by a hierarchical prediction and composite thresholding (HPCT) computer device 510 (in the example embodiment). Figure 5 (as shown in the diagram) to be executed. Procedure 300 includes steps using a dataset to classify the data as to whether it contains anomalies. In an example embodiment, system 100 is trained and procedure 300 is for analyzing real-time or real-time data from one or more sensors or measuring devices associated with the tool to be analyzed. In some embodiments, steps 304 to 314 of procedure 300 are for using the data obtained through procedure 200 (in the diagram). Figure 2 The training dataset (shown in the image) is used to train system 100.

[0039] HPCT computer device 510 acquires 302 real-time datasets from at least one of the sensors and / or measuring devices of the monitoring tool. In some embodiments, the real-time dataset includes data after the tool has been used. In other embodiments, the real-time dataset includes data before and after the tool has been used.

[0040] HPCT computing device 510 calibrates 304 and aligns with the real-time dataset. This ensures that time blocks used for each dataset contain the same amount of time and information from the same sensors. In some embodiments, HPCT computing device 510 divides the time in the dataset to align with the time blocks used in the training dataset. For example, if the training dataset contains one-minute time blocks, then HPCT computing device 510 also adjusts the real-time dataset to one-minute time blocks. In different embodiments, different data collection sets may collect data along different timelines. HPCT computing device 510 aligns the dataset with the desired time series.

[0041] HPCT computing device 510 performs a 306 anomaly prediction model on the real-time dataset to determine whether the dataset contains safe or anomalous data based on the trained model. Next, HPCT computing device 510 generates a 308 sliding time window for each element in the real-time dataset. HPCT computing device 510 appends information that appeared before the dataset within a predetermined time period. For example, HPCT computing device 510 may append information that appeared 30 minutes, two hours, or six days before the real-time dataset to be analyzed. This transforms the discrete data in the dataset into continuous data.

[0042] The HPCT computing device 510 generates a 310 random probability distribution curve to analyze a dataset with an upper bound. The random probability distribution curve uses random probabilities to generate an upper bound for detection purposes. When data in the dataset exceeds the generated upper bound, the HPCT computing device 510 marks the corresponding data as an anomaly. The random probability distribution curve determines the upper limit for each time point analyzed. This line is used as a composite threshold to determine whether the data is safe or contains anomalies.

[0043] During training, the HPCT computer device 510 provides 312 feedback and experience learning to compare predicted results with actual data. For example, it checks whether the dataset contains anomalies or whether the HPCT computer device 510 detects said anomalies, or whether the HPCT computer device 510 predicts anomalies in the security data. During real-time data processing, the HPCT computer device 510 uses past data to determine whether an alert should be issued. In at least one embodiment, the HPCT computer device 510 compares the number of alerts issued during the current time period with the number of alerts issued in previous time periods. If the current number of alerts is greater than or equal to the previous number of alerts, then the HPCT computer device 510 issues an actual alert. Otherwise, the HPCT computer device 510 stores the analysis results of the dataset and continues with the next real-time dataset.

[0044] HPCT computing device 510 then displays the prediction results at 314. For example, HPCT computing device 510 captures 302 a real-time dataset containing anomalies. HPCT computing device 510 calibrates 304 the real-time dataset to include a 30-second time segment used to match other time segments used by system 100 during training. This may involve waiting to capture more data at 302, using previously captured data, and / or trimming some data from the real-time dataset to calibrate 304 the real-time dataset used for the model. Next, HPCT computing device 510 executes 306 the anomaly prediction model. The model is trained using the training dataset from procedure 200.

[0045] HPCT computer device 510 generates a time sliding window 308 for the real-time dataset. For example, the time sliding window may include data from thirty minutes prior to the real-time dataset. HPCT computer device 510 calculates a random probability distribution curve to determine a dynamic threshold for prediction to produce an effective prediction curve. HPCT computer device 510 compares the real-time dataset with the random probability distribution curve. If the real-time dataset does not exceed the random probability distribution curve at any point, the data is considered normal. If the real-time dataset exceeds the random probability distribution curve at any point, HPCT computer device 510 issues an alert. HPCT computer device 510 compares the current number of alerts within the current time period with the number of alerts in previous time periods. If the current number of alerts reaches or exceeds the previous number, a full alert is issued and 314 is displayed to the user.

[0046] When the HPCT computer device 510 trains the model, it compares the alert triggers with the actual data to determine if any anomalies exist. If an anomaly exists, the HPCT computer device 510 marks it as a success. If no anomaly exists, the HPCT computer device 510 marks the event as a false alarm and uses the feedback and experience learning component 124 to update the model.

[0047] Figure 4 This indicates the use of system 100 (in Figure 1 The flowchart of example procedure 400 for predicting data (shown in the example embodiment). In the example embodiment, the steps of procedure 400 are performed by a hierarchical prediction and composite thresholding (HPCT) computer device 510 (in the example embodiment). Figure 5 (as shown in the diagram) to be executed. Program 400 includes functions for displaying steps as shown in step 314 (in the diagram). Figure 3 The steps described in the diagram (shown in the diagram) are for predicting the results.

[0048] HPCT computer device 510 receives 402 prediction results, for example in program 300 (in Figure 3The HPCT computer device 510 determines whether the prediction result (as shown in the diagram) is abnormal. If so, the HPCT computer device 510 reports the abnormality (as shown in the diagram) to the user (as shown in the diagram). The HPCT computer device 510 also visualizes the prediction result (as shown in the diagram) for the user (as shown in the diagram). In an example embodiment, the prediction result is visualized for the user by displaying it on a chart in a dashboard on a webpage, for example... Figure 8 As described herein. In other embodiments, depending on the user's preferences, the prediction results may be displayed to the user in a variety of different ways.

[0049] Figure 5 This is a simplified block diagram of an example system 500 for training and using machine learning that incorporates hierarchical prediction and composite thresholding. In the example embodiment, system 500 is used to analyze tool wear to determine when tool maintenance is required. Additionally, system 500 is a real-time data analysis and classification computer system that includes a hierarchical prediction and composite thresholding (HPCT) computer device 510 (also referred to as an HPCT server) configured to analyze tool performance and predict future states based on the analysis.

[0050] In an example embodiment, sensor 525 is configured to receive input regarding the current status of one or more tools on the assembly line. In some embodiments, sensor 525 measures one or more attributes or characteristics of the product before and after the tool has been used. In other embodiments, sensor 525 measures one or more attributes or characteristics of the tool itself. Sensor 525 is connected to HPCT computer device 510 via various wired or wireless interfaces, including (but not limited to) networks such as local area networks (LANs) or wide area networks (WANs), dial-up connections, cable modems, internet connections, wireless, and special high-speed integrated services digital network (ISDN) lines. Sensor 525 receives data regarding the surface of a wafer and reports the data to HPCT computer device 510. In other embodiments, sensor 525 communicates with one or more user computer devices 505, and user computer devices 505 route measurement data to HPCT computer device 510 in real time or near real time. In some embodiments, a first sensor 525 measures a product attribute before tooling and a second sensor 525 measures the same attribute of the product after tooling.

[0051] As described in more detail above, the HPCT computer device 510 is programmed to receive sensor data from the tool in real time to predict when the tool may malfunction, allowing system 500 to respond to changes that could cause problems with the final product. The HPCT computer device 510 is programmed to: 1) receive current data; 2) detect at least one anomaly based on the current data; 3) generate a time window for the anomaly; 4) generate a random probability distribution curve for the time window; 5) compare the time window with the random probability distribution curve; and 6) notify the user if at least one anomaly exceeds the random probability distribution curve.

[0052] In an example embodiment, user computer device 505 is a computer containing a web browser or software application that enables user computer device 505 to communicate with HPCT computer device 510 using the Internet, a local area network (LAN), or a wide area network (WAN). In some embodiments, user computer device 505 is communicatively coupled to the Internet through a variety of interfaces, including (but not limited to) at least one of a network (e.g., the Internet, LAN, WAN, or Integrated Services Digital Network (ISDN)), dial-up connection, digital subscriber line (DSL), cellular phone connection, satellite connection, and cable modem. User computer device 505 can be any device capable of accessing a network (e.g., the Internet), including (but not limited to) desktop computers, laptop computers, personal digital assistants (PDAs), cellular phones, smartphones, tablet computers, tablet phones, or other network-based connectable devices.

[0053] Database server 515 is communicatively coupled to database 520, which stores data. In one embodiment, database 520 is a database containing historical data, model data, and sensor data. In some embodiments, database 520 is stored remotely on HPCT computer device 510. In some embodiments, database 520 is distributed. In an example embodiment, a person can access database 520 by logging into HPCT computer device 510 via user computer device 505.

[0054] Figure 6 Description of an embodiment of the present disclosure Figure 5 The example configuration of the client system shown is illustrated. User computer device 602 is operated by user 601. User computer device 602 may include (but is not limited to) sensor 525, HPCT computer device 510, and user computer device 505 (all within...). Figure 5(As shown in the diagram). User computer device 602 includes a processor 605 for executing instructions. In some embodiments, the executable instructions are stored in memory region 610. Processor 605 may include one or more processing units (e.g., in a multi-core configuration). Memory region 610 is any means that allows storage and retrieval of information (e.g., executable instructions and / or transaction data). Memory region 610 may include one or more computer-readable media.

[0055] User computer device 602 also includes at least one media output component 615 for presenting information to user 601. The media output component 615 is any component capable of conveying information to user 601. In some embodiments, the media output component 615 includes an output adapter (not shown), such as a video adapter and / or an audio adapter. The output adapter is operatively coupled to processor 605 and operatively coupled to an output device, such as a display device (e.g., a cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED) display, or "e-ink" display) or an audio output device (e.g., a speaker or headphones). In some embodiments, the media output component 615 is configured to present a graphical user interface (e.g., a web browser and / or client application) to user 601. For example, the graphical user interface may include an interface for viewing the analysis results of one or more tools. In some embodiments, user computer device 602 includes an input device 620 for receiving input from user 601. User 601 can use the input device 620 to (but is not limited to) select tools to view their analysis. For example, input device 620 may include a keyboard, pointing device, mouse, stylus, touch-sensitive panel (e.g., touch pad or touch screen), gyroscope, accelerometer, position detector, biometric input device, and / or audio input device. A single component (e.g., a touch screen) may serve as both an output device of media output component 615 and an input device 620.

[0056] User computer device 602 may also include components communicatively coupled to a remote device (e.g., HPCT computer device 510). Figure 5 The communication interface 625 shown in the figure. For example, the communication interface 625 may include a wired or wireless network adapter and / or a wireless data transceiver for use with a mobile telecommunications network.

[0057] For example, computer-readable instructions for providing a user interface to user 601 via media output component 615 and optionally receiving and processing input from input device 620 are stored in memory area 610. Among other possibilities, the user interface may include a web browser and / or client applications. A web browser enables a user (e.g., user 601) to display and interact with media and other information typically embedded in web pages or websites from HPCT computer device 510. Client applications allow user 601 to interact with, for example, HPCT computer device 510. For example, instructions may be stored via cloud services, and the output of instruction execution may be sent to media output component 615.

[0058] Processor 605 executes computer-executable instructions for implementing aspects of this disclosure. In some embodiments, processor 605 is transformed into a special-purpose microprocessor by executing computer-executable instructions or by being otherwise programmed.

[0059] Figure 7 Description of an embodiment of the present disclosure Figure 5 The server system instance configuration shown in the figure. Server computer device 701 may include (but is not limited to) HPCT computer device 510 and database server 515 (both are in Figure 5 (As shown in the diagram). The server computer device 701 also includes a processor 705 for executing instructions. Instructions may be stored in memory region 710. The processor 705 may include one or more processing units (e.g., in a multi-core configuration).

[0060] The processor 705 is operatively coupled to the communication interface 715, enabling the server computer device 701 to communicate with remote devices (e.g., another server computer device 701, another HPCT computer device 510, or a user computer device 505). Figure 5 (As shown in the diagram) communication. For example, communication interface 715 can receive requests from user computer device 505 via the Internet, such as... Figure 5 As explained in the text.

[0061] The processor 705 may also be operatively coupled to the storage device 734. The storage device 734 is adapted to store and / or retrieve data (e.g., but not limited to, with the database 520). Figure 5The storage device 734 is any computer-operated hardware associated with the data shown in the illustration. In some embodiments, the storage device 734 is integrated into the server computer device 701. For example, the server computer device 701 may include one or more hard disk drives as storage devices 734. In other embodiments, the storage device 734 is external to the server computer device 701 and is accessible by multiple server computer devices 701. For example, the storage device 734 may include a storage local area network (SAN), a network attached storage (NAS) system, and / or multiple storage units (e.g., hard disks and / or solid-state drives configured in a redundant array of low-cost disks (RAID).

[0062] In some embodiments, processor 705 is operatively coupled to storage device 734 via storage interface 720. Storage interface 720 is any component capable of providing processor 705 with access to storage device 734. For example, storage interface 720 may include an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and / or any component providing processor 705 with access to storage device 734.

[0063] Processor 705 executes computer-executable instructions for implementing aspects of this disclosure. In some embodiments, processor 705 is transformed into a special-purpose microprocessor by executing computer-executable instructions or by being otherwise programmed. For example, processor 705 is used with, for example... Figures 2 to 4 Programming instructions as described in the document.

[0064] Figure 8 This is used to illustrate the use of system 100 (in Figure 1 (As shown in the image) through programs 200, 300, and 400 (in...) Figures 2 to 4 Figure 800 shows an example of the results of the analysis performed (as shown in the figure). Figure 800 illustrates the predicted value 802 based on the model. Figure 800 also illustrates the results of the random probability distribution curve component 122 (in the figure). Figure 1 The system 100 generates a composite threshold 804 (as shown in the diagram). The system 100 compares the predicted value 802 with the composite threshold 804 to determine when an alarm should be triggered. In an example embodiment, the system 100 triggers an alarm when the predicted value 802 exceeds the composite threshold 804. In some other embodiments, the system 100 triggers an alarm when the predicted value 802 exceeds the composite threshold 804 a predetermined number of times within a predetermined time period. The predetermined number of times and the predetermined time period can be determined by the model based on training or by the user.

[0065] At least one of the technical solutions achieved through this system for solving technical problems may include: (i) improving the analysis of tool functionality; (ii) reducing material loss due to malfunction or improper tool maintenance; (iii) increasing the speed of tool analysis; (iv) improving the accuracy of tool analysis; (v) reducing unnecessary adjustments to the tool; and (vi) reducing false alarms and false negatives; and (vii) reducing overfitting errors.

[0066] The embodiments described herein relate to systems and methods for enhancing machine learning, and more specifically to enhancing machine learning for anomaly detection using hierarchical prediction and composite thresholding. More specifically, the anomaly detection analysis model is executed by a computing device to (1) determine the current state of the tool; (2) predict the future state of the tool based on the current state and the model; and (3) determine whether adjustments to the tool are needed based on the future state. The systems and methods described herein allow for tool status feedback in a shorter time, thereby enabling adjustments to improve tool maintenance with less lag time to achieve improved quality control and / or product yield. Computer systems, such as hierarchical prediction and composite thresholding (HPCT) computer devices and related computer systems, are described herein. As described herein, all such computer systems include processors and memory. However, any processor in a computer device mentioned herein may also refer to one or more processors, which may be in a single computing device or multiple computing devices operating in parallel. Furthermore, any memory in a computer device mentioned herein may also refer to one or more memories, which may be in a single computing device or multiple computing devices operating in parallel.

[0067] Furthermore, the computer systems discussed herein may include additional, fewer, or alternative functionalities (including those discussed elsewhere herein). The computer systems discussed herein may include or be implemented via computer-executable instructions stored on or via said computer-executable instructions on a non-transitory computer-readable medium.

[0068] In some embodiments, the system is designed to implement machine learning, enabling neural networks to "learn" to analyze, organize, and / or process data without explicit programming. Machine learning can be implemented using machine learning (ML) methods and algorithms. In embodiments, a machine learning (ML) module is configured to implement ML methods and algorithms. In some embodiments, ML methods and algorithms are applied to data inputs and produce machine learning (ML) outputs. Data inputs may include (but are not limited to): analog and digital signals (e.g., sound, light, motion, natural phenomena, etc.). Data inputs may further include: sensor data, image data, video data, and telematics data. ML outputs may include (but are not limited to): digital signals (e.g., information data converted from natural phenomena). ML outputs may further include: speech recognition, image or video recognition, medical diagnosis, statistical or financial models, processed signals, signal identification and recognition, autonomous vehicle decision-making models, robot behavior modeling, signal detection, fraud detection analysis, user input recommendation and personalization, game AI, skill acquisition, targeted marketing, big data visualization, weather forecasts, and / or information extracted from computer devices, users, households, vehicles, or trading parties. In some embodiments, the data input may include specific ML outputs.

[0069] In some embodiments, at least one of a plurality of ML methods and algorithms may be applied, which may include (but is not limited to): linear or logistic regression, example-based algorithms, normalization algorithms, deterministic trees, Bayesian networks, cluster analysis, association rule learning, artificial neural networks, deep learning, recurrent neural networks, Monte Carlo search trees, generative adversarial networks, dimensionality reduction, and support vector machines. In various embodiments, the implemented ML methods and algorithms are for at least one of a variety of classifications of machine learning, such as supervised learning, unsupervised learning, and reinforcement learning.

[0070] In one embodiment, the ML method and algorithm are for supervised learning, which involves identifying patterns in existing data to predict subsequently received data. Specifically, the ML method and algorithm for supervised learning are "trained" with training data containing instance inputs and associated instance outputs. Based on the training data, the ML method and algorithm can generate a prediction function that maps the output to the input and use the prediction function to produce ML outputs based on the data inputs. The instance inputs and instance outputs of the training data can include either the data inputs or ML outputs described above. For example, an ML module can receive training data including data associated with different received signals and their corresponding classifications, generate a model that maps the signal data to the classification data, and identify future signals and determine their corresponding categories.

[0071] In another embodiment, the ML method and algorithm are for unsupervised learning, which involves finding meaningful relationships in unorganized data. Unlike supervised learning, unsupervised learning does not involve user-initiated training based on instance inputs and associated outputs. In fact, in unsupervised learning, unlabeled data, which can be any combination of data inputs and / or ML outputs as described above, is organized according to relationships determined by the algorithm. In an embodiment, an ML module coupled to or communicating with a design system or integrated as a component of the design system receives unlabeled data (including event data, financial data, community data, geographic data, cultural data, signal data, and political data), and the ML module employs an unsupervised learning method (e.g., "clustering") to identify patterns and organize the unlabeled data into meaningful groups. For example, the newly organized data can be used to extract further information about potential classifications.

[0072] In another embodiment, the ML method and algorithm are for reinforcement learning, which involves optimizing the output based on feedback from a reward signal. Specifically, the ML method and algorithm for reinforcement learning may receive a user-defined reward signal definition, receive data input, utilize a decision model to generate an ML output based on the data input, receive a reward signal based on the reward signal definition and the ML output, and modify the decision model to receive a stronger reward signal for subsequent ML outputs. The reward signal definition may be based on either the data input or the ML output described above. In an embodiment, the ML module implements reinforcement learning in a user recommendation application. The ML module may utilize a decision model to generate a ranked list of options based on user information received from the user, and may further receive selection data based on the user's selection of one of the ranked options. A reward signal may be generated based on comparing the selection data with the ranking of the selected option. The ML module may update the decision model so that subsequent rankings more accurately predict optimal constraints.

[0073] The computer implementation methods discussed herein may include additional, fewer, or alternative actions (including those discussed elsewhere herein). The methods may be implemented via one or more local or remote processors, transceivers, servers, and / or sensors (e.g., processors, transceivers, servers, and / or sensors mounted on a vehicle or mobile device or associated with intelligent infrastructure or a remote server), and / or via computer-executable instructions stored on a non-transitory computer-readable medium.

[0074] As used herein, a processor may comprise any programmable system, including systems using microcontrollers, reduced instruction set circuitry (RISC), application-specific integrated circuits (ASICs), logic circuits, and any other circuitry or processor capable of performing the functions described herein. The examples above are for illustrative purposes only and are therefore not intended to limit the definition and / or meaning of the term "processor" in any way.

[0075] As used herein, the term "database" can refer to a data subject, a relational database management system (RDBMS), or both. As used herein, a database can contain any collection of data, including hierarchical databases, relational databases, flat document databases, object-relational databases, object-oriented databases, and any other structured collection of records or data stored in a computer system. The examples above are for illustrative purposes only and are therefore not intended to limit the definition and / or meaning of the term "database" in any way. Examples of RDBMS include (but are not limited to) […]. Database, MySQL DB2, SQL Server And PostgreSQL. However, any database that implements the systems and methods described herein can be used. (Oracle is a registered trademark of Oracle Corporation of Redwood Shore, California; IBM is a registered trademark of International Business Machines Corporation of Armonk, New York; Microsoft is a registered trademark of Microsoft Corporation of Redmond, Washington; and Sybase is a registered trademark of Sybase of Dublin, California).

[0076] In another embodiment, a computer program is provided, and the program is embodied on a computer-readable medium. In an exemplary embodiment, the system executes on a single computer system without needing to be connected to a server computer. In a further exemplary embodiment, the system... The system runs in a host environment (Windows is a registered trademark of Microsoft Corporation, Redmond, Washington). In yet another embodiment, the system operates in a host environment and It runs on a server environment (UNIX is a registered trademark of X / Open Ltd., located in Reading, Berkshire, UK). In other embodiments, the system is... The system runs on an environment (iOS is a registered trademark of Cisco Systems, Inc., located in San Jose, California). In yet another embodiment, the system is on a Mac. The system runs on an environment (Mac OS is a registered trademark of Apple Inc., located in Cupertino, California). In yet another embodiment, the system is... Running on an OS (Android is a registered trademark of Google, Inc., Mountain View, California). In another embodiment, the system is... It runs on an OS (Linux is a registered trademark of Linus Torvalds, Boston, Massachusetts). The application is flexible and designed to run in a variety of environments without compromising any major functionality. In some embodiments, the system includes multiple components distributed across multiple computing devices. One or more components are in the form of computer-executable instructions embodied in a computer-readable medium. The system and programs are not limited to the specific embodiments described herein. Furthermore, the components of each system and each program may be practiced independently and separately from other components and programs described herein. Each component and program may also be used in conjunction with other assemblies, packages, and programs.

[0077] As used herein, a component or step described in the singular and beginning with the word "a" should be understood to not exclude multiple components or steps unless such exclusion is explicitly stated. Furthermore, references to "exemplary embodiments" or "an embodiment" of the invention are not intended to be construed as excluding or incorporating the existence of additional embodiments with described features.

[0078] As used herein, the terms “software” and “firmware” are interchangeable and include any computer program that is stored in memory (including RAM, ROM, EPROM, EEPROM, and non-volatile RAM (NVRAM) memory) and executed by a processor. The memory types described above are for illustrative purposes only and therefore do not limit the types of memory that can be used to store computer programs.

[0079] Furthermore, as used herein, the term "real-time" refers to at least one of the following: the time when the associated event occurs, the time when predetermined data is measured and collected, the time when the data is processed, and the time when the system responds to the event and the environment. In the embodiments described herein, these activities and events occur substantially instantaneously.

[0080] The systems and programs are not limited to the specific embodiments described herein. Furthermore, components of each system and program may be practiced independently and separately from other components and programs described herein. Each component and program may also be used in conjunction with other assemblies, packages, and programs.

[0081] Furthermore, the computer systems discussed herein may include additional, fewer, or alternative functionalities (including those discussed elsewhere herein). The computer systems discussed herein may include or be implemented via computer-executable instructions stored on or via said computer-executable instructions on a non-transitory computer-readable medium.

[0082] Processors or processing components can be trained using supervised or unsupervised machine learning, and the machine learning program can employ neural networks, which can be convolutional neural networks, deep learning neural networks, reinforcement learning modules or programs, or combined learning modules or programs that have learned in two or more domains of interest. Machine learning may involve identifying and recognizing patterns in existing data to facilitate prediction of subsequent data. Models can be created based on instance inputs to make effective and reliable predictions on novel inputs.

[0083] Alternatively or concurrently, machine learning programs can be trained by inputting sample datasets or specific data (e.g., images, object statistics and information, historical estimates, and / or actual repair costs) into the program. Machine learning programs can utilize deep learning algorithms that can primarily focus on pattern recognition and can be trained after processing multiple instances. Machine learning programs may individually or in combination incorporate Bayesian program learning (BPL), speech recognition and synthesis, image or object recognition, optical character recognition, and / or natural language processing. Machine learning programs may also incorporate natural language processing, semantic analysis, automated reasoning, and / or machine learning.

[0084] Both supervised and unsupervised machine learning techniques can be used. In supervised machine learning, the processing component is provided with instance inputs and their associated outputs, and seeks to discover general rules that map the inputs to the outputs, so that when given subsequent novel inputs, the processing component can accurately predict the correct output based on the discovered rules. In unsupervised machine learning, the processing component may need to find its own structural context in unlabeled instance inputs. In one embodiment, machine learning techniques can be used to extract data about tool wear and use to predict future states.

[0085] Based on these analyses, the processing component can learn how to identify characteristics and patterns, which can then be applied to analyze image data, model data, and / or other data. For example, the processing component can learn to identify trends prior to tool misalignment or other problems by comparing measurements before and after tooling. The processing component can also learn how to identify trends that may not be easily apparent, such as trends prior to tool misalignment, based on collected scan data.

[0086] The methods and systems described herein can be implemented using computer programming or engineering techniques that include computer software, firmware, hardware, or any combination or subset thereof. As disclosed above, at least one technical problem with prior art systems is the need for a cost-effective and reliable system for analyzing data to predict future tool status and performance. The systems and methods described herein address the aforementioned technical problem. Furthermore, at least one of the technical solutions provided by the system to overcome the technical problem may include: (i) improving tool functionality analysis; (ii) reducing material loss due to malfunction or improper tool maintenance; (iii) increasing the speed of tool analysis; (iv) improving the accuracy of tool analysis; (v) reducing unnecessary tool adjustments; and (vi) reducing false alarms and false negatives; and (vii) reducing overfitting errors.

[0087] The described methods and systems can be implemented using computer programming or engineering techniques including computer software, firmware, hardware, or any combination or subset thereof, wherein the technical effect is achieved by performing at least one of the following steps: a) receiving multiple real-time datasets from one or more sensors associated with the tool to be analyzed; b) calibrating the multiple real-time datasets; c) generating a time sliding window for each of the multiple real-time datasets; d) generating a random probability distribution curve; e) comparing the random probability distribution curve with each time sliding window to determine whether the time sliding window contains anomalous data; f) generating a prediction result based on the comparison, wherein the prediction result indicates potential future problems of the tool to be analyzed; g) performing an anomaly prediction model on each of the multiple real-time datasets to determine whether the real-time dataset contains anomalous data; h) comparing the determination that the real-time dataset contains anomalous data with the determination that the corresponding time sliding window contains anomalous data; i) generating a prediction result based on the comparison of the two determinations; j) training the anomaly prediction model using multiple training datasets; k) removing anomalous data by removing data with anomalous characteristics. The process involves: l) extracting multiple raw datasets (where anomalies are not associated with the measured tool) and generating multiple training datasets by removing noisy data; m) classifying the multiple raw datasets as normal or anomalous; n) for each anomalous dataset, determining whether the observed anomaly is associated with the observed tool and another source; o) if the observed anomaly is associated with another source, then removing the corresponding anomalous dataset; p) aligning the remaining multiple datasets to match time periods; q) cleaning up any noisy datasets; r) generating multiple training datasets using the remaining multiple datasets; s) performing data clustering on the remaining multiple datasets to determine one or more relationships with the remaining multiple datasets; t) aligning multiple real-time datasets; u) adjusting the amount of time in each of the real-time datasets to equal a predetermined amount of time; v) adjusting each real-time dataset to include a predetermined amount of time; w) adjusting to include a predetermined number of data points from one or more sensors; x) generating a time sliding window by combining each real-time dataset with real-time data within a predetermined time period preceding the corresponding real-time dataset; and y) issuing an alert when a future problem is detected.

[0088] The method may be implemented via one or more local or remote processors, transceivers, servers, and / or sensors (e.g., processors, transceivers, servers, and / or sensors mounted on a vehicle or mobile device or associated with smart infrastructure or a remote server), and / or via computer-executable instructions stored on a non-transitory computer-readable medium. Furthermore, the computer system discussed herein may include additional, fewer, or alternative functionalities (including those discussed elsewhere herein). The computer system discussed herein may include or be implemented via computer-executable instructions stored on a non-transitory computer-readable medium.

[0089] As used herein, the term "non-transitory computer-readable media" is intended to mean any tangible computer-based device implemented with any method or technique for the short-term and long-term storage of information (e.g., computer-readable instructions, data structures, program modules and submodules, or other data in any device). Therefore, the methods described herein can be encoded as executable instructions embodied in tangible, non-transitory computer-readable media (including, but not limited to, storage devices and / or memory devices). When executed by a processor, such instructions cause the processor to perform at least a portion of the methods described herein. Furthermore, as used herein, the term "non-transitory computer-readable media" includes all tangible, computer-readable media, including (but not limited to) non-transitory computer storage devices, including (but not limited to) volatile and non-volatile media, and removable and non-removable media, such as solid-state, physical and virtual memory, CD-ROMs, DVDs, and any other digital source (e.g., networks or the Internet), and undeveloped digital components, with the sole exception of transient, propagating signals.

[0090] This written description uses examples to disclose various embodiments (including the best mode) and to enable those skilled in the art to practice these embodiments (including making and using any apparatus or system and performing any incorporated methods). The patentability of this disclosure is defined by the claims and may include other examples that will be apparent to those skilled in the art. Such other examples are intended to be within the scope of the claims, provided that they have structural components that are not different from the literal language of the claims, or that contain equivalent structural elements that are not substantially different from the literal language of the claims.

[0091] When describing elements of this disclosure or embodiments thereof, the articles “a” and “described” are intended to mean that one or more of the elements are present. The terms “comprising,” “including,” “containing,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to those listed. Terms indicating a particular orientation (e.g., “top,” “bottom,” “side,” etc.) are used for convenience of description and do not require any particular orientation of the described items.

[0092] Since various changes can be made to the above construction and method without departing from the scope of this disclosure, all matters contained in the above description and shown in the accompanying drawings should be interpreted as illustrative and non-limiting.

Claims

1. A computer apparatus comprising at least one processor in communication with at least one memory device, wherein the at least one processor is programmed to: Receive multiple real-time datasets from one or more sensors associated with the tool to be analyzed, wherein each of the multiple real-time datasets includes multiple sensor data from the one or more sensors; The multiple real-time datasets are calibrated to ensure that the time blocks of each dataset contain the same amount of time and information. An anomaly prediction model is executed on the calibrated real-time dataset to determine whether the dataset contains safe data or anomalous data based on the trained model; Generate a time sliding window for a real-time dataset containing anomalous data, wherein the time sliding window includes a time subset within the corresponding real-time dataset and sensor data from the one or more sensors during the time subset; The upper bound for the time sliding window is generated using a random probability distribution curve; For each point in the time sliding window, the upper limit of the random probability distribution curve is compared with the sensor data of the time sliding window to issue an alert, wherein the alert is issued based on the sensor data exceeding the upper limit; Compare the number of warnings issued for the time sliding window with the number of warnings issued since the previous time period; as well as An alarm is displayed if the number of warnings issued for the time sliding window is greater than or equal to the number of warnings issued since the previous time period.

2. The computer apparatus of claim 1, wherein the at least one processor is further programmed to train the anomaly prediction model using multiple training datasets.

3. The computer apparatus of claim 2, wherein the at least one processor is further programmed to generate the plurality of training datasets by removing datasets with anomalies and by removing noisy data, wherein the anomalies are not associated with the measured tool.

4. The computer apparatus of claim 3, wherein the at least one processor is further programmed to: Extract multiple raw datasets; The multiple original datasets are classified as normal or abnormal. For each anomalous dataset, determine whether the observed anomaly is associated with the observed tool and another source; If the observed anomaly is associated with another source, then remove the corresponding anomaly dataset; Align the remaining datasets to match time periods; Clean up any noisy datasets; as well as The remaining datasets are used to generate the multiple training datasets.

5. The computer apparatus of claim 4, wherein the at least one processor is further programmed to perform data clustering on the remaining plurality of datasets to determine one or more relationships with the remaining plurality of datasets.

6. The computer apparatus of claim 1, wherein the at least one processor is further programmed to: Align with the multiple real-time datasets; and The time amount in each of the real-time datasets is adjusted to be equal to the predetermined time amount.

7. The computer apparatus of claim 6, wherein the at least one processor is further programmed to adjust each real-time dataset to include a predetermined amount of time.

8. The computer apparatus of claim 6, wherein the at least one processor is further programmed to adjust each real-time dataset to include a predetermined number of data points from the one or more sensors.

9. The computer apparatus of claim 1, wherein the at least one processor is further programmed to generate the time sliding window by combining each real-time dataset with real-time data over a predetermined time period preceding the corresponding real-time dataset.

10. A method for an analysis tool, the method being implemented on a computer device including at least one processor communicating with at least one memory device, wherein the method comprises: Receive multiple real-time datasets from one or more sensors associated with the tool to be analyzed, wherein each of the multiple real-time datasets includes multiple sensor data from the one or more sensors; The multiple real-time datasets are calibrated to ensure that the time blocks of each dataset contain the same amount of time and information. An anomaly prediction model is executed on the calibrated real-time dataset to determine whether the dataset contains safe data or anomalous data based on the trained model; Generate a time sliding window for a real-time dataset containing anomalous data, wherein the time sliding window includes a time subset within the corresponding real-time dataset and sensor data from the one or more sensors during the time subset; The upper bound for the time sliding window is generated using a random probability distribution curve; For each point in the time sliding window, the upper limit of the random probability distribution curve is compared with the sensor data of the time sliding window to issue an alert, wherein the alert is issued based on the sensor data exceeding the upper limit; Compare the number of warnings issued for the time sliding window with the number of warnings issued since the previous time period; as well as An alarm is displayed if the number of warnings issued for the time sliding window is greater than or equal to the number of warnings issued since the previous time period.

11. The method of claim 10, further comprising: The anomaly prediction model is trained using multiple training datasets, wherein the multiple training datasets are generated as follows: Extract multiple raw datasets; The multiple original datasets are classified as normal or abnormal. For each anomalous dataset, determine whether the observed anomaly is associated with the observed tool and another source; If the observed anomaly is associated with another source, then remove the corresponding anomaly dataset; Align the remaining datasets to match time periods; Clean up any noisy datasets; as well as The remaining datasets are used to generate the multiple training datasets.

Images