A continuous learning model hot updating method for industrial product defect detection

Abstract

This application provides a continuous learning model hot update method for defect detection in industrial products, relating to the field of industrial manufacturing technology. This method aims to solve the problem of production interruption caused by the need for updates to existing defect detection models. Its core lies in: first, achieving adaptive intelligent triggering of hot updates by calculating the drift between real-time data and historical defect distributions online. Then, after triggering, a lightweight sub-branch running parallel to the main network is launched. This sub-branch uses forward propagation only, without backpropagation, for efficient fine-tuning, greatly reducing computational overhead. Finally, the updated sub-branch is integrated into the main network to obtain a new model, which is then rigorously evaluated and compared with the old model. Model switching is only completed when performance meets the standards; otherwise, automatic rollback occurs. This allows for continuous, safe, and efficient online learning and iteration of the defect detection model without interrupting production.

Description

Technical Field

[0001] This application relates to the field of industrial manufacturing technology, and in particular to a hot update method for a continuous learning model for defect detection in industrial products. Background Technology

[0002] In the industrial manufacturing sector, with the continuous advancement of industrial upgrading, products are developing towards precision and customization, which places higher demands on the real-time performance and accuracy of defect detection in the production process. In modern industrial manufacturing, product manufacturing and defect detection are usually carried out simultaneously to achieve immediate detection and handling of product quality anomalies. Currently, mainstream defect detection models generally adopt a cold update strategy when dealing with new defect types or changes in production data distribution. This requires interrupting the production process, completing offline data collection, model retraining, and deployment verification before restarting the detection service. This update method has significant drawbacks in continuous, tightly coupled industrial production environments: First, any stoppage in the production line will cause a complete shutdown, resulting in direct capacity loss; second, the equipment reset, system restart, and parameter calibration processes after production stoppage will further consume additional time and resources. Furthermore, as the production process continues, new product defect types will constantly emerge, requiring the detection model to have rapid adaptability. Traditional stoppage update modes, due to their long cycles and slow response times, are unable to meet the real-time requirements of modern industry for production efficiency and quality control. Therefore, there is an urgent need in this field for a hot update method that enables online learning and real-time iteration of defect detection models without interrupting production or stopping the machine, so as to ensure the continuity, efficiency and quality control of industrial manufacturing processes. Summary of the Invention

[0003] This application provides a hot update method for continuous learning models for defect detection in industrial products, aiming to solve the technical problem that defect detection models are difficult to continuously adapt to new defects in continuous industrial manufacturing processes, and that traditional update methods require production interruption for offline upgrades.

[0004] This application provides a hot update method for a continuous learning model for defect detection in industrial products, including:

[0005] Historical data is acquired and distribution drift is detected to obtain a quantitative indicator of distribution drift. The quantitative indicator of distribution drift is then compared with a preset threshold to determine whether a hot update process is triggered.

[0006] When the quantitative index of distribution drift exceeds a preset threshold, a hot update process is triggered, a lightweight sub-branch is added and enabled, the sub-branch runs in parallel with the main network, and is updated in a way that only propagates forward and does not require backpropagation.

[0007] The updated sub-branch is integrated into the main network to obtain the updated model;

[0008] Calculate the performance index of the updated model and compare it with the performance benchmark. If the performance index is better than or equal to the performance benchmark, replace the old model with the updated model. The performance benchmark is the performance index of the old model, and the old model is the original main network without sub-branches.

[0009] Furthermore, historical data is acquired and distribution drift is detected to obtain a quantitative indicator of distribution drift. This quantitative indicator is then compared with a preset threshold to determine whether a hot update process should be triggered. Specifically, this includes:

[0010] During the stable operation phase of the industrial product defect detection model, historical defective products are collected and their deep features are extracted. The feature mean vector of these deep features is then calculated. With covariance matrix Establish a baseline feature distribution ;

[0011] For each batch of data arriving online Extract the batch data The batch features are determined, and the mean vector of the batch features is calculated. Calculate the mean vector Compared with the baseline characteristic distribution Mahalanobis distance between , as a quantitative indicator of distribution drift;

[0012] When the Mahalanobis distance of multiple consecutive batches of defect samples Exceeding the preset threshold When a significant distribution drift is detected in the production environment, a hot update process is automatically triggered.

[0013] Furthermore, the Mahalanobis distance The calculation formula is:

[0014] ;

[0015] In the formula, This is the matrix transpose operation.

[0016] Furthermore, updating the sub-branch using only forward propagation and without back propagation includes:

[0017] Weight the secondary branch This represents the product of two low-rank matrices. The first low-rank matrix , Second low-rank matrix ,and , For the real number space, min It is a minimum value function. and The original matrix dimension, Let be the rank of the low-rank matrix after decomposition, and let be the first low-rank matrix. Random Gaussian initialization, second low-rank matrix Initially, it is an all-zero matrix;

[0018] In the current parameters A set of disturbances sampled from the surrounding area Generate candidate parameters ; where the current parameter It contains all the trainable parameters of the secondary branch, including the first low-rank matrix. Second low-rank matrix All elements in;

[0019] For each candidate parameter Perform forward propagation on batches of online data and compute entropy minimization loss;

[0020] Based on entropy minimization loss, the parameters are updated in the direction of the disturbance with the lowest loss.

[0021] Furthermore, the formula for calculating the entropy minimization loss is as follows:

[0022] ;

[0023] In the formula, To minimize the loss due to entropy, For batch data, For the model to class The predicted probability, Indicates the current batch of data Samples from the middle Find the expected value. For a single sample, This is a logarithmic operator.

[0024] Furthermore, based on the entropy minimization loss, the calculation process of updating parameters in the perturbation direction with the lowest loss is expressed as follows:

[0025] ;

[0026] In the formula, For the updated parameters, The parameters before the update. For learning rate, For the number of samples, For baseline loss, The standard deviation of the loss. Candidate parameters Minimize the entropy loss. i Represents the perturbation in the current sample. N This represents the number of samples.

[0027] Furthermore, the merging process of integrating the updated secondary branch into the main network can be represented as follows:

[0028] ;

[0029] In the formula, The main network weights after hot update. Main network weights, This is the first low-rank matrix. It is the second low-rank matrix.

[0030] Furthermore, the secondary branch is a low-rank adaptive module that runs parallel to the main network.

[0031] Furthermore, the method for determining whether the performance indicator is better than or equal to the performance benchmark is as follows:

[0032] ;

[0033] In the formula, For the performance metrics of the updated model, As a performance benchmark, This is the tolerance threshold.

[0034] Furthermore, the performance indicators include at least one of the following:

[0035] Classification accuracy metrics include recall, precision, or F1 score;

[0036] Indicators of forecast uncertainty include average forecast entropy or forecast confidence.

[0037] Model inference efficiency metrics include average inference time per sample.

[0038] The continuous learning model hot update method for defect detection in industrial products provided in this application has at least the following beneficial effects:

[0039] 1) This application implements a precise and adaptive model update triggering mechanism. Through online distribution drift detection based on Mahalanobis distance, the system can intelligently identify concept drift in the production process and trigger hot updates only when there is a substantial change in data distribution. This mechanism effectively overcomes the resource waste caused by traditional fixed-cycle or blind updates, ensuring the model's keen perception and timely response to changes in operating conditions, and guaranteeing the necessity and effectiveness of each update from the source.

[0040] 2) This application establishes a highly efficient and low-cost online learning capability. By combining a forward propagation optimization strategy with a low-rank adaptive module, model fine-tuning is completed without relying on backpropagation. This method significantly reduces the high memory and computational overhead of traditional gradient descent methods, making it possible to achieve real-time, lightweight model hot updates on resource-constrained industrial edge devices, greatly improving the system's practicality and deployment value in real industrial environments.

[0041] 3) This application establishes a secure and reliable version management and risk control system. Through reparameterization merging technology and a rigorous performance evaluation window mechanism, it ensures seamless integration and robust performance during model updates. The system can automatically and quickly revert to a stable version when the updated model's performance fails to meet standards, minimizing the risk of production accidents caused by model iterations. This closed-loop security system provides a reliable and trustworthy intelligent hot-update solution for continuous industrial production environments. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] Figure 1 A technical roadmap for a continuous learning model hot update method for defect detection in industrial products, provided in the embodiments of this application;

[0044] Figure 2 This is a flowchart illustrating a continuous learning model hot update method for industrial product defect detection, provided in an embodiment of this application.

[0045] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0047] The collection, storage, use, processing, transmission, provision, and disclosure of relevant data and information in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0048] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0049] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0050] In the industrial manufacturing sector, with the increasing precision and customization of products, defect detection faces severe challenges. Currently, mainstream detection models must resort to offline updates when faced with new defects or changes in data distribution, leading to production line shutdowns, significant capacity losses, and failing to meet the real-time quality control requirements of modern industry. While the industry recognizes the importance of hot updates, existing methods face key issues in real-world industrial scenarios, including high computational overhead, high update risks, and difficulty in guaranteeing stability. To overcome these shortcomings, this application provides a hot update method for a continuous learning model for industrial product defect detection. By constructing an intelligent triggering and secure update mechanism, the model can continuously learn and rapidly adapt to new defect types without interrupting production, effectively solving the downtime problems caused by traditional cold update methods.

[0051] like Figure 1 The diagram shown illustrates the technical roadmap for a continuous learning model hot-update method for industrial product defect detection, provided in this application. This method, based on a decoupled master-slave network structure and a distributed drift sensing mechanism, enables online continuous learning and seamless hot-update of the defect detection model in industrial scenarios. The overall technical process is as follows:

[0052] First, during the stable operation phase of the model, the system collects historical defect features to establish a baseline distribution, which serves as a reference for subsequent judgments on changes in data distribution. During actual operation, the system performs online distribution drift calculations on the real-time generated online data, quantifying the difference between the current data distribution and the baseline distribution.

[0053] Based on the distribution drift calculation results, the system performs a hot update trigger judgment: if the distribution drift does not exceed the preset threshold, the judgment is negative and the system returns to continue monitoring; if the distribution drift is significant and exceeds the threshold, the judgment is positive and the hot update process is automatically triggered.

[0054] After a hot update is triggered, the system constructs a low-rank adaptive sub-branch, which serves as a lightweight online learning module running in parallel with the main network. Next, the system fine-tunes the sub-branch through forward propagation, employing a forward-only strategy to optimize its parameters and avoid the high computational overhead of traditional backpropagation.

[0055] The system continuously determines whether the fine-tuning has converged: if it has not converged, it is judged as no, and the fine-tuning process continues; if it has converged, it is judged as yes, and the sub-branch reparameterization merging is performed, integrating the optimized sub-branch knowledge into the main network to form an updated unified model.

[0056] The updated model enters the evaluation phase, where the system verifies whether the model's performance meets the standards: if the performance is better than or equal to the old model, it is judged as yes, and the new model is launched to complete the hot update; if the performance does not meet the standards, it is judged as no, and the old version is automatically rolled back to ensure the absolute reliability of the production system.

[0057] This method, through the aforementioned technical approach, enables the industrial defect detection model to continuously learn and adapt to new defect types without interrupting production, effectively improving the long-term adaptability and operational efficiency of the industrial quality inspection system.

[0058] For details, please refer to Figure 2 The diagram shown is a flowchart of a continuous learning model hot update method for industrial product defect detection provided in an embodiment of this application. The continuous learning model hot update method for industrial product defect detection includes the following steps S10-S40.

[0059] S10: Acquire historical data and perform distribution drift detection to obtain a quantitative indicator of distribution drift. Then, compare the quantitative indicator of distribution drift with a preset threshold to determine whether to trigger the hot update process.

[0060] The purpose of step S10 is to achieve intelligent perception and decision-making. This step continuously monitors production data to determine whether the model needs to be updated, thereby achieving on-demand triggering and avoiding the blindness caused by fixed-period updates. This saves computing resources and ensures that the model responds promptly to environmental changes.

[0061] In some embodiments, step S10 establishes a mechanism for real-time perception of changes in data distribution through a statistical hypothesis prior method, providing an accurate trigger signal for model hot updates, avoiding unnecessary computational overhead and potential risks from blind hot updates. Step S10 specifically includes the following steps (not shown in S101-S103).

[0062] S101: Historical Distribution Modeling. During the stable operation phase of the industrial product defect detection model, historical defective products are collected and their deep features are extracted, and their feature mean vector is calculated. With covariance matrix To establish a baseline feature distribution .

[0063] S102: Online distribution drift calculation. For each batch of data arriving online. Extract the batch features and calculate the mean vector of the batch features. Subsequently, its characteristic distribution is calculated compared with the baseline. Mahalanobis distance between As a quantitative indicator of distribution drift:

[0064] ;

[0065] In the formula, This is the matrix transpose operation.

[0066] This distance metric fully considers the correlation between the various dimensions of the features and is more sensitive to changes in distribution.

[0067] S103: Update trigger judgment. When the Mahalanobis distance of multiple consecutive batches of defective samples... Exceeding the preset threshold When a significant distribution drift is detected in the production environment, the system automatically triggers a hot update process. This continuous judgment mechanism effectively avoids false triggering caused by anomalies in a single batch of tests.

[0068] S20: When the quantitative index of distribution drift exceeds the preset threshold, a hot update process is triggered, and a lightweight sub-branch is started. The sub-branch runs in parallel with the main network and is updated in a way that only propagates forward and does not require backpropagation.

[0069] The purpose of step S20 is to achieve efficient and low-cost learning. This step is responsible for rapidly absorbing new knowledge after triggering, and through forward propagation and lightweight sub-branches, it significantly reduces the computational overhead and memory consumption of hot updates, making real-time model updates possible on resource-constrained industrial edge devices.

[0070] In some embodiments, after a hot update is triggered, the system starts a lightweight sub-branch, which is a low-rank adaptive module running in parallel with the main network. This sub-branch is updated using a forward-only propagation method without backpropagation, significantly reducing computational overhead while ensuring update effectiveness. This enables the model to achieve hot updates in resource-constrained industrial environments. Specifically, the method of updating the sub-branch using forward-only propagation without backpropagation can be implemented as follows: S201-S202.

[0071] S201: Secondary branch parameter initialization. Secondary branch weights. This represents the product of two low-rank matrices, i.e. The first low-rank matrix , Second low-rank matrix ,and , For the real number space, min It is a minimum value function. and The original matrix dimension, Let be the rank of the low-rank matrix after decomposition. Initially, the first low-rank matrix... Random Gaussian initialization, second lowest rank moment The initial matrix is ​​all zeros to ensure that the performance of the main network is not affected in the early stages of merging.

[0072] S202: Forward Propagation Optimization. Parameters are optimized using a forward-adaptive strategy. Specifically, using an evolutionary strategy, with the current parameters... A set of disturbances sampled from the surrounding area Generate candidate parameters Current parameters It contains all the trainable parameters of the secondary branch, including the first low-rank matrix. Second low-rank matrix All elements in As a set of disturbances The perturbation parameters in the data. For each candidate parameter... Forward propagation is performed on batches of online data, and the entropy minimization loss is calculated using the following formula:

[0073] ;

[0074] In the formula, To minimize the loss due to entropy, For batch data, For the model to class The predicted probability, Indicates the current batch of data Samples from the middle Find the expected value. For a single sample, This is a logarithmic operator.

[0075] Finally, the parameter update is performed using the perturbation direction that results in the least loss:

[0076] ;

[0077] In the formula, For the updated parameters, The parameters before the update. For learning rate, For the number of samples, For baseline loss, The standard deviation of the loss. Candidate parameters Minimize the entropy loss. i Represents the perturbation in the current sample. N This represents the number of samples.

[0078] By using the steps S201 and S202 described above, backpropagation can be completely avoided, significantly reducing memory usage and computational overhead.

[0079] S30: Integrate the updated sub-branch into the main network to obtain the updated model.

[0080] The purpose of step S30 is to achieve seamless knowledge fusion. This step is responsible for safely integrating the newly learned knowledge into the main model. Through reparameterization merging, it achieves seamless and lossless integration of the sub-branch and the main network, forming a unified and efficient new model, eliminating the additional inference overhead brought by the parallel structure.

[0081] In some embodiments, the optimized sub-branch knowledge is seamlessly integrated into the main network to form an efficient and unified inference model, eliminating additional computational overhead. Regarding the main network weights... The merger process is as follows:

[0082] ;

[0083] In the formula, These are the main network weights after hot updates.

[0084] S40: Calculate the performance index of the updated model and compare it with the performance benchmark. If the performance index is better than or equal to the performance benchmark, replace the old model with the updated model. The performance benchmark is the performance index of the old model, and the old model is the original main network without sub-branches.

[0085] Step S40 serves to achieve safety verification and assurance. This step is responsible for ensuring the reliability of the update process. By establishing a performance evaluation and automatic rollback mechanism, it ensures that the version switch is only completed when the performance meets the standards, minimizing the risk of production accidents caused by model updates and ensuring the continuous and stable operation of the production line.

[0086] In some embodiments, step S40 may be implemented through the following steps S401-S402.

[0087] S401: The merged new model enters the preset evaluation window period, and the system monitors its performance indicators. This includes the average prediction entropy of online data and the recall rate for known defective samples. Meanwhile, the older model runs in parallel in the background as a performance benchmark. .

[0088] In some embodiments, the performance metrics include at least one of the following metrics:

[0089] Classification accuracy metrics include recall, precision, or F1 score;

[0090] Indicators of forecast uncertainty include average forecast entropy or forecast confidence.

[0091] Model inference efficiency metrics include average inference time per sample.

[0092] S402: If the new model performance indicators Stable and outperforms or equals the performance benchmark ,Right now ( If the system reaches a tolerance threshold, the new model will be officially launched. Otherwise, the system will automatically revert to the old model before the update, ensuring absolute reliability of the production environment. This mechanism ensures that version switching is only completed when performance meets the standards, minimizing the risk of hot updates.

[0093] In summary, the method described in this application has the following three major advantages:

[0094] (1) Hot update architecture based on decoupling of main and secondary branches: By decoupling the defect detection model into a main network that stably executes detection tasks and a lightweight secondary branch that is dedicated to online learning, the parallel execution of model updates and production tasks is realized at the structural level, thereby ensuring the continuous operation of the industrial production line and realizing true non-stop hot update.

[0095] (2) Data distribution-driven intelligent triggering mechanism: A distribution drift detection module based on online data perception was constructed, which can automatically and accurately identify the evolution and changes of defect patterns in the production environment, and make autonomous decisions to trigger the hot update process accordingly, so that the detection system has the ability to adapt and continuously iterate.

[0096] (3) Efficient and secure continuous learning and integration system: After the update is triggered, the system performs online fine-tuning of the sub-branch through the forward propagation optimization strategy, which greatly reduces the computational overhead. It also uses reparameterization technology to seamlessly merge the learning results into the main network, forming a closed-loop process from learning, integration to verification. Ultimately, it achieves continuous and reliable improvement of the defect detection model performance without interrupting production.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A continuous learning model hot update method for industrial product defect detection, characterized in that, include: Historical data is acquired and distribution drift is detected to obtain a quantitative indicator of distribution drift. The quantitative indicator of distribution drift is then compared with a preset threshold to determine whether a hot update process is triggered. When the quantitative index of distribution drift exceeds a preset threshold, a hot update process is triggered, a lightweight sub-branch is added and enabled, the sub-branch runs in parallel with the main network, and is updated in a way that only propagates forward and does not require backpropagation. The updated sub-branch is integrated into the main network to obtain the updated model; Calculate the performance index of the updated model and compare it with the performance benchmark. If the performance index is better than or equal to the performance benchmark, replace the old model with the updated model. The performance benchmark is the performance index of the old model, and the old model is the original main network without sub-branches. Historical data is acquired and distribution drift is detected to obtain a quantitative indicator of distribution drift. This quantitative indicator is then compared with a preset threshold to determine whether a hot update process should be triggered. Specifically, this includes: During the stable operation phase of the industrial product defect detection model, historical defective products are collected and their deep features are extracted. The feature mean vector of these deep features is then calculated. With covariance matrix Establish a baseline feature distribution ; For each batch of data arriving online Extract the batch data The batch features are determined, and the mean vector of the batch features is calculated. Calculate the mean vector Compared with the baseline characteristic distribution Mahalanobis distance between , as a quantitative indicator of distribution drift; When the Mahalanobis distance of multiple consecutive batches of defect samples Exceeding the preset threshold When a significant distribution drift is detected in the production environment, a hot update process is automatically triggered.

2. The method for hot updating of a continuous learning model for defect detection in industrial products according to claim 1, characterized in that, Mahalanobis distance The calculation formula is: In the formula, This is the matrix transpose operation.

3. The hot update method for continuous learning models for defect detection in industrial products according to claim 1, characterized in that, The methods for updating the sub-branch using only forward propagation and without back propagation include: Weight the secondary branch This represents the product of two low-rank matrices. The first low-rank matrix , Second low-rank matrix ,and , For the real number space, min It is a minimum value function. and The original matrix dimension, Let be the rank of the low-rank matrix after decomposition, and let be the first low-rank matrix. Random Gaussian initialization, second low-rank matrix Initially, it is an all-zero matrix; In the current parameters A set of disturbances sampled from the surrounding area Generate candidate parameters ; where the current parameter It contains all the trainable parameters of the secondary branch, including the first low-rank matrix. Second low-rank matrix All elements in; For each candidate parameter Perform forward propagation on batches of online data and compute entropy minimization loss; Based on entropy minimization loss, the parameters are updated in the direction of the disturbance with the lowest loss.

4. The hot update method for continuous learning models for defect detection in industrial products according to claim 3, characterized in that, The formula for calculating the entropy minimization loss is: In the formula, To minimize the loss due to entropy, For batch data, For the model to class The predicted probability, Indicates the current batch of data Samples from the middle Find the expected value. For a single sample, This is a logarithmic operator with a base of 2.

5. The method for hot updating of a continuous learning model for defect detection in industrial products according to claim 3, characterized in that, The calculation process of updating parameters in the perturbation direction with the lowest loss, based on entropy minimization loss, is expressed as follows: In the formula, For the updated parameters, The parameters before the update. For learning rate, For the number of samples, For baseline loss, The standard deviation of the loss. Candidate parameters Minimize the entropy loss. i Represents the perturbation in the current sample. N This represents the number of samples.

6. The method for hot updating of a continuous learning model for defect detection in industrial products according to claim 1, characterized in that, The process of integrating the updated secondary branch into the main network is represented as: In the formula, The main network weights after hot update. Main network weights, This is the first low-rank matrix. It is the second low-rank matrix.

7. The method for hot updating of a continuous learning model for defect detection in industrial products according to claim 1, characterized in that, The sub-branch is a low-rank adaptive module that runs parallel to the main network.

8. The method for hot updating of a continuous learning model for defect detection in industrial products according to claim 1, characterized in that, The method for determining whether the performance indicator is better than or equal to the performance benchmark is as follows: In the formula, For the performance metrics of the updated model, As a performance benchmark, This is the tolerance threshold.

9. The method for hot updating of a continuous learning model for defect detection in industrial products according to any one of claims 1 to 8, characterized in that, The performance indicators include at least one of the following: Classification accuracy metrics include recall, precision, or F1 score; Indicators of forecast uncertainty include average forecast entropy or forecast confidence. Model inference efficiency metrics include average inference time per sample.

Images