Drift monitoring apparatus for ai model and method thereof
The AI model drift monitoring device addresses the challenge of data drift detection by configuring data sets and applying statistical metrics to visualize and analyze drift, ensuring timely model updates and performance maintenance.
Patent Information
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- SK CO LTD
- Filing Date
- 2021-11-19
- Publication Date
- 2026-07-29
AI Technical Summary
Existing AI models face challenges in detecting data drift accurately and efficiently, leading to performance degradation, necessitating the need for precise methods to monitor and analyze data drift at the feature level.
An AI model drift monitoring device and method that configures reference and prediction data sets, applies drift evaluation metrics to calculate statistical differences, and adjusts monitoring settings based on analysis results to detect and visualize data drift.
The solution effectively identifies data drift by visualizing statistical differences and provides feature-level drift scores, enabling easy detection and analysis of factors contributing to drift, thereby facilitating timely model updates.
Smart Images
Figure 112021133713702-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to an AI model drift monitoring device and method for monitoring data drift over time of a distributed AI model, and in particular, to an AI model drift monitoring device and method for configuring a reference data set and a prediction data set for evaluating the performance of an AI model, and detecting data drift by comparing the statistical difference between the two data sets. Background Technology
[0002] Artificial Intelligence (AI) models are developed using various languages, data, and libraries, and there are many constraints in deploying and running these models to suit various usage environments.
[0003] Furthermore, the inference results of machine learning in AI models change subtly depending on the applied data and features, and data drift occurs over time. Here, data drift refers to the case where the predicted data generated by the AI model based on input data deviates from a preset range when compared to reference data—that is, when it exceeds a threshold.
[0004] If data drift occurs in an AI model, machine learning must be re-run using new training data to update the AI model to an optimized version.
[0005] Therefore, there is a need for research and development regarding methods to detect data drift in AI models and to analyze data drift more precisely at the feature level that constitutes the data. The problem to be solved
[0006] The present invention was created in consideration of the above circumstances, and the objective of the present invention is to provide an AI model drift monitoring device and method that configures a reference data set and a prediction data set to detect performance degradation of an AI model, and monitors data drift by comparing the statistical differences between the two data sets. means of solving the problem
[0007] An AI model drift monitoring device according to one embodiment of the present invention includes: a monitoring setting unit that selects and configures a reference data set and a prediction data set and sets an execution cycle for monitoring data drift of an AI model; a monitoring result analysis unit that analyzes the reference data set and the prediction data set according to the monitoring cycle and determines data drift based on whether there is a deviation from a preset reference range by comparing the statistical difference between the two data sets; and a monitoring execution modification unit that updates the setting and execution cycle of monitoring for data drift according to the analysis result of the monitoring result analysis unit.
[0008] In one embodiment, the monitoring result analysis unit can determine data drift by applying a drift evaluation metric, which is an algorithm that calculates the difference in probability distributions of data constituting the reference data set and the prediction data set.
[0009] In one embodiment, the monitoring setting unit may include a target feature setting unit that selects target features to be analyzed among the features constituting the data and sets a threshold value for each feature.
[0010] In one embodiment, the monitoring result analysis unit may include a feature analysis unit that monitors and analyzes the drift trend of the target features over time.
[0011] In one embodiment, the feature analysis unit may analyze categorical features by converting them into a probability distribution such that the sum of the total probabilities is 1, and numerical features by assuming them to a Gaussian distribution such that the sum of the total probabilities is 1, for the normalization of target features of a plurality of data constituting a data set.
[0012] In one embodiment, the monitoring execution modification unit may include a feature-specific threshold modification unit that resets an appropriate threshold for each feature according to the analysis result of the feature analysis unit.
[0013] In one embodiment, the monitoring setting unit may include an alarm setting unit that sets a Slack channel to notify information when, based on the analysis result of the monitoring result analysis unit, the predicted data deviates from a preset reference range and data drift occurs.
[0014] In one embodiment, the reference data set may be a training data set composed of data used when creating an AI model, or a historical data set composed of data input after serving the AI model.
[0015] A method for monitoring AI model drift according to one embodiment of the present invention comprises: (a) selecting and configuring a reference data set and a prediction data set, and setting an execution cycle for monitoring the data drift of an AI model; (b) analyzing the reference data set and the prediction data set according to the monitoring cycle, and determining data drift based on whether there is a deviation from a preset reference range by comparing the statistical difference between the two data sets; and (c) updating the setting and execution cycle of monitoring for data drift according to the determination result.
[0016] In one embodiment, the step (a) may include selecting target features to be analyzed among the features constituting the data and setting a threshold value for each feature.
[0017] In one embodiment, step (b) may include a step of monitoring and analyzing the drift trend of the target features over time.
[0018] In one embodiment, the step (b) may include, for normalization of target features of a plurality of data constituting a data set, converting categorical features into a probability distribution such that the sum of all probabilities is 1, and assuming numerical features into a Gaussian distribution such that the sum of all probabilities is 1.
[0019] In one embodiment, step (c) may include a step of resetting an appropriate threshold value for each feature according to the analysis result of step (b).
[0020] In one embodiment, step (a) may include the step of setting a Slack channel to notify information if, as a result of the analysis of step (b), the predicted data deviates from a preset reference range and data drift occurs.
[0021] In one embodiment, step (b) may include a step of determining data drift by applying a drift evaluation metric, which is an algorithm that calculates the difference in probability distributions of data constituting the reference data set and the prediction data set. Effects of the invention
[0022] The AI model drift monitoring device and method according to the present invention monitors data drift resulting from the degradation of the prediction performance of a distributed AI model, and has the effect of easily identifying data drift by visualizing the statistical difference between the two data sets as a graph after configuring a reference data set and a prediction data set.
[0023] In addition, the AI model drift monitoring device and method according to the present invention provides a feature-level drift score to analyze features that cause data drift, and monitors drift trends by feature to identify factors affecting data drift. Brief explanation of the drawing
[0024] FIG. 1 is a block diagram of an AI model drift monitoring device according to one embodiment of the present invention. Figure 2a is a diagram illustrating the distribution of data constituting the reference data set and the prediction data set, and Figure 2b is a graph showing the calculation result of applying KLD, a probability distribution difference calculation algorithm. FIGS. 3a and FIGS. 3b illustrate exemplary screens of the AI model's drift monitoring list. FIGS. 4 and FIGS. 5 illustrate exemplary screens for registering and modifying drift monitoring of an AI model. FIGS. 6 and 7 illustrate exemplary screens regarding the drift monitoring results of an AI model. FIGS. 8 and 9 illustrate exemplary screens regarding drift information of an AI model. FIG. 10 is a flowchart of an AI model drift monitoring method according to one embodiment of the present invention. Specific details for implementing the invention
[0025] Hereinafter, embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the present invention, if it is determined that a detailed description of related known components or functions could obscure the essence of the invention, such detailed description is omitted.
[0026] FIG. 1 is a block diagram of an AI model drift monitoring device according to one embodiment of the present invention.
[0027] Referring to FIG. 1, the AI model drift monitoring device (10) of the present invention includes a monitoring setting unit (100), a monitoring result analysis unit (200), and a monitoring execution modification unit (300).
[0028] The monitoring setting unit (100) selects and configures a reference data set and a prediction data set, and sets an execution cycle for monitoring data drift.
[0029] Here, the reference data set can be a training data set or a historical data set. The training data set can be the whole or a subset of training data used when creating an AI model, and the training data can be, for example, refined data from measurements detected by sensors for training the AI model.
[0030] The historical dataset can be the whole or a subset of actual data input after the AI model is deployed and served. Since it consists of data input in the 'past' at the time of running the monitoring analysis on the AI model, it is referred to as the historical dataset.
[0031] The prediction data set can be the whole or a subset of prediction data produced by the AI model based on historical data input after the AI model is served.
[0032] In one embodiment of the present invention, the monitoring setting unit (100) may include a target feature setting unit (110) that selects target features to be analyzed among the features constituting the data and sets a threshold value for each feature.
[0033] Additionally, the monitoring setting unit (100) may include an alarm setting unit (120) for setting a slack channel to notify information when, based on the analysis results of the monitoring result analysis unit (200), the predicted data deviates from a preset standard range and data drift occurs.
[0034] The monitoring result analysis unit (200) analyzes the reference data set and the prediction data set according to the monitoring cycle and compares the statistical difference between the two data sets to determine whether data drift occurs based on whether there is a deviation from a preset reference range.
[0035] When the reference data set is a training data set, the monitoring result analysis unit (200) compares the statistical difference between the training data set, which is the whole or subset of training data used to create the AI model, and the prediction data set, which is the whole or subset of prediction data produced by the AI model based on actual data (past data) input after serving the AI model, for example, measurements detected through a sensor.
[0036] When the reference data set is a past data set, the monitoring result analysis unit (200) sets the past data set, which is the whole or subset of past data input into the AI model before the monitoring is executed, as the reference data set, and then compares the prediction data set, which is the whole or subset of prediction data produced by the AI model based on past data input at a time after the reference data set (past data set) is set, with the reference data set (past data set).
[0037] The monitoring result analysis unit (200) determines data drift by applying a drift evaluation metric, which is an algorithm that calculates the difference in probability distributions of data constituting a reference data set and a prediction data set.
[0038] As an algorithm for calculating the difference in probability distributions, commercial Jensen Shannon Divergence (JSD) and / or Kullback-Leibler Divergence (KLD) may be used, but are not limited thereto.
[0039] The difference in probability distributions calculated through JSD or KLD can be displayed as a graph, and KLD has values from 0 to infinity, while JSD has values from 0 to 1.
[0040] While KLD does not satisfy the commutative law when calculating the difference in distribution between two data sets, KLD(A,B)≠KLD(B,A) and JSD satisfies the commutative law, so JSD(A,B)=JSD(B,A) and JSD is evaluated as a more stable calculation method.
[0041] In one embodiment of the present invention, the monitoring result analysis unit (200) may include a feature analysis unit (210) that monitors and analyzes the drift trend over time of target features set by the monitoring setting unit (100).
[0042] The feature analysis unit (210) can analyze the target features of multiple data constituting the data set by converting categorical features into a probability distribution such that the sum of the total probabilities is 1, and assuming numerical features into a Gaussian distribution such that the sum of the total probabilities is 1. The feature analysis unit (210) can analyze the drift of numerical features by sampling N, for example, 100, items from the estimated Gaussian distribution.
[0043] Categorical features refer to features whose values have a specific range, while numerical features refer to features whose values have a specific numerical value.
[0044] Additionally, the monitoring result analysis unit (200) may include a graphic processing unit (220) that visualizes the drift trend of data or features and displays it as a graph. The graphic processing unit (220) may display data values or feature values as a graph over time based on the monitoring cycle.
[0045] Additionally, the monitoring result analysis unit (200) may include a drift history lookup unit (230) that provides historical information on drifts recorded and stored for each monitoring period.
[0046] The monitoring execution modification unit (300) updates the settings and execution cycle of monitoring for data drift based on the analysis results of the monitoring result analysis unit (200). The settings and execution cycle of monitoring can be adjusted by considering the usage environment in which the AI model is applied, the possible frequency of data drift occurrence, and the severity of damage caused by data drift occurrence.
[0047] In one embodiment of the present invention, the monitoring execution modification unit (300) may include a feature-specific threshold modification unit (310) that resets an appropriate threshold value for each feature according to the analysis result of the feature analysis unit (210).
[0048] Since the impact on inducing data drift varies by feature constituting the data, different thresholds can be set for each feature to monitor drift trends, and an adjustment operation can be performed to reset the thresholds based on the monitoring results.
[0049] Figure 2a is a diagram illustrating the distribution of data constituting the reference data set and the prediction data set, and Figure 2b is a graph showing the calculation result of applying KLD, a probability distribution difference calculation algorithm.
[0050] Referring to FIG. 2a, A represents the distribution of data constituting a reference data set, and B represents the distribution of data constituting a prediction data set. The data of A and B can be displayed on an XY plane as in FIG. 2a based on the numerical values of the features constituting the data. The present invention determines whether data drift occurs by comparing the statistical difference between A and B.
[0051] Referring to Figure 2b, the results calculated by applying KLD to the reference data set and the prediction data set are illustrated. As a result of the calculation, C is a state where no data drift occurs because the result value appears within a preset threshold for data drift, and D is a state where data drift occurs because the result value appears outside the threshold.
[0052] FIGS. 3a and FIGS. 3b illustrate exemplary screens of the AI model's drift monitoring list.
[0053] Referring to FIG. 3a, a data drift monitoring list of a deployed AI model is displayed. The monitoring list may include, for example, the analysis cycle of the AI model, whether drift is occurring, the model status, reference data, the last execution time, and the next execution time information. A monitoring registration button (102) can navigate to a drift registration screen to monitor the AI model's drift.
[0054] The AI model shown in Fig. 3a represents an AI model registered in the drift monitoring list, and the drift status indicated as N means that no data drift occurred in the AI model.
[0055] Referring to Fig. 3b, it shows a case where the deployed AI model is not registered in the data drift monitoring list, and since it is not registered, a 'No data available' message may be displayed.
[0056] FIGS. 4 and FIGS. 5 illustrate exemplary screens for registering and modifying drift monitoring of an AI model.
[0057] Referring to FIG. 4, the monitoring description item (104) may contain the type, form, and specific details of the AI model's drift. The reference time item (106) sets the reference time for monitoring the AI model. When the reference data applied to the AI model is training data, the reference time, that is, the first execution date of monitoring, must be a point in time after one analysis cycle has elapsed from the service start time of the AI model so that monitoring can be executed at the same time interval.
[0058] The notification setting item (122) activates a Slack channel to send a notice when a value exceeding a threshold is detected as a result of the AI model's monitoring execution.
[0059] The analysis cycle item (108) sets the analysis cycle of AI model monitoring, for example, the left side of the drawing illustrates the basic method of setting the analysis cycle, and the right side illustrates the Corn expression.
[0060] The basic method allows setting the analysis cycle for AI model monitoring in daily, weekly, or monthly units. A Corn expression can be composed of a string consisting of unit expressions separated into six input items; each input item can contain detailed information regarding seconds, minutes, hours, days, months, and days of the week, with the input items separated by spaces. In other words, compared to the basic method, Corn expressions allow for setting the monitoring cycle with greater detail, down to the second, minute, and hour levels.
[0061] Referring to FIG. 5, the registration screen for AI model drift monitoring is shown, and the type of data applied to the machine learning of the AI model is selected in the reference data item (130).
[0062] If the selected data is training data, the path is automatically set, and schema information contained in the paths of the training data used to create the AI model—such as target features and feature types—is automatically retrieved and displayed on the screen.
[0063] To monitor AI model drift, a reference time—that is, the initial execution time of the monitoring—must be set. The reference time for monitoring can be reset by changing it.
[0064] The schema setting item (132) is automatically called and set when training data for the AI model is registered, and when training data is not registered, the schema can be set manually.
[0065] In one embodiment, for the automatic retrieval of training data of an AI model, when the AI model is moved to a modification screen and training data is registered, target features, feature types, etc. regarding the training data can be automatically retrieved in the schema setting item (132).
[0066] The schema setting item (132) can be divided into, for example, target feature, type, and threshold item, and the user can select a specific target feature and manually set the type and threshold. The data type can be, for example, numerical, categorical, etc., but is not limited thereto.
[0067] When the user checks the threshold batch setting item (134) and enters a % value, the corresponding % value is applied in batches to the threshold of the schema setting item (132).
[0068] In one embodiment of the present invention, a specific % value may be set as a threshold value by using the values of target features regarding the training data used to generate the AI model as reference values.
[0069] As a result of monitoring the AI model, if a result value exceeding a threshold of a specific target feature is detected, a notice may be sent to the Slack channel as described above.
[0070] FIGS. 6 and 7 illustrate exemplary screens regarding the drift monitoring results of an AI model.
[0071] Referring to FIG. 6, the drift information summary item (202) may include a monitoring ID, monitoring status, execution time, and drift information. The monitoring ID indicates the ID of the entity that performed the monitoring, and the monitoring status may indicate whether monitoring is enabled.
[0072] The presence of drift can be indicated as Y if the AI model detects data drift, and as N if it does not.
[0073] The top drift feature item (212) displays top features with large drift magnitudes and their values. The drawing is configured to display three top features, but is not limited to this, and various numbers of top features may be displayed in categories required by the user for AI model analysis.
[0074] The refresh button (204) updates and displays the AI model drift result information on the screen.
[0075] The drift trend item (222) can set the monitoring period, and the drift trend can be produced as a graph during the set period.
[0076] The feature selection item (214) displays a list of features registered for drift monitoring, and when a specific feature is selected, the drift trend for that feature can be generated as a graph.
[0077] Referring to FIG. 7, the graph area (224) illustrates the drift trend by feature selected through the feature selection item (214), and the change trend by date / time / feature is displayed. When a specific point in time is clicked on the graph, information can be displayed as a popup table regarding the feature at that point in time.
[0078] The drift history item (232) may display the drift size, threshold, and drifting status of all target features.
[0079] The user can modify the existing drift monitoring list by clicking the drift monitoring modification item (206).
[0080] FIGS. 8 and 9 illustrate exemplary screens regarding drift information of an AI model.
[0081] Referring to FIG. 8, the drift description item (302) displays the type of drift and the monitoring reference time.
[0082] The notification setting item (304) displays information about the Slack channel to send a notice when a value exceeding a threshold is detected as a result of running the monitoring, and can be configured to allow modification of the Slack channel information.
[0083] The analysis period item (306) sets the analysis period of drift, and the period can be set in daily / weekly / monthly units.
[0084] The reference data item (308) indicates the location where the training data is stored. For the accuracy of drift monitoring, the reference data cannot be modified, but the reference time for monitoring can be changed.
[0085] FIG. 9 shows drift information of an AI model, and when the AI model drift monitoring registration is executed in FIG. 5, a screen like FIG. 9 may be displayed. When the user clicks the registration modification button (312), the screen switches to a screen like FIG. 5, allowing the target feature, type, and threshold information to be changed.
[0086] FIG. 10 is a flowchart of an AI model drift monitoring method according to one embodiment of the present invention.
[0087] Referring to FIG. 10, according to the AI model drift monitoring method of the present invention, an AI model drift monitoring device (10) selects and configures a reference data set and a prediction data set, sets an execution cycle for monitoring the data drift of an AI model (S100), analyzes the reference data set and the prediction data set according to the monitoring cycle, determines the data drift based on whether there is a deviation from a preset reference range by comparing the statistical difference between the two data sets (S200), and updates the setting and execution cycle of monitoring for data drift according to the determination result of step S200 (S300).
[0088] In one embodiment of the present invention, the AI model drift monitoring device (10) can, in step S100, select target features to be analyzed among the features constituting the data and set a threshold value for each feature.
[0089] The degree to which each feature influences the induction of data drift varies, and the AI model drift monitoring device (10) can analyze the features that cause data drift based on the feature level drift score and monitor the drift trend for each feature to identify the factors influencing data drift.
[0090] The AI model drift monitoring device (10) can set up a Slack channel to notify information when, in step S100, the prediction data deviates from a preset reference range and data drift occurs as a result of the analysis in step S200. When a notification regarding data drift is transmitted through the Slack channel, follow-up measures for updating the AI model, such as executing machine learning of the AI model using new training data, can be taken.
[0091] The AI model drift monitoring device (10) can monitor and analyze the drift trend of target features over time in step S200, and furthermore, for normalization of the target features of multiple data constituting the data set, categorical features can be converted into a probability distribution such that the sum of the total probabilities is 1, and numerical features can be analyzed by assuming a Gaussian distribution such that the sum of the total probabilities is 1.
[0092] The AI model drift monitoring device (10) can reset an appropriate threshold value for each feature in step S300 based on the drift trend analysis results of the target features executed in step S200.
[0093] Here, a specific percentage value can be reset as a feature-specific threshold, using the values of target features regarding the training data used to generate the AI model as reference values. Since the sensitivity of data drift caused by each feature is different, the feature-specific threshold can be set differently for each.
[0094] The AI model drift monitoring device (10) can determine data drift in step S300 by applying a drift evaluation metric, which is an algorithm that calculates the difference in probability distributions of data constituting a reference data set and a prediction data set. As an algorithm for calculating the difference in probability distributions, commercial Jensen Shannon Divergence (JSD) and / or Kullback Leibler Divergence (KLD) may be used.
[0095] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.
[0096] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.
[0097] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.
[0098] Although the present invention has been described in detail with reference to preferred embodiments, the present invention is not limited to the above-described embodiments, and the technical concept of the present invention extends to the scope in which various modifications or alterations are possible by anyone with ordinary knowledge in the technical field to which the present invention belongs, without departing from the gist of the present invention as claimed in the following claims. Explanation of the symbols
[0099] 10: AI Model Drift Monitoring Device 100: Monitoring Settings 102: Registration Monitoring Button 104: Monitoring Description Item 106: Reference Time Item 108: Analysis Cycle Item 110: Target feature setting section 120: Alarm setting section 122: Alarm setting item 130: Reference data item 132: Schema Settings Item 134: Batch Threshold Settings Item 200: Monitoring Result Analysis Section 202: Drift Information Summary Item 204: Refresh button 206: Drift monitoring modification item 210: Feature Analysis Section 212: Top Drift Feature Items 214: Feature Selection Item 220: Graphics Processing Unit 222: Drift Trend Item 224: Graph Area 230: Drift History Lookup 232: Drift History Item 300: Monitoring Execution Modification Unit 302: Drift Description Item 304: Notification Settings Item 306: Analysis Cycle Item 308: Reference data item 310: Feature-specific threshold modification unit 312: Register Edit Button
Claims
Claim 1 An AI model drift monitoring device comprising: a monitoring setting unit that selects and configures a reference data set and a prediction data set and sets an execution cycle for monitoring data drift of an AI model; a monitoring result analysis unit that analyzes the reference data set and the prediction data set according to the monitoring cycle and determines data drift based on whether there is a deviation from a preset reference range by comparing the statistical difference between the two data sets; and a monitoring execution modification unit that updates the setting and execution cycle of monitoring for data drift according to the analysis result of the monitoring result analysis unit; wherein the monitoring setting unit selects target features to be analyzed among the features constituting the data and sets a threshold value for each feature, and the monitoring execution modification unit includes a feature-specific threshold value modification unit that resets an appropriate threshold value for each feature according to the analysis result of the feature analysis unit; wherein the monitoring result analysis unit determines that data drift has occurred if a result value appears when a feature deviates from the corresponding threshold value, and the monitoring setting unit displays whether the feature has been selected in response to a user input selecting features to monitor for data drift. Claim 2 An AI model drift monitoring device according to claim 1, wherein the monitoring result analysis unit determines data drift by applying a drift evaluation metric, which is an algorithm for calculating the difference in probability distributions of data constituting the reference data set and the prediction data set. Claim 3 delete Claim 4 An AI model drift monitoring device according to claim 1, characterized in that the monitoring result analysis unit includes a feature analysis unit that monitors and analyzes the drift trend of the target features over time. Claim 5 An AI model drift monitoring device according to claim 4, wherein the feature analysis unit is characterized by converting categorical features into a probability distribution such that the sum of all probabilities is 1 for normalization of target features of a plurality of data constituting a data set, and analyzing numerical features by assuming a Gaussian distribution such that the sum of all probabilities is 1. Claim 6 delete Claim 7 The AI model drift monitoring device according to claim 1, wherein the monitoring setting unit includes an alarm setting unit that sets a Slack channel to notify information when data drift occurs as a result of analysis by the monitoring result analysis unit, such that the predicted data deviates from a preset reference range. Claim 8 An AI model drift monitoring device according to claim 1, characterized in that the reference data set is a training data set composed of data used when creating an AI model, or a historical data set composed of data input after serving the AI model. Claim 9 (a) a step of selecting and configuring a reference data set and a prediction data set and setting an execution cycle for monitoring data drift of an AI model; (b) a step of analyzing the reference data set and the prediction data set according to the monitoring cycle and determining data drift based on whether there is a deviation from a preset reference range by comparing the statistical difference between the two data sets; and (c) a step of updating the setting and execution cycle of monitoring for data drift according to the determination result; wherein step (a) includes a step of selecting target features to be analyzed among the features constituting the data and setting a threshold value for each feature; step (c) includes a step of resetting an appropriate threshold value for each feature according to the analysis result of step (b); step (b) determines that data drift has occurred if a result value appears when a feature deviates from the threshold value; and step (a) is characterized by indicating whether each feature has been selected in response to a user input selecting features to monitor for data drift. Claim 10 delete Claim 11 A method for monitoring AI model drift according to claim 9, wherein the above step (b) includes a step of monitoring and analyzing the drift trend of the target features over time. Claim 12 A method for monitoring AI model drift according to claim 11, wherein the above step (b) comprises the step of converting categorical features into a probability distribution such that the sum of all probabilities is 1 for normalization of target features of a plurality of data constituting a data set, and assuming numerical features into a Gaussian distribution such that the sum of all probabilities is 1. Claim 13 delete Claim 14 A method for monitoring AI model drift according to claim 9, wherein the above step (a) includes the step of setting a Slack channel to notify information when, as a result of the analysis of the above step (b), the predicted data deviates from a preset reference range and data drift occurs. Claim 15 In claim 9, the above step (b) is characterized by including a step of determining data drift by applying a drift evaluation metric, which is an algorithm for calculating the difference in probability distributions of data constituting the reference data set and the prediction data set. Claim 16 A method for monitoring AI model drift according to claim 9, characterized in that the reference data set is a training data set composed of data used when creating an AI model, or a historical data set composed of data input after serving the AI model.