Artificial intelligence model serving framework and method

Abstract

An AI model serving framework and method are provided. The AI model serving method according to an embodiment of the present invention comprises selecting one from among a plurality of AI models so as to process received data when receiving data from devices constituting a system, wherein each of the plurality of AI models is executed in a plurality of sub-tasks driven in a single container. Therefore, even if there are numerous real-world devices in a Kubernetes environment in which one GPU is allocated to one container, an AI model-based service can be provided.

Description

Artificial Intelligence Model Serving Framework and Method

[0001] The present invention relates to AI model serving, and more particularly, to a framework and method for serving AI models required to predict the future and detect malfunctions using data from a real environment in a digital twin platform.

[0002] Digital twins are a technology that represents objects and systems identical to real-world objects and systems in a virtual space and synchronizes them with the real world. By synchronizing virtual objects and systems with real-world data collected to create digital twins, future data can be predicted or anomalies in virtual objects and systems can be detected.

[0003] Predicting future data or detecting anomalies utilizes AI models, and it's known that AI models are best implemented in container-based cloud systems. The problem arises when real-world systems, such as IoT systems, contain a vast number of data-providing devices, which inevitably leads to a large number of containers. Considering Kubernetes' resource environment, where one GPU is allocated per container, this makes building a digital twin platform impossible.

[0004] Meanwhile, to adapt to real-world data, AI models undergo periodic additional training. If performance deteriorates as a result of additional training, they are rolled back to the previous AI model or to a model from a different point in time in the past. However, this runs counter to the AI ​​model's updating process, so a method is needed to ensure that AI model updates are maintained even during rollbacks.

[0005] The present invention has been devised to solve the above problems, and the purpose of the present invention is to provide an AI model serving framework that can create multiple sub-tasks within a single container and have the sub-tasks execute different AI models, and a digital twin platform that applies the framework.

[0006] In addition, another object of the present invention is to provide a method for updating an existing AI model by generating a new AI model by weighting the weights of previous AI models and verifying their performance when rolling back an AI model.

[0007] An AI model serving method according to one embodiment of the present invention for achieving the above object includes: a step of receiving data from devices constituting a real-world system; a step of creating a virtual system replicating the real-world system in a virtual space using the received data; a step of selecting one of a plurality of AI models; and a step of processing the received data with the selected AI model, wherein the plurality of AI models are each executed in a plurality of sub-tasks running in a single container.

[0008] The processing step may be to predict future data or detect anomalies in the system using the selected AI model.

[0009] Subtasks can share a single GPU. While sharing common code within a single container, subtasks can run different AI models. Subtasks can have different functions. Depending on the device providing the data, subtasks can select AI models with different weight settings.

[0010] The AI ​​model serving method according to the present invention further includes a step of updating AI models by additional learning; and the update step may be to update the AI ​​model by mixing AI models from past points in time if the performance of the additionally learned AI model is lower than the performance of the AI ​​model before the additional learning.

[0011] The update step may involve selecting a set number of AI models in order of performance from among AI models from a past point in time, mixing the weights of the selected AI models to create next-generation AI models, and updating the AI ​​model with one of the generated next-generation AI models.

[0012] If the best performing AI model among the next generation of AI models outperforms the AI ​​model before further training, the AI ​​model can be updated, and if the performance is poor, the next generation of AI models can be generated.

[0013] According to another aspect of the present invention, a digital twin system is provided, comprising: a first computing module that receives data from devices constituting a real-world system and creates a virtual system replicating the real-world system in a virtual space using the received data; a second computing module that selects one of a plurality of AI models and processes the received data with the selected AI model; wherein the plurality of AI models are each executed in a plurality of sub-tasks running in a single container.

[0014] According to another aspect of the present invention, an AI model serving method is provided, comprising: a step of receiving data from devices constituting a system; a step of selecting one of a plurality of AI models; and a step of processing the received data with the selected AI model; wherein the plurality of AI models are each executed in a plurality of sub-tasks running in a single container.

[0015] According to another aspect of the present invention, an AI model serving system is provided, comprising: a first computing module for receiving data from devices constituting the system; and a second computing module for selecting one of a plurality of AI models and processing the received data with the selected AI model; wherein the plurality of AI models are each executed in a plurality of sub-tasks running in a single container.

[0016] As described above, according to embodiments of the present invention, by creating multiple sub-tasks within a single container and enabling the sub-tasks to execute different AI models, it becomes possible to provide AI model-based digital twin services even when there are a large number of real-world devices in a Kubernetes environment where one GPU is allocated to one container.

[0017] In addition, according to embodiments of the present invention, when an AI model is rolled back, a new AI model is created by weighting the weights of previous AI models, and the performance is verified to update the existing AI model, thereby enabling an update to a better AI model even when rolled back.

[0018] Figure 1 is a digital twin platform according to one embodiment of the present invention;

[0019] Figure 2 shows the structure and operation of the AI ​​model serving framework.

[0020] Figure 3 is a genetic algorithm-based AI model retraining method according to another embodiment of the present invention.

[0021] Hereinafter, the present invention will be described in more detail with reference to the drawings.

[0022] In an embodiment of the present invention, an AI model serving framework and method for implementing a digital twin platform utilizing an AI model are proposed in a case where data is generated from many devices, such as in an IoT environment.

[0023] This is a technology for efficiently implementing AI models in a Kubernetes environment that are used to improve the operation and performance of real-world systems, such as predicting the future or detecting system malfunctions early, using real-time data transmitted from the real-world environment in a digital twin.

[0024] In addition, an embodiment of the present invention proposes a method of updating an AI model by mixing AI models that had good performance at a past point in time when the AI ​​model was rolled back to create an AI model with improved performance.

[0025] FIG. 1 is a diagram illustrating a digital twin platform according to one embodiment of the present invention. The digital twin platform according to the embodiment of the present invention is implemented as a container-based cloud system and is configured to include a prediction POD (110), an anomaly detection POD (120), and a management POD (130).

[0026] The management POD (130) is a computing module that receives time series data in real time from a number of devices existing in an actual environment, preprocesses it, and creates containers necessary to replicate / visualize a virtual system based on the data to create a digital twin in a virtual space.

[0027] The prediction POD (110) is a computing module in which containers are created that provide AI models for predicting future data from time series data of devices preprocessed by the management POD (130).

[0028] The anomaly detection POD (120) is a computing module in which containers are created that provide AI models for detecting anomalies in a monitoring target system from time series data of IoT devices preprocessed by the management POD (130).

[0029] Containers created in the prediction POD (110) and anomaly detection POD (120) perform inference using AI models, AI model verification and performance evaluation, and additional learning. The right side of Fig. 1 illustrates the structure of the anomaly detection AI model serving framework built in one container of the anomaly detection POD (120).

[0030] As shown, in the anomaly detection POD (120), multiple sub-tasks are created in the container to provide / operate multiple AI models in a single container.

[0031] The structure and operation of the AI ​​model serving framework are illustrated in detail in Fig. 2. As shown on the left side of Fig. 2, a number of sub-tasks (value anomaly detection sub-task, RTT anomaly detection sub-task, RSSI anomaly detection sub-task) are created in the container to perform anomaly detection tasks.

[0032] Each subtask detects anomalies for different parameters (value, RTT, RSSI) and uses different AI models for anomaly detection. Specifically, 1) the value anomaly detection subtask uses a 2-layer LSTM, 2) the RTT anomaly detection subtask uses a 3-layer LSTM, and 3) the RSSI anomaly detection subtask uses a Transformer.

[0033] Additionally, sub-tasks created in a single container share common code for inference using AI models and for training and testing AI models, while selectively switching the code of the AI ​​model to be used to load the corresponding AI model in order to use different AI models.

[0034] Meanwhile, the AI ​​models executed by the sub-tasks are differentiated by device. For example, as illustrated on the right side of Figure 2, the AI ​​models used by the value anomaly detection sub-task to process data from Device 1, Device 2, and Device 3 are different.

[0035] However, what is different here is the weights of the AI ​​model, and the structure of the AI ​​model is the same. That is, the value anomaly detection sub-task processes data from devices 1, 2, and 3 with LSTM (2-layer), but the weights of LSTM (2-layer) are set differently depending on the type of device.

[0036] Although the AI ​​model structure is identical, the RTT anomaly detection subtask and RSSI anomaly detection subtask apply different weights depending on the device. Accordingly, the AI ​​model loaded by the subtask is distinguished by the sub_task name and device_name, as shown on the right side of Figure 2.

[0037] Meanwhile, sub-tasks share a single GPU, as they are created and run within a single container.

[0038] So far, the structure and operation of the AI ​​model serving framework within a single container of the anomaly detection POD have been described in detail. As described, the configuration in which sub-tasks within a single container share common code but utilize AI models of different types and structures, and even within the same sub-task, AI models with weights trained differently depending on the device, can be equally applied to the containers of the prediction POD (110).

[0039] Meanwhile, the subtask periodically performs additional training on the AI ​​models it uses to adapt to real-world data. However, additional training can sometimes degrade the AI ​​model's performance. In this case, the subtask modifies the AI ​​model's weights. Instead of reverting to a specific past weight, it generates new weights based on a genetic algorithm.

[0040] FIG. 3 is a diagram provided to explain a genetic algorithm-based AI model retraining method according to another embodiment of the present invention.

[0041] If the performance of the AI ​​model is degraded as a result of additional learning, as shown in the upper part of Fig. 3, J AI models with excellent performance are selected by verifying the N most recent past AI models with the current data, and the J selected first-generation AI models are sorted in order of performance.

[0042] As shown in the center of the following Figure 3, N second-generation AI models are created by combining the weights of J AI models. For example, a new AI model-AB is created by weighting the weights of AI model-A and AI model-B, and a new AI model-DRB is created by weighting the weights of AI model-D and AI model-E.

[0043] Afterwards, the N second-generation AI models generated are verified, J second-generation AI models with superior performance are selected, and the selected J second-generation AI models are sorted in order of performance.

[0044] As shown in the lower part of the following Figure 3, weights of J second-generation AI models are combined to create N third-generation AI models, J third-generation AI models with superior performance as a result of verification are selected, and the selected J third-generation AI models are sorted in order of performance.

[0045] After verification with the current data, if the best performing third-generation AI model outperforms the previous AI model, the AI ​​model is updated with the weights of the third-generation AI model. Conversely, if the best performing third-generation AI model underperforms the previous AI model, a fourth-generation AI model is created and the above process is repeated.

[0046] So far, in cases where data is generated from many devices, such as in an IoT environment, an AI model serving framework for implementing a digital twin platform utilizing AI models and a method for updating an AI model by mixing AI models with good performance from the past when rolling back an AI model have been described in detail with preferred embodiments.

[0047] In the above examples, multiple sub-tasks are created within a single container and the sub-tasks can execute different AI models, enabling the provision of AI model-based digital twin services even in a Kubernetes environment where one GPU is allocated to one container, even when there are a large number of real-world devices.

[0048] Additionally, when rolling back an AI model, a new AI model is created by weighting the weights of previous AI models, and its performance is verified to update the existing AI model, enabling it to be updated to a better AI model even when rolling back.

[0049] Meanwhile, it goes without saying that the technical idea of ​​the present invention can also be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.

[0050] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. A step of receiving data from devices that constitute a real-world system; A step of creating a virtual system that replicates a real-world system in a virtual space using received data; A step of selecting one of multiple AI models; A step of processing the received data with a selected AI model; Many AI models, An AI model serving method characterized in that each of the multiple sub-tasks is executed in a single container.

2. In claim 1, The processing steps are: An AI model serving method characterized by predicting future data or detecting system anomalies using a selected AI model.

3. In claim 1, Sub-tasks are, An AI model serving method characterized by sharing and using a single GPU.

4. In claim 1, Sub-tasks are, An AI model serving method characterized by being able to run different AI models while sharing common code in a single container.

5. In claim 4, Sub-tasks are, A method for serving AI models characterized by different functions.

6. In claim 5, Sub-tasks are, An AI model serving method characterized by selecting AI models with different weights set depending on the device that provided the data.

7. In claim 1, Further comprising a step of updating AI models by additional learning; The update steps are: An AI model serving method characterized in that, if the performance of an additionally learned AI model is lower than the performance of an AI model before additional learning, the AI ​​model is updated by mixing AI models from past points in time.

8. In claim 7, The update steps are: An AI model serving method characterized by selecting a predetermined number of AI models in order of performance from among AI models of a past point in time, generating next-generation AI models by mixing the weights of the selected AI models, and updating the AI ​​model with one of the generated next-generation AI models.

9. In claim 8, An AI model serving method characterized in that if the best performing AI model among the next generation AI models outperforms the AI ​​model before further learning, the AI ​​model is updated, and if the performance is poor, next generation AI models are generated.

10. A first computing module that receives data from devices that constitute a real-world system and creates a virtual system that replicates the real-world system in a virtual space using the received data; A second computing module that selects one of a plurality of AI models and processes the received data with the selected AI model; Many AI models, A digital twin system characterized by each executing a number of sub-tasks running in a single container.

11. Step of receiving data from devices that constitute the system; a step of selecting one of multiple AI models; and A step of processing the received data with a selected AI model; Many AI models, An AI model serving method characterized in that each of the multiple sub-tasks is executed in a single container.

12. A first computing module that receives data from devices constituting the system; and A second computing module that selects one of a plurality of AI models and processes the received data with the selected AI model; Many AI models, An AI model serving system characterized in that each sub-task is executed in a single container.

Images