Edge device having edge application that manages model

Abstract

An edge device having an edge application that manages a model mounted on the edge device, according to an embodiment of the present disclosure, may comprise: a communication interface; a memory that stores an artificial neural network-based determination model that infers whether or not a product or a component is defective; and a processor that collects data, acquires the inference result regarding whether or not the product or the component is defective, by using the collected data as an input into the determination model according to execution of the edge application, and transmits the data and the inference result to an edge conductor server through the communication interface.

Description

Edge devices equipped with edge applications that manage models

[0001] The present invention relates to an edge application for managing models suitable for the environment of edge devices used in manufacturing sites.

[0002] Machine Learning Operations (MLOps) is a portmanteau of Machine Learning (ML) and Operations (Ops), and is a technology that aims to deploy and maintain machine learning models reliably and efficiently.

[0003] In other words, MLOps refers to technologies that implement and automate continuous integration, continuous deployment, and continuous learning for machine learning systems. In manufacturing, MLOps can be used to train and create global models, enabling them to be deployed on edge devices.

[0004] At manufacturing sites, data is collected to train a judgment model that determines whether a product is good or bad. Edge devices are installed on each manufacturing line, allowing data to be collected for judgment model training.

[0005] Each edge device deployed on a production line producing the same product or part has a different environment. This means that factors such as noise, lighting, and humidity influx from each edge device's location may vary, resulting in differing data collected from each edge device.

[0006] However, conventional MLOps applied a single judgment model uniformly learned using data sets collected from edge devices to edge devices without considering the different environments of each edge device.

[0007] Accordingly, the problem of inference results of the model applied to each edge device becoming inaccurate may arise.

[0008] Additionally, conventional edge devices are limited to passive functions that output inference results using the onboard judgment model.

[0009] Accordingly, it was difficult to actively respond to the operating status or failure status of edge devices, and there was a problem that a lot of time and effort was required to apply a new judgment model even when the prediction accuracy of the judgment model was low.

[0010] The present invention aims to provide an edge application that enables edge computing without limitations on the hardware and operating system of an edge device.

[0011] The present invention aims to provide an edge application that can maintain a stable edge computing state by periodically transmitting the operating status or malfunction of an edge device to an external server.

[0012] The present invention aims to provide an edge application that transmits the inference results and input data of a judgment model installed in an edge device to an edge conductor server, thereby enabling the determination of the need for a new judgment model.

[0013] An edge device having an edge application for managing a model mounted on the edge device according to one embodiment of the present disclosure may include a communication interface, a memory for storing an artificial neural network-based judgment model for inferring whether a product or a component is defective, and a processor for collecting data, using data collected according to execution of the edge application as input to the judgment model, obtaining an inference result on whether the product or the component is defective, and transmitting the data and the inference result to an edge conductor server through the communication interface.

[0014] A model management method of an edge device having an edge application for managing a model according to an embodiment of the present invention may include a step of collecting data, a step of using data collected according to execution of the edge application as input to a judgment model for inferring whether a product or component is defective, a step of obtaining an inference result regarding whether the product or component is defective, and a step of transmitting the data and the inference result to an edge conductor server.

[0015] According to an embodiment of the present invention, edge computing is enabled without restrictions on the hardware and operating system of an edge device.

[0016] The present invention can maintain a stable edge computing state by periodically transmitting the operating status or failure status of an edge device to an external server.

[0017] The present invention enables rapid replacement of a judgment model with a lower accuracy installed in an edge device with a new judgment model, thereby maintaining the performance of the judgment model in an improved state.

[0018] Figure 1 illustrates an AI device according to one embodiment of the present disclosure.

[0019] FIG. 2 illustrates an AI server according to one embodiment of the present disclosure.

[0020] FIG. 3 is a diagram for explaining the configuration of an AI advisor system according to one embodiment of the present disclosure.

[0021] FIG. 4 is a flowchart illustrating an operation method of an AI conductor according to an embodiment of the present disclosure.

[0022] FIG. 5 is a drawing illustrating a specific configuration of an artificial intelligence system according to an embodiment of the present invention.

[0023] FIG. 6 is a diagram illustrating a process of measuring the prediction accuracy of each of a plurality of models learned by each of a plurality of algorithms according to one embodiment of the present disclosure.

[0024] FIG. 7 is a drawing showing an execution screen of an edge UI application according to one embodiment of the present invention.

[0025] FIG. 1 illustrates an AI device (100) according to one embodiment of the present disclosure.

[0026] The AI ​​device (100) can be implemented as a fixed device or a movable device, such as a TV, a projector, a mobile phone, a smart phone, a desktop computer, a laptop, an edge device for digital broadcasting, a PDA (personal digital assistant), a PMP (portable multimedia player), a navigation device, a tablet PC, a wearable device, a set-top box (STB), a DMB receiver, a radio, a washing machine, a refrigerator, a desktop computer, digital signage, a robot, a vehicle, etc.

[0027] Referring to FIG. 1, an edge device (100) may include a communication unit (110), an input unit (120), a learning processor (130), a sensing unit (140), an output unit (150), a memory (170), and a processor (180).

[0028] The communication unit (110) can transmit and receive data with external devices such as other AI devices (100a to 100e) or AI servers (200) using wired or wireless communication technology. For example, the communication unit (110) can transmit and receive sensor information, user input, learning models, control signals, etc. with external devices.

[0029] The communication unit (110) may be named a communication interface.

[0030] The communication technologies used by the communication unit (110) include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth (Bluetooth™), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

[0031] The input unit (120) can obtain various types of data.

[0032] The input unit (120) may include a camera for inputting a video signal, a microphone for receiving an audio signal, a user input unit for receiving information from a user, etc. Here, the camera or microphone may be treated as a sensor, and a signal obtained from the camera or microphone may be referred to as sensing data or sensor information.

[0033] The input unit (120) can obtain input data to be used when obtaining output using learning data and learning models for model learning. The input unit (120) can also obtain unprocessed input data, in which case the processor (180) or learning processor (130) can extract input features as preprocessing for the input data.

[0034] The learning processor (130) can train a model composed of an artificial neural network using learning data. Here, the trained artificial neural network may be referred to as a learning model. The learning model can be used to infer result values ​​for new input data other than the learning data, and the inferred values ​​can be used as a basis for making decisions regarding certain actions.

[0035] The running processor (130) can perform AI processing together with the running processor (240) of the AI ​​server (200).

[0036] The running processor (130) may include memory integrated or implemented in the AI ​​device (100). Alternatively, the running processor (130) may be implemented using memory (170), external memory directly coupled to the AI ​​device (100), or memory maintained in an external device.

[0037] The sensing unit (140) can obtain at least one of internal information of the AI ​​device (100), information about the surrounding environment of the AI ​​device (100), and user information using various sensors.

[0038] Sensors included in the sensing unit (140) include a proximity sensor, a light sensor, an acceleration sensor, a magnetic sensor, a gyro sensor, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone, a lidar, a radar, etc.

[0039] The output unit (150) can generate output related to vision, hearing, or touch.

[0040] The output unit (150) may include a display unit that outputs visual information, a speaker that outputs auditory information, a haptic module that outputs tactile information, etc.

[0041] The memory (170) can store data that supports various functions of the AI ​​device (100). For example, the memory (170) can store input data, learning data, learning models, learning history, etc. obtained from the input unit (120).

[0042] The processor (180) may determine at least one executable operation of the AI ​​device (100) based on information determined or generated using a data analysis algorithm or a machine learning algorithm. Then, the processor (180) may control components of the AI ​​device (100) to perform the determined operation.

[0043] To this end, the processor (180) can request, retrieve, receive or utilize data from the running processor (130) or memory (170), and control components of the AI ​​device (100) to execute at least one of the executable operations, a predicted operation or an operation determined to be desirable.

[0044] When the processor (180) requires connection to an external device to perform a determined operation, it can generate a control signal for controlling the external device and transmit the generated control signal to the external device.

[0045] The processor (180) can obtain intent information for user input and determine the user's requirements based on the obtained intent information.

[0046] The processor (180) can obtain intent information corresponding to the user input by using at least one of a STT (Speech To Text) engine for converting voice input into a string or a natural language processing (NLP) engine for obtaining intent information of natural language.

[0047] At least one of the STT engine or the NLP engine may be configured with an artificial neural network, at least in part, trained according to a machine learning algorithm. Furthermore, at least one of the STT engine or the NLP engine may be trained by the learning processor (130), the learning processor (240) of the AI ​​server (200), or through distributed processing thereof.

[0048] The processor (180) can collect history information including the operation details of the AI ​​device (100) or the user's feedback on the operation, and store the information in the memory (170) or the learning processor (130), or transmit the information to an external device such as an AI server (200). The collected history information can be used to update the learning model.

[0049] The processor (180) can control at least some of the components of the AI ​​device (100) to drive an application program stored in the memory (170). Furthermore, the processor (180) can operate two or more of the components included in the AI ​​device (100) in combination to drive the application program.

[0050] FIG. 2 illustrates an AI server according to one embodiment of the present disclosure.

[0051] Referring to FIG. 2, the AI ​​server (200) may refer to a device that trains an artificial neural network using a machine learning algorithm or utilizes a trained artificial neural network. Here, the AI ​​server (200) may be composed of multiple servers to perform distributed processing, and may be defined as a 5G network. The AI ​​server (200) may be included as part of the AI ​​device (100) and may perform at least a portion of the AI ​​processing.

[0052] The AI ​​server (200) may include a communication unit (210), memory (230), a learning processor (240), and a processor (260).

[0053] The communication unit (210) can transmit and receive data with an external device such as an AI device (100).

[0054] The memory (230) may include a model storage unit (231). The model storage unit (231) may store a model (or artificial neural network, 231a) being learned or learned through the learning processor (240).

[0055] A learning processor (240) can train an artificial neural network (231a) using learning data. The learning model can be used while mounted on the AI ​​server (200) of the artificial neural network, or can be mounted on an external device such as an AI device (100).

[0056] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (230).

[0057] The processor (260) can use a learning model to infer a result value for new input data and generate a response or control command based on the inferred result value.

[0058] FIG. 3 is a diagram for explaining the configuration of an artificial intelligence system according to one embodiment of the present disclosure.

[0059] Referring to FIG. 3, the AI ​​advisor system (30) may include a plurality of edge devices (100-1 embedded 100-n), an edge conductor server (200-1), and an AI conductor server (200-2).

[0060] In FIG. 3, the edge conductor server (200-1) or AI conductor server (200-2) is illustrated as one, but may be configured as two or more.

[0061] As the edge conductor server (200-1) is connected to multiple edge devices, an overall pyramid structure can be formed.

[0062] Each of the plurality of edge devices (100-1 to 100-n) may be an example of the AI ​​device (100) of FIG. 1. Each of the plurality of edge devices (100-1 to 100-n) may include all of the components of the AI ​​device (100) of FIG. 1.

[0063] Each of the plurality of edge devices (100-1 to 100-n) may be an inspection device placed on a production line at a manufacturing site. Each of the plurality of edge devices (100-1 to 100-n) may be a device for acquiring sensing data during the manufacturing process of a single product or component.

[0064] Each of the multiple edge devices (100-1 to 100-n) may have an edge application installed. The edge application can collect data suitable for creating a learning model and cleanse the collected data. This can improve data quality.

[0065] An edge application can transmit device information of an edge device, such as an edge device or inspection PC, to the edge conductor server (200-1). The device information of the edge device may include one or more of hardware specification information (CPU, memory capacity, storage capacity) and operating system information.

[0066] The edge conductor server (200-1) can receive multiple data sets and multiple device information from multiple edge devices.

[0067] The edge conductor server (200-1) can generate a plurality of modeling codes based on each of a plurality of data sets, and transmit the generated plurality of modeling codes and the plurality of data sets to the AI ​​conductor server (200-2).

[0068] The edge conductor server (200-1) can store Docker images in a Docker registry server from which they can be downloaded, or store Docker images itself.

[0069] The edge conductor server (200-1) can secure a Docker image that matches the hardware specification information of the edge device and transfer the Docker image to the edge device.

[0070] The Docker Registry server can be included in the edge conductor server (200-1).

[0071] The edge conductor server (200-1) can communicate with multiple edge devices (100-1 to 100-n) via a short-range wireless communication standard or Internet communication.

[0072] Each of the plurality of edge devices (100-1 to 100-n) can collect one or more of structured data or unstructured data.

[0073] Each of the multiple edge devices (100-1 to 100-n) may have one or more sensors connected.

[0074] Each of the plurality of edge devices (100-1 to 100-n) can receive one or more of structured data or unstructured data from a connected sensor and transmit the received data to the edge conductor server (200-1).

[0075] The edge conductor server (200-1) can transmit structured data and unstructured data received from each edge device to the AI ​​conductor server (200-2).

[0076] As another example, the edge conductor server (200-1) can generate transformed data by converting unstructured data into structured data. For example, the edge conductor server (200-1) can convert an acoustic signal into a power spectrum in the frequency domain through a Fourier transform.

[0077] In another embodiment, a data conversion unit for converting unstructured data into structured data may be provided in the AI ​​conductor server (200-2) instead of the edge conductor server (200-1).

[0078] The edge conductor server (200-1) can create a Docker image suitable for each edge device to support various types of hardware and operating systems of the edge device.

[0079] The edge conductor server (200-1) can generate a docker image that can be run on each edge device based on device information received from each of the plurality of edge devices (100-1 to 100-n).

[0080] That is, the edge conductor server (200-1) can generate a Docker image list including a plurality of Docker images corresponding to a plurality of edge devices (100-1 to 100-n).

[0081] The edge conductor server (200-1) and the AI ​​conductor server (200-2) can perform wireless communication.

[0082] The AI ​​conductor server (200-2) can generate a plurality of different models corresponding to each of a plurality of edge devices based on a plurality of modeling codes received from the edge conductor server (200-1).

[0083] The AI ​​conductor server (200-2) can transmit multiple generated models to the edge conductor server (200-1).

[0084] The edge conductor server (200-1) can transmit each of the plurality of models received from the AI ​​conductor server (200-2) to each of the plurality of edge devices (100-1 to 100-n).

[0085] Each of the plurality of edge devices (100-1 to 100-n) may be equipped with a different model received from the edge conductor server (200-1).

[0086] According to an embodiment of the present invention, a plurality of streams corresponding to each of a plurality of edge devices (100-1 to 100-n) may be constructed. That is, there may be a number of streams for each number of edges.

[0087] Each stream can be a series of processes for learning a base model and loading it onto an edge device through a base model applied to each edge device and parameters used in the base model.

[0088] If the base model applied to the edge device changes, the stream can be rebuilt with the changed base model and parameters.

[0089] Once the stream is set up, relabeling can be performed on the data collected by the edge device, and the base model can be retrained using the data and relabeled data.

[0090] FIG. 4 is a ladder diagram for explaining the operation method of an AI advisor system that generates and applies a model suitable for each edge device in a production site according to one embodiment of the present invention.

[0091] The model described below may be, but need not be limited to, an artificial neural network-based judgment model that determines whether a product or part is good or bad, learned using a deep learning or machine learning algorithm.

[0092] Referring to FIG. 4, the first edge device (100-1) can transmit the first device information and the first data set to the edge conductor server (200-1) (S401).

[0093] The first device information may include one or more of the operating system, memory capacity, CPU, and hardware information of the first edge device (100-1).

[0094] The first data set may be data collected by the first edge device (100-1).

[0095] In one embodiment, the processor (180) of the first edge device (100-1) can collect sensing data through the sensing unit (140).

[0096] In another embodiment, the processor (180) of the first edge device (100-1) can collect sensing data through a sensor that communicates with the first edge device (100-1).

[0097] The processor (180) of the first edge device (100-1) can transmit the first data set, which is sensing data collected through the communication unit (210), to the edge conductor server (200-1).

[0098] The first data set may include one or more of structured data or unstructured data.

[0099] Structured data can be a type of data that has a label that can determine pass (or OK) or fail (or NG), and unstructured data can be an unstructured type of data that cannot determine pass or fail.

[0100] Structured data can be a type of data that a person can visually judge as good or bad, and unstructured data can be a type of data, such as sound data, that a person cannot visually see.

[0101] Structured data can be either image data or sensor sensing value data.

[0102] Unstructured data can be audio data.

[0103] The edge conductor server (200-1) can generate a first modeling code to be used for modeling the model based on the received first data set (S403).

[0104] The first modeling code may be a code required for modeling a model to be mounted on the first edge device (100-1). The first modeling code may be a code indicating a base model to be applied to the first edge device (100-1) and parameters used in the base model.

[0105] The first modeling code may include multiple codes corresponding to each of the multiple algorithms. Each of the multiple algorithms may be used to train multiple models in parallel.

[0106] The processor (260) of the edge conductor server (200-1) can obtain a model suitable for the type of the first data set and obtain a first modeling code for modeling the obtained model.

[0107] The memory (230) of the edge conductor server (200-1) may store modeling codes corresponding to each of a plurality of data types.

[0108] The processor (260) of the edge conductor server (200-1) can obtain a first modeling code matching the first data set type from the memory (230).

[0109] The edge conductor server (200-1) can transmit the first data set and the generated first modeling code to the AI ​​conductor server (200-2) (S405).

[0110] The processor (260) of the edge conductor server (200-1) can transmit the first data set and the first modeling code to the AI ​​conductor server (200-2) via the communication unit (210).

[0111] The AI ​​conductor server (200-2) can select an available server by considering resources upon receiving the first data set and the first modeling code (S407).

[0112] The AI ​​Conductor server (200-2) may include multiple modeling servers. Each of the multiple modeling servers may generate multiple models in parallel. The multiple modeling servers may be provided separately from the AI ​​Conductor server (200-2).

[0113] The processor (260) of the AI ​​conductor server (200-2) can determine an available server capable of performing modeling by considering the resources of each of the multiple modeling servers.

[0114] The processor (260) of the AI ​​conductor server (200-2) can determine an available server by considering one or more resources of the memory capacity and CPU performance of each modeling server.

[0115] The AI ​​conductor server (200-2) can create multiple models using the first data set and the first modeling code through the selected available server (S409).

[0116] The processor (260) of the AI ​​conductor server (200-2) can transmit a control command to a determined available server to train multiple models using the first data set and the first modeling code. The available server can train multiple models in parallel using the first data set and the first modeling code according to the received control command.

[0117] Each of the multiple models may be a model that aims to infer as accurately as possible whether a product is good or bad by an edge device operating in the field.

[0118] Each of the multiple models may be an artificial intelligence-based model that receives one or more of the transformed data corresponding to the structured data or unstructured data for the product as input and infers whether the product is good or bad.

[0119] An available server can train multiple models in parallel using the same primary data set. Each of the multiple models can be trained using different algorithms or the same algorithm.

[0120] The AI ​​conductor server (200-2) can select the first model with the optimal performance among the multiple generated models (S411).

[0121] The processor (260) of the AI ​​conductor server (200-2) may select the model with the highest prediction accuracy among the multiple learned models as the first model with optimal performance. If the model is a judgment model, the prediction accuracy may indicate the rate at which a good or defective product is accurately classified.

[0122] The AI ​​conductor server (200-2) can transmit the selected first model to the edge conductor server (200-1) (S415).

[0123] The processor (260) of the AI ​​conductor server (200-2) can transmit the selected first model to the edge conductor server (200-1) through the communication unit (210).

[0124] The processor (260) of the AI ​​conductor server (200-2) can transmit the selected first model to the edge conductor server (200-1) to distribute the first model to the first edge device (100-1).

[0125] The processor (260) of the AI ​​conductor server (200-2) can transmit one or more of the parameters constituting the selected first model and the code for driving the first model to the edge conductor server (200-1) through the communication unit (210).

[0126] The edge conductor server (200-1) can generate a first docker image based on the received first model and first device information (S415).

[0127] The processor (260) of the edge conductor server (200-1) can generate a first docker image based on the first device information so that the first model can operate normally on the first edge device (100-1).

[0128] The first Docker image may include a software package for independent execution of edge applications mounted on edge devices.

[0129] The first Docker image can contain the files required to run the edge application, including the edge application's code, runtime, system tools, and libraries.

[0130] The edge conductor server (200-1) can transmit the first model and the first docker image to the first edge device (100-1) (S417).

[0131] The processor (260) of the edge conductor server (200-1) can transmit the first model and the first docker image to the first edge device (100-1) via the communication unit (210) to load the first model onto the first edge device (100-1).

[0132] The first edge device (100-1) can load the first model onto the first edge device (100-1) based on the received first docker image (S419).

[0133] The processor (180) of the first edge device (100-1) may be equipped with a first model optimized for the first edge device (100-1) based on the first docker image. The processor (180) of the first edge device (100-1) may control the execution of the first model through an edge application.

[0134] The edge application installed on the first edge device (100-1) can infer the judgment result of a product or component through the first model. The edge application installed on the first edge device (100-1) can transmit the inferred judgment result to the edge conductor server (200-1). The edge conductor server (200-1) can transmit the judgment result to the AI ​​conductor server (200-2).

[0135] The second edge device (100-2) can transmit second device information and a second data set to the edge conductor server (200-1) (S421).

[0136] The second device information may include one or more of the operating system, memory capacity, CPU, and hardware information of the second edge device (100-2).

[0137] The second data set may be data collected by the second edge device (100-2).

[0138] In one embodiment, the processor (180) of the second edge device (100-2) can collect sensing data through the sensing unit (140).

[0139] In another embodiment, the processor (180) of the second edge device (100-2) can collect sensing data through a sensor that communicates with the second edge device (100-2).

[0140] The processor (180) of the second edge device (100-2) can transmit the second data set, which is sensing data collected through the communication unit (210), to the edge conductor server (200-2).

[0141] The second data set may include one or more of structured data or unstructured data.

[0142] The edge conductor server (200-1) can generate a second modeling code to be used for modeling the model based on the received second data set (S423).

[0143] The second modeling code may be a code required for modeling a model to be mounted on the second edge device (100-2). The second modeling code may be a code indicating a base model to be applied to the second edge device (100-2) and parameters used in the base model.

[0144] The second modeling code may include multiple codes corresponding to each of the multiple algorithms. Each of the multiple algorithms may be used to train multiple models in parallel.

[0145] The processor (260) of the edge conductor server (200-1) can obtain a model suitable for the type of the second data set and obtain a second modeling code for modeling the obtained model.

[0146] The memory (230) of the edge conductor server (200-1) may store modeling codes corresponding to each of a plurality of data types.

[0147] The processor (260) of the edge conductor server (200-1) can obtain a second modeling code matching the second data set type from the memory (230).

[0148] The first modeling code and the second modeling code may be the same or different.

[0149] The edge conductor server (200-1) can transmit the second data set and the generated second modeling code to the AI ​​conductor server (200-2) (S425).

[0150] The processor (260) of the edge conductor server (200-1) can transmit the second data set and the second modeling code to the AI ​​conductor server (200-2) through the communication unit (210).

[0151] The AI ​​conductor server (200-2) can select an available server by considering resources upon receiving the second data set and the second modeling code (S427).

[0152] The AI ​​Conductor server (200-2) may include multiple modeling servers. Each of the multiple modeling servers may generate multiple models in parallel. The multiple modeling servers may be provided separately from the AI ​​Conductor server (200-2).

[0153] The processor (260) of the AI ​​conductor server (200-2) can determine an available server capable of performing modeling by considering the resources of each of the multiple modeling servers.

[0154] The processor (260) of the AI ​​conductor server (200-2) can determine an available server by considering one or more resources of the memory capacity and CPU performance of each modeling server.

[0155] The AI ​​conductor server (200-2) can create multiple models using the second data set and the second modeling code through the selected available server (S429).

[0156] The processor (260) of the AI ​​conductor server (200-2) can transmit a control command to a determined available server to train multiple models using a second data set and a second modeling code. The available server can train multiple models in parallel using the second data set and the second modeling code according to the received control command.

[0157] Each of the multiple models may be a model that aims to infer as accurately as possible whether a product is good or bad by an edge device operating in the field.

[0158] Each of the multiple models may be an artificial intelligence-based model that receives one or more of the transformed data corresponding to the structured data or unstructured data for the product as input and infers whether the product is good or bad.

[0159] An available server can train multiple models in parallel using the same second data set. Each of the multiple models can be trained using different algorithms or the same algorithm.

[0160] The AI ​​conductor server (200-2) can select a second model with optimal performance among the multiple generated models (S431).

[0161] The processor (260) of the AI ​​conductor server (200-2) may select the model with the highest prediction accuracy among the multiple trained models as the second model with optimal performance. If the model is a judgment model, the prediction accuracy may indicate the rate at which a good or defective product is accurately classified.

[0162] The AI ​​conductor server (200-2) can transmit the selected second model to the edge conductor server (200-1) (S435).

[0163] The processor (260) of the AI ​​conductor server (200-2) can transmit the selected second model to the edge conductor server (200-1) through the communication unit (210).

[0164] The processor (260) of the AI ​​conductor server (200-2) can transmit the selected second model to the edge conductor server (200-1) to distribute the second model to the second edge device (100-2).

[0165] The processor (260) of the AI ​​conductor server (200-2) can transmit one or more of the parameters constituting the selected second model and the code for driving the second model to the edge conductor server (200-1) through the communication unit (210).

[0166] The edge conductor server (200-1) can generate a second docker image based on the received second model and second device information (S437).

[0167] The processor (260) of the edge conductor server (200-1) can generate a second docker image based on the second device information so that the second model can operate normally on the second edge device (100-2).

[0168] The second Docker image may include a software package for independent execution of edge applications mounted on edge devices.

[0169] The second Docker image can contain the files required to run the edge application, including the edge application's code, runtime, system tools, and libraries.

[0170] The edge conductor server (200-1) can transmit the second model and the second docker image to the second edge device (100-2) (S437).

[0171] The processor (260) of the edge conductor server (200-1) can transmit the second model and the second docker image to the second edge device (100-1) via the communication unit (210) to load the second model onto the second edge device (100-2).

[0172] The second edge device (100-2) can load the second model onto the second edge device (100-2) based on the received second docker image (S439).

[0173] The processor (180) of the second edge device (100-2) may be equipped with a second model optimized for the second edge device (100-1) based on the second docker image. The processor (180) of the second edge device (100-1) may control the execution of the second model through an edge application.

[0174] The edge application installed on the second edge device (100-2) can infer the judgment result of a product or component through the second model. The edge application installed on the second edge device (100-2) can transmit the inferred judgment result to the edge conductor server (200-1). The edge conductor server (200-1) can transmit the judgment result to the AI ​​conductor server (200-2).

[0175] In this way, according to embodiments of the present invention, an optimized model tailored to each of a plurality of edge devices can be distributed. This significantly improves the accuracy of determining whether each edge device is a good or bad product.

[0176] FIG. 5a is a drawing illustrating a specific configuration of an AI advisor system having a three-stage structure according to an embodiment of the present invention, and FIG. 5b is a drawing illustrating a configuration of an available server that learns multiple models in parallel according to an embodiment of the present invention.

[0177] Referring to FIG. 5, the AI ​​conductor server (200-2) may include a wireless communication unit (310), a plurality of available servers (320-1 to 320-n), an optimal model verification unit (330), a model selection unit (350), a display (370), and a processor (390).

[0178] The AI ​​conductor server (200-2) may be an example of the AI ​​server (200) of FIG. 2. In this case, the wireless communication unit (310) of FIG. 5 may be an example of the communication unit (210) of FIG. 2, and the processor (390) of FIG. 5 may have the same configuration as the processor (260) of FIG. 2.

[0179] The wireless communication unit (310) can perform wireless communication with the edge conductor server (200-1). The wireless communication unit (310) can receive structured data and unstructured data from the edge conductor server (200-1).

[0180] The wireless communication unit (310) may also receive converted data, in which unstructured data is converted into structured data, from the edge conductor server (200-1).

[0181] A data conversion unit (not shown) can create converted data by assigning labels to unstructured data.

[0182] The data transformation unit can convert structured data into a graph, assign a label, and combine the label with the structured data to create transformed data.

[0183] The data transformation unit can generate transformation data for input data required for model learning.

[0184] Each of the multiple available servers (320-1 to 320-n) can train multiple models in parallel using structured data and transformed data.

[0185] Each of the multiple models may be a judgment model that aims to infer as accurately as possible whether a product is good or bad by an edge device operating in the field.

[0186] Each of the multiple models may be a model that receives one or more of the transformed data corresponding to the structured data or unstructured data for the product as input, and infers whether the product is good or bad.

[0187] Each available server can train four models in parallel using the same data. Each of the four models can be trained using different learning methods.

[0188] Referring to FIG. 5b, each available server (320) may include a first model learning unit (321), a second model learning unit (322), a third model learning unit (323), and a fourth model learning unit (324).

[0189] In Fig. 5b, it is assumed that each available server (320) includes four model learning units, but this is merely an example and may include a greater number of model learning units.

[0190] The first model learning unit (321) may include a first preprocessing unit (321-1), a first data set processing unit (321-2), a first training unit (321-3), and a first model generation unit (321-4).

[0191] The first preprocessing unit (321-1) can preprocess the first data set received from the first edge device (100-1). The first preprocessing unit (321-1) can remove unusable data from the first data set.

[0192] The first data set processing unit (321-2) can calculate a data processing value including at least one of the average value, median value, and maximum value of the data from which useless data has been removed.

[0193] The first training unit (321-3) can learn the first judgment model using the first learning algorithm based on the data processing value.

[0194] The first learning algorithm may be a random forest classification algorithm. Random forest classification may be a technique that generates multiple decision trees and combines the prediction values ​​of each of the generated decision trees to generate a final prediction value.

[0195] The first model generation unit (321-4) can generate the first judgment model by determining the parameters of the first judgment model learned through the first training unit (321-3).

[0196] The second model learning unit (322) may include a second preprocessing unit (322-1), a second data set processing unit (322-2), a second training unit (322-3), and a second model generation unit (322-4).

[0197] The second preprocessing unit (322-1) can preprocess a first data set including structured data and transformed data. The second preprocessing unit (322-1) can remove unusable data from the first data set.

[0198] The second data set processing unit (322-2) can calculate a data processing value including at least one of the average value, median value, and maximum value of the data from which useless data has been removed.

[0199] The second training unit (322-3) can learn the second judgment model using the second learning algorithm based on the data processing value.

[0200] The second learning algorithm may be the Categorical Boosting (CAT Boost) classification algorithm. The CAT Boost classification algorithm may be an algorithm that classifies data by processing categorical features based on gradient descent.

[0201] The second model generation unit (322-4) can generate a second judgment model by determining the parameters of the second judgment model learned through the second training unit (322-3).

[0202] The third model learning unit (323) may include a third preprocessing unit (323-1), a third data processing unit (323-2), a third training unit (323-3), and a third model generation unit (323-4).

[0203] The third preprocessing unit (323-1) can preprocess a first data set including structured data and transformed data. The third preprocessing unit (323-1) can remove unusable data from the first data set.

[0204] The third data processing unit (323-2) can calculate a data processing value including at least one of the average value, median value, and maximum value of the data from a data set from which useless data has been removed.

[0205] The third training unit (323-3) can learn the third model using the third learning algorithm based on the data processing value.

[0206] The third learning algorithm may be the Gradient Boosting Classification algorithm. Gradient Boosting Classification uses gradient descent to adjust the weights of multiple learners to minimize prediction errors, and allows the next learner to learn from the error between the previous learner's prediction and the actual value.

[0207] The third model generation unit (323-4) can generate a third judgment model by determining the parameters of the third judgment model learned through the third training unit (323-3).

[0208] The fourth model learning unit (324) may include a fourth preprocessing unit (324-1), a fourth data processing unit (324-2), a fourth training unit (324-3), and a fourth model generation unit (324-4).

[0209] The fourth preprocessing unit (324-1) can preprocess the first data set. The fourth preprocessing unit (324-1) can remove unusable data from the structured data and converted data.

[0210] The fourth data processing unit (324-2) can calculate a data processing value including at least one of the average value, median value, and maximum value of the data from which useless data has been removed.

[0211] The fourth training unit (324-3) can learn the fourth judgment model using the fourth learning algorithm based on the data processing value.

[0212] The fourth learning algorithm may be the NGB Classification algorithm. The NGB Classification algorithm uses the assumption of independence between features, a fundamental assumption of Naive Bayes, to estimate the probability distribution of data and classify the data based on this.

[0213] The fourth model generation unit (324-4) can generate the fourth judgment model by determining the parameters of the fourth judgment model learned through the fourth training unit (324-3).

[0214] Each training unit can measure the weight values ​​of each intermediate layer among the multiple layers that make up the model and verify the measured weight values.

[0215] Each training unit can stop model learning if the validation score of each weight value of the intermediate layers is less than the reference score.

[0216] That is, each training unit can stop learning or adjust the weight values ​​if there are weight values ​​in the intermediate layers that have an effect of lowering the model's prediction accuracy.

[0217] Each training unit can adjust the weight values ​​of intermediate layers, which have the effect of lowering the prediction accuracy as each layer is stacked.

[0218] The optimal model verification unit (330) can measure the accuracy of each of the multiple judgment models.

[0219] The optimal model verification unit (330) can measure the prediction accuracy of each of multiple models using the N-fold cross validation method.

[0220] The N-fold cross validation method can be a method of dividing a given data set into N parts, using one part as validation data, and the remaining N-1 parts as training data, repeating N times so that each part is used as validation data once.

[0221] The model selection unit (350) can select the judgment model with the highest accuracy among multiple judgment models as the first model to be transmitted to the first edge device (100-1).

[0222] Through the above process, the AI ​​conductor server (200-2) can generate multiple judgment models based on the second data set received from the second edge device (100-2), and select the judgment model with the highest prediction accuracy among the generated multiple judgment models as the second model.

[0223] The AI ​​conductor server (200-2) can generate multiple judgment models based on the second data set using other available servers.

[0224] A plurality of judgment models based on a first data set received from a first edge device (100-1) and a plurality of judgment models based on a second data set received from a second edge device (100-2) can be simultaneously learned and generated through different available servers.

[0225] The processor (390) of the AI ​​conductor server (200-2) can distribute different models to each of the multiple edge devices. That is, the processor (390) of the AI ​​conductor server (200-2) can generate a model with different parameters based on data received from each edge device.

[0226] The edge conductor server (200-1) can transmit different judgment models to each of the multiple edge devices (100-1, 100-2) it manages.

[0227] The edge conductor server (200-1) can transmit a first model to a first edge device (100-1) and a second model with different parameters from the first model to a second edge device (100-2).

[0228] In another embodiment, multiple models can be trained using a single algorithm. In this case, each of the multiple models can be input with identical or different data.

[0229] Each model learning unit can learn the model using any one of the four algorithms described above.

[0230] The optimal model verification unit (330) and model selection unit (350) of FIG. 5b may be included in the running processor (240) of FIG. 2.

[0231] As another example, the optimal model verification unit (330) and the model selection unit (350) may be included in the processor (260, 390) of FIG. 2.

[0232] The display (370) can display the learning process of multiple models.

[0233] The processor (390) can control the overall operation of the AI ​​conductor server (200-2).

[0234] FIG. 6 is a diagram illustrating a process of measuring the prediction accuracy of each of a plurality of models learned by each of a plurality of algorithms according to one embodiment of the present disclosure.

[0235] Referring to Fig. 6, the training data set can be divided into N parts. The training data set can include structured data, unstructured data, and transformed data.

[0236] In Figure 6, N may be 4, but this is only an example.

[0237] The optimal model verification unit (330) can classify the training data set into four folds.

[0238] The optimal model verification unit (330) has one of the four folds as test data and the remaining three folds as training data.

[0239] Training data can be data used to learn a model, and test data can be data used to test whether the model has been learned well while preventing overfitting of the model.

[0240] The optimal model verification unit (330) can measure the first prediction result of the model when the first fold is used as test data.

[0241] The optimal model verification unit (330) can use the second fold as test data and the remaining folds as training data. When the second fold is used as test data, the optimal model verification unit (330) can measure the second prediction result of the model.

[0242] In this way, the optimal model verification unit (330) can measure the first to fourth prediction results.

[0243] The optimal model verification unit (330) can obtain the average of the first to fourth prediction results as the prediction accuracy of the model.

[0244] The optimal model verification unit (330) can perform N-fold cross-validation for each of the multiple models and obtain multiple prediction accuracies corresponding to each of the multiple models.

[0245] Meanwhile, the optimal model verification unit (330) can obtain the prediction accuracy of each of multiple models simultaneously and in parallel.

[0246] Below, we describe in detail the edge applications installed on each edge device.

[0247] Edge application features

[0248] Edge applications can be applications that enable edge devices to perform computing functions. The actions performed by edge applications can be actions performed by edge devices.

[0249] Edge applications can be installed without restrictions on personal computers, local servers, cloud servers, embedded devices, etc., and can perform computing according to the hardware performance of the installed environment.

[0250] Edge applications can be upgraded to the latest version via wireless communication with the Edge Conductor server (200-1). Edge devices can receive updated Edge applications from the Edge Conductor server (200-1).

[0251] The edge application can periodically transmit the operating status or failure status of the model installed on the edge device, and the operating status and failure status of the edge device to the edge conductor server (200-1) to maintain a stable computing state.

[0252] An edge application can transmit the inference results of the installed model to the edge conductor server (200-1). The inference results may include probability values.

[0253] The edge conductor server (200-1) can simultaneously communicate with multiple edge applications and control their functions. In other words, the edge conductor server (200-1) can centrally control multiple edge applications.

[0254] Edge applications can download a selected AI solution from among multiple AI solutions from the Edge Conductor server (200-1). Furthermore, edge applications can reinstall and re-execute new AI solutions based on user demand.

[0255] When an AI solution is installed on an edge device, an artificial intelligence model related to the AI ​​solution is trained in an AI conductor server (200-2), and the trained artificial intelligence model can be automatically distributed to the edge device.

[0256] An AI solution can be configured as a pipeline that receives data collected from edge devices, preprocesses the received data, creates an AI model using an AI algorithm based on the preprocessed data, and transmits the created AI model to the edge device.

[0257] As another example, an AI solution may consist of a pipeline that receives data collected from edge devices, preprocesses the received data, generates inference results using AI algorithms based on the preprocessed data, and delivers the generated inference results to edge applications.

[0258] Information, warnings, and errors that occur as the pipeline runs can be collected by the edge conductor server (200-1) to detect whether the AI ​​solution is operating normally.

[0259] When data requiring inference is generated on an edge device, the edge application can detect this and generate inference results.

[0260] Edge devices can display AI inference results on their displays through the edge viewer of edge applications.

[0261] Edge applications can transmit the inference results generated each time AI inference is performed, as well as the input data used for inference, to the edge conductor server (200-1). This allows users to review the inference results and determine whether a new AI model is needed.

[0262] When a user requests the creation of a new AI model, the AI ​​Conductor server (200-2) can create the new model and automatically deploy it to the edge application. This ensures that the edge device's inference performance remains fast.

[0263] The original data (input data) and inference results used in AI inference can be transmitted to the edge conductor server (200-1) at each inference. The edge conductor server (200-1) can manage the history of the received original data and inference results.

[0264] The original data (input data) used for AI inference may be sensing data collected by edge devices.

[0265] The original data can be used to retrain AI models. This allows for better management of data that impacts model performance, potentially enabling the development of high-performance AI solutions.

[0266] FIG. 7 is a drawing showing an execution screen of an edge UI application according to one embodiment of the present invention.

[0267] Edge UI applications can communicate with edge applications and execute various functions of edge applications.

[0268] Describes the execution screen (700) of the Edge UI application.

[0269] Number 1 can indicate the connection status with the edge application.

[0270] Number 2 can indicate the name of the model currently applied to the edge device.

[0271] Number 3 can indicate the operation mode of the edge application.

[0272] Button 4 may be a button indicating a request for inference results or data collection from a model installed on an edge device.

[0273] Number 5 can display the inferred image in real time.

[0274] Number 6 can represent real-time inference results. The inference result can be either OK or NG.

[0275] Number 7 can display statistics on the inferred results for the day. The statistics can include the number of counts judged OK and the number of counts judged NG.

[0276] The aforementioned disclosure can be implemented as computer-readable code on a program-recorded medium. The computer-readable medium includes any type of recording device that stores data readable by a computer system.

[0277] Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. In addition, the computer may include a processor (180) of an artificial intelligence device.

Claims

1. For an edge device equipped with an edge application that manages models mounted on the edge device, communication interface; A memory storing an artificial neural network-based judgment model that infers whether a product or part is defective; and A processor that collects data, uses the data collected according to the execution of the edge application as input to the judgment model, obtains an inference result on whether the product or the component is defective, and transmits the data and the inference result to the edge conductor server through the communication interface. Edge devices.

2. In paragraph 1, The above inference results are Contains the probability value and the judgment result on whether it is defective or not. Edge devices.

3. In paragraph 1, The above processor Transmitting the operation status or failure of the edge device or the judgment model to the edge conductor server through the communication interface. Edge devices.

4. In paragraph 1, The above processor Transmitting device information including hardware information and operating system information of the edge device to the edge conductor server, receiving a docker image and the judgment model from the edge conductor server, and loading the judgment model on the edge device based on the docker image. Edge devices.

5. In paragraph 1, The above processor Whenever the above judgment model generates the above inference result, the above inference result and the above data are transmitted to the edge conductor server. Edge devices.

6. A method for managing models of an edge device equipped with an edge application for managing models, Steps to collect data; A step of obtaining an inference result on whether the product or part is defective by using the data collected according to the execution of the edge application as input to a judgment model that infers whether the product or part is defective; and A step of transmitting the above data and the above inference result to an edge conductor server. How Edge devices work.

7. In paragraph 6, The above inference results are Contains the probability value and the judgment result on whether it is defective or not. How to manage models for edge devices 8. In paragraph 6, Further comprising a step of transmitting the operation status or failure of the edge device or the judgment model to the edge conductor server. How to manage models for edge devices 9. In paragraph 6, A step of transmitting device information including hardware information and operating system information of the edge device to the edge conductor server; A step of receiving a docker image and the judgment model from the edge conductor server; and Further comprising a step of loading the judgment model onto the edge device based on the above Docker image. How to manage models for edge devices 10. In paragraph 6, The above transmitting steps are Further comprising a step of transmitting the inference result and the data to the edge conductor server each time the judgment model generates the inference result. How to manage models for edge devices

Images