Method and system for customizing artificial intelligence model on basis of user feedback

Abstract

A method for customizing an artificial intelligence model on the basis of user feedback, performed by a processor, may comprise the steps of: outputting a first prediction result by using a pre-trained artificial intelligence model on the basis of input data and user output request data; receiving, from a user device, a change request for the artificial intelligence model received through a first user interface displayed on the user device; modifying the artificial intelligence model in response to the change request; outputting a second prediction result by using the modified artificial intelligence model; and providing a second user interface so that the user device can display the second prediction result.

Description

Method and system for customizing artificial intelligence models based on user feedback

[0001] The following embodiments relate to a method and system for customizing an artificial intelligence model based on user feedback.

[0002] Artificial intelligence models have established themselves as essential tools across various fields thanks to their ability to process large-scale data and generate predictive insights. These models are trained using data-driven learning algorithms and are widely used in diverse applications, such as medical diagnosis and financial forecasting. However, even when exhibiting high accuracy, AI models sometimes generate predictions or parameters that conflict with the common sense or domain knowledge of human experts. Such discrepancies can undermine confidence in AI models and limit their practical application.

[0003] Several factors contribute to these discrepancies. One reason may be that because AI models rely heavily on the quality and representativeness of training data, they can propagate and amplify inherent biases or imbalances within the data. Furthermore, due to the nature of high-dimensional data, complex interactions can occur between features that may not align with the intuitive relationships understood by domain experts. Additionally, AI models that are overfitted to training data may generate unreliable predictions by learning patterns independent of noise rather than generalizable insights. The complexity of the non-linear relationships modeled by AI makes interpreting results difficult, especially when they conflict with expected patterns.

[0004] Addressing these issues is essential to ensuring the reliability and interpretability of artificial intelligence models. Therefore, there is a growing need for systems and methods that can customize the parameters and predictions of AI models based on user feedback, particularly from domain experts. Such systems can bridge the gap between algorithmic output and human intuition, ultimately improving AI models to better meet actual expectations and needs.

[0005] A related prior art document is Korean Patent Publication No. 10-2020-0120557 (Publication date: October 21, 2020).

[0006] The technical problem that the present disclosure aims to solve is to provide a system for customizing an artificial intelligence model based on user feedback and a method for performing the same.

[0007] The technical problems that this disclosure aims to solve are not limited to those described above, and problems not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings.

[0008] A method for customizing an artificial intelligence model based on user feedback, performed by a processor according to one embodiment of the present disclosure, may include: outputting a first prediction result using a pre-trained artificial intelligence model based on input data and user output request data; receiving a request for change to the artificial intelligence model received from the user device through a first user interface displayed on the user device; modifying the artificial intelligence model in response to the change request; outputting a second prediction result using the modified artificial intelligence model; and providing a second user interface so that the user device can display the second prediction result. Here, the step of receiving the change request may include: displaying the first user interface on the user device, which includes bars representing the influence setting values ​​of a plurality of input variables included in the input data; automatically readjusting the influence setting values ​​of the remaining input variables, excluding the first input variable among the plurality of input variables, to the overall ratio when the first bar representing the influence setting value of the first input variable among the plurality of input variables is dragged and dropped; and receiving a change request for the readjusted plurality of input variables from the user device.

[0009] According to an embodiment, at least one of the input data may be data received from the user device or set according to a predetermined standard by the processor.

[0010] According to an embodiment, a request for change to the artificial intelligence model includes a request for change to the input data, and the input data may include a plurality of input variables and data for the plurality of input variables.

[0011] According to an embodiment, a request for change to the input data may include a request for change to the influence of at least one of the plurality of input variables.

[0012] According to an embodiment, a request for change to the input data may include a request to delete at least one of the plurality of input variables or a request to add an input variable to the plurality of input variables.

[0013] According to an embodiment, a request for change to the input data may include a request for change to at least some of the data for at least one of the plurality of input variables.

[0014] According to an embodiment, a request for change to the input data may include a request for change to the correlation between a plurality of the input variables.

[0015] According to an embodiment, a request for change to the artificial intelligence model may include a request for change to the first prediction result.

[0016] According to an embodiment, a request for change to the artificial intelligence model may include a request for change to the prediction result of the artificial intelligence model, which takes some of the input data as input.

[0017] According to an embodiment, the step of modifying the artificial intelligence model may include the step of modifying a plurality of weights included in the artificial intelligence model, and the step of retraining the artificial intelligence model using the modified plurality of weights.

[0018] According to an embodiment, the method may further include the step of providing a user interface for receiving a change request for the artificial intelligence model.

[0019] A system for structuring an artificial intelligence model based on user feedback according to one embodiment of the present disclosure may include a user device configured to transmit input data, output request data, and a change request received through a first user interface to a service server and to display a prediction result after modification; and a service server configured to output a prediction result before modification using an artificial intelligence model that takes the input data as input, modify the artificial intelligence model in response to the change request, output a prediction result after modification using the modified artificial intelligence model, and provide a second user interface so that the user device can display the prediction result after modification. Here, the user device may display the first user interface including bars representing influence setting values ​​of a plurality of input variables included in the input data, and when a first bar representing the influence setting value of a first input variable among the plurality of input variables is dragged and dropped, the influence setting values ​​of the remaining input variables excluding the first input variable among the plurality of input variables may be automatically readjusted to match the overall ratio, and a change request for the readjusted plurality of input variables may be transmitted to the service server.

[0020] The technical solutions of the present disclosure are not limited to the technical solutions described above, and technical solutions not mentioned will be clearly understood by those skilled in the art from this specification and the attached drawings.

[0021] According to embodiments of the present disclosure, by receiving global variables, the influence of partial variables included in input data from a user, and the correlations between variables, and by modifying and updating an artificial intelligence model, the user can derive a desired or more accurate prediction result.

[0022] According to embodiments of the present disclosure, by providing natural language-based and visualization-based menus that allow the user to adjust global variables, the influence of partial variables, and correlations between variables included in the input data, a user interface can be provided that enables the user to intuitively and easily modify and update an artificial intelligence model.

[0023] According to the embodiments of the present disclosure, since a user can train an artificial intelligence model by directly adding data for training, the user can derive desired or more accurate prediction results.

[0024] The effects of the present disclosure are not limited to those described above, and unmentioned effects will become apparent to those skilled in the art from the present specification and the accompanying drawings.

[0025] FIG. 1 is a conceptual diagram of a customization system for an artificial intelligence model based on user feedback according to one embodiment of the present disclosure.

[0026] FIG. 2 is a block diagram of a customization system for an artificial intelligence model based on user feedback according to one embodiment of the present disclosure.

[0027] FIG. 3 is a block diagram showing the configuration of a service server that provides a customization service for an artificial intelligence model based on user feedback according to one embodiment of the present disclosure.

[0028] FIG. 4 is an exemplary diagram illustrating a user interface showing input variables and the degree of influence of input variables according to one embodiment of the present disclosure.

[0029] FIGS. 5 to 7 are exemplary drawings illustrating a user interface before input data is modified and a user interface after input data is modified, according to one embodiment of the present disclosure.

[0030] FIG. 8 is an exemplary diagram illustrating a user interface for modifying a portion of input data according to one embodiment of the present disclosure.

[0031] FIG. 9 is an exemplary diagram illustrating a user interface for modifying the correlation of input data according to one embodiment of the present disclosure.

[0032] FIG. 10 is an exemplary diagram illustrating a user interface showing a prediction result according to one embodiment of the present disclosure.

[0033] FIG. 11 is a flowchart illustrating a method for customizing an artificial intelligence model based on user feedback performed by a processor according to one embodiment of the present disclosure.

[0034] Specific structural or functional descriptions of embodiments according to the concept of the present disclosure disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present disclosure, and embodiments according to the concept of the present disclosure may be implemented in various forms and are not limited to the embodiments described herein.

[0035] Embodiments according to the concept of the present disclosure may be subject to various modifications and may take various forms; therefore, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present disclosure to specific disclosed forms, and includes modifications, equivalents, or substitutions that fall within the spirit and scope of the present disclosure.

[0036] Terms such as "first" or "second" may be used to describe various components, but said components shall not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present disclosure, the first component may be named the second component, and similarly, the second component may be named the first component.

[0037] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Expressions describing the relationships between components, such as "between," "exactly between," or "directly adjacent to," should be interpreted in the same way.

[0038] The terms used herein are used merely to describe specific embodiments and are not intended to limit the disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0039] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this disclosure pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0040] In this specification, the term "processor" may refer to hardware capable of performing functions and operations according to each name described in this specification, computer program code capable of performing specific functions and operations, or an electronic recording medium loaded with computer program code capable of performing specific functions and operations.

[0041] In other words, the term "processor" may mean hardware for carrying out the technical concept of the present disclosure, software for driving said hardware, a functional combination of said hardware and said software, and / or a structural combination of said hardware and said software.

[0042] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.

[0043] FIG. 1 is a conceptual diagram of a customization system for an artificial intelligence model based on user feedback according to one embodiment of the present disclosure.

[0044] A customization system for an artificial intelligence model based on user feedback (hereinafter referred to as the "system") may include a user device (200), a service server (100), and / or a database (400).

[0045] According to one embodiment, in the system, the service server (100) can use an artificial intelligence model to output a prediction result and provide it to the user device (200) in response to a prediction result request from the user device (200).

[0046] For example, a user device (200) can send a request to a service server (100) to predict the user's fuel consumption.

[0047] The service server (100) can set input data. The input data may be data set by the service server (100) according to predetermined criteria, or data received from a user device (200). The input data may include multiple input variables. The service server (100) can set a setting value for the influence of each input variable according to predetermined criteria.

[0048] The service server (100) can respond to a fuel consumption prediction request received from a user device (200) by outputting input data and data requested by the user, i.e., a prediction result for fuel consumption, and can transmit the prediction result to the user device (200). The user can check the prediction result displayed through the user interface of the user device (200).

[0049] A user may request a change to the artificial intelligence model using a user device (200). The artificial intelligence model may generate predictions or parameters that conflict with the common sense or domain knowledge of people, particularly experts. Accordingly, the user may request a change to the input data using a user device (200). Here, the input data may include input data input to the artificial intelligence model, parameters of the artificial intelligence model, and / or the degree of influence or correlation between the input variables included in the input data.

[0050] The service server (100) may provide a user interface to the user device (200) for receiving a request to change an artificial intelligence model from the user. The user interface provided to the user device (200) may provide various tools for modifying prediction results, and the user may use these various tools.

[0051] For example, the user interface may include a tab or bar for modifying the artificial intelligence model. The user can view the prediction results of the artificial intelligence model and explanations of the variables through the user interface displayed via the user device (200).

[0052] As another example, the user interface may include tools that allow filtering data or adjusting various input variables using the drag-and-drop function of the cursor.

[0053] The user can adjust various input variables using tools provided through the user interface, and once the adjustment of the input variables is complete, the user can click the 'Retrain' button, 'Modify Model' button, or 'Update Model' button displayed on the user interface. As these buttons are clicked, the modified artificial intelligence model can be stored in the service server (100).

[0054] Meanwhile, the service server (100) can output a modified prediction result using the modified artificial intelligence model in response to a request for change to the artificial intelligence model, and can provide the output prediction result to the user device (200).

[0055] FIG. 2 is a block diagram of a customization system for an artificial intelligence model based on user feedback according to one embodiment of the present disclosure.

[0056] A system according to one embodiment of the present disclosure may include a service server (100), a user device (200), a database (300) connected to the user device (200), and / or a database (400) connected to the service server (100).

[0057]

[0058] The service server (100) may be implemented as a printed circuit board (PCB), such as a motherboard, an integrated circuit (IC), or a system on chip (SoC). For example, the service server (100) may be implemented as an application processor. The service server (100) may include a processor (10) and a memory (20) containing an artificial intelligence model (12). The processor (10) may process the artificial intelligence model stored in the memory (20). The processor (10) may execute computer-readable code (e.g., software) stored in the memory (20) and instructions triggered by the processor (10). The processor (10) may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in the program. For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), or a Field Programmable Gate Array (FPGA).

[0059] The artificial intelligence model (12) can be an xAI (Explainable Artificial Intelligence) or a deep learning model. By way of example, the artificial intelligence model 12 includes a Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), perceptron, multilayer perceptron, Feed Forward (FF), Radial Basis Network (RBF), Deep Feed Forward (DFF), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Auto Encoder (AE), VAE (Variational Auto Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics) Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning It may include Machine), ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), and / or AN (Attention Network).However, it will be understood by a person skilled in the art that the artificial intelligence model (12) is not limited to those exemplified and may include any artificial intelligence model. The artificial intelligence model (12) may be pre-trained, but this is merely one example. Furthermore, the learning method of the artificial intelligence model (12) is not specified as any one.

[0060] Meanwhile, the memory (20) may store instructions (or programs) executable by the processor (10) in addition to the artificial intelligence model (12). For example, the instructions may include instructions for executing the operation of the processor and / or the operation of each component of the processor.

[0061] The memory (20) can be implemented as a volatile memory device or a non-volatile memory device.

[0062] Volatile memory devices can be implemented as DRAM (dynamic random access memory), SRAM (static random access memory), T-RAM (thyristor RAM), Z-RAM (zero capacitor RAM), or TTRAM (Twin Transistor RAM).

[0063] A non-volatile memory device can be implemented as an EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM, Conductive Bridging RAM (CBRAM), FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, Molecular Electronic Memory Device, or Insulator Resistance Change Memory.

[0064] The user device (200) may be implemented in a PC (personal computer), server, or portable device. The portable device may be implemented as a laptop computer, mobile phone, smartphone, tablet PC (Personal Computer), mobile internet device (MID), PDA (personal digital assistant), EDA (enterprise digital assistant), digital still camera, digital video camera, PMP (portable multimedia player), PND (personal navigation device or portable navigation device), handheld game console, e-book, or smart device. The smart device may be implemented as a smart watch, smart band, or smart ring.

[0065] In one embodiment, the method for customizing an artificial intelligence model based on user feedback may be operated through a user device (200) in the form of an application or program. The application for customizing an artificial intelligence model based on user feedback may be an application running on a PC or a portable device. Such an application may display information processed by the user device (200) or service server (100) through a display, or a UI (user interface) or GUI (graphic user interface) according to the information processed by the user device (200) or service server (100) through a display. The information processed by the service server (100) may include input data, output data, parameters, and / or weights of the artificial intelligence model (12). Such an application may include a user interface, and said user interface may be linked with the system for customizing an artificial intelligence model based on user feedback.

[0066] The database (300) connected to the user device (200) may be a database connected to the user device (200) via a wired or wireless connection. For example, the database (300) may be a database built by the user of the user device (200). For another example, the database (300) may be a cloud-based database connected via a socket to a platform that processes index data. The database (300) may store input data, etc., that can be input into the artificial intelligence model (12).

[0067] The database (400) connected to the service server (100) may be a database connected to the service server (100) via a wired or wireless connection. For example, the database (400) may be a database built by the operator of the service server (100). The database (400) may store input data, etc., that can be input into the artificial intelligence model (12).

[0068] FIG. 3 is a block diagram showing the configuration of a service server that provides a customization service for an artificial intelligence model based on user feedback according to one embodiment of the present disclosure.

[0069] Referring to FIG. 3, the service server (100) may include a communication interface (110), memory (120), I / O interface (130), and a processor (140), and each component may communicate with one or more communication buses or signal lines.

[0070] The communication interface (110) can be connected to the user terminal (200) as well as other devices through a wired / wireless communication network to exchange data.

[0071] Meanwhile, a communication interface (110) that enables the transmission and reception of such data may include a wired communication pod (111) and a wireless circuit (112). Here, the wired communication port (111) may include one or more wired interfaces, for example, Ethernet, Universal Serial Bus (USB), and IEEE1394 (e.g., Apple’s FireWire, Sony’s i.Link, Texas Instruments’ Lynx). Additionally, the wireless circuit (112) may transmit and receive data with an external device via an RF (Radio Frequency) signal or an optical signal. Furthermore, wireless communication may use at least one of a plurality of communication standards, protocols, and technologies, such as GSM (Global System for Mobile Communications), EDGE (Enhanced Data rates for GSM Evolution), CDMA (Code-Division Multiple Access), TDMA (Time Division Multiple Access), Bluetooth, Wi-Fi, VoIP, Wi-MAX, and any other suitable communication protocol.

[0072] The memory (120) can store data for at least one process (algorithm) for providing a customization service for an artificial intelligence model or for a program that reproduces the process. In addition, the memory (120) can store additional processes for performing other operations. However, the data that the memory (120) can store is not limited to the examples given.

[0073] Meanwhile, the memory (120) can store various data used in the service server (100) as well as one or more artificial intelligence models built through prior training as needed.

[0074] In various embodiments, the memory (120) may include a volatile or non-volatile recording medium capable of storing various data, commands and / or information. For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), Random Access Memory (RAM), Static RAM (SRAM), Read-Only Memory (ROM), Programmable ROM (PROM), Electrically Erasable ROM (EEPROM), network storage, cloud, and blockchain database.

[0075] In various embodiments, the memory (120) may store at least one configuration of an operating system (121), a communication module (122), a user interface module (123), and one or more applications (124).

[0076] An operating system (121) (e.g., embedded operating systems such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) may include various software components and drivers for controlling and managing general system operations (e.g., memory management, storage device control, power management, etc.) and may support communication between various hardware, firmware, and software components.

[0077] The communication module (122) can support communication with another device through the communication interface (110). The communication module (122) may include various software components for processing data received by the wired communication port (111) or wireless circuit (112) of the communication interface (110).

[0078] The user interface module (123) can receive requests or input from a viewer from a keyboard, touch screen, and / or microphone, etc., through the I / O interface (130) and provide a user interface on the display.

[0079] The application (124) may include a program or module configured to be executed by one or more processors (140).

[0080] The I / O interface (130) can connect at least one of an input / output device (not shown) of the service server (100), such as a display, keyboard, touch screen, and microphone, to the user interface module (123). The I / O interface (130) can receive user input (e.g., voice input, keyboard input, touch input, etc.) together with the user interface module (123) and process commands based on the received user input.

[0081] The processor (140) is connected to a communication interface (110), memory (120), and I / O interface (130) to control the overall operation of the service server (100) and to execute various commands through applications and / or programs stored in memory (120).

[0082] The processor (140) may correspond to a computing device such as a CPU (Central Processing Unit) or an AP (Application Processor). Additionally, the processor (140) may be implemented in the form of an integrated chip (IC), such as a System on Chip (SoC) that integrates various computing devices. Alternatively, the processor (140) may include a module for computing artificial neural network models, such as a Neural Processing Unit (NPU).

[0083] Specifically, when a prediction result request is input from a user device, the processor (140) can output a first prediction result using a pre-trained artificial intelligence model based on input data and user output request data.

[0084] When a request for a change to the artificial intelligence model is received from a user device (200), the processor (140) can modify the artificial intelligence model in response to the request for change.

[0085] For example, a request for change to an artificial intelligence model may include a request for change to input data. The input data may include multiple input variables and data regarding the multiple input variables. A request for change to the multiple input data may include a request for change to the influence of at least one of the input variables, a request to delete at least one of the input variables, a request to add a new input variable to the existing input variables, and a request for change to the correlation between the input variables.

[0086] As another example, a change request for an artificial intelligence model may include a change request for a prediction result.

[0087] As another example, a request for change to an artificial intelligence model may include a request to change at least some of the data regarding at least one of multiple input variables. Furthermore, a request for change to an artificial intelligence model may include a request to change at least some of the total data regarding multiple input variables.

[0088] A request for a change to an artificial intelligence model may be a request for a change to some data that meets specific conditions requested by the user. For example, the user may request adjustments to variable influence, correlation, or prediction results only for some data that meet the conditions of "height 160cm or taller" and "age 30 years or older."

[0089] The processor (140) can output a second prediction result using the modified artificial intelligence model. In this specification, the output of the artificial intelligence model before modification may be referred to as the 'first prediction result', and the output of the artificial intelligence model after modification may be referred to as the 'second prediction result'.

[0090] The processor (140) may provide a user interface so that the user device (200) can display a second prediction result. To this end, the processor (140) may have one or more artificial intelligence models built through prior training. At this time, the training method for the artificial intelligence model is not specified as any one.

[0091] Meanwhile, the processor (140) may perform improvement operations, such as training data preprocessing, training, parameter change operations, or weight change operations, in order to improve the performance of the artificial intelligence model without interrupting the service while providing a customization service for the artificial intelligence model based on user feedback.

[0092] FIG. 4 is an exemplary diagram illustrating a user interface showing input variables and the degree of influence of input variables according to one embodiment of the present disclosure.

[0093] The user interface (40) of FIG. 4 shows the input variables and the degree of influence of the input variables on the output result when a first prediction result is output using a pre-trained artificial intelligence model based on input data and user output request data.

[0094] In one embodiment, the user output request data may be 'fuel consumption'. When a user's prediction result request is received from a user device (200) to output input data and a prediction result for 'fuel consumption', the service server (100) may set an artificial intelligence model for the received prediction result request. Specifically, the input data may be data set by the service server (100) according to predetermined criteria or data received from the user device (200). The input data may include multiple input variables. The service server (200) may set a setting value for the influence of each input variable according to predetermined criteria.

[0095] The user interface (40) of FIG. 4 displays the feature importance of multiple input variables for the prediction result output using an artificial intelligence model in response to the user's request for a prediction result. For example, based on the user's output request data, the input variables automatically recommended and input by the service server (100) may include GT1 power generation amount, weather station ambient temperature, weather station atmospheric pressure, weather station relative humidity, NG GOV OUT TEMP, GT1 fuel gas temperature, GT1 Pilot Stage CV POSN, Pilot Stage FG FLOW, GT1 OTC, GT2 fuel gas temperature, GT2 Pilot Stage CV POSN, GT2 Pilot Stage FG FLOW, ST condenser pressure, ST condenser inlet seawater temperature 1, HPS (Header) TEMP, HPS (Header) PRES, LPS (Header) TEMP, HRSG1 IP FLOW, HRSG1 LP FLOW, BOP SLP INV SPEED, and BOP CWP INV SPEED. Among the example input variables, the variable with the greatest influence on the current artificial intelligence model may be GT2 Pilot Stage CV POSN.

[0096] The ratio of Feature Importance for each of the multiple input variables can be represented as a bar graph (41) and / or a pie chart (42).

[0097] FIGS. 5 to 7 are exemplary drawings illustrating a user interface before modifying input data (i.e., input variables and the influence of input variables) and a user interface after modifying input data, according to one embodiment of the present disclosure.

[0098] FIG. 5 illustrates a user interface that can adjust the order of influence of input variables according to one embodiment of the present disclosure. The user interface (50_1) shown on the left side of FIG. 5 is a user interface before modification and corresponds to the default screen output in response to a user's request for a prediction result. The user interface (50_2) shown on the right side of FIG. 5 is a user interface after modification and corresponds to a modified screen in which a request for a change to the artificial intelligence model is reflected after the first prediction result is output.

[0099] For example, multiple input variables included in the input data may include VAR1, VAR2, VAR3, VAR4, VAR5, and VAR6. The influence of VAR1 may be initially set to 30%, the influence of VAR2 to 20%, the influence of VAR3 to 20%, the influence of VAR4 to 10%, the influence of VAR5 to 10%, and the influence of VAR6 to 10%.

[0100] A user who has checked the user interface (50_1) before modification can adjust the order of influence of the input variables (e.g., VAR1) by dragging and dropping the part where the input variables are listed or the button corresponding to the input variables up or down.

[0101] For example, the user can drag and drop VAR1 toward VAR2 and drag and drop VAR3 toward VAR4. In this way, the user can input a request to change the influence of at least one of the multiple input variables into the user device (200).

[0102] FIG. 6 illustrates a user interface that can adjust the influence setting value of an input variable according to one embodiment of the present disclosure. The user interface (60_1) shown on the left side of FIG. 6 is a user interface before modification and corresponds to the basic screen output in response to a user's request for a prediction result. The user interface (60_2) shown on the right side of FIG. 6 is a user interface after modification and corresponds to a modified screen in which a request for a change to the artificial intelligence model is reflected after the first prediction result is output.

[0103] For example, multiple input variables included in the input data may include VAR1, VAR2, VAR3, VAR4, VAR5, and VAR6. The influence of VAR1 may be initially set to 30%, the influence of VAR2 to 20%, the influence of VAR3 to 20%, the influence of VAR4 to 10%, the influence of VAR5 to 10%, and the influence of VAR6 to 10%.

[0104] A user who has checked the user interface (50_1) before modification can adjust the setting value of an input variable (e.g., VAR2) by dragging and dropping a bar representing the setting value of the input variable to the right or left.

[0105] For example, the user can drag and drop the bar representing the setting value of VAR2 to the right and the bar representing the setting value of VAR4 to the right. Since the total ratio must be set to 100%, as the setting values ​​of VAR2 and VAR4 are adjusted, the setting values ​​of VAR3 and / or VAR1 can be automatically adjusted. In this way, the user can input a request to change the influence of at least one of the multiple input variables into the user device (200).

[0106] FIG. 7 illustrates a user interface that can delete or add input variables according to one embodiment of the present disclosure. The user interface (70_1) shown on the left side of FIG. 7 is a user interface before modification and corresponds to the default screen displayed in response to a user's prediction result request. The user interface (70_2) shown on the right side of FIG. 7 is a user interface after modification and corresponds to a modified screen that reflects a request for a change to the artificial intelligence model after the first prediction result is displayed.

[0107] For example, multiple input variables included in the input data may include VAR1, VAR2, VAR3, VAR4, VAR5, and VAR6. The influence of VAR1 may be initially set to 30%, the influence of VAR2 to 20%, the influence of VAR3 to 20%, the influence of VAR4 to 10%, the influence of VAR5 to 10%, and the influence of VAR6 to 10%.

[0108] A user who has checked the user interface (70_1) before modification can delete or add the input variable by clicking the part or button corresponding to the input variable, or by dragging and dropping the bar that indicates the setting value of the input variable.

[0109] For example, the user can delete VAR2, VAR4, and VAR5 by dragging and dropping the bar representing the setting values ​​of VAR2, VAR4, and VAR5 to the left. Additionally, the user can add an additional variable (e.g., VAR7) as an input variable. For example, VAR7 can be added by clicking on an area excluding the area where multiple input variables and the setting values ​​of each input variable are displayed. In this way, the user can input a request to delete or add at least one of the multiple input variables to the user device (200).

[0110] As described above, the user can input a request for change to the artificial intelligence model and request retraining of the artificial intelligence model through the user interface of FIGS. 5 to 7 displayed on the user device (200).

[0111] Change requests and retraining requests for the artificial intelligence model input into the user device (200) can be transmitted to the service server.

[0112] FIG. 8 is an exemplary diagram illustrating a user interface for modifying a portion of input data according to one embodiment of the present disclosure.

[0113] FIG. 8 shows a user interface (80) that can change a portion of input data according to one embodiment of the present disclosure.

[0114] In one embodiment, the input data may include data for four input variables (e.g., input variable 0, input variable 1, input variable 2, and input variable 3). At least one of the plurality of input variables may be input variable 1. The user may decide to increase the weight of the latest data in the prediction result, and in this case, the setting value of the influence of the data of input variable 1 for the past week may be increased.

[0115] Here, at least one of the multiple input variables refers to the data from the past week corresponding to some samples of the data for input variable 1.

[0116] FIG. 9 is an exemplary diagram illustrating a user interface for modifying the correlation of input data according to one embodiment of the present disclosure.

[0117] FIG. 9 shows a user interface (90) that can change the correlation between a plurality of input variables according to one embodiment of the present disclosure.

[0118] The user interface (90) of FIG. 9 includes a graph (Partial Dependence Plot) that indicates the correlation between two input data among the input data when outputting the prediction results of the artificial intelligence model. The user can request a change to the correlation using various tools of the user interface (90) of FIG. 9. For example, the user can request to change the correlation between two input data from a positive correlation to a negative correlation. As another example, the user can request a slope correction in the graph.

[0119] FIG. 10 is an exemplary diagram illustrating a user interface showing a prediction result according to one embodiment of the present disclosure.

[0120] FIG. 10 shows a user interface (1000) that displays a prediction result of power generation (see 'Prediction Result of Power Generation' in FIG. 10), which is a prediction result of an artificial intelligence model according to one embodiment of the present disclosure.

[0121] The user interface (1000) of FIG. 10 includes a time series graph representing the predicted result of the power generation, which is the prediction result of an artificial intelligence model. The time series graph includes a graph representing the predicted result value of the power generation (see 'prediction' in FIG. 10) and a graph representing the actual value of the power generation (see 'ground-truth' in FIG. 10).

[0122] The user may request a change to the prediction result using various tools of the user interface (1000) of FIG. 10. For example, if the user determines that a value higher than a reference value has been output as the prediction result, the user may adjust the point position of the time series graph or use a filtering tool (1) to lower the prediction result. Specifically, the user may request a change to the prediction result of the artificial intelligence model for the entire input data, as well as request a change to the prediction result of the artificial intelligence model for a portion of the input data.

[0123] FIG. 11 is a flowchart illustrating a method for customizing an artificial intelligence model based on user feedback performed by a processor according to one embodiment of the present disclosure.

[0124] Referring to FIG. 11, the service server (100) can output a first prediction result using a pre-trained artificial intelligence model based on input data and user output request data (S100).

[0125] At least one of the input data may be data received from the user device (200) or set according to a predetermined standard by the processor.

[0126] The service server (100) can receive a request to change the artificial intelligence model from the user device (200) (S102).

[0127] A request for change to an artificial intelligence model may include a request for change to input data. The input data may include multiple input variables and data regarding the multiple input variables.

[0128] A request for change to input data may include a request for change to the influence of at least one of a plurality of input variables, a request for deletion of at least one of a plurality of input variables, a request for addition to add a new input variable to a plurality of input variables, a request for change to at least some of the data regarding at least one of a plurality of input variables, and / or a request for change to the correlation between a plurality of input variables.

[0129] A request for change to an artificial intelligence model may include a request for change to a first prediction result. A request for change to an artificial intelligence model may include a request for change to a prediction result of an artificial intelligence model that takes some of the input data as input.

[0130] The service server (100) can modify the artificial intelligence model in response to a request to change the artificial intelligence model (S104).

[0131] The step of modifying the artificial intelligence model (S104) may include modifying multiple weights included in the artificial intelligence model and retraining the artificial intelligence model using the modified multiple weights.

[0132] Meanwhile, the service server (100) can provide a user interface to the user device (200) for receiving a request to change the artificial intelligence model.

[0133] The service server (100) can output a second prediction result using a modified artificial intelligence model (S106).

[0134] The service server (100) can provide a user interface to the user device (200) so that the user device (200) can display a second prediction result (S108).

[0135] A service server (100) according to one embodiment may include a computer program stored on a non-transient computer-readable recording medium to perform S100 to S108 of FIG. 11 described above, combined with a computer or computing device which is hardware.

[0136] The service server (100) may be implemented as a computing device comprising at least one processor (140) that executes instructions of programs loaded into memory (120). The memory (120) may be loaded with a program comprising instructions described to execute S100 to S108 of FIG. 11 described above.

[0137] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0138] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0139] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0140] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0141] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

[0142] A method and system for customizing an artificial intelligence model based on user feedback as described above can be applied to various industrial sectors that provide prediction services using artificial intelligence models.

Claims

1. A step of outputting a first prediction result using a pre-trained artificial intelligence model based on input data and user output request data; A step of receiving a request for change to the artificial intelligence model received from the user device through a first user interface displayed on the user device; A step of modifying the artificial intelligence model in response to the above change request; A step of outputting a second prediction result using the modified artificial intelligence model; and The method includes the step of providing a second user interface so that the user device can display the second prediction result; The step of receiving the above change request is, A step of displaying the first user interface on the user device, the first user interface comprising bars representing respective influence setting values ​​of a plurality of input variables included in the input data; When a first bar representing the influence setting value of the first input variable among the plurality of input variables is dragged and dropped, the influence setting values ​​of the remaining input variables excluding the first input variable among the plurality of input variables are automatically readjusted to the overall ratio; and A step comprising receiving a request from the user device to change the plurality of input variables that have been readjusted; A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

2. In Paragraph 1, At least one of the above input data is, Data received from the above user device or set according to a predetermined standard by the above processor, A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

3. In Paragraph 2, A request for changes to the above artificial intelligence model is, Includes a request for change to the above input data, and The above input data includes a plurality of input variables and data for the plurality of said input variables, A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

4. In Paragraph 3, A request for changes to the above input data is, A request for a change to the influence of at least one of the plurality of the above input variables, A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

5. In Paragraph 4, A request for changes to the above input data is, A request to delete at least one of the plurality of input variables or a request to add an input variable to the plurality of input variables, A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

6. In Paragraph 5, A request for changes to the above input data is, A request for change to at least some of the data for at least one of the plurality of input variables, A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

7. In Paragraph 6, A request for changes to the above input data is, A request to change the correlation between multiple of the above input variables, A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

8. In Paragraph 1, A request for changes to the above artificial intelligence model is, including a request for a change to the above first prediction result, A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

9. In Paragraph 8, A request for changes to the above artificial intelligence model is, A request for a change to the prediction result of the artificial intelligence model using some of the above input data as input, A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

10. In Paragraph 1, The step of modifying the above artificial intelligence model; is, A step of modifying a plurality of weights included in the above artificial intelligence model; and A step of retraining the artificial intelligence model using a plurality of modified weights; comprising A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

11. In Paragraph 10, Further comprising the step of providing a user interface for receiving the change request for the artificial intelligence model; A method for customizing an artificial intelligence model based on user feedback, performed by a processor.

12. A user device configured to transmit input data, output request data, and a change request received through a first user interface to a server, and to display a predicted result after modification; and A service server comprising a processor configured to output a prediction result before modification using an artificial intelligence model that takes the above input data as input, modify the artificial intelligence model in response to the change request, output a prediction result after modification using the modified artificial intelligence model, and provide a second user interface so that the user device can display the prediction result after modification; The above user device is, Displaying the first user interface including bars representing the influence setting values ​​of a plurality of input variables included in the input data, and when the first bar representing the influence setting value of the first input variable among the plurality of input variables is dragged and dropped, automatically readjusting the influence setting values ​​of the remaining input variables excluding the first input variable among the plurality of input variables to match the overall ratio, and transmitting a change request for the readjusted plurality of input variables to the service server. A system that customizes artificial intelligence models based on user feedback.

13. A non-transient computer-readable recording medium comprising instructions for executing the method according to paragraph 1 on a computer.

Images