Method and electronic device for outputting predicted determination results for a task requiring a user's determination
The electronic device uses an AI model to learn from user judgment history, ensuring reliable prediction outputs align with individual user characteristics, addressing the inconsistency in AI-based decision support systems.
Patent Information
- Application Number
- KR1020260054296
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2026-03-25
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2046-03-25
AI Technical Summary
Existing AI-based decision support systems fail to adequately reflect the individual judgment characteristics of personnel, leading to varying outcomes based on personal experience and criteria, and struggle when personnel are absent.
An electronic device that utilizes an artificial intelligence model to learn from a user's past judgment history, calculates the reliability of predicted judgment results, and outputs these results based on a threshold value, ensuring they align with the user's judgment characteristics.
The system effectively generates judgment results that reflect individual user characteristics, improving decision-making reliability and efficiency by providing accurate predictions when personnel are present or absent.
Smart Images

Figure 112026036607307-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present disclosure relates to a method and electronic device for outputting information on a user's predicted judgment result regarding a task requiring a user's judgment. Specifically, the invention relates to a method and electronic device for obtaining information on a user's predicted judgment result regarding a second judgment request by inputting a first judgment information request and a user's first judgment data regarding a task requiring a user's judgment into an artificial intelligence model, calculating the reliability of the user's predicted judgment result information regarding the second judgment request information by comparing the obtained information on the user's predicted judgment result regarding the second judgment request information with the user's second judgment data, and outputting information on a user's predicted judgment result regarding a third judgment request information based on comparing the reliability with a threshold value. Background Technology
[0002] In various industries, there may be tasks where decision-making relies on the experience and judgment of the person in charge. For example, tasks such as reviewing abnormal transactions at financial institutions, handling customer inquiries, insurance underwriting, dispute resolution at e-commerce platforms, or internal approval processes can be handled based on the 담당자's judgment. In some cases, these tasks may be difficult to process solely using data-driven rules, and different outcomes may result depending on the person's experience, work history, or judgment criteria.
[0003] Recently, technologies utilizing artificial intelligence models to support such decision-making processes may be proposed. For example, a system that provides recommended judgment results for specific tasks using an AI model trained on past work processing records could be utilized. Such a system can be used to improve work efficiency by providing reference information to personnel when making decisions.
[0004] However, existing AI-based decision support systems may face the problem of failing to adequately reflect the individual judgment characteristics of the personnel in charge. For example, even when different personnel handle the same task, different judgment results may be derived depending on each person's experience or criteria. Furthermore, it may be difficult to process tasks based on the judgment of a person in charge when they are absent. Therefore, there is a growing need for technology that learns from a personnel's past judgment history to provide predictive judgment results that reflect their judgment characteristics, and supports task processing based on the reliability of these predictions. means of solving the problem
[0005] An electronic device according to one embodiment may include a memory for storing a program or at least one instruction, at least one processor for executing at least one instruction stored in the memory, and a display.
[0006] An electronic device according to one embodiment can obtain first judgment request information regarding a task requiring a user's judgment.
[0007] An electronic device according to one embodiment can obtain the user's first judgment data regarding the first judgment request information.
[0008] An electronic device according to one embodiment can obtain user prediction judgment result information regarding the second judgment request information by inputting the second judgment request information into an artificial intelligence model trained using the first judgment request information and the first judgment data as training data.
[0009] An electronic device according to one embodiment can obtain the user's second judgment data regarding the second judgment request information.
[0010] An electronic device according to one embodiment can calculate the reliability of the user's predicted judgment result information regarding the second judgment request information based on comparing the second judgment data and the user's predicted judgment result information regarding the second judgment request information.
[0011] An electronic device according to one embodiment inputs third judgment request information into an artificial intelligence model when the reliability is above a threshold value, and the third The user's predicted judgment result information regarding the judgment request information can be output through the display. Brief explanation of the drawing
[0012] FIG. 1 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 2 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 3 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 4 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 5 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 6 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 7 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 8 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 9 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 10 is a flowchart illustrating a method of operation of an electronic device according to one embodiment. FIG. 11 is a block diagram illustrating the components of an electronic device according to one embodiment. Specific details for implementing the invention
[0013] The terms used in this specification will be briefly explained, and the invention will be described in detail.
[0014] The terms used in this invention have been selected based on currently widely used general terms, taking into account their functions within the invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.
[0015] When a part of a specification is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part" or "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.
[0016] Additionally, the description 'at least one of A, B, and C' means that it may be any one of 'A', 'B', 'C', 'A and B', 'A and C', 'B and C', and 'A, B, and C'.
[0017] It should be understood that the combinations of blocks and flowcharts in each flowchart can be executed by one or more computer programs containing computer-executable instructions. One or more computer programs may be stored entirely in a single memory or may be partitioned and stored in multiple different memories.
[0018] All functions or operations described in this document may be processed by a single processor or a combination of processors. A single processor or a combination of processors is a circuitry that performs processing and may include circuitry such as an AP (Application Processor), CP (Communication Processor), GPU (Graphical Processing Unit), NPU (Neural Processing Unit), MPU (Microprocessor Unit), SoC (System on Chip), IC (Integrated Chip), etc.
[0019] A processor may include various processing circuits and / or multiple processors. For example, the term “processor” as used herein, including in the claims, may include at least one processor and various processing circuits. In at least one processor, one or more processors may be configured to perform the various functions described herein in a distributed manner, individually and / or collectively. As used herein, “processor,” “at least one processor,” and “one or more processors” may be configured to perform various functions. However, these terms cover, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor can perform all functions. Additionally, at least one processor may include a combination of processors performing various functions of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.
[0020] The following describes embodiments with reference to the attached drawings so that those skilled in the art can easily implement the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.
[0021] In the embodiments of this specification, the term "user" may mean a person controlling a system, function, or operation.
[0022] FIG. 1 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0023] In step 110, an electronic device according to one embodiment can obtain first judgment request information regarding a task requiring a user's judgment.
[0024] The first judgment request information may refer to information provided to request a user to perform a judgment regarding a specific task. The first judgment request information may include information describing an object or situation that the user must judge during the course of performing the task. The first judgment request information may include information that induces the user to select or decide on one or more judgment results based on various data, conditions, or situations occurring during the course of performing the task. However, it is not limited thereto.
[0025] The first judgment request information may include at least one of work subject information, work situation information, work condition information, or work-related data. Work subject information may refer to information regarding a target object or target event for which the user must perform a judgment. Work situation information may refer to information describing the situation or context in which a judgment is made. Work condition information may refer to information including criteria, rules, or constraints that must be considered when performing a judgment. Work-related data may refer to information including data items referenced for performing a judgment.
[0026] For example, the first judgment request information may include transaction data, transaction time information, transaction amount information, or transaction location information for determining whether a transaction is abnormal in business related to financial transactions. For example, the first judgment request information may include insurance claim amount information, medical certificate information, treatment history information, or insurance subscription condition information for determining whether to pay an insurance claim in insurance underwriting business. For example, the first judgment request information may include dispute-related information, such as transaction records between a seller and a buyer, product description information, shipping information, or user inquiry details, in dispute processing business arising from an e-commerce platform. However, it is not limited thereto.
[0027] The first judgment request information may be generated by querying business-related data stored in a database. The first judgment request information may be generated based on data received from an external system. The first judgment request information may be generated based on business request data entered through a user interface. For example, an electronic device according to one embodiment may receive business processing request data from an internal business management system, a customer management system, or a transaction processing system, and generate the first judgment request information based on the received data. However, it is not limited thereto.
[0028] The first judgment request information may be configured in the form of structured data or unstructured data. For example, the first judgment request information may include structured data, such as transaction record data or insurance claim data, configured in a table-like data structure. For example, the first judgment request information may include unstructured data, such as text data, such as user inquiry messages, consultation records, or work description documents. For example, the first judgment request information may include multimedia data, such as image data, voice data, or video data. However, it is not limited thereto.
[0029] An electronic device according to one embodiment may obtain first judgment request information by receiving first judgment request information from an external server or an internal system via a network. An electronic device according to one embodiment may obtain first judgment request information by querying business data stored in a local storage device. An electronic device according to one embodiment may obtain first judgment request information by receiving business request information entered through a user interface. However, it is not limited thereto.
[0030] An electronic device according to one embodiment may store the acquired first judgment request information in the form of a data structure. For example, an electronic device according to one embodiment may store the first judgment request information in the form of a data record including a plurality of data fields. For example, an electronic device according to one embodiment may store the first judgment request information in a data structure including a task identifier, a time of task occurrence, task target data, and task description information. However, it is not limited thereto.
[0031] Consultation duties may include tasks in which a consultation representative determines and provides a response to a customer inquiry or consultation request. The content of the response may be determined based on the consultation representative's experience, operational guidelines, or consultation history. For example, consultation duties may include responding to inquiries regarding the use of financial services. For example, consultation duties may include responding to inquiries regarding order cancellations or refund requests for e-commerce services. However, they are not limited thereto.
[0032] An electronic device according to one embodiment may process consultation request information as one embodiment of judgment request information. The consultation request information may include inquiry content or a consultation request message entered by a customer. For example, the consultation request information may include the sentence "When will the refund be processed?" entered by a customer. For example, the consultation request information may include the sentence "Please tell me how to change my password" entered by a customer. However, it is not limited thereto.
[0033] An electronic device according to one embodiment may acquire consultation request information as first judgment request information. For example, the electronic device may receive consultation request data generated by a customer consultation system. For example, the electronic device may receive a customer message entered through a chat interface and store it as first judgment request information. However, it is not limited thereto.
[0034] In step 120, an electronic device according to one embodiment can obtain the user's first judgment data regarding the first judgment request information.
[0035] The first judgment data may refer to information representing the result of a judgment made by the user regarding the first judgment request information. The first judgment data may refer to data including the result of a judgment selected or decided by the user regarding a business subject or business situation included in the first judgment request information. The first judgment data may include information expressing the result derived from the user's decision-making process in data form.
[0036] The first judgment data may include at least one of binary selection data, multiple selection data, grade evaluation data, or numeric judgment data. Binary selection data may refer to data representing the result of selecting one of two options for a specific judgment target. Multiple selection data may refer to data representing the result of selecting one or more options from three or more options. Grade evaluation data may refer to data representing the result of evaluating a specific target according to a predefined grade system. Numeric judgment data may refer to data containing a judgment result expressed in numeric form for a specific target.
[0037] For example, the first judgment data may include binary selection data containing either "normal transaction" or "abnormal transaction" as a result of determining whether a financial transaction is abnormal. For example, the first judgment data may include multi-selection data containing either "payment approved," "additional review required," or "payment denied" as a result of determining whether to pay insurance benefits in an insurance claim review task. For example, the first judgment data may include grade evaluation data such as "low," "average," "high," or "urgent" as a result of evaluating the importance of an inquiry in a customer inquiry processing task. For example, the first judgment data may include numeric judgment data expressed as a score in the range of 1 to 5 points as a result of evaluating the reliability of a review in a product review review task. However, it is not limited thereto.
[0038] The first judgment data may be stored in a form corresponding to the first judgment request information. For example, the first judgment data may be stored in association with a request identifier that identifies the first judgment request information. For example, the first judgment data may be stored in the form of a data record together with the business target data included in the first judgment request information. However, it is not limited thereto.
[0039] An electronic device according to one embodiment may obtain first judgment data by receiving a judgment result input through a user interface. For example, an electronic device according to one embodiment may obtain first judgment data by displaying first judgment request information on a display and receiving input from a user through a button selection, a checkbox selection, or an input field. For example, an electronic device according to one embodiment may obtain first judgment data by receiving a judgment result selected by a user through touch input, mouse input, or keyboard input. However, it is not limited thereto.
[0040] An electronic device according to one embodiment may obtain first judgment data by receiving judgment result data transmitted from an external system. For example, an electronic device according to one embodiment may receive business processing result data from an internal business processing system, a screening management system, or a customer management system, and obtain the received business processing result data as first judgment data. However, it is not limited thereto.
[0041] An electronic device according to one embodiment may store first judgment request information and first judgment data in a corresponding form. For example, an electronic device according to one embodiment may combine the first judgment request information and the first judgment data into a single data record and store them. For example, an electronic device according to one embodiment may store them in the form of a data pair in which the first judgment request information is input data and the first judgment data is output data.
[0042] A representative may determine and input a response regarding the consultation request information. The response input by the representative may be stored as the first judgment data. For example, if a representative inputs the response "Refunds are processed within 3 business days after payment," that response may be stored as the first judgment data. For example, if a representative inputs the response "Password changes are available in the account settings menu," that response may be stored as the first judgment data. However, it is not limited to these examples.
[0043] In step 130, an electronic device according to one embodiment can input second judgment request information into an artificial intelligence model trained using first judgment request information and first judgment data as training data to obtain user prediction judgment result information regarding second judgment request information.
[0044] The second judgment request information may refer to judgment request information newly provided for a task requiring a user's judgment. The second judgment request information may be composed of a data structure identical or similar in form to the first judgment request information. The second judgment request information may include at least one of task target information, task situation information, task condition information, or task-related data.
[0045] For example, the second judgment request information may include transaction amount information, transaction time information, transaction location information, or counterparty account information for determining whether a financial transaction is abnormal. For example, the second judgment request information may include medical certificate information, treatment history information, or insurance contract condition information provided to determine whether to pay insurance benefits in insurance claim review operations. For example, the second judgment request information may include data including order records, payment information, shipping information, or user inquiry details related to dispute resolution operations occurring on an e-commerce platform. However, it is not limited thereto.
[0046] An artificial intelligence model may refer to a model trained to learn the user's judgment characteristics based on the correspondence relationship between first judgment request information and first judgment data. The artificial intelligence model may include a model trained to infer a judgment result that the user is likely to perform regarding the judgment request information provided as input data.
[0047] Artificial intelligence models may be constructed based on neural network structures. For example, an artificial intelligence model may be a model that includes deep neural networks, recurrent neural networks, convolutional neural networks, or transformer structures. For example, an artificial intelligence model may include a natural language processing model or a multimodal model capable of processing multiple input data to analyze text data or business data included in judgment request information. However, it is not limited thereto.
[0048] An electronic device according to one embodiment may provide second judgment request information as input data for an artificial intelligence model. The artificial intelligence model may analyze data features included in the input second judgment request information to infer a judgment result expected to be performed by the user regarding the second judgment request information.
[0049] The predicted judgment result information may refer to information indicating the judgment result that the user is predicted to perform regarding the second judgment request information. The predicted judgment result information may include at least one of a binary selection result, a multiple selection result, a grade evaluation result, or a numerical judgment result.
[0050] For example, the predictive judgment result information may include a judgment result indicating whether a specific transaction in financial transaction review tasks corresponds to either a normal transaction or an abnormal transaction. For example, the predictive judgment result information may include a judgment result indicating whether the payment of insurance benefits is approved or whether further review is required in insurance claim review tasks. For example, the predictive judgment result information may include a judgment result predicting the importance of an inquiry in customer inquiry processing tasks as a grade such as "low," "medium," "high," or "urgent." However, it is not limited thereto.
[0051] Predicted judgment result information can be generated in the form of output data of an artificial intelligence model. For example, the artificial intelligence model may receive second judgment request information as input and generate probability distribution data containing probability values corresponding to each of multiple judgment result candidates. For example, the artificial intelligence model may determine the judgment result having the highest probability value among the multiple judgment result candidates as the predicted judgment result information. However, it is not limited thereto.
[0052] An electronic device according to one embodiment can obtain prediction judgment result information output from an artificial intelligence model. For example, an electronic device according to one embodiment can obtain prediction judgment result information by receiving judgment result data or judgment result probability data generated as the output of an artificial intelligence model. However, it is not limited thereto.
[0053] An electronic device according to one embodiment may store acquired prediction judgment result information in the form of data. For example, an electronic device according to one embodiment may store second judgment request information and prediction judgment result information in a corresponding form. For example, an electronic device according to one embodiment may generate and store a data record including second judgment request information, prediction judgment result information, and judgment result probability information. However, it is not limited thereto.
[0054] An electronic device according to one embodiment can train an artificial intelligence model using consultation request information and response data from a consultation representative as training data. The artificial intelligence model can be trained to predict the response expected to be performed by the consultation representative regarding the content of the consultation request. For example, when a consultation request sentence such as "When is the refund processed?" is entered, the artificial intelligence model can predict a response such as "The refund will be processed within 3 business days after payment." For example, when a consultation request sentence such as "How to change the password?" is entered, the artificial intelligence model can predict a response such as "You can change the password in the account settings menu." However, it is not limited thereto.
[0055] An electronic device according to one embodiment can generate predicted response information by inputting the consultation request information into an artificial intelligence model when new consultation request information is input. The predicted response information may include consultation response content expected to be performed by a consultation representative. For example, the electronic device can predict a response sentence such as "guidance on refund processing period" in response to a consultation request sentence. For example, the electronic device can predict a response sentence such as "guidance on password change procedure" in response to a consultation request sentence. However, it is not limited thereto.
[0056] An electronic device according to one embodiment may generate user-specific personal profile information. User-specific personal profile information may refer to a data set for expressing the judgment characteristics of a specific user. For example, user-specific personal profile information may include parameter information representing the judgment patterns of a specific user. For example, user-specific personal profile information may include feature vector information representing the decision-making tendencies of a specific user. However, it is not limited thereto.
[0057] An electronic device according to one embodiment may store user-specific personal profile information in the form of internal parameters of an artificial intelligence model. The internal parameters of an artificial intelligence model may refer to learned numerical information that determines the relationship between input data and output results. For example, the internal parameters of an artificial intelligence model may include weights or bias values of a neural network model. For example, the internal parameters of an artificial intelligence model may include attention weights or feedforward network parameters of a transformer-based model. However, they are not limited thereto.
[0058] An electronic device according to one embodiment may store user-specific personal profile information in the form of a feature vector. A feature vector may refer to vector data that quantifies and expresses the judgment characteristics of a specific user. For example, a feature vector may include a numerical vector representing the user's decision-making tendencies. For example, a feature vector may include a vector composed of multiple numerical values representing problem-solving methods, risk avoidance tendencies, or customer service styles. However, it is not limited thereto.
[0059] An electronic device according to one embodiment can express differences in judgment characteristics between users using a feature vector. For example, a specific dimension value of the feature vector may indicate the degree to which a user prefers conservative decisions in the problem-solving process. For example, a specific dimension value of the feature vector may indicate the frequency with which a user uses empathetic expressions in response to customer inquiries. However, it is not limited thereto.
[0060] An electronic device according to one embodiment may store user-specific personal profile information in the form of a rule set. A rule set may refer to a set of rule information that defines judgment rules performed by a user in specific situations. For example, a rule set may include rules that define processing procedures to be performed preferentially when specific types of business request information are input. For example, a rule set may include decision rules defined to select a specific judgment result when specific conditions are satisfied. However, it is not limited thereto.
[0061] An electronic device according to one embodiment may store user-specific personal profile information in a form that combines a plurality of data structures. For example, user-specific personal profile information may simultaneously include learned parameters of an artificial intelligence model, feature vectors representing user judgment characteristics, and rule sets corresponding to specific business situations. For example, user-specific personal profile information may be composed of a hybrid structure that combines artificial intelligence model parameters and rule-based decision rules. However, it is not limited thereto.
[0062] An electronic device according to one embodiment can generate user-customized judgment results using user-specific personal profile information. For example, when the same judgment request information is input, the electronic device can generate different predicted judgment result information based on user-specific personal profile information. For example, the electronic device can generate a response reflecting User A's consultation style and a response reflecting User B's consultation style for the same customer inquiry message, respectively. However, it is not limited thereto.
[0063] In step 140, an electronic device according to one embodiment can obtain the user's second judgment data regarding the second judgment request information.
[0064] The second judgment data may refer to information representing the result of a judgment actually performed by the user regarding the second judgment request information. The second judgment data may include information expressing, in data form, the result of a judgment selected or decided by the user regarding a business subject, business situation, or business condition included in the second judgment request information. The second judgment data may refer to data representing the actual user judgment result corresponding to the second judgment request information.
[0065] The second judgment data may include at least one of binary selection data, multiple selection data, grade evaluation data, or numeric judgment data. Binary selection data may refer to data representing the result of selecting one of two options for a specific judgment target. Multiple selection data may refer to data representing the result of selecting one or more options from three or more options. Grade evaluation data may refer to data representing the result evaluated according to a predefined grade system for the judgment target. Numeric judgment data may refer to data containing a judgment result expressed as a numeric value for the judgment target.
[0066] For example, the second judgment data may include a judgment result indicating whether a specific transaction in a financial transaction review task corresponds to either a normal transaction or an abnormal transaction. For example, the second judgment data may include a judgment result selecting one of "payment approved," "additional review required," or "payment denied" regarding whether to pay insurance benefits in an insurance claim review task. For example, the second judgment data may include a judgment result evaluating the importance of an inquiry in a customer inquiry processing task as a grade such as "low," "average," "high," or "urgent." For example, the second judgment data may include a numerical judgment result evaluating the reliability of a review in a product review task as a score ranging from 1 to 5 points. However, it is not limited thereto.
[0067] An electronic device according to one embodiment may obtain second judgment data by receiving a judgment result input through a user interface. For example, an electronic device according to one embodiment may obtain second judgment data by displaying second judgment request information through a display and receiving input from a user through a button selection, a checkbox selection, or an input field. For example, an electronic device according to one embodiment may obtain second judgment data by receiving a judgment result selected by a user through touch input, mouse input, or keyboard input. However, it is not limited thereto.
[0068] An electronic device according to one embodiment may acquire second judgment data by receiving a user's judgment result through voice input or gesture input. For example, an electronic device according to one embodiment may convert a judgment result spoken by a user into text data through a voice recognition module and acquire the converted text data as second judgment data. For example, an electronic device according to one embodiment may analyze a user's gesture recognized through a camera or sensor to identify the user's judgment selection result and acquire the identified result as second judgment data. However, it is not limited thereto.
[0069] An electronic device according to one embodiment can obtain second judgment data by receiving business processing result data transmitted from an external system. For example, an electronic device according to one embodiment may receive business processing result data from an internal business management system, a screening management system, or a customer management system, and obtain the received business processing result data as second judgment data. However, it is not limited thereto.
[0070] The second judgment data may be stored in a form corresponding to the second judgment request information. For example, the second judgment data may be stored in association with a request identifier that identifies the second judgment request information. For example, an electronic device according to one embodiment may combine the second judgment request information and the second judgment data into a single data record for storage. For example, an electronic device according to one embodiment may store the data in the form of a data pair in which the second judgment request information is input data and the second judgment data is output data. Such data pairs may be used for comparative analysis between predicted judgment result information and actual judgment results or for performance evaluation of an artificial intelligence model. However, they are not limited thereto.
[0071] In step 150, an electronic device according to one embodiment can calculate the reliability of the user's predicted judgment result information regarding the second judgment request information based on comparing the second judgment data and the user's predicted judgment result information regarding the second judgment request information.
[0072] The reliability of predictive judgment result information may refer to information indicating the extent to which the predictive judgment result generated by the artificial intelligence model matches the actual user's judgment. The reliability of predictive judgment result information may include a value that quantitatively expresses the accuracy or reliability of the prediction result generated by the artificial intelligence model regarding specific judgment request information. The reliability of predictive judgment result information may be expressed in the form of a numerical value and may be calculated based on the comparison result between the predictive judgment result and the actual judgment result.
[0073] An electronic device according to one embodiment can determine whether there is a match between second judgment data and predicted judgment result information. For example, if the second judgment data is determined to be an "abnormal transaction" and the predicted judgment result information is also predicted to be an "abnormal transaction," the two judgment results may be determined to be consistent. For example, if the second judgment data is determined to be a "payment refusal" and the predicted judgment result information is predicted to be a "payment approval," the two judgment results may be determined to be different. However, the invention is not limited thereto.
[0074] An electronic device according to one embodiment can calculate the reliability of the predicted judgment result information based on whether there is a match between the judgment results. For example, if the predicted judgment result information and the second judgment data match, the reliability value can be set to 1, and if the two judgment results differ from each other, the reliability value can be set to 0. For example, the reliability value can be set as a binary evaluation value expressed as 0 or 1. However, it is not limited thereto.
[0075] An electronic device according to one embodiment can calculate the reliability of predicted judgment result information based on comparison results accumulated for a plurality of judgment request information. For example, an electronic device according to one embodiment can determine whether there is a match between the predicted judgment result information and the actual judgment result for the last 100 judgment request information, and if 85 of the judgment results match the actual judgment result, the reliability value can be calculated as 0.85. For example, if 150 of the last 200 judgment request information match the actual judgment result, the reliability value can be calculated as 0.75. However, it is not limited thereto.
[0076] The reliability of the prediction judgment result information may be expressed in the form of a probability value. For example, the reliability of the prediction judgment result information may be expressed as a probability value in the range of 0 to 1. For example, the reliability of the prediction judgment result information may include probability values such as 0.60, 0.75, or 0.92. For example, if the reliability value is 0.90 or higher, it may be determined that the prediction judgment result information is likely to match the actual user judgment to a very high degree. For example, if the reliability value is 0.50 or lower, it may be determined that the prediction judgment result information is likely to match the actual user judgment to a low degree. However, it is not limited thereto.
[0077] The reliability of the prediction judgment result information may be expressed in the form of a score value. For example, the reliability of the prediction judgment result information may be expressed as a score value ranging from 0 to 100 points. For example, if the agreement rate between the prediction judgment result information and the actual judgment result for recent judgment request information is 82%, the reliability value may be set to 82 points. For example, if the agreement rate between the prediction judgment result and the actual judgment result is 65%, the reliability value may be set to 65 points. However, it is not limited thereto.
[0078] An electronic device according to one embodiment can calculate the reliability of the predicted judgment result information by considering the probability value of the judgment result output by an artificial intelligence model together with the second judgment data. For example, if the artificial intelligence model outputs a probability value of 0.93 for a specific judgment result and the judgment result matches the actual second judgment data, the reliability value can be set to 0.93. For example, if the artificial intelligence model outputs a probability value of 0.88 for a specific judgment result but the judgment result does not match the actual judgment result, the reliability value can be set to 0.12 or 0. However, it is not limited thereto.
[0079] An electronic device according to one embodiment can calculate the reliability of predicted judgment result information according to a judgment result type or a task type. For example, an electronic device according to one embodiment may set the reliability value for a task type to 0.91 if the agreement ratio between the predicted judgment result and the actual judgment result for a financial transaction review task is 0.91. For example, if the agreement ratio between the predicted judgment result and the actual judgment result for an insurance underwriting task is 0.78, the reliability value for a task type may be set to 0.78. However, it is not limited thereto.
[0080] An electronic device according to one embodiment may store the reliability of calculated prediction judgment result information in the form of data. For example, an electronic device according to one embodiment may generate and store a data record including second judgment request information, prediction judgment result information, second judgment data, and a calculated reliability value. For example, an electronic device according to one embodiment may store record data including a judgment request identifier, prediction judgment result information, actual judgment result, and reliability value in a database. However, it is not limited thereto.
[0081] An electronic device according to one embodiment may determine whether to utilize prediction judgment result information based on a calculated reliability value. For example, an electronic device according to one embodiment may determine the prediction judgment result information as a reliable result if the reliability value is 0.80 or higher or 80 points or higher. For example, an electronic device according to one embodiment may provide the prediction judgment result information to a user as reference information if the reliability value is in the range of 0.60 or higher and less than 0.80. For example, an electronic device according to one embodiment may not use the prediction judgment result information for automatic judgment and may set it as a subject for user review if the reliability value is less than 0.60. However, it is not limited thereto.
[0082] In step 160, the electronic device according to one embodiment inputs third judgment request information into an artificial intelligence model when the reliability is above a threshold value, and the third The user's predicted judgment result information regarding the judgment request information can be output through the display.
[0083] The third judgment request information may refer to judgment request information newly provided in relation to a task requiring the user's judgment. The third judgment request information may be composed of a data structure identical or similar in form to the first judgment request information or the second judgment request information. The third judgment request information may include at least one of task target information, task situation information, task condition information, or task-related data.
[0084] For example, third-party judgment request information may include transaction amount information, transaction time information, transaction location information, or counterparty account information provided to determine whether a specific transaction is abnormal in financial transaction review tasks. For example, third-party judgment request information may include medical certificate information, treatment history information, or insurance contract condition information provided to determine whether to pay insurance benefits in insurance claim review tasks. For example, third-party judgment request information may include data including order records, payment information, shipping information, or user inquiry details related to dispute resolution tasks occurring on an e-commerce platform. However, it is not limited thereto.
[0085] Reliability may refer to a value indicating the degree to which the prediction judgment result information generated by the artificial intelligence model matches the actual user's judgment result. Reliability may include a value calculated based on the comparison result between the second judgment data and the prediction judgment result information. Reliability may be expressed in the form of a probability value in the range of 0 to 1 or a score value in the range of 0 to 100. For example, if the reliability value is high, such as 0.85, 0.90, or 0.95, it may be determined that the artificial intelligence model reflects the user's judgment characteristics with high accuracy. For example, if the reliability value is 85 points or 90 points or higher, it may be determined that the prediction judgment result information generated by the artificial intelligence model matches the actual user's judgment to a high degree. However, it is not limited thereto.
[0086] A threshold value may refer to a standard value set to determine whether to provide information on the prediction judgment results generated by an artificial intelligence model to a user. The threshold value may include a pre-set confidence standard value. For example, the threshold value may be set to a probability value such as 0.80, 0.85, or 0.90. For example, the threshold value may be set to a score value such as 80 points, 85 points, or 90 points. However, it is not limited thereto.
[0087] An electronic device according to one embodiment may generate predicted judgment result information for new judgment request information using an artificial intelligence model when the calculated confidence value is greater than or equal to a threshold value. For example, if the calculated confidence value is 0.90 and the threshold value is set to 0.85, the electronic device may generate predicted judgment result information for the third judgment request information using an artificial intelligence model. For example, if the calculated confidence value is 92 points and the threshold value is set to 85 points, the electronic device may generate predicted judgment result information for the third judgment request information using an artificial intelligence model. However, it is not limited thereto.
[0088] An electronic device according to one embodiment may provide third judgment request information as input data for an artificial intelligence model. The artificial intelligence model may analyze data features included in the third judgment request information to generate a judgment result predicted to be performed by the user regarding the third judgment request information.
[0089] The predicted judgment result information may refer to information indicating the judgment result that the user is predicted to perform regarding the third judgment request information. The predicted judgment result information may include at least one of a binary selection result, a multiple selection result, a grade evaluation result, or a numerical judgment result.
[0090] For example, the predictive judgment result information may include a judgment result indicating whether a specific financial transaction falls under either a "normal transaction" or an "abnormal transaction." For example, in insurance claim review operations, the predictive judgment result information may include a judgment result regarding whether to pay insurance benefits, such as "payment approved," "additional review required," or "payment denied." For example, in customer inquiry processing operations, the predictive judgment result information may include a judgment result predicting the importance of an inquiry as a grade such as "low," "medium," "high," or "urgent." However, it is not limited thereto.
[0091] An electronic device according to one embodiment can acquire prediction judgment result information output from an artificial intelligence model. An electronic device according to one embodiment can output the acquired prediction judgment result information through a display to provide it to a user.
[0092] For example, an electronic device according to one embodiment may display predicted judgment result information along with third judgment request information on a display screen. For example, an electronic device according to one embodiment may display information including a probability value along with a predicted judgment result on a display screen, such as "Probability of abnormal transaction: 92%". For example, an electronic device according to one embodiment may display predicted judgment result information in the form of "Payment approval (predicted probability: 0.88)" in an insurance underwriting task. However, it is not limited thereto.
[0093] An electronic device according to one embodiment may provide prediction judgment result information in a visual form. For example, an electronic device according to one embodiment may display the prediction judgment result information on a display in the form of text, a graph, or a color display. For example, an electronic device according to one embodiment may visualize and display the result information by providing a green indicator for a prediction judgment result with high reliability and a yellow indicator when reliability is relatively low. However, it is not limited thereto.
[0094] An electronic device according to one embodiment may store result data including third judgment request information, prediction judgment result information, and a confidence value. For example, an electronic device according to one embodiment may generate a data record including a third judgment request information identifier, prediction judgment result information, a prediction probability value, and result output time information and store it in a database. However, it is not limited thereto.
[0095] An electronic device according to one embodiment may provide predicted response information to a counselor. For example, the electronic device may display a predicted response sentence in the form of a "recommended response" on a counseling screen. For example, the electronic device may display a predicted response sentence in an autocomplete form at the top of an input area on a counseling screen. However, it is not limited thereto.
[0096] An electronic device according to one embodiment can automatically transmit predicted response information to a customer when the reliability of the predicted response information is above a threshold value. For example, the electronic device can automatically transmit a response sentence, such as "Notice of refund processing period," to a customer chat screen. For example, the electronic device can automatically provide a response sentence, such as "Notice of how to change password," through a customer chat interface. However, it is not limited thereto.
[0097] An electronic device according to one embodiment can calculate the reliability of an artificial intelligence model by comparing the response content actually entered by a counselor with the predicted response content. For example, the electronic device can calculate the semantic similarity between a predicted response sentence and an actual counseling response sentence. For example, the electronic device can calculate the prediction accuracy by using whether the counseling response classification results match. However, it is not limited thereto.
[0098] An electronic device according to one embodiment can retrain an artificial intelligence model using consultation response prediction results and actual consultation response data as retraining data. For example, the electronic device may store consultation request sentences, predicted response sentences, and actual consultation response sentences as retraining data. For example, the electronic device may update an artificial intelligence model to learn the response patterns of a consultation agent using accumulated consultation data. However, it is not limited thereto.
[0099] A personalized consultation chatbot may refer to an AI-based consultation system configured to generate consultation responses by learning the judgment characteristics of a user performing consultation tasks. A personalized consultation chatbot can generate consultation responses that reflect that user's judgment criteria, response methods, or problem-solving patterns by learning the consultation records of a specific user performing consultation tasks. For example, a personalized consultation chatbot can generate responses to inquiries about financial products by learning the consultation records of a financial service consultant. For example, a personalized consultation chatbot can generate responses to inquiries regarding order cancellation or refund requests by learning the consultation records of an e-commerce customer center consultant. However, it is not limited to these examples.
[0100] An electronic device according to one embodiment may process consultation request information as one embodiment of judgment request information. The consultation request information may include the content of an inquiry transmitted by a customer through a consultation channel. For example, the consultation request information may include a message "When will the delivery arrive?" entered by a customer into a chat interface. For example, the consultation request information may include a message "How is a refund request processed?" entered by a customer. However, it is not limited thereto.
[0101] An electronic device according to one embodiment may store a consultation response entered by a consultation representative regarding consultation request information as judgment data. For example, if a consultation representative enters a response such as "Delivery arrives within an average of 2 to 3 days after ordering," such response may be stored as judgment data. For example, if a consultation representative enters a response such as "You can apply for a refund on My Page," such response may be stored as judgment data. However, it is not limited thereto.
[0102] An electronic device according to one embodiment can manage consultation data based on an identification code corresponding to a specific consultation agent. The identification code may refer to code information for identifying a consultation agent. For example, the identification code may include a consultation agent ID or account identification information. For example, the identification code may include a consultation agent number or consultation system user identification information. However, it is not limited thereto.
[0103] An electronic device according to one embodiment can separate and store consultation data of a specific consultant based on an identification code. For example, the electronic device can store consultation data of consultant A separately from consultation data of consultant B. For example, the electronic device can manage the chat consultation records of consultant A as a separate data set. However, it is not limited thereto.
[0104] An electronic device according to one embodiment can train an artificial intelligence model corresponding to a specific counselor by using counseling data of that counselor as training data. For example, the electronic device can train an artificial intelligence model AI_A using counseling data of counselor A. For example, the electronic device can train an artificial intelligence model AI_B using counseling data of counselor B. However, it is not limited thereto.
[0105] An electronic device according to one embodiment may perform a consultation data learning step. The consultation data learning step may refer to a step of training an artificial intelligence model using past consultation data of a specific consultant. For example, the electronic device may collect past chat consultation records of veteran consultant A. For example, the electronic device may collect consultation record data including customer inquiry content, consultant A's response content, and consultation processing results. However, it is not limited thereto.
[0106] An electronic device according to one embodiment may perform a counselor support step. The counselor support step may refer to a step of providing a draft counseling response generated by an artificial intelligence model to a counselor. For example, when a new counselor performs a chat counseling with a customer, the electronic device may input counseling request information into the artificial intelligence model. For example, the electronic device may display a draft counseling response generated by the artificial intelligence model on a counseling screen. However, it is not limited thereto.
[0107] An electronic device according to one embodiment may perform an automatic consultation response step when a consultation representative is absent. The automatic consultation response step may refer to a step in which an artificial intelligence model generates a consultation response based on the consultation method of a specific consultation representative. For example, if a customer inquiry is received during a time when consultation representative A is not working, the electronic device may input consultation request information into the artificial intelligence model AI_A. For example, the electronic device may provide the consultation response generated by the artificial intelligence model AI_A through a customer chat interface. However, it is not limited thereto.
[0108] An electronic device according to one embodiment may store the results of an automated consultation response as consultation data. For example, the electronic device may store consultation request information, generated consultation responses, and consultation processing results as data records. For example, the electronic device may use the stored consultation data as data for training an artificial intelligence model. However, it is not limited thereto.
[0109] An electronic device according to one embodiment may provide the results of an automated consultation response for a consultation representative to review. For example, the electronic device may display an automated consultation response record when the consultation representative logs into a business system. For example, the electronic device may provide a user interface for the consultation representative to check the automatically generated consultation response. However, it is not limited thereto.
[0110] An electronic device according to one embodiment can retrain an artificial intelligence model using the results of a counselor's review as retraining data. For example, the electronic device may store counseling request information, predicted counseling responses, and actual counseling responses modified by the counselor as retraining data. For example, the electronic device may update the artificial intelligence model to reflect the counselor's counseling style using the stored retraining data. However, it is not limited thereto.
[0111] FIG. 2 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0112] Steps 110 and 120 of Fig. 2 correspond to steps 110 and 120 of Fig. 1, respectively.
[0113] In step 210, an electronic device according to one embodiment may determine an identification code to identify a user among a plurality of users who will process the first judgment request information based on the first judgment request information.
[0114] An identification code may refer to identification information used to distinguish or identify a specific user among multiple users. The identification code may include user identification data for designating the user responsible for processing specific judgment request information. The identification code may include a user identifier used to distinguish user accounts in an electronic device or business processing system.
[0115] The identification code may be configured in the form of a character code, a numeric code, or a code combining characters and numbers. For example, the identification code may include a user ID for identifying a user account. For example, the identification code may include an employee number, account number, or user code value generated to identify a user account in an internal business system. For example, the identification code may include a user identification code composed of a combination of characters and numbers, such as "USR001", "EMP1023", or "A-2034". However, it is not limited thereto.
[0116] An electronic device according to one embodiment may determine a user to process a corresponding task by analyzing task type information or task target information included in the first judgment request information. For example, if the first judgment request information corresponds to a financial transaction review task, the electronic device may identify one user among the users belonging to the user group responsible for financial transaction review tasks. For example, if the first judgment request information corresponds to an insurance underwriting task, the electronic device may identify one user among the users belonging to the user group responsible for insurance underwriting tasks. However, it is not limited thereto.
[0117] An electronic device according to one embodiment may determine a user to process a corresponding task by utilizing task attribute information included in the first judgment request information. For example, if the transaction amount included in the first judgment request information is greater than or equal to a preset amount threshold, the electronic device may determine an identification code corresponding to a user having the authority to review high-risk transactions. For example, if the insurance claim amount included in the first judgment request information is greater than or equal to a certain amount, the electronic device may determine an identification code corresponding to a user responsible for high-value insurance underwriting. However, it is not limited thereto.
[0118] An electronic device according to one embodiment may determine an identification code using user role information or user authorization information. User role information may refer to information indicating the type of work that a specific user can perform or the area of work in charge. User authorization information may refer to information indicating the scope of work that a specific user can access or process.
[0119] For example, user role information may include role information such as "transaction reviewer," "insurance underwriter," or "customer inquiry handler." For example, user authority information may include authority to review transactions exceeding a certain amount, authority to review specific types of insurance claims, or authority to handle inquiries regarding specific customer tiers. However, it is not limited thereto.
[0120] An electronic device according to one embodiment can determine an identification code based on user information stored in a user database. The user database may refer to a storage area that stores data including user identification information, user role information, or user authority information for a plurality of users.
[0121] For example, the user database may include user records containing a user ID, user name, user role information, and user authority information. For example, the user database may include user role information indicating that the user corresponding to "USR001" is a user responsible for financial transaction review tasks. However, it is not limited thereto.
[0122] An electronic device according to one embodiment can identify a user to handle a corresponding task by comparing task type information included in the first judgment request information with user role information stored in a user database. For example, if the first judgment request information corresponds to "abnormal transaction review task," the electronic device can identify one of the users having "abnormal transaction reviewer" role information in the user database. For example, if the first judgment request information corresponds to "insurance claim review task," the electronic device can identify one of the users having "insurance reviewer" role information in the user database. However, it is not limited thereto.
[0123] An electronic device according to one embodiment can determine an identification code corresponding to an identified user. For example, the electronic device can determine a user ID corresponding to an identified user in a user database as the identification code. For example, the electronic device can determine an account number or employee number corresponding to an identified user as the identification code. However, it is not limited thereto.
[0124] An electronic device according to one embodiment may store a determined identification code together with first judgment request information. For example, an electronic device according to one embodiment may generate a data record including the first judgment request information and the identification code and store it in a database. For example, an electronic device according to one embodiment may store data including a first judgment request information identifier, an identification code, and judgment request creation time information. However, it is not limited thereto.
[0125] In step 220, an electronic device according to one embodiment can identify a user corresponding to an identification code among a plurality of users.
[0126] An electronic device according to one embodiment can identify a user corresponding to an identification code using user information stored in a user database. The user database may refer to a storage area that stores data including user identification information, user account information, user role information, or user authority information for a plurality of users.
[0127] For example, the user database may include user records containing a user ID, user name, user role information, and user authority information. For example, the user database may store a user record containing the name, assigned duties, and account information of a user corresponding to an identification code "USR001". For example, the user database may include role information indicating that a user corresponding to an identification code "EMP1023" is a user responsible for insurance underwriting duties. However, it is not limited thereto.
[0128] An electronic device according to one embodiment can compare an identification code with user identification information stored in a user database. The electronic device can search for user identification information among the user identification information stored in the user database that is identical to or corresponds to the identification code. As a result of the search, the electronic device can identify a user record corresponding to the identification code.
[0129] For example, the electronic device can search for a user record with a user ID of "USR001" among the user records stored in the user database. The electronic device can search for a user record with an employee number of "1023" among the user records stored in the user database. The electronic device can search for a user record with an account number of "A-2045" among the user records stored in the user database. However, it is not limited thereto.
[0130] An electronic device according to one embodiment can identify a user corresponding to an identification code by verifying a user record corresponding to the identification code. For example, the electronic device can verify in a user database a user record where the username is "Hong Gil-dong" that corresponds to an identification code "USR001". For example, the electronic device can verify in a user database that a user corresponding to an identification code "EMP1023" is a user responsible for financial transaction review tasks. However, it is not limited thereto.
[0131] An electronic device according to one embodiment may further acquire user information related to an identified user. The user information may include a user name, user role information, user authority information, or user department affiliation information.
[0132] For example, the electronic device can obtain the user's name, department information, and assigned duties corresponding to the identification code "USR001". For example, the electronic device can obtain the user's role information and authority information corresponding to the identification code "EMP1023". However, it is not limited thereto.
[0133] An electronic device according to one embodiment may identify a user corresponding to an identification code and then designate that user as the user to process the first judgment request information. For example, the electronic device may set a user account corresponding to the identification code as the user account responsible for the first judgment request information. For example, the electronic device may store the first judgment request information and the identified user account in a corresponding form. However, it is not limited thereto.
[0134] An electronic device according to one embodiment may store first judgment request information and identified user information as a single data record. For example, the electronic device may generate a data record including a first judgment request information identifier, an identification code, user account information, and judgment request creation time information, and store it in a database. However, it is not limited thereto.
[0135] FIG. 3 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0136] Steps 110 and 120 of FIG. 3 correspond to steps 110 and 120 of FIG. 1 and FIG. 2, respectively, and steps 130 and 140 of FIG. 3 correspond to steps 130 and 140 of FIG. 1, respectively.
[0137] In step 310, an electronic device according to one embodiment can obtain first judgment data based on an identification code.
[0138] An electronic device according to one embodiment can search for first judgment data based on a user corresponding to an identification code.
[0139] An electronic device according to one embodiment can search for a judgment data storage area using first judgment request information and an identification code. The judgment data storage area may include a database, a log storage, or a business process record storage that stores user judgment result data.
[0140] For example, the judgment data storage area may store a judgment data record containing multiple judgment records. The judgment data record may include a judgment request identifier, a user identification code, judgment result information, and judgment execution time information. For example, the judgment data record may include data such as "Judgment Request ID: JR-20240101-001", "User ID: USR001", "Judgment Result: Abnormal Transaction", and "Judgment Time: 2024-01-01 10:35:21". However, it is not limited thereto.
[0141] An electronic device according to one embodiment can search a judgment data storage area based on user identification information corresponding to an identification code. For example, the electronic device can search for a judgment data record where the user ID is "USR001". For example, the electronic device can search for a judgment data record where the user number is "EMP1023". However, it is not limited thereto.
[0142] An electronic device according to one embodiment can identify judgment data corresponding to a first judgment request information among the searched judgment data records. For example, the electronic device can identify a judgment data record in which a judgment request identifier corresponds to the first judgment request information. For example, the electronic device can identify judgment result data corresponding to the judgment request identifier "JR-20240101-001". However, it is not limited thereto.
[0143] An electronic device according to one embodiment may acquire judgment result information included in an identified judgment data record as first judgment data. For example, the electronic device may acquire a judgment result value of "abnormal transaction" included in the judgment data record as first judgment data. For example, the electronic device may acquire a judgment result value of "payment approval" included in the judgment data record as first judgment data. However, it is not limited thereto.
[0144] An electronic device according to one embodiment may store the acquired first judgment data in a form corresponding to the first judgment request information. For example, the electronic device may generate a data record including a first judgment request information identifier, an identification code, and the first judgment data and store it in a database. For example, the electronic device may generate and store a training data record including the first judgment request information, an identification code, the first judgment data, and judgment execution time information. However, it is not limited thereto.
[0145] In step 320, an electronic device according to one embodiment can train an artificial intelligence model based on an identification code, first judgment request information, and first judgment data.
[0146] An electronic device according to one embodiment may generate a training data set using an identification code, first judgment request information, and first judgment data. The training data set may refer to a set of data configured for training an artificial intelligence model. The training data set may include a plurality of training samples.
[0147] For example, an electronic device may set first judgment request information as input data and first judgment data as correct answer data to generate a training data set for training an artificial intelligence model. However, it is not limited thereto.
[0148] Each training sample may be composed of a data structure including an identification code, first judgment request information, and first judgment data. For example, one training sample may be composed of a data structure including a user identification code "USR001", judgment request information "Transaction amount: 2,500,000 won, Transaction location: Overseas, Transaction time: 02:15", and a judgment result "Abnormal transaction". For example, one training sample may be composed of a data structure including a user identification code "EMP1023", judgment request information "Insurance claim amount: 4,500,000 won, Treatment period: 3 days, Diagnosis certificate code: D123", and a judgment result "Payment approved". However, it is not limited thereto.
[0149] An electronic device according to one embodiment may configure a user-specific learning data set based on an identification code. The electronic device may classify learning samples having the same identification code into a single user data group. For example, the electronic device may configure a plurality of judgment request information and judgment result data corresponding to the identification code "USR001" into a single user learning data set. For example, the electronic device may configure a plurality of judgment request information and judgment result data corresponding to the identification code "EMP1023" into another user learning data set. However, it is not limited thereto.
[0150] An electronic device according to one embodiment can train an individual artificial intelligence model for each user using a user-specific learning data set. A user-specific artificial intelligence model may refer to a model configured to learn judgment characteristics corresponding to a specific user. A user-specific artificial intelligence model may be trained using judgment request information and judgment result data related to the user.
[0151] For example, the electronic device may train a first user artificial intelligence model using only the judgment data corresponding to the identification code "USR001". For example, the electronic device may train a second user artificial intelligence model using only the judgment data corresponding to the identification code "EMP1023". In this way, an independent artificial intelligence model for each user may be trained using judgment data sets corresponding to different users. However, it is not limited thereto.
[0152] An electronic device according to one embodiment can iteratively update model parameters during the learning process of an artificial intelligence model. For example, the electronic device can calculate the difference between a prediction result and an actual judgment result using a loss function. For example, the electronic device can update the weight parameters of the artificial intelligence model using a backpropagation algorithm. However, it is not limited thereto.
[0153] An electronic device according to one embodiment can train an artificial intelligence model by repeatedly using a plurality of training samples. For example, the electronic device can repeatedly train an artificial intelligence model using 1,000 or more training samples or 10,000 or more training samples. For example, the electronic device can improve the prediction accuracy of the artificial intelligence model by repeatedly inputting a training data set over a plurality of training epochs. However, it is not limited thereto.
[0154] An electronic device according to one embodiment may store an artificial intelligence model trained for each user. For example, the electronic device may store a first user artificial intelligence model corresponding to the identification code "USR001". For example, the electronic device may store a second user artificial intelligence model corresponding to the identification code "EMP1023". For example, the electronic device may create and store a model management table that corresponds user identification codes with artificial intelligence model identifiers. However, it is not limited thereto.
[0155] An electronic device according to one embodiment may subsequently select and use an artificial intelligence model corresponding to a specific user when judgment request information is input. For example, when judgment request information containing the identification code "USR001" is input, the electronic device may select a first user artificial intelligence model to generate a predicted judgment result. For example, when judgment request information containing the identification code "EMP1023" is input, the electronic device may select a second user artificial intelligence model to generate a predicted judgment result. However, it is not limited thereto.
[0156] FIG. 4 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0157] Step 140 of Fig. 4 corresponds to Step 140 of Fig. 1 and Fig. 3, and Steps 150 and 160 of Fig. 4 correspond to Steps 150 and 160 of Fig. 1, respectively.
[0158] In step 410, an electronic device according to one embodiment can calculate reliability based on at least one of the degree of agreement between the user’s predicted judgment result information regarding the second judgment data and the second judgment request information, or the degree of agreement between the distribution of the first judgment data and the distribution of the second judgment data.
[0159] An electronic device according to one embodiment can calculate the degree of agreement between two data by comparing the user's predicted judgment result information regarding the second judgment data and the second judgment request information. The degree of agreement may refer to a value indicating how much the prediction result generated by the artificial intelligence model matches the actual user judgment result.
[0160] For example, if the second judgment data is "abnormal transaction" and the prediction judgment result information is "abnormal transaction (prediction probability 0.91)", the electronic device may determine that the two data match. For example, if the second judgment data is "normal transaction" and the prediction judgment result information is "abnormal transaction (prediction probability 0.76)", the electronic device may determine that the two data do not match.
[0161] An electronic device according to one embodiment can calculate a match ratio by comparing second judgment data and predicted judgment result information with respect to a plurality of second judgment request information. For example, the electronic device can calculate the degree of match as 0.90 if the predicted result and the actual judgment result match in 90 of the 100 judgment request data. For example, the electronic device can calculate the degree of match as 0.85 if the predicted result and the actual judgment result match in 170 of the 200 judgment request data. However, it is not limited thereto.
[0162] An electronic device according to one embodiment can calculate reliability using the degree of agreement between the distribution of first judgment data and the distribution of second judgment data. The distribution may refer to the ratio of judgment result types appearing in a plurality of judgment data.
[0163] For example, if the "normal transaction" judgment result in the first judgment data is 70% and the "abnormal transaction" judgment result is 30%, the distribution of the first judgment data may include a ratio of 0.70 for normal transactions and 0.30 for abnormal transactions. For example, if the "normal transaction" judgment result in the second judgment data is 68% and the "abnormal transaction" judgment result is 32%, the distribution of the second judgment data may include a ratio of 0.68 for normal transactions and 0.32 for abnormal transactions.
[0164] An electronic device according to one embodiment can calculate the degree of distribution agreement by calculating the difference between a first judgment data distribution and a second judgment data distribution. For example, if the difference in the normal transaction ratio is 0.02 and the difference in the abnormal transaction ratio is 0.02, the electronic device may determine that the difference between the two distributions is small. For example, if the first judgment data distribution is normal transactions 0.70 and abnormal transactions 0.30 and the second judgment data distribution is normal transactions 0.50 and abnormal transactions 0.50, the electronic device may determine that the difference between the two distributions is large. However, it is not limited thereto.
[0165] An electronic device according to one embodiment can calculate an additional distribution match using the distribution of the user's recent judgment data and the distribution of past judgment data. Recent judgment data may refer to judgment result data performed by the user during a recent certain period. Past judgment data may refer to judgment result data accumulated during a period prior to the recent judgment data.
[0166] For example, recent judgment data may include the data of the last 30 judgment results. For example, recent judgment data may include the data of 50 judgment results performed over the last 7 days. For example, past judgment data may include the data of 2,000 or more judgment results accumulated over the previous 6 months. However, it is not limited thereto.
[0167] An electronic device according to one embodiment can calculate the degree of change in a user's judgment pattern by comparing the distribution of recent judgment data with the distribution of past judgment data. For example, if the ratio of "normal transactions" in past judgment data is 0.70 and the ratio of "abnormal transactions" is 0.30, the electronic device can set the corresponding distribution as the past judgment distribution. For example, if the ratio of "normal transactions" in recent judgment data is 0.72 and the ratio of "abnormal transactions" is 0.28, the electronic device can determine that the difference between the two distributions is small, such as 0.02 or less. For example, if the ratio of "normal transactions" in recent judgment data is 0.55 and the ratio of "abnormal transactions" is 0.45, the electronic device can determine that the difference between the two distributions has increased significantly. However, it is not limited thereto.
[0168] An electronic device according to one embodiment can calculate the stability of a judgment pattern using the difference between a recent judgment data distribution and a past judgment data distribution. For example, if the difference between the two distributions is 0.03 or less, the electronic device may determine that the user judgment pattern is stable. For example, if the difference between the two distributions is 0.15 or more, the electronic device may determine that the user judgment pattern has changed. However, it is not limited thereto.
[0169] An electronic device according to one embodiment may calculate reliability using at least one of the degree of agreement between second judgment data and prediction judgment result information, the degree of agreement between a first judgment data distribution and a second judgment data distribution, or the degree of agreement between a recent judgment data distribution and a past judgment data distribution. For example, if the prediction result agreement is 0.90, the distribution agreement is 0.92, and the judgment pattern stability is 0.88, the electronic device may calculate reliability as 0.90 by averaging the three values. For example, if the prediction result agreement is 0.85, the distribution agreement is 0.80, and the judgment pattern stability is 0.78, the electronic device may calculate reliability as 0.82 using a weighted average. However, it is not limited thereto.
[0170] An electronic device according to one embodiment may express the calculated reliability in the form of a probability value or a score value. For example, the electronic device may calculate the reliability in the form of a probability value such as 0.91 or 0.83. For example, the electronic device may calculate the reliability in the form of a score value such as 91 points or 83 points. However, it is not limited thereto.
[0171] An electronic device according to one embodiment can determine to what extent the prediction result of an artificial intelligence model accurately reflects the characteristics of actual user judgment using a calculated reliability. For example, if the reliability is 0.90 or higher, the electronic device may determine that the prediction result of the artificial intelligence model matches the user judgment to a high degree. For example, if the reliability is 0.70 or lower, the electronic device may determine that the prediction result of the artificial intelligence model does not sufficiently reflect the characteristics of user judgment. However, it is not limited thereto.
[0172] FIG. 5 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0173] Step 160 of Fig. 5 corresponds to Step 160 of Figs. 1 and 4.
[0174] In step 510, if the reliability is below a threshold value, the electronic device according to one embodiment can retrain an artificial intelligence model using the second judgment request information, the user's prediction judgment result information regarding the second judgment request information, and the second judgment data as retraining data.
[0175] An electronic device according to one embodiment may determine that the predictive performance of an artificial intelligence model does not sufficiently reflect user judgment characteristics when the calculated confidence level is below a threshold. For example, if the threshold is set to 0.85 and the confidence level is 0.72 or 0.68, the electronic device may determine that the predictive performance of the artificial intelligence model is insufficient. For example, if the threshold is set to 85 points and the confidence level is 70 points or 75 points, the electronic device may determine that the predictive performance of the artificial intelligence model does not meet the standard level.
[0176] An electronic device according to one embodiment may configure second judgment request information, prediction judgment result information, and second judgment data as retraining data when the reliability is determined to be below a threshold value. Retraining data may refer to training data additionally used to complement or improve an existing artificial intelligence model.
[0177] An electronic device according to one embodiment may configure second judgment request information, predicted judgment result information, and second judgment data into a single retraining sample. For example, a single retraining sample may be configured with a data structure including judgment request information "Transaction amount: 3,200,000 won, Transaction location: Overseas, Transaction time: 03:10", a predicted result "Abnormal transaction (prediction probability 0.76)", and an actual judgment result "Normal transaction". For example, a single retraining sample may be configured with a data structure including judgment request information "Insurance claim amount: 5,000,000 won, Treatment period: 2 days", a predicted result "Payment approval (prediction probability 0.63)", and an actual judgment result "Payment denial". However, it is not limited thereto.
[0178] An electronic device according to one embodiment can retrain an existing artificial intelligence model using retraining data. Retraining may refer to a learning process in which the weight parameters of an existing trained artificial intelligence model are updated again based on additional training data.
[0179] For example, the electronic device can update model parameters by inputting retraining data while the weight parameters of an existing artificial intelligence model are set to initial values. For example, the electronic device can calculate the difference between the prediction result of the artificial intelligence model and the actual judgment result using a loss function. For example, the electronic device can adjust the weight parameters of the artificial intelligence model using a backpropagation algorithm. However, it is not limited to these methods.
[0180] An electronic device according to one embodiment can individually retrain an artificial intelligence model for a user corresponding to an identification code. For example, the electronic device can retrain a first user artificial intelligence model based on user judgment data corresponding to the identification code "USR001". For example, the electronic device can retrain a second user artificial intelligence model based on user judgment data corresponding to the identification code "EMP1023". In this way, independent artificial intelligence models can be retrained for different users, respectively. However, it is not limited thereto.
[0181] An electronic device according to one embodiment may store an artificial intelligence model after retraining is complete. For example, the electronic device may store artificial intelligence model data including retrained weight parameters in a model repository. For example, the electronic device may update a model management table including an artificial intelligence model identifier corresponding to a user identification code. However, it is not limited thereto.
[0182] An electronic device according to one embodiment can generate predicted judgment result information for subsequently input judgment request information using a retrained artificial intelligence model. For example, the electronic device can predict a user's judgment result for new judgment request information using a retrained user-specific artificial intelligence model. However, it is not limited thereto.
[0183] FIG. 6 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0184] Step 150 of Fig. 6 corresponds to Step 150 of Fig. 1 and Fig. 4, and Step 160 of Fig. 6 corresponds to Step 160 of Fig. 1, Fig. 4 and Fig. 5.
[0185] In step 610, an electronic device according to one embodiment can obtain similarity between the first judgment request information, the second judgment request information, and the third judgment request information.
[0186] The first judgment request information may refer to judgment request data related to tasks requiring user judgment. The second judgment request information may refer to judgment request data used for comparison between the prediction result of an artificial intelligence model and the actual user judgment result. The third judgment request information may refer to new judgment request data used to predict the user's judgment result using an artificial intelligence model.
[0187] The first judgment request information, the second judgment request information, and the third judgment request information may include information on the subject of the work, information on the situation of the work, information on the conditions of the work, or work-related data. For example, in financial transaction review work, the first judgment request information may include transaction amount information, transaction time information, transaction location information, or counterparty account information. For example, in insurance claim review work, the second judgment request information may include insurance claim amount, treatment period, or medical certificate information. For example, in e-commerce dispute processing work, the third judgment request information may include order information, payment information, or shipping information. However, it is not limited thereto.
[0188] An electronic device according to one embodiment can convert a first judgment request information, a second judgment request information, and a third judgment request information into a feature vector form. A feature vector may refer to data in which the data included in the judgment request information is expressed as a numerical data vector. A feature vector may include a data set in which various attribute values included in the judgment request information are converted into numerical data.
[0189] For example, in financial transaction review tasks, transaction amount information may be expressed as a numeric value. For example, transaction time information may be converted into a time interval value or a time index value. For example, transaction location information may be converted into a country code or region code value. However, it is not limited thereto.
[0190] An electronic device according to one embodiment may generate a feature vector for each of the first judgment request information, the second judgment request information, and the third judgment request information. For example, a first feature vector may be generated for the first judgment request information. For example, a second feature vector may be generated for the second judgment request information. For example, a third feature vector may be generated for the third judgment request information. However, it is not limited thereto.
[0191] An electronic device according to one embodiment can calculate the similarity between a first feature vector and a third feature vector. For example, the electronic device can calculate the cosine similarity between two feature vectors. For example, the electronic device can calculate the Euclidean distance-based similarity between two feature vectors. However, it is not limited thereto.
[0192] An electronic device according to one embodiment can calculate similarity between a second feature vector and a third feature vector. For example, the electronic device can calculate similarity by calculating the difference in data attributes between second judgment request information and third judgment request information. For example, the electronic device can calculate similarity based on the difference in transaction amount, transaction location, and transaction time included in the two judgment request information. However, it is not limited thereto.
[0193] An electronic device according to one embodiment can calculate the similarity between a first judgment request information and a second judgment request information. For example, the electronic device can determine whether the two judgment request information corresponds to the same type of business. For example, the electronic device can determine whether the two judgment request information corresponds to the same type of transaction or the same type of insurance claim. However, it is not limited thereto.
[0194] An electronic device according to one embodiment can calculate the similarity between the first judgment request information, the second judgment request information, and the third judgment request information in the form of a numerical value. For example, the similarity may be calculated as a value in the range of 0 to 1. For example, the similarity may be calculated as a value such as 0.92 or 0.88. For example, the similarity may be calculated as a value such as 0.35 or 0.40. However, it is not limited thereto.
[0195] An electronic device according to one embodiment can identify past judgment request information similar to the third judgment request information using a calculated similarity. For example, the electronic device may determine that the two judgment request information are similar to a high degree if the similarity between the third judgment request information and the first judgment request information is 0.90 or higher. For example, the electronic device may determine that the two judgment request information correspond to similar business situations if the similarity between the third judgment request information and the second judgment request information is 0.80 or higher. However, it is not limited thereto.
[0196] An electronic device according to one embodiment may store similarity between a first judgment request information, a second judgment request information, and a third judgment request information in the form of data. For example, the electronic device may generate a data record including a first judgment request information identifier, a second judgment request information identifier, a third judgment request information identifier, and a similarity value, and store it in a database. For example, the electronic device may generate and store data in the form of a similarity matrix between judgment request information. However, it is not limited thereto.
[0197] An electronic device according to one embodiment may determine reference data for generating prediction judgment result information of an artificial intelligence model using the acquired similarity. For example, the electronic device may select past judgment request information having the highest similarity to the third judgment request information. For example, the electronic device may select a plurality of judgment request information having a similarity of 0.85 or higher and utilize them as reference data in the process of generating prediction judgment results. However, it is not limited thereto.
[0198] In step 620, an electronic device according to one embodiment can obtain user prediction judgment result information regarding third judgment request information based on similarity.
[0199] An electronic device according to one embodiment can identify past judgment request information similar to the third judgment request information by using the similarity between the third judgment request information, the first judgment request information, and the second judgment request information. Similarity may refer to a value indicating the degree of similarity of data characteristics between judgment request information. Similarity may be expressed as a value between 0 and 1. For example, if the similarity is 0.92 or 0.88, the two judgment request information may be judged to be similar to a high degree. For example, if the similarity is 0.45 or 0.32, the two judgment request information may be judged to be similar to a relatively low degree. However, it is not limited thereto.
[0200] An electronic device according to one embodiment can identify a first judgment request information having the highest similarity by comparing the similarity between the third judgment request information and the first judgment request information. For example, the electronic device can identify a judgment request information having a similarity of 0.90 or higher among a plurality of first judgment request information. For example, the electronic device can select the top 5 first judgment request informations with the highest similarity to the third judgment request information. However, it is not limited thereto.
[0201] An electronic device according to one embodiment may additionally identify similar judgment request information by comparing the similarity between third judgment request information and second judgment request information. For example, the electronic device may identify second judgment request information with a similarity of 0.85 or higher. For example, the electronic device may select the top 10 second judgment request information with high similarity to the third judgment request information. However, it is not limited thereto.
[0202] An electronic device according to one embodiment may refer to identified similar judgment request information and corresponding judgment result data. The judgment result data may include a judgment result actually performed by a user regarding the judgment request information or a predicted judgment result information generated by an artificial intelligence model.
[0203] For example, if the judgment result corresponding to the first judgment request information having a similarity of 0.93 to the third judgment request information is an "abnormal transaction," the electronic device may use the judgment result data as reference data. For example, if the judgment result corresponding to the second judgment request information having a similarity of 0.90 to the third judgment request information is a "normal transaction," the electronic device may use the judgment result data as reference data. However, it is not limited thereto.
[0204] An electronic device according to one embodiment can generate predicted judgment result information using judgment result data corresponding to a plurality of judgment request information with high similarity. For example, the electronic device may determine "abnormal transaction" as the predicted judgment result information if, among the top 5 similar judgment request information, 4 judgment results are "abnormal transaction" and 1 judgment result is "normal transaction." For example, the electronic device may determine "payment approval" as the predicted judgment result information if, among the top 10 similar judgment request information, 7 judgment results are "payment approval" and 3 judgment results are "payment rejection." However, it is not limited thereto.
[0205] An electronic device according to one embodiment can calculate prediction judgment result information using similarity values as weights. For example, the judgment result of judgment request information with a similarity of 0.95 may be reflected with a high weight of 0.95. For example, the judgment result of judgment request information with a similarity of 0.80 may be reflected with a weight of 0.80. However, it is not limited thereto.
[0206] For example, if the judgment result of judgment request information with a similarity of 0.95 to the third judgment request information is "abnormal transaction" and the judgment result of judgment request information with a similarity of 0.85 to the third judgment request information is "normal transaction," the electronic device may calculate a weight sum to determine the final predicted judgment result. For example, if the weight sum of "abnormal transaction" is 1.80 and the weight sum of "normal transaction" is 1.20, the electronic device may determine "abnormal transaction" as the predicted judgment result information. However, it is not limited thereto.
[0207] An electronic device according to one embodiment can obtain determined prediction judgment result information in the form of data. For example, the electronic device can generate data in the form of "Prediction result: Abnormal transaction, Prediction probability: 0.87". For example, the electronic device can generate data in the form of "Prediction result: Payment approval, Prediction probability: 0.82". However, it is not limited thereto.
[0208] An electronic device according to one embodiment may store acquired prediction judgment result information in a form corresponding to third judgment request information. For example, the electronic device may generate a data record including a third judgment request information identifier, prediction judgment result information, and a prediction probability value and store it in a database. For example, the electronic device may store judgment history data including third judgment request information, a similar judgment request information identifier, and a similarity value. However, it is not limited thereto.
[0209] FIG. 7 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0210] Step 150 of FIG. 7 corresponds to Step 150 of FIG. 1, FIG. 4, and FIG. 6, and Step 160 of FIG. 7 corresponds to Step 160 of FIG. 1, FIG. 4 to FIG. 6.
[0211] In step 710, in a first operation mode in which a user is making a judgment about a task, if the reliability is within a first threshold range, the electronic device according to one embodiment, 3 Information regarding the user's predicted judgment result for the judgment request information can be output along with the confidence level.
[0212] The first operation mode may refer to an operation state in which an electronic device provides judgment assistance information to a user while the user is performing a task requiring a judgment. The first operation mode may include a state in which the user is reviewing judgment request information through a business processing system, a business processing terminal, or a user interface of an electronic device. For example, the first operation mode may include a state in which the user is checking a business processing screen, such as a financial transaction review screen, an insurance underwriting screen, or a customer inquiry processing screen. For example, the first operation mode may include a state in which the user reviews third judgment request information displayed on the display of the electronic device and activates a user interface for inputting a judgment. However, it is not limited thereto.
[0213] An electronic device according to one embodiment can determine whether a user is in a first operation mode. For example, the electronic device may determine that the user is in the first operation mode if the user activates a judgment request information screen while logged into a business processing system. For example, the electronic device may determine that the user is in the first operation mode if the user is operating a judgment input interface on a screen displaying third judgment request information. However, it is not limited thereto.
[0214] The first threshold range may refer to a confidence interval set to provide prediction judgment result information to the user as reference information. The first threshold range may refer to a range that includes a preset lower confidence limit and an upper confidence limit. For example, the first threshold range may include a confidence range of 0.70 or higher and 0.90 or lower. For example, the first threshold range may include a confidence range of 70 points or higher and 90 points or lower. However, it is not limited thereto.
[0215] An electronic device according to one embodiment can determine whether a calculated reliability is included within a first threshold range. For example, if the calculated reliability is 0.82 and the first threshold range is 0.70 or higher and 0.90 or lower, the electronic device can determine that the reliability is included within the first threshold range. For example, if the calculated reliability is 88 points and the first threshold range is 70 points or higher and 90 points or lower, the electronic device can determine that the reliability is included within the first threshold range. However, it is not limited thereto.
[0216] An electronic device according to one embodiment may provide a user with predicted judgment result information regarding a third judgment request information when the reliability is included within a first threshold range. The predicted judgment result information may refer to a judgment result that an artificial intelligence model expects the user to perform regarding the third judgment request information. For example, the predicted judgment result information may include a result in which a specific financial transaction is predicted to be highly likely to be an "abnormal transaction." For example, the predicted judgment result information may include a judgment result predicted as "payment approval" in an insurance claim review process. However, it is not limited thereto.
[0217] An electronic device according to one embodiment can output prediction judgment result information along with reliability through a display. For example, the electronic device can display information in the form of "Prediction judgment result: Abnormal transaction, Reliability: 0.82" on a display screen. For example, the electronic device can display information in the form of "Prediction judgment result: Payment approval, Reliability: 88%" on a display screen. However, it is not limited thereto.
[0218] An electronic device according to one embodiment may provide predictive judgment result information as reference information to assist user judgment. For example, the electronic device may display the information in an auxiliary area of a screen so that the user can refer to the predictive judgment result information before performing a judgment on the third judgment request information. For example, the electronic device may display the predictive judgment result information in an area adjacent to an interface for the user to input a judgment result. However, it is not limited thereto.
[0219] An electronic device according to one embodiment may display prediction judgment result information and a reliability value in a visual form. For example, the electronic device may display the prediction judgment result information in green when the reliability is 0.80 or higher. For example, the electronic device may display the prediction judgment result information in yellow when the reliability is 0.70 or higher but less than 0.80. However, it is not limited thereto.
[0220] An electronic device according to one embodiment may store third judgment request information, predicted judgment result information, and a confidence value as a single result data. For example, the electronic device may generate a data record including a third judgment request information identifier, predicted judgment result information, a confidence value, and result output time information, and store it in a database. For example, the electronic device may generate and store log data that records the history of the predicted judgment result information being provided to a user. However, it is not limited thereto.
[0221] FIG. 8 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0222] Step 150 of FIG. 8 corresponds to Step 150 of FIG. 1, FIG. 4, FIG. 6 and FIG. 7, and Step 160 of FIG. 8 corresponds to Step 160 of FIG. 1 and FIG. 4 through FIG. 7.
[0223] In step 810, in a second operation mode in which the user is not performing a judgment regarding the task, the electronic device according to one embodiment, when the reliability is within a second threshold range, the third Based on the user's predicted judgment result information regarding the judgment request information, tasks according to the third judgment request information can be processed.
[0224] The second operation mode may refer to an operational state in which an electronic device performs task processing based on the prediction results of an artificial intelligence model while the user is not directly performing task judgments. The second operation mode may include a state in which the user is not logged into the task processing system, a state in which the user has not activated the task judgment screen, or a state in which the user has not performed judgment input for a certain period of time or longer. For example, the second operation mode may include a state in which the user is logged out of the task system. For example, the second operation mode may include a state during nighttime or holiday hours when the user is not using the task terminal. However, it is not limited thereto.
[0225] An electronic device according to one embodiment can determine whether a user is in a second operation mode based on the user's input state or the system usage state. For example, the electronic device may determine that the user is in a second operation mode if no user input occurs for a certain period of time. For example, the electronic device may determine that the user is in a second operation mode if the business system login session is terminated. However, it is not limited thereto.
[0226] The second threshold range may refer to a confidence interval in which an electronic device can automatically perform task processing based on the prediction judgment result of an artificial intelligence model. The second threshold range may be set as a confidence interval higher than the first threshold range. For example, the second threshold range may include a confidence range of 0.90 or higher and 1.00 or lower. For example, the second threshold range may include a confidence range of 90 points or higher and 100 points or lower. However, it is not limited thereto.
[0227] An electronic device according to one embodiment can determine whether a calculated reliability is included in a second threshold range. For example, if the calculated reliability is 0.93 and the second threshold range is 0.90 or higher and 1.00 or lower, the electronic device can determine that the reliability is included in the second threshold range. For example, if the calculated reliability is 95 points and the second threshold range is 90 points or higher and 100 points or lower, the electronic device can determine that the reliability is included in the second threshold range. However, it is not limited thereto.
[0228] An electronic device according to one embodiment can automatically process a task related to a judgment request by using predicted judgment result information regarding third judgment request information when the reliability is included in a second threshold range. The predicted judgment result information may refer to a judgment result that an artificial intelligence model expects a user to perform regarding the third judgment request information.
[0229] For example, if the information regarding a third-party judgment request relates to financial transaction review tasks, the electronic device may perform a process to approve the transaction as a normal transaction if the information regarding the predictive judgment result is a "normal transaction." For example, if the information regarding a third-party judgment request relates to insurance claim review tasks, the electronic device may perform a process to approve the payment of insurance benefits if the information regarding the predictive judgment result is a "payment approval." However, it is not limited thereto.
[0230] An electronic device according to one embodiment can generate commands for a business processing system based on information regarding prediction judgment results. For example, the electronic device can generate business processing commands such as "transaction approval processing," "insurance claim payment approval processing," or "automatic classification of customer inquiries." For example, the electronic device can automatically record judgment result values in the database of the business processing system. However, it is not limited thereto.
[0231] An electronic device according to one embodiment may store automatically processed business results in the form of data. For example, the electronic device may generate a data record including a third judgment request information identifier, prediction judgment result information, a confidence value, and automatic processing time information, and store it in a database. For example, the electronic device may record the history of automatically processed business results as log data. However, it is not limited thereto.
[0232] An electronic device according to one embodiment may provide automatically processed business results to a user thereafter. For example, when a user reconnects to the business system, the electronic device may display a list of automatically processed judgment results through a user interface. For example, the electronic device may provide a review interface to enable user review of the automatically processed judgment results. However, it is not limited thereto.
[0233] In step 820, if the reliability of the electronic device according to one embodiment is within a third threshold range, the third judgment request information and the third Information regarding the user's predicted judgment result for the judgment request information can be transmitted to an external device.
[0234] The third threshold range may refer to a confidence interval where the reliability of the prediction judgment result information generated by the artificial intelligence model is insufficient for the electronic device to process alone, but is appropriate for requesting additional review by a user or another system. The third threshold range may be set to a confidence interval lower than the second threshold range. For example, the third threshold range may include a confidence range of 0.60 or higher and less than 0.90. For example, the third threshold range may include a confidence range of 60 points or higher and less than 90 points. However, it is not limited thereto.
[0235] An electronic device according to one embodiment can determine whether a calculated reliability is included in a third threshold range. For example, if the calculated reliability is 0.72 and the third threshold range is 0.60 or higher and less than 0.90, the electronic device can determine that the reliability is included in the third threshold range. For example, if the calculated reliability is 75 points and the third threshold range is 60 points or higher and less than 90 points, the electronic device can determine that the reliability is included in the third threshold range. However, it is not limited thereto.
[0236] An electronic device according to one embodiment may transmit third judgment request information and predicted judgment result information to an external device when the reliability falls within a third threshold range. The third judgment request information is information regarding a task requiring a user's judgment and may include judgment target data, task identification information, or task processing condition information. For example, in a financial transaction review task, the third judgment request information may include transaction amount, transaction time, and counterparty identification information. For example, in an insurance underwriting task, the third judgment request information may include claim amount, medical treatment item, and insurance contract identification information. However, it is not limited thereto.
[0237] Predicted judgment result information may refer to the judgment result that the AI model is expected to make to the user regarding third-party judgment request information. For example, the predicted judgment result information may include a result in which a specific financial transaction is highly likely to be judged as "suspected abnormal transaction." For example, the predicted judgment result information may include a judgment result predicted as "additional review required" in insurance claim review tasks. However, it is not limited thereto.
[0238] An external device may refer to another computing device capable of communicating with an electronic device. An external device may include a user terminal device, an administrator terminal device, or a separate business processing server. For example, an external device may include a terminal device of an abnormal transaction reviewer at a financial institution. For example, an external device may include a business terminal device of an insurance underwriter. However, it is not limited thereto.
[0239] An electronic device according to one embodiment may generate third judgment request information and predicted judgment result information in the form of a single data packet or message. For example, the electronic device may generate transmission data including a third judgment request information identifier, predicted judgment result information, and a confidence value. For example, the electronic device may generate message data including summary information of the third judgment request information and predicted judgment result information. However, it is not limited thereto.
[0240] An electronic device according to one embodiment can transmit generated transmission data to an external device through a communication network. The communication network may include a wired network or a wireless network. For example, the communication network may include an Internet-based TCP / IP network. For example, the communication network may include an internal corporate network or a cloud-based network. However, it is not limited thereto.
[0241] An electronic device according to one embodiment may generate notification information so that an external device can review third judgment request information and predicted judgment result information. For example, the electronic device may transmit a "task requiring review" notification message to display on the user interface of the external device. For example, the electronic device may transmit a notification message in the form of a push notification or email to the external device. However, it is not limited thereto.
[0242] An electronic device according to one embodiment may receive an actual judgment result regarding third judgment request information from an external device. For example, the external device may transmit a judgment result performed by a user or an administrator to the electronic device. For example, the electronic device may store the judgment result received from the external device as third judgment data. However, it is not limited thereto.
[0243] An electronic device according to one embodiment may utilize a judgment result received from an external device as training data for improving the performance of an artificial intelligence model. For example, the electronic device may store third judgment request information, prediction judgment result information, and a judgment result input from an external device as retraining data. For example, the electronic device may use the stored retraining data to further train or retrain the artificial intelligence model. However, it is not limited thereto.
[0244] In step 830, if the reliability is within the fourth threshold range, the electronic device according to one embodiment may suspend business processing according to the third judgment request information until the user's third judgment data regarding the third judgment request information is obtained.
[0245] The fourth threshold range may refer to a confidence interval where the reliability of the prediction judgment result information generated by the artificial intelligence model is low, making it difficult for an electronic device to automatically perform task processing using the prediction judgment result information. The fourth threshold range may be set to a confidence interval lower than the third threshold range. For example, the fourth threshold range may include a confidence interval of less than 0.60. For example, the fourth threshold range may include a confidence interval of less than 60 points. However, it is not limited thereto.
[0246] An electronic device according to one embodiment can determine whether a calculated reliability is included in a fourth threshold range. For example, if the calculated reliability is 0.42 and the fourth threshold range is set to less than 0.60, the electronic device can determine that the reliability is included in the fourth threshold range. For example, if the calculated reliability is 45 points and the fourth threshold range is set to less than 60 points, the electronic device can determine that the reliability is included in the fourth threshold range. However, it is not limited thereto.
[0247] An electronic device according to one embodiment may not immediately perform business processing based on the third judgment request information when the reliability falls within the fourth threshold range. The third judgment request information may refer to information regarding a task requiring a user's judgment. For example, in a financial transaction review task, the third judgment request information may include transaction amount, transaction time, and counterparty identification information. For example, in an insurance claim review task, the third judgment request information may include claim amount, medical treatment item, and insurance contract identification information. However, it is not limited thereto.
[0248] An electronic device according to one embodiment may set the task processing based on the third judgment request information to a pending state. A pending state may mean a state in which the task is not automatically processed and awaits user judgment. For example, the electronic device may set the task status value corresponding to the third judgment request information to a "waiting," "pending," or "user judgment required" status value. For example, the electronic device may update the task processing status field stored in the task management database to "waiting state." However, it is not limited thereto.
[0249] An electronic device according to one embodiment may provide third judgment request information to a user to obtain the user's third judgment data. For example, the electronic device may display the third judgment request information on the display of a user terminal device. For example, the electronic device may register the third judgment request information in the waiting task list of a task management system. However, it is not limited thereto.
[0250] Third judgment data may refer to the results of judgments actually made by the user regarding the information requested for a third judgment. For example, third judgment data may include results where the user judged a specific financial transaction as an "abnormal transaction" or a "normal transaction." For example, third judgment data may include results where the user judged a claim as "payment approved" or "payment denied" in insurance claim review tasks. However, it is not limited thereto.
[0251] An electronic device according to one embodiment may acquire third judgment data through user input. For example, the electronic device may receive input in which a user selects a judgment result from a user interface. For example, the electronic device may receive input in which a user selects a judgment result button from a task processing screen. However, it is not limited thereto.
[0252] An electronic device according to one embodiment may not perform business processing based on the third judgment request information until the third judgment data is acquired. For example, the electronic device may not perform approval processing or rejection processing of the transaction until the third judgment data is input. For example, the electronic device may not perform approval processing or rejection processing of insurance payment until the third judgment data is input. However, it is not limited thereto.
[0253] An electronic device according to one embodiment may perform business processing based on third judgment request information after acquiring third judgment data. For example, if the third judgment data is a "normal transaction," the electronic device may perform transaction approval processing. For example, if the third judgment data is a "payment approval," the electronic device may perform insurance payment approval processing. However, it is not limited thereto.
[0254] An electronic device according to one embodiment may store third judgment request information and third judgment data. For example, the electronic device may store a data record in a database that includes a third judgment request information identifier, third judgment data, and judgment execution time information. For example, the electronic device may store the third judgment data as training data for improving the performance of an artificial intelligence model. However, it is not limited thereto.
[0255] FIG. 9 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0256] Step 160 of FIG. 9 corresponds to Step 160 of FIG. 1 and FIG. 4 through FIG. 8.
[0257] In step 910, if the reliability is within a fourth threshold range, the electronic device according to one embodiment may store third judgment request information and user prediction judgment result information regarding the third judgment request information.
[0258] An electronic device according to one embodiment may store third judgment request information and predicted judgment result information in the form of a data structure. For example, the electronic device may generate a data record including a third judgment request information identifier, predicted judgment result information, and a confidence value. For example, the electronic device may generate a data record including third judgment request information source data, predicted judgment result information, and prediction execution time information. However, it is not limited thereto.
[0259] An electronic device according to one embodiment may store generated data records in a storage device or a database. For example, the electronic device may record the data in a table of a relational database management system. For example, the electronic device may record the data in a log storage server or cloud storage. However, it is not limited thereto.
[0260] An electronic device according to one embodiment may utilize the stored third judgment request information and predicted judgment result information as waiting data for requesting a subsequent user judgment. For example, the electronic device may manage the stored data as business data included in a "user judgment waiting list." For example, the electronic device may register the stored data in a business queue requiring user review. However, it is not limited thereto.
[0261] An electronic device according to one embodiment may utilize stored data as data for comparison with the user's actual judgment data thereafter. For example, when a user inputs a judgment result regarding third judgment request information, the electronic device may compare the stored predicted judgment result information with the actual judgment result. For example, the electronic device may store the comparison result to use it as retraining data for an artificial intelligence model. However, it is not limited thereto.
[0262] An electronic device according to one embodiment can perform performance analysis of an artificial intelligence model using stored third judgment request information and prediction judgment result information. For example, the electronic device can calculate the degree of agreement between the prediction judgment result information and the actual judgment result. For example, the electronic device can generate statistical data for analyzing prediction accuracy by confidence interval. However, it is not limited thereto.
[0263] In step 920, when the operation mode of the electronic device according to one embodiment is switched from the second operation mode to the first operation mode, the electronic device can obtain the user's third judgment data regarding the third judgment request information.
[0264] An electronic device according to one embodiment can detect whether the operation mode of the electronic device switches from a second operation mode to a first operation mode. For example, the electronic device can detect the operation mode switch by receiving an event in which a user logs into a business processing system. For example, the electronic device can detect the operation mode switch by detecting an event in which a user activates a business processing screen containing judgment request information. However, it is not limited thereto.
[0265] An electronic device according to one embodiment may provide a third judgment request information stored during a second operation mode to a user. The third judgment request information may include business data in which reliability is included within a fourth threshold range. For example, the third judgment request information may include financial transaction review request information stored in a pending state without being automatically processed. For example, the third judgment request information may include insurance claim review request information awaiting user judgment. However, it is not limited thereto.
[0266] An electronic device according to one embodiment may display third judgment request information to a user through a user interface. For example, the electronic device may display the third judgment request information on a "judgment waiting task list" screen. For example, the electronic device may display the third judgment request information on a display sorted according to priority. However, it is not limited thereto.
[0267] Third judgment data may refer to the results of judgments actually made by the user regarding the information requested for a third judgment. For example, third judgment data may include the result of a user determining a specific transaction as a "normal transaction" or an "abnormal transaction" in financial transaction review tasks. For example, third judgment data may include the result of a user determining "payment approval" or "payment denial" in insurance claim review tasks. However, it is not limited thereto.
[0268] An electronic device according to one embodiment may obtain third judgment data through user input. For example, the electronic device may receive input in which a user selects a judgment result button on a user interface. For example, the electronic device may receive input in which a user enters a judgment result value into an input field. However, it is not limited thereto.
[0269] An electronic device according to one embodiment may store the acquired third judgment data as data corresponding to the third judgment request information. For example, the electronic device may store a data record in a database that includes a third judgment request information identifier, third judgment data, and judgment execution time information. For example, the electronic device may store the third judgment data to be used as data for training an artificial intelligence model thereafter. However, it is not limited thereto.
[0270] An electronic device according to one embodiment may perform business processing based on third judgment request information after acquiring third judgment data. For example, if the third judgment data is a "normal transaction," the electronic device may perform transaction approval processing. For example, if the third judgment data is a "payment approval," the electronic device may perform insurance payment approval processing. However, it is not limited thereto.
[0271] In step 930, an electronic device according to one embodiment can retrain an artificial intelligence model using third judgment request information, user prediction judgment result information regarding the third judgment request information, and third judgment data as retraining data.
[0272] An electronic device according to one embodiment may generate third judgment request information, predicted judgment result information, and third judgment data into a single retraining data set. The retraining data set may refer to a data structure that includes both the prediction result generated by an artificial intelligence model and the actual user judgment result. For example, the retraining data set may include a data record containing a third judgment request information identifier, predicted judgment result information, third judgment data, and a confidence value. For example, the retraining data set may include a data record containing the third judgment request information source data, predicted judgment result information, third judgment data, and judgment execution time information. However, it is not limited thereto.
[0273] An electronic device according to one embodiment may utilize a generated retraining data set as training data for an artificial intelligence model. For example, the electronic device may use third judgment request information data as input data for the artificial intelligence model. For example, the electronic device may use third judgment data as correct label data for the artificial intelligence model. However, it is not limited thereto.
[0274] An electronic device according to one embodiment can update the parameters of an artificial intelligence model by reflecting the difference between the prediction judgment result information and the third judgment data. For example, if the prediction judgment result information is a "normal transaction" and the third judgment data is an "abnormal transaction," the electronic device can calculate a learning error value for the corresponding judgment request information. For example, the electronic device can update the weight parameters of the artificial intelligence model using the calculated learning error value. However, it is not limited thereto.
[0275] An electronic device according to one embodiment can further train an artificial intelligence model using a retraining data set. For example, the electronic device can retrain the artificial intelligence model by adding the retraining data set to existing training data. For example, the electronic device can retrain the artificial intelligence model using a batch training method when a certain number of retraining data sets or more have been accumulated. However, it is not limited thereto.
[0276] An electronic device according to one embodiment can evaluate the performance of an artificial intelligence model after retraining. For example, the electronic device can calculate the degree of agreement between a prediction judgment result and an actual user judgment result after retraining. For example, the electronic device can calculate the accuracy value or error rate value of the artificial intelligence model after retraining. However, it is not limited thereto.
[0277] An electronic device according to one embodiment may use an artificial intelligence model that has completed retraining in a subsequent process of processing judgment request information. For example, the electronic device may generate predictive judgment result information for new judgment request information using the retrained artificial intelligence model. For example, the electronic device may generate predictive judgment result information that more accurately reflects user judgment characteristics using the retrained artificial intelligence model. However, it is not limited thereto.
[0278] FIG. 10 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.
[0279] Step 160 of FIG. 10 corresponds to Step 160 of FIG. 1 and FIG. 4 through 9.
[0280] In step 1010, if the electronic device according to one embodiment detects that the user has changed to a new user based on the identification code, it may set the weight parameters of the artificial intelligence model as the initial parameters of the artificial intelligence model for the new user.
[0281] An electronic device according to one embodiment may determine a user as a new user if the acquired identification code is not included in the list of existing user identification codes. For example, if "USER1023" and "USER1024" are registered in the user management database and the identification code "USER2031" is entered, the electronic device may determine the user "USER2031" as a new user. For example, if the number of user accounts registered in the user management database is 25 and a new user account is created, the electronic device may determine that account as a new user. However, it is not limited thereto.
[0282] A new user may include a user who takes over a task after an existing user ceases to perform that task. Situations in which an existing user ceases to perform a task may include internal personnel transfers, departmental changes, or resignation. For example, if a user performing specific financial transaction review tasks resigns, another user may take over those tasks. For example, if a user performing insurance claim review tasks moves to another department, another user may take over those insurance claim review tasks. However, this is not limited to these cases.
[0283] An electronic device according to one embodiment can detect that an identification code corresponding to an existing user is no longer in use. The electronic device can check the account status of an existing user by utilizing user account management information from a user management database or a business processing system. For example, the electronic device may determine that the user is no longer performing business if the user account status changes to "deactivated," "suspended," or "scheduled for deletion." For example, the electronic device may determine that the user is not performing business if a login record corresponding to a specific user identification code does not exist for a certain period of time. However, it is not limited thereto.
[0284] An electronic device according to one embodiment may assign judgment request information corresponding to tasks performed by an existing user to a new user. For example, the electronic device may assign financial transaction review tasks processed by an existing user to a new user. For example, the electronic device may reassign insurance underwriting tasks handled by an existing user to a new user account. However, it is not limited thereto.
[0285] An electronic device according to one embodiment may determine a user as a new user when a new identification code different from an existing user identification code is associated with a corresponding task. For example, if the existing user identification code "EMP1023" is no longer in use and the new user identification code "EMP2051" is assigned to the same task group, the electronic device may determine the user "EMP2051" as a new user. For example, if a new user account is registered to the same task role after an existing user account has been deactivated, the electronic device may determine the user as a new user. However, it is not limited thereto.
[0286] A new user may be a user who continues to perform tasks previously performed by an existing user. While a new user handles the same type of tasks, they may have different judgment criteria or patterns than the existing user. For example, a new user may perform the same financial transaction review tasks as an existing user, but their criteria for judging transaction risk may differ. For example, a new user may perform the same insurance claim review tasks as an existing user, but their criteria for payment approval may differ. However, this is not limited to these cases.
[0287] An electronic device according to one embodiment may separately generate an artificial intelligence model corresponding to a new user when a new user replaces an existing user to perform tasks. For example, the electronic device may not use an artificial intelligence model trained using existing user judgment data as is. For example, the electronic device may set an artificial intelligence model corresponding to a new user to an initial parameter state and then perform training using new user judgment data. However, it is not limited thereto.
[0288] An electronic device according to one embodiment may set weight parameters of an artificial intelligence model corresponding to a new user as initial parameters when the new user is determined to be a new user. The weight parameters of the artificial intelligence model may refer to learning parameters used by the artificial intelligence model in the process of processing input data. For example, the weight parameters may include connection weight values or bias parameter values of a neural network model. For example, the weight parameters may include coefficient values of a regression model or parameter values of a classification model. However, they are not limited thereto.
[0289] Initial parameters may refer to an initial learning state for the artificial intelligence model to operate prior to a stage where sufficient judgment data from new users has been accumulated. Initial parameters may include reference model parameters to maintain business continuity. For example, initial parameters may include weight parameter values of a base model previously trained using all user judgment data. For example, initial parameters may include weight parameter values of a model trained on the average judgment patterns of users belonging to a specific organization or department, provided, however, that they are not limited thereto.
[0290] The use of initial parameters may serve the purpose of providing predictive judgment result information even in the initial stages where sufficient judgment data from new users has not yet been secured. For example, even when a new user accesses a business processing system for the first time, the electronic device can generate predictive judgment result information regarding a judgment request by utilizing an artificial intelligence model with applied initial parameters. For instance, regarding financial transaction review tasks or insurance underwriting tasks processed by a new user, the electronic device can provide predictive judgment result information using a basic model. However, it is not limited thereto.
[0291] Setting initial parameters may serve the purpose of preventing interruptions in business processing. For example, even if judgment data for a new user does not exist, the electronic device can process judgment request information using a basic model. For instance, the electronic device can maintain the workflow without interruption by providing predictive judgment result information regarding tasks assigned to the new user. However, it is not limited to this.
[0292] An electronic device according to one embodiment may create an artificial intelligence model instance corresponding to a new user identification code. For example, the electronic device may create an artificial intelligence model object corresponding to the user identification code "USER2031". For example, the electronic device may create a model management table that maps the user identification code to an artificial intelligence model identifier. However, it is not limited thereto.
[0293] An electronic device according to one embodiment can initialize an artificial intelligence model corresponding to a new user using set initial parameters. For example, the electronic device can set the weight values of the new user model as initial parameter values. For example, the electronic device can load the artificial intelligence model with the applied initial parameters into memory and set it to a state ready for execution. However, it is not limited thereto.
[0294] An electronic device according to one embodiment can progressively train an artificial intelligence model when judgment data from a new user is accumulated. For example, the electronic device can update the weight parameters of the artificial intelligence model using first judgment data or third judgment data performed by the new user. For example, when a certain number of judgment data or more is accumulated, the electronic device can switch the training state to an artificial intelligence model specialized for the new user. However, it is not limited thereto.
[0295] In step 1020, an electronic device according to one embodiment can obtain first judgment data of a new user regarding first judgment request information.
[0296] A new user may refer to a user whom the electronic device determines to be a newly registered user by comparing them with the list of existing user identification codes. A new user may include a user account newly registered in the business processing system or a newly generated user identification code. For example, if a user with an identification code not registered in the user management database logs into the business processing system, the electronic device may determine that user as a new user. For example, if a new employee within the organization creates an account in the business processing system, the electronic device may determine that user as a new user. However, it is not limited thereto.
[0297] An electronic device according to one embodiment may provide first judgment request information to a new user. For example, the electronic device may display the first judgment request information through a display of a new user terminal device. For example, the electronic device may display the first judgment request information on a judgment request screen of a business processing system. However, it is not limited thereto.
[0298] An electronic device according to one embodiment may receive a judgment result performed by a new user regarding first judgment request information in the form of input data. For example, the electronic device may receive input in which the new user selects a judgment result button, such as "Approve" or "Reject," in a user interface. For example, the electronic device may receive input in which the new user enters a judgment result into a text input field. However, it is not limited thereto.
[0299] The first judgment data may refer to the judgment results actually performed by the new user regarding the first judgment request information. For example, the first judgment data may include the result of the new user judging a specific transaction as a "normal transaction" or an "abnormal transaction" in financial transaction review tasks. For example, the first judgment data may include the result of the new user judging as "payment approved" or "payment denied" in insurance claim review tasks. However, it is not limited thereto.
[0300] An electronic device according to one embodiment can acquire first judgment data using input from a new user. For example, the electronic device can collect a judgment result value selected by the new user in the form of data. For example, the electronic device can generate a judgment result entered by the new user as a data record corresponding to a first judgment request information identifier. However, it is not limited thereto.
[0301] An electronic device according to one embodiment may store the acquired first judgment data in a data storage device or a database. For example, the electronic device may store a data record in a database that includes a first judgment request information identifier, first judgment data, and judgment execution time information. For example, the electronic device may link a new user identification code with the first judgment data and store it in a user-specific judgment data table. However, it is not limited thereto.
[0302] An electronic device according to one embodiment may utilize stored first judgment data as training data for an artificial intelligence model corresponding to a new user. For example, the electronic device may use first judgment request information as input data for the artificial intelligence model. For example, the electronic device may use the first judgment data as correct label data for the artificial intelligence model. However, it is not limited thereto.
[0303] An electronic device according to one embodiment can learn judgment patterns specialized for new users as judgment data of new users accumulates. For example, the electronic device can update weight parameters of an artificial intelligence model using 50 or 100 or more judgment data processed by a new user. For example, when new user judgment data accumulates to a certain number or more, the electronic device can train an artificial intelligence model into a personalized model that reflects the individual judgment characteristics of the user. However, it is not limited thereto.
[0304] In step 1030, an electronic device according to one embodiment can train an artificial intelligence model for a new user using the first judgment request information and the new user's first judgment data as training data.
[0305] An electronic device according to one embodiment may generate a first judgment request information and a new user's first judgment data into a single training data set. The training data set may refer to a combination of input data and correct answer data used for training an artificial intelligence model. For example, the first judgment request information may be used as input data for an artificial intelligence model. For example, the new user's first judgment data may be used as correct answer label data for an artificial intelligence model. However, it is not limited thereto.
[0306] An electronic device according to one embodiment may generate or select an artificial intelligence model corresponding to a new user identification code. For example, the electronic device may generate an instance of an artificial intelligence model corresponding to the user identification code "USER2031". For example, the electronic device may select a new user model using a model management table that maps the user identification code to an artificial intelligence model identifier. However, it is not limited thereto.
[0307] An electronic device according to one embodiment can update weight parameters of an artificial intelligence model using a training data set. For example, the electronic device can calculate the difference between the prediction result of the artificial intelligence model regarding the first judgment request information and the first judgment data of a new user. For example, the electronic device can update weight parameters of the artificial intelligence model using the calculated learning error value. However, it is not limited thereto.
[0308] An electronic device according to one embodiment can repeatedly train an artificial intelligence model as new user judgment data accumulates. For example, the electronic device can perform additional training when 10 or 50 or more judgment request information processed by a new user accumulates. For example, the electronic device can retrain the artificial intelligence model using a batch learning method with accumulated training data at regular time intervals. However, it is not limited thereto.
[0309] An electronic device according to one embodiment can train an artificial intelligence model to reflect the judgment characteristics of a new user based on the accumulation of new user judgment data. For example, the electronic device can learn a tendency for a new user to repeatedly judge a specific type of transaction as a "normal transaction." For example, the electronic device can learn a tendency for a new user to judge a specific type of insurance claim as requiring "additional review." However, it is not limited thereto.
[0310] An electronic device according to one embodiment can generate predictive judgment result information for subsequently input judgment request information using a trained artificial intelligence model. For example, the electronic device can generate predictive judgment result information that reflects the judgment patterns of new users. For example, the electronic device can provide work judgment support information using a personalized judgment model specialized for new users. However, it is not limited thereto.
[0311] FIG. 11 is a block diagram illustrating the components of an electronic device according to one embodiment.
[0312] An electronic device (1100) according to one embodiment may include a display (1110), a processor (1120), and a memory (1130).
[0313] A display (1110) may refer to an output device for an electronic device (1100) to visually provide information to a user. The display (1110) may include a screen display device included in the electronic device (1100) or an external screen display device connected to the electronic device. For example, the display (1110) may include a Liquid Crystal Display (LCD). For example, the display (1110) may include an Organic Light Emitting Diode (OLED) based display device. However, it is not limited thereto.
[0314] The display (1110) can display various information generated by the electronic device (1100) in the form of a screen. The display (1110) can display a user interface screen including judgment request information, prediction judgment result information, or reliability information. For example, the display (1110) can display a work review screen including the content of the third judgment request information. For example, the display (1110) can display a judgment auxiliary information screen including both prediction judgment result information and reliability values. However, it is not limited thereto.
[0315] The display (1110) may display user interface elements for interaction between the user and the electronic device (1100). The user interface elements may include a judgment result selection button, an input field, or a task status display area. For example, the display (1110) may display a screen including a judgment result selection button such as "Normal Transaction," "Abnormal Transaction," or "Additional Review Required." For example, the display (1110) may display a screen including an information display area for checking detailed data of the judgment request information. However, it is not limited thereto.
[0316] The display (1110) can display information provided to the user in various visual forms. The visual forms may include text information, numeric information, or graphic information. For example, the display (1110) may display text and numeric information such as "Prediction judgment result: Normal transaction, Reliability: 0.87". For example, the display (1110) may display a screen including a bar graph or color display information indicating the reliability level. However, it is not limited thereto.
[0317] The display (1110) may include a display device embedded in the electronic device (1100) or a display device connected to an external device. For example, the display (1110) may include a screen display device embedded in a terminal device such as a smartphone, tablet, or laptop. For example, the display (1110) may include a screen display device of an external monitor or a business terminal connected to a business processing server. However, it is not limited thereto.
[0318] The display (1110) may include a display device that includes a touch input function. The touch input function may refer to a function that allows a user to perform input by directly touching the screen. For example, the display (1110) may include a capacitive touch screen display device. For example, the display (1110) may include a pressure-sensing touch screen display device. However, it is not limited thereto.
[0319] The display (1110) can display various work status information during the work processing performed by the electronic device (1100). For example, the display (1110) can display work status information such as a judgment waiting state, an automatic processing state, or a state requiring user review. For example, the display (1110) can display a screen including a list of judgment request information, a record of judgment results, or a work processing history. However, it is not limited thereto.
[0320] An electronic device (1100) according to one embodiment may further include a communication unit (not shown).
[0321] The communication unit can communicate with an external device or server through at least one wired or wireless communication network by means of a processor (310).
[0322] The communication unit may include at least one short-range communication module that performs communication according to communication standards such as Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC / RFID, Wi-Fi Direct, UWB, or ZIGBEE, and a long-range communication module that performs communication with a server to support long-range communication according to long-range communication standards. The long-range communication module may perform communication through a communication network according to 3G, 4G, and / or 5G communication standards, or a network for internet communication.
[0323] The processor (1120) is configured to control a series of processes to operate the electronic device (1100) according to the embodiments described above, and may be composed of one or more processors.
[0324] The processor (1120) may be composed of hardware components that perform arithmetic, logic, and input / output operations and signal processing. One or more processors included in the processor (1120) may be circuitry such as a System on Chip (SoC) or an Integrated Circuit (IC). The processor (1120) may be composed of at least one of, for example, a Central Processing Unit, a microprocessor, a Graphic Processing Unit, Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), and Field Programmable Gate Arrays (FPGAs), but is not limited thereto.
[0325] The processor (1120) can write data to memory (1130) or read data stored in memory (1130), and in particular, can process data according to predefined operation rules by executing a program or at least one instruction stored in memory (1130).
[0326] The memory (1130) may store various data, programs, or applications for driving and controlling an electronic device (1100) according to one embodiment. The memory (1130) may include non-volatile memory such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), ROM (Read-Only Memory), and at least one of an EEPROM (Electrically Erasable Programmable Read-Only Memory) and a PROM (Programmable Read-Only Memory), and volatile memory such as RAM (Random Access Memory) or SRAM (Static Random Access Memory).
[0327] The memory (1130) may store instructions, data structures, and program code that can be read by the processor (1120). In the following embodiments, the processor (1120) may be implemented by executing the instructions or code of the program stored in the memory (1130).
[0328] The method according to the above-described embodiment can be performed by an electronic device comprising a sensor, a memory, and a processor.
[0329] That is, the processor (1120) can control the operation of the electronic device (1100) according to one embodiment so that the electronic device (1100) according to one embodiment performs the operations described above by executing at least one instruction and / or instruction stored in the memory (1130).
[0330] An electronic device according to one embodiment may include a memory for storing a program or at least one instruction, at least one processor for executing at least one instruction stored in the memory, and a display.
[0331] An electronic device according to one embodiment can obtain first judgment request information regarding a task requiring a user's judgment.
[0332] An electronic device according to one embodiment can obtain the user's first judgment data regarding the first judgment request information.
[0333] An electronic device according to one embodiment can obtain user prediction judgment result information regarding the second judgment request information by inputting the second judgment request information into an artificial intelligence model trained using the first judgment request information and the first judgment data as training data.
[0334] An electronic device according to one embodiment can obtain the user's second judgment data regarding the second judgment request information.
[0335] An electronic device according to one embodiment can calculate the reliability of the user's predicted judgment result information regarding the second judgment request information based on comparing the second judgment data and the user's predicted judgment result information regarding the second judgment request information.
[0336] An electronic device according to one embodiment inputs third judgment request information into an artificial intelligence model when the reliability is above a threshold value, and the third The user's predicted judgment result information regarding the judgment request information can be output through the display.
[0337] An electronic device according to one embodiment can determine an identification code to identify a user among a plurality of users who will process the first judgment request information based on the first judgment request information.
[0338] An electronic device according to one embodiment can identify a user corresponding to an identification code among a plurality of users.
[0339] An electronic device according to one embodiment can obtain first judgment data based on an identification code.
[0340] An electronic device according to one embodiment can train an artificial intelligence model based on an identification code, first judgment request information, and first judgment data.
[0341] An electronic device according to one embodiment can calculate reliability based on at least one of the degree of agreement between the user’s predicted judgment result information regarding the second judgment data and the second judgment request information, or the degree of agreement between the distribution of the first judgment data and the distribution of the second judgment data.
[0342] An electronic device according to one embodiment can retrain an artificial intelligence model using second judgment request information, user prediction judgment result information regarding the second judgment request information, and second judgment data as retraining data when the reliability is below a threshold value.
[0343] An electronic device according to one embodiment can obtain similarity between first judgment request information, second judgment request information and third judgment request information.
[0344] An electronic device according to one embodiment can obtain user's predicted judgment result information regarding third judgment request information based on similarity.
[0345] An electronic device according to one embodiment, in a first operation mode in which a user is making a judgment regarding a task, if the reliability is within a first threshold range, a third Information regarding the user's predicted judgment result for the judgment request information can be output along with the confidence level.
[0346] An electronic device according to one embodiment, in a second operation mode in which a user is not performing a judgment regarding a task, when the reliability is within a second threshold range, a third Based on the user's predicted judgment result information regarding the judgment request information, tasks according to the third judgment request information can be processed.
[0347] An electronic device according to one embodiment, in a second operation mode in which a user is not performing a judgment regarding a task, if the reliability is within a third threshold range, third judgment request information and third Information regarding the user's predicted judgment result for the judgment request information can be transmitted to an external device.
[0348] An electronic device according to one embodiment may suspend business processing based on third judgment request information until the user's third judgment data regarding third judgment request information is obtained, when the reliability is within a fourth threshold range in a second operation mode in which the user is not performing a judgment regarding business.
[0349] An electronic device according to one embodiment may store third judgment request information and user's predicted judgment result information regarding the third judgment request information when the reliability is within a fourth threshold range.
[0350] An electronic device according to one embodiment can obtain user's third judgment data regarding third judgment request information when the operation mode of the electronic device is switched from a second operation mode to a first operation mode.
[0351] An electronic device according to one embodiment can retrain an artificial intelligence model using third judgment request information, user prediction judgment result information regarding the third judgment request information, and third judgment data as retraining data.
[0352] An electronic device according to one embodiment can set the weight parameters of an artificial intelligence model as initial parameters of an artificial intelligence model for a new user when it detects that the user has changed to a new user based on an identification code.
[0353] An electronic device according to one embodiment can obtain first judgment data of a new user regarding first judgment request information.
[0354] An electronic device according to one embodiment can train an artificial intelligence model for a new user by using first judgment request information and first judgment data of a new user as training data.
[0355] A method according to one embodiment of the present disclosure may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.
[0356] Additionally, the method according to the disclosed embodiments may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product.
[0357] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product may include a product in the form of a software program (e.g., a downloadable app) that is electronically distributed through a manufacturer of an electronic device or an electronic market (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a server of the manufacturer, a server of the electronic market, or a storage medium of a relay server that temporarily stores the software program.
[0358] A computer program product may include a storage medium of a server or a storage medium of a client device in a system composed of a server and a client device. Alternatively, if a third device (e.g., a smartphone) is communicationally connected to the server or client device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include the S / W program itself, which is transmitted from the server to the client device or the third device, or from the third device to the client device.
[0359] In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.
[0360] For example, a server (e.g., a cloud server or an artificial intelligence server, etc.) can execute a computer program product stored on the server to control a client device connected to the server in communication to perform a method according to the disclosed embodiments.
[0361] Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention.
Claims
Claim 1 An electronic device for outputting predictive judgment result information regarding a task requiring user judgment, wherein the electronic device comprises: a memory storing a program or at least one instruction; at least one processor executing at least one instruction stored in the memory; and a display; wherein, by the at least one processor executing at least one instruction stored in the memory, the electronic device obtains first judgment request information regarding a task requiring user judgment, obtains first judgment data of the user regarding the first judgment request information, inputs second judgment request information to an artificial intelligence model trained using the first judgment request information and the first judgment data as training data to obtain predictive judgment result information of the user regarding the second judgment request information, obtains second judgment data of the user regarding the second judgment request information, calculates the reliability of the user's predictive judgment result information regarding the second judgment request information based on a comparison of the second judgment data and the user's predictive judgment result information regarding the second judgment request information, and if the reliability is greater than or equal to a threshold value, inputs third judgment request information to the artificial intelligence model to output the user's predictive judgment result information regarding the third judgment request information through the display, and wherein the user regarding the task In a first operation mode where a judgment is being performed, if the reliability is within a first threshold range, the user's predicted judgment result information regarding the third judgment request information is output along with the reliability, and in a second operation mode where the user is not performing a judgment on the task, if the reliability is within a second threshold range, the task according to the third judgment request information is processed according to the user's predicted judgment result information regarding the third judgment request information.An electronic device that, when the above reliability is within a third threshold range, transmits the third judgment request information and the user's predicted judgment result information regarding the third judgment request information to an external device; when the above reliability is within a fourth threshold range, suspends business processing according to the third judgment request information until the user's third judgment data regarding the third judgment request information is obtained; when the above reliability is within the fourth threshold range, stores the third judgment request information and the user's predicted judgment result information regarding the third judgment request information; when the operation mode of the electronic device switches from the second operation mode to the first operation mode, obtains the user's third judgment data regarding the third judgment request information; and retrains the artificial intelligence model using the third judgment request information, the user's predicted judgment result information regarding the third judgment request information, and the third judgment data as retraining data. Claim 2 An electronic device according to claim 1, wherein the electronic device, by executing at least one instruction stored in the memory, determines an identification code to identify a user among a plurality of users who will process the first judgment request information based on the first judgment request information, and identifies the user corresponding to the identification code among the plurality of users. Claim 3 In paragraph 2, the electronic device, by executing at least one instruction stored in the memory by the at least one processor, acquires the first judgment data based on the identification code and trains the artificial intelligence model based on the identification code, the first judgment request information and the first judgment data. Claim 4 In paragraph 3, the electronic device calculates the reliability based on at least one of the degree of agreement between the user’s prediction judgment result information regarding the second judgment data and the second judgment request information, or the degree of agreement between the distribution of the first judgment data and the distribution of the second judgment data, by the at least one processor executing at least one instruction stored in the memory. Claim 5 An electronic device according to claim 4, wherein the electronic device calculates a judgment pattern stability based on the difference between the distribution of the user's recent judgment data and the distribution of past judgment data by the at least one processor executing at least one instruction stored in the memory, and calculates the reliability using the degree of agreement between the user's predicted judgment result information regarding the second judgment data and the second judgment request information, the degree of agreement between the distribution of the first judgment data and the distribution of the second judgment data, and the judgment pattern stability. Claim 6 An electronic device according to claim 4, wherein, by the at least one processor executing at least one instruction stored in the memory, the electronic device retrains the artificial intelligence model using the second judgment request information, the user’s prediction judgment result information regarding the second judgment request information, and the second judgment data as retraining data when the reliability is below a threshold value. Claim 7 An electronic device according to claim 6, wherein the electronic device, by executing at least one instruction stored in the memory, obtains a similarity between the first judgment request information, the second judgment request information, and the third judgment request information, and obtains a user's prediction judgment result information regarding the third judgment request information based on the similarity. Claim 8 In paragraph 2, the electronic device, by executing at least one instruction stored in the memory, sets the weight parameters of the artificial intelligence model as initial parameters of the artificial intelligence model for the new user when it is detected that the user has changed to a new user based on the identification code, obtains the first judgment data of the new user regarding the first judgment request information, and trains the artificial intelligence model for the new user using the first judgment request information and the first judgment data of the new user as training data. Claim 9 delete Claim 10 delete
Citation Information
Patent Citations
Action selection system, action selection method, predictive machine training system, predictive machine training method, and predictive machine
JP2008508581A
Evaluation support program, evaluation support method, and information processing device
JP7275591B2
Generative AI-based decision-making method amd system
KR1020250154773A
Proxy for selective use of human and artificial intelligence in a natural language understanding system
US10789943B1
Learning device, learning method, learning program, and dialogue device
WO2025032740A1
