Training system for mutual education between counselor and agent via real-time cousultation analytics

The mutual teacher learning system enhances consultation response quality by analyzing real-time data to provide immediate feedback and warnings, improving counselor performance through continuous learning and risk prevention.

WO2026235352A1PCT designated stage Publication Date: 2026-11-12DIQUEST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2026/005879
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-07
Filing Date
2026-05-07
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing customer consultation systems face challenges in ensuring consistent consultation quality due to reliance on manual information searches and agent experience, leading to decreased satisfaction and regulatory risks from inappropriate expressions, with limited systematic analysis and feedback for improving response quality.

Method used

A mutual teacher learning system that automatically analyzes real-time consultation data to identify key keywords and topics, provides real-time knowledge information and warnings, and incorporates feedback into a learning algorithm to enhance response accuracy and consistency.

Benefits of technology

Improves the quality and speed of consultation responses through continuous feedback-based learning, minimizing deviations in counselor capabilities and identifying potential complaint risks proactively.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2026005879_12112026_PF_FP_ABST
    Figure KR2026005879_12112026_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed is a training system for mutual education between a counselor and an agent via real-time consultation analytics, the system comprising: a consultation sentence analysis summary unit that generates consultation sentence summary information summarizing real-time consultation information that is generated as consultation between a counselor and a customer proceeds; a consultation knowledge information provision unit that identifies, from a pre-stored knowledge base, consultation knowledge information having a high relevance score for a consultation sentence and provides same to a counselor, on the basis of a result of analyzing a core keyword and a consultation topic via a pre-stored relevance evaluation algorithm; a real-time warning output unit that outputs a warning for an identified keyword; and a feedback training unit that enhances the performance of the consultation sentence analysis summary unit and the consultation knowledge information provision unit by receiving feedback information on sentence summary information and enabling analysis and learning of feedback evaluation information via a feedback learning algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Mutual learning system between counselors and agents through real-time consultation analysis

[0001] The present invention relates to a mutual teacher learning system between counselors and agents through real-time consultation analysis. Specifically, it relates to a technology that supports mutual teacher-based learning between counselors and agents by automatically analyzing real-time consultation data between a counselor and a customer to identify key keywords and consultation topics, providing related consultation knowledge information in real time, outputting warnings regarding expressions of risk during consultation, and providing consultation summary information to the counselor after the consultation ends, thereby reflecting feedback information based on the consultation summary information from the counselor into a learning algorithm that has been stored as training data.

[0002]

[0003] [National R&D projects that supported this invention]

[0004] [Project ID] 2710008210

[0005] [Project No.] RS-2024-00343989

[0006] [Ministry Name] Ministry of Science and ICT

[0007] [Project Management (Specialized) Agency Name] Korea Institute of Information & Communications Technology Planning & Evaluation

[0008] [Research Project Name] Next-Generation Generative AI Technology Development (R&D)

[0009] [Research Project Title] Data Characteristics and Generative AI Models for Social and Ethical Learning

[0010] [Name of Project Performing Organization] Daiquest Co., Ltd.

[0011] [Research Period] 2024.04.01 ~ 2026.12.31

[0012] Existing customer consultation systems operate by having agents respond to customer inquiries in real-time, relying on manual information searches or experience to provide answers. This approach makes it difficult to ensure consistency in consultation quality; in the case of new agents, this frequently leads to decreased customer satisfaction due to a lack of response capabilities. Furthermore, the failure to detect prohibited words or inappropriate expressions used during consultations in real-time can result in complaints or regulatory risks. Additionally, there is a problem in that the lack of systematic analysis and feedback regarding agents' responses or handling methods after the consultation ends limits the potential for improving the quality of consultation services.

[0013] Accordingly, the industry is developing various technologies to solve the aforementioned problems.

[0014] As an example, Korean registered patent 10-2537051 (AI-based counselor mis-selling evaluation system) discloses a technology for evaluating product information related to financial products.

[0015] However, the aforementioned prior art discloses only a technology that determines whether mis-selling has occurred by simply converting audio files into text and comparing them with a script. It does not disclose a technology that supports mutual teacher-based learning between a counselor and an agent by automatically analyzing real-time consultation data between a counselor and a customer to identify key keywords and consultation topics, providing related consultation knowledge information in real time, outputting warnings regarding risk expressions during the consultation, and providing consultation summary information to the counselor after the consultation ends, thereby reflecting feedback based on the consultation summary information into a learning algorithm stored as training data. Consequently, there is a growing need for a technology capable of addressing this issue.

[0016] Accordingly, the present invention is designed to solve the problems of the aforementioned existing technologies. Its purpose is to improve the accuracy and speed of real-time consultation responses, minimize deviations in response quality due to differences in counselor capabilities, and enhance the response quality of the consultation support system through continuous feedback-based learning. This is achieved by automatically analyzing real-time consultation data between a counselor and a customer to identify key keywords and consultation topics, providing related consultation knowledge information in real time, outputting warnings regarding risk expressions during the consultation, and providing consultation summary information to the counselor after the consultation ends. By reflecting feedback based on the consultation summary information into a learning algorithm stored as training data, the invention supports mutual teacher-based learning between counselors and agents.

[0017] A mutual teaching learning system between a counselor and an agent through real-time consultation analysis, implemented by a computing device comprising one or more processors and one or more memories for storing instructions executable by said processors according to an embodiment of the present invention, wherein, upon receiving real-time consultation information generated as consultation between a counselor and a customer proceeds, the system preprocesses said real-time consultation information through a stored natural language processing algorithm to identify core keywords and consultation topics included in the identified consultation sentences, and generates consultation sentence summary information that summarizes said consultation sentences; a consultation knowledge information providing unit, upon completion of the function execution of said consultation sentence analysis summarizing unit, identifies consultation knowledge information with a high correlation score for said consultation sentences from a pre-stored knowledge base based on the result of analyzing said core keywords and said consultation topics through a stored correlation evaluation algorithm, and provides it to the counselor; and a real-time warning output unit, which outputs a real-time warning for said identified keyword when, during the function execution of said consultation knowledge information providing unit, a keyword satisfying a pre-set risk condition among a plurality of keywords included in said consultation sentences is identified. The system is characterized by including a feedback learning unit that, when consultation sentence summary information is provided to the counselor upon termination of consultation between the counselor and the customer, receives feedback evaluation information regarding the consultation sentence summary information from the counselor, stores the feedback evaluation information in a learning database, and analyzes and learns the feedback evaluation information through a previously stored feedback learning algorithm to enhance the performance of the consultation sentence analysis summary unit and the consultation knowledge information provision unit.

[0018] The above consultation sentence analysis summary unit, upon receiving real-time consultation information generated as a consultation between a counselor and a customer proceeds, preprocesses the real-time consultation information through a stored natural language processing algorithm to identify the consultation sentence corresponding to the real-time consultation information, performs a tokenization process on the consultation sentence to tokenize the consultation sentence, performs a vectorizing process on the tokens to calculate a vector value for each token quantified based on the position of each token in the consultation sentence, and calculates a sentence vector value for the consultation sentence; and when the calculation of the sentence vector value is completed, the core keyword identification unit reflects the sentence vector value to an importance judgment model, which is a partial model included in the stored natural language processing algorithm, calculates an importance score between keywords included in the consultation sentence, and identifies core keywords expressing the customer's main interests in the consultation sentence. It is preferable to include a consultation topic identification unit that, when the function of the core keyword identification unit is completed, reflects the sentence vector value in a topic classification model, which is a part of the model included in the previously stored natural language processing algorithm, to identify the consultation topic regarding the consultation proceeding between the counselor and the customer reflected in the consultation sentence.

[0019] When the generation of the sentence vector value is completed, the above consultation sentence analysis summary unit can reflect the sentence vector value in a summary model, which is part of a model included in the previously stored natural language processing algorithm, so that the summary model generates consultation sentence summary information that summarizes the core information of the consultation sentence based on the result of determining the contextual importance of the consultation sentence.

[0020] It is preferable that the counseling knowledge information providing unit comprises: a frequency analysis unit that, when the function of the counseling sentence analysis summary unit is completed, calculates the co-occurrence frequency and individual occurrence probability for each combination through the result of preprocessing the core keywords and counseling topics identified in the counseling sentence and the plurality of counseling knowledge information stored in the knowledge base using the interdependence model of the previously stored association evaluation algorithm; and a selection knowledge information providing unit that, when the calculation of the co-occurrence frequency and individual occurrence probability is completed, provides at least one counseling knowledge information among the plurality of counseling knowledge information stored in the knowledge base, in which the association score for the counseling sentence is greater than or equal to a preset threshold value, to the counselor account of the counselor currently conducting counseling with the customer, based on the association score calculated by reflecting the co-occurrence frequency and individual occurrence probability in the interdependence model.

[0021] It is preferable that the real-time warning output unit identifies multiple keywords included in a consultation sentence through a stored natural language processing algorithm while the function of the consultation knowledge information provision unit is being performed, checks whether they are identical to a reference keyword included in each of the multiple risk categories based on the pre-set risk conditions, and if at least one of the multiple keywords is identical to the reference keyword, outputs a warning message matched to the reference keyword to the counselor in real time.

[0022] The above feedback learning unit preferably includes: an evaluation information storage unit that, when consultation sentence summary information is provided to the counselor upon termination of consultation between the counselor and the customer, receives feedback evaluation information regarding said consultation sentence summary information from the counselor and stores said feedback evaluation information in a learning database; and a learning pattern derivation reflection unit that, when the storage of said feedback evaluation information in the learning database is completed, analyzes the detailed information included in said feedback evaluation information, namely the original of said sentence summary information, whether said sentence summary information has been modified, the content of the modification, whether it has been accepted, the selected item, the evaluation score, and the reason for feedback, through said previously stored feedback learning algorithm, and reflects the derived learning pattern value in said previously stored natural language processing algorithm and said previously stored correlation evaluation algorithm.

[0023] The counselor and agent mutual teaching learning system through the above-described real-time consultation analysis preferably further includes a trend analysis unit that, when a pre-set trend analysis condition is satisfied as multiple consultation sentence summary information is stored in the summary database, generates trend analysis result information including trend trend information, rapidly rising keywords, expanded interest group information, and issue topic history information based on the result of analyzing multiple consultation sentence summary information accumulated and stored in the summary database through a pre-stored trend analysis algorithm.

[0024] The trend analysis unit may include: a time series information generation unit that identifies a consultation topic included in each of the multiple consultation sentence summary information stored in the summary database, identifies a consultation type category having the same identification value as the identified consultation topic among the pre-set consultation type categories, classifies the multiple consultation sentence summary information, and generates time series information that aggregates the number of times a topic appears per unit period; and a trend trend generation unit that, when the generation of the time series information is completed, analyzes the time series information through a pre-stored trend analysis algorithm to generate trend trend information capable of identifying the increasing trend, decreasing trend, recurring occurrence pattern, and abnormal fluctuation of the consultation volume by the consultation topic.

[0025] The trend analysis unit comprises: a frequency change rate calculation unit that identifies key keywords included in each of the multiple consultation sentence summary information stored in the summary database, calculates the frequency of occurrence of the key keywords for each preset time interval, and calculates the rate of change in frequency of occurrence according to the time interval for each key keyword using the calculated frequency of occurrence; and a surge keyword identification unit that, when the function of the frequency change rate calculation unit is completed, identifies as surge keywords a key keyword whose frequency of occurrence in the current time interval increases rapidly by more than a preset appearance threshold ratio compared to the average frequency of occurrence in the previous time interval through the rate of change in frequency of occurrence according to the time interval for each identified key keyword. It is possible to further include a group information generation unit that, when the function of the above-mentioned surge keyword identification unit is completed, calculates the co-occurrence frequency and individual occurrence probability for each combination through the result of preprocessing the core keyword identified in the consultation sentence and the past core keyword identified prior to the current point in time using the interdependence model of the previously stored association evaluation algorithm, and then, based on the association score calculated by reflecting the co-occurrence frequency and individual occurrence probability in the interdependence model, identifies past core keywords among the past core keywords whose association score for the core keyword is greater than or equal to a preset association score, and groups the identified past core keywords and the core keywords to generate expanded interest group information.

[0026] The trend analysis unit may include: an emotional intensity calculation unit that analyzes multiple consultation sentence summary information stored in the summary database using a previously stored sentiment analysis algorithm to identify positive and negative keywords for consultation summary sentences corresponding to each of the multiple consultation sentence summary information, and calculates the emotional intensity through each of the identified positive and negative keywords; an index grouping unit that, when the calculation of the emotional intensity is completed, reflects the emotional intensity in a reliability classification model of the previously stored sentiment analysis algorithm to calculate an emotional quality index for the consultation summary sentences, identifies the consultation type of the consultation summary sentence corresponding to the calculated emotional quality index as a consultation topic, and groups the emotional quality index corresponding to each of the multiple consultation sentence summary information to the identified consultation type; and an issue topic information generation unit that, when the function of the index grouping unit is completed, identifies the rate of change of the emotional quality index in units of a previously set time interval, and if the rate of change of the identified emotional quality index remains below a reference index for a predetermined period, determines the consultation topic based on the consultation type corresponding to the emotional quality index as an issue topic with a high probability of civil complaints occurring, and generates issue topic history information.

[0027] Through the mutual teacher learning system between counselors and agents based on real-time consultation analysis of the present invention, real-time consultation information between counselors and customers is automatically analyzed to identify key keywords and consultation topics, and based on this, related knowledge information is provided in real time or warnings of risk expressions are issued, thereby effectively improving the quality of consultation response and response speed.

[0028] In addition, feedback evaluation information collected from counselors after the counseling session ends is accumulated in a learning database, and based on this, the quality of counseling sentence summaries and the accuracy of knowledge information provision can be continuously improved, thereby effectively enhancing the learning performance of counselors and counseling support agents.

[0029] Finally, by systematically analyzing trend patterns by consultation topic, surging keywords, expanded interest groups, and the history of changes in sentiment quality based on accumulated consultation sentence summary information, it is possible to identify issue topics with a high potential for increased customer complaints at an early stage and prevent complaint risks in advance.

[0030] FIG. 1 is a block diagram illustrating a counselor and agent mutual teacher learning system through real-time counseling analysis according to one embodiment of the present invention.

[0031] FIG. 2 is a block diagram illustrating the counseling sentence analysis summary section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to an embodiment of the present invention.

[0032] FIG. 3 is a block diagram illustrating a counseling knowledge information provision unit of a counselor and agent mutual teacher learning system through real-time counseling analysis according to an embodiment of the present invention.

[0033] FIG. 4 is a block diagram illustrating the feedback learning section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to an embodiment of the present invention.

[0034] FIG. 5 is a block diagram illustrating the trend analysis section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to an embodiment of the present invention.

[0035] FIG. 6 is another block diagram illustrating the trend analysis section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to one embodiment of the present invention.

[0036] FIG. 7 is another block diagram illustrating the trend analysis section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to one embodiment of the present invention.

[0037] FIG. 8 is a drawing for explaining an example of the internal configuration of a computing device according to an embodiment of the present invention.

[0038] Hereinafter, various embodiments and / or aspects are disclosed with reference to the drawings. For illustrative purposes, numerous specific details are disclosed in the following description to aid in a general understanding of one or more aspects. However, it will also be recognized by those skilled in the art that these aspects may be practiced without such specific details. The following description and the accompanying drawings describe specific exemplary aspects of one or more aspects in detail. However, these aspects are exemplary, and some of the various methods in the principles of the various aspects may be used, and the description is intended to include all such aspects and their equivalents.

[0039] As used herein, terms such as "examples," "examples," "aspects," "examples," etc., may not be interpreted as implying that any aspect or design described is better or more advantageous than other aspects or designs.

[0040] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that the relevant feature and / or component is present, but not to exclude the presence or addition of one or more other features, components and / or groups thereof.

[0041] Additionally, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0042] Furthermore, in the embodiments of the present invention, all terms used herein, including technical or scientific terms, unless otherwise defined, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the embodiments of the present invention.

[0043] FIG. 1 is a block diagram illustrating a counselor and agent mutual teacher learning system through real-time counseling analysis according to one embodiment of the present invention.

[0044] Referring to FIG. 1, a counselor and agent mutual teacher learning system (100) (hereinafter referred to as the mutual teacher learning system) through real-time counseling analysis, implemented as a computing device comprising one or more processors and one or more memories for storing instructions that can be executed by said processors, may include a counseling sentence analysis summary unit (101), a counseling knowledge information provision unit (103), a real-time warning output unit (105), and a feedback learning unit (107).

[0045] According to one embodiment, when the consultation sentence analysis summary unit (101) receives real-time consultation information (101a) generated as consultation between a counselor and a customer proceeds, it can preprocess the real-time consultation information (101a) through a stored natural language processing algorithm (109) to identify key keywords and consultation topics included in the identified consultation sentence, and generate consultation sentence summary information that summarizes the consultation sentence.

[0046] According to one embodiment, when the consultation sentence analysis summary unit (101) receives real-time consultation information (101a) generated as consultation between a counselor and a customer proceeds, it can preprocess the real-time consultation information (101a) through a preprocessing model included in a stored natural language processing algorithm (109).

[0047] In relation to the above, the consultation sentence analysis summary unit (101) can perform a refinement process such as removing stop words, organizing punctuation, separating sentences, and distinguishing speakers within a conversation when preprocessing the real-time consultation information (101a), and then divide the consultation sentences corresponding to the preprocessed real-time consultation information (101a) into sentence units.

[0048] According to one embodiment, the consultation sentence analysis summary unit (101) can generate a sentence vector value for each consultation sentence by performing a tokenization process for each consultation sentence divided into sentence units using a tokenization model included in a stored natural language processing algorithm (109), dividing into semantic units, and by performing a vectorization process for each divided token using a vectorization model to calculate a vector value that reflects location information and semantic similarity information within the consultation sentence.

[0049] According to one embodiment, when the consultation sentence analysis summary unit (101) calculates a sentence vector value for a consultation sentence through the vectorization model, it inputs the sentence vector value into an attention-based importance scoring model included in the previously stored natural language processing algorithm (109) to calculate the mutual association and contextual importance between tokens spoken within the consultation sentence.

[0050] Accordingly, the consultation sentence analysis summary unit (101) can identify key keywords that reflect the customer's main requirements or sensitive interests within the consultation sentence based on the results of calculating importance, and can derive a consultation topic by classifying the semantic topic of the consultation sentence through a topic classification model included in a stored natural language processing algorithm (109).

[0051] In addition, the consultation sentence analysis summary unit (101) can input the sentence vector value into a summary generation model included in a stored natural language processing algorithm (109) based on the identified core keyword and consultation topic, thereby generating consultation sentence summary information (101b) in which the semantic core of the consultation sentence is condensed.

[0052] This process is performed to improve the quality of consultation sentence summary information. While it is possible to directly input the sentence vector values ​​of the consultation sentences into the summary model, inputting identified keywords or topics as conditions allows for controlling the direction of the summary results or generating more topic-consistent summaries.

[0053] According to one embodiment, a stored natural language processing algorithm (109) may include a pre-processing model, a tokenization model, a vectorization model, an importance scoring model, a topic classification model, and a summary generation model to perform automatic analysis and summarization of real-time consultation information (101a).

[0054] According to one embodiment, the preprocessing model may be a model that refines the consultation sentence into a structured form by removing stopwords, special characters, and punctuation included in the received real-time consultation information (101a), and performing processes such as speaker identification, sentence segmentation, and reordering based on speech units.

[0055] Furthermore, the tokenization model divides refined consultation sentences into individual words or phrases (tokens) by splitting them into semantic or morpheme units, which can serve as the basic input for subsequent vectorization and importance evaluation processes. The vectorization model converts each tokenized word or phrase into a high-dimensional vector representation by reflecting contextual and positional information, thereby enabling the numerical representation of the meaning of each consultation sentence.

[0056] In addition, the importance judgment model may be a model that evaluates the interrelationships and contextual importance between each token within the generated sentence vector values, and identifies core keywords by selecting semantically key words from customer requests or agent responses. The topic classification model may be a model that takes the entire sentence vector of the consultation sentence or core keywords as input and classifies the topic contained in the consultation sentence into one of multiple consultation types.

[0057] Finally, the summary model may be a model that generates a summary sentence condensing the semantic core of the consultation sentence based on the above sentence vector values, identified key keywords, and consultation topics, and generates consultation sentence summary information categorized into customer request summary, agent response summary, etc.

[0058] According to one embodiment, the consultation sentence summary information may be information that summarizes key information, such as the gist of a customer's request, the content of a counselor's response, and emotional expressions to be mindful of during the consultation, in the form of natural language sentences.

[0059] According to one embodiment, when the function of the counseling sentence analysis summary unit (101) is completed, the counseling knowledge information providing unit (103) can identify counseling knowledge information with a high correlation score for the counseling sentence from a previously stored knowledge base based on the result of analyzing the core keyword and the counseling topic through a previously stored correlation evaluation algorithm (111) and provide it to the counselor.

[0060] According to one embodiment, the counseling knowledge information providing unit (103) can analyze the core keywords and counseling topic information through a previously stored correlation evaluation algorithm (111) after the function of the counseling sentence analysis summary unit (101) is completed. Accordingly, the counseling knowledge information providing unit (103) can quantify the semantic intention inherent in the counseling sentence through the analysis results.

[0061] In relation to the above, the association evaluation algorithm (111) may include a mutual dependency-based probability model. In relation to the above, the mutual dependency-based probability model (mutual dependency model) can derive an association score for each pair combination by calculating the frequency of joint occurrence and the probability of individual occurrence between each of the multiple counseling knowledge information items in a prior-stored knowledge base and the core keyword and counseling topic.

[0062] At this time, the counseling knowledge information providing unit (103) can select one or more counseling knowledge information that has the highest semantic correlation with the counseling sentence based on the magnitude of the correlation score, and provide it in real time to the account interface or screen of the counselor currently performing counseling.

[0063] According to one embodiment, the consultation knowledge information may include, for example, processing procedures for cases similar to customer inquiries (summary information of past similar consultation cases, processing history, or history of cases causing complaints), related policy descriptions (description information regarding company internal policies, processing procedures, relevant regulations, or legal notices), legal notice phrases, and response template sentences (information on structured natural language response sentences written in advance according to the type of customer question), and the consultation knowledge information may be utilized as automatic recommendation information to improve the accuracy and consistency of customer responses.

[0064] According to one embodiment, the real-time warning output unit (105) can output a real-time warning for the identified keyword when a keyword satisfying a preset risk condition is identified among a plurality of keywords included in the consultation sentence while the function of the consultation knowledge information providing unit (103) is being performed.

[0065] More specifically, while the function of the counseling knowledge information providing unit (103) is being performed, the real-time warning output unit (105) identifies multiple keywords included in the counseling sentence through a stored natural language processing algorithm (109), checks whether they are identical to a reference keyword included in each of the multiple risk categories based on the pre-set risk conditions, and if at least one of the multiple keywords is identical to the reference keyword, it can output a warning message matched to the reference keyword to the counselor in real time.

[0066] According to another embodiment, the real-time warning output unit (105) can determine whether each keyword is the same as a keyword based on a pre-set risk condition by utilizing a risk expression discrimination model included in a pre-stored natural language processing algorithm (109).

[0067] According to one embodiment, the pre-established risk conditions are divided into multiple categories including expressions included in a pre-registered list of prohibited words, excessive definitive expressions that may cause customer misunderstanding or mis-selling, negative emotional keywords with a high likelihood of causing emotions or complaints, and expressions corresponding to omissions or violations in standard consultation scripts in product descriptions or contract-related conversations, and each category may have a pre-registered list of reference keywords.

[0068] According to one embodiment, when a keyword satisfying at least one of the preset risk conditions is identified in a consultation sentence, the real-time warning output unit (105) outputs a warning message matched with the keyword in real time to the counselor visually or audibly, and if necessary, outputs a recommended phrase that can replace the keyword (e.g., "Please soften the expression," "Can be replaced with the following phrase: ~") together, thereby enabling the prevention of risks during consultation in advance.

[0069] At this time, the real-time warning output unit (105) can output a real-time warning in the form of a pop-up notification, warning sound, or highlighting on the counselor screen, and can apply a warning intensity differently depending on the warning level or transmit the warning history to the administrator system.

[0070] According to one embodiment, when the feedback learning unit (107) provides counseling sentence summary information to the counselor as the counseling between the counselor and the customer ends, it receives feedback evaluation information regarding the counseling sentence summary information from the counselor, stores the feedback evaluation information in a learning database, and analyzes and learns the feedback evaluation information through a previously stored feedback learning algorithm, thereby enhancing the performance of the counseling sentence analysis summary unit and the counseling knowledge information providing unit.

[0071] At this time, the feedback learning unit can enhance the performance regarding the content consistency and accuracy of natural language expression of the counseling summary information of the counseling sentence analysis summary unit (101) and the performance regarding the accuracy of the correlation judgment of the counseling knowledge information of the counseling knowledge information providing unit (103).

[0072] More specifically, the feedback learning unit (107) can receive feedback evaluation information regarding the counseling sentence summary information from the counselor after providing the counselor with the counseling sentence analysis summary information generated in the counseling sentence summary unit (101) as the counseling between the counselor and the customer ends.

[0073] In relation to the above, the feedback evaluation information is response information based on the counselor's actual usage experience regarding the content of the counseling sentence summary information, and may include the original text of the counseling sentence summary information, whether the counseling summary information was modified, the content of the modification, whether the counselor accepted or rejected the original counseling summary information, evaluation categories for multiple selection items (e.g., unclear, excessive length, off-topic, etc.), evaluation score, and reason for feedback.

[0074] For example, the above feedback evaluation information can include the following details as details: the automatically generated consultation summary "The customer inquired about the billing details for last month, and the agent checked and provided the billing details," Modification Status: Yes, Modification Details: "The customer inquired about the details of the billing amount for March 28 (19,800 won)," Selected Evaluation Item: "Missing key information," Score Rating: 3.5 / 5, Reason for Feedback: "It is inconvenient because I have to explain it again if the billing amount or date is missing."

[0075] According to one embodiment, the feedback learning unit (107) can analyze and learn qualitative and quantitative evaluation data regarding counseling sentence summary information and counseling knowledge information provision history by storing feedback evaluation information received from a counselor in a learning database and inputting the stored evaluation information into a previously stored feedback learning algorithm (113).

[0076] More specifically, the feedback learning unit (107) stores feedback evaluation information received from a counselor in a learning database and analyzes and learns the feedback evaluation information through a previously stored feedback learning algorithm (113), thereby updating the previously stored natural language processing algorithm (109) used in the counseling sentence analysis summary unit (101) to enhance the function of the sentence analysis summary unit (101).

[0077] In relation to the above, the feedback learning unit (107) can improve the consistency of the content of the counseling sentence summary information generated by the counseling sentence analysis summary unit (101) and the accuracy of the natural language expression by reflecting the learning pattern values ​​derived from analyzing and learning feedback evaluation information through the previously stored feedback learning algorithm (113), expression patterns evaluated as frequently modified or unclear, vocabulary or sentence structures repeatedly avoided by the counselor, and cases where the summary content is excessively long or deviates from the main point, into the previously stored natural language processing algorithm (109).

[0078] Additionally, the feedback learning unit (107) can improve the accuracy of determining the topic suitability and relevance of counseling knowledge information recommended by the counseling knowledge information provider (103) by reflecting the case-based learning pattern values ​​derived by analyzing and learning feedback evaluation information through the previously stored feedback learning algorithm (113) into the previously stored relevance evaluation algorithm (111), which evaluates the counseling knowledge information provided by the counselor as having low relevance or as not matching the topic.

[0079] In other words, the present invention may be an invention for realizing a mutual teaching structure in which the counselor and the system improve each other by not only receiving information from the system but also utilizing the feedback left by the counselor to train the system again, thereby providing more accurate and useful information to the counselor thereafter.

[0080] FIG. 2 is a block diagram illustrating the counseling sentence analysis summary section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to an embodiment of the present invention.

[0081] Referring to FIG. 2, a counselor and agent mutual teacher learning system through real-time counseling analysis (e.g., counselor and agent mutual teacher learning system through real-time counseling analysis of FIG. 1 (100)) (hereinafter referred to as the mutual teacher learning system) implemented as a computing device comprising one or more processors and one or more memories for storing instructions that can be executed by said processors may include a counseling sentence analysis summary unit (200) (e.g., counseling sentence analysis summary unit (101) of FIG. 1).

[0082] According to one embodiment, when the consultation sentence analysis summary unit (200) receives real-time consultation information (201a) generated as consultation between a counselor and a customer proceeds, it can preprocess the real-time consultation information (201a) through a stored natural language processing algorithm to identify key keywords and consultation topics included in the identified consultation sentence, and generate consultation sentence summary information that summarizes the consultation sentence.

[0083] According to one embodiment, the consultation sentence analysis summary unit (200) may include a vector value calculation unit (201), a core keyword identification unit (205), and a consultation topic identification unit (205) as detailed configurations for performing the above-described function.

[0084] According to one embodiment, when the vector value calculation unit (201) receives real-time consultation information (201a) generated as consultation between a counselor and a customer proceeds, it preprocesses the real-time consultation information (201a) through a stored natural language processing algorithm to identify a consultation sentence corresponding to the real-time consultation information (201a), performs a tokenization process on the consultation sentence to tokenize the consultation sentence, and performs a vectorizing process on the tokens to calculate a vector value for each token quantified based on the position of each token in the consultation sentence, thereby calculating a sentence vector value for the consultation sentence.

[0085] According to one embodiment, when the vector value calculation unit (201) receives real-time consultation information (201a) generated as consultation between a counselor and a customer proceeds, it can preprocess the real-time consultation information (201a) through a preprocessing model included in a stored natural language processing algorithm, separate it into sentence units, and remove unnecessary punctuation, special characters, stop words, etc., thereby refining the consultation sentence into a form suitable for semantic analysis.

[0086] According to one embodiment, the vector value calculation unit (201) can be configured to analyze the equivalent sentence through a tokenization model included in a stored natural language processing algorithm, divide the consultation sentence into tokens in word units or semantic units, and consider the contextual position and order of each token.

[0087] Afterwards, the vector value calculation unit (201) can apply a vectorization model to each token included in the tokenized consultation sentence to calculate a high-dimensional vector value that reflects the contextual meaning, location information, and topic relevance of the corresponding token, and can generate a sentence vector value representing the entire consultation sentence by combining the vector values ​​of the individual tokens calculated in this way.

[0088] According to one embodiment, when the calculation of the sentence vector value is completed, the core keyword identification unit (203) reflects the sentence vector value to an importance judgment model (207), which is a part of the model included in the previously stored natural language processing algorithm, to calculate an importance score between the keywords included in the consultation sentence and can identify the core keywords that express the customer's main interests in the consultation sentence.

[0089] According to one embodiment, when the core keyword identification unit (203) completes the calculation of sentence vector values ​​for a consultation sentence, it can evaluate the relative importance of multiple tokens (words or phrases) included in the sentence through the result of analyzing the sentence vector values ​​using an importance judgment model (207) included in a stored natural language processing algorithm.

[0090] According to one embodiment, the importance judgment model (207) may include an attention-based weight calculation technique or a transformer-family structure, and may calculate an importance score for each token by comprehensively considering the contextual relationship, location information, frequency of expression, semantic role within the conversation, etc., between each token constituting the sentence vector value.

[0091] According to one embodiment, the core keyword identification unit (203) can automatically select and identify one or more core keywords that can represent the customer's request intent or sensitive interests within the consultation sentence, based on the importance score calculated by the importance judgment model (207).

[0092] According to one embodiment, the core keyword is a keyword that can be utilized to classify counseling topics, search for related counseling knowledge information, or generate counseling summary sentences, and can be created as a separate keyword information object and stored within the system.

[0093] According to one embodiment, when the function of the core keyword identification unit (203) is completed, the consultation topic identification unit (205) can identify the consultation topic of the consultation between the counselor and the customer reflected in the consultation sentence by reflecting the sentence vector value in the topic classification model (209), which is a part model included in the previously stored natural language processing algorithm.

[0094] According to one embodiment, the counseling topic identification unit (205) can automatically classify counseling topics by analyzing semantic content included in the counseling sentence, by reflecting the sentence vector value in a topic classification model (209) included in a stored natural language processing algorithm when the function of the core keyword identification unit (203) is completed.

[0095] According to one embodiment, the topic classification model (209) is a classification model that predicts whether a consultation sentence is included in any of a number of predefined consultation topic categories (e.g., fee inquiry, service cancellation, address change, payment error, etc.), and may be a model that comprehensively analyzes the semantic center, contextual flow, sentence structure, etc. of the sentence using the sentence vector value and core keyword as input values.

[0096] According to one embodiment, the counseling topic identification unit (205) derives one or more counseling topics corresponding to the counseling sentences based on the classification result, and the derived topic information can be stored as a separate topic identification result so that it can be used as reference information in various subsequent processing steps for recommending counseling knowledge information, analyzing trends, and learning based on feedback evaluation.

[0097] According to one embodiment, when the generation of the sentence vector value is completed, the consultation sentence analysis summary unit (200) may reflect the sentence vector value in a summary model, which is part of a model included in the previously stored natural language processing algorithm, so that the summary model may generate consultation sentence summary information that summarizes the core information of the consultation sentence based on the result of determining the contextual importance of the consultation sentence.

[0098] According to one embodiment, when the calculation of the sentence vector value is completed, the consultation sentence analysis summary unit (200) inputs the sentence vector value into a summary model, which is part of a model included in a stored natural language processing algorithm, to evaluate the overall contextual flow and semantic importance contained in the consultation sentence.

[0099] According to one embodiment, the summary model may be a Transformer-based sentence summary network or a natural language generation model with an encoder-decoder structure, and may be a model that condenses and summarizes the semantic core of the corresponding consultation sentence into a concise sentence form by comprehensively considering key keywords, topic information, sentence structure, and length information included in the sentence vector value.

[0100] Accordingly, the consultation sentence analysis summary unit (200) can generate consultation sentence summary information composed of a customer request summary, a counselor response summary, etc., based on the output result of the summary model, and the generated consultation sentence summary information can be stored in a separate structure or transmitted within the system so as to be utilized for subsequent knowledge information linkage, feedback evaluation, trend analysis, etc. At this time, in the process of performing the function of the summary model, the core keyword and consultation topic information may be information that includes the main purpose and content of the consultation regarding the generated consultation sentence summary information, as the core keyword and consultation topic information are utilized as conditional input values ​​of the summary model.

[0101] FIG. 3 is a block diagram illustrating a counseling knowledge information provision unit of a counselor and agent mutual teacher learning system through real-time counseling analysis according to an embodiment of the present invention.

[0102] Referring to FIG. 3, a counselor and agent mutual teacher learning system through real-time counseling analysis (e.g., counselor and agent mutual teacher learning system through real-time counseling analysis of FIG. 1 (100)) (hereinafter referred to as the mutual teacher learning system) implemented as a computing device comprising one or more processors and one or more memories for storing instructions that can be executed by said processors may include a counseling knowledge information providing unit (300) (e.g., counseling knowledge information providing unit (103) of FIG. 1).

[0103] According to one embodiment, when the function of the counseling knowledge information providing unit (300) is completed, the counseling knowledge information providing unit (300) can identify counseling knowledge information with a high correlation score for the counseling sentence from a previously stored knowledge base based on the result of analyzing the core keyword and the counseling topic through a previously stored correlation evaluation algorithm (301a) and provide it to the counselor.

[0104] According to one embodiment, the counseling knowledge information providing unit (300) may include an appearance frequency analysis unit (301) and a selection knowledge information providing unit (303) as detailed configurations for performing the above-described function.

[0105] According to one embodiment, when the function of the consultation sentence analysis summary unit (e.g., consultation sentence analysis summary unit (101) of FIG. 1) is completed, the appearance frequency analysis unit (301) can calculate the joint appearance frequency and individual appearance probability for each combination (a pair of a core keyword and consultation topic extracted from the consultation sentence and a consultation knowledge information item stored in the knowledge base) through the result of preprocessing the core keyword and consultation topic identified in the consultation sentence and a plurality of consultation knowledge information stored in the knowledge base through the interdependence model (305) of the previously stored association evaluation algorithm (301a).

[0106] According to one embodiment, the occurrence frequency analysis unit (301) can analyze the relationship between the core keywords and counseling topics identified in the counseling sentence through the interdependence model (305) and each of the plurality of counseling knowledge information stored in the knowledge base.

[0107] According to one embodiment, the occurrence frequency analysis unit (301) can form pairs between each core keyword and consultation topic included in the consultation sentence and each consultation knowledge information item stored in the knowledge base through the interdependence model (305), and then calculate the joint occurrence frequency and individual occurrence probability for each pair.

[0108] According to one embodiment, the interdependence model (305) is a model included in the previously stored association evaluation algorithm (301a), and is a statistical-based model for numerically evaluating the relationship between a core keyword and a consultation topic identified in a consultation sentence and each of a plurality of consultation knowledge information items stored in a knowledge base, and may be a model that calculates the frequency of co-occurrence and the probability of individual occurrence for each pair, and calculates an association score based on the calculated frequency of co-occurrence and the probability of individual occurrence.

[0109] In relation to the above, the interdependence model may be a model that learns the occurrence patterns between each keyword / topic and knowledge information item based on past counseling history or a pre-constructed learning corpus, and thereby is capable of identifying knowledge information that has a high semantic correlation with keywords and topics extracted from arbitrary counseling sentences.

[0110] According to one embodiment, the co-occurrence frequency can be calculated based on the number of times a specific keyword and topic and a specific counseling knowledge information item appear simultaneously in past counseling history or a learning corpus.

[0111] In addition, the above individual occurrence probability can be calculated based on the probability that a core keyword, consultation topic, or consultation knowledge information stored in the knowledge base appears alone in the entire consultation data (a set of all consultation history data collected and stored in advance by the system).

[0112] In relation to the above, the above-mentioned co-occurrence frequency and individual occurrence probability can be used as basic data for calculating association scores and can be used as information for selecting counseling knowledge information that has a high semantic correlation with counseling sentences.

[0113] According to one embodiment, when the calculation of the joint appearance frequency and individual appearance probability is completed, the selection knowledge information providing unit (303) may provide at least one counseling knowledge information among a plurality of counseling knowledge information stored in the knowledge base, in which the association score for the counseling sentence is greater than or equal to a preset threshold value, to the counselor account of the counselor currently conducting counseling with the customer, based on the association score calculated by reflecting the joint appearance frequency and individual appearance probability in the interdependence model (305).

[0114] According to one embodiment, when the calculation of the joint occurrence frequency and the individual occurrence probability is completed, the selection knowledge information providing unit (303) can calculate an association score for each pair by reflecting the joint occurrence frequency and the individual occurrence probability in an interdependence model (305).

[0115] In relation to the above, the association score is a value converted into a probability-based statistical figure representing the semantic association between core keywords and counseling topics within a counseling sentence and multiple counseling knowledge information items, and may be a score calculated based on the degree of interdependence considering the probability of individual occurrence relative to the frequency of joint occurrence.

[0116] According to one embodiment, the selection knowledge information providing unit (303) can filter only the counseling knowledge information that exceeds a preset threshold value by comparing each calculated association score with a preset threshold value. At this time, the preset threshold value may be a fixed value or a user-set value within the system, and may be a reference value that is dynamically adjusted according to factors such as the importance of the counseling topic, the complaint generation rate, the counselor grade, or the recency of the knowledge information.

[0117] According to one embodiment, the selection knowledge information providing unit (303) may sort the filtered consultation knowledge information in descending order based on the association score or adjust the output order by applying a pre-set priority policy.

[0118] According to one embodiment, a pre-set priority policy is information that includes response urgency, usage frequency, recency, etc., as output criteria for filtered consultation knowledge information. Specifically, it may be information that serves as a standard to enable a counselor to quickly identify and utilize knowledge information suitable for real-time consultation situations by providing knowledge information related to topics with a high probability of customer dissatisfaction first, or by placing items frequently used in past consultations or recently updated information at the top.

[0119] According to one embodiment, when the sorting and priority application is completed, the selection knowledge information providing unit (303) can provide one or more finally selected consultation knowledge information in real time to the account interface of the counselor currently conducting the consultation, and the counselor can directly select one of the provided information to use in customer response or compare and refer to multiple items.

[0120] FIG. 4 is a block diagram illustrating the feedback learning section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to an embodiment of the present invention.

[0121] Referring to FIG. 4, a counselor and agent mutual teacher learning system through real-time counseling analysis (e.g., counselor and agent mutual teacher learning system through real-time counseling analysis of FIG. 1 (100)) (hereinafter referred to as the mutual teacher learning system) implemented as a computing device comprising one or more processors and one or more memories for storing instructions that can be executed by said processors may include a feedback learning unit (400) (e.g., feedback learning unit (107) of FIG. 1).

[0122] According to one embodiment, when the counselor provides counseling sentence summary information to the counselor as the counseling between the counselor and the customer ends, the feedback learning unit (400) receives feedback evaluation information (401a) regarding the counseling sentence summary information from the counselor, stores the feedback evaluation information (401a) in a learning database, and analyzes and learns the feedback evaluation information (401a) through a previously stored feedback learning algorithm (405), thereby enhancing the performance of the counseling sentence analysis summary unit (e.g., counseling sentence analysis summary unit (101) of FIG. 1) and the counseling knowledge information providing unit (e.g., counseling knowledge information providing unit (103) of FIG. 1).

[0123] According to one embodiment, the feedback learning unit (400) may include an evaluation information storage unit (401) and a learning pattern derivation reflection unit (403) as detailed configurations for performing the above-described function.

[0124] According to one embodiment, when the evaluation information storage unit (401) provides consultation sentence summary information to the counselor as the consultation between the counselor and the customer ends, it may receive feedback evaluation information regarding the consultation sentence summary information from the counselor and store the feedback evaluation information in a learning database.

[0125] According to one embodiment, the evaluation information storage unit (401) can provide the counselor with consultation sentence summary information generated by the function performance of the consultation sentence analysis summary unit after the consultation between the counselor and the customer has ended.

[0126] In relation to the above, when the evaluation information storage unit (401) receives feedback evaluation information regarding the counseling sentence summary information from a counselor, the received feedback evaluation information can be stored in a learning database.

[0127] According to one embodiment, the consultation sentence summary information is a summary sentence automatically generated during a real-time consultation or at the time of consultation termination, and may be information in the form of natural language text that concisely summarizes the core content of the consultation, such as the gist of the customer's request or the gist of the counselor's response.

[0128] For example, the consultation sentence summary information may include a sentence such as, "The customer inquired about a billing error, and the agent checked the details and explained."

[0129] According to one embodiment, the feedback evaluation information (401a) may be information that includes whether the summary sentence has been modified, the modified content, whether it has been accepted, optional evaluation items, evaluation scores, and descriptive opinions of the counselor, as a result of the counselor evaluating the consistency with the actual counseling content, clarity of expression, and appropriateness of the summary regarding the counseling sentence summary information provided to the counselor.

[0130] For example, if the original consultation sentence summary information provided to the counselor is provided as "The customer inquired about a billing error, and the counselor checked the details and explained," the feedback evaluation information (401a) generated as the counselor evaluates the consultation sentence summary information may include a modified sentence such as "The customer inquired about double billing of the March billing amount, and the counselor checked the details and informed whether a refund was possible," and the evaluation items may include "missing key details" and "unclear," and the evaluation score may include 3 points (out of 5). In addition, the above feedback evaluation information (401a) may also include a descriptive opinion written by the counselor, such as "Since the content inquired by the customer was not a simple error but double billing, I would like it to be expressed more clearly."

[0131] According to one embodiment, the evaluation information storage unit (401) can accumulate and store the feedback evaluation information (401a) entered by the counselor in a learning database, thereby enabling the feedback learning algorithm (405) to use it as learning data for improving the quality of the counseling summary.

[0132] According to one embodiment, when the learning pattern derivation reflection unit (403) completes the storage of the feedback evaluation information (401a) in the learning database, it can reflect the derived learning pattern value in the previously stored natural language processing algorithm and the previously stored correlation evaluation algorithm by analyzing the original sentence summary information, whether the sentence summary information has been modified, the modified content, whether it has been accepted, the selected item, the evaluation score, and the reason for feedback, which are detailed information included in the feedback evaluation information (401a), through the previously stored feedback learning algorithm (405).

[0133] According to one embodiment, the learning pattern derivation reflection unit (403) can derive a learning pattern value for improving consultation sentence summary information by analyzing a plurality of detailed information included in the feedback evaluation information (401a) through a previously stored feedback learning algorithm (405) when a pre-set learning start condition (e.g., accumulation of a certain amount of data or a set time) is satisfied while the feedback evaluation information (401a) is stored in a learning database.

[0134] More specifically, the learning pattern derivation reflection unit (403) can derive a learning pattern value for improving the quality of the counseling sentence summary by analyzing the correlation between the original, modification status and modification content, acceptance status, selectable evaluation items, evaluation score, and the counselor's free-form opinion of the counseling sentence summary information, which is detailed information included in the feedback evaluation information (401a), through the previously stored feedback learning algorithm (405).

[0135] In relation to the above, the learning pattern values ​​derived to improve the quality of counseling sentence summaries may include a modification frequency-based expression improvement pattern that identifies specific expressions or sentence structures that are repeatedly modified to avoid using such expressions or reflect them in an improved form when generating future summaries; a counselor-preferred expression pattern that analyzes the writing style or preferred vocabulary frequently used by counselors and reflects them in the sentence style of the natural language processing algorithm; and a summary composition criterion correction indicator that adjusts the threshold of the summary algorithm based on evaluation scores related to summary composition criteria such as summary length, keyword inclusion, and number of sentences.

[0136] According to one embodiment, the learning pattern derivation reflection unit (403) can derive a learning pattern value for improving the accuracy of correlation judgment by analyzing the correlation between the counselor's acceptance of the original counseling summary sentence information, which is detailed information included in the feedback evaluation information (401a), the selection evaluation item, the evaluation score, and the counselor's descriptive opinion through the previously stored feedback learning algorithm (405).

[0137] In relation to the above, the learning pattern value for improving the accuracy of association judgment may be a learning pattern value for correcting the weights or filtering criteria applied when calculating the association score between core keywords and consultation topics and consultation knowledge information in the association evaluation algorithm.

[0138] More specifically, the learning pattern values ​​derived to improve the accuracy of relevance judgment may include: a relevance score adjustment pattern applied to lower the relevance score or reduce the recommendation priority for a specific keyword / topic combination based on inappropriate cases between knowledge information items that counselors have repeatedly given low ratings to or rejected; a knowledge information preference-based weight correction pattern that identifies knowledge items that counselors consistently prefer or avoid regarding a specific counseling topic or keyword and reflects this in the weighting method; and a knowledge information filtering criterion adjustment indicator that strengthens or relaxes the knowledge information filtering criteria according to the distribution of evaluation scores or the cumulative frequency of selected negative evaluation items.

[0139] According to one embodiment, the learning pattern derivation reflection unit (403) can continuously improve the counseling sentence summary and knowledge information recommendation functions of the peer teacher learning system by reflecting each of the learning patterns derived as above into a stored natural language processing algorithm and a stored association evaluation algorithm.

[0140] More specifically, the stored natural language processing algorithm, whose function is enhanced by the learning pattern derivation reflection unit (403) above, can be optimized to avoid expressions that are repeatedly modified or avoided based on the counselor's feedback, and to generate natural and highly consistent counseling sentence summary information by reflecting the writing style or vocabulary preferred by the counselor.

[0141] In addition, the previously stored correlation evaluation algorithm, whose function is enhanced by the learning pattern derivation reflection unit (403), can improve the topic suitability and response effectiveness of the counseling knowledge information provided to the counselor by dynamically adjusting the correlation score between the core keyword and counseling topic and counseling knowledge information or correcting the filtering criteria through feedback-based learning.

[0142] Accordingly, the present invention can continuously improve the accuracy and reliability of counseling automation by realizing a cyclic learning structure in which the system improves its performance based on the actual usage experience of counselors.

[0143] FIG. 5 is a block diagram illustrating the trend analysis section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to an embodiment of the present invention.

[0144] Referring to FIG. 5, a counselor and agent mutual teacher learning system through real-time counseling analysis (e.g., counselor and agent mutual teacher learning system through real-time counseling analysis of FIG. 1 (100)) (hereinafter referred to as the mutual teacher learning system) implemented as a computing device comprising one or more processors and one or more memories that store instructions that can be executed by said processors may further include a trend analysis unit (500).

[0145] According to one embodiment, the trend analysis unit (500) can generate trend analysis result information including trend trend information, rapidly rising keywords, expanded interest group information, and issue topic history information generated based on the result of analyzing multiple consultation sentence summary information accumulated and stored in the summary database through a previously stored trend analysis algorithm when a pre-set trend analysis condition is satisfied as multiple consultation sentence summary information is stored in the summary database.

[0146] According to one embodiment, the trend analysis unit (500) may include a time series information generation unit (501) and a trend trend generation unit (503) as detailed configurations for performing the above-described function.

[0147] According to one embodiment, the time series information generation unit (501) can generate time series information (501a) that aggregates the number of times a topic appears per unit period by classifying the multiple consultation sentence summary information by identifying a consultation topic included in each of the multiple consultation sentence summary information stored in the summary database, identifying a consultation type category among the pre-set consultation type categories that has the same identification value as the identified consultation topic, and classifying the multiple consultation sentence summary information.

[0148] According to one embodiment, the time series information generation unit (501) identifies a counseling topic included in each of the plurality of counseling sentence summary information stored in the summary database, and matches a counseling type that matches the identified counseling topic among the pre-set counseling type categories to each of the plurality of counseling sentence summary information, thereby classifying the plurality of counseling sentence summary information by counseling type.

[0149] According to one embodiment, the time series information generation unit (501) aggregates consultation sentence summary information classified by consultation type based on a unit period (e.g., one day, one week, one month, etc.) and can calculate the frequency of appearance of consultation sentence summary information for each consultation type during the unit period. Accordingly, the time series information generation unit (501) can generate time series information (501a) representing the appearance pattern of consultation sentence summary information according to the passage of time for each consultation type.

[0150] At this time, the time series information (501a) is information configured to quantify and arrange the number of occurrences of consultation sentence summary information for each unit period or to compare and analyze the frequency of occurrence of multiple consultation types, and can subsequently be used as input data for detecting an increasing trend, decreasing trend, recurring occurrence pattern, and abnormal fluctuation in the consultation volume in the trend trend generation unit (503).

[0151] According to another embodiment, the time series information generation unit (501) may, as needed, analyze complex appearance relationships between multiple consultation types as well as a single consultation type, or generate time series information subdivided based on specific keywords.

[0152] According to one embodiment, when the generation of the time series information (501a) is completed, the trend generation unit (503) can generate trend information that can identify the increasing trend, decreasing trend, recurring occurrence pattern, and abnormal fluctuation of the consultation volume by consultation topic by analyzing the time series information through a previously stored trend analysis algorithm.

[0153] According to one embodiment, the trend generation unit (503) can quantitatively evaluate the pattern of change in consultation volume by consultation topic by analyzing the time series information (%01a) through a previously stored trend analysis algorithm based on the time series information (501a).

[0154] According to one embodiment, a previously stored trend analysis algorithm may include a plurality of analysis models to generate trend analysis result information based on consultation sentence summary information, and the plurality of analysis models may include a trend trend generation model, a surge keyword identification model, an expanded interest group generation model, and an sentiment quality-based issue topic detection model, and each model may be configured to generate trend analysis result information independently or in combination.

[0155] In relation to the above, the trend generation model is a model that constructs time-series data based on the frequency of appearance of consultation topics or core keywords included in consultation sentence summary information, and analyzes this data to identify increasing or decreasing trends, recurring patterns, and abnormal fluctuations in consultation volume over time; it may be a model utilized to detect long-term increasing trends in consultation issues, seasonal issue occurrence cycles, and unexpected sudden changes.

[0156] In addition, the core keyword identification model analyzes fluctuations in the frequency of occurrence of each core keyword across time intervals and performs the function of identifying a specific keyword as a surge keyword if its frequency of occurrence in the current time interval increases by more than a preset threshold ratio compared to the previous time interval; this model can be utilized to detect the possibility of urgent issues or rapid changes in customer interest at an early stage.

[0157] In addition, the extended interest group generation model is a model that calculates association scores with multiple core keywords identified in past consultation history based on core keywords identified in consultation sentences, and groups keywords with association scores exceeding a pre-set threshold to generate extended interest group information, which is a set of semantically close keywords; it may be a model utilized for the extended detection of derived issues or interest clusters related to a specific topic.

[0158] Finally, the sentiment quality-based issue topic detection model analyzes the sentiment quality index calculated from consultation sentence summary information on a time interval basis, and performs the function of identifying the relevant consultation topic as an issue topic with a high risk of complaint if the sentiment quality index falls below a standard for a specified period or if the ratio of negative sentiment dominance rises abnormally; this model may be utilized to detect signs of escalating customer dissatisfaction early and to preemptively block complaint risks.

[0159] According to one embodiment, the trend information is information that includes the result of quantitatively analyzing the pattern of fluctuation in the volume of consultation for a specific consultation topic based on time series information (501a) generated from consultation sentence summary information, and may include information that includes an increasing trend, a decreasing trend, a pattern of repeated occurrence, and whether there is an abnormal fluctuation in the volume of consultation by consultation topic.

[0160] In relation to the above, trend information may include increasing trend information (information quantifying the tendency of the volume of consultations regarding a specific consultation topic to continuously rise over time), decreasing trend information (information quantifying the tendency of the volume of consultations regarding a specific consultation topic to continuously fall over time), recurring occurrence pattern information (information indicating a tendency for the volume of consultations regarding a specific consultation topic to repeatedly increase or decrease at similar times during specific cycles (e.g., end of month, end of quarter, specific season, etc.), and abnormal fluctuation information (information including a history of abnormal surges or surges in the volume of consultations exceeding the existing fluctuation range).

[0161] More specifically, the trend generation unit (503) can analyze the time series information (501a) through a previously stored trend analysis algorithm to identify an increasing trend when the volume of consultation for a specific consultation topic continuously increases over time, and conversely, identify a decreasing trend when the volume of consultation continuously decreases.

[0162] In relation to the above, the increasing trend may be a pattern in which the volume of consultations on a specific topic tends to continuously rise over time when analyzing consultation volume data by unit period (e.g., day, week, month).

[0163] In addition, the decreasing trend may be a pattern in the consultation volume data by unit period in which the volume of consultations for a specific topic tends to continuously decrease over time.

[0164] According to one embodiment, the trend generation unit (503) can analyze the time series information (501a) through a previously stored trend analysis algorithm and identify a recurring pattern if the volume of consultations tends to increase or decrease repeatedly at similar times with a specific periodicity (e.g., end of month, end of quarter, season).

[0165] In relation to the above, the recurring pattern is a pattern in which the volume of consultations regarding a specific consultation topic tends to increase or decrease with a regular cycle, and may include, for example, a pattern in which the volume of consultations repeatedly increases at the end of a month, the end of a quarter, or a specific season (e.g., holidays, year-end and New Year).

[0166] According to one embodiment, the trend generation unit (503) can analyze the time series information (501a) through a previously stored trend analysis algorithm and, if an abnormal situation is detected where the volume of consultations rapidly increases or decreases outside the normal fluctuation range or prediction range, identify that an abnormal fluctuation has occurred regarding the consultation topic.

[0167] In relation to the above, anomaly detection involves detecting phenomena where the volume of consultations for a specific topic increases or decreases abnormally compared to normal appearance patterns; for example, it can be detected when consultations related to "connection errors" increase sharply on a specific day due to an unexpected large-scale service failure.

[0168] According to one embodiment, when the detection of the abnormal fluctuation is completed, the trend generation unit (503) can generate early warning data, such as the occurrence of a new issue or an increase in abnormal customer inquiries, through the detection results. In this regard, the early warning data can be provided to an administrator so that the administrator can respond to the abnormal fluctuation.

[0169] Accordingly, the above trend trend generation unit (503) can generate trend trend information based on information such as the trend of changes in consultation volume by consultation topic, periodicity, and abnormality.

[0170] Accordingly, the trend analysis unit (500) may include the trend trend information as one of the detailed information included in the trend analysis result information.

[0171] FIG. 6 is another block diagram illustrating the trend analysis section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to one embodiment of the present invention.

[0172] Referring to FIG. 6, a counselor and agent mutual teacher learning system through real-time counseling analysis (e.g., counselor and agent mutual teacher learning system through real-time counseling analysis of FIG. 1 (100)) (hereinafter referred to as the mutual teacher learning system) implemented as a computing device comprising one or more processors and one or more memories for storing instructions that can be executed by said processors may further include a trend analysis unit (600) (e.g., trend analysis unit (500) of FIG. 1).

[0173] According to one embodiment, the trend analysis unit (600) can generate trend analysis result information including trend trend information, rapidly rising keywords, expanded interest group information, and issue topic history information generated based on the result of analyzing multiple consultation sentence summary information accumulated and stored in the summary database through a previously stored trend analysis algorithm when a pre-set trend analysis condition is satisfied as multiple consultation sentence summary information is stored in the summary database.

[0174] According to one embodiment, the trend analysis unit (600) may include, as detailed configurations for performing the above-described function, a change rate calculation unit for appearance frequency (601), a surge keyword identification unit (603), and a group information generation unit (605).

[0175] According to one embodiment, the occurrence frequency change rate calculation unit (601) identifies a core keyword (601a) included in each of the multiple consultation sentence summary information stored in the summary database, calculates the occurrence frequency of the core keyword (601a) for each preset time interval, and calculates the change rate of the occurrence frequency according to the time interval for each core keyword (601a) through the calculated occurrence frequency.

[0176] According to one embodiment, the occurrence frequency change rate calculation unit (601) can identify a core keyword (601a) included in each of the multiple consultation sentence summary information stored in the summary database, and then calculate the occurrence frequency of the core keyword (601a) for each preset time interval (e.g., one day, one week, one month, etc.).

[0177] According to one embodiment, the pre-set time interval applies the same criteria as the time interval mentioned in FIG. 5 (e.g., one day, one week, one month, etc.) and can be used as a criterion for quantitatively analyzing the frequency of appearance and fluctuation trends of the consultation topic and core keywords of the consultation sentence summary information on an hourly basis.

[0178] According to one embodiment, the occurrence frequency change rate calculation unit (601) can quantitatively analyze the fluctuation trend of the occurrence frequency over time by calculating the rate of increase or decrease in the occurrence frequency of each key keyword (601a) in the current time interval compared to the previous time interval, based on the occurrence frequency of the key keyword (601a) calculated for each time interval.

[0179] At this time, the rate of change in the frequency of appearance can be calculated by subtracting the frequency of appearance in the previous time interval from the frequency of appearance in the current time interval of the core keyword (601a) and dividing it based on the frequency of appearance in the previous time interval, thereby identifying the relative surge or decrease of the core keyword (601a).

[0180] According to one embodiment, when the function of the occurrence frequency change rate calculation unit (601) is completed, the surge keyword identification unit (603) can identify a core keyword as a surge keyword (607) in which the occurrence frequency in the current time interval increases by more than a preset occurrence threshold ratio compared to the average occurrence frequency in the previous time interval through the change rate of occurrence frequency according to the time interval for each identified core keyword.

[0181] More specifically, the above-mentioned surge keyword identification unit (603) compares the average frequency of occurrence in the previous time interval with the frequency of occurrence in the current time interval for the core keyword (601a), and if the frequency of occurrence in the current time interval increases by more than the pre-set frequency threshold ratio compared to the previous time interval, the core keyword can be identified as a surge keyword (607).

[0182] In relation to the above, the pre-set occurrence threshold ratio can be set according to system or administrator settings, such as 150% or 200%, and can serve as a criterion for selecting only abnormally rapid increase phenomena rather than simple increase in occurrence.

[0183] According to one embodiment, the surge keyword identification unit (603) can identify the time of surge occurrence, surge rate, and related consultation type as additional information for each identified surge keyword (607), and structure and generate information about the surge keyword (607) so that it can be used for creating expanded interest groups, managing issue topic history, issuing real-time alerts, etc.

[0184] That is, the above-mentioned surge keyword (607) may be a keyword that can be used as basic data for various application functions, such as detecting changes in consultation trends, identifying urgent response issues, and optimizing the distribution of consultation resources.

[0185] According to one embodiment, when the function of the surge keyword identification unit (603) is completed, the group information generation unit (605) calculates the co-occurrence frequency and individual occurrence probability for each combination through the result of preprocessing the core keyword identified in the consultation sentence and the past core keyword identified prior to the current time point using the interdependence model of the previously stored association evaluation algorithm (609), and then, based on the association score calculated by reflecting the co-occurrence frequency and individual occurrence probability in the interdependence model, identifies past core keywords among the past core keywords whose association score for the core keyword is greater than or equal to a pre-set association score, and groups the identified past core keywords and the core keywords to generate expanded interest group information (information used for detecting derived issues or expanding interest clusters).

[0186] According to one embodiment, when the function of the surge keyword identification unit (603) is completed, the group information generation unit (605) can analyze the core keywords through the interdependence model included in the previously stored association evaluation algorithm (609) to analyze the relationship between the core keywords at the current time and the core keywords identified in the past consultation history.

[0187] More specifically, the group information generation unit (605) can perform preprocessing on each of the core keywords identified in the consultation sentence at the current time (current core keywords) and the multiple core keywords stored in the past consultation database (past core keywords), and then calculate the co-occurrence frequency and individual occurrence probability of the preprocessed pair of current core keywords and past core keywords through the interdependence model.

[0188] Afterwards, the group information generation unit can calculate an association score through the joint occurrence frequency and individual occurrence probability of the calculated keyword pairs.

[0189] According to one embodiment, the group information generation unit (605) can select past core keywords included in keyword pairs whose calculated association score is greater than or equal to a preset association score standard, and group past core keywords that have high semantic similarity or thematic association with the core keywords at the current time.

[0190] For example, the group information generation unit (605) can calculate an association score by calculating the joint occurrence frequency and individual occurrence probability for keyword pairs for each of the multiple core keywords stored in the past consultation database, while the current core keyword "delivery delay" is identified in the current consultation sentence.

[0191] At this time, the group information generation unit (605) can generate extended interest group information by identifying past core keywords included in keyword pairs where the calculated association score is greater than or equal to a preset association score standard, such as "delivery cancellation," "delivery change request," and "courier company issue," and grouping the identified past core keywords into extended interest groups for the current core keywords.

[0192] According to one embodiment, a grouped set of past core keywords whose association score for current core keywords is equal to or greater than a preset association score standard is managed as expanded interest group information and can be utilized as strategic data in various application fields such as consultation trend analysis, derivative issue detection, expansion of new issue clusters, and proactive response to customer complaints.

[0193] In other words, the aforementioned expanded interest group information is the result of selecting and grouping keywords whose relevance scores exceed a pre-set standard among multiple past core keywords identified in past consultation history and core keywords identified in current consultation sentences. It is information composed of a single set of various keywords that are semantically close or related to a specific consultation topic or customer issue. By including not only keywords directly connected to the current consultation issue but also derived keywords semantically connected through the analysis of past consultation patterns, it can be utilized to manage related problem clusters in an integrated manner rather than being limited to a single issue; to predict potential derived issues that are not directly apparent through groups of keywords highly related to the current consultation topic; or to support flexible cluster management that reflects changing customer issue trends by adding new keywords to existing interest clusters by consultation type or expanding cluster boundaries. Furthermore, it can be utilized for various business decisions, such as establishing real-time consultation response strategies, sending advance notifications, optimizing consultation resource allocation, developing new products, and improving policies.

[0194] Accordingly, the trend analysis unit (600) may include the surge keyword and the expanded interest group information as one of the details included in the trend analysis result information.

[0195] FIG. 7 is another block diagram illustrating the trend analysis section of a counselor and agent mutual teacher learning system through real-time counseling analysis according to one embodiment of the present invention.

[0196] Referring to FIG. 7, a counselor and agent mutual teacher learning system through real-time counseling analysis (e.g., counselor and agent mutual teacher learning system through real-time counseling analysis of FIG. 1 (100)) (hereinafter referred to as the mutual teacher learning system) implemented as a computing device comprising one or more processors and one or more memories for storing instructions that can be executed by said processors may further include a trend analysis unit (700) (e.g., trend analysis unit (500) of FIG. 1).

[0197] According to one embodiment, the trend analysis unit (700) can generate trend analysis result information including trend trend information, rapidly rising keywords, expanded interest group information, and issue topic history information generated based on the result of analyzing multiple consultation sentence summary information accumulated and stored in the summary database through a previously stored trend analysis algorithm when a pre-set trend analysis condition is satisfied as multiple consultation sentence summary information is stored in the summary database.

[0198] According to one embodiment, the trend analysis unit (700) may include, as detailed configurations for performing the above-described function, an emotion intensity calculation unit (701), an index grouping unit (703), and an issue topic information generation unit (705).

[0199] According to one embodiment, the emotion intensity calculation unit (701) can analyze multiple counseling sentence summary information stored in a summary database through a previously stored sentiment analysis algorithm (707), identify positive keywords and negative keywords for counseling summary sentences corresponding to each of the multiple counseling sentence summary information, and calculate the emotion intensity through each of the identified positive keywords and negative keywords.

[0200] According to one embodiment, the emotion intensity calculation unit (701) can identify positive keywords and negative keywords included in the counseling summary sentences corresponding to each counseling sentence summary information through the result of analyzing a plurality of counseling sentence summary information stored in a summary database using a previously stored sentiment analysis algorithm (707).

[0201] More specifically, the emotion intensity calculation unit (701) receives text within the consultation summary sentence as input through a previously stored sentiment analysis algorithm (707) and starts analysis, and can identify positive keywords and negative keywords included in the consultation summary sentence based on a predefined list of positive keywords (e.g., "satisfaction," "fast," "gratitude") and a list of negative keywords (e.g., "dissatisfaction," "delay," "refund").

[0202] In relation to the above, the emotion intensity calculation unit (701) can calculate the emotion intensity by analyzing each of the positive keyword and negative keyword through the previously stored sentiment analysis algorithm (707) and comprehensively considering the keyword occurrence frequency, location (whether it is emphasized within a sentence), and semantic intensity.

[0203] In relation to the above, emotional intensity may be a value that quantitatively calculates the strength of the positive or negative emotional expression represented by the counseling summary sentence based on the frequency of appearance of each positive and negative keyword included in the counseling summary sentence, their position within the sentence, the emphasis of the expression, and the emotional weight of the keyword itself.

[0204] For example, the above-mentioned emotional intensity calculation unit (701) can assign a high positive emotional intensity when the number of positive keywords in the counseling summary sentence is greater than the number of negative keywords and is placed in the beginning of the main sentence or in the emphasis expression, and conversely, can calculate a high negative emotional intensity when a large number of negative keywords appear or are used in a strong negative tone.

[0205] According to one embodiment, the emotional intensity calculation unit (701) can calculate an emotional intensity including positive emotional intensity and negative emotional intensity calculated for each of the consultation sentence summary information, thereby enabling it to be utilized in subsequent processing steps such as calculating an emotional quality index and evaluating the risk level of an issue topic.

[0206] According to one embodiment, when the calculation of the emotional intensity is completed, the index grouping unit (703) reflects the emotional intensity in the reliability classification model (709) of the previously stored sentiment analysis algorithm (707) to calculate an emotional quality index for the counseling summary sentence, identifies the counseling type of the counseling summary sentence corresponding to the calculated emotional quality index as a counseling topic, and groups the emotional quality index corresponding to each of the plurality of counseling sentence summary information to the identified counseling type.

[0207] According to one embodiment, when the calculation of the emotional intensity is completed, the index grouping unit (703) can calculate an emotion quality index for each counseling sentence summary information by reflecting the emotional intensity in the reliability classification model (709) included in the previously stored sentiment analysis algorithm (707).

[0208] According to one embodiment, the reliability classification model (709) included in the previously stored sentiment analysis algorithm may be a model that comprehensively analyzes the positive and negative sentiment intensities included in the counseling sentence summary information to determine the overall emotional nature (positive, negative, neutral) of the counseling summary sentence, and quantitatively evaluates the consistency, reliability, and quality level of the corresponding emotional expression to calculate an emotional quality index.

[0209] In relation to the above, the sentiment quality index may be a factor calculated by comprehensively considering the relative magnitude between positive and negative sentiment intensities, the frequency and location of major sentiment keywords (beginning of a sentence, whether emphasis phrases are included), and the concentration of negative sentiment keywords.

[0210] At this time, the value of the above-mentioned emotional quality index may be expressed as a continuous numerical value between 0 and 1, or classified into separately defined grades (e.g., Excellent, Good, Fair, Poor).

[0211] According to one embodiment, the index grouping unit (703) can additionally identify a consultation type (topic) corresponding to the consultation sentence summary information for each of the calculated emotional quality indices. In this regard, the consultation type may be classified into topics such as, for example, "fee inquiry," "cancellation request," "delivery delay," "refund request," "contract change," etc., and may be determined through the topic identification result included in the consultation sentence summary information or through a separate classification logic.

[0212] According to one embodiment, the index grouping unit (703) can group and store emotional quality indices by identified counseling topics, and can statistically classify emotional quality indices calculated for multiple counseling sentence summary information belonging to the same counseling topic into a single set.

[0213] Accordingly, the index grouping unit (703) can complete the grouping of the emotional quality index of at least one counseling sentence summary information belonging to the same counseling topic among the multiple counseling sentence summary information by grouping the emotional quality index corresponding to each of the multiple counseling sentence summary information.

[0214] According to another embodiment, the index grouping unit (703) can evaluate emotional quality variability by calculating the average emotional quality index for each counseling topic based on the emotional quality indices calculated for each counseling topic, and by calculating the variance or standard deviation of the emotional quality index.

[0215] In addition, the index grouping unit (703) can analyze whether positive or negative emotional expressions are dominant in a specific counseling topic by deriving the ratio of positive emotion dominance or the ratio of negative emotion dominance by counseling topic, and can identify in advance the possibility of counseling quality deterioration by deriving the ratio of counseling (emotion quality deterioration ratio) in which the emotional quality index is below a preset standard by counseling topic.

[0216] That is, the index grouping unit (703) can identify a specific counseling topic as a risk topic if a pattern is detected in which the emotional quality index for a specific counseling topic drops sharply for a certain period or the ratio of negative emotions rises abnormally. At this time, the identified risk topic can be transmitted to a real-time warning output unit (e.g., the real-time warning output unit (105) of FIG. 1) to output a real-time warning containing content that warns of the risk of a complaint in advance, or it can be included in the issue topic history information generated by the issue topic information generation unit (705) to be used to establish strategic response measures (e.g., modification of response manual, strengthening of counselor training, etc.) for improving counseling quality.

[0217] According to one embodiment, when the function of the index grouping unit (703) is completed, the issue topic information generation unit (705) identifies the rate of change of the emotional quality index in units of a preset time interval, and if the rate of change of the identified emotional quality index continues to be below the reference index for a predetermined period, it can determine a counseling topic based on a counseling type corresponding to the emotional quality index as an issue topic with a high probability of civil complaints occurring and generate issue topic history information.

[0218] According to one embodiment, the issue topic information generation unit (705) can monitor changes in the emotional quality index in units of a preset time interval (e.g., one day, one week, one month, etc.) based on the emotional quality index groups by counseling topic grouped by the function of the index grouping unit (703).

[0219] More specifically, the above-mentioned issue topic information generation unit (705) can evaluate whether the emotional quality of each counseling topic is improving, maintaining, or deteriorating through the result of calculating the rate of change of the emotional quality index corresponding to each counseling topic over time intervals.

[0220] In relation to the above, the issue topic information generation unit (705) may determine that the consultation topic is an issue topic with a high risk of causing complaints if the rate of change of the emotional quality index for each consultation topic remains below a preset standard index (e.g., a decrease of 10% or more in the emotional quality index compared to the previous period) for a predetermined period (e.g., 7 consecutive days).

[0221] According to one embodiment, the issue topic information generation unit (705) can generate issue topic history information for each consultation topic based on the judgment result as described above. In this regard, the issue topic history information may include detailed information such as the history of changes in the sentiment quality index, the history of increases in the negative sentiment ratio, the time of first detection of the risk topic, the duration, and whether improvement was made.

[0222] Accordingly, the above trend analysis unit (700) can include the above issue topic history information as one of the details included in the trend analysis result information.

[0223] FIG. 8 is a drawing for explaining an example of the internal configuration of a computing device according to an embodiment of the present invention.

[0224] FIG. 8 illustrates an example of the internal configuration of a computing device according to an embodiment of the present invention. In the following description, descriptions of unnecessary embodiments that overlap with the descriptions of FIG. 1 to 7 described above will be omitted.

[0225] As illustrated in FIG. 8, the computing device (10000) may include at least one processor (11100), memory (11200), peripheral interface (11300), input / output subsystem (11400), power circuit (11500), and communication circuit (11600). In this case, the computing device (10000) may correspond to a user terminal (A) connected to a haptic interface device or the aforementioned computing device (B).

[0226] The memory (11200) may include, for example, high-speed random access memory, a magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. The memory (11200) may include software modules, instruction sets, or various other data required for the operation of the computing device (10000).

[0227] At this time, access to memory (11200) from other components, such as the processor (11100) or peripheral device interface (11300), can be controlled by the processor (11100).

[0228] The peripheral device interface (11300) can connect input and / or output peripheral devices of the computing device (10000) to the processor (11100) and memory (11200). The processor (11100) can perform various functions for the computing device (10000) and process data by executing software modules or instruction sets stored in the memory (11200).

[0229] The input / output subsystem (11400) can connect various input / output peripherals to the peripheral interface (11300). For example, the input / output subsystem (11400) may include a controller for connecting peripherals such as a monitor, keyboard, mouse, printer, or, if necessary, a touchscreen or sensor to the peripheral interface (11300). According to another aspect, input / output peripherals may be connected to the peripheral interface (11300) without passing through the input / output subsystem (11400).

[0230] The power circuit (11500) can supply power to all or part of the components of the terminal. For example, the power circuit (11500) may include one or more power sources such as a power management system, a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or any other components for power generation, management, and distribution.

[0231] The communication circuit (11600) can enable communication with another computing device using at least one external port.

[0232] Alternatively, as described above, the communication circuit (11600) may enable communication with other computing devices by including an RF circuit and transmitting and receiving an RF signal, also known as an electromagnetic signal.

[0233] The embodiment of FIG. 8 is merely an example of a computing device (10000), and the computing device (11000) may have some components shown in FIG. 8 omitted, additional components not shown in FIG. 8 added, or a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may include, in addition to the components shown in FIG. 8, a touchscreen or a sensor, etc., and the communication circuit (1160) may include a circuit for RF communication of various communication methods (WiFi, 3G, LTE, Bluetooth, NFC, Zigbee, etc.). The components that can be included in the computing device (10000) may be implemented as hardware, software, or a combination of both hardware and software, including one or more integrated circuits specialized for signal processing or applications.

[0234] Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. An application to which the present invention is applied may be installed on a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file upon a request from the user terminal.

[0235] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0236] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed across networked computing devices and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0237] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0238] Although the embodiments have been described above with reference to limited embodiments and drawings, those skilled in the art can make various modifications and variations from the description above. For example, appropriate results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents. Therefore, other implementations, other embodiments, and equivalents to the claims below also fall within the scope of the claims.

Claims

1. A mutual teacher learning system between counselors and agents through real-time counseling analysis implemented by a computing device comprising one or more processors and one or more memories storing instructions executable by said processors, A consultation sentence analysis summary unit that, upon receiving real-time consultation information generated as a consultation between a counselor and a customer proceeds, preprocesses the real-time consultation information using a stored natural language processing algorithm to identify key keywords and consultation topics included in the identified consultation sentences, and generates consultation sentence summary information that summarizes the consultation sentences; When the function of the above-mentioned consultation sentence analysis summary unit is completed, a consultation knowledge information providing unit identifies consultation knowledge information with a high correlation score for the above-mentioned consultation sentence from a pre-stored knowledge base based on the result of analyzing the above-mentioned core keyword and the above-mentioned consultation topic through a previously stored correlation evaluation algorithm, and provides this information to the counselor; A real-time warning output unit that outputs a real-time warning for the identified keyword when a keyword satisfying a preset risk condition is identified among a plurality of keywords included in the consultation sentence while the function of the consultation knowledge information providing unit is being performed; and A counselor and agent mutual teaching learning system through real-time counseling analysis, characterized by including: a feedback learning unit that, when counseling sentence summary information is provided to the counselor upon termination of counseling between the counselor and the customer, receives feedback evaluation information regarding said counseling sentence summary information from the counselor, stores said feedback evaluation information in a learning database, and analyzes and learns said feedback evaluation information through a stored feedback learning algorithm to enhance the performance of said counseling sentence analysis summary unit and said counseling knowledge information provision unit.

2. In Paragraph 1, The summary of the analysis of the consultation sentences above is, A vector value calculation unit that, upon receiving real-time consultation information generated as a consultation between a counselor and a customer proceeds, preprocesses the real-time consultation information using a stored natural language processing algorithm to identify a consultation sentence corresponding to the real-time consultation information, performs a tokenization process on the consultation sentence to tokenize the consultation sentence, performs a vectorizing process on the tokens to calculate a vector value for each token quantified based on the position of each token in the consultation sentence, and calculates a sentence vector value for the consultation sentence; A core keyword identification unit that, when the calculation of the above sentence vector value is completed, reflects the above sentence vector value in an importance judgment model, which is a partial model included in the above-stored natural language processing algorithm, calculates an importance score between keywords included in the above consultation sentence, and identifies core keywords expressing the customer's main interests in the above consultation sentence; and A counselor and agent mutual teacher learning system through real-time counseling analysis, characterized by including: a counseling topic identification unit that, when the function of the core keyword identification unit is completed, reflects the sentence vector value to a topic classification model, which is a partial model included in the previously stored natural language processing algorithm, to identify a counseling topic regarding the counseling conducted between a counselor and a customer reflected in the counseling sentence.

3. In Paragraph 2, The summary of the analysis of the consultation sentences above is, A counselor and agent mutual teacher learning system through real-time counseling analysis, characterized by, when the generation of the sentence vector value is completed, reflecting the sentence vector value in a summary model which is part of a model included in the previously stored natural language processing algorithm, so that the summary model generates counseling sentence summary information that summarizes the core information of the counseling sentence based on the result of determining the contextual importance of the counseling sentence.

4. In Paragraph 1, The above consultation knowledge information provider is, When the function of the above consultation sentence analysis summary unit is completed, an occurrence frequency analysis unit that calculates the joint occurrence frequency and individual occurrence probability for each combination through the result of preprocessing the core keywords and consultation topics identified in the above consultation sentence and multiple consultation knowledge information stored in the knowledge base using the interdependence model of the previously stored association evaluation algorithm; and A counselor and agent mutual teaching learning system through real-time counseling analysis, characterized by including: a selection knowledge information providing unit that, when the calculation of the above-mentioned joint occurrence frequency and individual occurrence probability is completed, provides at least one counseling knowledge information among a plurality of counseling knowledge information stored in the knowledge base, wherein the association score for the above-mentioned counseling sentence is greater than or equal to a preset threshold value, to the counselor account of a counselor currently conducting counseling with a customer, based on the association score calculated by reflecting the above-mentioned joint occurrence frequency and individual occurrence probability in the above-mentioned interdependence model.

5. In Paragraph 1, The above real-time warning output unit is, A counselor and agent mutual teaching learning system through real-time counseling analysis, characterized by identifying multiple keywords included in a counseling sentence through a stored natural language processing algorithm while the function of the counseling knowledge information provision unit is performed, checking whether they are identical to a reference keyword included in each of the multiple risk categories based on the pre-set risk conditions, and if at least one of the multiple keywords is identical to the reference keyword, outputting a warning message matched to the reference keyword to the counselor in real time.

6. In Paragraph 1, The above feedback learning unit is, An evaluation information storage unit that, when consultation sentence summary information is provided to the counselor as consultation between the counselor and the customer ends, receives feedback evaluation information regarding said consultation sentence summary information from the counselor and stores said feedback evaluation information in a learning database; and A counselor and agent mutual teacher learning system through real-time counseling analysis, characterized by including: a learning pattern derivation reflection unit that, when the storage of the feedback evaluation information in the above-mentioned learning database is completed, analyzes the original text of the sentence summary information, whether the sentence summary information has been modified, the content of the modification, whether it has been accepted, the selected items, the evaluation score, and the reason for the feedback, which are detailed information included in the above-mentioned feedback evaluation information, through the above-mentioned stored feedback learning algorithm, and reflects the derived learning pattern value to the above-mentioned natural language processing algorithm and the above-mentioned correlation evaluation algorithm.

7. In Paragraph 1, The counselor and agent mutual teaching learning system through the above-mentioned real-time consultation analysis is, A counselor and agent mutual teaching system through real-time counseling analysis, further comprising: a trend analysis unit that generates trend analysis result information including trend trend information, rapidly rising keywords, expanded interest group information, and issue topic history information generated based on the result of analyzing multiple counseling sentence summary information accumulated and stored in the summary database through a stored trend analysis algorithm when a pre-set trend analysis condition is satisfied as multiple counseling sentence summary information is stored in the summary database.

8. In Paragraph 7, The above trend analysis department, A time series information generation unit that identifies a counseling topic included in each of the multiple counseling sentence summary information stored in the summary database, identifies a counseling type category having the same identification value as the identified counseling topic among the preset counseling type categories, classifies the multiple counseling sentence summary information, and generates time series information that aggregates the number of occurrences of topics per unit period; and A counselor and agent mutual teaching learning system through real-time counseling analysis, characterized by including: a trend trend generation unit that, when the generation of the above time series information is completed, analyzes the above time series information through a stored trend analysis algorithm to generate trend trend information capable of identifying the increasing trend, decreasing trend, recurring occurrence pattern, and abnormal fluctuation of counseling volume by counseling topic.

9. In Paragraph 7, The above trend analysis department, A frequency change rate calculation unit that identifies key keywords included in each of the multiple consultation sentence summary information stored in the summary database, calculates the frequency of occurrence of the key keywords for each preset time interval, and calculates the rate of change in frequency of occurrence according to the time interval for each key keyword through the calculated frequency of occurrence; When the function of the above-mentioned occurrence frequency change rate calculation unit is completed, a surge keyword identification unit identifies as a surge keyword a core keyword whose occurrence frequency in the current time interval increases rapidly by more than a preset occurrence threshold ratio compared to the average occurrence frequency in the previous time interval through the change rate of occurrence frequency according to the time interval for each identified core keyword; and A counselor and agent mutual teacher learning system through real-time counseling analysis, further comprising: a group information generation unit that, when the function of the above-mentioned surge keyword identification unit is completed, calculates the co-occurrence frequency and individual occurrence probability for each combination through the result of preprocessing the core keyword identified in the counseling sentence and the past core keyword identified prior to the current point in time using the interdependence model of the above-mentioned association evaluation algorithm, and then, based on the association score calculated by reflecting the co-occurrence frequency and individual occurrence probability in the above-mentioned interdependence model, identifies past core keywords among the above-mentioned core keywords whose association score for the core keyword is greater than or equal to a preset association score, and groups the identified past core keywords and the above-mentioned core keywords to generate expanded interest group information.

10. In Paragraph 7, The above trend analysis department, An emotional intensity calculation unit that analyzes multiple counseling sentence summary information stored in the summary database using a previously stored sentiment analysis algorithm to identify positive keywords and negative keywords for counseling summary sentences corresponding to each of the multiple counseling sentence summary information, and calculates an emotional intensity through each of the identified positive keywords and negative keywords; When the calculation of the above emotional intensity is completed, an index grouping unit that reflects the above emotional intensity in the reliability classification model of the previously stored sentiment analysis algorithm to calculate an emotional quality index for the above counseling summary sentence, identifies the counseling type of the counseling summary sentence corresponding to the calculated emotional quality index as a counseling topic, and groups the emotional quality index corresponding to each of the plurality of counseling sentence summary information to the identified counseling type; and A counselor and agent mutual teaching learning system through real-time counseling analysis, characterized by including: an issue topic information generation unit that, when the function of the index grouping unit is completed, identifies the rate of change of the emotional quality index in units of preset time intervals, and if the identified rate of change of the emotional quality index remains below a reference index for a predetermined period, determines a counseling topic based on a counseling type corresponding to the emotional quality index as an issue topic with a high probability of civil complaints occurring, and generates issue topic history information.