Apparatus for AI-based Data Analysis and Driving Method Thereof

Abstract

The present invention relates to an AI-based data analysis device and a method for operating the device. An AI-based data analysis device according to an embodiment of the present invention may include a communication interface unit that communicates with a client terminal device using a messenger-based chat service, and a control unit that divides collected social data into sentence units, has a generative artificial intelligence model interpret the meaning of each divided sentence to classify each sentence into one or more of an analysis target, a topic, and a sentiment according to predefined classification criteria, re-aggregates the classification results using a rule-based conditional expression to integrate data including non-standard expressions into the same semantic group, embeds the classification results into sentence units and stores them in a vector store, concurrently stores the classified and aggregated statistical data in a relational database, analyzes a natural language query received from the client terminal device to simultaneously query the vector store and the relational database, and combines the retrieved sentence embedding results and the statistical re-aggregation results to generate an analysis result in a natural language form that has improved accuracy and does not cause hallucinations, and provides it to the client terminal device as a response to the query.

Description

Technology Field

[0001] The present invention relates to an AI-based data analysis device and a method for operating the device, and more specifically, to an AI-based data analysis device and a method for operating the device that refines data collected from all channels, such as social media, e-commerce, online communities, and news sites, and enables generative artificial intelligence to understand context to automatically detect and predict key issues, purchase intentions, and public opinion trends for each post, and provides multi-purpose B2B (business to business) insights, such as consumer analysis, public opinion analysis, and market trend analysis, through conversational question-and-answer. Background Technology

[0002] Recently, companies and research institutions have been collecting and analyzing social data generated on online platforms—such as reviews, comments, community posts, and social media reactions—to identify customer perceptions, brand reputation, and social trends. Since this social data consists of large-scale unstructured text, automated analysis using Natural Language Processing (NLP) technology is typically performed. Conventional data analysis systems have primarily conducted keyword-based sentiment analysis or utilized pretrained language models (LLMs) such as BERT and KoBERT to perform document-level positive / negative judgments and extract key topics.

[0003] However, this approach reveals many problems. Existing keyword-based models or BERT-based models fail to fully understand the contextual meaning of a sentence. For instance, in the sentence "This product has a pretty design but lacks performance," the model must simultaneously distinguish between a positive view of 'design' and a negative view of 'performance'; however, word-unit-based models frequently misclassify because they fail to recognize the interrelationships within the entire sentence. Furthermore, expressions containing slang or non-standard language (e.g., "Samsung," "Samjeon") are not accurately recognized by the model, leading to omissions in data aggregation.

[0004] In addition, since existing models must be pre-trained to specific domains (e.g., home appliances, finance, games), they must be retrained or fine-tuned whenever new fields or keywords emerge. This leads to problems such as high costs for data construction and training, and difficulties in managing models on a project-by-project basis.

[0005] Furthermore, recently, attempts have been made to query data in natural language and obtain analysis results by utilizing generative AI (such as ChatGPT). However, existing question-answering systems rely on a single storage structure, such as performing semantic search using only a vector store or processing statistical queries using only a relational database. In this case, there are problems such as retrieving only a portion of the data or hallucinations occurring because the model generates excessive context. In particular, when large-scale text data is directly vector-embedded, there is a limitation in that the number of tokens increases excessively, leading to decreased response accuracy and skyrocketing costs. Prior art literature

[0006] Korean Patent Publication No. 10-2856954 (2025.09.03) Korean Patent Publication No. 10-2704192 (2024.09.03) The problem to be solved

[0007] The purpose of the embodiments of the present invention is to provide an AI-based data analysis device and a method for operating the device, which refine data collected from all channels, such as social media, e-commerce, online communities, and news sites, and enable generative artificial intelligence to understand the context to automatically detect and predict key issues, purchase intentions, and public opinion trends for each post, and provide multi-purpose B2B insights, such as consumer analysis, public opinion analysis, and market trend analysis, through conversational question-and-answer.

[0008] In addition, another objective of the embodiments of the present invention is to provide an AI-based data analysis device and a method for operating the device, which provide a data analysis technology capable of high-accuracy classification reflecting context by subdividing unstructured social data into sentence units, interpreting the meaning of each sentence using generative artificial intelligence, and then classifying and re-aggregating according to criteria such as analysis target, topic, and sentiment.

[0009] Furthermore, another objective of the embodiments of the present invention is to provide an AI-based data analysis device and a method for operating the device, which constitutes a hallucination-suppressing question-answering system that derives accurate and reliable analysis results by parallel querying of two stores (e.g., RAG structure) for a user's natural language query, by storing data embedded in sentence units in a vector store and classifying and aggregating results in parallel in a relational database.

[0010] Furthermore, another objective of the embodiments of the present invention is to provide an AI-based data analysis device and a method for operating the device, which provide an AI-Rule hybrid data processing structure capable of maintaining data quality and reducing inspection costs by correcting and integrating non-standard expressions with rule-based conditional expressions to prevent misclassification caused by the free output characteristics of generative artificial intelligence.

[0011] The problems of the present invention are not limited to those mentioned above, and other unmentioned problems will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0012] An AI-based data analysis device according to an embodiment of the present invention comprises: a communication interface unit that communicates with a client terminal device using a messenger-based chat service; and a control unit that divides collected social data into sentence units, in which a generative artificial intelligence model interprets the meaning of each divided sentence and classifies each sentence into one or more of an analysis target, a topic, and a sentiment according to a predefined classification criterion, re-aggregates the classification results using a rule-based conditional expression to integrate data including non-standard expressions into the same semantic group, embeds the classification results into sentence units and stores them in a vector store, and concurrently stores the classified and aggregated statistical data in a relational database, analyzes a natural language query received from the client terminal device to simultaneously query the vector store and the relational database, and combines the retrieved sentence embedding results and the statistical re-aggregation results to generate an analysis result based on semantic and quantitative grounds in the form of natural language and provides it to the client terminal device as a response to the query.

[0013] The control unit can recognize contextually pre-set entity names and sentiment words when the generative artificial intelligence model interprets the meaning of the entire sentence, classify them into one or more of an analysis target, a topic, and a sentiment, and generate metadata corresponding to the purpose of analysis of each sentence according to the classification result.

[0014] The control unit can re-aggregate non-standard expressions or modified terms among the classification results output by the generative artificial intelligence model into standardized word groups according to a predefined rule-based conditional expression, and generate statistical data based on the re-aggregation results.

[0015] The control unit executes a dual storage and search module to store the classified and re-aggregated data in a vector store in the form of sentence-unit embeddings expressed in a multidimensional vector structure reflecting sentence meaning, and simultaneously stores the statistical data in a relational database so that the vector store and the relational database can be queried simultaneously during a query.

[0016] The control unit can analyze the meaning of a natural language query input from the customer terminal device to automatically extract query conditions and statistical fields, query the vector store and the relational database based on the extracted conditions, combine the retrieved results, exclude data with a reliability level below a standard among the combined results to prevent hallucinations, and generate a final analysis result.

[0017] In addition, the method of operating an AI-based data analysis device according to an embodiment of the present invention comprises the steps of: a communication interface unit communicating with a client terminal device using a messenger-based chat service; and a control unit dividing collected social data into sentence units, a generative artificial intelligence model interpreting the meaning of each divided sentence and classifying each sentence into one or more of an analysis target, a topic, and a sentiment according to a predefined classification criterion, re-aggregating the classification results into a rule-based conditional expression to integrate data including non-standard expressions into the same semantic group, embedding the classification results into sentence units and storing them in a vector store, and storing the classified and aggregated statistical data in parallel in a relational database, analyzing a natural language query received from the client terminal device to simultaneously query the vector store and the relational database, and combining the retrieved sentence embedding results and the statistical re-aggregation results to generate an analysis result based on semantic and quantitative grounds in the form of natural language and providing it to the client terminal device as a response to the query.

[0018] The step of classifying one or more of the above can be performed by the generative artificial intelligence model, when interpreting the meaning of the entire sentence, recognizing contextually pre-set entity names and sentiment words to classify into one or more of the analysis target, topic, and sentiment, and generating metadata corresponding to the analysis purpose of each sentence according to the classification result.

[0019] The step of integrating into the same semantic group described above may re-aggregate non-standard expressions or modified terms among the classification results output by the generative artificial intelligence model into standardized word groups according to a predefined rule-based conditional expression, and generate statistical data based on the re-aggregation results.

[0020] The above control unit may include the step of executing a dual storage and search module to store the classified and re-aggregated data in a vector store in the form of sentence-unit embeddings expressed in a multidimensional vector structure reflecting sentence meaning, and simultaneously storing the statistical data in a relational database in parallel, thereby querying the vector store and the relational database simultaneously during a query.

[0021] The step of generating the above analysis results in a natural language form may include: a step of automatically extracting query conditions and statistical fields by analyzing the meaning of a natural language query input from the customer terminal device; and a step of generating a final analysis result by querying the vector store and the relational database based on the extracted conditions, combining the retrieved results, and then excluding data with a reliability level below a standard from the combined results to prevent hallucinations. Effects of the invention

[0022] According to an embodiment of the present invention, a generative artificial intelligence model interprets the meaning of the entire sentence and classifies each sentence into an analysis target, topic, and sentiment, thereby eliminating context recognition errors that occurred in word-centered keyword-based analysis. Accordingly, even when positive and negative sentiments alternate within the same sentence, or when non-standard expressions such as slang or parody are included, the meaning is accurately grasped, and the classification accuracy is greatly improved.

[0023] Furthermore, according to an embodiment of the present invention, non-standard expressions or modified terms among the results generated in the form of free output by a generative AI are re-aggregated using rule-based conditional expressions, thereby integrating different expressions so that they can be recognized as the same entity. For example, since modified terms such as 'Samsung' and 'Samjeon' can be standardized to 'Samsung', the uncertainty of the AI ​​output can be corrected and data quality can be maintained consistently. As a result, the cost and time required for data cleaning and verification can be reduced by adjusting only project-specific rules without retraining the model.

[0024] According to an embodiment of the present invention, a Retrieval-Augmented Generation (RAG) structure is provided in which classified sentence data is stored in a vector store in the form of sentence-unit embeddings, and aggregated statistical data is stored in parallel in a relational database, thereby querying both stores simultaneously upon a user's natural language query. Accordingly, semantic search results and numerical data results can be combined simultaneously, thereby realizing the generation of objective and reliable responses based on data.

[0025] Furthermore, according to an embodiment of the present invention, data embedded in sentence units is stored in the vector store instead of the entire document, and only the necessary range is re-aggregated in the DB according to query conditions, thereby reducing the number of input tokens of the model by more than 90%. This effectively suppresses the occurrence of hallucinations caused by the model's context overload and enables the continuous generation of highly accurate response results. Consequently, the embodiment of the present invention structurally solves the reliability problem of generative artificial intelligence.

[0026] In the embodiment of the present invention, when a user queries in natural language through a messenger-based chat interface, the AI ​​analyzes data in real time and responds with results, allowing even non-experts to intuitively view data without writing complex queries. Therefore, the embodiment of the present invention can improve the efficiency of data analysis tasks for companies and institutions and be utilized to derive business insights based on large-scale social data.

[0027] The effects according to the present invention are not limited to those exemplified above, and various other effects are included in this specification. Brief explanation of the drawing

[0028] FIG. 1 is a diagram showing an AI-based data analysis system according to an embodiment of the present invention. FIGS. 2a to 2d are drawings for explaining the main functions of the AI-based data analysis device of FIG. 1. Figure 3 is a block diagram illustrating the detailed structure of the AI-based data analysis device of Figure 1. Figure 4a is a block diagram illustrating the detailed structure of the AI-based data analysis unit of Figure 3. Figure 4b is a diagram showing the operation and data flow of the generative AI classification unit and the AI ​​chat function unit of Figure 4a. Figure 5 is a flowchart showing the operation process of the AI-based data analysis device of Figure 1. Specific details for implementing the invention

[0029] The present invention is not limited to the embodiments described below but can be implemented in various different forms. These embodiments are merely illustrative of the content of the invention and are provided to inform those skilled in the art of the scope of the invention in detail. The present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0030] The embodiments described herein will be described with reference to cross-sectional and / or plan views, which are exemplary illustrations of the invention. In the drawings, the illustrated regions are depicted for the effective description of the technical content. Accordingly, the regions illustrated in the drawings are schematic in nature, and the shapes of the regions illustrated in the drawings are intended to illustrate specific forms of the device regions and are not intended to limit the scope of the invention. Although terms such as first, second, third, etc., have been used to describe various components in the various embodiments of this specification, these components should not be limited by such terms. These terms are used merely to distinguish one component from another. The embodiments described and illustrated herein also include their complementary embodiments.

[0031] The terms used in the specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used in the specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.

[0032] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0033] Hereinafter, the concept of the present invention and embodiments according thereto will be described in detail with reference to the drawings.

[0034] FIG. 1 is a drawing showing an AI-based data analysis system according to an embodiment of the present invention, and FIG. 2a to 2d are drawings for explaining the main functions of the AI-based data analysis device of FIG. 1.

[0035] As illustrated in FIG. 1, an AI-based data analysis system (90) according to an embodiment of the present invention is a system for providing social data classification and analysis services using, for example, generative AI, and includes some or all of a customer terminal device (100), a communication network (110), an AI-based data analysis device (120), and a third-party device (130).

[0036] Here, "including some or all" means that some components, such as a third-party device (130), may be omitted to configure the AI-based data analysis system (90) of FIG. 1, or that some or all components constituting the AI-based data analysis device (120) may be integrated into a network device (e.g., wireless switching device, gateway, etc.) constituting the communication network (110), and is described as including all to facilitate a sufficient understanding of the invention.

[0037] The customer terminal device (100) performs a natural language query using the chat service of the AI-based data analysis device (120) and performs the operation of receiving and displaying the analysis results. The customer terminal device (100) collects a natural language query entered by a user and transmits it to the AI-based data analysis device (120) via a network, and is configured to allow the user to view the data analysis results interactively by displaying the response data received from the data analysis device (120) on a screen or visualizing it in a summarized form. The user can input a query in the form of a general conversation sentence through a messenger-based chat service installed on the customer terminal device (100).

[0038] For example, when a query such as “How positive has the ‘Dreamy’ brand been over the past three months?” or “What are the major sentiment keywords related to product complaints?” is entered, the client terminal device (100) can convert the query sentence into a query format that can be recognized by the AI-based data analysis device (120) and transmit it. The AI-based data analysis device (120) can interpret the meaning of the query using an internal generative artificial intelligence model and generate response data by concurrently querying data stored in a vector store and a relational database. The response generated at this time may include a descriptive text in natural language or analysis result data such as graphs and statistics.

[0039] When the client terminal device (100) receives response data from the AI-based data analysis device (120), it immediately displays it on the user's screen and provides it in the form of a chat service where conversational responses are made. Based on the displayed response content, the user can continuously perform additional queries or detailed analyses, and the client terminal device (100) transmits these subsequent queries to the AI-based data analysis device (120) in the same manner to repeatedly receive responses. That is, the client terminal device (100) is not limited to a simple output interface, but can operate as a conversational question-and-answer execution module that collects the user's query intent in the form of natural language and performs the entire process of data inquiry, receiving analysis results, and displaying results through real-time communication with the AI-based data analysis device (120). Therefore, the user can check the classification results, sentiment analysis, statistical information, etc. of social data in real time using the chat service of the AI-based data analysis device (120) through the client terminal device (100) without using a separate analysis tool or writing a query statement, and can maximize data analysis efficiency by automating complex data processing processes based on artificial intelligence.

[0040] The communication network (110) includes both wired and wireless communication networks. For example, a wired or wireless internet network may be used or interconnected as the communication network (110). Here, the wired network includes internet networks such as cable networks or public telephone networks (PSTN), and the wireless communication network includes CDMA, WCDMA, GSM, EPC (Evolved Packet Core), LTE (Long Term Evolution), WiBro networks, etc. Of course, the communication network according to the embodiment of the present invention is not limited thereto and may be used as an access network for a next-generation mobile communication system to be implemented in the future, for example, a cloud computing network under a cloud computing environment, a 5G network, a 6G network, etc. For example, if the communication network is a wired communication network, the access point within the communication network (110) can be connected to the exchange of a telephone company, etc., but if it is a wireless communication network, it can be connected to an SGSN or GGSN (Gateway GPRS Support Node) operated by a telecommunications company to process data, or connected to various relay stations such as a BTS (Base Transceiver Station), NodeB, e-NodeB to process data.

[0041] The communication network (110) may include access points. Access points include small base stations such as femto or pico base stations that are often installed within a building. Here, femto or pico base stations are classified according to the maximum number of customer terminal devices (100) or third-party devices (130) that can be connected in the classification of small base stations. Of course, the access point includes a short-range communication module for performing short-range communication such as Zigbee and Wi-Fi with customer terminal devices (100) or third-party devices (130). The access point may use TCP / IP or RTSP (Real-Time Streaming Protocol) for wireless communication. Here, short-range communication can be performed using various standards other than Wi-Fi, such as Bluetooth, Zigbee, infrared (IrDA), RF (Radio Frequency) such as UHF (Ultra High Frequency) and VHF (Very High Frequency), and ultra-wideband communication (UWB). Accordingly, the access point can extract the location of the data packet, specify the best communication path for the extracted location, and transmit the data packet along the specified communication path to the next device, such as an AI-based data analysis device (120). The access point can share multiple lines in a typical network environment and includes, for example, routers, repeaters, and relays.

[0042] In addition, the AI-based data analysis device (120) according to the embodiment of the present invention solves the problem that general-purpose generative artificial intelligence services are not suitable for large-scale social data analysis at the individual post level because they rely on limited data such as credible news. In other words, the AI-based data analysis device (120) is configured to provide a reliable generative AI-based social big data analysis service by collecting all social data collected from various platforms, classifying it at the sentence level, and analyzing and providing the results in a statistical form. To this end, the AI-based data analysis device (120) operates based on the following two main functions. First, the sentence-level classification function divides crawled social data into sentences through a generative artificial intelligence model, interprets the meaning of each sentence, and classifies it into analysis targets, topics, sentiments, etc., according to predefined criteria. Through this, it secures higher accuracy than keyword-based analysis and can semantically classify sentences containing non-standard expressions or slang. Second, the AI ​​Chat-based data inquiry function interprets natural language queries entered by users and generates statistical and analysis results that meet the query conditions by concurrently querying data stored in a vector store and a relational database. Subsequently, the generative AI model generates a response in the form of natural language based on the retrieved data and provides it to the client terminal device (100), thereby enabling the user to interactively check the data analysis results. Ultimately, the AI-based data analysis device (120) combines the semantic understanding ability of the generative AI with a RAG-based search and response structure, thereby enabling precise social big data analysis based on real data, which is difficult for general-purpose AI to perform.

[0043] More specifically, the AI-based data analysis device (120) can perform various operations for sentence-unit classification. As illustrated in FIG. 2a, the AI-based data analysis device (120) interprets the meaning of a sentence using a generative artificial intelligence model, classifies the sentence into an analysis target, topic, and sentiment according to classification criteria, and then performs accurate and flexible sentence-unit AI classification by integrating non-standard expressions by applying rule-based conditional expressions. The AI-based data analysis device (120) of FIG. 1 can perform sentence-unit AI classification operations to classify collected social data at the sentence level. The AI-based data analysis device (120) first collects social data crawled from various online platforms, divides the data into sentence units, and interprets the meaning of each sentence using a generative artificial intelligence model. At this time, the generative artificial intelligence model recognizes sentences at the context level rather than the word level, thereby classifying sentences with higher accuracy compared to keyword-based aggregation methods. In addition, since it requires almost no additional training (fine-tuning) or detailed adjustment compared to existing pre-trained language models (BERT, etc.), it can be efficiently applied to various projects. Accordingly, the AI-based data analysis device (120) can utilize the OpenAI API to extract sentences from social big data collected according to predefined classification rules and classify each sentence into one or more categories among analysis target, topic and sentiment.

[0044] However, instances where generative AI ignores given classification guidelines can frequently occur during the classification process. For instance, when given the instruction to "enter the exact brand name that matches the provided list" and the brand name "Samsung," if the expression within the post appears in a modified form such as "Samsung" or "Samjeon," the model may output it exactly as is. This problem is not completely resolved even if the classification guidelines are specifically modified or presented clearly. To address this, the embodiment of the present invention introduces a method in which the 'analysis target' mentioned in the post is entered exactly as it is, and then aggregates the data using rule-based conditional expressions. For instance, if the analysis target is "Dreamy," the researcher (or service representative) pre-writes a conditional expression such as ("Dreamy" = "Dreamy" OR "Dreamy" OR "Dreame") and aggregates the data based on the rule after classification is complete. In other words, the above conditional expression can be viewed as being pre-set through programming. By using this method, even if some data is omitted during the aggregation process, immediate correction is possible by simply modifying the conditional expression after verification by a researcher (or after automatic verification by the program), and it is possible to flexibly respond to various unpredictable variations of expression.

[0045] Similar difficulties also exist in the process of classifying 'topics'. If a separate explanation is provided to the model, overly conservative results may be generated, and if only abbreviated keywords are presented without explanation, the classification scope tends to expand excessively. For example, if the keyword "air conditioner performance" is presented, the model tends to classify it into an expanded scope of topics that includes not only cooling performance but also energy efficiency, noise, and convenience features. However, even in the case of such overclassification, since the model performs the classification, the researcher (or programmatically, hereinafter referred to as the researcher) can directly review the original text of the classification and the reasons for it, and perform necessary modifications and corrections, which has the advantage of increasing verification efficiency. Therefore, the embodiments of the present invention adopt a stepwise classification method that gradually narrows down from a broad scope of topics to specific details without presenting a separate explanatory text.

[0046] Consequently, researchers can adjust the 'brand aggregation' and 'topic setting' optimized for each project through iterative testing, and in particular, providing the model with a specific and appropriate list of topics is very important for performing accurate sentence-level classification. As such, through the sentence-level AI classification operation illustrated in FIG. 2a, the AI-based data analysis device (120) can generate high-accuracy sentence-level semantic classification results by having the generative AI interpret the meaning of the entire sentence and classify each sentence into an analysis target, topic, or sentiment according to predefined classification criteria, and then re-aggregating the results into rule-based conditional expressions to integrate data containing non-standard expressions into the same semantic group. FIG. 2a clearly illustrates the sequence of the generative AI classification operation.

[0047] Meanwhile, referring to FIG. 2c, the AI-based data analysis device (120) illustrated in FIG. 1 performs a natural language-based data retrieval and analysis AI Chat operation to retrieve data according to a user's natural language query and generate a reliable answer based on the result. An embodiment of the present invention may apply a Retrieval-Augmented Generation (RAG) method to provide a reliable data-based answer. The AI-based data analysis device (120) may design a RAG pipeline in which social data is collected and classified, the result is stored in a vector store (or a first storage), and a generative AI retrieves the data and generates an answer according to a request from a user (or a client terminal device).

[0048] While the use of a vector store is essential for generating accurate answers, calculating numerical statistics using only a vector store leads to long processing times and complex system design. Accordingly, in an embodiment of the present invention, data is stored in a dual storage structure; raw data is stored in an OpenAI vector store using an embedding method, while statistical data is uploaded to a MySQL relational database (or a second storage) after being pre-aggregated. Through this structure, semantic-based search and numerical-based statistical processing can be performed simultaneously.

[0049] In addition, in an embodiment of the present invention, a query generation AI (BuzzSearch) is constructed to present a list of fields and a detailed data schema to the model in advance, and by forcing the output of answers in JSON format, it is possible to induce the automatic generation of stable and consistent queries. Through this process, the AI-based data analysis device (120) according to an embodiment of the present invention can complete a RAG pipeline in which automatic query generation based on natural language queries, data retrieval, and response generation are linked. However, the following technical problems may occur in the above process.

[0050] First, there is the issue of cost associated with the OpenAI vector store. Even when applying the recommended chunking method, it is difficult to selectively retrieve specific posts, and there is a problem where costs skyrocket simply by storing large volumes of data. To address this, the embodiment of the present invention introduces the open-source Chroma, which supports metadata filtering functions, to enable a transition to a local vector store. As a result, sophisticated searching with detailed filter conditions becomes possible, and storage and retrieval costs—excluding sentence embedding costs—can be significantly reduced. However, since query filter conditions between Chroma and MySQL were intermittently generated differently, the embodiment of the present invention is designed to query both stores simultaneously based on the same query structure. Accordingly, data consistency is ensured, and highly accurate responses can be generated by combining vector search and statistical query results in parallel.

[0051] The second issue is the problem of hallucinations caused by massive amounts of data. When the amount of data retrieved by the AI-based data analysis device (120) exceeds the context capacity of the model, a hallucination occurs in which the model generates information that does not actually exist. If the original social data is used as is, thousands to tens of thousands of tokens are generated, making it difficult to derive an accurate response; even if an aggregation table stored in MySQL is used, tens of thousands of tokens are generated according to the query conditions, causing the same phenomenon. To solve this, two technical measures can be applied in the embodiments of the present invention. First, the vector store stores only sentences classified by semantic units instead of the entire document. Second, the MySQL relational database is configured to provide a pre-aggregated statistical table by re-aggregating only the necessary range according to the user query conditions. Through these structural improvements, the AI-based data analysis device (120) can improve search accuracy while reducing the number of tokens in the query results by more than 90%, alleviate the context burden on the model, and effectively suppress hallucinations.

[0052] Meanwhile, the AI-based data analysis device (120) of FIG. 1 according to an embodiment of the present invention can perform advanced analysis functions such as complex queries, numerical calculations, and past context inference by improving the aforementioned unidirectional pipeline structure and introducing an Agentic Loop-based self-feedback structure as shown in FIG. 2d. As shown in FIG. 2d, the AI-based data analysis device (120) can respond to complex queries, numerical calculations, and past context inference by improving the unidirectional pipeline structure into an Agentic Loop structure and performing the utilization of external tools and self-feedback processes, and can perform intelligent analysis operations capable of continuous performance improvement through a post-unit analysis function.

[0053] More specifically, regarding the improvement of the AI-based data analysis device (120) from a unidirectional pipeline structure to an agentic loop, the aforementioned AI model for analysis (AIM) has a unidirectional pipeline structure that responds sequentially according to predetermined rules. While this structure has the advantage of being able to respond quickly to simple queries, it has the limitation of being difficult to respond to requests that are complex or exceed the basic capabilities of the model. Accordingly, the AI-based data analysis device (120) according to the embodiment of the present invention may introduce the concept of an agentic loop. An agentic loop refers to a structure in which the model utilizes external tools or reviews intermediate inference procedures to achieve a goal, and if necessary, delegates specific tasks to specialized AI modules or modifies and supplements the response itself. Through such a self-feedback loop, the model can efficiently process complex requests that are difficult to handle with a unidirectional pipeline structure, such as complex queries, numerical calculations, and the use of past context. In other words, the AI-based data analysis device (120) can verify its own judgment and gradually improve the quality of the response through the agentic loop structure.

[0054] In addition, regarding the limitation of viewing and analyzing posts at the post level, the AI-based data analysis device (120) of FIG. 1 has difficulty directly reviewing original data exceeding a certain number of tokens due to the limitation of the model's context capacity. However, in social data analysis, functions such as aggregating the number of posts containing specific keywords, summarizing specific posts, or identifying comment types are essential. Currently, in the embodiment of the present invention, sentence-level data is provided by pre-aggregating it, and the model only queries classified sentence-level data rather than the entire text. Therefore, there are limitations to context analysis or total aggregation at the post level, and detailed analysis at the post level can only be performed to a limited extent. Accordingly, in the embodiment of the present invention, advanced data analysis functions that consider the entire context, such as aggregation, summarization, and comment type identification at the post level, can be implemented through future improvements. When this function is implemented, it becomes possible to simultaneously secure accuracy and interpretability by combining the sentence-level classification results and the post-level context analysis results.

[0055] Furthermore, for the AI-based data analysis device (120), improving the precision of category and brand classification is another improvement task. In other words, the model does not always accurately classify brands or categories, and accordingly, a process is required to align the recognition of category names between the researcher (or through an interface or program with the researcher) and the model. In particular, in the initial stage, it is essential for the researcher to empirically adjust the category names. This can be interpreted as a kind of labeling action. However, there are cases where the model performs incorrect classification even after undergoing a repetitive verification process. Accordingly, the embodiment of the present invention may additionally introduce a system to optimize the response through fine-tuning of the model and evaluate the classification results in order to increase classification precision. In particular, since the model tends to accurately present the reason even if it incorrectly classifies a category, it is effective to utilize this characteristic so that a separate evaluation AI reviews the classification results and automatically performs reclassification if the reliability score is below a certain standard. Through this, the AI-based data analysis device (120) can continuously improve classification accuracy and maintain optimized data classification quality for each project. Accordingly, the AI-based data analysis device (120) can autonomously process requests including complex queries or multi-stage inference by extending the unidirectional pipeline structure based on an Agentic Loop as shown in FIG. 2d, and can continuously improve performance through the extension of post-unit analysis and improvement of category and brand classification precision.

[0056] The third-party device (130) functions as an external device that is connected to the AI-based data analysis device (120) via a communication network to provide social data or to manage and verify analysis results. The third-party device (130) can be implemented in two main forms. The first is an external portal server or platform server that provides social data, and the second is a computer or terminal device used by an analysis service operator (e.g., a researcher, an inspector, etc.). The third-party device (130) illustrated in FIG. 1 can be configured as an external portal server or SNS platform server that provides social data, or as a computer terminal of a researcher or others who operate and inspect analysis services, and is connected to the AI-based data analysis device (120) via a communication network to support processes before and after data analysis, such as data collection, inspection, and conditional expression management.

[0057] In the first form, the third-party device (130) may be configured in the form of an internet portal, news site, community, blog, SNS, or an external data server providing an open API. This device provides unstructured text data (Social Data), such as reviews, comments, posts, and articles posted by service users or general users, so that the AI-based data analysis device (120) can collect it. The third-party device (130) transmits the social data to the AI-based data analysis device (120) in various ways, such as web crawling, API calls, and data feed transmission, and the data may include metadata such as the body of the post, author information, posting date, and platform identifier. In addition, in some implementation examples, the third-party device (130) may be directly linked with the AI-based data analysis device (120) to perform preprocessing functions such as filtering, duplicate removal, and format conversion at the time of data collection. Through this, the AI-based data analysis device (120) can efficiently receive social data in a refined form and utilize it for sentence-level analysis and RAG-based querying.

[0058] In the second form, the third-party device (130) refers to a computer, laptop, or other terminal device used by a researcher, data analyst, or corporate representative who operates or inspects an AI-based data analysis service. In this case, the third-party device (130) is connected to the same network environment or secure network (VPN, private network, etc.) as the AI-based data analysis device (120) to monitor analysis results or manage settings such as classification criteria, conditional expressions, and query schemas. For example, the researcher can register or modify classification rules (e.g., "analysis target = brand name matching conditional expression") through the third-party device (130), and can review sentence-level classification results or sentiment analysis results performed by the AI-based data analysis device (120) to perform correction and re-aggregation operations if necessary. Additionally, the third-party device (130) can be used to verify the performance of the RAG pipeline built by the AI-based data analysis device (120) and to evaluate the response accuracy of the query generation AI (BuzzSearch) or the results of the Agentic Loop-based feedback. That is, the third-party device (130) performs the role of continuously improving the reliability and accuracy of the AI-based data analysis device (120) as not only a simple data provider but also as an entity for verifying and managing the quality of the analysis service.

[0059] In summary, the third-party device (130) performs the function of supplying external social data or internally verifying and supplementing analysis services, and operates as a core input source and auxiliary management device for the AI-based data analysis device (120). Through this structure, the AI-based data analysis device (120) can collect various forms of social data generated from external platforms in real time and provide a flexible data analysis environment in which verification of classification criteria and modification of conditional expressions through the researcher terminal are immediately reflected.

[0060] In addition to the above, the customer terminal device (100), communication network (110), AI-based data analysis device (120), and third-party device (130) of FIG. 1 can perform various operations, and since the relevant details may continue to be covered later, the details will be replaced by those details.

[0061] FIG. 3 is a block diagram illustrating the detailed structure of the AI-based data analysis device of FIG. 1, and FIG. 4 is a block diagram illustrating the detailed structure of the AI-based data analysis unit of FIG. 3.

[0062] As illustrated in FIG. 3, the AI-based data analysis device (120) of FIG. 1 according to an embodiment of the present invention is a device that provides social data classification and analysis services using, for example, generative AI, and includes some or all of a communication interface unit (300), a control unit (310), an AI-based data analysis unit (320), and a storage unit (330).

[0063] Here, "including some or all" means that the AI-based data analysis device (120) of FIG. 1 may be configured with some components, such as the storage unit (330), omitted, or the AI-based data analysis unit (320) may be configured by integrating it into other components, such as the control unit (310). To facilitate a sufficient understanding of the invention, it is described as including all.

[0064] The communication interface unit (300) is a component included within the AI-based data analysis device (120) that performs data transmission and reception with a client terminal device (100) using a messenger-based chat service and a third-party device (130) providing social data. The communication interface unit (300) receives a natural language query input from the client terminal device (100), transmits the corresponding analysis result back to the client terminal device (100), and also receives and transmits commands such as collecting social data from the third-party device (130) or updating data and statistics when necessary. At this time, the communication interface unit (300) is connected via the Internet, a private network (VPN), or a cloud-based communication network (110), and can use bidirectional asynchronous protocols such as REST API, WebSocket, or gRPC during the data transmission and reception process.

[0065] In particular, the query data input from the client terminal device (100) consists of text in a natural language format, and the communication interface unit (300) converts it into a data structure (e.g., JSON or XML format, etc.) that can be interpreted by the AI-based data analysis device (120) and transmits it to the control unit (310). The control unit (310) analyzes the query and transmits a classification and inquiry request to the AI-based data analysis unit (320), receives the result, and responds to the client terminal device (100) again through the communication interface unit (300).

[0066] Additionally, the communication interface unit (300) receives social data collected in real-time or periodically from a third-party device (130), converts the format of the received data to match the internal schema of the AI-based data analysis device (120), and controls it to be stored in the storage unit (330). In this process, the communication interface unit (300) can detect errors that may occur during data transmission, request retransmission, or perform encryption and decryption processing.

[0067] In addition, the communication interface unit (300) manages the session state and authentication information (Token, Key, etc.) between the customer terminal device (100) and the AI-based data analysis device (120), thereby ensuring that only authorized users can perform data queries and analysis requests. This prevents data leakage or unauthorized queries caused by external access.

[0068] In summary, the communication interface unit (300) acts as an input / output gateway for the AI-based data analysis device (120) and supports stable communication of the AI-based data analysis service through functions such as receiving queries and transmitting responses from the customer terminal device (100), controlling data collection and transmission from the third-party device (130), converting data formats, and managing session authentication.

[0069] The control unit (310) is a central control module that controls the overall data flow and the operation of each component within the AI-based data analysis device (120), and may be configured in the form of an IC chip including a processor such as a CPU, MPU, GPU, or NPU (Neural Processing Unit) and a memory such as RAM. The control unit (310) executes a control program stored in internal memory to comprehensively control a series of operations such as data input / output, query processing, analysis requests, and result return between the communication interface unit (300), the AI-based data analysis unit (320), and the storage unit (330). In practice, the processor of the control unit (310) may be configured to have computational performance sufficient to directly perform data classification and chat service operations performed by the AI-based data analysis unit (320); however, to aid in understanding the present invention, in the actual embodiment, specific AI operations such as data classification, sentiment analysis, and natural language question-and-answer are described as being performed by the AI-based data analysis unit (320). Accordingly, the control unit (310) can control the operation of the AI-based data analysis unit (320) and transmit the execution result to the communication interface unit (300) or storage unit (330).

[0070] That is, the control unit (310) interprets a natural language query received from a client terminal device (100) through a communication interface unit (300), transmits a request for sentence-level classification, sentiment analysis, or statistical data inquiry to an AI-based data analysis unit (320) according to the content and format of the query, and controls the communication interface unit (300) to transmit the result to the client terminal device (100) upon receiving the analysis result. In addition, the control unit (310) controls the storage of social data collected from a third-party device (130) in a storage unit (330) and manages the data update cycle, transmission format, and security authentication status. Through this, the control unit (310) coordinates the entire process so that the entire data analysis process is performed sequentially and stably. In one embodiment, the control unit (310) is responsible for the control flow of scheduling sentence-unit classification operations (see FIG. 2a), natural language-based question-and-answer operations (see FIG. 2b and FIG. 2c), and agentic loop-based feedback operations (see FIG. 2d) performed by the AI-based data analysis unit (320) in the order of operations, allocating necessary system resources (e.g., CPU, GPU, memory, etc.), verifying the analysis results, and transmitting them externally. In addition, the control unit (310) monitors the operational status of each component and, if an error or data inconsistency is detected during the classification process, instructs the AI-based data analysis unit (320) to reprocess or requests data retransmission from a connected third-party device (130). As such, the control unit (310) serves as a central processing unit that manages the overall operation of the AI-based data analysis device (120) and can perform operations such as synchronization of data input / output, management of inter-process communication, error recovery, and security authentication control.

[0071] The AI-based data analysis unit (320) is a core processing module that analyzes social data collected within the AI-based data analysis device (120) and generates response data according to the user's natural language query. The AI-based data analysis unit (320) performs sentence-unit AI classification operations (see FIG. 2a), natural language query-based data lookup and analysis AI chat operations (see FIG. 2b and FIG. 2c), and Agentic Loop-based feedback and improvement operations (see FIG. 2d) according to the control signal of the control unit (310). The AI-based data analysis unit (320) uses a Generative AI model to understand the context of unstructured social data and classifies each sentence into semantic units according to predefined classification criteria. In addition, it generates accurate and reliable analysis results by concurrently querying a Vector Store and a Relational Database (RDB) and performs the role of suppressing hallucinations. Specifically, the AI-based data analysis unit (320) may be composed of a generative AI classification unit (400) that performs sentence-unit data classification and sentiment analysis, and an AI chat function unit (410) that analyzes natural language queries and retrieves and responds to data based on RAG.

[0072] The generative AI classification unit (400) performs the operation of classifying social data collected within the AI-based data analysis unit (320) in sentence units. The generative AI classification unit (400) interprets each sentence of the collected data in semantic units using a generative AI model such as an OpenAI API, and classifies each sentence into one or more of an analysis target, topic, and sentiment according to predefined classification criteria. At this time, since the generative AI classification unit (400) determines the meaning of the entire sentence based on contextual understanding rather than simple keyword matching, it can secure higher accuracy than existing keyword-based classification methods. In addition, it requires almost no additional training or fine-tuning compared to existing BERT-based models, so it can be efficiently applied to various projects. However, during the classification process, a problem may occur where the generative AI ignores guidelines or recognizes modified expressions (e.g., "Samsung", "Samjeon") as they are. Accordingly, the generative AI classification unit (400) inputs the 'analysis target' mentioned in the post exactly as it is, and then applies rule-based conditional expressions to re-aggregate non-standard expressions with the same meaning into a single semantic group.

[0073] For example, consistent data aggregation is performed by applying a conditional expression such as ("Dreamy" = "Dreamy" OR "Dreamy" OR "Dreame") to an analysis target called "Dreamy". Additionally, the generative AI classification unit (400) performs classification based on abbreviated keywords without separate explanations to prevent an excessively narrow or broad range when classifying 'topics', and stores the reasons for classification and supporting data together so that researchers can verify and correct the results later. This allows researchers to empirically adjust the 'brand aggregation' and 'topic criteria' optimized for each project, and enables accurate and flexible sentence-unit AI classification. Consequently, the generative AI classification unit (400) analyzes the meaning of sentences using a generative artificial intelligence model and classifies and re-aggregates sentences according to predefined criteria to convert unstructured social data into structured semantic data.

[0074] The generative AI classification unit (400) stores data in a dual manner. The generative AI classification unit (400) stores original data in a vector store in an embedding form, and uploads statistical data to a MySQL-based relational database after pre-aggregation. Here, the term "embedding form" refers to a data representation form in which each sentence of the collected social data is converted into a multidimensional vector reflecting the meaning of the sentence using a generative artificial intelligence model. This embedding form is used as base data for quantitatively comparing the semantic similarity between sentences or for searching for similar sentences. The semantic-based multidimensional embedding form is not a simple numerical storage but a vector representation that reflects the meaning of the sentence. That is, the embedding form is a data representation format in which the sentence is converted into a numerical vector (or semantic coordinate).

[0075] The AI ​​chat function unit (or execution unit) (410) performs the function of querying data according to the user's natural language query within the AI-based data analysis unit (320) and generating the analysis results in the form of a natural language response. The AI ​​chat function unit (410) adopts the RAG method to provide reliable data-based responses. That is, instead of the generative AI generating the response itself, it parallel queries the analysis data stored in the vector store and the relational database (RDB) and constructs the response sentence based on the results. Subsequently, it uses the query generation AI (BuzzSearch) to present the field list and data schema to the model and enforces the output of a response in JSON format to perform stable query generation. To overcome the limitations of the OpenAI vector store, which does not support metadata filters, the AI ​​chat function unit (410) may use a local vector store such as Chroma; in this case, sophisticated searching using detailed filter conditions becomes possible and sentence embedding costs can be reduced. Additionally, data consistency is ensured by parallel querying the vector store and the relational database using the same query.

[0076] In addition, the AI ​​chat function (410) stores only sentences classified by semantic units instead of the entire document in the vector store to prevent hallucinations that occur when querying large-scale data that exceeds the context capacity of the model, and in the MySQL relational database, it re-aggregates and provides only the range that matches the user query conditions from the pre-aggregated statistical tables. Through this, the number of tokens in the query data is reduced by more than 90%, thereby alleviating the context burden on the model and enabling accurate and reliable question-and-answer responses. As a result, the AI ​​chat function (410) operates as an AI question-and-answer engine that interprets natural language queries and generates data-based responses by querying the vector store and the relational database in parallel.

[0077] As described above, the AI-based data analysis unit (320) may be composed of a generative AI classification unit (400) and an AI chat function unit (410) as illustrated in FIG. 4a. The generative AI classification unit (400) classifies social data in sentence units and semantically re-aggregates it to generate a refined data set, and the AI ​​chat function unit (410) performs natural language question-and-answer based on the classification results and generates reliable analysis results, which are then provided to the customer terminal device (100) through the control unit (310). With this configuration, the AI-based data analysis unit (320) can implement an entire AI analysis pipeline extending from data collection → sentence-unit classification → RAG-based query → response generation, and provides a data analysis service that secures both accuracy and reliability by combining the semantic interpretation capability of generative artificial intelligence with the structural accuracy of the database. The specific operation and data flow of the generative AI classification unit (400) and the AI ​​chat function unit (410) shown in FIG. 4a are clearly illustrated in FIG. 4b.

[0078] The storage unit (330) is provided within the AI-based data analysis device (120) and is a module responsible for temporarily storing various information generated during the data classification and question-and-answer process, or for managing and transmitting data to be processed. The storage unit (330) may be composed of non-volatile storage media such as memory, SSD, and HDD, and in particular, temporarily stores sentence-unit classification results, sentiment analysis results, statistical data, and AI chat response history generated by the AI-based data analysis unit (320). Subsequently, the data is transmitted to the DB (120a) shown in FIG. 1 and is permanently stored and managed. That is, the storage unit (330) functions as a temporary buffer and intermediate data management module within the AI-based data analysis device (120), and performs data transmission and synchronization with the DB (120a) according to the instructions of the control unit (310). At this time, the storage unit (330) can protect the classification results or analysis response data from being altered or lost by performing integrity checks and encryption processing before and after data transmission.

[0079] Data transmitted from the AI-based data analysis unit (320) is stored in the following two forms. For Vectorized Embeddings, semantic vectors extracted in sentence units by the Generative AI Classification Unit (400) are temporarily stored and subsequently uploaded to the vector store area of ​​the DB (120a). For Aggregated Statistics, results generated in MySQL or a pseudo-relational DB format by the AI ​​Chat Function Unit (410) are stored and the data is synchronized to the relational database area of ​​the DB (120a).

[0080] Additionally, the storage unit (330) can support version control and backup of data. For example, by comparing and storing the difference between existing data and new data according to classification rule changes or aggregation condition modifications performed by the AI-based data analysis unit (320), it is possible to enable data history tracking and reprocessing error recovery. In one embodiment, the storage unit (330) can apply a memory caching structure to increase the input / output speed of data and maintain temporary results in a cache state before uploading to the DB (120a) to improve the response speed of the AI ​​chat function unit (410). Additionally, the storage unit (330) can classify and store results received from the AI-based data analysis unit (320) by format, such as sentence data, statistical data, and log data.

[0081] Consequently, the storage unit (330) serves as an internal storage area of ​​the AI-based data analysis device (120) to temporarily store intermediate data generated during the analysis process and permanently synchronize it with the DB (120a) of FIG. 1, thereby ensuring the stability and reliability of data processing.

[0082] In addition to the above, the communication interface unit (300), control unit (310), AI-based data analysis unit (320), and storage unit (330) of FIG. 3 can perform various operations, and other details have been sufficiently explained above, so they will be replaced with those details.

[0083] Meanwhile, the communication interface unit (300), control unit (310), AI-based data analysis unit (320), and storage unit (330) of FIG. 3 according to an embodiment of the present invention are composed of hardware modules that are physically separated from each other, but each module may store software for performing the above operations internally and execute it. However, since the software is a set of software modules and each module can be formed as hardware, the configuration will not be specifically limited to software or hardware. For example, the storage unit (330) may be storage or memory which is hardware. However, since it is also possible to store information in a software repository, the above content will not be specifically limited.

[0084] In addition, as another embodiment of the present invention, the control unit (310) may include a CPU and memory and may be formed as a single chip. The CPU includes a control circuit, an arithmetic logic unit (ALU), an instruction interpretation unit, and a registry, and the memory may include RAM. The control circuit may perform control operations, the arithmetic logic unit may perform operations on binary bit information, and the instruction interpretation unit may include an interpreter or a compiler to perform operations that convert high-level language into machine language and machine language into high-level language, and the registry may be involved in software data storage. According to the above configuration, for example, at the beginning of operation of the AI-based data analysis device (120) of FIG. 1, the data operation processing speed can be rapidly increased by copying a program stored in the AI-based data analysis unit (320), loading it into memory, i.e., RAM, and then executing it. In the case of a deep learning model, it may be loaded into GPU memory instead of RAM and executed by using the GPU to accelerate the execution speed.

[0085] Figure 5 is a flowchart showing the operation process of the AI-based data analysis device of Figure 1.

[0086] For convenience of explanation, referring to FIG. 5 together with FIG. 1, an AI-based data analysis device (120) according to an embodiment of the present invention performs data communication with a client terminal device (100) using a messenger-based chat service (S500). The AI-based data analysis device (120) establishes a bidirectional communication channel with the client terminal device (100) through an internet network or a cloud network and receives query data in a natural language format from the client terminal device (100). At this time, the client terminal device (100) can transmit the user's query sentence in the form of text or voice input on a messenger platform, and the AI-based data analysis device (120) converts it into a data structure capable of natural language analysis.

[0087] The AI-based data analysis device (120) performs a user authentication procedure to verify access rights before transmitting the received query data to the internal data analysis unit. To this end, the AI-based data analysis device (120) performs authentication using a security protocol such as a session authentication token, API key, or OAuth method, and determines whether to receive the query based on the authentication result. The AI-based data analysis device (120) parses information such as the query term, target range, analysis request type, and response format included in the query data to set processing conditions for the query and starts an analysis process corresponding to it. In addition, the AI-based data analysis device (120) can recognize the conversation context by maintaining a session state with the client terminal device (100) and storing and managing previous query history and response results. At this stage, the AI-based data analysis device (120) can transmit status response messages, query processing progress, or error notification messages in real time according to the request of the client terminal device (100). This allows the client to intuitively check the progress status of the analysis request within the messenger environment.

[0088] Additionally, the AI-based data analysis device (120) can automatically determine the response format according to the purpose of the user's request. For example, in the case of a simple query, a response is generated in the form of a natural language sentence, and in the case of a statistical query or sentiment analysis result, a response is configured in the form of a table, graph, or JSON data. The generated response is then transmitted to the client terminal device (100) and displayed in real time on the messenger screen.

[0089] In summary, the AI-based data analysis device (120) receives a natural language query from a client terminal device (100) via a messenger-based chat service and performs a two-way communication procedure including security authentication, data structuring, session maintenance, and response transmission. This step serves as the basis for the sentence-unit data classification and natural language-based analysis steps performed thereafter.

[0090] Furthermore, the AI-based data analysis device (120) can perform the operation of dividing the collected social data into sentence units, and a generative artificial intelligence model interprets the meaning of each divided sentence and classifies each sentence into one or more of an analysis target, a topic, and a sentiment according to a predefined classification criterion (S510). The AI-based data analysis device (120) classifies the collected social data into sentence units, re-aggregates the classification results into rule-based conditional expressions to integrate non-standard expressions, stores the results in parallel in a vector store and a relational database, and then queries both stores according to a natural language query from a customer terminal device (100) to provide the combined analysis results in natural language form.

[0091] The AI-based data analysis device (120) can classify sentences with much higher accuracy compared to existing statistical analysis methods that rely on simple keyword matching, as it is possible to interpret the meaning of a sentence while considering its context by using a generative artificial intelligence model. In addition, it has the advantage of being immediately applicable to various project environments without separate prior training or fine-tuning processes. Based on the classification results, the AI-based data analysis device (120) integrates non-standard expressions (e.g., 'Samsung', 'Samjeon', etc.) that have the same meaning but different notation methods into the same semantic group by re-aggregating them using rule-based conditional expressions. For example, conditional expressions can be automatically applied or corrected through researcher verification so that different notation data such as "Dreamy", "Dreamy", and "Dreame" are all recognized as the same analysis target.

[0092] The AI-based data analysis device (120) embeds the data classified and re-aggregated in this manner into sentence units, converts it into a vector form, and stores the result in a vector store. At the same time, the aggregated statistical data is stored in parallel in a relational database. Through this dual data storage structure, the AI-based data analysis device (120) can simultaneously perform sentence-unit semantic search and numerical-based statistical analysis. The AI-based data analysis device (120) then analyzes a natural language query received from a client terminal device (100) and simultaneously queries the vector store and the relational database according to the purpose and target range of the query. The retrieved sentence embedding results and statistical re-aggregation results are combined within the AI-based data analysis device (120) to form the final response data.

[0093] The AI-based data analysis device (120) generates highly reliable analysis results with improved accuracy and no hallucinogenic effects by combining the contextual understanding results (vector-based search results) of generative artificial intelligence and quantitative data (statistical re-aggregation results). The generated analysis results are converted into natural language form and transmitted to the client terminal device (100). Accordingly, the client can check accurate and clear-of-foundation analysis results in real time within a messenger environment.

[0094] In summary, the AI-based data analysis device (120) divides social data into sentence units and classifies it based on meaning, re-aggregates non-standard expressions using conditional expressions to integrate them into the same semantic group, stores the classification results in parallel in a vector store and a relational database, and simultaneously queries both stores in response to a natural language query from a client terminal device (100) to provide accurate, non-illusory analysis results as natural language responses.

[0095] Furthermore, the AI-based data analysis device (120) reviews the generated analysis results according to internal verification logic, and if an error or discrepancy is detected, it reclassifies or re-aggregates the corresponding sentence or statistical data. The verified analysis results are finally updated through a data synchronization procedure between the vector store and the relational database. Through this, the AI-based data analysis device (120) continuously maintains the quality of analysis and minimizes the possibility of hallucinations.

[0096] The AI-based data analysis device (120) according to an embodiment of the present invention may be improved in various ways in the future. For example, by operating a multi-generative AI model in a collaborative structure, it may provide an optimized response for each query type, or by introducing a continuous learning function that reflects user feedback, it may improve classification accuracy and response consistency. In addition, it may expand multimodal data analysis functions such as multilingual and image / voice data, and simultaneously improve processing speed and cost efficiency by combining a local AI engine with an open-source vector store.

[0097] In addition to the above, the AI-based data analysis device (120) of FIG. 1 can perform various operations, and other details have been sufficiently explained above, so they will be replaced with those details.

[0098] Meanwhile, although it has been described that all components constituting an embodiment of the present invention are combined or operate in combination, the present invention is not necessarily limited to such an embodiment. That is, within the scope of the purpose of the present invention, all components may be selectively combined in one or more ways to operate. Furthermore, while all components may each be implemented as a single independent piece of hardware, they may also be implemented as a computer program having a program module that performs some or all of the combined functions on one or more pieces of hardware by selectively combining some or all of the components. The codes and code segments constituting the computer program can be easily inferred by those skilled in the art of the present invention. An embodiment of the present invention may be implemented by storing such a computer program on a non-transitory computer-readable medium, reading it, and executing it by a computer.

[0099] Here, a non-transient readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the programs described above may be stored and provided on non-transient readable recording media such as CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0100] Although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention. Explanation of the symbols

[0101] 100: Client terminal device 110: Communication network 120: AI-based data analysis device 130: Third-party device 300: Communication interface unit 310: Control unit 320: AI-based data analysis unit 330: Storage unit 400: Generative AI Classification Unit 410: AI Chat Function Unit

Claims

Claim 1 A communication interface unit that communicates with a client terminal device using a messenger-based chat service; and a control unit that divides collected social data into sentence units, and a generative artificial intelligence (AI) model interprets the meaning of each divided sentence to classify each sentence into one or more of an analysis target, a topic, and a sentiment according to a predefined classification criterion, re-aggregates the classification results using a rule-based conditional expression to integrate data containing non-standard expressions into the same semantic group, embeds the classification results into sentence units and stores them in a vector store, concurrently stores the classified and aggregated statistical data in a relational database, analyzes a natural language query received from the client terminal device to simultaneously query the vector store and the relational database, and combines the retrieved sentence embedding results and statistical re-aggregation results to generate an analysis result based on semantic and quantitative grounds in the form of natural language and provides it to the client terminal device as a response to the query. Claim 2 In claim 1, the control unit is an AI-based data analysis device that, when the generative artificial intelligence model interprets the meaning of the entire sentence, recognizes contextually pre-set entity names and sentiment words, classifies them into one or more of an analysis target, a topic, and a sentiment, and generates metadata corresponding to the purpose of analysis of each sentence according to the classification result. Claim 3 In claim 1, the control unit re-aggregates non-standard expressions or modified terms among the classification results output by the generative artificial intelligence model into standardized word groups according to a predefined rule-based conditional expression, and generates statistical data based on the re-aggregation results, an AI-based data analysis device. Claim 4 In claim 1, the control unit executes a dual storage and search module to store the classified and re-aggregated data in a vector store in the form of sentence unit embeddings expressed in a multidimensional vector structure reflecting sentence meaning, and simultaneously stores the statistical data in a relational database in parallel, thereby querying the vector store and the relational database simultaneously during a query, an AI-based data analysis device. Claim 5 In claim 1, the control unit analyzes the meaning of a natural language query input from the customer terminal device to automatically extract conditions and statistical fields of the query, queries the vector store and the relational database based on the extracted conditions to combine the retrieved results, and then excludes data with a reliability level below a standard among the combined results to prevent hallucinations and generates a final analysis result. Claim 6 A method for operating an AI-based data analysis device, comprising: a step in which a communication interface unit communicates with a client terminal device using a messenger-based chat service; and a step in which a control unit divides collected social data into sentence units, and a generative artificial intelligence model interprets the meaning of each divided sentence and classifies each sentence into one or more of an analysis target, a topic, and a sentiment according to a predefined classification criterion, and re-aggregates the classification results into a rule-based conditional expression to integrate data including non-standard expressions into the same semantic group, embeds the classification results into sentence units and stores them in a vector store, and concurrently stores the classified and aggregated statistical data in a relational database, analyzes a natural language query received from the client terminal device to simultaneously query the vector store and the relational database, and combines the queryed sentence embedding results and the statistical re-aggregation results to generate an analysis result based on semantic and quantitative grounds in the form of natural language and provides it to the client terminal device as a response to the query. Claim 7 In claim 6, the step of classifying one or more is a method of operating an AI-based data analysis device, wherein when the generative artificial intelligence model interprets the meaning of the entire sentence, it recognizes contextually pre-set entity names and sentiment words to classify one or more of an analysis target, a topic, and a sentiment, and generates metadata corresponding to the purpose of analysis of each sentence according to the classification result. Claim 8 In claim 6, the step of integrating into the same semantic group comprises re-aggregating non-standard expressions or modified terms among the classification results output by the generative artificial intelligence model into a standardized word group according to a predefined rule-based conditional expression, and generating statistical data based on the re-aggregation results, in a method of operating an AI-based data analysis device. Claim 9 In claim 6, the steps of parallel storage and simultaneous querying include the step of the control unit executing a dual storage and search module to store the classified and re-aggregated data in a vector store in the form of sentence unit embeddings expressed in a multidimensional vector structure reflecting sentence meaning, and simultaneously storing the statistical data in a relational database in parallel, and querying the vector store and the relational database simultaneously during a query. Claim 10 In claim 6, the step of generating the analysis result in a natural language form comprises: a step of automatically extracting conditions and statistical fields of the query by analyzing the meaning of a natural language query input from the customer terminal device; and a step of generating a final analysis result by querying the vector store and the relational database based on the extracted conditions, combining the retrieved results, and excluding data with a reliability level below a standard among the combined results so as not to cause hallucinations.

Images