Active Metadata-based Automated Data Collection Control System and Method

Abstract

The present invention relates to an active metadata-based data collection control device comprising: one or more processors; and a memory for storing instructions executed by the one or more processors; wherein the one or more processors, by executing the instructions, acquire collected data from an external data source, manage metadata and user behavior data based on the collected data, generate an extension candidate signal for data collection extension based on the metadata and the user behavior data, generate a Trigger Event based on the extension candidate signal, and control a data collection workflow based on the Trigger Event.

Description

Technology Field

[0001] The present invention relates to data collection and data management technology. More specifically, the present invention relates to an active metadata-based data collection control system and method that automatically expands the scope of data collection and controls the data collection workflow based on metadata analysis results and user behavior data.

[0002] The present invention relates to a trigger-based data collection control structure that dynamically expands the scope of data collection in response to changes in data demand and user interest in a data portal environment, and controls the execution of a data collection workflow by converting data analysis results into trigger events. Background Technology

[0003] Recently, various data portals and platforms have been supporting data exploration, search, and utilization by collecting data from external sources—such as social media, news, blogs, and public data portals—to build data catalogs and providing them to users. These systems are widely used to systematically manage large-scale data assets and improve user accessibility.

[0004] However, conventional data collection systems generally operate by adjusting the data collection scope based on predefined keywords, Cron-based static scheduling, or manual settings by operators. Consequently, there are limitations in flexibly expanding or reconfiguring the data collection scope in response to changes in data demand or user interest.

[0005] For example, even if user interest in a specific topic or keyword increases rapidly, data collection systems based on conventional technology find it difficult to automatically expand the scope of relevant data collection by reflecting this in real-time or near-real-time. Furthermore, even when data within a specific data category is concentrated on certain keywords or areas, the functionality to automatically identify data gaps and actively derive additional collection targets to supplement them is insufficient.

[0006] Furthermore, although user behavior data such as search patterns, click history, and navigation flow is crucial information reflecting actual data demand, conventional data collection systems fail to effectively incorporate this data into their collection strategies. As a result, discrepancies may arise between actual user demand and the collected data, leading to problems where the data users need is not secured in a timely manner.

[0007] Meanwhile, conventional data catalog systems, such as DataHub and Amundsen, primarily focus on metadata management, data lineage management, and data exploration and search functions. However, these systems do not sufficiently provide a structure capable of reconstructing subsequent data collection strategies using metadata generated from data collection results, automatically expanding the scope of data collection by reflecting user behavior data, or dynamically controlling the execution of data collection workflows.

[0008] Therefore, there is a need for active data collection control technology that can proactively respond to changes in data demand and user interests, automatically expand the scope of data collection based on metadata analysis results and user behavior data, and dynamically control data collection execution. The problem to be solved

[0009] The present invention aims to provide an active metadata-based data collection control technology that can automatically expand the scope of data collection based on metadata analysis results.

[0010] The present invention aims to provide a user behavior-based data collection control technology that analyzes user search history and data usage history to resolve discrepancies between user demand and data retention status, and can automatically expand data collection regarding topics of user interest.

[0011] The present invention aims to provide a data expansion decision structure capable of deriving data collection expansion candidates from metadata analysis results and user behavior analysis results, and generating trigger events by determining whether to execute data collection based on these candidates.

[0012] The present invention aims to provide a trigger-based data collection control structure capable of dynamically changing the data collection scope, data collection target, and data collection timing by controlling the execution of a data collection workflow based on trigger events.

[0013] The present invention aims to provide an active metadata-based data expansion feedback loop structure capable of continuously reconfiguring data collection strategies by feeding back metadata generated from data collection results and user behavior data. means of solving the problem

[0014] The present invention relates to an active metadata-based data collection control device comprising: one or more processors; and a memory for storing instructions executed by the one or more processors; wherein the one or more processors are configured to acquire collected data from an external data source by executing the instructions, manage metadata and user behavior data based on the collected data, generate an extension candidate signal for data collection extension based on the metadata and user behavior data, generate a Trigger Event based on the extension candidate signal, and control a data collection workflow based on the Trigger Event.

[0015] Additionally, the above one or more processors are configured to analyze the metadata by executing the above instruction to derive an extension candidate corresponding to at least one of an associated keyword, an extension target category, and a data coverage gap area.

[0016] In addition, the above one or more processors are configured to analyze the user behavior data by executing the above instruction to derive an extension candidate corresponding to at least one of user interest topics, uncollected keywords, related topics, and areas with insufficient data relative to user demand.

[0017] Additionally, the one or more processors are configured to integrate and analyze the extension candidate derived based on the metadata and the extension candidate derived based on the user behavior data when generating the extension candidate signal by executing the instruction.

[0018] Additionally, the above one or more processors are configured to determine a priority for the integratedly analyzed expansion candidates by executing the above instruction, based on at least one of the level of data demand, user interest, degree of data coverage insufficiency, correlation with external events, necessity of expansion, and expected utilization.

[0019] Additionally, the above one or more processors are configured to generate the Trigger Event by executing the above instruction, such that it includes at least one of information regarding a new collection keyword, an expanded target category, an external data source for collection, a collection scope, a collection time, a collection cycle, a priority, and execution conditions.

[0020] Additionally, the above one or more processors are configured to perform at least one of adding new collection keywords, adding expansion target categories, changing external data sources to be collected, changing the collection time, changing the collection cycle, and expanding the collection scope when controlling the data collection workflow by executing the above instruction.

[0021] Additionally, the one or more processors are configured to reconstruct the data collection strategy by executing the instruction, thereby reflecting the newly collected data and corresponding metadata in subsequent analysis in accordance with the execution of the data collection workflow.

[0022] The present invention relates to a data collection control method performed by an active metadata-based data collection control device comprising one or more processors, comprising: an acquisition step of acquiring collected data from an external data source; an analysis step of analyzing metadata and user behavior data generated or extracted from the collected data; an expansion candidate signal generation step of generating an expansion candidate signal for data collection expansion based on the analysis results of the metadata and the analysis results of the user behavior data; a determination step of integrating and analyzing the expansion candidate signals and determining a priority; a trigger event generation step of generating a trigger event based on the expansion candidate signals for which the priority has been determined; and an execution step of executing or re-executing a data collection workflow based on the trigger event.

[0023] Additionally, the analysis step includes the step of analyzing the metadata to derive an extension candidate corresponding to at least one of an associated keyword, an extension target category, and a data coverage gap area.

[0024] Additionally, the analysis step includes the step of analyzing the user behavior data to derive expansion candidates corresponding to at least one of user interest topics, uncollected keywords, related topics, and areas with insufficient data relative to user demand.

[0025] Additionally, the judgment step includes a step of mutually comparing extension candidates generated based on the analysis results of the metadata with extension candidates generated based on the analysis results of the user behavior data to identify duplicate or similar candidates, and determining the priority by integrating and analyzing them.

[0026] Additionally, the above-mentioned judgment step includes a step of determining the priority of the expansion candidate signals based on at least one of the level of data demand, user interest, degree of data coverage insufficiency, correlation with external events, necessity of expansion, and expected utilization.

[0027] Additionally, the Trigger Event generation step comprises the step of generating a Trigger Event that includes at least one of information regarding a new collection keyword, an expanded target category, an external data source for collection, a collection scope, a collection time, a collection cycle, a priority, and execution conditions.

[0028] In addition, the execution step includes a step of performing at least one of adding new collection keywords, adding expansion target categories, changing external data sources to be collected, changing the collection time, changing the collection cycle, and expanding the collection scope. Effects of the invention

[0029] The present invention can automatically expand the scope of data collection by analyzing metadata generated from collected data, thereby improving the proactiveness and flexibility of data collection compared to conventional methods that relied on fixed collection rules.

[0030] Since the present invention can incorporate user behavior data, such as user search history, click history, and navigation paths, into data collection control, it has the effect of mitigating the discrepancy between actual user demand and the data retention status and increasing the possibility of securing data on topics of user interest.

[0031] The present invention generates extended candidate signals based on metadata analysis results and user behavior analysis results, and determines priorities by integrating and analyzing them, thereby having the effect of more rationally determining the selection of data collection targets and the execution order.

[0032] The present invention converts an extended candidate signal into a Trigger Event and controls the execution of a data collection workflow based thereon, thereby enabling the dynamic change of the data collection range, collection target, and collection timing.

[0033] The present invention can reflect newly collected data and corresponding metadata back into subsequent analysis, thereby providing a closed-loop data collection control structure that continuously reconfigures the data collection strategy. Brief explanation of the drawing

[0034] FIG. 1 is a block diagram showing the overall configuration of an active metadata-based data collection control system (10) according to one embodiment of the present invention. FIG. 2 is a flowchart showing the overall operation flow of an active metadata-based data collection control method according to an embodiment of the present invention. FIG. 3 is a block diagram showing the detailed configuration of an active metadata analysis module (510) according to one embodiment of the present invention. FIG. 4 is a block diagram showing the detailed configuration of a user behavior analysis module (540) according to one embodiment of the present invention. Specific details for implementing the invention

[0035] Specific details of the embodiments are included in the detailed description and drawings.

[0036] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0037] FIG. 1 is a block diagram showing the overall configuration of an active metadata-based data collection control system (10) according to one embodiment of the present invention.

[0038] According to one embodiment of the present invention, each component of the active metadata-based data collection control system (10) may be implemented by one or more computing devices. The one or more computing devices may include at least one processor, a memory, and a communication interface, and the memory may store one or more program instructions executed by the processor. By executing the program instructions, the processor may perform the functions of the data collection unit (300), trigger control unit (400), data management unit (500), or sub-components described below.

[0039] According to one embodiment of the present invention, the data collection unit (300), the trigger control unit (400), and the data management unit (500) may be implemented in dedicated hardware, but are not limited thereto, and may be implemented in a functional configuration in which at least one processor executes program instructions stored in memory. For example, the data collection unit (300) may be implemented by a data collection program module or instruction set for performing interaction with an external data source (200), the trigger control unit (400) may be implemented by a control program module or instruction set for performing Trigger Event generation and workflow execution control based on extended candidate signals, and the data management unit (500) may be implemented by a management program module or instruction set for performing storage, management, and analysis of collected data, metadata, and user behavior data.

[0040] According to one embodiment of the present invention, the active metadata analysis module (510), metadata management module (520), real data management module (530), and user behavior analysis module (540) included in the data management unit (500), and the trigger management unit (410), trigger storage unit (420), and collected data expansion determination module (430) included in the trigger control unit (400) may each be implemented in the form of an independent program module, library, service, task, process, or thread. In addition, these configurations do not necessarily need to be implemented in a physically separated form, but may be implemented as integrated or distributed software configurations on one or more processors.

[0041] According to one embodiment of the present invention, the memory may include volatile memory or non-volatile memory and may store program instructions, collected data, metadata, user behavior data, Trigger Event information, or workflow execution information. Additionally, the processor may execute program instructions stored in the memory to generate extension candidate signals based on metadata analysis results and user behavior analysis results, determine the priority of the extension candidate signals, generate a Trigger Event based on the determination result, and control the execution of a data collection workflow according to the generated Trigger Event.

[0042] According to one embodiment of the present invention, the one or more computing devices may be implemented as a single server, a plurality of servers, cloud computing infrastructure, a virtual machine, a container-based execution environment, or a combination thereof. For example, the data collection unit (300) may be executed on a collection server adjacent to an external data source (200), the data management unit (500) may be executed on a separate analysis server or storage server, and the trigger control unit (400) may be executed on a control server that orchestrates the data collection workflow. However, the present invention is not limited thereto, and the configurations may be implemented distributed across a single computing device or a plurality of computing devices.

[0043] According to one embodiment of the present invention, the program instruction may be stored in a computer-readable recording medium, and the active metadata-based data collection control method described herein may be performed by executing the program instruction stored in the recording medium by one or more processors.

[0044] As described above, the active metadata-based data collection control system (10) of the present invention can collect various types of data from an external data source (200). The external data source (200) may include, for example, a social media (210), news (220), a blog (230), a data mart (240), and a data portal (250). Here, the social media (210) may provide user posts, comments, hashtags, reaction information, etc., the news (220) may provide article body, article meta-information, publication time information, etc., and the blog (230) may provide posts, tags, category information, etc. Additionally, the data mart (240) may provide structured data for analysis or aggregated data by topic, and the data portal (250) may provide information related to public data or open data sets.

[0045] According to one embodiment of the present invention, a data collection unit (300) may be configured to acquire collected data from an external data source (200). The collected data of the present invention may refer to target data collected by the data collection unit (300) from the external data source (200). The collected data may include unstructured data, semi-structured data, or structured data, and may include, for example, text data, image-related metadata, posting time information, author information, category information, tag information, link information, statistical information, or data in the form of a structured record. Additionally, the data collection unit (300) may acquire the collected data using different collection methods depending on the type or characteristics of the external data source (200). For example, the data collection unit (300) may collect the collected data through API calls, crawling, feed reception, batch linkage, or data linkage interfaces. Accordingly, the active metadata-based data collection control system (10) can secure various forms of collected data from a plurality of different external data sources.

[0046] According to one embodiment of the present invention, a data collection unit (300) may be configured to acquire collected data from an external data source (200) and to transmit the collected data to a data management unit (500). The 'collected data + metadata' of FIG. 1 may mean that the collected data obtained by the data collection unit (300) from the external data source (200) and metadata extracted or generated based on the collected data are provided together to the data management unit (500). Here, the collected data may refer to raw data collected from the external data source (200) to the system (10). For example, the collected data may include the body of a post, article content, blog post content, data records in tabular form, link information, image-related information, or other source data. Meanwhile, metadata may refer to information about data extracted or generated from the collected data, rather than the collected data itself. For example, metadata may include keyword information, subject classification information, creation time information, source information, data format information, category information, tag information, summary information, quality information, or additional information for determining relevance.

[0047] According to one embodiment of the present invention, the data management unit (500) is configured to store, manage, and analyze collected data and metadata provided by the data collection unit (300), and may include an active metadata analysis module (510), a metadata management module (520), a real data management module (530), and a user behavior analysis module (540).

[0048] According to one embodiment of the present invention, a data portal (100) may include a portal server (110) and a portal user interface (120). The portal server (110) may be configured to receive user requests within an active metadata-based data collection control system (10) and to provide services related to data search, data lookup, and data exploration. The portal user interface (120) is an interface for interaction between a user and an active metadata-based data collection control system (10) and may receive user actions such as search input, click, exploration, lookup, or selection.

[0049] The 'user behavior data' illustrated in FIG. 1 may refer to user activity information generated through the data portal (100). For example, user behavior data may include user search keywords, search request history, click history, navigation path, category movement pattern, viewing frequency, time spent, and repeated usage history. The data portal (100) may generate or collect such user behavior data and provide it to the data management unit (500), and the data management unit (500) may analyze user interest topics, changes in user demand, or data usage patterns based on the provided user behavior data.

[0050] According to one embodiment of the present invention, a portal server (110) may be configured to process data search, data lookup, and data exploration services provided through a data portal (100). Specifically, the portal server (110) may receive a user request from a portal user interface (120), generate a data search result, a data lookup result, or a data exploration result corresponding to the user request, and provide it to the portal user interface (120). Additionally, the portal server (110) may mediate the interaction between the user and the active metadata-based data collection control system (10) within the data portal (100) by processing search terms, selection information, click information, search path information, or lookup request information entered by the user.

[0051] According to one embodiment of the present invention, the portal user interface (120) may be configured as a user interface that enables a user to perform data search, data inquiry, and data exploration through the data portal (100). Specifically, the portal user interface (120) may receive user inputs such as user search term input, category selection, data selection, click, scroll, page movement, or inquiry request, and may output search results, inquiry results, or exploration results corresponding thereto. In addition, the portal user interface (120) may generate user activity information such as search history, click history, exploration path, category movement pattern, or repeated inquiry history that occurs during the user's data usage process, and the generated user activity information may be transmitted to the data management unit (500) through the portal server (110) and utilized as user behavior data.

[0052] According to one embodiment of the present invention, an active metadata analysis module (510) may be configured to analyze metadata provided by a data collection unit (300) and generate analysis information for actively controlling a data collection strategy. Here, the metadata may be information about data extracted from or generated from collected data, and the active metadata analysis module (510) may not merely store or classify the metadata but may generate active metadata to determine changes in data demand and the need for data expansion.

[0053] For example, the active metadata analysis module (510) can derive new keyword candidates, expansion category candidates, or areas for supplementing data collection that can be expanded from existing collected keywords by analyzing changes in collection volume by keyword, correlation between keywords, co-occurrence patterns, data distribution within categories, data coverage gaps, data generation patterns over time, correlations with changes in user interest, and external event or issue information. Additionally, the active metadata analysis module (510) can derive new data collection targets that can be derived from existing data by analyzing relationships between metadata and data lineage. The analysis results generated in this way can be provided to the data expansion judgment module as expansion candidate signals for expanding data collection.

[0054] According to one embodiment of the present invention, a metadata management module (520) may be configured to store, classify, update, and manage metadata corresponding to collected data provided by a data collection unit (300). Here, metadata is information about data extracted or generated from collected data, and may include, for example, keyword information, category information, source information, creation time information, tag information, data format information, summary information, quality information, and association information. By managing such metadata in a structured form, the metadata management module (520) can provide basic data for the active metadata analysis module (510) to perform subsequent analysis. Additionally, the metadata management module (520) can provide a management foundation for expanding the scope of data collection, identifying the status of data coverage, and reconstructing data collection strategies by continuously updating metadata in response to the inflow of new collected data and maintaining a linkage relationship with previously stored metadata. Accordingly, the metadata management module (520) can perform the function of consistently managing metadata within the active metadata-based data collection control system (10) and providing it for other modules to utilize.

[0055] According to one embodiment of the present invention, the actual data management module (530) may be configured to store, manage, and provide collected data provided by the data collection unit (300). Here, the collected data may refer to raw data collected from an external data source (200) to the active metadata-based data collection control system (10), and may include, for example, the body of a post, article content, blog post content, data records in tabular form, link information, image-related information, or other source data. The actual data management module (530) may store and manage such collected data according to the type, source, category, creation time, or collection time of the data, and may enable the search, viewing, and analysis utilization of each collected data in conjunction with metadata managed by the metadata management module (520). In addition, the actual data management module (530) may provide a basis for determining whether to expand the scope of data collection and for reconstructing subsequent data collection strategies by providing actual data that is the subject of processing for the active metadata analysis module (510) and the user behavior analysis module (540).

[0056] According to one embodiment of the present invention, a user behavior analysis module (540) may be configured to collect and analyze user activity data generated in a data portal and generate a signal for expanding data collection that reflects user demand. Here, user activity data may include user search keywords, click history, navigation path, category movement patterns, view frequency, time spent, or history of repeated use. The user behavior analysis module (540) may not merely store such user activity data but may interpret changes in user interest and data requirements to derive targets and scopes for which data collection expansion is necessary.

[0057] For example, the user behavior analysis module (540) can identify uncollected keywords where searches exist but data is insufficient or non-existent by analyzing whether there is a match between user search keywords and actual collected data. Additionally, it can derive candidate topics for expansion related to a specific topic by analyzing user search paths and consecutive click patterns, and can generate a scope for expanding data collection by identifying interest diffusion paths derived from a specific topic by analyzing user usage flow and category movement patterns. Furthermore, it can identify areas where data is insufficient relative to user demand by comparing and analyzing user behavior data with existing data coverage, and define such areas as targets for expanding data collection. Additionally, the user behavior analysis module (540) can expand the scope for expanding data collection by generating synonyms, superordinate concepts, subordinate concepts, or related keywords based on user search keywords.

[0058] As described, the data management unit (500) can generate an expansion candidate signal for expanding data collection based on the results of analyzing collected data, metadata, and user behavior data, and provide it to the trigger control unit (400). The 'expansion candidate signal' of FIG. 1 may refer to candidate information or judgment basis information that can be utilized to expand the scope of data collection, add data collection targets, or adjust the timing of data collection. For example, the expansion candidate signal may include new keyword candidates, related topic candidates, information on data coverage gap areas, information on data insufficient areas relative to user demand, collection target candidates corresponding to specific events or issues, or priority collection target information. The trigger control unit (400) can determine whether to execute the data collection workflow, the execution target, and the execution conditions using the expansion candidate signal provided by the data management unit (500).

[0059] As described, the trigger control unit (400) may include a trigger management unit (410), a trigger storage unit (420), and a collected data expansion determination module (430). The collected data expansion determination module (430) may determine whether data collection expansion is necessary based on expansion candidate signals provided by the data management unit (500), and may generate a Trigger Event according to the result of the determination. The 'Trigger Event' of FIG. 1 may refer to event information for instructing or requesting the execution of a data collection workflow, the designation of additional collection targets, the expansion of the collection scope, or the adjustment of the collection timing. For example, the Trigger Event may include information regarding new collection keywords, expansion target categories, priority collection targets, collection conditions, execution timing, or execution priority.

[0060] According to one embodiment of the present invention, the trigger management unit (410) can receive a Trigger Event provided by the data collection expansion judgment module (430) and manage whether to execute and the execution conditions of the data collection workflow. Specifically, the trigger management unit (410) can register, update, or delete the Trigger Event, and when multiple Trigger Events are generated, it can manage the priority, execution time, target of application, and collection conditions of each Trigger Event. In addition, the trigger management unit (410) can identify and adjust duplicate or conflicting Trigger Events, and can generate execution information to be provided to the data collection unit (300) based on the managed Trigger Events.

[0061] According to one embodiment of the present invention, the trigger storage unit (420) may be configured to store Trigger Events generated by the collected data expansion judgment module (430) and managed by the trigger management unit (410). The trigger storage unit (420) may store new collection keywords, expansion target categories, priority collection targets, collection conditions, execution time, priority, or processing status information corresponding to each Trigger Event. Additionally, the trigger storage unit (420) may provide reference information for determining whether duplicate Trigger Events of the same or similar Trigger Events are generated, or for determining whether previously generated Trigger Events are reused, by maintaining history information of previously stored Trigger Events.

[0062] As described, the trigger control unit (400) can provide a workflow execution command to the data collection unit (300) based on a Trigger Event managed by the trigger management unit (410). The 'workflow execution command' of FIG. 1 may refer to control information for directing the execution, re-execution, change of execution conditions, or expansion of the collection scope of a data collection workflow. For example, the workflow execution command may include new collection keywords, expansion target categories, collection target sources, collection time, collection cycle, collection scope, priority, or execution condition information. The data collection unit (300) can perform a data collection workflow for an external data source (200) according to the workflow execution command provided by the trigger control unit (400).

[0063] FIG. 2 is a flowchart showing the overall operation flow of an active metadata-based data collection control method according to an embodiment of the present invention.

[0064] According to the present invention, in the step (S110) of performing metadata and user behavior analysis, the active metadata-based data collection control system (10) can analyze metadata generated or extracted from collected data and user behavior data obtained through the data portal (100). Specifically, the active metadata-based data collection control system (10) can analyze changes in the amount of data collected by keyword, correlations between keywords, co-occurrence patterns, data distribution within categories, data coverage gap areas, data generation patterns over time, or correlations with external events based on the metadata. In addition, the active metadata-based data collection control system (10) can analyze user search keywords, click history, search paths, category movement patterns, recurring view history, or user usage flow based on user behavior data to identify changes in user interest, uncollected keywords, topics expandable based on search, and areas with insufficient data relative to user demand.

[0065] According to the present invention, in the step of generating an extension candidate signal (S120), the active metadata-based data collection control system (10) can generate an extension candidate signal for expanding data collection based on the metadata analysis results and user behavior analysis results derived in the step of performing metadata and user behavior analysis (S110). Specifically, the active metadata-based data collection control system (10) can derive associated keyword candidates, extension category candidates, supplementary collection targets corresponding to data coverage gap areas, or new collection targets corresponding to external events from the metadata analysis results.

[0066] Additionally, the active metadata-based data collection control system (10) can derive uncollected keywords, related topic candidates based on search paths, derived collection targets corresponding to user interest diffusion paths, or supplementary collection targets corresponding to areas with insufficient data relative to user demand from the results of user behavior analysis. The expanded candidate signals generated in this way may include information regarding new collection keywords, expansion target categories, related topic candidates, priority collection targets, or areas requiring data supplementation.

[0067] According to the present invention, in the step of integrated analysis of expansion candidate signals and priority determination (S130), the active metadata-based data collection control system (10) can integrate and analyze a plurality of expansion candidate signals generated in the expansion candidate signal generation step (S120) and determine the priority for each expansion candidate signal. Specifically, the active metadata-based data collection control system (10) can identify duplicate or similar candidates by comparing the expansion candidate signals generated based on metadata analysis results with the expansion candidate signals generated based on user behavior analysis results, and can merge or refine them. In addition, the active metadata-based data collection control system (10) can determine the priority for each expansion candidate signal by evaluating the level of data demand, user interest, degree of data coverage insufficient, correlation with external events, necessity of expansion, or expected utilization.

[0068] According to the present invention, in the Trigger Event generation step (S140), the active metadata-based data collection control system (10) can generate a Trigger Event based on the extension candidate signals selected and assigned a priority in the extension candidate signal integration analysis and priority determination step (S130). Specifically, the active metadata-based data collection control system (10) can generate event information for each extension candidate that includes information necessary for the execution of a data collection workflow.

[0069] For example, a Trigger Event may include information regarding new collection keywords, expansion target categories, external data sources for collection, collection scope, collection time, collection cycle, priority, or execution conditions. Additionally, the active metadata-based data collection control system (10) may generate a Trigger Event for an expansion candidate with a high priority among a plurality of expansion candidate signals, or generate a Trigger Event by integrating Trigger Events corresponding to duplicate or similar expansion candidates.

[0070] According to the present invention, in the Trigger Event-based data collection workflow execution step (S150), the active metadata-based data collection control system (10) can execute the data collection workflow based on the Trigger Event generated in the Trigger Event generation step (S140). Specifically, the active metadata-based data collection control system (10) can execute or re-execute the data collection workflow according to new collection keywords, expansion target categories, external data sources to be collected, collection scope, collection time, collection cycle, priority, or execution conditions included in the Trigger Event.

[0071] Accordingly, the active metadata-based data collection control system (10) can expand the scope of data collection for external data sources (200), add new data collection targets, or dynamically change the execution timing and execution conditions of existing data collection. In addition, since the newly collected data and the corresponding metadata can be reused for subsequent analysis through the Trigger Event-based data collection workflow execution step (S150), the active metadata-based data collection control system (10) can perform closed-loop data collection control that continuously reconstructs the data collection strategy by feeding back the data collection results.

[0072] FIG. 3 is a block diagram showing the detailed configuration of an active metadata analysis module (510) according to one embodiment of the present invention.

[0073] According to one embodiment of the present invention, the metadata preprocessing and normalization block (511) may be configured to refine and standardize the collected data and metadata provided from the data collection unit (300) into a form that allows for subsequent analysis. Here, the metadata preprocessing and normalization block (511) may provide consistent analysis criteria by correcting format differences, representation differences, or structural differences included in the data collected from different external data sources (200).

[0074] For example, the metadata preprocessing and normalization block (511) can normalize keyword notation methods, tag formats, category names, date and time formats, source notation formats, or attribute naming systems. Additionally, the metadata preprocessing and normalization block (511) can perform removal of duplicate metadata, correction of missing information, removal of unnecessary information, text cleaning, or structuring processing.

[0075] According to one embodiment of the present invention, the keyword association analysis block (512) may be configured to derive association relationships between keywords by analyzing metadata that has been refined and normalized in the metadata preprocessing and normalization block (511). Specifically, the keyword association analysis block (512) may derive new keyword candidates that can be linked to existing collected keywords by analyzing the co-occurrence frequency, co-occurrence pattern, association, similarity, or distribution relationship by category of keywords included in the collected data.

[0076] For example, the keyword association analysis block (512) can identify associated keywords that appear repeatedly with a specific keyword, derived keywords used together within a specific topic, or semantically similar or adjacent keywords. Additionally, the keyword association analysis block (512) can analyze the connection strength or association pattern between multiple keywords to generate new collection keyword candidates or associated topic candidates for expanding the scope of data collection.

[0077] According to one embodiment of the present invention, the coverage gap analysis block (513) may be configured to identify areas of insufficient or gaps in data coverage by analyzing the metadata refined and normalized in the metadata preprocessing and normalization block (511). Specifically, the coverage gap analysis block (513) can identify areas where data is excessively concentrated in a specific area or, conversely, areas where data is insufficient by analyzing the data distribution by keyword, the amount of data collected by category, the data density by topic, the degree of data accumulation by time interval, or the degree of data bias by source.

[0078] For example, the coverage gap analysis block (513) can determine that a specific area is a data gap area requiring supplementary collection if data is concentrated in some keywords or specific topics within a specific category and there is insufficient data in other related areas. Additionally, the coverage gap analysis block (513) can identify a specific period of data as missing or insufficient collection of specific external data sources as a coverage gap from a time-series perspective.

[0079] According to one embodiment of the present invention, the demand and trend analysis block (514) may be configured to identify changes in data demand and trends by topic by analyzing metadata refined and normalized in the metadata preprocessing and normalization block (511). Specifically, the demand and trend analysis block (514) may derive topics or keywords requiring expanded data collection in the future by analyzing temporal changes in the frequency of appearance by keyword, increases or decreases in the volume of mentions for a specific topic, data growth trends by category, patterns of issue diffusion by source, or time-series-based change patterns.

[0080] For example, the demand and trend analysis block (514) can determine that if the volume of mentions for a specific keyword or specific topic increases within a short period, it is a signal of increased demand or rising interest, and if a continuous data increase trend appears in a specific category, it can identify that category as a candidate for expanded collection. Additionally, the demand and trend analysis block (514) can predict keywords or topics that are likely to see increased demand in the future by using time series analysis or trend prediction techniques.

[0081] According to one embodiment of the present invention, an event-based keyword generation block (515) may be configured to generate new keyword candidates for data collection expansion based on event information or issue information input from the outside. Here, the event information may include social issues, policy changes, disaster occurrences, changes in industrial trends, issues related to specific services or products, seasonal events, or real-time trend information. The event-based keyword generation block (515) may extract keywords, related words, or derived words included in such event information, or generate new collected keywords related to the event using pre-set rules or natural language processing techniques. For example, when a specific external event occurs, the event-based keyword generation block (515) may derive not only keywords directly related to the event, but also keywords related to related topics, derived issues, or corresponding categories.

[0082] According to the present invention, the expansion candidate signal generation block (516) may be configured to generate an expansion candidate signal for data collection expansion by synthesizing the analysis results provided from the keyword association analysis block (512), the coverage gap analysis block (513), the demand and trend analysis block (514), and the event-based keyword generation block (515). Specifically, the expansion candidate signal generation block (516) may collect new keyword candidates, related topic candidates, data gap area information, demand increase target information, or event-response keyword information derived from each block, and structure them into a signal form suitable for subsequent processing. For example, the expansion candidate signal generation block (516) may group identical or similar candidates among the candidates derived from a plurality of analysis blocks, or provide additional information to each candidate, such as candidate type, basis of occurrence, related category, related external data source, or whether priority review is required.

[0083] FIG. 4 is a block diagram showing the detailed configuration of a user behavior analysis module (540) according to one embodiment of the present invention.

[0084] According to one embodiment of the present invention, a user behavior data preprocessing block (541) may be configured to refine and standardize user behavior data provided from a data portal (100) into a form that allows for subsequent analysis. Here, the user behavior data may include user search keywords, search request history, click history, navigation path, category movement pattern, view frequency, dwell time, or repeated usage history. The user behavior data preprocessing block (541) may provide consistent analysis criteria by correcting for differences in data format, differences in recording methods, or discrepancies in time information collected from different types of user behavior.

[0085] For example, the user behavior data preprocessing block (541) can perform standardization of search term notation formats, structuring of click events and view events, reconstruction of user behavior by session, alignment of time information, removal of duplicate logs, correction of missing information, or removal of unnecessary logs. Additionally, the user behavior data preprocessing block (541) can normalize the user behavior data into a form that allows for search patterns, search flows, or demand-coverage comparison for subsequent analysis.

[0086] According to one embodiment of the present invention, the search pattern analysis block (542) may be configured to identify user search patterns by analyzing user behavior data that has been refined and normalized in the user behavior data preprocessing block (541). Specifically, the search pattern analysis block (542) can identify topics or keywords of which user interest is concentrated by analyzing the frequency of appearance of search keywords entered by the user, whether repeated searches are performed, changes in searches over time, associations between consecutive search terms, or the intensity of searches during a specific period.

[0087] For example, the search pattern analysis block (542) can determine that a specific search keyword is a keyword of interest with high user demand if the keyword is entered repeatedly or increases rapidly within a short period of time. Additionally, the search pattern analysis block (542) can analyze the correspondence between user search keywords and data currently held within the active metadata-based data collection control system (10) to identify keywords or topics for which search demand exists but sufficient data has not been secured.

[0088] According to one embodiment of the present invention, the search path analysis block (543) may be configured to identify the user's search path by analyzing user behavior data that has been refined and normalized in the user behavior data preprocessing block (541). Specifically, the search path analysis block (543) can identify how the user expands their interest from a specific topic to an associated or derived topic by analyzing the user's category movement order, data selection order, consecutive click flow, page transition pattern, or in-session search flow.

[0089] For example, the search path analysis block (543) can determine that the movement path is a topic-linked search pattern when a user repeatedly moves to other related categories or associated data after viewing data of a specific keyword or specific category. Additionally, the search path analysis block (543) can analyze the search flow that appears commonly among multiple users to identify an extended area of ​​interest derived from a specific topic or an area of ​​associated data that needs to be collected together.

[0090] According to one embodiment of the present invention, the demand-coverage gap analysis block (544) may be configured to analyze the difference between user demand and current data coverage based on user behavior data refined and normalized in the user behavior data preprocessing block (541). Specifically, the demand-coverage gap analysis block (544) may compare the level of user interest indicated by user search keywords, click history, search frequency, view concentration, or repeated usage patterns with the distribution status of currently secured collected data and metadata within the active metadata-based data collection control system (10).

[0091] For example, the demand-coverage gap analysis block (544) can identify an area with a demand-coverage gap if, despite high user search and view demand for a specific keyword or specific topic, the amount of collected data corresponding to the keyword or topic is insufficient or the range of related data is narrow. Additionally, the demand-coverage gap analysis block (544) can determine that an area requires supplementary collection even if, despite repeated user access to a specific category or specific detailed topic, sufficient data corresponding to that area is not provided.

[0092] According to one embodiment of the present invention, the keyword expansion block (545) may be configured to generate keyword candidates for data collection expansion based on analysis results provided from the search pattern analysis block (542), the search path analysis block (543), and the demand-coverage gap analysis block (544). Specifically, the keyword expansion block (545) may generate new collection keywords, related keywords, derived keywords, or expansion target topic candidates by combining interest keywords identified based on user search patterns, related topic candidates derived based on user search paths, and supplementary collection target areas identified based on demand-coverage gap analysis results.

[0093] Additionally, the keyword expansion block (545) can expand the scope of data collection more broadly by generating synonyms, superordinate concepts, subordinate concepts, or contextually related expansion words. For example, the keyword expansion block (545) can derive not only the search term itself where user demand is concentrated, but also related topics explored together with the search term or derived keywords related to areas where data is lacking. Accordingly, the keyword expansion block (545) can generate expansion keyword candidates based on user behavior data and output them to the data expansion judgment module (430), and the data expansion judgment module (430) can determine the necessity and priority of data collection expansion based on the output keyword candidates.

[0094] The scope of the present invention is not limited to the embodiments described above but may be implemented in various forms of embodiments within the scope of the appended claims. It is deemed that the scope of the claims of the present invention includes various modifications that are possible by anyone with ordinary knowledge in the technical field to which the invention pertains, without departing from the essence of the invention claimed in the claims. Explanation of the symbols

[0095] 10: Active Metadata-Based Data Collection Control System 100: Data Portal 110: Portal Server 120: Portal User Interface 200: External data source 210: SNS 220: News 230: Blog 240: Data Mart 250: Data Portal 300: Data Collection Unit 400: Trigger control unit 410: Trigger Management Department 420: Trigger storage 430: Collected Data Extension Decision Module 500: Data Management Department 510: Active Metadata Analysis Module 511: Metadata Preprocessing and Normalization Block 512: Keyword Association Analysis Block 513: Coverage Gap Analysis Block 514: Demand and Trend Analysis Block 515: Event-based keyword generation block 516: Extended Candidate Signal Generation Block 520: Metadata Management Module 530: Real Data Management Module 540: User Behavior Analysis Module 541: User Behavior Data Preprocessing Block 542: Search Pattern Analysis Block 543: Search Path Analysis Block 544: Demand-Coverage Gap Analysis Block 545: Keyword Expansion Block S110: Step to perform metadata and user behavior analysis S120: Extended candidate signal generation step S130: Extended candidate signal integration analysis and priority determination step S140: Trigger Event Generation Step S150: Trigger Event-based Data Collection Workflow Execution Steps

Claims

Claim 1 An active metadata-based data collection control device comprising: one or more processors; and a memory for storing instructions executed by the one or more processors; wherein the one or more processors, by executing the instructions, acquire collected data from an external data source, manage metadata and user behavior data based on the collected data, generate an extension candidate signal for data collection extension based on the metadata and the user behavior data, generate a Trigger Event based on the extension candidate signal, and control a data collection workflow based on the Trigger Event. Claim 2 An active metadata-based data collection control device according to claim 1, wherein the one or more processors are configured to analyze the metadata by executing the instruction to derive an extension candidate corresponding to at least one of an associated keyword, an extension target category, and a data coverage gap area. Claim 3 An active metadata-based data collection control device according to claim 1, wherein the one or more processors are configured to analyze the user behavior data by executing the instruction to derive an extension candidate corresponding to at least one of a user interest topic, an uncollected keyword, an associated topic, and a data shortage area relative to user demand. Claim 4 An active metadata-based data collection control device according to claim 1, wherein the one or more processors are configured to integrate and analyze the extension candidate derived based on the metadata and the extension candidate derived based on the user behavior data when generating the extension candidate signal by executing the instruction. Claim 5 An active metadata-based data collection control device according to claim 4, wherein the one or more processors are configured to determine a priority for the integratedly analyzed expansion candidates based on at least one of the level of data demand, user interest, degree of data coverage insufficiency, external event correlation, necessity of expansion, and expected utilization by executing the instruction. Claim 6 An active metadata-based data collection control device according to claim 1, wherein the one or more processors are configured to generate the Trigger Event by executing the instruction, such that the Trigger Event includes at least one of information regarding a new collection keyword, an expanded target category, an external data source for collection, a collection range, a collection time, a collection cycle, a priority, and an execution condition. Claim 7 An active metadata-based data collection control device according to claim 1, wherein the one or more processors are configured to perform at least one of adding new collection keywords, adding expansion target categories, changing external data sources to be collected, changing the collection time, changing the collection cycle, and expanding the collection scope when controlling the data collection workflow by executing the command. Claim 8 An active metadata-based data collection control device according to claim 1, wherein the one or more processors are configured to reconstruct a data collection strategy by executing the instruction, thereby reflecting newly collected data and corresponding metadata in subsequent analysis in accordance with the execution of the data collection workflow. Claim 9 A data collection control method performed by an active metadata-based data collection control device comprising one or more processors, comprising: an acquisition step of acquiring collected data from an external data source; an analysis step of analyzing metadata and user behavior data generated or extracted from the collected data; an expansion candidate signal generation step of generating an expansion candidate signal for data collection expansion based on the analysis results of the metadata and the analysis results of the user behavior data; a determination step of performing integrated analysis of the expansion candidate signal and determining a priority; a trigger event generation step of generating a trigger event based on the expansion candidate signal with the determined priority; and an execution step of executing or re-executing a data collection workflow based on the trigger event. Claim 10 In claim 9, the analysis step comprises the step of analyzing the metadata to derive an extension candidate corresponding to at least one of an associated keyword, an extension target category, and a data coverage gap area; a method. Claim 11 In claim 9, the analysis step comprises the step of analyzing the user behavior data to derive an extension candidate corresponding to at least one of user interest topics, uncollected keywords, related topics, and areas with insufficient data relative to user demand; a method. Claim 12 In claim 9, the judgment step comprises: a step of mutually comparing an extension candidate generated based on the analysis result of the metadata and an extension candidate generated based on the analysis result of the user behavior data to identify duplicate or similar candidates, and integrating and analyzing them to determine the priority. Claim 13 In claim 9, the judgment step comprises determining the priority of the expansion candidate signal based on at least one of the level of data demand, user interest, degree of data coverage insufficiency, correlation with external events, necessity of expansion, and expected utilization; a method. Claim 14 In claim 9, the Trigger Event generation step comprises: generating a Trigger Event including at least one of information regarding a new collection keyword, an expanded target category, an external data source for collection, a collection scope, a collection time, a collection cycle, a priority, and execution conditions; a method. Claim 15 In claim 9, the execution step comprises the step of performing at least one of adding new collection keywords, adding expansion target categories, changing external data sources to be collected, changing the collection time, changing the collection cycle, and expanding the scope of collection; a method.

Images