System and method for cohort analysis and reliability grade-linked automatic report generation using natural language processing ai agent

Abstract

A system and method for automatically generating reports are disclosed, which automatically perform cohort analysis using a natural language processing AI agent and automatically generate reliable customized reports. To this end, the present invention provides a system for automatically generating reports comprising: a database in which a plurality of health data is stored; a data standardization unit that converts the plurality of health data stored in the database into the structure of a pre-configured common data model; an AI agent that receives an analysis request input in natural language, determines an analysis purpose, defines a cohort corresponding to the analysis purpose, selects an analysis technique corresponding to the defined cohort, generates an analysis code corresponding to the selected analysis technique, and executes it on data of the common data model structure; a reliability evaluation unit that calculates a reliability grade for the execution result of the AI ​​agent and controls subsequent processing; and a report generation unit that generates a report whose structure is determined based on at least one of the analysis purpose, the analysis technique, and the reliability grade. According to the present invention, even a general user without programming or statistical expertise can extract patient groups with desired conditions from vast medical and health data and rapidly perform advanced analysis using only natural language queries.

Description

Technology Field

[0001] The present invention relates to a technology for health data analysis and report provision, and more specifically, to a system and method for automatically generating reports that converts heterogeneous source health data into a standardized common data model and automatically performs cohort analysis using a natural language processing AI agent, controls the language model of the AI ​​agent to operate a deterministic analysis engine only through the invocation of a verified tool without directly generating analysis code, calculates a reliability rating based on multiple indicators for the analysis results to differentially control the scope of exposure and subsequent processing methods of the results, and links this with insurance product evaluation, etc., to automatically generate safe and reliable customized reports. Background Technology

[0003] With the recent rapid growth of the digital healthcare industry alongside the Fourth Industrial Revolution, vast amounts of health data, such as electronic medical records (EMR), health checkup data, genomic data, and life-logs from medical institutions, are being accumulated explosively.

[0004] Beyond simple disease treatment, such health data is being utilized as a key decision-making resource across various industries, including predicting the risk of specific diseases, identifying patient cohort characteristics, and even planning and developing new insurance products.

[0005] However, real-world health data has a structural limitation in that database schemas and code systems differ depending on the source institutions or systems from which they are collected. For example, heterogeneous data is scattered, with Hospital A using its own proprietary medical codes, Hospital B using standardized KCD (Korean Standard Classification of Diseases and Causes of Death) codes, and insurance companies primarily using EDI billing codes.

[0006] In order to integrate and analyze such fragmented data, data must be cleaned and preprocessed for mapping whenever an analysis objective arises; this not only consumes enormous amounts of time and cost but also makes it difficult to guarantee data integrity. To address this, common data models such as the OMOP-CDM (Observational Medical Outcomes Partnership Common Data Model) are being introduced worldwide, but consistently transforming vast amounts of source data in bulk at the standardization layer remains a highly technical challenge.

[0007] Even setting aside data standardization issues, the process of deriving meaningful analysis results from an established database also presents a very high barrier to entry. For traditional cohort analysis, analysts must define patient eligibility criteria based on medical and statistical knowledge and implement them using complex SQL queries or statistical programming languages ​​such as R or Python. While UI-based tools like OHDSI ATLAS exist for defining cohorts, the learning curve is still very steep, making it difficult for general users—such as insurance product planners or executives—to handle intuitively.

[0008] With the recent advancements in artificial intelligence (AI) and natural language processing (NLP) technologies, tools such as Criteria2Query and OMOP Navigator have emerged that convert users' natural language queries into database queries. However, most of these prior technologies remain limited to 'search and extraction' functions, which mechanically translate user questions into SQL queries to return results. In the medical and insurance domains, data analysis requires a process that goes beyond simply extracting numbers to evaluate whether the results are statistically reliable and clinically significant.

[0009] In actual analysis environments, data quality issues frequently occur, such as the sample size of the extracted cohort being too small, the number of variables being insufficient compared to the number of events input into the analysis model, a high number of missing values ​​in key variables, or severe data bias toward specific regions and medical institutions.

[0010] Existing natural language-based analysis tools had a limitation in that they exposed query execution results to users as is, without verifying the 'reliability of the analysis results' at all.

[0011] Furthermore, if large language models (LLMs) are left to freely access databases and reason on their own, two types of errors occur in the field of insurance product development, where numerical accuracy is required.

[0012] The first error is hallucination, in which the language model generates unfounded patient numbers, ratios, or statistics based on prior knowledge, or states that it performed methodologies that were not actually carried out. In particular, falsified figures that closely approximate the total true value are difficult to distinguish through visual inspection.

[0013] The second error is non-determinism, which occurs when a language model is allowed to generate free-form queries; since the definition of the denominator, left-truncation, and decimal cell handling methods vary with each call, irreproducible and different figures are produced for the same query. This non-determinism cannot be directly applied to actuarial and argument judgments, which require statistical certainty.

[0014] Therefore, there is a need to develop a report generation technology capable of structurally blocking the aforementioned illusions and non-determinism through a control structure that goes beyond the level of translating natural language queries into analysis code, limits the behavioral space of the language model to calls to verified tools, delegates the generation of analysis code to a deterministic compiler, and verifies the output figures by comparing them with actual data.

[0015] The limitations of such prior art pose significant risks in the business environment. For example, if an insurance product planner calculates premium rates based on 'incidence rates of specific disease groups' derived through natural language queries, even though these results are derived from a small amount of data that is coincidentally biased, the insurer may suffer massive financial losses or face bad debt risks.

[0016] Therefore, there is an urgent need to develop an integrated intelligent report generation system that supports non-experts in defining cohorts and executing analyses using only natural language, based on an infrastructure capable of batch converting heterogeneous source data into standard models. Simultaneously, by automatically evaluating the statistical and clinical reliability of derived analysis results and controlling the exposure of unreliable outcomes or suggesting alternatives, this system can prevent structural errors in data-driven decision-making in advance. Prior art literature

[0018] Republic of Korea Published Patent No. 10-2021-0098048 (Published Aug. 10, 2021) Republic of Korea Registered Patent No. 10-2742546 (Announced Dec. 16, 2024) Republic of Korea Published Patent No. 10-2024-0049613 (Published Apr. 16, 2024) Republic of Korea Registered Patent No. 10-1885111 (Announced Aug. 3, 2018) Republic of Korea Registered Patent No. 10-1923654 (Announced Nov. 29, 2018) The problem to be solved

[0019] Accordingly, the first objective of the present invention is to provide a cohort analysis and reliability-based report automatic generation system that standardizes different source health data into a common data model structure, allows an AI agent to automatically perform tasks ranging from cohort definition to code execution in response to an analysis request input in natural language, limits the language model of the AI ​​agent to the invocation of a typed tool, delegates the generation of analysis code to a deterministic compiler, and verifies the calculated values ​​by comparing them with actual data to prevent errors caused by the illusion and non-determinism of the language model, and differentially controls result exposure and subsequent processing according to the reliability grade based on multiple indicators of the execution results.

[0020] Furthermore, the second objective of the present invention is to provide a method for cohort analysis and automatic reliability-based report generation that supports users lacking expertise in data analysis in safely obtaining only verified analysis results and preventing the risk of relying on biased or erroneous data by specifying a methodology from the analysis flow initiated by a user's natural language query through a systemic safety mechanism called reliability verification to the visualization of the final report. means of solving the problem

[0022] To achieve the first objective of the present invention described above, one embodiment of the present invention provides a report automatic generation system comprising: a database in which a plurality of health data collected from different sources and having different data structures are stored; a data standardization unit that converts the plurality of health data stored in the database into the structure of a pre-set common data model; an AI agent that receives an analysis request input in natural language, determines an analysis purpose, defines a cohort corresponding to the analysis purpose, selects an analysis technique that matches the defined cohort, generates an analysis code corresponding to the selected analysis technique, and executes it on data of the common data model structure; a reliability evaluation unit that calculates a reliability grade for the execution result of the AI ​​agent and controls subsequent processing; and a report generation unit that generates a report whose structure is determined based on at least one of an analysis purpose received from the AI ​​agent, an analysis technique, and a reliability grade received from the reliability evaluation unit.

[0023] In addition, to achieve the second objective of the present invention, one embodiment of the present invention provides a method for automatically generating a report, comprising: a standardization step in which a data standardization unit extracts a plurality of health data collected from different sources and having different data structures from a database and converts them into the structure of a pre-configured common data model; an agent control step in which an AI agent receives an analysis request input in natural language, determines an analysis purpose based on the received analysis request, defines a cohort corresponding to the determined analysis purpose, selects an analysis technique for the defined cohort, generates an analysis code corresponding to the selected analysis technique, and executes it on the data of the converted common data model structure stored in the database; a reliability evaluation step in which a reliability evaluation unit calculates a reliability grade for the execution result of the agent control step and controls subsequent processing; and a report generation step in which a report generation unit automatically generates a report whose structure is determined based on at least one of the analysis purpose transmitted in the agent control step, the analysis technique, and the reliability grade calculated in the reliability evaluation step. Effects of the invention

[0025] According to the present invention, technical entry barriers to data analysis can be effectively overcome by integrating the entire process—from the batch standardization of fragmented heterogeneous source health data to cohort definition and analysis execution via natural language-based AI agents, as well as result reliability verification and customized report generation—into a single intelligent pipeline. Through this, even general users without programming or statistical expertise can extract patient groups meeting desired conditions from vast medical and health data and rapidly perform advanced analysis using only natural language queries.

[0026] Furthermore, the present invention goes beyond merely deriving and providing analysis results based on user queries; it can automatically control subsequent processing by evaluating the statistical and clinical reliability grades of the derived results and restricting information exposure or suggesting alternatives based on those levels. Through this organic control flow, it prevents incomplete results containing statistical errors, such as insufficient sample size or data bias, from being provided to users without filtering. Ultimately, this prevents the risk of users making critical decisions based on incorrect data and enhances the safety of data utilization.

[0027] Furthermore, by immediately providing a final report that comprehensively reflects the user's initial analysis intent, appropriate analysis techniques, and reliability information strictly verified and controlled by the system, the present invention enables the user to make safe and efficient data-driven decisions based solely on verified insights without errors.

[0028] Furthermore, the present invention isolates the language model of an AI agent as a conversational orchestrator that invokes only predefined tools, rather than as an analyst who freely handles data; generates analysis code in a parameterized form using a deterministic compiler; and provides all calculated figures after verification by comparing them with actual data. This prevents the generation of baseless figures and unreproducible results that occur in the absence of a control structure, even when using the same language model and data, and has the effect of providing reproducible and verified analysis results.

[0029] Furthermore, the present invention ensures reproducibility by recording the specifications used in the analysis, the compiled query, and the hash and confidence judgment of the result values ​​together, thereby verifying that the same values ​​are produced when re-executed for the same input. Brief explanation of the drawing

[0031] FIG. 1 is a block diagram schematically showing the integrated configuration of a report automatic generation system according to one embodiment of the present invention. FIG. 2 is a block diagram schematically showing the integrated configuration of a report automatic generation system according to another embodiment of the present invention. FIG. 3 is a flowchart showing the deterministic processing flow in which a data standardization unit according to an embodiment of the present invention converts source data into the structure of a common data model. FIG. 4 is a conceptual diagram showing the processing structure and results of a controlled language model and an uncontrolled language model according to one embodiment of the present invention. FIG. 5 is a diagram showing the processing flow in which a natural language analysis request is converted into a parameterized query in the harness structure of an AI agent according to one embodiment of the present invention. FIG. 6 is a conceptual diagram showing the configuration of a triple line of defense controlling language model inference of an AI agent according to one embodiment of the present invention. FIG. 7 is a flowchart illustrating the basis verification and answer regeneration process of an output verifier according to one embodiment of the present invention. FIG. 8 is a diagram showing the learning and inference steps of a risk prediction model according to one embodiment of the present invention. FIG. 9 is a flowchart illustrating a method for automatically generating a report according to the present invention. FIG. 10 is a flowchart showing the detailed sub-processes of the agent control step according to the present invention and the control flow when preconditions are not met. FIG. 11 is a flowchart showing the detailed sub-processes and bias detection logic of the reliability evaluation step according to the present invention. FIG. 12 is a flowchart showing detailed sub-processes of the report generation step according to the present invention. Specific details for implementing the invention

[0032] Hereinafter, a system for automatically generating cohort analysis and reliability grade linked reports using a natural language processing AI agent according to preferred embodiments of the present invention (hereinafter abbreviated as "automatic report generation system") will be described in detail with reference to the attached drawings.

[0034] FIG. 1 is a block diagram schematically showing the integrated configuration of a report automatic generation system according to one embodiment of the present invention, and FIG. 2 is a block diagram schematically showing the integrated configuration of a report automatic generation system according to another embodiment of the present invention.

[0035] Referring to FIG. 1, the automatic report generation system according to the present invention comprises a database (100) that loads heterogeneous non-standard health data collected from different sources, a data standardization unit (200) that standardizes and converts the health data into a common data model (CDM) structure capable of integrated analysis, an AI agent (300) that interprets a natural language analysis request to dynamically determine a cohort and analysis technique and automatically executes an analysis code, a reliability evaluation unit (400) that calculates a reliability grade by synthesizing statistical and data integrity indicators of the execution results and determines the overall control flow, and a report generation unit (500) that automatically generates a report with a determined exposure range and structure based on the reliability grade and analysis metadata.

[0036] In addition, the present invention may optionally further include a knowledge management unit (600) that integrates and manages metadata and system knowledge information to assist in data standardization and agent computation as shown in FIG. 2, and a product evaluation unit (700) that calculates business value by calculating the size of the insured customer group and the risk rate based on the execution results of the AI ​​agent (300).

[0038] Below, each component will be described in more detail with reference to the drawings.

[0039] Referring to FIGS. 1 and FIGS. 2, the automatic report generation system according to the present invention includes a database (100).

[0040] The above database (100) is a physical or logical storage medium in which multiple health data collected from different sources and having different data structures are stored.

[0041] The above different sources include servers of heterogeneous medical institutions, public health institution terminals, or private wearable devices.

[0042] The source data loaded into the above database (100) includes electronic medical records (EMR), health checkup results, and prescription history, and for each source, the table structure, column names, and diagnostic code system including the Korean Standard Classification of Diseases and Causes of Death (KCD) or the International Classification of Diseases (ICD) are stored in a non-standardized state that differs from one another.

[0043] The above database (100) stores standard data converted into the structure of a common data model by the data standardization unit (200). At this time, the converted standard data can be stored based on a lakehouse architecture that combines the advantages of a data lake and a data warehouse so that large-scale analysis, cohort generation, and data history management can be efficiently performed. In this case, the AI ​​agent (300) reads the standard data stored in the database (100) and uses it as the target for execution of analysis code.

[0044] In a specific embodiment, the system according to the present invention has a database (100) in which data to be analyzed is stored, and a conversation memory database in which business data including session information, conversation history, and reports is stored, which are physically or logically separated from each other.

[0045] The above database (100) is used only for statistical calculation of standardized data, and the conversation memory database is used for managing the conversation context. Through this separation, the resulting numerical value of the statistical calculation is structurally prevented from being mixed into and contaminated by the conversation context.

[0047] Referring to FIGS. 1 and FIGS. 2, the automatic report generation system according to the present invention includes a data standardization unit (200).

[0048] The above data standardization unit (200) is a computing device that extracts multiple health data stored in the database (100) and converts them into the structure of a pre-set Common Data Model (CDM). Here, the Common Data Model includes the OMOP-CDM (Observational Medical Outcomes Partnership Common Data Model) structure for multi-institutional observational studies.

[0049] The above data standardization unit (200) converts the patient identification information of the source data into a PERSON table, converts the diagnosis information into a CONDITION_OCCURRENCE table, converts the prescription information into a DRUG_EXPOSURE table, and converts the test value information into a MEASUREMENT table.

[0050] And the data standardization unit (200) converts the examination and screening information of the source data into a MEASUREMENT table, converts the medical institution visit and treatment utilization into a VISIT_OCCURRENCE table, converts the procedure information into a PROCEDURE_OCCURRENCE table, and converts the observation information into an OBSERVATION table.

[0051] Additionally, the data standardization unit (200) maps the source KCD or ICD code to a standard code of a standard vocabulary system including SNOMED-CT, RxNorm, or LOINC.

[0052] The data standardization unit (200) is configured to calculate a mapping reliability that indicates the certainty of the correspondence relationship between a source code and a standard code for each data item being converted when converting heterogeneous health data into the structure of a common data model. Here, the mapping reliability is calculated by normalizing it to a value between 0 and 1 based on at least one of the following: whether the source code is queried in a direct correspondence relationship in the crosswork table of the standard vocabulary system, the number of join steps of the bridge table traversed to reach the correspondence relationship, and whether multiple standard codes corresponding to a single source code are queried. At this time, since the calculation of the mapping reliability is performed by a deterministic operation based on a pre-established rule rather than by inference from a learned model, the same mapping reliability is always calculated for the same source code.

[0053] The above data standardization unit (200) suspends automatic conversion for items where the calculated mapping reliability is below a preset threshold value, such as 0.7 to 0.8, and generates an isolation flag to classify the item as a subject for administrator review. Here, the item to which the isolation flag is assigned is notified to the administrator terminal and is reflected in the conversion after undergoing manual mapping or correction of the mapping rule.

[0054] Additionally, the data standardization unit (200) comprises a procedure for converting source data into the structure of a common data model, which is performed according to a predetermined rule rather than by a learning model that performs inference at runtime, as a deterministic extraction, transformation, and loading (ETL) procedure. This deterministic extraction, transformation, and loading procedure is performed in batches at the time of loading the source data, and subsequent analysis requests are processed on the standard data that has been converted.

[0055] More specifically, the data standardization unit (200) maps the Korean Standard Classification of Diseases and Causes of Death (KCD) codes for diagnostic items to the standard concept of SNOMED-CT using the correspondence relationship defined in the crosswork of the standard vocabulary system, connects the drug names for drug items to the Anatomical Therapeutic Chemical Classification (ATC) code through a bridge table that passes through the institution's internal drug code system, and connects the medical benefit claim codes for procedure items to the standard concept of SNOMED-CT through a table join that passes through the Electronic Data Interchange (EDI) code.

[0056] At this time, the data standardization unit (200) does not discard items that do not correspond to the standard code of the standard vocabulary system during the conversion process, but preserves them as local concepts assigned a pre-set identifier range and loads them into the common data model. Through this, traceability with the source data is maintained even for source items that are not standardized, and the item is prevented from being omitted from the calculation of the denominator in subsequent analysis.

[0057] In addition, the data standardization unit (200) performs an integrity check to verify temporal consistency before loading the converted standard data into the database (100). This integrity check detects at least one of the following as a time inconsistency: when the diagnosis date, prescription date, or visit date is prior to the subject's birth date; when the end date of an event is prior to the start date of an event; and when an event exists outside the observation period. The detected time inconsistency items are isolated from loading into the standard table, recorded in a separate isolation area, notified to the administrator terminal, and reflected after undergoing verification of the source data or correction of the mapping rules.

[0058] The above data standardization unit (200) stores the converted standard data in the database (100).

[0060] Referring to FIG. 2, the automatic report generation system according to the present invention may further include a knowledge management unit (600).

[0061] The knowledge management unit (600) is a module that stores original data structure information including column meanings and code systems of non-standardized tables stored in the database (100). Additionally, the knowledge management unit (600) collects and integrates metadata including classification rules, mapping rules, reliability judgment criteria, and report templates from an administrator terminal or a pre-configured external system.

[0062] Specifically, the knowledge management unit (600) provides a computational baseline for the AI ​​agent (300) by explicitly managing definitions of disease groups and drug groups, cohort definition rules for analysis purposes, and mapping rules between analysis purposes and statistical techniques.

[0063] The knowledge management unit (600) provides the stored data structure information and updated mapping rules to the data standardization unit (200) to support conversion to a common data model.

[0064] The knowledge management unit (600) provides the stored classification rules, reliability judgment criteria, and report templates to the AI ​​agent (300), reliability evaluation unit (400), and report generation unit (500), respectively, so that they are utilized as operation criteria for each component.

[0065] In a specific embodiment, the knowledge management unit (600) according to the present invention manages a technique rule table that maps an analysis purpose to a statistical technique, a cohort template, and a precondition guard as a single source of code constants that cannot be arbitrarily changed by the analyst.

[0066] This technique rule table maps precondition guards regarding left-truncation and selection bias to incidence analysis, and precondition guards regarding the number of events relative to the number of model variables to risk prediction, and the precondition guards are exposed via dialogue prior to specification approval.

[0067] Additionally, the knowledge management unit (600) distinguishes and manages requests that cannot be answered with data and requests that can only be approximated. More specifically, requests including death and survival, fixed values ​​of insurance premiums, individual unit probabilities, and cross-combination of examination information and diagnostic information belonging to different collection paths are rejected as unsupported, and requests including age standardization and multivariate disturbance correction are explicitly notified of the alternative method actually applied. These support boundaries are not treated as silent defaults but are explicitly notified to the user.

[0069] Referring to FIGS. 1 and FIGS. 2, the automatic report generation system according to the present invention includes an AI agent (300).

[0070] The above AI agent (300) is a control module that receives an analysis request input in natural language from a user terminal (800) and determines an analysis purpose based on the received analysis request through a linked natural language processing engine. Here, the natural language processing engine includes a generative language model and interprets the meaning of the analysis request to derive an analysis purpose including a target disease, a target group, and a type of analysis.

[0071] Here, the generative language model of the natural language processing engine is composed of a commercial large-scale language model called through an external application programming interface (API). In one embodiment, the generative language model is a GPT-4.1 family model, and the AI ​​agent (300) calls the model through a conversational language model call interface, but the target endpoint and authentication key are injected as environment variables of the execution environment and are not fixed in the code. Accordingly, even when the base language model is replaced or updated, the definition of the tool function, the deterministic compiler, and the configuration of the output verifier described below are not changed. The AI ​​agent (300) according to the present invention does not require an open-source language model installed on its own server or a separately fine-tuned language model, and by limiting the behavioral space of the language model to the harness structure described below, it ensures the reproducibility of the output value while using a general-purpose commercial language model.

[0072] Specifically, the natural language processing engine quantitatively identifies the purpose of the analysis by extracting an intent slot from the received analysis request that includes at least one of age, gender, reference disease, result event or indicator, observation period, and comparison or exclusion condition.

[0073] In this case, the determination of the intent slot is not achieved solely through the output of the generative language model, but is accomplished by combining a first process in which the generative language model proposes candidates for the intent slot as arguments to a typed tool function according to instructions declared at the system prompt, and a second process in which a deterministic rule parser re-extracts the value corresponding to the intent slot from the user's source text.

[0074] Here, the deterministic rule parser includes a set of regular expressions corresponding to the notation formats for age ranges, gender, periods, and quantities, respectively, and extracts values ​​specified in the source text to compare with the values ​​in the proposed intent slots. If the two do not match as a result of the comparison, the value re-extracted from the source text is adopted first, and if there are missing items in the proposed intent slots, they are supplemented with the re-extracted values.

[0075] Through this dual extraction structure, the scope of analysis is prevented from varying differently from the user's request by the language model arbitrarily supplementing age ranges or observation periods that do not exist in the source text, and it does not require a separately trained slot extraction model or named entity recognition model.

[0076] The AI ​​agent (300) is interconnected with the knowledge management unit (600) to extract metadata mapped to the determined analysis purpose, and defines a cohort corresponding to the clinical condition to be analyzed based on the extracted metadata. Here, the definition of the cohort includes the process of calculating the inclusion and exclusion conditions of the target group using standard codes of the common data model. At this time, the definition of the cohort is composed of a basic cohort structure among a reference disease group versus a comparison group, an event group versus a non-event group, a comorbidity group within the reference disease group, a target group versus a reference population, and a candidate group for acceptance, according to the determined analysis purpose.

[0077] The AI ​​agent (300) above selects an analysis technique corresponding to a defined cohort. Here, the analysis technique includes at least one of survival analysis such as Kaplan-Meier estimation or Cox proportional hazards regression, logistic regression analysis, calculation of hazard ratio or odds ratio, and cluster analysis.

[0078] More specifically, the AI ​​agent (300) selects a target analysis technique mapped to the extracted intent slot and the analysis purpose according to the pre-set analysis technique determination logic. For example, for the purpose of analyzing the risk of disease occurrence (over time), a Kaplan-Meier estimation or Cox regression technique based on the time-to-event axis is mapped; for the purpose of predicting event retention and occurrence, a logistic regression or machine learning classification model based on the classification axis is mapped; and for the purpose of predicting the size of medical expenses and insurance payouts, a generalized linear model (GLM) or distribution model based on the continuous and distribution axes is mapped. Additionally, for the purpose of estimating ratios, a prevalence and incidence rate calculation technique is mapped; for the purpose of comorbidity, an association analysis or comorbidity network technique is mapped; and for the purpose of estimating market size, an inclusion and extrapolation technique with an acceptance criterion filter applied is mapped, respectively.

[0079] When an analysis technique is selected, the AI ​​agent (300) generates an executable form of analysis code corresponding to the selected analysis technique. Here, the generation of the analysis code is performed by a deterministic compiler included in the AI ​​agent (300), and does not mean that the language model directly writes the analysis code as described below. At this time, the analysis code includes the form of a Python script, an R script, or a structured query language (SQL) query.

[0080] The AI ​​agent (300) reads standard data stored in the database (100) and executes the analysis code.

[0081] The AI ​​agent (300) has logic for checking whether the data characteristic preconditions for the selected analysis technique are satisfied prior to the execution of the analysis code. Here, the preconditions include statistical requirements for each analysis technique. For example, the preconditions include the assumption of normality of the sample and the minimum sample size for the t-test, the assumption of proportional hazards for Cox proportional hazards regression, and the homogeneity of variance between groups for the analysis of variance.

[0082] Furthermore, the aforementioned preconditions include the existence of a time variable, which is essential when applying survival analysis techniques, and whether the Events Per Variable (EPV) criterion is satisfied.

[0083] If the preconditions are met according to the above test results, the AI ​​agent (300) generates and executes analysis code through a deterministic compiler according to the selected analysis technique.

[0084] Conversely, if the preconditions are not met, the AI ​​agent (300) controls the communication interface to switch to a pre-set alternative analysis technique or to impose a penalty on the reliability grade calculation process of the linked reliability evaluation unit (400).

[0085] In a specific manner, the transition to the alternative analysis technique described above includes switching to a nonparametric test including the Mann-Whitney U test or the Wilcoxon signed-rank test when the normality assumption is not met, switching to an accelerated life model or a time-dependent covariate model when the proportional hazards assumption is not met, and applying a dimensionality reduction technique including principal component analysis when the dimensionality of the variable is excessive. Additionally, if the preconditions for survival analysis are not met, the alternative analysis technique is automatically switched to logistic regression analysis.

[0086] FIG. 4 is a conceptual diagram showing a comparison of the processing structure and results of a controlled language model and an uncontrolled language model according to an embodiment of the present invention. The left side of FIG. 4 shows the prior art of an uncontrolled language model, and the right side shows the processing structure of a controlled language model according to the present invention.

[0087] Referring to FIG. 4, the AI ​​agent (300) according to the present invention has a harness structure that controls the language model so that it does not directly generate analysis code. As shown in FIG. 4 (a), when the language model directly generates a freeform query, the definition of the denominator and the application of left truncation vary with each call, resulting in baseless numbers and unreproducible results. On the other hand, as shown in FIG. 5 (b), the AI ​​agent (300) according to the present invention limits the behavioral space of the language model to calls to a plurality of tool functions where the parameters are specified as types, and does not provide a tool to execute arbitrary queries. Accordingly, a string containing a disease name is not directly connected to the query but is replaced with a standard concept identifier after querying the concept registry.

[0088] FIG. 5 is a diagram showing the processing flow in which a natural language analysis request is converted into a parameterized query in the harness structure of an AI agent according to one embodiment of the present invention.

[0089] Referring to FIG. 5, a natural language analysis request in an AI agent (300) according to the present invention is converted into a parameterized query through the following processing flow.

[0090] First, a system prompt in which the aforementioned invariant is declared is combined with a received natural language analysis request and provided as input to a language model. The language model operates in an inference layer that repeats thinking, tool calls, and observations, but is finitely sealed by pre-set recursion and time limits, and records of tool calls remaining without a response are removed through a recovery process.

[0091] The inference layer extracts intent information slots including age, gender, reference disease, result event, and observation period from a natural language analysis request, and determines the purpose of analysis based on the extracted intent information slots. At this time, classification rules, mapping rules, connection tables between purpose and technique, and premise guards stored in the knowledge management unit (600) are provided, respectively, to the combination step of the system prompt, the inference layer, the determination step of the purpose of analysis, and the determination step of the tool call, so that the method is determined by the rules of the knowledge management unit (600) rather than the language model selecting a method based on sentence patterns.

[0092] Next, the above language model determines the call to any one of the tool functions where the parameter is specified as a type, and at this time, a tool for executing an arbitrary query is not provided.

[0093] As a result of the above tool call, a CohortSpec is output, and the deterministic compiler compiles the CohortSpec into fixed query templates and bound parameters to generate a query-only parameterized query. As such, since the output of the language model is limited to the CohortSpec rather than the query, the language model cannot arbitrarily change the definition of the denominator or the application of left truncation.

[0094] The above AI agent (300) includes a deterministic compiler that receives a structured specification (CohortSpec) of a cohort and generates analysis code consisting of a fixed query template and bound parameters.

[0095] The queries generated by such a deterministic compiler are restricted to lookup-only use and executed only through a single parameterized execution path, thereby being configured to structurally make the generation and execution of free-form queries by the language model structurally impossible. In this case, the structured specification includes at least one of a disease identifier, an indicator, an observation period, an event type, a stratification axis, gender and age restrictions, a left-truncation period, and an event confirmation definition.

[0096] Additionally, the AI ​​agent (300) applies a double approval gate to the execution of ratio analysis. More specifically, it includes a first process in which a language model proposes a structured specification, and a second process in which an analysis corresponding to the approved specification is executed only in the turn after the proposed structured specification is approved by the user terminal (800), and an execution request for an invalid specification identifier or an unapproved specification is rejected.

[0097] Furthermore, the AI ​​agent (300) configures the conversation orchestration of the language model into an inference loop that repeats thinking, tool calls, and observations, and finitely seals the inference loop by applying a pre-set recursion upper limit and a pre-set time upper limit. In addition, it performs a recovery process to remove records of tool calls remaining without a response from the conversation history so that the damaged conversation history does not hinder the processing of subsequent requests.

[0098] FIG. 6 is a conceptual diagram showing the configuration of a triple line of defense controlling language model inference of an AI agent according to one embodiment of the present invention.

[0099] Referring to FIG. 6, the control structure of the AI ​​agent (300) according to the present invention includes three lines of defense deployed at different times in the lifecycle of a request.

[0100] The first line of defense, deployed prior to generation, blocks the language model from bypassing the gate through dialogue by declaring an invariant in the language model that cites numerical values ​​only from the tool's results and does not calculate fixed values ​​for individual unit probabilities and premiums.

[0101] The second line of defense deployed during generation makes the generation of free-form queries impossible by providing only parameterized tools and delegating the generation of queries to the aforementioned deterministic compiler.

[0102] A third line of defense deployed after generation compares the values ​​included in the answer with actual data and, if a violation is detected, performs regeneration or replacement to block unverified values ​​from reaching the user terminal (800).

[0103] These three lines of defense share responsibility, so that if one line fails to catch an error, another line detects it.

[0105] FIG. 7 is a flowchart illustrating the basis verification and answer regeneration process of an output verifier according to one embodiment of the present invention.

[0106] Referring to FIG. 7, the AI ​​agent (300) further includes a deterministic output validator that operates after the language model has generated the final answer.

[0107] The above output verifier performs a basis verification by comparing each value included in the answer with the value returned by the tool and the data cell value resulting from the execution of the deterministic compiler, and blocks values ​​that are not compared or fall outside the preset tolerance as having no basis.

[0108] The above-mentioned verification of evidence includes a test that blocks at least one of the following: a non-coherence where the numerator exceeds the denominator, an assertion regarding a parental population that is not actually narrowed, a composite of prices calculated by multiplying incidence rates and treatment costs, and the cross-combination of screening and diagnostic information belonging to different collection channels.

[0109] If the output verifier detects a violation, the AI ​​agent (300) injects correction instructions to regenerate and re-verify the answer, and if the violation is not resolved even through regeneration, the final answer is replaced with a deterministic response that expresses only true facts.

[0110] Additionally, the AI ​​agent (300) may optionally further include a semantic review module that re-evaluates semantic residual errors, request fulfillment, and data truthfulness through an independent language model that the deterministic output verifier fails to capture. The semantic review module is kept in a deactivated state by default to account for the system's latency and processing efficiency, and is provided in a toggle form to enable activation according to preset conditions, thereby functioning as a fourth line of defense to complement the triple line of defense.

[0111] In a specific embodiment, the AI ​​agent (300) according to the present invention has a deterministic analysis engine in which the calculation formula of each analysis technique is implemented in code, and the language model calls the deterministic analysis engine by passing parameters to it.

[0112] In the aforementioned deterministic analysis engine, the prevalence rate is calculated by dividing the number of carriers meeting the event confirmation definition by the observable population, and the incidence rate is calculated by dividing the number of new cases by the observation person-time; the observation person-time is calculated by converting the number of days obtained by dividing each subject's observation interval by year into years and summing them up.

[0113] In this case, a new occurrence of the incidence rate is recognized as new only if the date of the initial diagnosis is after the point in time calculated by adding the said washout period to the start of observation, by applying a washout period to separate a diagnosis immediately after the start of observation from existing cases.

[0114] The above left-truncation period is determined singly according to the priority of session settings, specification settings, and default values. Additionally, the definition of confirmed event includes at least one of the following: being diagnosed more than a preset number of times on different dates, being diagnosed more than a preset number of times within a preset period, and being accompanied by an inpatient visit, thereby preventing overestimation that occurs when a single diagnosis is considered an event.

[0115] In addition, the deterministic analysis engine calculates a Wilson score interval for each prevalence value and, for the relative risks of the comorbidity analysis, calculates a confidence interval based on the Mantel-Haenszel aggregated relative risk stratified by sex and age and the Greenland-Robins variance estimator.

[0116] More specifically, the Wilson score confidence interval is calculated for the prevalence cell according to the following mathematical formula 1 for the number of events k, the number of trials n, the ratio p=k / n, and the pre-set z value.

[0117] [Mathematical Formula 1]

[0118]

[0119] Furthermore, the deterministic analysis engine is configured such that the relative risk of a cell with a small number of events is contracted in the direction of the null value on a logarithmic scale based on the actuarial credibility of the reliability evaluation unit (400), so that when the number of events is insufficient, the adjusted relative risk converges to no signal.

[0120] In addition, the aforementioned deterministic analysis engine derives the cumulative incidence rate by applying an actuarial life table method to calculate effective risk groups and conditional risks within the interval, and calculates the median, interquartile range, and 90th percentile using a percentile function instead of the mean value for medical expense statistics, along with the co-payment rate.

[0121] In addition, it can be configured to calculate a coefficient of determination-based trend equation and next-year extrapolated values ​​by simultaneously fitting linear and log-linear models through least squares regression to the time series trend.

[0122] FIG. 8 is a diagram showing the learning and inference steps of a risk prediction model according to one embodiment of the present invention.

[0123] Referring to Fig. 8, the risk prediction of the deterministic analysis engine is performed by dividing it into a learning phase and an inference phase.

[0124] In the learning step illustrated in Fig. 8 (a), the deterministic analysis engine receives cohort data and model variables as input, fits logistic regression coefficients through iterative reweighted least squares (IRLS), and stabilizes the coefficients by applying Ridge regularization. Subsequently, the area under the curve (AUC) and correction error are calculated for the fitted coefficients to verify the performance of the model, and a grade is determined based on the number of events relative to the number of model variables (EPV).

[0125] In the inference step illustrated in Fig. 8 (b), the deterministic analysis engine applies the coefficients fitted in the learning step to the cohort data to be analyzed to calculate a risk score, divides the calculated risk score into low-risk, medium-risk, and high-risk bands, and calculates the observation rate for each band. At this time, the AI ​​agent (300) according to the present invention does not calculate the probability at the individual level but outputs only the observation rate for each band, and the subsequent processing of the calculated observation rate is controlled according to the grade of the reliability evaluation unit (400).

[0126] Meanwhile, the module performing risk prediction may be configured independently of other output paths of the deterministic analysis engine, and may optionally be configured to be connected to the analysis execution path of the AI ​​agent (300) so that the output result is transmitted to the reliability evaluation unit (400). In this case as well, the learning phase and the inference phase of the risk prediction module are performed separately from each other, and the output of the inference phase is limited to the observation ratio by band.

[0127] The AI ​​agent (300) according to the present invention specifies as a support boundary that it does not perform multivariate disturbance correction and age standardization, limits the adjustment of disturbance to stratification of gender and age, and does not calculate fixed values ​​of individual unit probabilities and premiums.

[0128] The above AI agent (300) further includes a deterministic integrity verifier that re-verifies the algebraic invariant of the report value with code after statistical calculation by the deterministic analysis engine.

[0129] The integrity verifier verifies at least one of whether the sum of subtotals matches the total, whether the estimate is in the range of 0 to 100, whether the confidence interval includes the estimate, whether the cumulative occurrence is monotonic, whether hospitalization and surgery events are a subset of the occurrence events, and whether the number of holders is less than or equal to the population size, and blocks the passage of results that violate the invariant.

[0130] In a specific embodiment, when the AI ​​agent (300) according to the present invention calculates the simplified screening risk rate, it divides the accident observation window and the notification inquiry window so that they do not overlap, using the observation end date for each subject as the anchor, rather than the global reference point. At this time, the claim data is used as a confidence interval only up to a pre-set period, and the inquiry window exceeding the said period is indicated as compressed. In addition, it is specified that the calculated risk rate is not a future prediction but the occurrence rate within the observation window of the corresponding group, and the output subject to verification and comparison is configured to offset the common bias within the observation window by using an adjustment coefficient obtained by dividing the risk rate by the form by the reference risk rate.

[0132] Referring to FIGS. 1 and FIGS. 2, the automatic report generation system according to the present invention includes a reliability evaluation unit (400).

[0133] The reliability evaluation unit (400) is a computing device that controls subsequent processing of the system by calculating a reliability grade based on a plurality of evaluation indicators for the result of executing the analysis code of the AI ​​agent (300). Here, the plurality of evaluation indicators include the statistical significance of the execution result itself, the mapping reliability calculated by the data standardization unit (200), and the penalty score transmitted from the AI ​​agent (300).

[0134] In a specific embodiment, according to the present invention, a plurality of evaluation indicators collected by the reliability evaluation unit (400) may be configured to explicitly include a sample size (N), which is the number of subjects in a cohort; an EPV, which indicates the number of events relative to the number of model variables; a missing rate, which is the ratio of missing values ​​of key variables; a ratio of the median or minimum follow-up period relative to the target window; a standardized mean difference, which indicates the standardized mean difference of covariates; and a distribution bias, which is the proportion occupied by a specific top institution or region.

[0135] The above reliability evaluation unit (400) calculates a comprehensive reliability score by normalizing each of the multiple evaluation indicators and applying a pre-set weight to the weighted sum, and the penalty score is reflected by subtracting it from the comprehensive reliability score.

[0136] The above reliability evaluation unit (400) calculates the calculated overall reliability score by classifying it into one of a plurality of reliability grades according to a plurality of pre-set intervals.

[0137] The calculation of the overall reliability score and reliability grade of the above reliability evaluation unit (400) is performed by calculation using a pre-set threshold and a fixed formula, rather than by inference from a separately learned machine learning model. Accordingly, the same reliability grade is always calculated for the same execution result, and the basis for the grade calculation can be broken down and presented by indicator.

[0138] At this time, the reliability evaluation unit (400) applies a dual evaluation method in combination so that when one decisive indicator is missing, the average value does not obscure it when calculating the overall reliability score and the final reliability grade. Specifically, the reliability evaluation unit (400) combines a first evaluation method that limits the upper limit of the overall reliability grade by first adopting the grade of the indicator that recorded the lowest score among multiple evaluation indicators, and a second evaluation method that calculates the weighted sum of the scores for each indicator to finally determine the overall reliability grade.

[0139] In a specific embodiment, the reliability evaluation unit (400) according to the present invention calculates the reliability of the sample based on the actuarial limited-fluctuation credibility, in addition to the weighted sum-based second evaluation method and the lowest score indicator-priority first evaluation method described above.

[0140] More specifically, the reliability evaluation unit (400) uses z and k derived from a pre-set confidence level and tolerance to determine the number of perfect credit events n full =(z / k)2 Defines and quantifies the reliability of the sample by calculating the partial credit coefficient Z through the following mathematical formula 2 for the number of observed events E.

[0141] [Mathematical Formula 2]

[0142]

[0143] In this case, actuarial credit is used for the calculation and notification of reliability, and is not used for the mixing to determine the price rate.

[0144] In addition, the reliability evaluation unit (400) applies a lowest grade priority method in conjunction with the overall grade upper limit limit by the first evaluation method, by adopting the lowest grade among the cell credit rating and the sample number rating as the overall grade, thereby preventing the lack of a specific indicator from being concealed by the weighted average of the second evaluation method.

[0145] If the above overall grade is the highest grade, the entire report is generated and the marketability evaluation is performed; if it is the middle grade, the report is generated with a warning indication and the marketability evaluation is performed conditionally; and if it is the lowest grade, subsequent processing is controlled so that the exposure of the analysis body is suppressed, alternatives are proposed, and the marketability evaluation is blocked.

[0146] In addition, the reliability evaluation unit (400) performs a directional limit notification that automatically indicates the selection bias and limits of the sample at the top of the report through a fixed phrase derived from the calculated grade.

[0147] Furthermore, the reliability evaluation unit (400) generates a reproducibility specification that records the structured specification used in the analysis, the hash of the compiled query, the hash of the result value, and the gate determination together, thereby enabling verification that the same value is produced when the same input is re-executed.

[0148] In a specific embodiment, the reliability grade according to the present invention is classified into grades 1 to 5, and a higher grade is assigned as the overall reliability score is higher.

[0149] Among the aforementioned multiple reliability grades, a specific grade that has been set is established as a threshold grade that serves as the trigger criterion for subsequent processing. For example, in an embodiment where the reliability grades are classified into three levels (A / B / C grades) of high, medium, and low, in the case of the highest grade, Grade A, the generation of the entire report and the marketability evaluation proceed normally without restriction. On the other hand, in the case of the medium grade, Grade B, the report is generated but a warning is inserted and its direct use for decision-making is restricted, and the marketability evaluation is performed conditionally. In the case of the lowest grade, Grade C, the exposure of the analysis body results within the report is blocked, an alternative technique including at least one of cohort expansion, observation period adjustment, or variable reduction is proposed, and subsequent processing is controlled to restrict the progress of the marketability evaluation.

[0150] Additionally, the reliability evaluation unit (400) is equipped with a bias detection logic that detects a statistical discrepancy between the concentration of distribution by medical institution or region of a cohort defined by the AI ​​agent (300) and the reference population distribution. Here, the reference population distribution includes the resident registration population distribution of Statistics Korea or the distribution of National Health Insurance subscribers.

[0151] The above statistical discrepancy can be calculated based on at least one of the chi-square goodness-of-fit test statistic, Kullback-Leibler divergence (KL divergence), or standardized residual between the distribution of the cohort and the distribution of the reference population.

[0152] If the detected discrepancy exceeds a preset threshold, the reliability evaluation unit (400) lowers the reliability grade by a preset step due to bias in the data sample. Subsequently, the reliability evaluation unit (400) controls the operation of the report generation unit (500) and the AI ​​agent (300) in conjunction to differentially apply at least one of the following: limiting the exposure range of the execution result within the report, blocking the direct use of the execution result for decision-making, and adjusting the subsequent analysis so that the AI ​​agent (300) performs a subsequent analysis using an alternative technique, according to the calculated final reliability grade.

[0154] Referring to FIG. 2, the automatic report generation system according to the present invention may further include a marketability evaluation unit (700).

[0155] The above-mentioned product evaluation unit (700) is an evaluation module that receives the clinical execution results of the AI ​​agent (300) and converts them into product indicators for insurance and financial purposes. Based on prediction data obtained through trend extrapolation of a specific disease, the above-mentioned product evaluation unit (700) quantifies and calculates the size of the target customer group targeting the corresponding cohort, the risk rate by age, and the risk of future medical expenses. In addition to this, it generates an insurance productization possibility evaluation result by synthesizing the possibility of separating high-risk groups, the possibility of selecting low-risk groups, and the possibility of designing coverage as additional evaluation factors.

[0156] The above-mentioned marketability evaluation unit (700) calculates a plurality of signals, including scale, risk separability, the monotonicity of the risk slope by age, and the reliability range, respectively, using a fixed formula, combines them to calculate a marketability score, and applies a value based on the reliability calculated by the reliability evaluation unit (400) as a multiplication upper limit to the combined result. Accordingly, if the reliability is low, the marketability score is structurally limited to a low value.

[0157] In this case, the calculation of marketability indicators, including the size of the target customer group and the calculation of future medical expense risks, is performed by applying a fixed formula to observations calculated by the deterministic analysis engine, rather than running a separately trained machine learning model. Accordingly, the marketability evaluation results are also reproduced as identical values ​​for the same input.

[0159] Referring to FIGS. 1 and FIGS. 2, the automatic report generation system according to the present invention includes a report generation unit (500).

[0160] The above report generation unit (500) is an interface device that generates a report whose structure is determined based on at least one of the analysis purpose and analysis technique information received from the AI ​​agent (300) and the reliability grade received from the reliability evaluation unit (400).

[0161] The above report generation unit (500) receives a report template stored in the knowledge management unit (600) and determines a report structure and template corresponding to at least one of the analysis purpose, analysis technique and reliability grade.

[0162] Additionally, the report generation unit (500) determines the layout and text composition of the report based on the combination of the evaluation results of the product evaluation unit (700) and the reliability grade calculated by the reliability evaluation unit (400).

[0163] In a specific embodiment, the report generation unit (500) according to the present invention generates a report including both clinical analysis results and product evaluation results according to a combination criterion having two axes of reliability grade and product evaluation results, when the reliability grade is a higher grade and the product evaluation result is above a preset standard, conditionally restricts or blocks the reflection of product evaluation results when the reliability grade is below a specific grade, and injects a warning message warning that the execution result is not directly used in decision-making into the template of the report.

[0164] The overall operation flow of the system according to the present invention is described with specific examples as follows.

[0165] In a first embodiment, when a natural language query such as "analysis of the possibility of developing an insurance product for patients with pre-existing conditions in their 50s" is input through a user terminal (800), the AI ​​agent (300) generates a hypertension cohort, analyzes the size by gender and age, comorbidities, hospitalization risk, and medical cost risk, and then subdivides the risk groups. Subsequently, after verification by the reliability evaluation unit (400), the product evaluation unit (700) evaluates the productability and outputs a final report.

[0166] In a second embodiment, when a query such as "analysis of a customer group with a high risk of developing diabetes among hypertensive patients" is input, the AI ​​agent (300) sets the reference cohort as hypertension, the resulting event as diabetes, and the observation period, and then runs a survival analysis technique including Cox proportional hazards regression to generate a report defining a high-risk group.

[0167] In a third embodiment, when a query such as "estimation of the size of potential customers for commercialization among chronic disease patients" is entered, the AI ​​agent (300) performs a distribution analysis on a cohort of patients to calculate potential acquisition candidates and derives a report estimating the market size, thereby enabling even non-experts to immediately obtain advanced insurance and medical statistical insights through text input.

[0169] The following describes the methodological control flow executed chronologically by the system described above. Since each step described below is performed by the components with the same names explained earlier, explanations of overlapping technical details are referenced.

[0171] FIG. 9 is a flowchart illustrating a method for automatically generating a report according to the present invention. Referring to FIG. 9, the method for automatically generating a report according to the present invention includes a standardization step (S100) for reconstructing collected heterogeneous source data into a common standard that facilitates integrated analysis, an agent control step (S200) for processing data by independently determining the analysis target and technique based on a user's natural language query, a reliability evaluation step (S300) for verifying the bias and integrity of the analyzed data and assigning an objective grade to the result, and a report generation step (S400) for dynamically customizing the layout of the final result document and the level of information exposure by synthesizing the derived reliability and the purpose of the analysis.

[0172] In the above standardization step (S100), the data standardization unit (200) extracts multiple health data collected from different sources and having different data structures from the database (100) and converts them into the structure of a pre-set common data model.

[0173] This standardization step (S100) includes the process of calculating the mapping reliability for each data item being converted when the data standardization unit (200) converts the data into the structure of a common data model, and isolating and classifying items for administrator review where the calculated mapping reliability is below a preset threshold value. At this time, the standardized data that has been converted is stored in the database (100).

[0174] Additionally, the standardization step (S100) further includes a process in which the data standardization unit (200) detects items where the diagnosis date is prior to the birth date, items where the end date of an event is prior to the start date, and events located outside the observation period as time contradiction items prior to loading the converted standard data, and withholds the loading of the detected time contradiction items and records them in a separate isolation area.

[0175] In the above agent control step (S200), the AI ​​agent (300) receives and processes an analysis request input in natural language and executes the analysis code on the data of the common data model structure.

[0176] FIG. 10 is a flowchart illustrating the detailed sub-processes of the agent control step according to the present invention and the control flow when preconditions are not met. Referring to FIG. 10, the agent control step (S200) includes the process of determining the purpose of analysis by parsing the meaning of a received natural language analysis request by an AI agent (300) (S210), the process of defining a cohort by calculating inclusion and exclusion conditions of a target group to match the determined purpose of analysis (S220), and the process of selecting an analysis technique corresponding to the defined cohort (S230).

[0177] Afterwards, in the agent control step (S200), the AI ​​agent (300) performs a process of checking whether the data characteristic preconditions for the selected analysis technique are satisfied (S240).

[0178] If it is calculated that the preconditions are satisfied in the above inspection process (S240), the AI ​​agent (300) sequentially performs the process (S250) of generating analysis code corresponding to the analysis technique through a deterministic compiler, and the process (S260) of executing the analysis code on the data of the converted common data model structure stored in the database (100).

[0179] On the other hand, if the preconditions are not satisfied in the above inspection process (S240), the AI ​​agent (300) performs the process of automatically switching internal logic to a pre-set alternative analysis technique or notifying and assigning a penalty factor to the reliability grade calculation process of the reliability evaluation unit (400) (S245).

[0180] In a specific embodiment, the agent control step (S200) according to the present invention includes a process in which a language model of an AI agent (300) proposes a structured specification of a cohort through a call to a tool function in which a parameter is specified as a type, and a process in which a deterministic compiler of the AI ​​agent (300) generates a query-only parameterized analysis code from the structured specification. In this case, the language model does not directly generate a free-form query.

[0181] Additionally, the agent control step (S200) may further include a process of comparing the numerical value included in the answer generated by the language model with the data value, which is the execution result of the analysis code, and blocking numerical values ​​that are not compared or exceed a preset tolerance. Here, the language model is a component included in the AI ​​agent (300) that operates in the inference layer, which repeats thinking, tool calling, and observation, and the answer is an output generated by the language model including numerical values ​​cited from the results of the tool. On the other hand, the data value is an output produced by the analysis code generated by the deterministic compiler included in the AI ​​agent (300), rather than the language model, being executed on standard data stored in the database (100), and the language model does not directly write the analysis code. Therefore, the source and path of production of the answer and the data value are different from each other, and the comparison means detecting and blocking the inclusion of numerical values ​​in the answer that are not based on the execution result by the language model. At this time, if a violation is detected, correction guidelines are injected to regenerate the answer, and if the violation is not resolved even through regeneration, the above answer is replaced with a response that expresses only true facts.

[0182] In the above reliability evaluation step (S300), the reliability evaluation unit (400) calculates a reliability grade for the data execution result of the agent control step (S200) and controls the subsequent processing of the system.

[0183] FIG. 11 is a flowchart illustrating the detailed sub-processes and bias detection logic of the reliability evaluation step according to the present invention. Referring to FIG. 11, the reliability evaluation step (S300) includes a process (S310) in which a reliability evaluation unit (400) normalizes a plurality of evaluation indicators including statistical significance of execution results, mapping reliability, and penalty score, and then calculates a comprehensive reliability score by weighted summing, and classifies the calculated comprehensive reliability score into one of a plurality of reliability grades according to a preset interval.

[0184] Additionally, the reliability evaluation step (S300) includes a process (S320) in which the reliability evaluation unit (400) performs a bias detection logic to calculate the statistical discrepancy between the concentration of the cohort's distribution by medical institution or region and the reference population distribution, and a process (S330) in which the calculated discrepancy is greater than or equal to a preset threshold.

[0185] If, in the above judgment process (S330), it is determined that the discrepancy is greater than or equal to a threshold value, the reliability evaluation unit (400) performs a process (S340) of lowering the reliability grade calculated by reflecting the bias of the data sample.

[0186] When the final reliability grade is determined, the reliability evaluation unit (400) performs a process (S350) of controlling the report generation unit (500) and the AI ​​agent (300) in conjunction to differentially apply at least one of the following: limiting the exposure scope of the execution result in the report, blocking the possibility of direct use in clinical or business decision-making, and suggesting alternative techniques according to the calculated grade.

[0187] Meanwhile, the present invention may further include a product evaluation step in which the product evaluation unit (700) converts the execution result of the agent control step (S200) into an insurance productization indicator, including the size of the target customer group, risk rate by age, and medical cost risk. This product evaluation step may be performed after the agent control step (S200) and before the report generation step (S400) described later.

[0188] In the above report generation step (S400), the report generation unit (500) automatically generates a report whose structure is determined based on at least one of the analysis purpose and analysis technique transmitted in the agent control step (S200) and the reliability grade calculated in the reliability evaluation step (S300).

[0189] FIG. 12 is a flowchart illustrating a detailed sub-process of the report generation step according to the present invention. Referring to FIG. 12, the report generation step (S400) includes a process (S410) in which a report generation unit (500) receives at least one of an analysis purpose, an analysis technique, and a reliability grade, and a process (S420) in which a report template stored in a knowledge management unit (600) is provided and a report structure and template are determined based on the metadata of the received information.

[0190] At this time, the report generation unit (500) performs the process (S430) of configuring the main body of the report by differentially reflecting the exposure range of each item in the report of the execution result according to the reliability grade, and controlling to conditionally reflect or block the evaluation result of the product evaluation unit (700) when the reliability grade is lower than a specific grade set in advance.

[0191] Afterwards, the report generation unit (500) terminates the control flow of the system by performing the process (S440) of mapping the analysis execution result data to the corresponding area of ​​the determined template and placing text and visualization elements, and the process (S450) of generating and outputting a report in which the exposure range is finally reflected.

[0193] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as set forth in the following claims. Explanation of the symbols

[0195] 100 : Database 200 : Data Standardization Department 300: AI Agent 400: Reliability Evaluation Department 500 : Report Generation Department 600 : Knowledge Management Department 700: Product Evaluation Department 800: User Terminal

Claims

Claim 1 A report automatic generation system comprising: a database in which multiple health data collected from different sources and having different data structures are stored; a data standardization unit that converts the multiple health data stored in the database into the structure of a pre-configured common data model; an AI agent that receives an analysis request input in natural language, determines an analysis purpose, defines a cohort corresponding to the analysis purpose, selects an analysis technique that matches the defined cohort, generates an analysis code corresponding to the selected analysis technique, and executes it on the data of the common data model structure; a reliability evaluation unit that calculates a reliability grade for the execution result of the AI ​​agent and controls subsequent processing; and a report generation unit that generates a report whose structure is determined based on at least one of the analysis purpose, analysis technique, and reliability grade received from the AI ​​agent and the reliability evaluation unit, wherein the reliability evaluation unit performs a bias detection logic that detects a discrepancy between the distribution concentration of the cohort defined by the AI ​​agent by medical institution or region and the standard population distribution, and if the discrepancy is greater than or equal to a threshold value, the reliability grade is downgraded. Claim 2 A report automatic generation system according to claim 1, wherein the reliability evaluation unit calculates a reliability grade based on a plurality of evaluation indicators for the execution result, and controls the operation of the report generation unit and the AI ​​agent to differentially apply at least one of limiting the exposure range of the execution result within the report, blocking its direct use in decision-making, and subsequent analysis adjustments proposing alternative techniques according to the calculated reliability grade. Claim 3 A report automatic generation system characterized in that, in claim 2, the reliability evaluation unit quantifies the reliability of a sample by calculating a partial credit coefficient based on the ratio of the number of observed events to the number of perfect credit events defined from a pre-set reliability level and tolerance, and controls the subsequent processing by applying a lowest grade priority method that adopts the lowest grade among the cell-specific credit grades classified according to the quantified reliability of the sample and the sample number grades classified according to the number of cohort subjects among the plurality of evaluation indicators as the reliability grade. Claim 4 A report automatic generation system according to claim 1, further comprising a knowledge management unit that stores data structure information including the semantics and code system of the table columns of the database, and collects and stores metadata including classification rules, mapping rules, reliability judgment criteria, and report templates from an administrator terminal or a pre-configured external system, wherein the AI ​​agent interacts with the knowledge management unit to extract metadata mapped to the determined analysis purpose and determine the cohort definition and analysis technique, and the data standardization unit receives the data structure information and mapping rules stored in the knowledge management unit and performs conversion to the common data model. Claim 5 A report automatic generation system according to claim 1, characterized in that the AI ​​agent checks whether the data characteristic preconditions for the selected analysis technique are satisfied, and if the preconditions are not satisfied, switches to a pre-set alternative analysis technique or is linked to impose a penalty on the reliability grade calculation process of the reliability evaluation unit. Claim 6 delete Claim 7 A report automatic generation system according to claim 1, further comprising a productability evaluation unit that converts the execution results of the AI ​​agent into insurance productability indicators to evaluate productability including target customer group size, risk rate, and medical expense risk, and wherein the report generation unit determines the structure of the report based on the combination of the evaluation results of the productability evaluation unit and the reliability grade calculated by the reliability evaluation unit. Claim 8 A report automatic generation system according to claim 1, wherein the AI ​​agent limits the behavior of the language model to the call of a plurality of tool functions, each parameter designated as a type, but does not provide a tool for executing arbitrary queries, and includes a deterministic compiler that receives a structured specification of a cohort and generates the analysis code for querying only, consisting of a fixed query template and bound parameters, and wherein the analysis code is executed only through a single parameterized execution path, thereby excluding the generation and execution of free-form queries by the language model. Claim 9 A report automatic generation system according to claim 8, wherein the AI ​​agent includes an output verifier that performs a basis verification by comparing each numerical value included in the answer generated by the language model with a data cell value which is the execution result of the deterministic compiler, and blocks numerical values ​​that are not compared or fall outside a preset tolerance as having no basis, and when the output verifier detects a violation, injects a correction guideline to regenerate and re-verify the answer, and if the violation is not resolved even through regeneration, replaces the answer with a response that expresses only true facts. Claim 10 A report automatic generation system according to claim 1, wherein the AI ​​agent further includes an integrity verifier that re-verifies an algebraic invariant for the execution result, including at least one of whether the sum of subtotals and the total match, whether the estimate is within a preset range, whether the confidence interval includes the estimate, and whether the number of holders is less than or equal to the number of populations, and blocks the passage of results that violate the invariant. Claim 11 A report automatic generation system characterized in that, in claim 8, the reliability evaluation unit generates a reproducibility specification that records together the structured specification used in the analysis, the hash of the compiled query, the hash of the result value, and the reliability determination, thereby enabling verification that the same value is produced when re-executed for the same input. Claim 12 A report automatic generation system according to claim 1, wherein the AI ​​agent proposes a candidate for an intent slot as an argument to a tool function typed according to instructions declared in a system prompt by a language model, and a deterministic rule parser re-extracts a value corresponding to the intent slot from the source text of the analysis request and compares it with the value of the proposed intent slot, wherein if the comparison result is inconsistent, the re-extracted value is adopted first, and if there are missing items, the intent slot is confirmed by supplementing with the re-extracted value. Claim 13 A standardization step in which a data standardization unit extracts multiple health data collected from different sources with different data structures from a database and converts them into the structure of a pre-established common data model; an agent control step in which an AI agent receives an analysis request input in natural language, determines an analysis objective based on the received analysis request, defines a cohort corresponding to the determined analysis objective, selects an analysis technique for the defined cohort, generates an analysis code corresponding to the selected analysis technique, and executes it on the data of the converted common data model structure stored in the database; and a reliability evaluation step in which a reliability evaluation unit calculates a reliability grade for the execution result of the agent control step and controls subsequent processing. A method for automatically generating a report, wherein the report generation unit includes a report generation step in which the structure of the report is determined based on at least one of the analysis purpose, analysis technique, and reliability grade calculated in the reliability evaluation step, and the reliability evaluation step includes a process of performing a bias detection logic in which the reliability evaluation unit calculates the disparity between the distribution concentration of the cohort defined in the agent control step and the reference population distribution, and a process of lowering the reliability grade if the calculated disparity is greater than or equal to a preset threshold. Claim 14 A method for automatically generating a report according to claim 13, wherein the reliability evaluation step comprises a process in which the reliability evaluation unit calculates a reliability grade based on a plurality of evaluation indicators for the execution result, and controls the differential application of at least one of limiting the exposure range of the execution result within the report, blocking its direct use in decision-making, and proposing an alternative technique according to the calculated reliability grade. Claim 15 A method for automatically generating a report according to claim 13, wherein the agent control step includes a process in which the AI ​​agent checks whether the data characteristic preconditions for the selected analysis technique are satisfied and automatically switches to an alternative analysis technique if they are not satisfied, and the reliability evaluation step further includes a process in which the reliability evaluation unit limits the exposure range of the execution result within the report generated by the report generation unit according to the reliability grade. Claim 16 delete

Images