Intelligent acquisition, integration and statistical analysis method and device for NHANES data

Abstract

The application provides an intelligent acquisition, integration and statistical analysis method and device for NHANES data, and the method comprises the following steps: NHANES outline grabbing and local database management are performed; XPT data file batch downloading and progress tracking are performed; Codebook variable specification grabbing, structured analysis and local indexing are performed; variable semantic classification, cross-cycle variable matching and research index intelligent inference are performed based on AI; candidate research cohorts, inclusion and exclusion conditions and variable combination suggestions are generated based on AI; a dialogic data exploration assistant is used to assist users in querying NHANES data in natural language; multi-file merging, data quality detection and statistical analysis are performed to generate a visual report containing analysis results, quality prompts and preliminary explanations. The application can help researchers significantly shorten the preparation time from public data to available research data sets.

Description

Technical Field

[0001] This invention relates to the field of data integration and analysis technology, specifically to a method and apparatus for intelligent acquisition, integration, and statistical analysis of NHANES data. Background Technology

[0002] Currently, when using publicly available data from the National Health and Nutrition Examination Survey (NHANES) in the United States for research projects and papers, one may encounter problems such as cumbersome data acquisition, repetitive data processing, and a high barrier to entry for statistical analysis.

[0003] Therefore, there is a need to provide a statistical analysis method for NHANES data, targeting researchers and graduate students in medical and public health schools, as well as researchers in hospital clinical and epidemiological research. This method aims to address the problems they face when using publicly available data from the National Health and Nutrition Examination Survey (NHES) for research projects and papers, such as cumbersome data acquisition, repetitive data processing, and a high barrier to entry in statistics. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method and apparatus for intelligent acquisition, integration and statistical analysis of NHANES data. This invention can help researchers significantly shorten the preparation time from public data to usable research datasets.

[0005] To solve the above problems, the technical solution of the present invention is as follows:

[0006] A method for intelligent acquisition, integration, and statistical analysis of NHANES data includes the following steps:

[0007] Perform NHANES outline retrieval and local database management;

[0008] Perform batch downloading and progress tracking of XPT data files;

[0009] Perform codebook variable specification capture, structured parsing, and local indexing;

[0010] AI-based variable semantic classification, cross-period variable matching, and intelligent prediction of research indicators;

[0011] AI-generated candidate study cohorts, inclusion / exclusion criteria, and variable combination suggestions;

[0012] The intelligent agent conversational data exploration assistant helps users query NHANES data using natural language;

[0013] Perform multi-file merging, data quality checks, and statistical analysis to generate a visual report that includes analysis results, quality tips, and preliminary explanations.

[0014] Preferably, the steps of NHANES outline capture and local database management specifically include: capturing the survey period, research components and data file list from the US Centers for Disease Control and Prevention website entry level by level, writing them into the local SQLite database in a three-level structure of period-component-file, supporting subsequent direct loading, clearing or rebuilding from the database.

[0015] Preferably, the steps for batch downloading and tracking XPT data files specifically include: the user selects multiple data files in a tree view, a background thread downloads them sequentially to the local download directory, files that already exist are automatically skipped, the overall progress and the current file progress are provided during the download process, and stopping midway is supported.

[0016] Preferably, the step of performing Codebook variable specification capture, structured parsing, and local indexing specifically includes: capturing the variable names, descriptions, and value ranges from the official codebook webpage of the U.S. Centers for Disease Control and Prevention for the selected data file and writing them into a database for subsequent indicator prediction and merging.

[0017] Preferably, the steps of performing variable semantic classification, cross-period variable matching, and intelligent prediction of research indicators based on AI specifically include: NHANES may have different variable names for the same indicator in different years. AI is used to identify their semantic relationships. Based on variable names, variable descriptions, components to which they belong, and historical period information, a semantic vector model is used to perform similarity matching on variables and identify synonymous variables, near-synonymous variables, or substitute variables across survey periods.

[0018] Preferably, the step of generating candidate study cohorts, inclusion / exclusion criteria, and variable combination suggestions based on AI specifically includes: having AI recommend inclusion criteria, exclusion criteria, exposure variables, outcome variables, and covariates based on the research topic; and intelligently generating candidate study designs based on the research topic input by the user, including target population, inclusion / exclusion criteria, exposure factors, outcome indicators, covariates, and recommended combinations of data files.

[0019] Preferably, the step of assisting the user in querying NHANES data using natural language through the intelligent agent conversational data exploration assistant specifically includes: providing a dialog window based on Markdown rendering through the intelligent agent conversational data exploration assistant, and the model can call local tools to read database content to assist the user in querying NHANES data using natural language.

[0020] Preferably, the steps of merging multiple files, performing data quality checks and statistical analysis, and generating a visual report containing analysis results, quality tips, and preliminary explanations specifically include: using AI to determine variable types and recommend statistical methods; automatically recommending descriptive statistics, group comparisons, trend tests, regression models, or weighted analysis methods based on the types of exposure variables, outcome variables, and covariates; and generating a visual report containing analysis results, quality tips, and preliminary explanations.

[0021] Furthermore, the present invention also provides an intelligent acquisition, integration, and statistical analysis apparatus for NHANES data, including a processor and a memory for storing executable instructions of the processor, the processor being configured to execute the intelligent acquisition, integration, and statistical analysis method for NHANES data as described above by executing the executable instructions.

[0022] Compared with existing technologies, this invention helps researchers significantly shorten the preparation time from public data to usable research datasets by capturing NHANES outlines, capturing Codebook variable descriptions, downloading batch XPT files, inferring AI indicators, exploring natural language data, and merging and exporting multiple files to the same desktop client. It also presents preliminary analysis conclusions in shareable formats such as HTML reports, allowing researchers to focus more on the research problem itself rather than data manipulation. Attached Figure Description

[0023] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0024] Figure 1 This is a flowchart of the intelligent acquisition, integration, and statistical analysis method for NHANES data according to the present invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0026] Specifically, this invention provides an intelligent method for the acquisition, integration, and statistical analysis of NHANES data, such as... Figure 1 As shown, the method includes the following steps:

[0027] S1: Perform NHANES outline capture and local database management;

[0028] Specifically, the system retrieves the survey cycle, research components, and data file list from the U.S. Centers for Disease Control and Prevention (CDC) website, writes them into a local SQLite database in a three-level structure of cycle-component-file, and supports subsequent loading, clearing, or rebuilding from the database.

[0029] S2: Perform batch downloading and progress tracking of XPT data files;

[0030] Specifically, users can select multiple data files in the tree view, and a background thread will download them to the local download directory in sequence. Files that already exist will be automatically skipped. The download process provides the overall progress and the progress of the current file, and supports stopping the download midway.

[0031] S3: Perform codebook variable specification capture, structured parsing, and local indexing;

[0032] Specifically, the variable names, descriptions, and value ranges from the CDC's official codebook webpage are extracted from the selected data files and written into the database for subsequent indicator prediction and merging.

[0033] S4: AI-based variable semantic classification, cross-period variable matching, and intelligent prediction of research indicators;

[0034] The same indicator in NHANES may have different variable names in different years, and AI can identify their semantic relationships. For example, based on variable name, variable description, component, and historical period information, semantic vector models are used to perform similarity matching on variables, identifying synonymous, near-synonymous, or substitute variables across survey periods.

[0035] S5: AI-based generation of candidate study cohorts, inclusion / exclusion criteria, and variable combination suggestions;

[0036] Specifically, AI can recommend inclusion criteria, exclusion criteria, exposure variables, outcome variables, and covariates based on the research topic. Based on the user-input research topic, it intelligently generates candidate research designs, including the target population, inclusion and exclusion criteria, exposure factors, outcome indicators, covariates, and recommended combinations of data files.

[0037] S6: Assists users in querying NHANES data using natural language through an intelligent agent-based conversational data exploration assistant;

[0038] Specifically, the intelligent agent conversational data exploration assistant provides a dialog window based on Markdown rendering. The model can call local tools to read database content to assist users in querying NHANES data in natural language.

[0039] S7: Performs multi-file merging, data quality checks, and statistical analysis, generating a visual report that includes analysis results, quality tips, and preliminary explanations.

[0040] Specifically, AI determines the variable type and recommends statistical methods: based on the types of exposure variables, outcome variables, and covariates, it automatically recommends descriptive statistics, group comparisons, trend tests, regression models, or weighted analysis methods. It generates a visual report containing analysis results, quality tips, and preliminary interpretations.

[0041] The method of this invention can help researchers significantly shorten the preparation time from publicly available data to usable research datasets, and present preliminary analysis conclusions in shareable formats such as HTML reports, allowing researchers to focus more on the research question itself rather than data transfer.

[0042] In another embodiment, the present invention also provides an intelligent acquisition, integration, and statistical analysis apparatus for NHANES data, including a processor and a memory for storing executable instructions of the processor, the processor being configured to execute the intelligent acquisition, integration, and statistical analysis method for NHANES data as described in the above embodiment by executing the executable instructions.

[0043] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for intelligent acquisition, integration, and statistical analysis of NHANES data, characterized in that, The system includes: Perform NHANES outline retrieval and local database management; Perform batch downloading and progress tracking of XPT data files; Perform codebook variable specification capture, structured parsing, and local indexing; AI-based variable semantic classification, cross-period variable matching, and intelligent prediction of research indicators; AI-generated candidate study cohorts, inclusion / exclusion criteria, and variable combination suggestions; The intelligent agent conversational data exploration assistant helps users query NHANES data using natural language; Perform multi-file merging, data quality checks, and statistical analysis to generate a visual report that includes analysis results, quality tips, and preliminary explanations.

2. The intelligent acquisition, integration, and statistical analysis method for NHANES data according to claim 1, characterized in that, The steps for NHANES outline retrieval and local database management include: retrieving the survey period, research components, and data file list from the US Centers for Disease Control and Prevention website, writing them into a local SQLite database in a three-level structure of period-component-file, and supporting subsequent loading, clearing, or rebuilding from the database.

3. The intelligent acquisition, integration, and statistical analysis method for NHANES data according to claim 1, characterized in that, The steps for batch downloading and tracking XPT data files specifically include: the user selects multiple data files in a tree view, the background thread downloads them sequentially to the local download directory, files that already exist are automatically skipped, the overall progress and the current file progress are provided during the download process, and stopping the download midway is supported.

4. The intelligent acquisition, integration, and statistical analysis method for NHANES data according to claim 1, characterized in that, The steps of capturing, parse, and indexing the Codebook variable descriptions specifically include: capturing the variable names, descriptions, and value ranges from the official codebook webpage of the U.S. Centers for Disease Control and Prevention for the selected data file and writing them into the database for subsequent indicator prediction and merging.

5. The intelligent acquisition, integration, and statistical analysis method for NHANES data according to claim 1, characterized in that, The steps of using AI for variable semantic classification, cross-period variable matching, and intelligent prediction of research indicators specifically include: NHANES may have different variable names for the same indicator in different years. AI is used to identify their semantic relationships. Based on variable names, variable descriptions, components to which they belong, and historical period information, a semantic vector model is used to perform similarity matching on variables, and synonymous, near-synonymous, or substitute variables across survey periods are identified.

6. The intelligent acquisition, integration, and statistical analysis method for NHANES data according to claim 1, characterized in that, The steps for generating candidate study cohorts, inclusion / exclusion criteria, and variable combination suggestions based on AI specifically include: having AI recommend inclusion criteria, exclusion criteria, exposure variables, outcome variables, and covariates based on the research topic; and intelligently generating candidate study designs based on the research topic input by the user, including target population, inclusion / exclusion criteria, exposure factors, outcome indicators, covariates, and recommended combinations of data files.

7. The intelligent acquisition, integration, and statistical analysis method for NHANES data according to claim 1, characterized in that, The steps of assisting users in querying NHANES data using natural language through the intelligent agent conversational data exploration assistant specifically include: providing a dialog window based on Markdown rendering through the intelligent agent conversational data exploration assistant, and the model can call local tools to read database content to assist users in querying NHANES data using natural language.

8. The intelligent acquisition, integration, and statistical analysis method for NHANES data according to claim 1, characterized in that, The steps of merging multiple files, performing data quality checks and statistical analysis, and generating a visual report containing analysis results, quality tips, and preliminary explanations specifically include: using AI to determine variable types and recommend statistical methods; automatically recommending descriptive statistics, group comparisons, trend tests, regression models, or weighted analysis methods based on the types of exposure variables, outcome variables, and covariates; and generating a visual report containing analysis results, quality tips, and preliminary explanations.

9. A smart acquisition, integration, and statistical analysis device for NHANES data, characterized in that, The device includes a processor and a memory for storing executable instructions of the processor, the processor being configured to perform the intelligent acquisition, integration, and statistical analysis method for NHANES data as described in any one of claims 1 to 8 by executing the executable instructions.

Images