Autism database sharing platform based on ETL process

Abstract

The invention discloses an autism database sharing platform based on an ETL process, and belongs to the technical field of information technologies. The sharing platform comprises a data extraction module, a data conversion module, a data loading module, a metadata management module and an access control module. According to the method, efficient integration of multi-source data is achieved through the ETL process, the problems of data dispersion and format inconsistency are solved, metadata management and data standardization technologies are adopted, the understandability and utilization efficiency of the data are improved, the data security and privacy protection module is adopted to perform privacy protection on the data, and the data security and privacy protection efficiency is improved. A safe and reliable data sharing environment is established; the system can support and promote data communication and cooperation between researchers and professionals.

Description

Technical Field

[0001] The present invention belongs to the technical field of information technology, and in particular relates to an autism database sharing platform based on ETL process. Background Art

[0002] Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder that usually appears in early childhood and affects patients' social interactions, communication abilities, and behavioral patterns. Although significant progress has been made in the study of autism in recent years, the etiology and pathogenesis of ASD are still not fully understood. Researchers generally believe that the occurrence of autism is the result of the combined effects of genetic and environmental factors. Therefore, comprehensive analysis of multi-source data is essential for a deep understanding and study of ASD.

[0003] Currently, data used in ASD research come from a variety of sources, including clinical data, genetic data, questionnaire data, and neuroimaging data. However, these data are often scattered across different databases and research institutions, and lack a unified data sharing platform, making it difficult for researchers to effectively integrate and analyze multiple data. This data island phenomenon limits interdisciplinary collaboration and comprehensive research, and affects the progress of autism research and the optimization of diagnosis and treatment plans.

[0004] In addition, the existing autism data sharing platform has the following problems in data integration and management:

[0005] Inconsistent data formats: The data formats and standards of different data sources are not unified, which makes data integration and analysis difficult.

[0006] Complex data cleaning and preprocessing: The quality of multi-source data varies, requiring complex data cleaning and preprocessing steps, which increases the workload of researchers.

[0007] Insufficient data security and access control: Autism data involves sensitive information of patients. Existing platforms are deficient in data security and privacy protection, which affects the willingness to share data.

[0008] Lack of standardized metadata descriptions: Metadata descriptions are not unified, resulting in inefficient data retrieval and utilization.

[0009] Therefore, it is particularly necessary to establish a unified autism database sharing platform. Through this platform, scattered ASD-related data can be integrated to form a comprehensive database, providing researchers with convenient data access and analysis tools, and promoting interdisciplinary cooperation. Summary of the invention

[0010] The present invention aims to solve the problems of autism research data being scattered, inconsistent in format, and unable to be shared, and to provide an autism database sharing platform based on an ETL process.

[0011] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0012] An autism database sharing platform based on ETL process, characterized by comprising the following modules: data extraction module, data conversion module, data loading module and metadata management module;

[0013] The data extraction module is used to extract autism-related data from different data sources;

[0014] The data sources include hospital databases, research institute data warehouses, gene sequencing platforms, etc.; the autism-related data include clinical data, gene data, questionnaire scale data, neuroimaging data, and audio-visual data;

[0015] The data conversion module is used to convert the format of the autism-related data extracted by the data extraction module according to the metadata standard and perform data integration, so that the autism-related data of the same patient are combined into one autism metadata;

[0016] The data loading module is used to store the autism metadata obtained by the data conversion module into the autism database, so that users of the sharing platform can access the autism metadata in the database, and support data exchange and cooperation between researchers and professionals;

[0017] The metadata management module is used to manage metadata items of the data sharing platform, including semantic consistency checking function, user modification function and automatic version control function.

[0018] Preferably, the autism database sharing platform also includes an access control module; the access control module includes user identity authentication, authority management and access log functions, which are used to encrypt data and control access to the autism database, ensure data security and user privacy protection, and establish a safe and reliable data sharing environment.

[0019] Preferably, the user identity authentication is used to perform multi-factor authentication (MFA) on the user before the user accesses the autism database, including identity authentication using at least two authentication methods including username and password, dynamic verification code, fingerprint recognition, or other biometric technologies, to ensure that only authorized users can access the database.

[0020] The permission management adopts the role-based access control (RBAC) method to set the access rights and operation rights of different roles in the system; the set roles include: (1) Researchers: can only access non-sensitive data that has been anonymized for scientific research analysis; (2) Doctors: can access the complete medical records of the patients they are responsible for; (3) Administrators: can manage system permissions and monitor access logs, but are not allowed to directly access patient data.

[0021] The access log function is used to record detailed information of each data access, including user identity, access time, access content, operation type (query, update, delete) and source IP address.

[0022] Preferably, the semantic consistency check function is used to ensure that data from different sources maintain consistency at the semantic level based on preset metadata standards, reduce data comprehension errors caused by terminology differences, and improve data availability and accuracy.

[0023] The user modification function is used to allow authorized users or administrators to add, delete, and modify operations.

[0024] The automatic version control function is used to automatically generate a version record when metadata is added, deleted or modified, and the record content includes the change time, operator, and change content; the version control function supports tracing back historical data and recording and tracking the evolution process of data.

[0025] Preferably, the data extraction module adopts a combination of periodic retrieval and real-time extraction to ensure the timeliness and update frequency of data and prevent data omission.

[0026] Preferably, the data conversion module converts the format of the extracted autism-related data according to the metadata standard and performs data integration to solve the problems of inconsistent data formats and data quality. The specific steps include:

[0027] Data cleaning: remove duplicate data, correct erroneous data, and fill in missing values.

[0028] Data standardization: Convert data into a unified format and unit specified by metadata specifications to ensure consistency of data from different sources.

[0029] Data integration: Integrate data from different sources of the same patient according to unified standards to form comprehensive autism metadata.

[0030] The beneficial effects of the present invention are as follows:

[0031] Efficient data integration: Efficient integration of multi-source data is achieved through the ETL process to solve the problems of data dispersion and inconsistent formats.

[0032] Standardized data management: Use metadata management and data standardization technology to improve data understandability and utilization efficiency.

[0033] Safe and reliable data sharing: Use data security and privacy protection modules to protect data privacy and establish a safe and reliable data sharing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the description, are used to explain the technical solutions of the present invention.

[0035] Figure 1 A schematic diagram of a process of an autism database sharing platform based on an ETL process is provided in an embodiment of the present invention.

[0036] Figure 2 A schematic diagram of a clinical data metadata item specification provided in an embodiment of the present invention.

[0037] Figure 3 A schematic diagram of a gene data metadata item specification provided by an embodiment of the present invention.

[0038] Figure 4 A schematic diagram of questionnaire data metadata item specifications provided by an embodiment of the present invention.

[0039] Figure 5 A schematic diagram of a neuroimaging data metadata item specification provided by an embodiment of the present invention.

[0040] Figure 6 A schematic diagram of a metadata item specification for audio and video data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the technical solution of the present invention is further described below in conjunction with embodiments.

[0042] The autism database sharing platform based on the ETL process of this embodiment includes: a data extraction module, a data conversion module, a data loading module, a metadata management module and an access control module.

[0043] The data extraction module is used to extract autism-related data from different data sources by combining periodic retrieval and real-time extraction, so as to ensure the timeliness and update frequency of the data and prevent data omission; the data sources include hospital databases, research institution data warehouses, gene sequencing platforms, etc.; the autism-related data include clinical data, gene data, questionnaire scale data, neuroimaging data, audio and video data, etc.

[0044] The data conversion module is used to convert the format of the autism-related data extracted by the data extraction module according to the metadata standard and perform data integration, so that the autism-related data of the same patient are merged into one autism metadata, solving the problems of inconsistent data formats and data quality.

[0045] The specific steps include:

[0046] Data cleaning: remove duplicate data, correct erroneous data, and fill in missing values.

[0047] Data standardization: Convert data into a unified format and unit specified by metadata specifications to ensure consistency of data from different sources.

[0048] Data integration: Integrate data from different sources of the same patient according to unified standards to form comprehensive autism metadata.

[0049] The loading module is used to store the autism metadata obtained by the data conversion module into the autism database, so that users of the sharing platform can access the autism metadata in the database, and support data exchange and cooperation between researchers and professionals.

[0050] The access control module includes user identity authentication, authority management and access log functions, and is used to encrypt data and control access to the autism database, ensure data security and user privacy protection, and establish a safe and reliable data sharing environment.

[0051] The user identity authentication is used to perform multi-factor authentication (MFA) on the user before the user accesses the autism database, including identity authentication using at least two authentication methods including username and password, dynamic verification code, fingerprint recognition, or other biometric technologies, to ensure that only authorized users can access the database.

[0052] The permission management adopts the role-based access control (RBAC) method to set the access rights and operation rights of different roles in the system; the set roles include: (1) Researchers: can only access non-sensitive data that has been anonymized for scientific research analysis; (2) Doctors: can access the complete medical records of the patients they are responsible for; (3) Administrators: can manage system permissions and monitor access logs, but are not allowed to directly access patient data.

[0053] The access log function is used to record detailed information of each data access, including user identity, access time, access content, operation type (query, update, delete) and source IP address.

[0054] The metadata management module is used to manage metadata items of the data sharing platform, including semantic consistency checking function, user modification function and automatic version control function.

[0055] Specifically, the access log function is used to record detailed information of each data access, including user identity, access time, access content, operation type (query, update, delete) and source IP address.

[0056] The semantic consistency check function is used to ensure that data from different sources maintains consistency at the semantic level based on preset metadata standards, reduce data comprehension errors caused by terminology differences, and improve data availability and accuracy.

[0057] The user modification function is used to allow authorized users or administrators to add, delete, and modify operations.

[0058] The automatic version control function is used to automatically generate a version record when metadata is added, deleted or modified, and the record content includes the change time, operator, and change content; the version control function supports tracing back historical data and recording and tracking the evolution process of data.

[0059] Designing appropriate metadata specifications can ensure that key information is accurately captured, stored, and analyzed, thereby providing support for autism research and treatment. The metadata specifications, after conversion according to the unified standard, are as follows:

[0060] 1. Clinical Data

[0061] Clinical data is data collected in medical and health research, usually related to basic information of patients, health status, diseases, treatments and other medical-related information. Figure 2 .

[0062] 2. Genetic Data

[0063] Through research and analysis of domestic and foreign gene databases, their metadata mainly includes number, title, gene-related biological information, submission time, editing history, etc.; gene database metadata is divided into identification dimension, content dimension and management dimension. Among them, the identification dimension is a description of gene identification information, the content dimension is a description of gene-related biological information, and the management dimension is a description of gene management information. Figure 3 .

[0064] 3. Questionnaire scale data

[0065] The metadata specification of questionnaire scales is designed to effectively manage and use questionnaire scales that can be used to diagnose autism patients. Questionnaire scale metadata is divided into identification dimension, content dimension and management dimension. Among them, the identification dimension is a description of the identification information of the clinical questionnaire, the content dimension is a description of the main content aspects of the questionnaire designed to collect patient or participant information in medical and clinical research, and the management dimension is a description of the management information of the clinical questionnaire. See the questionnaire scale metadata for details. Figure 4 .

[0066] 4. Neuroimaging Data

[0067] The purpose of neuroimaging data specifications is to ensure the integrity, consistency, and availability of data. Therefore, in the process of designing specifications, all data should be graded and classified as follows:

[0068] The first level is to obtain the basic information of the patient, including the basic medical information of the hospital and the basic data of the patient's condition;

[0069] The second level scans the device information, including recording the device model, software version, and coil type;

[0070] The third level is scanning parameter information, including modality, resolution, number of slices and slice thickness, TR / TE, flip angle, and acquisition direction;

[0071] Level 4, data processing information, including preprocessing steps, file formats, software and tools, and parameter settings;

[0072] Level 5, data quality information, including motion parameters, signal-to-noise ratio, and artifacts;

[0073] Level 6, supplementary information, including experimental design, behavioral data, and annotations;

[0074] Level 7, management information.

[0075] The design of the neuroimaging data specification of the multimodal database is a great supplement to the sharing of clinical medical data resources. The detailed design of its metadata can be found in Figure 5 .

[0076] 5. Audio and video data

[0077] Video and audio data, such as video and audio data of family interactions and daily life. Its metadata is divided into identification information, physical feature information and management information. For detailed design, see Figure 6 .

Claims

1. An autism database sharing platform based on ETL process, characterized by: It includes the following modules: data extraction module, data conversion module, data loading module, metadata management module and access control module; The data extraction module is used to extract autism-related data from different data sources; The data sources include hospital databases, research institute data warehouses, gene sequencing platforms, etc.; the autism-related data include clinical data, gene data, questionnaire scale data, neuroimaging data, and audio-visual data; The data conversion module is used to convert the format of the autism-related data extracted by the data extraction module according to the metadata standard and perform data integration, so that the autism-related data of the same patient are combined into one autism metadata; The data loading module is used to store the autism metadata obtained by the data conversion module into the autism database, so that users of the sharing platform can access the autism metadata in the database, and support data exchange and cooperation between researchers and professionals; The metadata management module is used to manage metadata items of the data sharing platform, including semantic consistency checking function, user modification function and automatic version control function.

2. The autism database sharing platform based on ETL process as claimed in claim 1, characterized in that: The autism database sharing platform also includes an access control module; the access control module includes user identity authentication, authority management and access log functions, which are used to encrypt data and control access to the autism database, ensure data security and user privacy protection, and establish a safe and reliable data sharing environment.

3. The autism database sharing platform based on ETL process as claimed in claim 2, characterized in that: The user identity authentication is used to perform multi-factor authentication on the user before the user accesses the autism database, including user name and password, dynamic verification code, fingerprint recognition, or at least two authentication methods of other biometric technologies to ensure that only authorized users can access the database; The permission management adopts a role-based access control method to set the access rights and operation rights of different roles in the system; The roles set include: (1) Researchers: can only access anonymized non-sensitive data for scientific research analysis; (2) Doctors: can access the complete medical records of the patients they are responsible for; (3) Administrators: can manage system permissions and monitor access logs, but are not allowed to directly access patient data; The access log function is used to record detailed information of each data access, including user identity, access time, access content, operation type and source IP address.

4. An autism database sharing platform based on ETL process as described in claim 2 or 3, characterized in that: The semantic consistency check function is used to ensure that data from different sources are consistent at the semantic level based on preset metadata standards, reduce data comprehension errors caused by terminology differences, and improve data availability and accuracy; The user modification function is used to add, delete, and modify operations for a given authorized user or administrator; The automatic version control function is used to automatically generate a version record when metadata is added, deleted or modified, and the record content includes the change time, operator, and change content; the version control function supports tracing back historical data and recording and tracking the evolution process of data.

5. The autism database sharing platform based on ETL process as claimed in claim 4, characterized in that: The data extraction module adopts a combination of regular retrieval and real-time extraction to ensure the timeliness and update frequency of data and prevent data omission.

6. The autism database sharing platform based on ETL process as claimed in claim 5, characterized in that: The data conversion module converts the format of the extracted autism-related data according to the metadata standard and performs data integration to solve the problems of inconsistent data formats and data quality; The specific steps include: Data cleaning: remove duplicate data, correct erroneous data, and fill in missing values; Data standardization: converting data into a unified format and unit specified by metadata specifications to ensure consistency of data from different sources; Data integration: Integrate data from different sources of the same patient according to unified standards to form comprehensive autism metadata.

Images