Uniformity AI multi-modal data acquisition method and platform

Abstract

The embodiment of the invention relates to the technical field of data acquisition, and particularly discloses a uniform AI multi-modal data acquisition method and platform. According to the embodiment of the invention, data acquisition is carried out from a plurality of preset data sources to obtain original data; performing data preprocessing on the original data, and performing feature extraction based on an AI large language model to obtain feature data; performing data cleaning on the feature data to obtain data variables; performing matching and filling on the data variable and a preset form template to generate form data; and storing the form data in a database. The method can simplify and optimize a data acquisition process, utilizes an AI technology to intelligently extract and analyze data, assists a user in automatically completing related work of data acquisition, filling and exporting, and structuralizes and standardizes hospital clinical medical record data, so that the efficiency and the result quality of scientific research work are remarkably improved, and the scientific research efficiency is improved. The innovation and development of the scientific research field are further promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of data acquisition, and particularly relates to a unified AI multi-modal data acquisition method and platform. Background Art

[0002] In today's scientific research field, information collection, management, and analysis constitute key parts of the scientific research process. However, the management of clinical research forms poses challenges to scientific researchers.

[0003] Traditional form management methods are inefficient, lack necessary flexibility, and have poor user experience. These problems seriously threaten the accuracy and usability of data, thus restricting the progress and quality of scientific research work. Especially in hospital databases, the dispersion of data storage, permission differences, and professional restrictions make it difficult for many scientific research doctors to directly and accurately obtain the required data. They have to rely on the assistance of professionals, which not only raises the threshold of data acquisition but also increases the difficulty of acquisition, significantly reducing scientific research efficiency. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a unified AI multi-modal data acquisition method and platform, aiming to solve the problems raised in the background art.

[0005] To achieve the above purpose, the embodiments of the present invention provide the following technical solutions:

[0006] A unified AI multi-modal data acquisition method, the method specifically includes the following steps:

[0007] Collect data from multiple preset data sources to obtain raw data;

[0008] Perform data preprocessing on the raw data, and based on the AI large language model, perform feature extraction to obtain feature data;

[0009] Perform data cleaning on the feature data to obtain data variables;

[0010] Match and fill the data variables with a preset form template to generate form data;

[0011] Store the form data in a database.

[0012] As a further limitation of the technical solution of the embodiments of the present invention, the raw data includes audio data, picture data, PDF data, and / or text data.

[0013] As a further limitation of the technical solution of the embodiments of the present invention, the performing data preprocessing on the raw data, and based on the AI large language model, performing feature extraction to obtain feature data specifically includes the following steps:

[0014] Remove the noise and / or abnormally formatted data in the original data to obtain valid data;

[0015] Based on the AI large language model, extract features from the valid data to obtain feature data.

[0016] As a further limitation of the technical solution of the embodiment of the present invention, in the step of removing the noise and / or abnormally formatted data in the original data to obtain valid data: perform audio noise reduction processing on the audio data; perform image enhancement and text area localization processing on the image data; perform recognition processing on the PDF data.

[0017] As a further limitation of the technical solution of the embodiment of the present invention, in the step of extracting features from the valid data based on the AI large language model to obtain feature data: based on the AI large language model, extract multiple key features from the text data, including word segmentation, part-of-speech tagging, and named entities; combine the AI large language model and OCR technology to identify the text content in the image data and perform key extraction; based on the AI large language model, perform spectral analysis on the audio data to extract acoustic features.

[0018] As a further limitation of the technical solution of the embodiment of the present invention, the step of performing data cleaning on the feature data to obtain data variables specifically includes the following steps:

[0019] Identify redundant data and error data in the feature data;

[0020] Clean the redundant data and the error data to obtain data variables.

[0021] As a further limitation of the technical solution of the embodiment of the present invention, the step of matching and filling the data variables with a preset form template to generate form data specifically includes the following steps:

[0022] Match the data variables with a preset form template to determine the matching positions;

[0023] Extract specified variable values;

[0024] Automatically fill the specified variable values into the matching positions to generate form data.

[0025] As a further limitation of the technical solution of the embodiment of the present invention, in the step of matching the data variables with a preset form template to determine the matching positions, match the specified variables, filter out the redundant variables, determine the field positions in the form template, and obtain the matching positions.

[0026] As a further limitation of the technical solution of the embodiment of the present invention, the step of storing the form data in a database specifically includes the following steps:

[0027] Match the storage location;

[0028] Store the form data into the database according to the storage location;

[0029] Perform version control on the form data.

[0030] Unified AI multi-modal data acquisition platform, the platform includes a data acquisition unit, a data preprocessing unit, a data cleaning unit, a data matching and filling unit, and a data storage unit, where:

[0031] The data acquisition unit is used to acquire raw data from multiple preset data sources;

[0032] The data preprocessing unit is used to preprocess the raw data and perform feature extraction based on the AI large language model to obtain feature data;

[0033] The data cleaning unit is used to clean the feature data to obtain data variables;

[0034] The data matching and filling unit is used to match and fill the data variables with a preset form template to generate form data;

[0035] The data storage unit is used to store the form data into the database.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] In the embodiment of the present invention, raw data is acquired by acquiring data from multiple preset data sources; the raw data is preprocessed, and feature extraction is performed based on the AI large language model to obtain feature data; the feature data is cleaned to obtain data variables; the data variables are matched and filled with a preset form template to generate form data; the form data is stored in the database. It can streamline and optimize the data acquisition process, and use AI technology for intelligent extraction and analysis of data, assist users to automatically complete the relevant work of data acquisition, filling, and export, and structure and standardize the hospital clinical medical record data, thereby significantly improving the efficiency and quality of scientific research work, and further promoting innovation and development in the field of scientific research. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0039] Figure 1Shows the application process schematic diagram of the method provided by the embodiments of the present invention.

[0040] Figure 2 Shows the application architecture diagram of the platform provided by the embodiments of the present invention. Detailed implementation manners

[0041] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] It can be understood that traditional form management methods are inefficient, lack necessary flexibility, and have poor user experience. These problems seriously threaten the accuracy and practicality of data, and thus limit the progress and quality of scientific research work. Especially in hospital databases, the dispersion of data storage, permission differences, and professional restrictions make it difficult for many scientific research doctors to directly and accurately obtain the required data, and they have to rely on the assistance of professionals. This not only raises the threshold for data acquisition, but also increases the difficulty of acquisition, significantly reducing the efficiency of scientific research.

[0043] To solve the above problems, the embodiments of the present invention collect raw data from multiple preset data sources, preprocess the raw data, and perform feature extraction based on the AI large language model to obtain feature data; clean the feature data to obtain data variables; match and fill the data variables with a preset form template to generate form data; and store the form data in a database. It can streamline and optimize the data collection process, and use AI technology for intelligent extraction and analysis of data, assist users in automatically completing the related work of data collection, filling, and export, and structure and standardize the hospital clinical medical record data, thereby significantly improving the efficiency and quality of scientific research work, and further promoting innovation and development in the scientific research field.

[0044] Figure 1 Shows the application process schematic diagram of the method provided by the embodiments of the present invention.

[0045] Specifically, a unified AI multi-modal data collection method, the method specifically includes the following steps:

[0046] Step 1: Collect data from multiple preset data sources to obtain raw data.

[0047] In the embodiments of the present invention, data is collected from multiple preset data sources to obtain raw data including audio data, picture data, PDF data, and / or text data, and the multiple data sources include audio data sources, picture data sources, PDF data sources, and / or text data sources.

[0048] In the embodiment of the present invention, the collected data is uniformly managed, including data storage, retrieval, export, etc. It can perform multi-dimensional retrieval based on multiple dimensions (such as departments, specific diseases, patients), and provide a data visualization function to help users intuitively understand the data distribution.

[0049] Step 2: Perform data preprocessing on the original data, and based on the AI large language model, perform feature extraction to obtain feature data.

[0050] In the embodiment of the present invention, by identifying noise and / or abnormal format data in the original data, and performing elimination processing on the noise and / or abnormal format data (specifically including: performing audio noise reduction processing on audio data; performing image enhancement and text area localization processing on image data; performing recognition processing on PDF data, etc.), effective data is obtained. Then, based on the AI large language model, multiple key features are extracted from the text data, including word segmentation, part-of-speech tagging, and named entities; combining the AI large language model and OCR technology, the text content in the image data is recognized and key extraction is performed; based on the AI large language model, spectral analysis is performed on the audio data to extract acoustic features, so as to perform feature extraction on the effective data and obtain feature data.

[0051] Step 3: Perform data cleaning on the feature data to obtain data variables.

[0052] In the embodiment of the present invention, by identifying redundant data and error data in the feature data, and then performing cleaning processing on the redundant data and error data, data variables are obtained.

[0053] Step 4: Match and fill the data variables with a preset form template to generate form data.

[0054] In the embodiment of the present invention, the data variables are subjected to matching analysis with a preset form template, the specified variables are matched, the redundant variables are filtered, the field positions in the form template are determined to obtain the matching positions, and the specified variable values corresponding to the specified variables are extracted. The specified variable values are automatically filled into the matching positions to generate form data.

[0055] Step 5: Store the form data in a database.

[0056] In the embodiment of the present invention, by matching the storage location, and then according to the storage location, the form data is stored in the database, and version control is performed on the form data for subsequent query and analysis.

[0057] In the embodiments of the present invention, a flexible CRF form design function is provided, which supports customizing form styles, field types, and logical verification rules. Users can quickly construct forms by dragging and dropping, and perform online preview and editing. In addition, a form template management function is provided to facilitate the reuse of common forms.

[0058] Furthermore, Figure 2 The application architecture diagram of the platform provided by the embodiments of the present invention is shown.

[0059] Among them, in another preferred embodiment provided by the present invention, the unified AI multi-modal data acquisition platform includes:

[0060] A data acquisition unit for acquiring raw data by collecting data from multiple preset data sources.

[0061] In the embodiments of the present invention, the data acquisition unit collects data from multiple preset data sources to obtain raw data including audio data, picture data, PDF data, and / or text data. The multiple data sources include audio data sources, picture data sources, PDF data sources, and / or text data sources.

[0062] A data preprocessing unit for preprocessing the raw data and extracting features based on the AI large language model to obtain feature data.

[0063] In the embodiments of the present invention, the data preprocessing unit identifies noise and / or abnormal format data in the raw data, and eliminates the noise and / or abnormal format data (specifically including: performing audio noise reduction processing on audio data; performing picture enhancement and text area localization processing on picture data; performing recognition processing on PDF data, etc.) to obtain valid data. Then, based on the AI large language model, multiple key features are extracted from the text data, including word segmentation, part-of-speech tagging, and named entities; combining the AI large language model and OCR technology, the text content in the picture data is recognized and key extraction is performed; based on the AI large language model, spectral analysis is performed on the audio data to extract acoustic features, so as to extract features from the valid data and obtain feature data.

[0064] A data cleaning unit for cleaning the feature data to obtain data variables.

[0065] In the embodiments of the present invention, the data cleaning unit identifies redundant data and error data in the feature data, and then cleans the redundant data and error data to obtain data variables.

[0066] A data matching and filling unit for matching and filling the data variables with a preset form template to generate form data.

[0067] In an embodiment of the present invention, the data matching and filling unit performs matching analysis on data variables and a preset form template, matches specified variables, filters out redundant variables, determines the field positions in the form template to obtain matching positions, extracts the specified variable values corresponding to the specified variables, and automatically fills the specified variable values into the matching positions to generate form data.

[0068] The data storage unit is used to store the form data in a database.

[0069] In an embodiment of the present invention, the data storage unit stores the form data in the database by matching the storage positions and then according to the storage positions, and performs version control on the form data for subsequent query and analysis.

[0070] In a specific embodiment of the present invention, it is known that patient A has undergone multiple examinations in the hospital, and a total of H examination reports of this patient have been obtained, which are examination reports made on different dates at different time periods. The report types include blood routine, urine routine, heart research institute, biochemical immunity, autoantibodies, hematology, etc., a total of F types. The CRF form needs to fill in blood glucose values, including fasting, 1-hour interval, 2-hour interval, and a total of N fields such as blood routine. And the three blood glucose values are not in the same report, and the blood glucose value report only shows the detection time, and the time interval needs to be calculated by oneself. There are a total of M data fields that need to be calculated independently. Try to ensure that all the filled data is on the same day. If not, then take the closest date. If the same field exists in different reports, then try to take the one with the closest date. Set an algorithm to find the corresponding values from these reports for filling, and try to reduce the number of report traversals.

[0071] The algorithm formula is as follows:

[0072] Define sets and symbols:

[0073] Let R be the set of all examination reports of patient A, (R = {r1, r2,..., rH});

[0074] Let D be the set of all possible dates, (D = {d1, d2,..., dK});

[0075] Let T be the set of all possible detection times, (T = {t1, t2,..., tL});

[0076] Let F be the set of all fields in the CRF form, (F = {f1, f2,..., fN});

[0077] Let M be the set of data fields that need to be calculated independently, (M ⊆ F);

[0078] Let dataMap be a mapping that maps field f to a tuple containing value v and date d, i.e., dataMap: F → {(v, d)}.

[0079] Define the functions:

[0080] sortReports(R) → R′: Sorts the set of reports R by date to obtain the sorted set of reports R′;

[0081] extractInfo(r) → (d, t, fields): Extracts date d, time t, and set of fields fields from report r;

[0082] isBloodSugar(f) → bool: Determines whether field f is a blood sugar value field;

[0083] calculateTimePoint(t) → timePoint: Calculates the blood sugar time point based on the detection time t;

[0084] updateDataMap(dataMap, f, v, d, timePoint) → dataMap′: Updates the data mapping dataMap;

[0085] performCalculation(v) → v′: Performs the necessary calculations on field value v;

[0086] getLatestValue(dataMap, f) → (v, d): Gets the latest value v and date d of field f from the data mapping dataMap;

[0087] isSameDay(d1, d2) → bool: Determines whether two dates d1 and d2 are the same day.

[0088] The mathematical model is as follows:

[0089] 1. Initialize CRF and dataMap:

[0090] CRF = {}

[0091] dataMap = {}

[0092] 2. Sort the set of reports R:

[0093] R′ = sortReports(R)

[0094] 3. Traverse the sorted set of reports R′:

[0095]

[0096] if is Blood Sugar(f)

[0097] timePoint = calculateTimePoint(t)

[0098] dataMap = updateDataMap(dataMap,f,v,d,timePoint)

[0099] else if f ∈ M

[0100] v′ = performCalculation(v)

[0101] dataMap = updateDataMap(dataMap,f,v′,d)

[0102] Else

[0103] dataMap = updateDataMap(dataMap,f,v,d)

[0104] Fill in the CRF form:

[0105]

[0106] (v,d) = getLatestValue(dataMap,f).

[0107] It should be understood that although the steps in the flowcharts of the embodiments of the present invention are shown sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0108] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0109] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0110] The above embodiments only represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent for the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be subject to the appended claims.

[0111] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A unified AI multimodal data collection method, characterized in that: The method specifically comprises the following steps: Collect data from multiple preset data sources to obtain raw data; Preprocess the raw data, extract features based on the AI ​​large language model, and obtain feature data; Performing data cleaning on the characteristic data to obtain data variables; Matching and filling the data variables with a preset form template to generate form data; The form data is stored in a database.

2. The unified AI multimodal data collection method according to claim 1, characterized in that: The original data includes audio data, picture data, PDF data and / or text data.

3. The unified AI multimodal data collection method according to claim 2, characterized in that: The data preprocessing of the raw data and feature extraction based on the AI ​​large language model to obtain feature data specifically includes the following steps: Removing noise and / or abnormal format data from the original data to obtain valid data; Based on the AI ​​large language model, feature extraction is performed on the valid data to obtain feature data.

4. The unified AI multimodal data collection method according to claim 3, characterized in that: The removing of noise and / or abnormal format data in the original data to obtain valid data comprises: performing audio noise reduction processing on the audio data; The image data is subjected to image enhancement and text area positioning processing; and the PDF data is subjected to recognition processing.

5. The unified AI multimodal data collection method according to claim 3, characterized in that: Based on the AI ​​big language model, feature extraction is performed on the valid data to obtain feature data: based on the AI ​​big language model, multiple key features are extracted from the text data, including word segmentation, part-of-speech tagging and named entities; the AI ​​big language model and OCR technology are combined to identify the text content in the image data and perform key extraction; based on the AI ​​big language model, the audio data is spectrally analyzed to extract acoustic features.

6. The unified AI multimodal data collection method according to claim 1, characterized in that: The data cleaning of the feature data to obtain data variables specifically includes the following steps: Identifying redundant data and erroneous data in the feature data; The redundant data and the erroneous data are cleaned to obtain data variables.

7. The unified AI multimodal data collection method according to claim 1, characterized in that: The step of matching and filling the data variables with the preset form template to generate form data specifically includes the following steps: Matching the data variable with a preset form template to determine a matching position; Extract the value of the specified variable; The specified variable value is automatically filled into the matching position to generate form data.

8. The unified AI multimodal data collection method according to claim 7, characterized in that: The data variables are matched with a preset form template, and in determining a matching position, the specified variables are matched, redundant variables are filtered, and the field position in the form template is determined to obtain a matching position.

9. The unified AI multimodal data collection method according to claim 1, characterized in that: The storing of the form data in a database specifically comprises the following steps: Matching storage locations; According to the storage location, storing the form data in a database; The form data is version controlled.

10. Unified AI multimodal data collection platform, characterized by: The platform includes a data acquisition unit, a data preprocessing unit, a data cleaning unit, a data matching and filling unit, and a data storage unit, wherein: A data acquisition unit, used to collect data from multiple preset data sources to obtain raw data; A data preprocessing unit, used to perform data preprocessing on the raw data, and perform feature extraction based on an AI large language model to obtain feature data; A data cleaning unit, used to clean the characteristic data to obtain data variables; A data matching and filling unit, used to match and fill the data variables with a preset form template to generate form data; The data storage unit is used to store the form data in a database.

Images