Data fusion method and device, equipment and storage medium

Abstract

The invention discloses a data fusion method, apparatus and device, and a storage medium. The method comprises the steps of obtaining scale data in each database; by means of a preset semantic analysis model, scale fields in the scale data are transferred into standard names in the CDISC standard, and a field mapping relation and a field value mapping relation are constructed; converting the scale data into target data conforming to the CDISC standard based on the field mapping relation and the field value mapping relation; and performing data fusion on the target data according to the weight of each database by adopting a weighted average method to obtain a data fusion result. The scale field in the scale data is transferred into the standard name by utilizing the preset semantic analysis model, the scale data is converted into the target data meeting the CDISC standard on the basis of the field mapping relation and the field value mapping relation, and finally, the target data is fused by adopting the weighted average method, so that the data fusion efficiency is improved. And the efficiency and reusability of cross-column research data fusion are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data fusion method, device, equipment and storage medium. Background Art

[0002] With the development of mobile Internet, the data generated is also growing explosively, and more and more knowledge metadata are scattered on the Internet. However, due to the heterogeneity of content on the Internet and its loose organizational structure, it has brought great challenges to the knowledge interconnection in the big data environment. Therefore, it is necessary to explore the knowledge interconnection method that is in line with the development and changes of network information resources and can adapt to the cognitive needs of users from a new perspective based on the knowledge organization principles in the big data environment, so as to reveal the integrity and relevance of human cognition at a deeper level.

[0003] Cross-cohort research data refers to a data set obtained by aggregating and fusing cohort data from different sources and types. In the medical field, with the development of medical and health informationization and medical information systems, a huge amount of medical cohort research data has been accumulated. Existing data fusion is mainly achieved with the help of data integration platforms, which not only requires users to manually select and configure mapping relationships, but also has a complex and error-prone process, takes a long time, and lacks a unified cross-cohort fusion standard. Each cohort uses a customized standard, which is not conducive to reuse.

[0004] Therefore, there is an urgent need for a data fusion method that can effectively improve the efficiency and reusability of cross-column research data fusion. Summary of the invention

[0005] The main purpose of the present invention is to provide a data fusion method, device, equipment and storage medium, aiming to solve the technical problems of low efficiency and reusability of cross-column research data fusion in the prior art.

[0006] To achieve the above object, the present invention provides a data fusion method, which comprises the following steps:

[0007] Obtaining scale data from each database;

[0008] Using a preset semantic analysis model, the scale fields in the scale data are converted into standard names in the CDISC standard, and a field mapping relationship and a field value mapping relationship are constructed;

[0009] Converting the scale data into target data that complies with the CDISC standard based on the field mapping relationship and the field value mapping relationship;

[0010] The weighted average method is adopted to perform data fusion on the target data according to the weight of each database to obtain a data fusion result.

[0011] Optionally, after the step of adopting a weighted average method to fuse the target data according to the weights of the databases to obtain a data fusion result, the step further includes:

[0012] Converting the data fusion result into database data according to the field mapping relationship and the field value mapping relationship;

[0013] Determining a data format of a target database, and determining a format conversion rule based on the database data and the data format of the target database;

[0014] The database data is structurally spliced ​​based on the format conversion rule to obtain target data that meets the requirements of the target database.

[0015] Optionally, after the step of adopting a weighted average method to fuse the target data according to the weights of the databases to obtain a data fusion result, the step further includes:

[0016] Monitor the data fusion process in real time and generate fusion monitoring logs based on the monitoring results;

[0017] Evaluate the data fusion quality based on the preset data fusion evaluation index and the fusion monitoring log to obtain an evaluation result;

[0018] When the evaluation result indicates that the data fusion quality does not meet the preset fusion requirements, an intelligent alarm mechanism is used to generate early warning prompt information according to the evaluation result.

[0019] Optionally, the step of using a preset semantic analysis model to convert the scale fields in the scale data into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship includes:

[0020] Performing semantic analysis on the scale fields in the scale data using a preset semantic analysis model, and extracting semantic features of the scale fields according to the semantic analysis results;

[0021] Intelligently search and match the semantic features with the SDTM standard in the CDISC standard to obtain the matching result;

[0022] Based on the matching result, the table fields are translated into standard names in the CDISC standard, and a field mapping relationship and a field value mapping relationship are constructed.

[0023] Optionally, the step of intelligently searching and matching the semantic features with the SDTM standard in the CDISC standard to obtain the matching result includes:

[0024] Based on the semantic features, intelligent retrieval is performed in the SDTM standard in the CDISC standard to find the standard field entity corresponding to the semantic features;

[0025] Intelligent matching is performed based on the standard field entity to determine the standard name and label value in the CDISC standard corresponding to each of the semantic features;

[0026] The standard names and vertebral values ​​in the CDISC standard corresponding to each of the semantic features are used as matching results.

[0027] Optionally, the step of converting the meter data into target data that complies with the CDISC standard based on the field mapping relationship and the field value mapping relationship includes:

[0028] Cleaning the scale data according to the numerical range of the CDISC standard to obtain cleaned scale data;

[0029] According to the format requirements of the CDISC standard, the format of the cleaned scale data is standardized to obtain processed scale data;

[0030] The processed scale data is normalized according to the field mapping relationship and the field value mapping relationship to obtain target data that complies with the CDISC standard.

[0031] Optionally, the step of adopting a weighted average method to fuse the target data according to the weights of the databases to obtain a data fusion result includes:

[0032] According to the purpose of data fusion, correlation analysis is performed on the target data of each database to obtain correlation analysis results;

[0033] Determine the corresponding weight assigned to each of the databases based on the correlation analysis result;

[0034] The weighted average method is adopted to perform data fusion on the target data according to the weight of each database to obtain a data fusion result.

[0035] In addition, to achieve the above object, the present invention also provides a data fusion device, the device comprising:

[0036] A data acquisition module is used to acquire the scale data in each database;

[0037] A relationship building module, used to use a preset semantic analysis model to convert the scale fields in the scale data into standard names in the CDISC standard, and to build a field mapping relationship and a field value mapping relationship;

[0038] A data conversion module, used for converting the meter data into target data conforming to the CDISC standard based on the field mapping relationship and the field value mapping relationship;

[0039] The data fusion module is used to adopt a weighted average method to fuse the target data according to the weights of the databases to obtain a data fusion result.

[0040] In addition, to achieve the above objectives, the present invention also proposes a data fusion device, which includes: a memory, a processor, and a data fusion program stored in the memory and executable on the processor, wherein the data fusion program is configured to implement the steps of the data fusion method described above.

[0041] In addition, to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which a data fusion program is stored, and when the data fusion program is executed by a processor, the steps of the data fusion method described above are implemented.

[0042] The present invention discloses the method of obtaining scale data in each database; using a preset semantic analysis model, converting the scale fields in the scale data into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship; based on the field mapping relationship and the field value mapping relationship, converting the scale data into target data that conforms to the CDISC standard; using a weighted average method, performing data fusion on each target data according to the weight of each database, and obtaining a data fusion result. Since the present invention uses a preset semantic analysis model, converting the scale fields in the scale data in each database into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship, converting the scale data into target data that conforms to the CDISC standard, and finally using a weighted average method to perform data fusion on each target data, compared with the prior art, the present invention effectively improves the efficiency and reusability of cross-column research data fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of the first embodiment of the data fusion method of the present invention;

[0044] Figure 2 It is a schematic diagram of the process of intelligent matching of scale fields in the data fusion method of the present invention;

[0045] Figure 3 It is a flow chart of the second embodiment of the data fusion method of the present invention;

[0046] Figure 4 It is a schematic diagram of the flow of data processing in the data fusion method of the present invention;

[0047] Figure 5It is a flowchart of the third embodiment of the data fusion method of the present invention;

[0048] Figure 6 It is a schematic diagram of the technical principle of the data fusion method of the present invention;

[0049] Figure 7 It is a structural block diagram of the first embodiment of the data fusion device of the present invention;

[0050] Figure 8 It is a structural diagram of a data fusion device of a hardware operating environment involved in an embodiment of the present invention.

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0053] The embodiment of the present invention provides a data fusion method, referring to Figure 1 , Figure 1 FIG. 1 is a flow chart of the first embodiment of the data fusion method of the present invention.

[0054] In this embodiment, the data fusion method includes steps S10 to S40:

[0055] Step S10: Obtaining the meter data in each database.

[0056] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication and program running functions in the fusion scenario of medical cohort research data, such as a personal computer, a laptop computer, a mobile phone, a smart watch, etc., or an electronic device capable of realizing the above functions, a data fusion device, etc. The following takes a system including a data fusion device (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.

[0057] It needs to be explained that the above-mentioned databases can be databases of different sources and types, and the scale data in each database can represent medical research data across cohorts.

[0058] Step S20: using a preset semantic analysis model, converting the scale fields in the scale data into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship.

[0059] It should be noted that the above preset semantic analysis model can be a BERT model, which is a natural language processing model proposed by Google. It learns contextual information through a bidirectional encoder to make the understanding of words more accurate. The core of BERT is the Transformer architecture, which captures the relationship between each word and other words in a sentence through a self-attention mechanism. Unlike traditional unidirectional language models, BERT can understand the meaning of words from both left and right directions, so it performs well in a variety of NLP tasks, such as question-answering systems and text classification.

[0060] CDISC (Clinical Data Interchange Standards Consortium) standards: CDISC (Clinical Data Interchange Standards Consortium) standards are a set of standards that provide a unified framework for the collection, storage and exchange of clinical research data. It defines the structure and format of data to ensure that data between different systems and studies can be exchanged consistently and accurately. The main standards include SDTM (Study Data Table Standard), ADaM (Analysis Data Model) and CDASH (Data Collection Standard), which help improve data quality and efficiency.

[0061] In a specific implementation, a preset semantic analysis model can be used to perform semantic analysis on the scale fields in the scale data, and the semantic features of the scale fields can be extracted based on the semantic analysis results; the semantic features are intelligently retrieved and matched with the SDTM standard in the CDISC standard to obtain the matching results; based on the matching results, the scale fields are translated into standard names in the CDISC standard, and field mapping relationships and field value mapping relationships are constructed.

[0062] It should be explained that the step of intelligently searching and matching the semantic features with the SDTM standard in the CDISC standard to obtain the matching results includes: based on the semantic features, intelligently searching the SDTM standard in the CDISC standard to find the standard field entity corresponding to the semantic features; intelligently matching based on the standard field entities to determine the standard names and vertebral values ​​in the CDISC standard corresponding to each of the semantic features; and taking the standard names and vertebral values ​​in the CDISC standard corresponding to each of the semantic features as the matching results.

[0063] It should be noted that the SDTM (Study Data Tabulation Model) standard is part of the CDISC standard, which defines a standard structure for the tabulation and submission of human clinical trial (research) data. SDTM aims to simplify the data collection, management, analysis and reporting process and improve the efficiency of the audit and approval process of regulatory agencies. In addition, SDTM also provides standards for the submission of non-clinical data, including data in the fields of medical devices, pharmacogenomics and genetics research.

[0064] For example, refer to Figure 2 , Figure 2 It is a flowchart of intelligent matching of scale fields in the data fusion method of the present invention. Agent represents an intelligent agent, which is a core concept of artificial intelligence and is defined as an entity that can autonomously perceive information in its environment and make decisions based on this information to achieve specific goals or tasks. The system can use the BERT model to perform deep semantic recognition of scale fields, accurately extract their semantic features, and perform intelligent retrieval and intelligent matching with the SDTM standard in the CDISC standard. This process includes searching for CDISC standard entities that meet the requirements in the CDISC standard to ensure that the selected fields are fully compliant with the specifications. Next, the system performs further semantic analysis based on the standard value of the field and the scale parameters, constructs the corresponding relationship between the fields (i.e., the field mapping relationship), and generates the corresponding optionMapping (i.e., the field value mapping relationship) to determine the CDISC standard naming and CDISC standard value corresponding to the scale field from the CDISC standard entity through semantic recognition and intelligent matching. These steps effectively and precisely connect the scale fields to the CDISC standards, thereby automatically forming a final system entity that complies with the standards and saving this information in the database. This not only improves the standardization efficiency of the scale fields, but also significantly reduces manual intervention, ensuring consistency and accuracy in the data processing process.

[0065] Step S30: converting the scale data into target data that complies with the CDISC standard based on the field mapping relationship and the field value mapping relationship.

[0066] In order to ensure the consistency and compatibility of the data and ultimately make the data comply with the CDISC standard and prepare for subsequent data fusion, it is necessary to convert the scale data into target data that complies with the CDISC standard based on the field mapping relationship and the field value mapping relationship.

[0067] In a specific implementation, the scale data can be cleaned according to the numerical range of the CDISC standard to obtain the cleaned scale data; the format of the cleaned scale data can be standardized according to the format requirements of the CDISC standard to obtain the processed scale data; the processed scale data can be normalized according to the field mapping relationship and the field value mapping relationship to obtain the target data that complies with the CDISC standard.

[0068] It should be noted that data cleaning can include removing errors in the data and making judgments based on the numerical range of the CDISC standard to ensure the accuracy and completeness of the data. Format standardization can convert the date formats from different source databases into a unified format specified by the CDISC standard to ensure the consistency of the data during integration. Data normalization can convert the values ​​corresponding to the scale fields into standardized enumeration values ​​based on the field mapping relationship and the field value mapping relationship. This step ensures the consistency and compatibility of the data, and ultimately makes the data comply with the CDISC standard, preparing for subsequent data fusion.

[0069] Step S40: using a weighted average method to fuse the target data according to the weights of the databases to obtain a data fusion result.

[0070] In order to solve the data conflict problem, this embodiment adopts a weighted average method to effectively handle the data conflict problem of the same attribute from different source databases, thereby generating a standard data set.

[0071] In a specific implementation, a correlation analysis can be performed on the target data of each of the databases according to the purpose of data fusion to obtain a correlation analysis result; a corresponding weight assigned to each of the databases is determined based on the correlation analysis result; and a weighted average method is used to perform data fusion on each of the target data according to the weight of each of the databases to obtain a data fusion result.

[0072] This embodiment discloses obtaining the scale data in each database; using a preset semantic analysis model, converting the scale fields in the scale data into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship; based on the field mapping relationship and the field value mapping relationship, converting the scale data into target data that conforms to the CDISC standard; using the weighted average method, performing data fusion on each target data according to the weight of each database, and obtaining a data fusion result. Since this embodiment uses a preset semantic analysis model to convert the scale fields in the scale data of each database into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship, converting the scale data into target data that conforms to the CDISC standard, and finally using the weighted average method to perform data fusion on each target data, compared with the prior art, this embodiment effectively improves the efficiency and reusability of cross-column research data fusion.

[0073] refer to Figure 3 , Figure 3 FIG. 4 is a flow chart of a second embodiment of a data fusion method according to the present invention.

[0074] Based on the first embodiment described above, in this embodiment, after step S40, steps S501 to S503 are further included:

[0075] Step S501: converting the data fusion result into database data according to the field mapping relationship and the field value mapping relationship.

[0076] Step S502: determining the data format of the target database, and determining a format conversion rule based on the database data and the data format of the target database.

[0077] Step S503: Structural splicing of the database data is performed based on the format conversion rule to obtain target data that meets the requirements of the target database.

[0078] For example, refer to Figure 4 , Figure 4 The figure is a flow chart of data processing in the data fusion method of the present invention. The figure includes three key stages: conversion, fusion and reverse conversion. The fusion engine of the present invention systematically processes the data of different source databases through these three stages to ensure the accuracy and consistency of the data, thereby providing users with a high-quality fusion database (i.e., target database).

[0079] The conversion stage is the first step of data fusion, which aims to lay a solid foundation for data fusion. First, this stage performs data cleaning and format standardization. Data cleaning includes removing errors in the data and making judgments based on the numerical range of the CDISC standard to ensure the accuracy and completeness of the data. Format standardization converts the date formats from different source databases into a unified format specified by the CDISC standard to ensure the consistency of the data during integration. Next, data normalization is performed to convert the data in the database into standardized enumeration values ​​based on the scale mapping rules (i.e., field mapping relationship and field value mapping relationship). This step ensures the consistency and compatibility of the data, and ultimately makes the data comply with the CDISC standard, preparing for subsequent data fusion.

[0080] The fusion stage is the second step of data fusion, and its core is to merge the transformed data into a consistent view. This process involves integrating information from multiple databases to form a complete and unified data record. In order to solve the data conflict problem, this embodiment adopts the weighted average method, which allows the user to assign different weights to each source database when starting the fusion, and perform weighted average on these data values ​​to obtain the final result (CDISC standard data set, i.e., data fusion result). In this way, data conflicts of the same attributes from different source databases can be effectively handled, thereby generating a standard data set.

[0081] The reverse conversion stage is the third step of data fusion, which aims to convert the fused standard data set (data fusion result) into a database format normally used in the system so that researchers can effectively use these data to carry out new research projects. In this stage, the present invention proposes a reverse conversion method. Specifically, according to the set mapping relationship (i.e., field mapping relationship and field value mapping relationship), the fused standard data set is reversely converted into a data format that meets the system usage requirements. This conversion improves the practicality of the data, allowing researchers to more conveniently and intuitively apply these data for in-depth analysis and research, and can conduct subsequent follow-up updates based on this database.

[0082] This embodiment discloses obtaining the scale data in each database; using a preset semantic analysis model, converting the scale fields in the scale data into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship; based on the field mapping relationship and the field value mapping relationship, converting the scale data into target data that conforms to the CDISC standard; using a weighted average method, performing data fusion on each target data according to the weight of each database to obtain a data fusion result; converting the data fusion result into database data according to the field mapping relationship and the field value mapping relationship; determining the data format of the target database, and determining a format conversion rule based on the database data and the data format of the target database; structurally splicing the database data based on the format conversion rule to obtain target data that conforms to the requirements of the target database. Compared with the prior art, this embodiment effectively improves the practicality of the fused data.

[0083] refer to Figure 5 , Figure 5 FIG. 4 is a flow chart of a third embodiment of a data fusion method according to the present invention.

[0084] Based on the above embodiments, in this embodiment, after step S40, steps S601 to S603 are further included:

[0085] Step S601: Monitor the data fusion process in real time and generate a fusion monitoring log according to the monitoring results.

[0086] Step S602: Evaluate the data fusion quality based on the preset data fusion evaluation index and the fusion monitoring log to obtain an evaluation result.

[0087] Step S603: When the evaluation result indicates that the data fusion quality does not meet the preset fusion requirements, an intelligent alarm mechanism is used to generate early warning information according to the evaluation result.

[0088] It should be noted that the preset data fusion evaluation indicators may include data fusion completion, error number and level, etc.

[0089] It should be understood that this embodiment realizes real-time monitoring of the entire data fusion process, tracks data changes and processing indicators, detects potential problems such as data loss or delay, and records detailed logs. The intelligent alarm mechanism can immediately issue an alarm when a serious anomaly is detected, and notify relevant personnel via email and SMS so that users can respond quickly and take corrective measures to ensure the stability of the data fusion process and the high quality of the fused data.

[0090] For example, refer to Figure 6 , Figure 6The schematic diagram of the technical principle of the data fusion method of the present invention. The CDISC standard definition module is used to manage the standardization of scales and fields in each database. The module allows users to easily view and manage the set standard fields through a visual interface, and supports manual inspection and personalized settings for fields that have not set standards. The system can display the standard naming and options of the fields to help users clarify the CDISC mapping relationship of the fields. The IDs, standard naming, and option mapping relationships of all fields are stored in the database to ensure the persistence and consistency of the data. Through this data structure, the module can efficiently manage the standardization process of the fields, ensure the consistency of data between different systems, and simplify data fusion. The fusion engine systematically processes the data of different source databases through three key stages: conversion, fusion, and reverse conversion to ensure the accuracy and consistency of the data, thereby providing users with a high-quality target database. In terms of data fusion quality monitoring, the system focuses on fusion anomaly monitoring, data quality reporting, data monitoring, and alarming. The data mapping rules and field conversion rules represent the field value mapping relationship and the field mapping relationship respectively. After acquiring the scale data in each database and enabling the agent, the present invention performs standard retrieval and standard support through the CDISC standard library (including the research data table standard SDTM, domain Domain, and variable Variable).

[0091] This embodiment discloses real-time monitoring of the data fusion process, and generating a fusion monitoring log according to the monitoring results; evaluating the data fusion quality based on preset data fusion evaluation indicators and the fusion monitoring log to obtain an evaluation result; when the evaluation result indicates that the data fusion quality does not meet the preset fusion requirements, using an intelligent alarm mechanism to generate early warning information according to the evaluation result. Compared with the prior art, this embodiment ensures the stability of the data fusion process and the quality of the fused data.

[0092] In addition, an embodiment of the present invention further provides a storage medium, on which a data fusion program is stored. When the data fusion program is executed by a processor, the steps of the data fusion method described above are implemented.

[0093] Reference Figure 7 , Figure 7 It is a structural block diagram of the first embodiment of the data fusion device of the present invention.

[0094] like Figure 7 As shown, the data fusion device proposed in the embodiment of the present invention includes: a data acquisition module 701, a relationship construction module 702, a data conversion module 703 and a data fusion module 704.

[0095] The data acquisition module 701 is used to acquire the meter data in each database.

[0096] The relationship building module 702 is used to use a preset semantic analysis model to convert the scale fields in the scale data into standard names in the CDISC standard, and to build a field mapping relationship and a field value mapping relationship.

[0097] The data conversion module 703 is used to convert the meter data into target data that complies with the CDISC standard based on the field mapping relationship and the field value mapping relationship.

[0098] The data fusion module 704 is used to adopt a weighted average method to fuse the target data according to the weights of the databases to obtain a data fusion result.

[0099] The embodiment of the present device discloses obtaining the scale data in each database; using a preset semantic analysis model, converting the scale fields in the scale data into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship; based on the field mapping relationship and the field value mapping relationship, converting the scale data into target data that conforms to the CDISC standard; using the weighted average method, performing data fusion on each target data according to the weight of each database, and obtaining a data fusion result. Since the embodiment of the present device uses a preset semantic analysis model, converting the scale fields in the scale data of each database into standard names in the CDISC standard, and constructing a field mapping relationship and a field value mapping relationship, converting the scale data into target data that conforms to the CDISC standard, and finally using the weighted average method to perform data fusion on each target data, compared with the prior art, the embodiment of the present device effectively improves the efficiency and reusability of cross-column research data fusion.

[0100] Based on the above-mentioned first embodiment of the data fusion device of the present invention, a second embodiment of the data fusion device of the present invention is proposed.

[0101] In this embodiment, the data fusion module 704 is also used to convert the data fusion result into database data according to the field mapping relationship and the field value mapping relationship; determine the data format of the target database, and determine the format conversion rules based on the database data and the data format of the target database; structurally splice the database data based on the format conversion rules to obtain target data that meets the requirements of the target database.

[0102] Other embodiments or specific implementations of the data fusion device of the present invention may refer to the above-mentioned method embodiments, which will not be described in detail here.

[0103] The present application provides a data fusion device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data fusion method in the above-mentioned embodiment 1.

[0104] Reference below Figure 8 , which shows a schematic diagram of the structure of a data fusion device suitable for implementing the embodiment of the present application. The data fusion device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The data fusion device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0105] like Figure 8 As shown, the data fusion device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data fusion device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data fusion device to communicate with other devices wirelessly or by wire to exchange data. Although the data fusion device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0106] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0107] The data fusion device provided by the present application adopts the data fusion method in the above embodiment, which can solve the technical problems of low efficiency and reusability of cross-column research data fusion in the prior art. Compared with the prior art, the beneficial effects of the data fusion device provided by the present application are the same as the beneficial effects of the data fusion method provided by the above embodiment, and other technical features in the data fusion device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0108] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0109] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0110] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0111] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0113] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data fusion method, characterized in that: The method comprises: Obtaining scale data from each database; Using a preset semantic analysis model, the scale fields in the scale data are converted into standard names in the CDISC standard, and a field mapping relationship and a field value mapping relationship are constructed; Converting the scale data into target data that complies with the CDISC standard based on the field mapping relationship and the field value mapping relationship; The weighted average method is adopted to perform data fusion on the target data according to the weight of each database to obtain a data fusion result.

2. The data fusion method according to claim 1, characterized in that: After the step of adopting the weighted average method to fuse the target data according to the weights of the databases to obtain the data fusion result, the method further includes: Converting the data fusion result into database data according to the field mapping relationship and the field value mapping relationship; Determining a data format of a target database, and determining a format conversion rule based on the database data and the data format of the target database; The database data is structurally spliced ​​based on the format conversion rule to obtain target data that meets the requirements of the target database.

3. The data fusion method according to claim 1, characterized in that: After the step of adopting the weighted average method to fuse the target data according to the weights of the databases to obtain the data fusion result, the method further includes: Monitor the data fusion process in real time and generate fusion monitoring logs based on the monitoring results; Evaluate the data fusion quality based on the preset data fusion evaluation index and the fusion monitoring log to obtain an evaluation result; When the evaluation result indicates that the data fusion quality does not meet the preset fusion requirements, an intelligent alarm mechanism is used to generate early warning prompt information according to the evaluation result.

4. The data fusion method according to claim 1, characterized in that: The step of using a preset semantic analysis model to convert the scale fields in the scale data into standard names in the CDISC standard and constructing a field mapping relationship and a field value mapping relationship includes: Performing semantic analysis on the scale fields in the scale data using a preset semantic analysis model, and extracting semantic features of the scale fields according to the semantic analysis results; Intelligently search and match the semantic features with the SDTM standard in the CDISC standard to obtain the matching result; Based on the matching result, the table fields are translated into standard names in the CDISC standard, and a field mapping relationship and a field value mapping relationship are constructed.

5. The data fusion method according to claim 4, characterized in that: The step of intelligently searching and matching the semantic features with the SDTM standard in the CDISC standard to obtain the matching result comprises: Based on the semantic features, intelligent retrieval is performed in the SDTM standard in the CDISC standard to find the standard field entity corresponding to the semantic features; Intelligent matching is performed based on the standard field entity to determine the standard name and label value in the CDISC standard corresponding to each of the semantic features; The standard names and vertebral values ​​in the CDISC standard corresponding to each of the semantic features are used as matching results.

6. The data fusion method according to claim 1, characterized in that: The step of converting the meter data into target data that complies with the CDISC standard based on the field mapping relationship and the field value mapping relationship includes: Cleaning the scale data according to the numerical range of the CDISC standard to obtain cleaned scale data; According to the format requirements of the CDISC standard, the format of the cleaned scale data is standardized to obtain processed scale data; The processed scale data is normalized according to the field mapping relationship and the field value mapping relationship to obtain target data that complies with the CDISC standard.

7. The data fusion method according to claim 1, characterized in that: The step of adopting the weighted average method to fuse the target data according to the weights of the databases to obtain the data fusion result includes: According to the purpose of data fusion, correlation analysis is performed on the target data of each database to obtain correlation analysis results; Determine the corresponding weight assigned to each of the databases based on the correlation analysis result; The weighted average method is adopted to perform data fusion on the target data according to the weight of each database to obtain a data fusion result.

8. A data fusion device, characterized in that: The device comprises: A data acquisition module is used to acquire the scale data in each database; A relationship building module, used to use a preset semantic analysis model to convert the scale fields in the scale data into standard names in the CDISC standard, and to build a field mapping relationship and a field value mapping relationship; A data conversion module, used for converting the meter data into target data conforming to the CDISC standard based on the field mapping relationship and the field value mapping relationship; The data fusion module is used to adopt a weighted average method to fuse the target data according to the weights of the databases to obtain a data fusion result.

9. A data fusion device, characterized in that: The device comprises: a memory, a processor, and a data fusion program stored in the memory and executable on the processor, wherein the data fusion program is configured to implement the steps of the data fusion method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a data fusion program, and when the data fusion program is executed by the processor, the steps of the data fusion method according to any one of claims 1 to 7 are implemented.

Images