Knowledge base management method, system, terminal and storage medium based on data governance

Abstract

The application relates to the technical field of big data analysis, and specifically provides a knowledge base management method and system based on data management, a terminal and a storage medium, which comprise the following steps: obtaining an initial data table and metadata of the initial data table in a source database from the source database; converting the initial data table into a standard data template, binding the metadata with the standard data template, and deleting the initial data table; extracting identity information from the standard data template, generating a unique code for the identity information based on a preset identity information conversion rule, and setting the unique code as an index number of the standard data template; extracting keywords from content information of the standard data template, screening matched terms from a term library based on the keywords, and converting the content information into the matched terms. The application eliminates the large consumption of storage resources caused by storing initial data tables, guarantees the standardization of data storage, and improves data query efficiency.

Description

Technical Field

[0001] This invention belongs to the field of big data analytics technology, specifically relating to a knowledge base management method, system, terminal, and storage medium based on data governance. Background Technology

[0002] Health and medical big data is characterized by its wide distribution, numerous nodes, and significant management challenges. It involves numerous medical institutions, manufacturers, and business systems, and the quality of data governance is greatly influenced by the professional competence of personnel. Currently, establishing a health and medical big data knowledge base faces challenges due to the large volume of data, numerous interfacing parties, and inconsistent data table formats across different hospital databases, leading to chaotic data processing, hindering subsequent data analysis and querying. Furthermore, synchronizing all these data tables to the knowledge base results in substantial consumption of storage resources. Summary of the Invention

[0003] To address the problems of chaotic data formats hindering analysis and querying, and the excessive storage resources consumed by initial data tables in existing technologies, this invention provides a knowledge base management method, system, terminal, and storage medium based on data governance to solve the aforementioned technical problems.

[0004] In a first aspect, the present invention provides a knowledge base management method based on data governance, comprising: Obtain the initial data table and its metadata from the source database. Convert the initial data table into a standard data template, bind the metadata to the standard data template, and delete the initial data table; Identity information is extracted from a standard data template, a unique code is generated for the identity information based on a preset identity information conversion rule, and the unique code is set as the index number of the standard data template; Keyword extraction is performed on the content information of the standard data template, and matching terms are selected from the terminology database based on the keywords to convert the content information into the matching terms.

[0005] In an optional implementation, the initial data table is converted into a standard data template, the metadata is bound to the standard data template, and the initial data table is deleted, including: The initial data table is converted into database table fields by calling an SQL script, and the database table fields are written into a standard template to obtain a standard data template.

[0006] In one optional implementation, identity information is extracted from a standard data template, a unique code is generated for the identity information based on a preset identity information conversion rule, and the unique code is set as the index number of the standard data template, including: The identity information is parsed into information items and information item content; The information item is queried for a matching code based on a preset information item code; Calculate the MD5 value of the information item content, and combine the encoding with the MD5 value to form a unique code; Select key information items from the information items as primary information items, and use the unique code composed of the primary information item code and the MD5 value of the primary information item content as the primary index number; When multiple primary index numbers exist, use an OR statement for querying.

[0007] In an optional implementation, the method further includes: Receive a query request, extract identity information from the query request, and convert the identity information into a target unique code; Determine whether the target unique code contains the target primary index number: If so, query the target standard data template containing the target master index number in the unique code, and extract the initial data table from the source database based on the metadata bound to the target standard data template. Then, return the queried target standard data template and the initial data table to the requesting end. If not, query the target standard data template containing the target unique code in the unique code, group the target standard data templates according to whether the primary index number in the unique code is the same or related, and feed the grouping results back to the requesting end for selection. After receiving the selection instruction, feed the corresponding target standard data template and the corresponding initial data table back to the requesting end.

[0008] Secondly, the present invention provides a knowledge base management system based on data governance, comprising: The data acquisition module is used to acquire the initial data table and the metadata of the initial data table in the source database from the source database; The data conversion module is used to convert the initial data table into a standard data template, bind the metadata to the standard data template, and delete the initial data table. The index setting module is used to extract identity information from a standard data template, generate a unique code for the identity information based on a preset identity information conversion rule, and set the unique code as the index number of the standard data template. The standard processing module is used to extract keywords from the content information of the standard data template, and to filter matching terms from the terminology database based on the keywords, thereby converting the content information into the matching terms.

[0009] In an optional implementation, the data conversion module includes: The script processing unit is used to call an SQL script to convert the initial data table into database table fields and write the database table fields into a standard template to obtain a standard data template.

[0010] In an optional implementation, the index setting module includes: The information parsing unit is used to parse identity information into information items and information item content; The encoding query unit is used to query the matching encoding of the information item according to the preset information item encoding; A code generation unit is used to calculate the MD5 value of the information item content and combine the encoding with the MD5 value to form a unique code; The index setting unit is used to select key information items from the information items as the main information items, and use the unique code composed of the main information item code and the MD5 value of the main information item content as the main index number; The query settings unit is used to perform queries using OR statements when multiple primary index numbers exist.

[0011] In an optional implementation, the system further includes: The request parsing module is used to receive query requests, extract identity information from the query requests, and convert the identity information into a target unique code. The index determination module is used to determine whether the target unique code contains the target primary index number; The first processing module is used to query the target standard data template containing the target primary index number in the unique code if the target unique code contains the target primary index number, and extract the initial data table from the source database based on the metadata bound to the target standard data template, and feed back the queried target standard data template and the initial data table to the requesting end. The second processing module is used to query the target standard data template that contains the target unique code if the target unique code does not contain the target primary index number, group the target standard data template according to whether the primary index number in the unique code is the same or related, and feed the grouping results back to the requesting end for selection. After receiving the selection instruction, the corresponding target standard data template and the corresponding initial data table are fed back to the requesting end.

[0012] Thirdly, a terminal is provided, including: Processor, memory, among which, This memory is used to store computer programs. The processor is used to retrieve and run the computer program from memory, causing the terminal to perform the terminal method described above.

[0013] Fourthly, a computer storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the methods described in the above aspects.

[0014] The beneficial effects of the present invention are that the knowledge base management method, system, terminal and storage medium based on data governance provided by the present invention, by converting the initial data table into a data template and saving only the metadata of the initial data table, eliminates the large consumption of storage resources of storing the initial data table, while ensuring the standardization of data storage, providing convenience for big data analysis, and improving data query efficiency by establishing an index with identity information.

[0015] Furthermore, the design principle of this invention is reliable, the structure is simple, and it has a very wide range of application prospects. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0018] Figure 2 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0022] The knowledge base management method based on data governance provided in this embodiment of the invention is executed by a computer device, and correspondingly, the knowledge base management system based on data governance runs on the computer device.

[0023] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The implementing entity can be a data governance-based knowledge base management system. Depending on different needs, the order of the steps in this flowchart can be changed, and some can be omitted.

[0024] like Figure 1 As shown, the method includes: Step 110: Obtain the initial data table and its metadata from the source database. Step 120: Convert the initial data table into a standard data template, bind the metadata to the standard data template, and delete the initial data table; Step 130: Extract identity information from the standard data template, generate a unique code for the identity information based on the preset identity information conversion rules, and set the unique code as the index number of the standard data template; Step 140: Extract keywords from the content information of the standard data template, and filter matching terms from the terminology database based on the keywords to convert the content information into the matching terms.

[0025] To facilitate understanding of the present invention, the following description further illustrates the knowledge base management method based on data governance provided by the present invention, using the principles of the data governance-based knowledge base management method and the process of managing the knowledge base based on data governance in the embodiments.

[0026] Specifically, the knowledge base management method based on data governance includes: S1. Obtain the initial data table and its metadata from the source database.

[0027] Establish network connections with the management consoles of multiple source databases and periodically collect updated initial data tables and metadata from the source databases.

[0028] S2. Convert the initial data table into a standard data template, bind the metadata to the standard data template, and delete the initial data table.

[0029] The initial data table is converted into database table fields by calling an SQL script, and the database table fields are written into a standard template to obtain a standard data template.

[0030] Specifically, a standard template is pre-set, which includes identity information, diagnosis details, and treatment plan items. The SQL script parses the contents of the initial data table and imports them into the corresponding items of the standard template to obtain the standard data template.

[0031] S3. Extract identity information from the standard data template, generate a unique code for the identity information based on the preset identity information conversion rules, and set the unique code as the index number of the standard data template.

[0032] The identity information is parsed into information items and information item content; the information item is queried and matched according to the preset information item code; the MD5 value of the information item content is calculated, and the code and the MD5 value are combined into a unique code; key information items are selected from the information items as primary information items, and the unique code composed of the primary information item code and the MD5 value of the primary information item content is used as the primary index number; when there are multiple primary index numbers, an OR statement is used for querying.

[0033] For example, identity information includes name, ID card number, and address. The name is coded as A, the ID card number as B, and the address as C. Calculate the MD5 hash of the name (M1), ID card number (M2), and address (M3) respectively. The unique code for this identity information is then AM1-BM2-CM3. Set this unique code as the filename of the standard data template.

[0034] If the ID card number is set as the primary information item, then the primary index number of this identity information is BM2. If the name is also set as the primary information item, then this identity information has two primary index numbers, namely AM1 and BM2. The two are associated, that is, the unique codes containing AM1 or containing BM2 are in the same group.

[0035] S4. Extract keywords from the content information of the standard data template, and filter matching terms from the terminology database based on the keywords, and convert the content information into the matching terms.

[0036] Keywords were extracted from the diagnostic and treatment plan items in the standard data template using the NLTK (Natural Language Toolkit) library. The terms with the highest similarity to the keywords were then queried from a pre-built terminology database, with Euclidean distance used for similarity calculation. The diagnostic and treatment plan items were then transformed into terms.

[0037] Configure field cleaning rules, regular expression cleaning rules, and complex logic cleaning rules to clean the data in the knowledge base and handle invalid, missing, and erroneous data.

[0038] The HIPAA's data anonymization scheme is adopted. HIPAA is divided into five parts (Titles), each addressing a specific issue in healthcare reform. The second issue addressed by the bill is simplified administration, focusing on healthcare-related information security. HIPAA establishes a comprehensive set of standards for receiving, transmitting, and maintaining healthcare information, ensuring the security of privacy and personally identifiable information. The core of this part is the privacy and security protection of information such as PHI (Protected Health Information). To ensure that patient medical privacy data is not leaked, the anonymization of patient-related privacy data must first be completed (using a static anonymization method).

[0039] S4. Query request processing.

[0040] Receive a query request, extract identity information from the query request, and convert the identity information into a target unique code.

[0041] Determine whether the target unique code contains the target primary index number, that is, whether the target unique code contains the encoding that protects the primary index number; If the target unique code contains the target primary index number, then query the target standard data template that contains the target primary index number in the unique code, and extract the initial data table from the source database based on the metadata bound to the target standard data template. Then, return the queried target standard data template and the initial data table to the requesting end.

[0042] If the target unique code does not contain the target primary index number, then the target standard data template containing the target unique code is queried. The target standard data templates are grouped according to whether the primary index numbers in the unique code are the same or related. Target standard data templates with the same or related primary index numbers in the unique code are grouped together. The grouping results are fed back to the requesting end for selection. The requesting end selects the group to view. After receiving the selection instruction, the initial data table of the corresponding target standard data template is obtained based on the bound metadata. The corresponding target standard data template and the corresponding initial data table are fed back to the requesting end.

[0043] In some embodiments, the data governance-based knowledge base management system may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the data governance-based knowledge base management system may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) Functionality of knowledge base management based on data governance.

[0044] In this embodiment, the data governance-based knowledge base management system can be divided into multiple functional modules according to its functions, such as... Figure 2As shown. The functional modules of system 200 may include: a data acquisition module 210, a data conversion module 220, an index setting module 230, and a standard processing module 240. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0045] The data acquisition module is used to acquire the initial data table and the metadata of the initial data table in the source database from the source database; The data conversion module is used to convert the initial data table into a standard data template, bind the metadata to the standard data template, and delete the initial data table. The index setting module is used to extract identity information from a standard data template, generate a unique code for the identity information based on a preset identity information conversion rule, and set the unique code as the index number of the standard data template. The standard processing module is used to extract keywords from the content information of the standard data template, and to filter matching terms from the terminology database based on the keywords, thereby converting the content information into the matching terms.

[0046] Optionally, as an embodiment of the present invention, the data conversion module includes: The script processing unit is used to call an SQL script to convert the initial data table into database table fields and write the database table fields into a standard template to obtain a standard data template.

[0047] Optionally, as an embodiment of the present invention, the index setting module includes: The information parsing unit is used to parse identity information into information items and information item content; The encoding query unit is used to query the matching encoding of the information item according to the preset information item encoding; A code generation unit is used to calculate the MD5 value of the information item content and combine the encoding with the MD5 value to form a unique code; The index setting unit is used to select key information items from the information items as the main information items, and use the unique code composed of the main information item code and the MD5 value of the main information item content as the main index number; The query settings unit is used to perform queries using OR statements when multiple primary index numbers exist.

[0048] Optionally, as an embodiment of the present invention, the system further includes: The request parsing module is used to receive query requests, extract identity information from the query requests, and convert the identity information into a target unique code. The index determination module is used to determine whether the target unique code contains the target primary index number; The first processing module is used to query the target standard data template containing the target primary index number in the unique code if the target unique code contains the target primary index number, and extract the initial data table from the source database based on the metadata bound to the target standard data template, and feed back the queried target standard data template and the initial data table to the requesting end. The second processing module is used to query the target standard data template that contains the target unique code if the target unique code does not contain the target primary index number, group the target standard data template according to whether the primary index number in the unique code is the same or related, and feed the grouping results back to the requesting end for selection. After receiving the selection instruction, the corresponding target standard data template and the corresponding initial data table are fed back to the requesting end.

[0049] Figure 3 This is a schematic diagram of the structure of a terminal 300 provided in an embodiment of the present invention. The terminal 300 can be used to execute the knowledge base management method based on data governance provided in the embodiment of the present invention.

[0050] The terminal 300 may include a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0051] The memory 320 can be used to store the execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile memory terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 is able to perform some or all of the steps in the above method embodiments.

[0052] The processor 310 serves as the control center of the storage terminal, connecting various parts of the electronic terminal via various interfaces and lines. It executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0053] The communication unit 330 is used to establish a communication channel, enabling the storage terminal to communicate with other terminals. It can receive user data sent by other terminals or send user data to other terminals.

[0054] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0055] Therefore, by converting the initial data table into a data template and saving only the metadata of the initial data table, this invention eliminates the large consumption of storage resources by storing the initial data table, while ensuring the standardization of data storage and providing convenience for big data analysis. At the same time, the indexing based on identity information improves data query efficiency. The technical effects achieved by this embodiment can be found in the description above, and will not be repeated here.

[0056] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or other media capable of storing program code. It includes several instructions to cause a computer terminal (which may be a personal computer, server, or a second terminal, network terminal, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0057] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0058] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0059] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0060] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0061] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the invention should also be covered within the protection scope of the invention. Therefore, the protection scope of the invention should be determined by the scope of the claims.

Claims

1. A knowledge base management method based on data governance, characterized in that, include: Obtain the initial data table and its metadata from the source database. A standard template is pre-set, which includes items such as identity information, diagnosis content, and treatment plan. Call the SQL script to convert the initial data table into database table fields, and write the database table fields into the corresponding items of the standard template to obtain the standard data template. Bind the metadata to the standard data template and delete the initial data table. The process involves extracting identity information from a standard data template, generating a unique code for the identity information based on preset identity information conversion rules, and setting the unique code as the index number of the standard data template. Specifically, this includes: parsing the identity information into information items and information item content; using preset information item encoding to generate a matching encoding for the information item query; calculating the MD5 value of the information item content and combining the encoding with the MD5 value to form a unique code; selecting key information items from the information items as primary information items, and using the unique code composed of the primary information item encoding and the MD5 value of the primary information item content as the primary index number. Keyword extraction is performed on the content information of the standard data template, and matching terms are selected from the terminology database based on the keywords to convert the content information into the matching terms; The method further includes: Receive a query request, extract identity information from the query request, and convert the identity information into a target unique code; Determine whether the target unique code contains the target primary index number: If so, query the target standard data template containing the target master index number in the unique code, and extract the initial data table from the source database based on the metadata bound to the target standard data template. Then, return the queried target standard data template and the initial data table to the requesting end. If not, query the target standard data template containing the target unique code in the unique code, group the target standard data templates according to whether the primary index number in the unique code is the same or related, and feed the grouping results back to the requesting end for selection. After receiving the selection instruction, feed the corresponding target standard data template and the corresponding initial data table back to the requesting end.

2. The method according to claim 1, characterized in that, Extracting identity information from a standard data template, generating a unique code for the identity information based on a preset identity information conversion rule, and setting the unique code as the index number of the standard data template, further includes: When multiple primary index numbers exist, use an OR statement for querying.

3. A knowledge base management system based on data governance, characterized in that, include: The data acquisition module is used to obtain the initial data table and the metadata of the initial data table in the source database from the source database; The data conversion module is used to convert the initial data table into database table fields by calling an SQL script based on a pre-set standard template, and write the database table fields into the corresponding items of the standard template to obtain a standard data template. The metadata is then bound to the standard data template and the initial data table is deleted. The standard template includes items for identity information, diagnosis details, and treatment plan. The index setting module is used to extract identity information from a standard data template, generate a unique code for the identity information based on a preset identity information conversion rule, and set the unique code as the index number of the standard data template. The standard processing module is used to extract keywords from the content information of the standard data template, and to filter matching terms from the terminology database based on the keywords, thereby converting the content information into the matching terms. The data conversion module includes: The script processing unit is used to call an SQL script to convert the initial data table into database table fields and write the database table fields into a standard template to obtain a standard data template. The index setting module includes: The information parsing unit is used to parse identity information into information items and information item content; The encoding query unit is used to query the matching encoding of the information item according to the preset information item encoding; A code generation unit is used to calculate the MD5 value of the information item content and combine the encoding with the MD5 value to form a unique code; The index setting unit is used to select key information items from the information items as the main information items, and use the unique code composed of the main information item code and the MD5 value of the main information item content as the main index number; The system also includes: The request parsing module is used to receive query requests, extract identity information from the query requests, and convert the identity information into a target unique code. The index determination module is used to determine whether the target unique code contains the target primary index number; The first processing module is used to query the target standard data template containing the target primary index number in the unique code if the target unique code contains the target primary index number, and extract the initial data table from the source database based on the metadata bound to the target standard data template, and feed back the queried target standard data template and the initial data table to the requesting end. The second processing module is used to query the target standard data template that contains the target unique code if the target unique code does not contain the target primary index number, group the target standard data template according to whether the primary index number in the unique code is the same or related, and feed the grouping results back to the requesting end for selection. After receiving the selection instruction, the corresponding target standard data template and the corresponding initial data table are fed back to the requesting end.

4. The system according to claim 3, characterized in that, The index setting module also includes: The query settings unit is used to perform queries using OR statements when multiple primary index numbers exist.

5. A terminal, characterized in that, include: The memory is used to store the knowledge base management program based on data governance; A processor, configured to implement the steps of the data governance-based knowledge base management method as described in any one of claims 1-2 when executing the data governance-based knowledge base management program.

6. A computer-readable storage medium storing a computer program, characterized in that, The readable storage medium stores a knowledge base management program based on data governance, which, when executed by a processor, implements the steps of the knowledge base management method based on data governance as described in any one of claims 1-2.

Images