Data table storage and management method and system

Abstract

The present invention provides a method for storing and managing data tables, comprising the following steps: receiving metadata formed by creating one or more form structures, parsing the metadata to obtain the form name, group name, data column number, stored data name, and stored data type included in the form structure, where the stored data type includes one or more data types; receiving form data filled in the form structure; creating an extended table structure, the extended table structure having extended table fields corresponding to the form name, group name, and data column number, and uniformly setting the type of data stored in the extended table structure to a first data type; filling the form data into the extended table structure based on the form name, group name, and data column number corresponding to the form data, where the type of data filled into the extended table structure is set to the first data type. The data table storage solution provided by this application can be compatible with different types of custom forms at the same time, realizing efficient storage and management of data.

Description

Technical Field

[0001] The present invention mainly relates to data management, and in particular to a method for storing and managing data tables. Background Art

[0002] In data management applications, the metadata and data tables of MDM (Master Data Management) data are stored in a business database. At the same time, the data tables correspond one by one to the forms defined by the metadata to address data analysis problems under the EAV (entity-attribute-value) model design. However, in this solution, all the data structures defined by EAV need to be instantiated into specific database tables, that is, a large number of library tables will be generated. The total number of online MDM tables can even reach approximately hundreds of thousands or more. In this case, whether using a general server storage or a server cluster storage solution, it will face the dilemma of a huge number of files and a sharp increase in the difficulty of management and maintenance (which can be called the explosion of quantity). Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a data storage solution that can be compatible with different types of custom forms at the same time, and achieve efficient storage and management of data tables.

[0004] To solve the above technical problem, the present invention provides a method for storing and managing data tables, including the following steps: receiving metadata formed by creating one or more form structures, parsing the metadata to obtain the form name, group name, data column number, stored data name, and stored data type included in the form structure, where the stored data type includes one or more data types; receiving form data filled in the form structure; creating an extended table structure, the extended table structure having extended table fields corresponding to the form name, group name, and data column number, and uniformly setting the type of data stored in the extended table structure to a first data type; filling the form data into the extended table structure based on the form name, group name, and data column number corresponding to the form data, where the type of data filled into the extended table structure is set to the first data type.

[0005] In an embodiment of the present invention, the method for storing and managing data tables further includes: receiving a read / write operation request for the form data; parsing the read / write operation request to obtain the form name, group name, and data column number to which the form data corresponding to the read / write operation request belongs; locating the corresponding extended table data in the extended table structure based on the form name, group name, and data column number, and obtaining the data type of the corresponding extended table data from the metadata; forming output data based on the data type obtained from the metadata and the extended table data to respond to the read / write operation request.

[0006] In an embodiment of the present invention, forming output data based on the data type obtained from the metadata and the extended table data in the method for storing and managing data tables to respond to the read / write operation request includes: judging whether the obtained data type is consistent with the first data type based on the data type obtained from the metadata; if not, performing type conversion on the corresponding extended table data, and using the data after the type conversion as the output data to respond to the read / write operation request; if consistent, using the extended table data as the output data to respond to the read / write operation request.

[0007] In an embodiment of the present invention, the one or more data types include integer type, floating-point type, character type, and / or string type.

[0008] In an embodiment of the present invention, in the extended table structure, the group name includes an extended table group style name and an extended table instance name.

[0009] In an embodiment of the present invention, the number of fields included in the extended table structure is greater than or equal to the sum of the corresponding numbers of the form name, group name, and data column number.

[0010] In an embodiment of the present invention, the stored data name is the clinical trial collection data name, and the clinical trial collection data name includes subject information name, drug information name, and / or test and assay item name.

[0011] In an embodiment of the present invention, the extended table structure and the extended table data are stored in a MySQL database.

[0012] The present invention also provides a data table storage and management system, including: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the method as described in any of the above embodiments.

[0013] The present invention also provides a computer-readable medium storing computer program code, and the computer program code implements the method as described in any of the above embodiments when executed by a processor.

[0014] Compared with the prior art, the data table storage solution provided by the present invention can be compatible with different types of custom forms at the same time, enabling data conversion, association, and calculation, being more flexible and avoiding waste of storage resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are provided to further understand the present application. They are incorporated into and constitute a part of this application. The accompanying drawings illustrate embodiments of the present application and, together with this specification, serve to explain the principles of the present application. In the accompanying drawings:

[0016] Figure 1 is a flowchart of a method for storing and managing data tables according to an embodiment of the present application;

[0017] Figure 2 is a flowchart of a method for storing and managing data tables according to another embodiment of the present application;

[0018] Figure 3 is a flowchart of a method for storing and managing data tables according to still another embodiment of the present application;

[0019] Figure 4 is a schematic diagram of a metadata storage method;

[0020] Figure 5 is a schematic diagram of an extended table structure according to an embodiment of the present application;

[0021] Figure 6 is a schematic diagram of a data table storage and management system according to an embodiment of the present application; DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present application. For those of ordinary skill in the art, without creative efforts, the present application can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the drawings represent the same structure or operation.

[0023] As shown in the present application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0024] Unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present application. In all the examples shown and discussed herein, any specific values should be construed as merely exemplary and not as limiting. Thus, other examples of the exemplary embodiments may have different values. It should be noted that like reference numerals and letters denote like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

[0025] In the present application, flowcharts are used to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the operations described above or below are not necessarily executed precisely in sequence. Instead, they may be executed in reverse order or simultaneously. Also, one or more operations may be added to these processes, or one or more operations may be removed from these processes.

[0026] Figure 1 It is a flowchart of a method for storing and managing data tables according to an embodiment of the present application. Refer to Figure 1 As shown, the present invention provides a method for storing and managing data tables, including step 101: receiving metadata formed by creating one or more form structures, parsing the metadata to obtain the form name, group name, data column number, stored data name, and stored data type included in the form structure, where the stored data type includes one or more data types; step 102: receiving form data filled in the form structure; step 103: creating an extended table structure, where the extended table structure has extended table fields corresponding to the form name, group name, and data column number, and the type of data stored in the extended table structure is uniformly set to a first data type; step 104: filling the form data into the extended table structure based on the form name, group name, and data column number corresponding to the form data, where the type of data filled into the extended table structure is set to the first data type.

[0027] Optionally, in some embodiments of the present invention, one or more data types include integer (int), floating point (float), character (char), and / or string (string).

[0028] Optionally, in some embodiments of the present invention, the stored data name includes a clinical trial collection data name, and the clinical trial collection data name includes a subject information name, a drug information name, and / or a test and assay item name. The subject information name includes a unified identification number, basic information such as height / weight / age, the experimental stage, the name of the project participated in, etc. The drug information name, for example, includes the drug name, administration time, and administration route. The test and assay item name, for example, includes the names of items such as blood pressure, blood sugar, blood routine test, body fluid assay, secretion assay, metabolite assay, and imaging examination.

[0029] Figure 4 It is a storage schematic diagram of some metadata (also known as a metadata table). Figure 4 The metadata in it is presented in the form of a table. In fact, the metadata represents the mapping relationship of the form structure, that is, the mapping relationship of each field in the form structure is stored through the metadata. Figure 4 Shown below is an example of a metadata table. In this example, there are two types of metadata tables in total. The metadata table g1 represents that the form structure has five fields, namely: name, height, weight, gender code, and gender, and their data types are: string type, integer type, floating-point type, integer type, and string type respectively. Figure 4 In it, f1 is the form name (or also called the form type name). Group g1 and group g2 ( Figure 4 written as the group name in this case) belong to the same form type f1.

[0030] For the convenience of understanding, the following is a form structure corresponding to a metadata table g1:

[0031] Table 1

[0032] Name Height Weight Gender Code Gender Zhang San 165 60.0 0 Female Li Si 182 78.2 1 Male

[0033] In the actual application scenario, users of the data collection and management system (such as users of the clinical trial data collection and management system, including medical staff, experiment managers, and experiment participants) create a form structure such as shown in Table 1 on the front-end interface of the system for operations such as data collection and viewing. After the form structure is created on the front end (including the interface that directly interacts with users, such as the web version interface), corresponding metadata will be formed on the database on the server side. When maintaining the system, the maintenance and management personnel of the data collection and management system can view and operate on the data through the database tables corresponding to the metadata. For example, Figure 4 the metadata corresponding to group g1 and group g2 under the form name f1 shown in it will form corresponding database tables. However, as the number of form structures created by front-end users increases, that is, the number of forms increases, the number of database tables will also increase accordingly, which will bring the problem of a sharp increase in the number of database tables as described above and cannot be effectively managed and maintained. Therefore, in the technical solution of this application, a large number of database tables are no longer generated, but an extended table is formed to facilitate the maintenance and management of the system. For details, please continue to refer to the description in this article.

[0034] The meaning of Table 1 above is that a patient named Zhang San has a height of 165, a weight of 60.0, and a gender code of 0, which means female; another patient named Li Si has a height of 182, a weight of 78.2, and a gender code of 1, which means male. In this embodiment, the units of height and weight are centimeters (cm) and kilograms (kg) respectively. This unit can be the default or set by the actual operator. For example, when the unit of height in this embodiment is modified to meters (m), the data content should become 1.65 and 1.82.

[0035] Similarly, the metadata table g2 has four fields, namely: drug name, administration time, administration route number, and administration route, and their data types are: string type, long integer type, string type, and string type respectively. For easy understanding, a form structure corresponding to the metadata table g2 is given below (its specific meaning will not be elaborated here):

[0036] Table 2

[0037] Drug Name Administration Time Administration Route Code Administration Route Amoxicillin 2020-01-22 09:00:01 01 Oral Penicillin 2020-01-23 10:00:01 02 Injection

[0038] Figure 5 is a schematic diagram of the extended table structure of an embodiment of the present application. With reference to Figure 1 、 4 -5 and Tables 1-2 given above, Tables 1 and 2 are the form structures, Figure 4 shown as metadata (or metadata table), Figure 5 the shown Form ID is the form name, Group ID is the group name, and column1-column 5 are the data column numbers.

[0039] In this embodiment, a data table storage and management method provided by the present application creates an extended table structure as Figure 5 shown. This extended table structure contains extended table fields corresponding to the form name, group name, and data column numbers, that is, extended table fields corresponding to the form name f1, group name g1, group name g2, and data column numbers column1 to data column number column5, etc., and fills in the form data contained in the form structures shown in Tables 1 and 2. The data types stored in the extended table structure are, for example, uniformly set to the first data type.

[0040] Specifically, as Figure 5As shown, the name of the extended table form is f1, which contains content from two types of metadata, g1 and g2. Although the number of data columns in the two forms is originally different, they are both represented by column1 to column5 in the extended table structure. It should be emphasized that in this extended table structure, the meanings of column1 to column5 are different according to different group names. For example, when the group name is g1, its column1 to 5 are name, height, weight, gender number, and gender respectively; when the group name is g2, its column1 to 4 are drug name, administration time, administration route number, and administration route respectively, and column5 is empty.

[0041] Optionally, in this embodiment, the first data type is string type. In some other embodiments, the first data type can also be integer type, floating-point type, character type, etc., and the present application does not make specific limitations here. Further, the first data type can be set as needed, and the present application does not make specific limitations here.

[0042] Figures 4 to 5 In the shown drawings, the form name is f1, which is only an example here. Figure 4 The shown group g1 and group g2 belong to the form name (or form type name) f1. The form type name also includes, for example, f2, f3,..., which include corresponding groups. Figure 5 The dimension (including the number of rows and columns) of the shown extended table can actually be more, for example, including dozens of columns or more. Figure 4 The corresponding columns of each row in the shown metadata (metadata table) are filled into the extended table from the left. The extra columns corresponding to the extended table maintain the initial value, such as being empty (NULL) or zero value, which actually depends on the type of the initial value of the extended table.

[0043] Figure 2 It is a flowchart of a data table storage and management method according to another embodiment of the present application. Refer to Figure 2 As shown, in an embodiment of the present invention, the data table storage and management method further includes: Step 201: Receive a read / write operation request for form data; Step 202: Parse the read / write operation request to obtain the form name, group name, and data column number of the form data corresponding to the read / write operation request; Step 203: Locate the corresponding extended table data in the extended table structure based on the form name, group name, and data column number, and obtain the data type of the corresponding extended table data from the metadata; Step 204: Form output data based on the data type obtained from the metadata and the extended table data to respond to the read / write operation request.

[0044] For better understanding, in combination with reference to Figures 3 - 5A description will be given with reference to Table 1-2. Specifically, in this embodiment, when a user (also referred to as an operator) of the clinical trial data collection and management system needs to query the administration time of amoxicillin for a patient, the user sends a read / write operation request through an operation. The data table storage and management method of the present application receives this request, parses it to obtain the form name f1, group name g2, and data column number column2 to which the form data "amoxicillin" belongs. Further, based on the above content, the corresponding extended table data is located in the extended table structure, that is, column2 of gv3 in group g2 of the f1 extended table structure, and its content indicates that the administration time of amoxicillin is 2020-01-22 09:00:01. The data type of the corresponding extended table data is obtained from the metadata g2 table. In this embodiment, the data type of the administration time content in the metadata g2 is long integer (long).

[0045] In an embodiment of the present invention, in the extended table structure, the group name includes an extended table group style name and an extended table group instance name. For example, in Figure 5 the extended table structure shown, the group name g1 may include the extended table group style name g1 and the extended table group instance name gv1. The same label is used for the group name g1 and the extended table group style name g1 here to represent the association before and after the data is filled into the extended table. In fact, different labels can also be used.

[0046] In the actual operation process, when the operator needs to query data, the first data type may not be consistent with the data type in the metadata, so it may cause the output to fail or not meet the subsequent processing requirements. In this case, it is necessary to correct the data type before outputting the data.

[0047] Figure 3 This is a flowchart of a data table storage and management method according to another embodiment of the present application. In Figure 3 the embodiment shown, forming output data based on the data type obtained from the metadata and the extended table data in response to a read / write operation request (i.e., the aforementioned step 204) in the data table storage and management method includes: Step 204a: Based on the data type obtained from the metadata, determine whether the obtained data type is consistent with the first data type; Step 204b: If they are not consistent, perform type conversion on the corresponding extended table data, and use the data after type conversion as the output data to respond to the read / write operation request. Step 204c: If they are consistent, use the extended table data as the output data to respond to the read / write operation request.

[0048] Further, in some other embodiments of the present application, the extended table data can also be calculated and then used as output data. For example, when there is an association between the administered dose and the patient's weight, age, etc., the calculated administered dose is used as the output data to respond to the read / write operation request.

[0049] In an embodiment of the present invention, the number of fields included in the extended table structure is greater than or equal to the sum of the corresponding numbers of the form name, group name, and data column number to meet the mapping and filling of table data.

[0050] In an embodiment of the present invention, the extended table structure and extended table data are stored in a MySQL database. MySQL is a relational database. Or rather, it is an associative database management system. The associative database stores data in different tables, so it can improve the response speed and increase the flexibility of the application.

[0051] The technical solution of the present application combines the data tables corresponding to multiple forms (or called database tables, library tables) to form an extended table, which is convenient for the unified management and maintenance of a large number of data tables. Moreover, through the type conversion operation, while meeting the requirements of unified storage and management, the accurate invocation of data in the data table is realized.

[0052] Figure 6 It is a schematic diagram of a data table storage and management system according to an embodiment of the present application. An embodiment of the present invention also proposes a data table storage and management system 30 as shown in Figure 6 According to Figure 6 , the data table storage and management system 30 may include an internal communication bus 31, a processor 32, a read-only memory (ROM) 33, a random access memory (RAM) 34, a hard disk storage 36, and a communication port 35. The data table storage and management system of the present application runs on the server side, for example, and the server side can communicate with the front end on the mobile device to realize data transfer.

[0053] The internal communication bus 31 can realize data communication between the components of the data table storage and management system 300. The processor 32 can make judgments and issue prompts. In some embodiments, the processor 32 may be composed of one or more processors. The communication port 35 can realize data communication between the data table storage and management system 30 and the outside. In some embodiments, the data table storage and management system 30 can send and receive information and data from the network through the communication port 35.

[0054] The data table storage and management system 30 may also include different forms of program storage units and data storage units, such as a hard disk 36, a read-only memory (ROM) 53, and a random access memory (RAM) 34, which are capable of storing various data files used for computer processing and / or communication, as well as possible program instructions executed by the processor 32. The processor executes these instructions to implement the main part of the method. The results processed by the processor are transmitted to the user device through the communication port and displayed on the user interface.

[0055] In addition, on the other hand, the present invention also proposes a computer-readable medium storing computer program code, and the computer program code implements the above-mentioned data table storage and management method when executed by a processor.

[0056] Some aspects of the present application can be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above-mentioned hardware or software can all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". The processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. In addition, aspects of the present application may be embodied as a computer product located in one or more computer-readable media, and the product includes computer-readable program coding. For example, the computer-readable media may include, but are not limited to, magnetic storage devices (such as hard disks, floppy disks, magnetic tapes...), optical discs (such as compact discs CD, digital versatile discs DVD...), smart cards, and flash memory devices (such as cards, sticks, key drives...).

[0057] The computer-readable media may contain a propagated data signal containing computer program coding, such as on a baseband or as part of a carrier wave. The propagated signal may have various forms of manifestation, including electromagnetic form, optical form, etc., or a suitable combination of forms. The computer-readable media can be any computer-readable media other than computer-readable storage media, and this medium can be connected to an instruction execution system, device, or equipment to implement communication, propagation, or transmission of a program for use. The program coding located on the computer-readable media can be propagated through any suitable medium, including radio, cable, fiber optic cable, radio frequency signal, or similar media, or any combination of the above media.

[0058] Meanwhile, this application uses specific terms to describe the embodiments of this application. For example, "an embodiment", "one embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0059] Similarly, it should be noted that, in order to simplify the description of this application disclosure and thus help the understanding of one or more inventive embodiments, in the previous description of the embodiments of this application, sometimes multiple features are grouped into one embodiment, drawing, or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this application are more than those mentioned in the claims. In fact, the features of the embodiment are fewer than all the features of the single embodiment disclosed above.

[0060] Although this application has been described with reference to the current specific embodiments, those of ordinary skill in the art should recognize that the above embodiments are only used to illustrate this application, and various equivalent changes or substitutions can be made without departing from the spirit of this application. Therefore, as long as the changes and modifications to the above embodiments are within the scope of the spirit of this application, they will fall within the scope of the claims of this application.

Claims

1. A method for storing and managing data tables, comprising the following steps: Receiving metadata created by one or more form structures, parsing the metadata to obtain the form name, group name, data column number, stored data name, and stored data type included in the form structure, where the stored data type includes one or more data types; Receiving form data filled in the form structure; Creating an extended table structure, where the extended table structure has extended table fields corresponding to the form name, group name, and data column number, and the type of data stored in the extended table structure is uniformly set to the first data type; Filling the form data into the extended table structure based on the form name, group name, and data column number corresponding to the form data, where the type of data filled into the extended table structure is set to the first data type; Receiving a read / write operation request for the form data; Parsing the read / write operation request to obtain the form name, group name, and data column number of the form data corresponding to the read / write operation request; Locating the corresponding extended table data in the extended table structure based on the form name, group name, and data column number, and obtaining the data type of the corresponding extended table data from the metadata; Forming output data based on the data type obtained from the metadata and the extended table data to respond to the read / write operation request.

2. The data table storage and management method according to claim 1, characterized in that Forming output data based on the data type obtained from the metadata and the extended table data to respond to the read / write operation request includes: Based on the data type obtained from the metadata, determining whether the obtained data type is consistent with the first data type; If not consistent, performing a type conversion on the corresponding extended table data, and using the data after the type conversion as the output data to respond to the read / write operation request; If consistent, using the extended table data as the output data to respond to the read / write operation request.

3. The data table storage and management method according to claim 1, characterized in that, The one or more data types include integer type, floating-point type, character type, and / or string type.

4. The method for storing and managing data tables according to claim 1, wherein, In the extended table structure, the group name includes an extended table group style name and an extended table group instance name.

5. The method for storing and managing a data table according to claim 1, wherein The number of fields included in the extended table structure is greater than or equal to the sum of the numbers corresponding to the form name, group name, and data column number.

6. The method for storing and managing data tables according to claim 1, wherein The stored data name includes a clinical trial collection data name, and the clinical trial collection data name includes a subject information name, a drug information name, and / or a test and assay item name.

7. The method for storing and managing a data table according to claim 1, wherein The extended table structure and the extended table data are stored in a MySQL database.

8. A data table storage and management system, comprising: A memory for storing instructions executable by a processor; And A processor for executing the instructions to implement the method according to any one of claims 1-7.

9. A computer-readable medium storing computer program code, where the computer program code implements the method according to any one of claims 1-7 when executed by a processor.

Images