Big data query method based on database and related device

Abstract

The present application provides a database-based big data query method and related devices. The method at least includes: in response to a user's data query request for a target object, a preliminary search of the target object in the display layer is performed to obtain the first index information of the target object; a first granularity index structure of the first index information is pre-established with a granularity level of at least the form level; based on the first index information, a deep query is performed on the target object in the index layer to obtain the second index information corresponding to the target object; a second granularity index structure of the second index information is pre-established with a granularity level of at least the variable level; the variable data associated with the second index information is queried, and the queried variable data is aggregated and processed as the target query result fed back to the user. Through a double-layer index structure and independent variable query, the data query efficiency is optimized and the applicable scenarios of the database are expanded.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a database-based big data query method and related devices. Background Art

[0002] In clinical medical research, medical data collection and management is one of the most core links.

[0003] In the related technologies, clinical research data collection systems often face problems such as large data volume, diverse data types, and complex query requirements. Existing database systems often find it difficult to simultaneously meet the requirements of high flexibility and high query efficiency. For example, traditional relational databases are less efficient when processing highly free data structure customization, and non-relational databases can process unstructured data, but perform poorly in complex queries and data integrity assurance.

[0004] Therefore, in view of the deficiencies in the prior art, it is urgent to propose a new database system to solve at least one of the above technical problems. Summary of the invention

[0005] The embodiment of the present application provides a database-based big data query method and related devices, which are used to achieve high-degree-of-freedom data structure customization through a double-layer index structure and independent variable query, provide efficient database-based big data query capabilities, and ensure data security and privacy. Thus, data query efficiency is optimized and the applicable scenarios of the database are expanded.

[0006] In a first aspect, an embodiment of the present application provides a database-based big data query method, the method is applied to a big data dynamic query system of a medical database, the system includes at least a display layer and an index layer, and a multi-layer index structure of multiple variable data is stored in the index layer, the method includes:

[0007] In response to a user's data query request for a target object, a preliminary search is performed on the target object in the presentation layer to obtain first index information of the target object; the first index information is pre-established by a first granularity index structure of a database; the granularity level of the first granularity index structure is at least a form level;

[0008] According to the first index information, a deep query is performed on the target object in the index layer to obtain second index information corresponding to the target object; the second index information is pre-established using a second granularity index structure of a database based on variable attributes of variable objects in the medical field; the granularity level of the second granularity index structure is at least a variable level;

[0009] Querying the variable data associated with the second index information, and performing aggregation processing on the queried variable data to provide the target query result as feedback to the user;

[0010] The queried variable data comes from at least one dynamic form associated with the second index information; and the second granularity index structure is dynamically adjusted based on the query frequency and data update status of the variable data in each dynamic form.

[0011] In a second aspect, the present application provides a big data query device based on a database, the device is applied to a big data dynamic query system of a medical database, the system comprises at least a display layer and an index layer, a multi-layer index structure of multiple variable data is stored in the index layer, the device comprises:

[0012] A search module is used to perform a preliminary search on the target object in the presentation layer in response to a user's data query request for the target object, so as to obtain first index information of the target object; the first index information is pre-established by a first granularity index structure of the database; the granularity level of the first granularity index structure is at least a form level;

[0013] A query module, configured to perform a deep query on the target object in the index layer according to the first index information to obtain second index information corresponding to the target object; the second index information is pre-established based on the variable attributes of the variable object in the medical field using a second granularity index structure of the database; the granularity level of the second granularity index structure is at least the variable level;

[0014] A feedback module, used for querying the variable data associated with the second index information, and performing aggregation processing on the queried variable data to provide feedback to the user as a target query result;

[0015] The queried variable data comes from at least one dynamic form associated with the second index information; and the second granularity index structure is dynamically adjusted based on the query frequency and data update status of the variable data in each dynamic form.

[0016] In a third aspect, the present application provides an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, and the device is used to implement the steps of the database-based big data query method provided in any embodiment of the present application.

[0017] In a sixth aspect, the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a storage controller, implements the steps of a database-based big data query method as provided in any embodiment of the present application.

[0018] Compared with the prior art, this application has at least the following beneficial effects:

[0019] The present application provides a database-based big data query method, which is applied to a big data dynamic query system of a medical database. The system includes at least a display layer and an index layer, and a multi-layer index structure of multiple variable data is stored in the index layer. In this method, in response to a user's data query request for a target object, a preliminary search is performed on the target object in the display layer to obtain the first index information of the target object; the first index information is pre-established by the first granular index structure of the database; the granularity level of the first granular index structure is at least the form level. Further, according to the first index information, a deep query is performed on the target object in the index layer to obtain the second index information corresponding to the target object; the second index information is pre-established using the second granular index structure of the database based on the variable attributes of the variable object in the medical field; the granularity level of the second granular index structure is at least the variable level. Finally, the variable data associated with the second index information is queried, and the queried variable data is aggregated and processed as the target query result fed back to the user. Among them, the queried variable data comes from at least one dynamic form associated with the second index information; the second granular index structure is dynamically adjusted based on the query frequency and data update status of the variable data in each dynamic form. In summary, through the double-layer index structure and independent variable query, a high degree of freedom in data structure customization is achieved, providing efficient database-based big data query capabilities and ensuring data security and privacy. This optimizes data query efficiency and expands the application scenarios of the database.

[0020] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of a big data query method based on a database is shown in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of the principle of a database-based big data query method shown in an embodiment of the present application;

[0023] Figure 3a A schematic diagram of an interface shown in an embodiment of the present application;

[0024] Figure 3b A schematic diagram of the structure of a database shown in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the structure of a big data query device based on a database shown in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of a database-based big data query device shown in an embodiment of the present application.

[0027] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0030] In clinical medical research, medical data collection and management is one of the most core links.

[0031] In the related technologies, clinical research data collection systems often face problems such as large data volume, diverse data types, and complex query requirements. Existing database systems often find it difficult to simultaneously meet the requirements of high flexibility and high query efficiency. For example, traditional relational databases are less efficient when processing highly free data structure customization, and non-relational databases can process unstructured data, but perform poorly in complex queries and data integrity assurance.

[0032] In addition, in related technologies, the security and privacy protection of clinical research data are also issues that cannot be ignored. The data collection system needs to ensure the privacy of data during transmission and storage, and comply with relevant legal and ethical regulations during data access and processing, which leads to complex data collection and management processes and a large workload.

[0033] Therefore, in view of the deficiencies in the prior art, it is urgent to propose a new database system to solve at least one of the above technical problems.

[0034] The embodiment of the present application provides a database-based big data query method provided by the present application, and the method is applied to a big data dynamic query system of a medical database, wherein the system includes at least a display layer and an index layer, and a multi-layer index structure of multiple variable data is stored in the index layer. In the method, in response to a user's data query request for a target object, a preliminary search is performed on the target object in the display layer to obtain the first index information of the target object; the first index information is pre-established by the first granular index structure of the database; the granularity level of the first granular index structure is at least the form level. Further, according to the first index information, a deep query is performed on the target object in the index layer to obtain the second index information corresponding to the target object; the second index information is pre-established using the second granular index structure of the database based on the variable attributes of the variable object in the medical field; the granularity level of the second granular index structure is at least the variable level. Finally, the variable data associated with the second index information is queried, and the queried variable data is aggregated and processed as the target query result fed back to the user. Among them, the queried variable data comes from at least one dynamic form associated with the second index information; the second granular index structure is dynamically adjusted based on the query frequency and data update status of the variable data in each dynamic form.

[0035] The database-based big data query method provided by this application realizes high-degree-of-freedom data structure customization through a double-layer index structure and independent variable query, provides efficient database-based big data query capabilities, and ensures data security and privacy. Thus, data query efficiency is optimized and the applicable scenarios of the database are expanded.

[0036] The database-based big data query solution provided in the embodiment of the present application can also be executed by an electronic device, which can be a server, a server cluster, or a cloud server. The electronic device can also be a terminal device such as a mobile phone, a computer, a tablet computer, a wearable device, or a special device (such as a special terminal device with a database-based big data query system, etc.). These electronic devices can also be equipped with the chips introduced in the above embodiments. Alternatively, these electronic devices can also be installed with a service program for executing a database-based big data query solution.

[0037] To solve the above problems, please refer to Figure 1 , Figure 1 The following is a flow chart of a method for querying big data based on a database according to an embodiment of the present application. Figure 1 As shown, at least the following steps are included:

[0038] S101 , in response to a user's data query request for a target object, perform a preliminary search for the target object in a presentation layer to obtain first index information of the target object.

[0039] In the embodiment of the present application, the database-based big data query method is applied to the big data dynamic query system of the medical database. The system includes at least a display layer and an index layer, and the index layer stores a multi-layer index structure of multiple variable data.

[0040] The system improves the storage and query efficiency of clinical research data through special index design, data organization and data structure optimization, meeting the dual requirements of data processing speed and accuracy.

[0041] In this system, the presentation layer is responsible for storing basic project information and form content, making it easy for users to query and browse related data. The presentation layer contains basic information such as project information, visit number, form number, and JSON data body of form content. The innovation lies in using JSON format to store form content, which improves the flexibility and accessibility of data and facilitates fast query and parsing. This method allows fast data reading and front-end display, improving the user interaction experience.

[0042] The index layer is responsible for parsing the variables in the form content and establishing a multi-layer index structure to quickly locate and access variable data. Parse the variables in the form content and establish new indexes based on variable attributes. These index structures have multiple levels, including a first granularity index structure and a second granularity index structure, which are used to quickly locate and access variable data. The system uses index strategies and data structure design to dynamically adjust the index to adapt to data updates and query frequencies to optimize the storage, retrieval and processing efficiency of clinical trial data. Further optionally, in an embodiment of the present application, the first index information is pre-established by the first granularity index structure of the database. The granularity level of the first granularity index structure is at least the form level. That is, the first granularity index structure of the pre-established database, the granularity level is at least the form level, is used to preliminarily search for the target object and obtain the first index information. The second granularity index structure of the database pre-established based on the variable attributes of the variable object in the medical field, the granularity level is at least the variable level, is used to deeply query the target object and obtain the second index information. The innovation lies in that the index layer adopts a multi-layer composite index structure, and improves the storage, retrieval and processing efficiency of clinical trial data through specific index strategies and dynamic adjustment mechanisms. The index structure is dynamically adjusted according to query requirements and data updates to achieve efficient data capture and processing.

[0043] In general, this system improves the efficiency of database systems in processing high-dimensional, dynamically changing scientific research data through innovative index design and data organization, meeting the needs of clinical research for data processing speed and accuracy. Through the collaboration of the presentation layer and the index layer, the system realizes dynamic query and efficient storage of big data in medical databases, providing an innovative solution for scientific research data processing.

[0044] It is further understood that the form level and variable level mentioned in the context refer to different granularity levels used to store and index data in a medical database system. The following is a specific understanding of the form level and variable level. The form level refers to the highest granularity level of data storage and indexing, which usually corresponds to a form (or a table) in the database. In a medical database system, a form can contain a set of associated data fields and information, representing a data set of a certain type or category. The form-level index structure is usually used for preliminary search and location of target objects, and is the basis for data organization and indexing in the system. The first granularity index structure can be established for the entire form to quickly retrieve and locate data.

[0045] The variable level refers to a more granular level of data storage and indexing, which is processed for each specific variable or attribute in the form. In a medical database system, a variable can represent a field or data item in a form, with independent attributes and values. The variable-level index structure is usually used for deep query and positioning of specific attributes or values ​​of the target object, and is the basis for more refined management and processing of data in the system. The second granularity index structure can be established for the attributes of each variable to support precise query and aggregate processing.

[0046] By establishing index structures of different granularities at the form level and variable level, the medical database system can achieve multi-level management and query of data, improving data retrieval efficiency and processing accuracy. The concepts of form level and variable level help the system organize and index data from both the overall and detailed levels to meet different query needs and data processing requirements.

[0047] S102: Perform a deep query on the target object in an index layer according to the first index information to obtain second index information corresponding to the target object.

[0048] In the embodiment of the present application, the second index information is pre-established using a second granularity index structure of a database based on the variable attributes of a variable object in the medical field. The granularity level of the second granularity index structure is at least the variable level. See the above introduction, and no further elaboration will be given here.

[0049] It is understandable that in the medical field, variable objects refer to data items in a form, i.e., variables, which have specific properties. The following is an introduction to variable properties based on variable objects in the medical field.

[0050] A variable object refers to a data field in a form, such as the patient's age, gender, medical history, medication, test results, etc. These variable objects are used to store detailed patient information and clinical data in medical databases, and are an important foundation for clinical research, disease management, and clinical decision support systems.

[0051] Variable attributes refer to additional information of variable objects, such as data type, value domain (value range), unit, description, etc. Variable attributes define additional constraints and descriptions for variables, which helps standardize and normalize data and improve the accuracy and efficiency of data processing. Through variable attributes, specific variable data can be quickly located and filtered to improve query efficiency. Variable attributes help the system to effectively manage and process variable data, such as performing data validation, conversion, and aggregation analysis. In the context of medical data sharing and exchange, variable attributes ensure the consistency and interoperability of data between different systems.

[0052] For example, data types include age (numeric), gender (character), and blood pressure (numeric). The value range of age is usually a positive integer, and the value range of gender is "male" or "female". The unit of blood pressure can be millimeters of mercury (mmHg) or kilopascals (kPa). The description of blood pressure may include the measurement method and measurement time.

[0053] In the medical database system, based on the variable attributes of the variable objects in the medical field, a second-level granularity index structure can be established to achieve deep query and precise positioning of variable data. In this way, the medical information system can efficiently retrieve and process clinical data to support clinical decision-making and scientific research.

[0054] S103, querying the variable data associated with the second index information, and performing aggregation processing on the queried variable data to serve as a target query result fed back to the user.

[0055] In an embodiment of the present application, the queried variable data comes from at least one dynamic form associated with the second index information. In a medical database system, the queried variable data comes from at least one dynamic form associated with the second index information. A dynamic form refers to a form dynamically generated according to specific requirements in a medical database system, which is used to store and display specific variable data. A dynamic form can be dynamically generated according to the user's query requirements and variable attributes, providing flexible data display and query functions.

[0056] The second index information is an index structure established in the index layer for deep query, usually for the attributes of medical variables. The second index information can help the system quickly locate and retrieve data related to the variable attributes, and achieve accurate data access and processing. For example, during the query process, the user initiates a query request for specific variable data, and the system locates the relevant dynamic form based on the first granularity index structure. According to the variable attribute information in the second granularity index structure, the variable data that meets the conditions is further screened and located. The queried variable data is displayed in the relevant dynamic form, and the user can view detailed information and perform further operations.

[0057] In practical applications, the medical database system can generate corresponding dynamic forms according to different research projects to support dynamic query and analysis of clinical data. Doctors can query patients' relevant variable data through dynamic forms to assist clinical decision-making and treatment plan formulation. The dynamic form has a flexible structure, which helps to organize and manage different types of variable data and improve the accessibility and availability of data. By associating with the second index information, the dynamic form realizes the dynamic query and display of variable data in the medical database, allowing users to easily obtain and analyze the required data information, improving the availability and efficiency of the medical information system.

[0058] In the embodiment of the present application, the second granularity index structure is dynamically adjusted based on the query frequency of the variable data in each dynamic form and the data update situation. In the medical database system, the second granularity index structure can be dynamically adjusted according to the query frequency of the variable data in each dynamic form and the data update situation. Exemplarily, the system can determine which variable data is frequently queried and which variable data is rarely queried by monitoring the query frequency of different variable data by the user. Then, according to the analysis results of the query frequency, the system can dynamically adjust the second granularity index structure, and preferentially index those variable data with high query frequency to improve the query efficiency and response speed of these data. Then, the update of the variable data in each dynamic form can be monitored in real time, including the addition, modification and deletion of data. Finally, according to the dynamic changes in the data update situation, the system can update the second granularity index structure in real time to ensure the consistency of the index information with the actual data, and improve the query accuracy and real-time performance of the data. Further optionally, based on the query frequency and data update situation, the system can adopt the following strategy to dynamically adjust the second granularity index structure. For example, for popular data with high query frequency and frequent updates, strengthen the index structure to improve query efficiency. Alternatively, for unpopular data with low query frequency and less updates, appropriately weaken the index structure to save system resources. Alternatively, monitor data updates and update index information in a timely manner to ensure that index data is synchronized with actual data.

[0059] In this way, dynamically adjusting the second granularity index structure can optimize data query performance according to actual usage, improving the overall efficiency of the system and user experience. Through dynamic adjustment, the system can flexibly adjust the index structure according to changes in data access patterns to adapt to different query requirements and data update conditions. By dynamically adjusting the second granularity index structure based on the query frequency and data update conditions of variable data in each dynamic form, the medical database system can more intelligently optimize data indexing and query efficiency, improving system performance and user experience.

[0060] In the big data dynamic query system of the medical database, the specific analysis of steps S101-S103 can produce the following beneficial effects: In step S101, the system can quickly locate the approximate location of the target object by obtaining the first index information through preliminary search, thereby speeding up the subsequent deep query. The preliminary search can help the system filter out a large amount of irrelevant data, reduce unnecessary query operations, reduce system load, and improve the overall performance of the system. In step S102, the system can more accurately locate the target object by obtaining the second index information through deep query, reduce the data scanning range, and improve the accuracy of data retrieval. The second index information is based on the second granularity index structure at the variable level, which can describe the variable attributes more carefully, help improve the query accuracy, and ensure the accuracy of the query results. In step S103, the queried variable data is aggregated and presented to the user as the final query result, which improves the readability and display effect of the data. Through the aggregation processing of the variable data associated with the dynamic form, the system can meet the diverse query needs of users and support complex data analysis and processing operations.

[0061] In general, the beneficial effects of steps S101-S103 are mainly reflected in improving query efficiency, reducing system load, accurately locating target objects, improving query accuracy, efficient data display, and supporting complex query requirements. These effects help optimize the data query and processing capabilities of the medical database system, improve system performance and user experience, and also provide effective support and assistance for the management and analysis of clinical research data.

[0062] As an optional embodiment, before performing a preliminary search on the target object in the presentation layer to obtain the first index information of the target object, as follows Figure 2 As shown, the following steps are also included:

[0063] S201, obtaining variable data to be stored;

[0064] S202, constructing a table layer data structure corresponding to the variable data in a database;

[0065] S203: Based on the form layer data structure, reconstruct the variable layer data structure corresponding to each form to construct multi-layer dynamic index information corresponding to the variable data.

[0066] In the embodiment of the present application, variable data includes: clinical research data input through the visual data model editing interface. Variable data refers to a data unit used to describe or represent a specific attribute or value in a medical database. In this embodiment, variable data includes clinical research data input through the visual data model editing interface. These data can be various medical information collected in clinical trials, such as basic patient information, symptom data, treatment plans, etc. By entering these data in the visual data model editing interface, users can intuitively enter, edit and manage the data.

[0067] In the embodiment of the present application, the visual data model editing interface includes input components dynamically constructed based on clinical research needs. The visual data model editing interface is a tool that provides a visual operation interface to meet the dynamic needs of the clinical research field. The editing interface may contain various dynamically constructed input components to facilitate users to customize the data entry interface according to actual needs. These input components can be text boxes, drop-down boxes, check boxes, date selectors, etc., for users to input various types of medical data information. For example, the visual data model editing interface can be Figure 3a The interface shown. Figure 3a In the interface shown, multiple diseases can be selected as input data. Further optionally, a calendar control can be called in the interface so that the date and time can be selected through the calendar control, such as the examination date, the disease development cycle, etc., and a specific value can also be entered. The calendar control supports setting the format and range of the time to meet different time data recording needs. Further optionally, a table component can also be provided for users to customize multiple columns and rows and enter a series of related data in the table. The table component can support different data type definitions for columns, such as values, text, dates, etc.

[0068] Further optionally, the property settings of the input component can be: variable name-class_cn is the identification name of each component, which is used to display the variable name in the data model. Unique id-uuid is the unique value of each component, which is used to uniquely identify the component in the data model. Type-type is the storage type of the specified data (such as integer, floating point number, string, date, etc.). Logical verification rules-roles are used to set the format verification of the filled-in value, and regular formulas are supported. Logical display rules-role_list are used to set the display logic of the component and define the dependency rules for data entry, such as displaying another component only when a certain option is selected. Formula calculation rules are rules for automatic calculation of data, such as calculating total scores, averages, etc., supporting simple to complex mathematical expressions.

[0069] For example, assume that the data with id 1 is a multiple-choice question, and the content is to choose whether to join the research group. The data with id 2 is a fill-in-the-blank question, and the content is the reason for not joining the group. The data with id 3 is a fill-in-the-blank question, and the content is the group number. The desired effect is that when the data with id1 selects yes, id2 is not displayed, and only id3 is displayed; when the data with id1 selects no, id3 is not displayed, and only id2 is displayed. The logic implemented by the code is controlled by the values ​​of role_list, show, and contain. The value of Show, true or false, is used to control whether the variable is displayed, and the value of contain is used to define the set of other variables involved in the variable. role_list is used to define the corresponding id number to be displayed when the value of the option is a certain value.

[0070] For example, after selecting multiple diseases or multiple attribute data as input data, the processing logic can be: first parse the json data to determine whether the type value is radio or checkbox, and then parse the optionlist as an option for display. Each option object contains name and selected. Name is the option name, and selected is whether it is selected. The system will listen to the click event of the option page. Once it occurs, it will scan each option in the optionlist and change the selected under the selected option to 1. Multiple selection scenarios include patient symptom selection, multiple treatment method selection, etc.

[0071] In an optional example, the processing of disease or attribute data will be used as an example to display and process JSON data containing radio or checkbox options. It will include parsing the JSON data, determining the display mode (single or multiple selection) based on the type value, displaying the option list and listening to the user's click event to update the option status according to the user's selection. The system will first parse the input JSON data. The JSON data structure can be similar to the following example: The type field indicates the type of display, which can be radio (single selection) or checkbox (multiple selection). Optionlist is a list containing option objects. Each option object contains two fields, name and selected. Selected indicates whether the option is selected. Furthermore, based on the value of the type field, the system will determine the display mode. If it is "radio", it will be presented as a single-select list; if it is "checkbox", it will be presented as a multiple-select box list. Then, the system will display specific option information to the user based on the data in optionlist. Next, the system will listen to the user's click event on the option page. Once the user clicks an option, the system will obtain the option information clicked and traverse the entire option list. For single selection, the system sets the selected attribute of the selected option to 1, and sets the selected attributes of other options to 0. For multiple selection, after the user clicks, only the selected attribute of the clicked option is changed. Finally, according to the user's click behavior, the system will traverse the option list and update the selected attribute of the corresponding option. If the option clicked by the user is matched, the selected of the option is changed to 1 (selected state), and the other options are updated to unselected state. In this way, the user can see the list of options on the interactive interface. After making a selection, the system will dynamically change the selected state and update the display state in time to provide real-time feedback to the user. Through the above steps, the system can parse and display the option information in the JSON data, and dynamically update the option state according to the user's click event, providing users with a convenient selection experience. This design can be applied to processing JSON data containing radio or checkbox options to achieve effective management of display and interaction.

[0072] In addition, in another embodiment, the implementation logic of the input component can also determine whether the type attribute of the json file is text. If so, the result under the json data is bidirectionally bound to the value of the front-end input component, and the value in the input is assigned to the result in the json by monitoring the input box change event of the front-end input component. The role attribute value in the json file is extracted and input into the regular matching method as a regular matching method parameter, and the result value is checked for regular rules.

[0073] Further optionally, formula calculations between multiple variables can be implemented by definition. For example, if you want to divide the value of the variable with id=2 by the square of 0.01 of the value of the variable with id=1, you need to configure the role to "&{2}& / (&{1}&*0.01*&{1}&*0.01)".

[0074] Through steps S201-S203, the variable data entered by the user through the visual data model editing interface can be processed and converted to construct the corresponding form layer data structure and variable layer data structure. These data structures will serve as the basis for indexing and will be used for subsequent data query and retrieval operations, thereby improving the data management and query efficiency of the system. By constructing dynamic index information, the system can more flexibly adapt to the changing needs of clinical research data, while also providing users with a better and more convenient data input and query experience.

[0075] The system interface involved in this application also provides some optional functions. For example, data validation, that is, implementing advanced data validation, including but not limited to data type checking, format matching, logical consistency verification, etc. For example, version control, that is, each modification and update of the data model will be saved as a different version, allowing users to track historical changes and restore old versions.

[0076] For example, in the process of medical data entry, to improve the quality and accuracy of data, built-in logical validation rules are usually used to verify the legitimacy of data. This validation rule is designed to check simple rules for single variables (such as body temperature and height), as well as linkage compound conditions between multiple variables, especially when cross-data point association judgment is required. Rule validation for linkage compound conditions of multiple variables is a validation rule designed for complex relationships between multiple data points. These rules usually involve association judgments between multiple data fields, such as the combination of logical conditions, comparison of numerical values, determination of time ranges, etc. Each association condition can be defined as an independent rule. A clear logical relationship needs to be defined for each compound condition, such as using logical operators (AND, OR) to connect different conditions. For example: Gender is male AND weight is greater than 100 kg AND blood sugar is normal within one week after surgery. The system will apply these compound rules in real time during the data entry or update stage and verify the data. The system will accurately identify and mark the data that does not meet the conditions according to the rule definition. When the data does not meet the compound condition rules, the system will provide detailed feedback information, indicating the specific conditions violated by the user, and may provide solutions or suggestions. These rule checks can be implemented by building complex judgment logic in the system's back-end business logic and combining database triggers, transaction processing and other technologies to ensure data consistency and correctness.

[0077] In an example scenario, suppose a judgment is made based on the disease name and physical examination data. If the disease name is "hypertension", then the blood pressure value in the physical examination data is judged to be abnormal. The system makes a linkage judgment based on the disease name filled in by the patient and the blood pressure value in the physical examination data to ensure the rationality of the data.

[0078] In another example scenario, assume that the target object's attributes include: male gender, weight greater than 100 kg, and it is necessary to determine whether the patient's blood sugar is normal within one week after surgery. Then, the patient's gender is male AND weight greater than 100 kg, and blood sugar is normal within one week after surgery. Based on the combination of these conditions, the system determines whether the patient meets the inclusion criteria and checks whether the insulin injection amount is normal.

[0079] Through the built-in multi-variable linkage compound conditional rule verification, the system can conduct more in-depth logical judgment and verification of medical data, ensuring the quality and accuracy of data entry while meeting the requirements of medical logic. The application of this complex rule verification provides more powerful logical support for medical data management, helping medical staff to better manage and use medical data.

[0080] Further optionally, in S202, the variable data is parsed and classified by form, and the variable data is stored in a corresponding database form; the form identifier of the variable data is marked as the first index information. In S203, the variable data in each database form is parsed and classified, and the variable identifier corresponding to the variable data after secondary classification is marked as the second index information.

[0081] Specifically, parse the json data, the data structure with the type value of public_bigtablelist is judged to be table type data; the content of the model is used as a template to add row data. The value of the tablelist is the data row data, which is used to store the added row data to form the table data. Whenever the user adds new data, the object data of the model will be added to the array of the tablelist. During the row data parsing process, under each row data, there is a property configlist that stores variables. Configlist is an array used to store column data under the row data. For blood routine tests, it is divided into several columns, inspection items, inspection values ​​and reference ranges. In the table data, the data validation logic for each row can be that since the columns under the same row data are divided into inspection items, inspection values ​​and reference ranges, the code can monitor the inspection value data, read the reference range and convert it into a regular expression to verify the inspection value data, thereby determining whether the inspection value is qualified.

[0082] For example, suppose there is a medical database system where users can enter basic patient information through a visual data model editing interface and associate clinical research data such as disease data and treatment plans. The system needs to build a multi-layer index structure based on this data to facilitate advanced search queries and logical judgments.

[0083] In S201, the user enters a patient's information, such as name, age, gender, symptoms, treatment plan, etc., through the visual data model editing interface. These data will be acquired by the system and used as variable data to be stored. In S202, the system parses and classifies the form according to the data entered by the user, and assigns different types of data to corresponding database forms. For example, basic patient information may be stored in one form, and symptom data and treatment plans may be stored in another form. While storing these data, the system will mark the form identifier of each variable data as the first index information. In this way, the system can quickly locate the corresponding data form according to the form identifier and improve data query efficiency.

[0084] In S203, the system performs variable analysis and classification for the variable data in each database form, and performs secondary classification on each variable. For example, for disease data, it can be further classified into symptoms, diagnosis results, examination data, etc. After analysis and classification, each variable data will be marked with a corresponding variable identifier as the second index information. These variable identifiers are used to construct multi-layer dynamic index information to support the system's advanced search query and logical judgment at the variable level.

[0085] Through the implementation process of the above examples, the system can classify and store the clinical research data entered by users according to forms and variables, and build a multi-layer index structure to achieve refined management and efficient query of data. This specific data storage and index design can improve the query efficiency and data retrieval accuracy of the system.

[0086] Specifically, in this specific system implementation, the construction of the first-layer data structure and the second-layer data structure, as well as the data processing process when the save button is triggered on the front-end page, are involved. The first-layer data structure is stored in the MySQL database and is used to store and echo data. It is usually used for the display and logical interaction of forms in the system. The data structure of this layer contains the configuration information of the form, the storage location of the variable data, etc. When the user performs a form operation on the front-end page, the system will read the corresponding information from the database and display it according to the configured form structure, and the user can perform operations such as data entry and modification. When the user triggers the save button on the front-end page, the system will perform the save operation. At this time, the system will scan and parse the data in Table_config, which is used to store the form configuration information, including the form fields, layout, validation rules, etc. The system will parse the variables in the configlist in the configuration information, recombining them into data in units of variables, and store them one by one in the second-layer data structure. This process will classify and store the variable data in different forms to form a data structure in units of variables. The second-layer data structure is used for the system's advanced search queries and logical judgments at the variable level. This layer of data structure has more fine-grained data classification and indexing, which facilitates the system to perform variable-level query operations. When the data in the first layer of data structure is parsed and recombined and stored in the second layer of data structure, the system can perform more accurate searches and logical judgments based on this data to support the system's advanced functional requirements.

[0087] This system design enables efficient data management and query, while supporting system-level search queries and logical judgments at the variable level. This process design of data storage, parsing, and reorganization helps improve system performance and user experience, and meets the complex needs of data management and query in the medical field.

[0088] It is worth noting that S201 corresponds to the process in which the system reads the corresponding information from the database and displays it when the user performs form operations on the front-end page in the system implementation, and the user enters and modifies the data. This step ensures that the system can accurately obtain the variable data entered by the user. S202 corresponds to the first-layer data structure part in the system implementation. When the user triggers the save button on the front-end page, the system scans and parses the data in Table_config, and stores the variable data in the corresponding database form according to the information configured in the form, and marks the form identifier for display and logical interaction. This step ensures that the variable data is correctly parsed, classified and stored in the database. S203 corresponds to the second-layer data structure part in the system implementation. After parsing and recombining the data of the first layer, the system stores the data in the second-layer data structure to support advanced search queries and logical judgments at the variable level. In this process, the system will mark the variable identifier for each variable data for more sophisticated data management and query. This step ensures that the system can perform more accurate searches and judgments based on the reorganized variable data.

[0089] Through these steps, the system can effectively manage and organize data, making it more efficient and accurate in storage and query. Each step has its specific purpose and operation in the implementation process, ensuring that the system can quickly respond to user requests at different levels while meeting complex business needs. The purpose of this design is to provide a stable and high-performance data management and interaction experience.

[0090] Further optionally, a one-to-one association relationship or a one-to-many association relationship is established between the second index information and the variable data in the database form. Wherein, the association relationship is set based on case attributes and / or clinical research attributes. In system design, a one-to-one or one-to-many association relationship is established between the second index information and the variable data in the database form in order to better manage and query data, and is set according to different case attributes and clinical research attributes.

[0091] In a one-to-one relationship, each second index information establishes a direct correspondence with the variable data in the database form. This relationship can be used for variable data that needs to be managed independently, such as some basic case attributes, such as the patient's name, age, etc. By establishing a one-to-one association, the system can quickly and accurately locate and access specific variable data, while also providing a concise and clear way of data management. For example, you can directly find the detailed information of a specific patient by case number or patient name.

[0092] In a one-to-many relationship, a second index information can be associated with multiple variable data in a database form. This relationship is suitable for data with complex structures or multiple attributes, such as clinical research attributes. By establishing a one-to-many association relationship, the system can organize and manage variable data according to different attribute dimensions, making it more flexible and extensible. For example, multiple index information can be established for different clinical data of the same case to achieve multi-angle data analysis and query.

[0093] The associations set based on case attributes and clinical research attributes can better meet different data management needs. For example, variable data can be classified according to case attributes such as case type, date of visit, etc., and more detailed index information can be established based on clinical research attributes such as symptoms and test results. The associations set based on attributes enable the system to manage and organize data according to specific business needs, improving data readability and query performance. At the same time, this design can also support the expansion and customization needs of the system, making the system more flexible and intelligent.

[0094] By establishing a one-to-one or one-to-many relationship and combining it with case attributes and clinical research attributes, the system can better manage and query variable data in the database, improve the data utilization value and analysis efficiency. This relationship design can make the system more efficient and flexible in complex data management and query scenarios.

[0095] Further optionally, it is assumed that each operation data of each variable data is associated with the corresponding first index information and / or second index information. Based on the above assumption, the variable data in each database form is analyzed and classified, and the variable identifier corresponding to the variable data after the secondary classification is marked as the second index information. It is also possible to query the unique identifier change logic chain of the corresponding variable data based on the first index information and / or the second index information; based on the unique identifier change logic chain, retrieve any historical version of the corresponding variable data.

[0096] Based on the above assumption, that is, each operation data of each variable data is associated with the corresponding first index information and / or second index information, the system can be further designed to support querying the unique identification change logic chain of the variable data and retrieving historical versions. In the system, each operation of the variable data will be recorded and associated with the first index information and / or the second index information. When it is necessary to query the unique identification change logic chain of a certain variable data, the system can locate the target variable data from the relevant index information according to the query conditions. By tracking the operation records of the variable data, the system can build a data change logic chain, including the order and timestamp information of operations such as creation, modification, and deletion of the variable data. The continuity and sequential relationship of these operation data will constitute a logical chain of unique identification changes.

[0097] In addition, based on the unique identification change logic chain, the system can realize the function of retrieving any historical version of the target variable data. By querying the historical operation records in the identification change logic chain, the system can find the version information of the target variable data at different time points. When the user needs to view or restore the historical version of the variable data, the system can retrieve the corresponding data version according to the request, thereby realizing data backtracking and historical version viewing.

[0098] While recording each variable data operation, ensure that the operation data is associated with the first index information and / or the second index information. In this way, the system can determine the specific variable data based on the index information at different levels to ensure the accuracy and consistency of the unique identifier. Maintaining the association between the operation data and the index information is a key step to ensure that the system can accurately track the data operation history, implement version management and data traceability.

[0099] Through the above design, the system can effectively support querying the unique identifier change logic chain based on the first index information and / or the second index information, and then retrieve the historical version of the variable data. This functional design can provide users with more comprehensive data management and query functions, while ensuring the integrity and traceability of the data, and providing more control and flexibility for the system's data operations.

[0100] As an optional embodiment, it is assumed that the database is set with multiple levels; the multiple levels at least include: project level, visit level, form level. Exemplarily, Figure 3b The relationship between multiple levels in the database shown. Among them, the project level is the highest level data organization unit, which is used to divide different research projects or business projects. Each project level can include multiple visits and forms, representing a complete research or business project. The visit level organizes data within the project, representing data collection at different time points or event points. Each visit level contains multiple forms for collecting data at specific time points. The form level is the most fine-grained data collection unit, which contains specific data fields and information. Each form level corresponds to specific data collection requirements. Based on this, the corresponding visit level database table and form level database table can also be set according to the project requirements of different project levels; the corresponding visit level database table and form level database table are bound to the input components of different project levels to collect clinical research data in different projects.

[0101] Specifically, according to different project requirements, corresponding visit-level database tables and form-level database tables are set for each project level. In this way, the data storage structure can be designed according to the characteristics of the project and the research requirements to ensure the consistency and accuracy of the data. The visit-level database table and the form-level database table are bound to the input components of different project levels. This means that during the data collection process, the system will automatically load the corresponding database table structure according to the project requirements in order to collect clinical research data in different projects. By binding the input components and database tables, the accuracy and completeness of data collection are ensured. The data of each project is stored in the database with different hierarchical structures, making data management clearer and more orderly, and facilitating subsequent data query and analysis operations. Through the design of the above method, the system can set different levels of database table structures according to project requirements to achieve multi-level data management and query. At the same time, corresponding visit-level and form-level database tables are set for each project, and they are bound to the input components to effectively support data collection and management. This method helps to improve the organization and operability of data and meet the collection and management needs of clinical research data for different projects.

[0102] As an optional embodiment, assuming that the case creation mode adopts the distribution mode. In the above steps, based on the form layer data structure, the variable layer data structure corresponding to each form is rebuilt to construct the multi-layer dynamic index information corresponding to the variable data, which can also be implemented as follows: create a case index identifier corresponding to each case; bind each variable data in each form to the corresponding case index identifier, so that each variable data performs an independent query based on the case index identifier.

[0103] In the case creation mode using the distribution mode, the variable layer data structure corresponding to each form is rebuilt, and multi-layer dynamic index information is created, which can better organize and manage case data and provide flexible query and analysis functions. For each form, the corresponding variable layer data structure is redesigned to organize and store the variable data in the form according to the logical relationship. This can include classifying and combining different types of variable data, setting the relationship between variables, etc. By reconstructing the data structure, the data characteristics of each form can be better grasped, providing more flexibility and customizability for subsequent data operations and queries. Furthermore, a unique case index identifier is created for each case as an identifier of the case data. This case index identifier can be used to associate and identify all data associated with the case, including variable data in each form. By introducing the case index identifier, the association between multiple forms can be achieved, and it is also convenient to query and analyze the overall case data. Then, each variable data in each form is bound according to the corresponding case index identifier. This means that each variable data can be uniquely identified by the case index identifier to achieve the association and integration between data. By binding the variable data with the case index identifier, case-based data query and analysis operations can be achieved. Users can retrieve and display all case-related data information through case index identifiers.

[0104] By implementing the above steps, comprehensive management and query functions of case data can be realized. Rebuilding the variable layer data structure, creating case index identifiers, and binding variable data to case index identifiers give the system stronger data analysis and operation capabilities. At the same time, this case data management model also helps to improve the consistency and integrity of data and provide better support for clinical research work.

[0105] Furthermore, in the above steps, each variable data in each form is respectively bound to the corresponding case index identifier, so that each variable data can be independently queried based on the case index identifier, and then the query condition corresponding to the case index identifier can be set; wherein the query condition corresponding to the case index identifier at least includes: the second index information is a set threshold, and / or, a data search condition that satisfies the custom setting. Furthermore, in the above steps, querying the variable data associated with the second index information can be implemented as: obtaining a collection of case index identifiers that meet the query conditions, and obtaining all corresponding variable data through the collection query.

[0106] For example, with case data volumes of more than 10,000, custom queries can be implemented through data template prefabrication and distribution mechanisms, combined with a two-layer data structure storage mechanism, which can greatly improve query efficiency. Assume that there is a table in the system that stores all case data, which contains detailed information for each case and the corresponding case index identifier and other related data. Based on the prefabrication of the data template, the user can select any specific variable as the query condition, map the query condition to the uuid of the variable (i.e., the first index information and / or second index information introduced above), and set a threshold (e.g., greater than or less than a certain value) and a custom data search condition (e.g., specific symptoms, diseases, etc.). Based on the query conditions set by the user, the system queries the corresponding uuid through the second-layer data structure, and filters out a set of case index identifiers that meet the conditions based on the set thresholds and custom search conditions. Further, based on the collection of case index identifiers, the system can query all variable data corresponding to these cases, thereby obtaining a set of all variable data that meets the query conditions.

[0107] For example, suppose the user wants to query all cases with a specific symptom when the value of a specific variable is greater than 100. The user first selects the specific variable as the query condition in the system, maps the query condition to the uuid of the variable and sets the threshold to be greater than 100, and sets a custom symptom search condition. The system queries the collection of index identifiers of cases that meet the conditions through the second-layer data structure, and then retrieves all variable data of the corresponding cases through these index identifiers, and finally obtains the data set of all cases that meet the conditions.

[0108] Through the above steps, the system can quickly and effectively filter out data sets that meet the requirements from massive case data based on the query conditions set by the user, realize efficient data query and analysis functions, and provide convenient and effective data management solutions for clinical medical research.

[0109] As an optional embodiment, it is assumed that the visual data model editing interface includes an input component dynamically constructed based on clinical research needs. Based on this, S201, obtaining the variable data to be stored, includes: accepting the variable data to be stored through the input component; using a pre-set regular expression and / or a conditional statement corresponding to the clinical research pathology to perform a preliminary verification on the variable data to be stored; sending the preliminary verification result and the variable data to be stored to the backend database, and performing a secondary verification on the variable data to be stored in the database to ensure data accuracy and completeness.

[0110] In the case where the visual data model editing interface contains input components dynamically constructed based on clinical research needs, the user inputs the variable data to be stored through the input components in the data model editing interface. These input components can be dynamically constructed according to clinical research needs to adapt to different data input scenarios and needs. After obtaining the variable data to be stored, the system first uses the pre-set regular expressions and / or conditional statements corresponding to clinical research pathology to perform preliminary verification on these data. This step can be used to check the format, legality and integrity of the input data to ensure data quality. After preliminary verification, the system sends the results of the preliminary verification and the variable data to be stored to the back-end database. In the database, the variable data to be stored is secondary verified to further confirm the accuracy and completeness of the data. The system will perform validity checks on the data according to the pre-set data verification rules to ensure that the data meets the specified standards. The variable data to be stored that has been preliminarily verified and secondary verified will be stored in the database after passing all verification steps. This ensures the quality and accuracy of the data and prevents data that does not meet the specifications from being written into the database.

[0111] For example, suppose a user enters a variable data in the visual data model editing interface. The system first receives the data through the dynamically constructed input component, and then applies the pre-set regular expressions and conditional statements to perform preliminary verification on the input data to ensure the data format and legality. Once the preliminary verification is passed, the system sends the preliminary verification results and data to the back-end database for secondary verification in the database. The database performs secondary confirmation on the data according to the set rules. If the data meets the specifications, it is finally stored in the database, otherwise an error message is returned. This process ensures that the data is strictly verified and checked before storage, ensuring the accuracy and integrity of the data.

[0112] Through the above process, the system can ensure that the variable data entered by the user meets the specifications after preliminary and secondary verification, thereby effectively managing and maintaining data quality and meeting the requirements for data accuracy and completeness in the field of clinical research.

[0113] In the embodiment of the present application, further optionally, the conditional statement corresponding to the clinical research pathology at least includes: a multi-variable linkage compound conditional statement. The linkage compound conditional statement is used to verify the association relationship between multiple variable data.

[0114] This linkage compound conditional statement can help the system verify the logical relationship between multiple variable data to ensure data consistency and integrity. The linkage compound conditional statement is a complex verification condition composed of the logical relationship between multiple variables. These conditions can involve the value ranges, relationships, constraints, etc. of multiple variables, in this way to verify whether the association relationship between these variables meets specific rules. For example, you can set the following compound conditions: when the value of variable A is greater than 100, the value of variable B must be within a specific range; or when the value of variable C is a certain value, the value of variable D must conform to a certain pattern.

[0115] When the user enters multiple variable data, the system will verify these variable data according to the pre-set linkage compound conditional statements. The system will check whether the logical relationship between each variable meets the set conditions one by one to ensure the legitimacy and consistency of the data. If there is a violation of the set conditions between the input multiple variable data, the system will promptly prompt the user with an error message and require the user to re-enter the data that meets the regulations to ensure the correctness of the data.

[0116] By verifying the relationship between multiple variables through linked compound conditional statements, the system can effectively ensure the consistency and integrity of the data. This mechanism can effectively prevent users from inputting incorrect information or omissions, and improve the quality and reliability of the data. At the same time, data verification based on linked compound conditional statements can help users better understand the logical relationship between data, so that they can be more cautious and accurate when entering data.

[0117] By introducing linked compound conditional statements to verify the correlation between multiple variables, the system can provide more comprehensive and flexible data verification functions, ensure the correctness and consistency of input data, and provide stronger support for clinical research.

[0118] Further optionally, based on the linkage compound conditional statement, other fields with associated relationships can also be dynamically verified and updated when a certain field is updated. Here, other fields can be fields in the same form or fields in other forms. Other fields can be calculated based on the updated fields, or they can have a logical association with the updated fields, such as belonging to the same user, the same user group, the same scientific research project, etc. In this way, the efficiency of data updating can be improved, making the data source more flexible and rich.

[0119] As an optional embodiment, the second granularity index structure can also be dynamically adjusted based on the query frequency of variable data in each dynamic form and the data update status; the optimized and adjusted second granularity index structure is recursively traversed, and data mapping processing is performed; wherein the recursive traversal processing is based on the nested structure planning of the second granularity index structure.

[0120] In this optional embodiment, the second granularity index structure can be dynamically adjusted based on the query frequency and data update of the variable data in each dynamic form to optimize system performance and query efficiency. The specific steps are as follows: the system monitors the query frequency and data update of different variable data in each dynamic form, and dynamically adjusts the second granularity index structure based on this information. Variables with high frequency queries may be designed as more accessible index structures to improve query speed, while data that is updated frequently may need to be stored in a more efficient data structure. The second granularity index structure after dynamic adjustment will be optimized by the system to ensure the efficiency and accuracy of data access. This may involve operations such as reordering indexes and optimizing query algorithms to ensure that the performance of the system is optimized during query and update. The optimized and adjusted second granularity index structure is recursively traversed to process data mapping operations. The system will traverse the index layer by layer according to the nested planning of the index structure, perform data mapping processing operations, and ensure that the required data can be retrieved quickly and accurately. The recursive traversal process is based on the nested structure planning of the second granularity index structure. The system will traverse the index structure layer by layer according to the set plan and perform data mapping processing. This approach ensures that the system can effectively handle complex index structures, quickly locate required data and perform related operations.

[0121] For example, suppose you need to design a recursive algorithm to traverse and process the above nested JSON data structure. The main goals include: go deep into each level and process all nested objects; perform specific processing on each configuration object, such as data validation or field update; and ensure that the termination condition of the recursion is that the current object no longer contains the configlist field. Then, define a function processConfigList, which accepts a JSON object as a parameter and checks whether the configlist field exists. If it exists, each nested object is processed recursively. During the recursive traversal, for each JSON object, we call another function processCurrentObject for processing. This function can contain specific business logic, such as checking whether a certain field exists and converting the format. If the configuration object contains a specific field (such as field), its value can be transformed or verified. For example, convert all field values ​​to uppercase. Other operations can be performed according to business needs, such as logging certain information or collecting statistical data. The termination condition of the recursive traversal is that the current JSON object no longer contains the configlist field. This condition is checked before each recursive call, and if it does not exist, the recursive process ends. For possible performance issues, especially when processing very deeply nested JSON data, optimization strategies may be required. In an environment that supports tail recursion optimization, recursion is refactored to conform to the tail recursion mode. Recursion is converted to an explicit stack-based iteration method to avoid overflow of the system call stack. For very large data structures, a custom stack is used to manage the recursive process to avoid relying on the system call stack. With the above method, complex and nested JSON data structures can be systematically processed to ensure that each internal object is properly processed. This recursive traversal mechanism not only maintains the clarity and readability of the code, but also can flexibly adapt to JSON data of various nesting depths and structural complexities.

[0122] Through the above steps, the system can dynamically adjust the index structure according to the frequency of data queries and updates. Through the optimized index structure and recursive traversal processing, efficient management and processing of data storage and retrieval are achieved. This dynamic adjustment and optimization can effectively improve the system performance and user experience, making data operations more efficient and convenient.

[0123] Further optionally, in the above steps, recursively traversing the optimized and adjusted second granularity index structure and performing data mapping processing include:

[0124] Through the difference comparison model, the key change points of the second granularity index structure before and after the optimization adjustment are identified; based on the key change point identification results, the second granularity index structure before and after the optimization adjustment is recursively traversed according to the nested structure to obtain the local structural information in the second granularity index structure before and after the optimization adjustment; based on the local structural information, data migration is performed to map the variable data from the second granularity index structure before the optimization adjustment to the second granularity index structure after the optimization adjustment.

[0125] In a further embodiment, first, the second granularity index structure before and after the optimization adjustment is compared using a difference comparison model to identify key change points. These change points may include newly added indexes, deleted indexes, changes in index fields, etc. This can help the system understand the differences between index structures and prepare for subsequent data migration. Furthermore, based on the identification results of key change points, the system recursively traverses the second granularity index structure before and after the optimization adjustment according to the nested structure to obtain local structural information in the index structure before and after the optimization adjustment. This includes dependencies between indexes, mapping relationships between index fields, etc. In this way, the system can establish a mapping relationship between index structures. After obtaining the local structural information, the system performs a data migration operation based on this information to map variable data from the second granularity index structure before the optimization adjustment to the second granularity index structure after the optimization adjustment. This includes extracting data from the old index structure and reorganizing and storing data according to the new index structure. Through the operation of the above steps, the system can effectively respond to the optimization adjustment of the index structure and ensure the accuracy and integrity of the data after the index structure is updated. The difference comparison model helps the system quickly identify key change points, while recursive traversal based on nested structures helps obtain local structural information and provide necessary support for data migration. The data migration operation ensures that data can be smoothly migrated from the old index structure to the new index structure, ensuring the stability and accuracy of the system.

[0126] This method can help the system ensure data continuity and consistency when optimizing and adjusting the index structure, thereby improving system performance and user experience.

[0127] In practical applications, the difference comparison model can be constructed based on a variety of algorithms. The specific algorithm selection depends on the application scenario and requirements. The following are some algorithms used to build a difference comparison model: The hash algorithm can map data into a hash value of fixed length, and judge the change of data by comparing the difference between two hash values. Common hash algorithms include MD5, SHA-1, SHA-256, etc. When the hash values ​​of two data blocks are different, it can be considered that there is a difference between them. Edit distance algorithms (such as Levenshtein distance and Hamming distance) can measure the similarity between two strings to judge the difference between them. This algorithm is usually used to compare the differences between text data. Machine learning algorithms (such as decision trees, random forests, neural networks, etc.) can be trained to obtain models for predicting the differences between data. This method is suitable for complex data comparison scenarios. Version control algorithms (such as the Diff algorithm in Git) can be used to compare the differences between different versions of code. This algorithm is usually used in source code management and version control systems. According to the specific application scenario and requirements, you can choose a suitable algorithm or combine multiple algorithms to build a difference comparison model for effective identification and comparison of differences between data.

[0128] As an optional embodiment, in the above steps, dynamically adjusting the second granularity index structure based on the query frequency and data update status of the variable data in each dynamic form includes:

[0129] The multi-branch decision tree in the index decision model is used to make preliminary decisions on the query frequency and data update status of the variable data in each dynamic form, so as to obtain the initial adjustment information of the variable data in each dynamic form;

[0130] By indexing the state change identification layer in the decision model, the database state change trend in the initial adjustment information of the variable data in each dynamic form is identified;

[0131] Through the pattern change recognition layer in the index decision model, the query pattern change trend of each variable data under each dynamic form is analyzed;

[0132] Through the dynamic adjustment layer in the index decision model, the database status change trend and the query mode change trend are integrated to adjust the data structure of each variable data under each dynamic form to obtain the optimized and adjusted second granularity index structure.

[0133] In this optional embodiment, for the query frequency and data update of the variable data in each dynamic form, first, the multi-branch decision tree in the index decision model is used to make a preliminary decision on the query frequency and data update of the variable data in each dynamic form to obtain the initial adjustment information of the variable data in each dynamic form. The multi-branch decision tree can provide a basis for subsequent decisions based on the different branches of the query frequency and update situation. Next, the state change recognition layer in the index decision model is used to identify the database state change trend in the initial adjustment information of the variable data in each dynamic form. Further optionally, the index decision model can be trained using reinforcement learning technology, especially the state change recognition layer in the index decision model. This step can help the system understand the trend of data changes in the database and provide guidance for subsequent index adjustments. Next, the query mode change trend of each variable data under each dynamic form is analyzed through the mode change recognition layer in the index decision model. Cluster learning can classify the variable data and identify the change trend of different query modes, which helps to better adjust for different types of data. Finally, the dynamic adjustment layer in the index decision model is used to integrate the database state change trend and the query mode change trend, and adjust the data structure of each variable data under each dynamic form to obtain the optimized and adjusted second granularity index structure. In this step, the system will adjust the index structure based on the comprehensive data analysis results to maximize query efficiency and system performance.

[0134] Through the above steps, the system can use different layers of the index decision model to comprehensively analyze the query frequency and update status of variable data in each dynamic form, thereby dynamically adjusting the second granularity index structure. This method combines reinforcement learning, clustering learning and dynamic adjustment techniques to more accurately optimize the index structure, improve the query efficiency and performance of the system, and provide users with a better experience.

[0135] Exemplarily, the construction of the index decision model can combine the ideas of decision tree learning, reinforcement learning, clustering algorithm and neural network to dynamically adjust the index structure and optimize database performance. Using the decision tree learning algorithm, a decision tree model is constructed based on input features (such as query type and frequency) to predict the optimal index structure. At this stage, the decision tree can provide a preliminary index decision path. The reinforcement learning algorithm is introduced to define the state (index structure and query mode of the current database), action (add or delete index) and reward (increase or decrease of query efficiency), so that the system can perform automated learning to adjust the index under different database states to achieve dynamic optimization. This stage helps the system adjust the index structure according to the real-time state to maximize the query efficiency. Use clustering algorithms to identify similar groups in query patterns and help optimize indexes to process these groups more efficiently. Based on the analysis results of the clustering algorithm, the system can adjust the index to better adapt to different query patterns. The neural network model is introduced to predict the query result pattern and further optimize the index. By inputting query features, the neural network model can predict which type of index can provide the fastest access speed, thereby guiding the adjustment of the index structure. In this way, the above decision tree learning, reinforcement learning, clustering algorithm and neural network are integrated into a multi-layer model. The system first uses decision trees to make preliminary indexing decisions, then fine-tunes the index structure through reinforcement learning to adapt to real-time changes, clustering algorithms to analyze query patterns, and neural networks to predict future trends and adjust indexes. This multi-model fusion strategy can maximize the efficiency and accuracy of index adjustment.

[0136] By integrating the above steps, the constructed index decision model can realize the adaptive optimization management of database indexes, thereby improving database query speed and system performance. The system will be able to intelligently adjust the index structure according to different query modes and real-time database status, providing users with a more efficient and stable database query experience.

[0137] In order to execute the database-based big data query method corresponding to the above method embodiment, to achieve the corresponding functions and technical effects. Figure 4 , Figure 4 The structural block diagram of a big data query device 400 based on a database provided in an embodiment of the present application is shown. For the sake of convenience, only the parts related to the present embodiment are shown. The big data query device 400 based on a database provided in an embodiment of the present application is applied to a big data dynamic query system of a medical database. The system includes at least a display layer and an index layer. The index layer stores a multi-layer index structure of multiple variable data. The provided big data query device 400 based on a database includes:

[0138] The search module 401 is used to perform a preliminary search on the target object in the presentation layer in response to a user's data query request for the target object, so as to obtain first index information of the target object; the first index information is pre-established by a first granularity index structure of the database; the granularity level of the first granularity index structure is at least a form level;

[0139] A query module 402 is used to perform a deep query on the target object in the index layer according to the first index information to obtain second index information corresponding to the target object; the second index information is pre-established based on the variable attributes of the variable object in the medical field using a second granularity index structure of the database; the granularity level of the second granularity index structure is at least the variable level;

[0140] Feedback module 403 is used to query the variable data associated with the second index information, and aggregate the queried variable data to serve as a target query result fed back to the user; wherein the queried variable data comes from at least one dynamic form associated with the second index information; the second granularity index structure is dynamically adjusted based on the query frequency and data update status of the variable data in each dynamic form.

[0141] As an optional embodiment, it also includes a storage module, which is used to: obtain the variable data to be stored before the search module 401 performs a preliminary search for the target object in the display layer to obtain the first index information of the target object; the variable data includes: clinical research data input through the visual data model editing interface; the visual data model editing interface contains input components dynamically constructed based on clinical research needs; in the database, construct a form layer data structure corresponding to the variable data; based on the form layer data structure, reconstruct the variable layer data structure corresponding to each form to construct multi-layer dynamic index information corresponding to the variable data.

[0142] As an optional embodiment, the storage module constructs a form layer data structure corresponding to the variable data in the database, which is specifically used to: perform form parsing and classification on the variable data, and store the variable data in the corresponding database form; mark the form identifier of the variable data as the first index information; based on the form layer data structure, rebuild the variable layer data structure corresponding to each form to construct multi-layer dynamic index information corresponding to the variable data, including: perform variable parsing and classification on the variable data in each database form, and mark the variable identifier corresponding to the variable data after the secondary classification as the second index information.

[0143] As an optional embodiment, a one-to-one association relationship or a one-to-many association relationship is established between the second index information and the variable data in the database form; wherein the association relationship is set based on case attributes and / or clinical research attributes.

[0144] As an optional embodiment, each operation data of each variable data is associated with the corresponding first index information and / or second index information; the storage module performs variable analysis and classification on the variable data in each database form, and marks the variable identifier corresponding to the variable data after the secondary classification as the second index information, and is also specifically used for: based on the first index information and / or the second index information, querying the unique identifier change logic chain of the corresponding variable data; based on the unique identifier change logic chain, retrieving any historical version of the corresponding variable data.

[0145] As an optional embodiment, the database is set up with multiple levels; the multiple levels include at least: project level, visit level, form level; the storage module is also used to: set up corresponding visit level database tables and form level database tables according to project requirements of different project levels; bind the corresponding visit level database tables and form level database tables to input components of different project levels to collect clinical research data in different projects.

[0146] As an optional embodiment, the case creation mode adopts a distribution mode; the storage module reconstructs the variable layer data structure corresponding to each form based on the form layer data structure to construct multi-layer dynamic index information corresponding to the variable data, and is also used to: create a case index identifier corresponding to each case; bind each variable data in each form to the corresponding case index identifier, so that each variable data can perform an independent query based on the case index identifier.

[0147] As an optional embodiment, the storage module binds each variable data in each form to the corresponding case index identifier, so that after each variable data performs an independent query based on the case index identifier, it is also used to: set the query conditions corresponding to the case index identifier; wherein the query conditions corresponding to the case index identifier include at least: the second index information is a set threshold, and / or, data search conditions that meet custom settings; the storage module queries the variable data associated with the second index information, specifically for: obtaining a collection of case index identifiers that meet the query conditions, and obtaining all corresponding variable data through the collection query.

[0148] As an optional embodiment, the visual data model editing interface includes an input component dynamically constructed based on clinical research needs; the storage module obtains the variable data to be stored, and is specifically used to: accept the variable data to be stored through the input component; use pre-set regular expressions and / or conditional statements corresponding to clinical research pathology to perform preliminary verification of the variable data to be stored; send the preliminary verification results and the variable data to be stored to the back-end database, and perform secondary verification on the variable data to be stored in the database to ensure data accuracy and completeness.

[0149] As an optional embodiment, the conditional statement corresponding to the clinical research pathology at least includes: a multi-variable linkage compound conditional statement; the linkage compound conditional statement is used to verify the correlation relationship between multiple variable data.

[0150] As an optional embodiment, the storage module is also used to: dynamically adjust the second granularity index structure based on the query frequency of variable data in each dynamic form and the data update status; recursively traverse the optimized and adjusted second granularity index structure, and perform data mapping processing; wherein the recursive traversal processing is based on the nested structure planning of the second granularity index structure.

[0151] As an optional embodiment, the storage module recursively traverses the second granularity index structure after optimization and adjustment, and performs data mapping processing, which is specifically used to: identify key change points of the second granularity index structure before and after optimization and adjustment through a difference comparison model; based on the key change point identification results, recursively traverse the second granularity index structure before and after optimization and adjustment according to the nested structure to obtain local structural information in the second granularity index structure before and after optimization and adjustment; perform data migration based on the local structural information to map variable data from the second granularity index structure before optimization and adjustment to the second granularity index structure after optimization and adjustment.

[0152] As an optional embodiment, the storage module dynamically adjusts the second granularity index structure based on the query frequency and data update status of the variable data in each dynamic form, specifically for:

[0153] A multi-branch decision tree in the index decision model is used to make preliminary decisions on the query frequency and data update status of the variable data in each dynamic form, so as to obtain the initial adjustment information of the variable data in each dynamic form; the database state change trend in the initial adjustment information of the variable data in each dynamic form is identified through the state change recognition layer in the index decision model; the query mode change trend of each variable data under each dynamic form is analyzed through the mode change recognition layer in the index decision model; the database state change trend and the query mode change trend are integrated through the dynamic adjustment layer in the index decision model, and the data structure of each variable data under each dynamic form is adjusted to obtain the optimized and adjusted second granularity index structure.

[0154] The above-mentioned database-based big data query device 400 can implement the database-based big data query method of the above-mentioned method embodiment. The options in the above-mentioned method embodiment are also applicable to this embodiment and will not be described in detail here. The rest of the contents of the embodiment of this application can refer to the contents of the above-mentioned method embodiment, and will not be repeated in this embodiment.

[0155] In order to execute the database-based big data query method corresponding to the above method embodiment, to achieve the corresponding functions and technical effects. Figure 5 , Figure 5 The structural block diagram of a big data query device 500 based on a database provided in an embodiment of the present application is shown. For the convenience of explanation, only the parts related to the present embodiment are shown. The big data query device 500 based on a database provided in an embodiment of the present application is applied to a big data dynamic query system of a medical database. The system at least includes a display layer and an index layer, and a multi-layer index structure of multiple variable data is stored in the index layer. The big data query device 500 based on a database includes at least a processor 501, a memory 502, and a communication interface 503.

[0156] The above-mentioned database-based big data query device can implement the database-based big data query method of the above-mentioned method embodiment. The options in the above-mentioned method embodiment are also applicable to this embodiment and will not be described in detail here. The rest of the contents of the embodiment of this application can refer to the contents of the above-mentioned method embodiment, and will not be repeated in this embodiment.

[0157] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a storage controller, the steps in any of the above method embodiments are implemented.

[0158] An embodiment of the present application provides a computer program product. When the computer program product runs on a storage controller, the storage controller implements the steps in the above-mentioned method embodiments when executing.

[0159] In several embodiments provided in the present application, it is understood that each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.

[0160] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0161] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A big data query method based on a database, characterized in that: The method is applied to a big data dynamic query system of a medical database, the system at least comprising a display layer and an index layer, the index layer storing a multi-layer index structure of multiple variable data, the method comprising: In response to a user's data query request for a target object, a preliminary search is performed on the target object in the presentation layer to obtain first index information of the target object; the first index information is pre-established by a first granularity index structure of a database; the granularity level of the first granularity index structure is at least a form level; According to the first index information, a deep query is performed on the target object in the index layer to obtain second index information corresponding to the target object; the second index information is pre-established using a second granularity index structure of a database based on variable attributes of variable objects in the medical field; the granularity level of the second granularity index structure is at least a variable level; Querying the variable data associated with the second index information, and performing aggregation processing on the queried variable data to provide the target query result as feedback to the user; The queried variable data comes from at least one dynamic form associated with the second index information; the second granularity index structure is dynamically adjusted based on the query frequency and data update status of the variable data in each dynamic form; Before the preliminary search for the target object in the presentation layer to obtain the first index information of the target object is performed, the method further includes: Obtaining variable data to be stored; the variable data includes: clinical research data input through a visual data model editing interface; the visual data model editing interface includes an input component dynamically constructed based on clinical research needs; In the database, constructing a table layer data structure corresponding to the variable data; Based on the form layer data structure, the variable layer data structure corresponding to each form is reconstructed to construct multi-layer dynamic index information corresponding to the variable data.

2. The database-based big data query method according to claim 1, characterized in that: The step of constructing a table layer data structure corresponding to the variable data in the database includes: Perform form parsing and classification on the variable data, and store the variable data in a corresponding database form; Marking the form identifier where the variable data is located as the first index information; The variable layer data structure corresponding to each form is reconstructed based on the form layer data structure to construct multi-layer dynamic index information corresponding to the variable data, including: Variable parsing and classification are performed on the variable data in each database form, and variable identifiers corresponding to the variable data after secondary classification are marked as the second index information.

3. The database-based big data query method according to claim 2, characterized in that: A one-to-one association relationship or a one-to-many association relationship is established between the second index information and the variable data in the database form; Wherein, the association relationship is set based on case attributes and / or clinical research attributes.

4. The database-based big data query method according to claim 2, characterized in that: Each operation data of each variable data is associated with the corresponding first index information and / or second index information; The step of performing variable analysis and classification on the variable data in each database form and marking the variable identifiers corresponding to the variable data after secondary classification as the second index information further includes: Based on the first index information and / or the second index information, query the unique identification change logic chain of the corresponding variable data; Based on the unique identifier change logic chain, any historical version of the corresponding variable data is retrieved.

5. The database-based big data query method according to claim 2, characterized in that: The database is arranged with multiple levels; The multiple levels at least include: project level, visit level, and form level; the method further includes: Set up corresponding visit-level database tables and form-level database tables according to project requirements at different project levels; Bind the corresponding visit-level database tables and form-level database tables to input components at different project levels to collect clinical research data in different projects.

6. The database-based big data query method according to claim 1, characterized in that: The visual data model editing interface includes input components dynamically constructed based on clinical research needs; The step of obtaining the variable data to be stored includes: Accepting variable data to be stored through the input component; Use pre-set regular expressions and / or conditional statements corresponding to clinical research pathology to perform preliminary verification of the variable data to be stored; The preliminary verification results and the variable data to be stored are sent to the back-end database, and the variable data to be stored are secondary verified in the database to ensure data accuracy and completeness.

7. The database-based big data query method according to claim 1, characterized in that: The method further comprises: Dynamically adjusting the second granularity index structure based on the query frequency and data update status of the variable data in each dynamic form; The optimized and adjusted second granularity index structure is recursively traversed, and data mapping processing is performed; wherein the recursive traversal processing is based on the nested structure planning of the second granularity index structure.

8. The database-based big data query method according to claim 7, characterized in that: The dynamically adjusting the second granularity index structure based on the query frequency and data update status of the variable data in each dynamic form includes: The multi-branch decision tree in the index decision model is used to make preliminary decisions on the query frequency and data update status of the variable data in each dynamic form, so as to obtain the initial adjustment information of the variable data in each dynamic form; By indexing the state change identification layer in the decision model, the database state change trend in the initial adjustment information of the variable data in each dynamic form is identified; Through the pattern change recognition layer in the index decision model, the query pattern change trend of each variable data under each dynamic form is analyzed; Through the dynamic adjustment layer in the index decision model, the database status change trend and the query mode change trend are integrated to adjust the data structure of each variable data under each dynamic form to obtain the optimized and adjusted second granularity index structure.

9. A big data query device based on a database, characterized in that: The device is applied to a big data dynamic query system of a medical database, the system at least comprising a display layer and an index layer, wherein a multi-layer index structure of multiple variable data is stored in the index layer, and the device comprises: A search module is used to perform a preliminary search on the target object in the presentation layer in response to a user's data query request for the target object, so as to obtain first index information of the target object; the first index information is pre-established by a first granularity index structure of the database; the granularity level of the first granularity index structure is at least a form level; A query module, configured to perform a deep query on the target object in the index layer according to the first index information to obtain second index information corresponding to the target object; the second index information is pre-established based on the variable attributes of the variable object in the medical field using a second granularity index structure of the database; the granularity level of the second granularity index structure is at least the variable level; A feedback module, used for querying the variable data associated with the second index information, and performing aggregation processing on the queried variable data to provide feedback to the user as a target query result; The queried variable data comes from at least one dynamic form associated with the second index information; the second granularity index structure is dynamically adjusted based on the query frequency and data update status of the variable data in each dynamic form; The device also includes a storage module, which is used to: obtain variable data to be stored before the search module performs a preliminary search for the target object in the display layer to obtain the first index information of the target object; the variable data includes: clinical research data input through a visual data model editing interface; the visual data model editing interface contains input components dynamically constructed based on clinical research needs; in a database, construct a form layer data structure corresponding to the variable data; based on the form layer data structure, reconstruct the variable layer data structure corresponding to each form to construct multi-layer dynamic index information corresponding to the variable data.

Images