Data query method, device, apparatus, and storage medium

Abstract

This invention relates to data processing, and provides a data query method, apparatus, device, and storage medium. The method performs task table deduplication on a single-layer lineage configuration list to obtain a target single-layer lineage list. Based on the number of task tables and thread pool information, the target single-layer lineage list is split into multiple sub-tables. Multiple execution threads are invoked to query these sub-tables in parallel, obtaining configuration dependency tables and corresponding dependency levels. A cross-layer lineage list is generated based on the configuration task tables, configuration dependency tables, and dependency levels. Responding to the query object in the dependency data query request, the cross-layer lineage list is traversed, improving the comprehensiveness and efficiency of query result generation. Furthermore, this invention also relates to blockchain technology, allowing the query results to be stored in the blockchain.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data query method, apparatus, device, and storage medium. Background Technology

[0002] With the development of fintech, the volume of business data has also grown rapidly, leading to relationships between data points similar to kinship in human society. Within these data kinship relationships, there exist both single-layer and cross-layer dependencies.

[0003] Currently, when querying the dependency tables of a data table, it is usually only possible to query the upstream or downstream direct dependency tables of that data table. However, when the data table has data tables with cross-level lineage dependencies, it is not possible to quickly and comprehensively query all the dependency tables corresponding to that data table. Summary of the Invention

[0004] In view of the above, it is necessary to provide a data query method, apparatus, device and storage medium that can solve the technical problem of how to quickly and comprehensively query all dependency tables corresponding to a data table.

[0005] On one hand, the present invention proposes a data query method, the data query method comprising:

[0006] The read single-layer bloodline configuration list is deduplicated in the task table to obtain the target single-layer bloodline list;

[0007] Based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the target single-layer lineage list is split into multiple sub-tables;

[0008] Multiple execution threads are invoked from the preset thread pool to query the multiple partitioned tables in parallel, thereby obtaining the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table.

[0009] Generate a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level;

[0010] In response to a dependency data query request, the cross-lineage list is traversed based on the query object in the dependency data query request to obtain the query result.

[0011] According to a preferred embodiment of the present invention, the step of performing task table deduplication processing on the read single-layer bloodline configuration list to obtain the target single-layer bloodline list includes:

[0012] Obtain the list identifier of the single-layer bloodline configuration list;

[0013] Read the field identifiers of all stored fields from the single-layer lineage configuration list;

[0014] Write the list identifier, the field identifier, and the preset deduplication requirements into the preset deduplication function to obtain the deduplication script;

[0015] If the preset deduplication requirement includes a copy return requirement, then the deduplication script is run to obtain the target single-layer lineage list; or

[0016] If the preset deduplication requirement does not include the copy return requirement, then the deduplication script is run, and the updated single-layer lineage configuration list is determined as the target single-layer lineage list.

[0017] According to a preferred embodiment of the present invention, before splitting the target single-layer lineage list into multiple sub-tables based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the method further includes:

[0018] Identify the storage location of the list field in the target single-layer lineage list;

[0019] Based on the storage location, the task position of the configuration task table in the target single-layer lineage list is identified;

[0020] Based on the task location, the number of multiple configuration task tables in the target single-layer lineage list is counted to obtain the number of task tables.

[0021] According to a preferred embodiment of the present invention, the thread pool information includes the number of idle threads and resource availability information corresponding to idle threads in the preset thread pool. The step of splitting the target single-layer lineage list based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool to obtain multiple sub-tables includes:

[0022] Calculate the ratio of the number of tasks to the configured value;

[0023] If the number of idle threads is less than the ratio, a new thread is created in the preset thread pool based on the resource availability information, and the number of new threads created is counted.

[0024] The target quantity is obtained by summing the number of idle units and the number of units created.

[0025] Based on the target quantity, the target single-layer bloodline list is split to obtain the multiple sub-tables.

[0026] According to a preferred embodiment of the present invention, the step of calling multiple execution threads from the preset thread pool to query the multiple partitioned tables in parallel, and obtaining the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table, includes:

[0027] A dictionary is created based on the task identifiers in the configured task table;

[0028] Each execution thread is invoked to read the task identifier from the dictionary and query the run table of the configuration task table corresponding to the task identifier in the multiple sub-tables as the first data table;

[0029] The query is performed to find the run table of the first data table in the multiple sub-tables as the second data table, until the second data table no longer has a run table in the multiple sub-tables. Then the query on the configuration task table is stopped, and the first data table and the second data table are determined as the configuration dependency table.

[0030] The dependency hierarchy from the configuration dependency table to the configuration task table is identified as the dependency level;

[0031] When the query of the configuration task table is completed, the task identifier is deleted from the dictionary, and the task identifier is re-read from the dictionary for querying until the dictionary is empty.

[0032] According to a preferred embodiment of the present invention, generating a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level includes:

[0033] Create a cross-layer configuration list based on the configuration task table;

[0034] Identify the dependency relationship between the configuration task table and the configuration dependency table;

[0035] Based on the aforementioned dependency relationship, the configuration dependency table corresponding to the configuration task table and the corresponding dependency level are written into the configuration list to obtain the cross-layer lineage list.

[0036] According to a preferred embodiment of the present invention, the step of traversing the cross-generational lineage list based on the query object in the dependent data query request to obtain the query result includes:

[0037] The query object is parsed from the request message of the dependent data query request;

[0038] Generate a query statement based on the query object;

[0039] The query statement is executed on the cross-lineage bloodline list to obtain the query results.

[0040] On the other hand, the present invention also proposes a data query device, the data query device comprising:

[0041] The deduplication unit is used to perform task table deduplication on the read single-layer bloodline configuration list to obtain the target single-layer bloodline list;

[0042] The splitting unit is used to split the target single-layer lineage list into multiple sub-tables based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool.

[0043] The query unit is used to call multiple execution threads from the preset thread pool to query the multiple partitioned tables in parallel, and obtain the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table.

[0044] A generation unit is used to generate a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level.

[0045] The traversal unit is used to respond to a dependency data query request, and based on the query object in the dependency data query request, traverse the cross-layer lineage list to obtain the query result.

[0046] On the other hand, the present invention also proposes an electronic device, the electronic device comprising:

[0047] Memory, which stores computer-readable instructions; and

[0048] The processor executes computer-readable instructions stored in the memory to implement the data query method.

[0049] On the other hand, the present invention also proposes a computer-readable storage medium storing computer-readable instructions, which are executed by a processor in an electronic device to implement the data query method.

[0050] As can be seen from the above technical solutions, this application, by performing task table deduplication on the single-layer lineage configuration list, can avoid duplicate task tables in the target single-layer lineage list, thereby avoiding the inability to reasonably split the multiple sub-tables. It also avoids the storage space occupation of the subsequently generated cross-layer lineage list. Furthermore, combined with the number of task tables and the thread pool information, the rationality of generating the multiple sub-tables can be further improved. Furthermore, by calling multiple execution threads to perform parallel queries on the multiple sub-tables, the query efficiency of the configuration dependency table and the dependency level is improved, thereby increasing the generation efficiency of the cross-layer lineage list. Furthermore, based on the query object, this application traverses the cross-layer lineage list, directly obtaining all related data tables of the query object from the cross-layer lineage list, improving query comprehensiveness and query efficiency. Attached Figure Description

[0051] Figure 1 This is a flowchart of a preferred embodiment of the data query method of the present invention.

[0052] Figure 2 This is a schematic diagram of the target single-layer lineage list in the data query method of the present invention.

[0053] Figure 3 This is a schematic diagram of the cross-lineage bloodline list in the data query method of the present invention.

[0054] Figure 4 This is a functional block diagram of a preferred embodiment of the data query device of the present invention.

[0055] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the data query method of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the data query method of the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0058] The data query method is applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0059] The electronic device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0060] The electronic devices may include network devices and / or user devices. The network devices include, but are not limited to, single network electronic devices, groups of multiple network electronic devices, or cloud computing-based systems consisting of a large number of hosts or network electronic devices.

[0061] The network in which the electronic device is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0062] 101. Perform task table deduplication on the read single-layer bloodline configuration list to obtain the target single-layer bloodline list.

[0063] In at least one embodiment of the present invention, the single-layer lineage configuration list can be a list generated based on business data configuration in the fintech field. From the single-layer lineage configuration list, only dependency tables that directly depend on the data table can be read.

[0064] The target single-layer lineage list refers to the list obtained after removing duplicate task lists. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of the target single-layer lineage list in the data query method of the present invention. Figure 2 The field identifiers include downstream table fields and upstream table fields. Figure 2The data tables in the table include: DB.TB, DB1.TB1, DB2.TB2, DB2.TB3, DB2.TB4, DB2.TB5, and DB2.TB6. The upstream tables of data table DB.TB include: DB1.TB1, DB2.TB2, and DB2.TB3. The upstream table of data table DB2.TB2 includes: DB2.TB4. The upstream table of data table DB2.TB3 includes: DB2.TB5. The upstream table of data table DB2.TB5 includes: DB2.TB6. The upstream table of data table DB2.TB5 includes: DB2.TB2. The downstream table of data table DB1.TB1 includes: DB.TB. The downstream table of data table DB2.TB2 includes: DB.TB. The downstream table of data table DB2.TB3 includes: DB.TB. The downstream table of data table DB2.TB4 includes: DB2.TB2. The downstream tables of data table DB2.TB5 include: data table DB2.TB3. The downstream tables of data table DB2.TB6 include: data table DB2.TB5. The downstream tables of data table DB2.TB2 include: data table DB2.TB5.

[0065] The representation of the target single-layer lineage list is similar to that of the single-layer lineage configuration list, and will not be described in detail here.

[0066] In at least one embodiment of the present invention, the electronic device performs task table deduplication processing on the read single-layer lineage configuration list to obtain a target single-layer lineage list including:

[0067] Obtain the list identifier of the single-layer bloodline configuration list;

[0068] Read the field identifiers of all stored fields from the single-layer lineage configuration list;

[0069] Write the list identifier, the field identifier, and the preset deduplication requirements into the preset deduplication function to obtain the deduplication script;

[0070] If the preset deduplication requirement includes a copy return requirement, then the deduplication script is run to obtain the target single-layer lineage list; or

[0071] If the preset deduplication requirement does not include the copy return requirement, then the deduplication script is run, and the updated single-layer lineage configuration list is determined as the target single-layer lineage list.

[0072] The list identifier is used to uniquely represent the single-layer bloodline configuration list. For example, the list identifier can be the list name of the single-layer bloodline configuration list.

[0073] The stored field refers to the field in the single-layer lineage configuration list. The field identifier is used to uniquely represent the stored field. For example, the field identifier can be an identifier of downstream table, upstream table, etc.

[0074] The preset deduplication requirements include, but are not limited to, the retention status of the duplicate task table. The retention status of the duplicate task table includes: retaining only the first occurrence of the duplicate task table; retaining only the last occurrence of the duplicate task table; deleting all duplicate task tables, etc.

[0075] The preset deduplication function can be the drop_duplicates() function.

[0076] The deduplication script can be: df.drop_duplicates(subset = ['upstream table', 'downstream table')).

[0077] `],keep='first',inplace=False`. Here, `first` means to keep only the first occurrence of the duplicate task table, and `False` means to return a copy file after deleting the duplicate task table.

[0078] The request to return a copy indicates that a copy file will be returned after deleting the duplicate task table.

[0079] The preset deduplication function can be used to quickly generate the deduplication script, and then calling the deduplication script can quickly perform task table deduplication on the single-layer lineage configuration list, thereby improving the generation efficiency of the target single-layer lineage list.

[0080] 102. Based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the target single-layer lineage list is split into multiple sub-tables.

[0081] In at least one embodiment of the present invention, the task table data refers to the number of configuration task tables in the target single-layer lineage list.

[0082] The preset thread pool refers to the thread usage mode in the electronic device.

[0083] The thread pool information includes the number of idle threads in the preset thread pool and resource availability information. The resource availability information may include remaining CPU resources, etc.

[0084] In at least one embodiment of the present invention, before splitting the target single-layer lineage list into multiple sub-lists based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the method further includes:

[0085] Identify the storage location of the list field in the target single-layer lineage list;

[0086] Based on the storage location, the task position of the configuration task table in the target single-layer lineage list is identified;

[0087] Based on the task location, the number of multiple configuration task tables in the target single-layer lineage list is counted to obtain the number of task tables.

[0088] The storage location can be row-based or column-based. When the storage location is row-based, the task location is column-based.

[0089] The list field allows for quick identification of the storage location, which in turn improves the efficiency of task location identification. Furthermore, by using the task location to count the number of multiple configured tasks, the accuracy of the task table count can be improved.

[0090] In at least one embodiment of the present invention, the electronic device splits the target single-layer lineage list based on the number of task tables in the target single-layer lineage list and the thread pool information of a preset thread pool, obtaining multiple sub-tables including:

[0091] Calculate the ratio of the number of tasks to the configured value;

[0092] If the number of idle threads is less than the ratio, a new thread is created in the preset thread pool based on the resource availability information, and the number of new threads created is counted.

[0093] The target quantity is obtained by summing the number of idle units and the number of units created.

[0094] Based on the target quantity, the target single-layer bloodline list is split to obtain the multiple sub-tables.

[0095] The configuration values ​​can be set according to actual needs.

[0096] In this embodiment, if the number of idle threads is less than the ratio, it means that the idle threads cannot complete the processing of the multiple partitioned tables within the demand period. Therefore, by creating the new thread, it can be ensured that the idle threads and the new thread can complete the processing of the multiple partitioned tables within the demand period. At the same time, by creating the new thread using the resource availability information, it can be avoided that the CPU resources of the electronic device are excessively occupied, which could lead to system crash.

[0097] In other embodiments, if the number of idle items is greater than or equal to the ratio, the electronic device determines the ratio as the target number.

[0098] 103. Call multiple execution threads from the preset thread pool to query the multiple partitioned tables in parallel, and obtain the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table.

[0099] In at least one embodiment of the present invention, the plurality of execution threads include the idle thread and the new thread.

[0100] The configuration dependency table refers to a data table that is directly or indirectly dependent on the configuration task table.

[0101] The dependency level refers to the dependency hierarchy between the configuration dependency table and the configuration task table. For example, the dependency level of table S1, which is a direct dependency of table A, is 1, and the dependency level of table S2, which is a direct dependency of table S1, is 2.

[0102] In at least one embodiment of the present invention, the electronic device invokes multiple execution threads from the preset thread pool to query the multiple partitioned tables in parallel, obtaining the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table, including:

[0103] A dictionary is created based on the task identifiers in the configured task table;

[0104] Each execution thread is invoked to read the task identifier from the dictionary and query the run table of the configuration task table corresponding to the task identifier in the multiple sub-tables as the first data table;

[0105] The query is performed to find the run table of the first data table in the multiple sub-tables as the second data table, until the second data table no longer has a run table in the multiple sub-tables. Then the query on the configuration task table is stopped, and the first data table and the second data table are determined as the configuration dependency table.

[0106] The dependency hierarchy from the configuration dependency table to the configuration task table is identified as the dependency level;

[0107] When the query of the configuration task table is completed, the task identifier is deleted from the dictionary, and the task identifier is re-read from the dictionary for querying until the dictionary is empty.

[0108] The travel table may include an upstream table and a downstream table.

[0109] By creating the dictionary and then deleting the corresponding task identifier from the field when the configuration task table completes the query, repeated loop queries on the configuration task table can be avoided. By performing progressive queries on the configuration task table, the comprehensiveness of the query on the configuration dependency table can be improved. Furthermore, by using the dependency hierarchy from the configuration dependency table to the configuration task table, the dependency level can be accurately identified. At the same time, by calling the multiple execution threads to perform parallel queries on the multiple sub-tables, the query efficiency of the multiple sub-tables can be improved.

[0110] In this embodiment, after completing the query of the upstream table of the configuration task table, the query of the downstream table of the configuration task table is executed.

[0111] 104. Generate a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level.

[0112] In at least one embodiment of the present invention, the cross-level lineage list includes the configuration task table, the configuration dependency table, and the dependency level. From the cross-level lineage list, all direct or indirect dependency tables of any configuration task table can be comprehensively obtained. For example... Figure 3 As shown, Figure 3 This is a schematic diagram of the cross-lineage bloodline list in the data query method of the present invention. Figure 3 The fields in the table include the downstream table, the upstream table, and the dependency level. Figure 3 The data tables include: DB.TB, DB1.TB1, DB2.TB2, DB2.TB3, DB2.TB4, DB2.TB5, and DB2.TB6. From... Figure 3 It can be read that the upstream tables of data table DB.TB include data table DB1.TB1, data table DB2.TB2, data table DB2.TB3, data table DB2.TB4, data table DB2.TB5, and data table DB2.TB6.

[0113] In at least one embodiment of the present invention, the electronic device generates a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level, including:

[0114] Create a cross-layer configuration list based on the configuration task table;

[0115] Identify the dependency relationship between the configuration task table and the configuration dependency table;

[0116] Based on the aforementioned dependency relationship, the configuration dependency table corresponding to the configuration task table and the corresponding dependency level are written into the configuration list to obtain the cross-layer lineage list.

[0117] The cross-layer configuration list includes the configuration task table.

[0118] The dependencies include upstream dependencies, downstream dependencies, etc.

[0119] By identifying the dependencies, the configuration dependency table and the corresponding dependency level can be accurately written into the configuration list, thereby improving the accuracy of generating the cross-lineage list.

[0120] 105. In response to the dependency data query request, based on the query object in the dependency data query request, the cross-layer lineage list is traversed to obtain the query result.

[0121] It should be emphasized that, to further ensure the privacy and security of the above query results, the query results can also be stored in a node of a blockchain.

[0122] In at least one embodiment of the present invention, the dependent data query request may be a request triggered and generated by business personnel in the fintech field. The request message of the dependent data query request carries the query object. The query object may be any configured task table; for example, the query object may be the data table DB.TB.

[0123] The query result refers to the configuration task table that has a dependency relationship with the query object, and the dependency level between the query object and the configuration task table.

[0124] In at least one embodiment of the present invention, the electronic device traverses the cross-generational lineage list based on the query object in the dependent data query request, and obtains query results including:

[0125] The query object is parsed from the request message of the dependent data query request;

[0126] Generate a query statement based on the query object;

[0127] The query statement is executed on the cross-lineage bloodline list to obtain the query results.

[0128] The request message can be used to quickly parse the query object, thereby improving the efficiency of generating the query statement. Furthermore, by executing the query statement on the cross-lineage list, the query efficiency of the query results can be improved.

[0129] As can be seen from the above technical solutions, this application, by performing task table deduplication on the single-layer lineage configuration list, can avoid duplicate task tables in the target single-layer lineage list, thereby avoiding the inability to reasonably split the multiple sub-tables. It also avoids the storage space occupation of the subsequently generated cross-layer lineage list. Furthermore, combined with the number of task tables and the thread pool information, the rationality of generating the multiple sub-tables can be further improved. Furthermore, by calling multiple execution threads to perform parallel queries on the multiple sub-tables, the query efficiency of the configuration dependency table and the dependency level is improved, thereby increasing the generation efficiency of the cross-layer lineage list. Furthermore, based on the query object, this application traverses the cross-layer lineage list, directly obtaining all related data tables of the query object from the cross-layer lineage list, improving query comprehensiveness and query efficiency.

[0130] like Figure 4 The diagram shown is a functional block diagram of a preferred embodiment of the data query device of the present invention. The data query device 11 includes a deduplication unit 110, a splitting unit 111, a query unit 112, a generation unit 113, a traversal unit 114, an identification unit 115, and a statistics unit 116. The module / unit referred to in this invention refers to a series of computer-readable instruction segments that can be acquired by the processor 13 and perform a fixed function, and are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0131] The deduplication unit 110 is used to perform task table deduplication processing on the read single-layer bloodline configuration list to obtain the target single-layer bloodline list.

[0132] The splitting unit 111 is used to split the target single-layer lineage list based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool to obtain multiple sub-tables;

[0133] The query unit 112 is used to call multiple execution threads from the preset thread pool to query the multiple sub-tables in parallel, and obtain the configuration dependency table corresponding to each configuration task table in the multiple sub-tables and the dependency level corresponding to the configuration dependency table.

[0134] Generation unit 113 is used to generate a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level;

[0135] Traversal unit 114 is used to respond to a dependency data query request, and based on the query object in the dependency data query request, traverse the cross-layer lineage list to obtain the query result.

[0136] In at least one embodiment of the present invention, the deduplication unit 110 is further configured to obtain the list identifier of the single-layer bloodline configuration list;

[0137] Read the field identifiers of all stored fields from the single-layer lineage configuration list;

[0138] Write the list identifier, the field identifier, and the preset deduplication requirements into the preset deduplication function to obtain the deduplication script;

[0139] If the preset deduplication requirement includes a copy return requirement, then the deduplication script is run to obtain the target single-layer lineage list; or

[0140] If the preset deduplication requirement does not include the copy return requirement, then the deduplication script is run, and the updated single-layer lineage configuration list is determined as the target single-layer lineage list.

[0141] In at least one embodiment of the present invention, before splitting the target single-layer lineage list into multiple sub-tables based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the identification unit 115 is used to identify the storage location of the list fields in the target single-layer lineage list in the target single-layer lineage list.

[0142] The identification unit 115 is further configured to identify the task position of the configuration task table in the target single-layer lineage list based on the storage location;

[0143] The statistics unit 116 is used to count the number of multiple configuration task tables in the target single-layer lineage list based on the task location, and obtain the number of task tables.

[0144] In at least one embodiment of the present invention, the thread pool information includes the number of idle threads and resource availability information corresponding to the idle threads in the preset thread pool, and the splitting unit 111 is also used to calculate the ratio of the number of tasks in the task table to the configured value.

[0145] If the number of idle threads is less than the ratio, a new thread is created in the preset thread pool based on the resource availability information, and the number of new threads created is counted.

[0146] The target quantity is obtained by summing the number of idle units and the number of units created.

[0147] Based on the target quantity, the target single-layer bloodline list is split to obtain the multiple sub-tables.

[0148] In at least one embodiment of the present invention, the query unit 112 is further configured to create a dictionary based on the task identifier of the configuration task table;

[0149] Each execution thread is invoked to read the task identifier from the dictionary and query the run table of the configuration task table corresponding to the task identifier in the multiple sub-tables as the first data table;

[0150] The query is performed to find the run table of the first data table in the multiple sub-tables as the second data table, until the second data table no longer has a run table in the multiple sub-tables. Then the query on the configuration task table is stopped, and the first data table and the second data table are determined as the configuration dependency table.

[0151] The dependency hierarchy from the configuration dependency table to the configuration task table is identified as the dependency level;

[0152] When the query of the configuration task table is completed, the task identifier is deleted from the dictionary, and the task identifier is re-read from the dictionary for querying until the dictionary is empty.

[0153] In at least one embodiment of the present invention, the generation unit 113 is further configured to create a cross-layer configuration list based on the configuration task table;

[0154] Identify the dependency relationship between the configuration task table and the configuration dependency table;

[0155] Based on the aforementioned dependency relationship, the configuration dependency table corresponding to the configuration task table and the corresponding dependency level are written into the configuration list to obtain the cross-layer lineage list.

[0156] In at least one embodiment of the present invention, the traversal unit 114 is further configured to parse the query object from the request message of the dependent data query request;

[0157] Generate a query statement based on the query object;

[0158] The query statement is executed on the cross-lineage bloodline list to obtain the query results.

[0159] As can be seen from the above technical solutions, this application, by performing task table deduplication on the single-layer lineage configuration list, can avoid duplicate task tables in the target single-layer lineage list, thereby avoiding the inability to reasonably split the multiple sub-tables. It also avoids the storage space occupation of the subsequently generated cross-layer lineage list. Furthermore, combined with the number of task tables and the thread pool information, the rationality of generating the multiple sub-tables can be further improved. Furthermore, by calling multiple execution threads to perform parallel queries on the multiple sub-tables, the query efficiency of the configuration dependency table and the dependency level is improved, thereby increasing the generation efficiency of the cross-layer lineage list. Furthermore, based on the query object, this application traverses the cross-layer lineage list, directly obtaining all related data tables of the query object from the cross-layer lineage list, improving query comprehensiveness and query efficiency.

[0160] like Figure 5 The diagram shown is a schematic diagram of the structure of an electronic device that implements the data query method of the present invention.

[0161] In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and computer-readable instructions, such as a data query program, stored in the memory 12 and executable on the processor 13.

[0162] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, electronic device 1 may also include input / output devices, network access devices, buses, etc.

[0163] The processor 13 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the electronic device 1 through various interfaces and lines, and executing the operating system of the electronic device 1, as well as various installed application programs and program code.

[0164] For example, the computer-readable instructions can be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units can be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer-readable instructions in the electronic device 1. For example, the computer-readable instructions can be divided into a deduplication unit 110, a splitting unit 111, a query unit 112, a generation unit 113, a traversal unit 114, an identification unit 115, and a statistics unit 116.

[0165] The memory 12 can be used to store the computer-readable instructions and / or modules. The processor 13 implements various functions of the electronic device 1 by running or executing the computer-readable instructions and / or modules stored in the memory 12 and calling the data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. The memory 12 may include non-volatile and volatile memory, such as: hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other storage devices.

[0166] The memory 12 can be the external memory and / or internal memory of the electronic device 1. Furthermore, the memory 12 can be a physical memory, such as a memory module, a TF card (Trans-flash Card), etc.

[0167] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, the computer-readable instructions can implement the steps of the various method embodiments described above.

[0168] The computer-readable instructions include computer-readable instruction code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer-readable instruction code, recording medium, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), and random access memory (RAM).

[0169] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data querying, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0170] Combination Figure 1 The memory 12 in the electronic device 1 stores computer-readable instructions to implement a data query method, and the processor 13 can execute the computer-readable instructions to achieve the following:

[0171] The read single-layer bloodline configuration list is deduplicated in the task table to obtain the target single-layer bloodline list;

[0172] Based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the target single-layer lineage list is split into multiple sub-tables;

[0173] Multiple execution threads are invoked from the preset thread pool to query the multiple partitioned tables in parallel, thereby obtaining the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table.

[0174] Generate a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level;

[0175] In response to a dependency data query request, the cross-lineage list is traversed based on the query object in the dependency data query request to obtain the query result.

[0176] Specifically, the specific implementation method of the processor 13 for the above-mentioned computer-readable instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0177] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0178] The computer-readable storage medium stores computer-readable instructions, which, when executed by the processor 13, are used to perform the following steps:

[0179] The read single-layer bloodline configuration list is deduplicated in the task table to obtain the target single-layer bloodline list;

[0180] Based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the target single-layer lineage list is split into multiple sub-tables;

[0181] Multiple execution threads are invoked from the preset thread pool to query the multiple partitioned tables in parallel, thereby obtaining the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table.

[0182] Generate a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level;

[0183] In response to a dependency data query request, the cross-lineage list is traversed based on the query object in the dependency data query request to obtain the query result.

[0184] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0186] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0187] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A data query method, characterized in that, The data query method includes: The read single-layer bloodline configuration list is deduplicated in the task table to obtain the target single-layer bloodline list; Based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the target single-layer lineage list is split into multiple sub-tables; Multiple execution threads are invoked from the preset thread pool to query the multiple partitioned tables in parallel, obtaining the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table. This includes: creating a dictionary based on the task identifier of the configuration task table; invoking each execution thread to read the task identifier from the dictionary and querying the run table of the configuration task table corresponding to the task identifier in the multiple partitioned tables as a first data table; querying the run table of the first data table in the multiple partitioned tables as a second data table, until the second data table no longer has a run table in the multiple partitioned tables, stopping the query on the configuration task table, and determining the first data table and the second data table as the configuration dependency table; identifying the dependency level from the configuration dependency table to the configuration task table as the dependency level; when the query on the configuration task table is completed, deleting the task identifier from the dictionary and re-reading the task identifier from the dictionary for querying, until the dictionary is empty. Generate a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level; In response to a dependency data query request, the cross-lineage list is traversed based on the query object in the dependency data query request to obtain the query result.

2. The data query method as described in claim 1, characterized in that, The process of deduplicating the read single-layer lineage configuration list to obtain the target single-layer lineage list includes: Obtain the list identifier of the single-layer bloodline configuration list; Read the field identifiers of all stored fields from the single-layer lineage configuration list; Write the list identifier, the field identifier, and the preset deduplication requirements into the preset deduplication function to obtain the deduplication script; If the preset deduplication requirement includes a copy return requirement, then the deduplication script is run to obtain the target single-layer lineage list; or If the preset deduplication requirement does not include the copy return requirement, then the deduplication script is run, and the updated single-layer lineage configuration list is determined as the target single-layer lineage list.

3. The data query method as described in claim 1, characterized in that, Before splitting the target single-level lineage list into multiple sub-lists based on the number of task tables in the target single-level lineage list and the thread pool information of the preset thread pool, the method further includes: Identify the storage location of the list field in the target single-layer lineage list; Based on the storage location, the task position of the configuration task table in the target single-layer lineage list is identified; Based on the task location, the number of multiple configuration task tables in the target single-layer lineage list is counted to obtain the number of task tables.

4. The data query method as described in claim 1, characterized in that, The thread pool information includes the number of idle threads and resource availability information corresponding to idle threads in the preset thread pool. Based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool, the target single-layer lineage list is split to obtain multiple sub-tables, including: Calculate the ratio of the number of tasks to the configured value; If the number of idle threads is less than the ratio, a new thread is created in the preset thread pool based on the resource availability information, and the number of new threads created is counted. The target quantity is obtained by summing the number of idle units and the number of units created. Based on the target quantity, the target single-layer bloodline list is split to obtain the multiple sub-tables.

5. The data query method as described in claim 1, characterized in that, The step of generating a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level includes: Create a cross-layer configuration list based on the configuration task table; Identify the dependency relationship between the configuration task table and the configuration dependency table; Based on the aforementioned dependency relationship, the configuration dependency table corresponding to the configuration task table and the corresponding dependency level are written into the configuration list to obtain the cross-layer lineage list.

6. The data query method as described in claim 1, characterized in that, The query results obtained by traversing the cross-generational lineage list based on the query object in the dependency data query request include: The query object is parsed from the request message of the dependent data query request; Generate a query statement based on the query object; The query statement is executed on the cross-lineage bloodline list to obtain the query results.

7. A data query device, characterized in that, The data query device includes: The deduplication unit is used to perform task table deduplication on the read single-layer bloodline configuration list to obtain the target single-layer bloodline list; The splitting unit is used to split the target single-layer lineage list into multiple sub-tables based on the number of task tables in the target single-layer lineage list and the thread pool information of the preset thread pool. A query unit is configured to invoke multiple execution threads from the preset thread pool to query the multiple partitioned tables in parallel, obtaining the configuration dependency table corresponding to each configuration task table in the multiple partitioned tables and the dependency level corresponding to the configuration dependency table. The query unit includes: creating a dictionary based on the task identifier of the configuration task table; invoking each execution thread to read the task identifier from the dictionary and querying the run table of the configuration task table corresponding to the task identifier in the multiple partitioned tables as a first data table; querying the run table of the first data table in the multiple partitioned tables as a second data table, until the second data table no longer has a run table in the multiple partitioned tables, stopping the query on the configuration task table, and determining the first data table and the second data table as the configuration dependency table; identifying the dependency level from the configuration dependency table to the configuration task table as the dependency level; when the query on the configuration task table is completed, deleting the task identifier from the dictionary and re-reading the task identifier from the dictionary for querying, until the dictionary is empty. A generation unit is used to generate a cross-layer lineage list based on the configuration task table, the configuration dependency table, and the dependency level. The traversal unit is used to respond to a dependency data query request, and based on the query object in the dependency data query request, traverse the cross-layer lineage list to obtain the query result.

8. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the data query method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which are executed by a processor in an electronic device to implement the data query method as described in any one of claims 1 to 6.

Images