Data asset retrieval method, apparatus, device, medium, and program product

Abstract

The present disclosure provides a data asset retrieval method, which can be applied to the fields of big data and financial technology. The method comprises: in response to a retrieval request, acquiring a plurality of fields of data to be retrieved, and calling data tables associated with the plurality of fields from a preset relational database; screening the data tables to obtain a minimum set of data tables covering the plurality of fields; screening key fields in the plurality of fields based on a preset rule; dividing the minimum set into at least one associated table set according to whether at least one key field associated with each data table exists; and outputting the associated table set and the key fields associated with the associated table set. The present disclosure also provides a data asset retrieval device, equipment, storage medium and program product.

Description

Technical Field

[0001] This disclosure relates to the field of big data technology, and specifically to a data asset retrieval method, apparatus, device, medium, and program product. Background Technology

[0002] With the rapid development of the internet age, the scale of enterprise data is increasing exponentially. How to quickly extract the required data and the relationships between different tables from massive amounts of data, so as to efficiently guide data development and business applications, has always been a difficult and painful problem in data asset management.

[0003] Data asset management aims to build an efficient communication platform for data management and application, thereby improving the level of data management and application. However, the process of data asset management still faces many problems or shortcomings, mainly reflected in the large data volume, wide coverage, difficulty in data collection, complexity of analysis and processing, and difficulty in fully mining the data. Summary of the Invention

[0004] In view of the above problems, this disclosure provides data asset methods, apparatus, devices, media and program products for improving data retrieval efficiency.

[0005] According to a first aspect of this disclosure, a data asset retrieval method is provided, comprising: in response to a retrieval request, obtaining multiple fields of data to be retrieved, retrieving data tables associated with the multiple fields from a preset relational database; filtering the data tables to obtain a minimum set of data tables covering the multiple fields; filtering key fields among the multiple fields based on preset rules; dividing the minimum set into at least one associated table set according to whether there is an intersection between at least one key field associated with each of the data tables, and outputting the associated table set and the key fields associated with the associated table set.

[0006] According to an embodiment of this disclosure, the step of filtering the data tables to obtain a minimum set of data tables covering the multiple fields includes: obtaining a set of tables of data tables associated with the multiple fields, and listing a first set of fields associated with each of the data tables; merging the first set of fields into groups of two without repetition; merging the combinations of the first set of fields into groups of two without repetition; repeating the merging operation until only one combination remains, wherein the data tables corresponding to the first set of fields included in the combination constitute the minimum set.

[0007] According to an embodiment of this disclosure, the step of combining the first field sets in pairs and performing a merging operation includes: comparing two first field sets in the combination to determine whether one of the first field sets in the combination contains the other first field set; when one of the first field sets in the combination contains the other first field set, merging the other first field set into the first field set; when one of the first field sets in the combination does not contain the other first field set, retaining the combination.

[0008] According to an embodiment of this disclosure, the step of combining each of the first field sets in pairs and merging the first field sets in the combinations includes: merging two combinations into one combination, determining whether there is a first field set in the combination that contains other first field sets; when there is a first field set in the combination that contains other first field sets, merging the first field sets that have an inclusion relationship and retaining the merged combination; when there is no first field set in the combination that contains other first field sets, retaining the combination.

[0009] According to an embodiment of this disclosure, the step of filtering key fields among the plurality of fields based on preset rules includes: dividing the plurality of key fields into a second field set according to their correspondence with the data table in the minimum set; arranging and combining the second field sets in pairs, and taking the intersection of the two second field sets in the combination, and taking the fields included in the union of the intersection as the key fields.

[0010] According to embodiments of this disclosure, the method further includes: filtering the key fields so that the key fields are associated with all data tables in the minimum set, and the number of key fields is minimized.

[0011] According to embodiments of this disclosure, dividing the minimum set into at least one associated table set based on whether there is an intersection of at least one key field associated with the data table, and outputting the associated table set and the key field associated with the associated table set includes: constructing a relationship network between the key field and the data tables in the minimum set, obtaining a connected subgraph of the relationship network, wherein there is an intersection of key fields between the data tables in the connected subgraph, and no intersection of key fields between the data tables in different connected subgraphs; constructing an associated table set from the data tables included in the connected subgraph, wherein the key fields included in the connected subgraph are the key fields associated with the associated table set.

[0012] According to an embodiment of this disclosure, constructing a relationship network between the key field and the data tables in the minimum set, and obtaining a connected subgraph of the relationship network, includes: using the key field and the data table associated with the key field as nodes, and using the association relationship as edges, constructing a relationship network to form at least one connected subgraph. The data tables and key fields in the connected subgraph are connected by at least one edge. When the number of connected subgraphs is greater than 1, there are no edges connecting different connected subgraphs.

[0013] A second aspect of this disclosure provides a data asset retrieval device, comprising: a request response module, configured to respond to a retrieval request, acquire multiple fields of data to be retrieved, and retrieve data tables associated with the multiple fields from a preset relational database; a data initial selection module, configured to filter the data tables to obtain a minimum set of data tables covering the multiple fields; a key field filtering module, configured to filter key fields among the multiple fields based on preset rules; and a data re-filtering module, configured to divide the minimum set into at least one associated table set based on whether there is an intersection of the key fields associated with the data tables, and output the associated table set and the key fields associated with the associated table set.

[0014] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the data asset retrieval method described above.

[0015] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the data asset retrieval method described above.

[0016] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned data asset retrieval method. Attached Figure Description

[0017] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0018] Figure 1 The illustrations depict application scenarios of data asset retrieval methods, apparatuses, devices, media, and program products according to embodiments of this disclosure.

[0019] Figure 2 A flowchart illustrating a data asset retrieval method according to an embodiment of the present disclosure is shown schematically.

[0020] Figure 3A flowchart illustrating operation S202 of the data asset retrieval method according to an embodiment of the present disclosure is shown schematically.

[0021] Figure 4A A flowchart illustrating operation S203 of the data asset retrieval method according to an embodiment of the present disclosure is shown schematically.

[0022] Figure 4B This illustration schematically shows another flowchart of the data asset retrieval method operation S203 according to an embodiment of the present disclosure;

[0023] Figure 5 A flowchart illustrating operation S204 of the data asset retrieval method according to an embodiment of the present disclosure is shown schematically.

[0024] Figure 6 A schematic diagram illustrating the structure of a data asset retrieval apparatus according to embodiments of the present disclosure is shown; and

[0025] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a data asset retrieval method according to an embodiment of the present disclosure. Detailed Implementation

[0026] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0030] The embodiments of this disclosure provide a data asset retrieval method, including: in response to a retrieval request, obtaining multiple fields of the data to be retrieved, retrieving data tables associated with the multiple fields from a preset relational database; filtering the data tables to obtain a minimum set of data tables covering the multiple fields; filtering key fields among the multiple fields based on preset rules; dividing the minimum set into at least one associated table set according to whether there is an intersection between at least one key field associated with each data table, and outputting the associated table set and the key fields associated with the associated table set.

[0031] It should be noted that the data asset retrieval method and apparatus provided in this disclosure can be used in data retrieval scenarios in the financial field, as well as in any field other than the financial field. The application fields of the data asset retrieval method and apparatus provided in this disclosure are not limited.

[0032] Figure 1 The illustration shows an application scenario of the data asset retrieval method and apparatus according to embodiments of the present disclosure.

[0033] like Figure 1 As shown, application scenario 100 according to this embodiment may include a data retrieval scenario in the financial field. Network 104 is used as a medium to provide a communication link between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links or fiber optic cables, etc.

[0034] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0035] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0036] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0037] It should be noted that the data asset retrieval method provided in this embodiment can generally be executed by server 105. Correspondingly, the data asset retrieval device provided in this embodiment can generally be located in server 105. The data asset retrieval method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the data asset retrieval device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0038] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0039] The following will be based on Figure 1 The described scene, through Figures 2-5 The data asset retrieval method of the disclosed embodiments is described in detail.

[0040] Figure 2 A flowchart illustrating a data asset retrieval method according to an embodiment of the present disclosure is shown.

[0041] like Figure 2 As shown, the data asset retrieval method of this embodiment includes operations S201 to S204, and the transaction processing method can be executed sequentially.

[0042] In operation S201, in response to the retrieval request, multiple fields of the data to be retrieved are obtained, and the data table associated with the multiple fields is retrieved from the preset relational database.

[0043] In this embodiment, a relational database can be pre-established to store relational data tables about "table name - field name", where there is a one-to-one correspondence between table names and field names, and the table names and field names form a composite primary key. Table names and field names can have many-to-many relationships, and the relational data tables store various representations of table names and field names. The relational data representation is shown in Table 1.

[0044] Table 1

[0045] Table Name (English) Table Name (in Chinese) Field name_English Field name_Chinese table_a Table a field_a field a

[0046] When a user inputs the fields they want to search, for example, 10 fields including f1, f2, f3, f4, f5, f6, f7, f8, f9, and f10, the system filters the data based on the relationship between the field names and table names to obtain the corresponding data tables. The relationship between fields and data tables can be one-to-many, and can be stored in the form of an Object, stored as {f n : [t1, t2, ...]}, t n This represents the corresponding data table. Specific examples include:

[0047] f1: [t1, t2],

[0048] f2: [t1],

[0049] f3: [t6],

[0050] f4: [t3, t6],

[0051] f5: [t2, t4],

[0052] f6: [t3, t5],

[0053] f7: [t1, t9],

[0054] f8: [t4, t 10 ],

[0055] f9: [t1, t2, t4, t7, t8, t9],

[0056] f 10 : [t7, t9, t 10 ]

[0057] The examples in the following embodiments will all be illustrated by this example.

[0058] In operation S202, the data table is filtered to obtain the smallest set of data tables covering multiple fields.

[0059] In this embodiment, the union of all data tables can be taken to initially filter the range of the required tables. A specific example is as follows:

[0060] {t1, t2}∪{t1}∪{t6}∪{t3, t6}∪{t2, t4}∪{t3, t5}∪{t1, t9}∪{t4, t 10}∪{t1, t2, t4, t7, t8, t9}∪{t7, t9, t 10}={t1, t2, t3, t4, t5, t6, t7, t8, t9, t 10}

[0061] Based on this, by filtering fields, the scope of the data table can be further narrowed down to ensure that the data table in the smallest set involves all fields.

[0062] By filtering the data table, the search scope can be greatly narrowed, the amount of data retrieved can be reduced, and search efficiency can be improved.

[0063] In operation S203, key fields are filtered from multiple fields based on preset rules.

[0064] By filtering key fields, the search scope can be further narrowed. Optionally, key fields can be fields that appear frequently in the smallest set of data tables, or fields with high weights calculated according to rules. Specific filtering rules can be customized according to the actual situation.

[0065] In operation S204, the minimum set is divided into at least one set of associated tables based on whether there is an intersection of at least one key field associated with each data table, and the set of associated tables and the key field associated with the set of associated tables are output.

[0066] In this embodiment, the minimum set of key fields and data tables greatly narrows the search scope, and the output of the associated table set and the key fields associated with the associated table set is the optimal search result.

[0067] The data asset retrieval method provided by the embodiments of this disclosure offers an efficient retrieval method, making up for the difficulties and inefficiencies in analyzing and retrieving massive amounts of data. At the same time, this technical solution is also user-friendly. Whether for developers or business personnel, the retrieval tools built based on this method can efficiently and quickly retrieve related data tables that meet their actual needs, as well as the relationships between related tables, thus eliminating the embarrassing situation of "difficulty in finding data".

[0068] Figure 3 A flowchart illustrating operation S202 of the data asset retrieval method according to an embodiment of the present disclosure is shown.

[0069] like Figure 3 As shown, in the data asset retrieval method operation S202 according to the embodiment of this disclosure, after obtaining the data table associated with all fields, in order to obtain the minimum set of the data table, operation S202 may further include operations S301 to S304.

[0070] In operation S301, obtain a set of tables of data tables associated with multiple fields, and list the first set of fields associated with each data table.

[0071] In this embodiment, taking 10 fields as an example, the fields include f1, f2, f3, f4, f5, f6, f7, f8, f9, and f10. The first set of fields associated with the data table includes:

[0072] t1: [f1, f2, f7, f9]

[0073] t2: [f1, f5, f9]

[0074] t3: [f4, f6],

[0075] t4: [f5, f8, f9]

[0076] t5: [f6],

[0077] t6: [f3, f4],

[0078] t7: [f9, f 10 ],

[0079] t8: [f9],

[0080] t9: [f7, f9, f 10 ],

[0081] t 10 : [f8, f 10 ]

[0082] In operation S302, the first field set is merged by grouping the two fields together without repetition.

[0083] Operation S302 may include: comparing two sets of first fields in the combination, determining whether one set of first fields in the combination contains the other set of first fields; when one set of first fields in the combination contains the other set of first fields, merging the other set of first fields into one set of first fields; when one set of first fields in the combination does not contain the other set of first fields, retaining the combination.

[0084] Specifically, based on the first field set obtained from operation S301, the first merging is performed, and the two subsequences are combined. The resulting combinations are:

[0085] {[f1, f2, f7, f9], [f1, f5, f9]}, {[f4, f6], [f5, f8, f9]}, {[f6], [f3, f4]}, {[f9, f 10 ]},{[f7,f9,f 10 ],[f7,f 10 ]}.

[0086] In operation S303, the combinations of each first field set are grouped into pairs without repetition, and the first field set in the combination is merged.

[0087] Operation S303 may include: merging two combinations into one combination, determining whether the combination contains a first field set or other first field sets; when the combination contains a first field set or other first field sets, merging the first field sets with inclusion relationships and retaining the merged combination; when the combination does not contain a first field set or other first field sets, retaining the combination.

[0088] In the combination and merging of combinations, the first field sets of the two combinations are combined, and the merging result is:

[0089] {[f1, f2, f7, f9], [f1, f5, f9], [f4, f6], [f5, f8, f9]}, {[f6], [f3, f4], [f9, f 10 ]},{[f7,f9,f 10 ], [f8, f 10 ]}.

[0090] In operation S304, the merge operation is repeated until only one combination remains. The data tables corresponding to the first set of fields included in the combination constitute the minimum set.

[0091] In this embodiment, after completing operation S303, if the combinations can still be merged, the third and fourth merging operations are performed until they can no longer be merged.

[0092] Based on the merging result of the embodiment in operation S303, a third merging can be performed, and the merging result is as follows:

[0093] {[f1, f2, f7, f9], [f1, f5, f9], [f4, f6], [f5, f8, f9], [f3, f4], [f9, f 10 ]},{[f7,f9,f 10 ], [f8, f 10 ]}.

[0094] The fourth merge was performed, and the result was:

[0095] {[f1, f2, f7, f9], [f1, f5, f9], [f4, f6], [f5, f8, f9], [f3, f4], [f7, f9, f 10 ], [f8, f 10 ]}.

[0096] Thus, this embodiment has obtained the minimum set of the first field set. Referring to the mapping relationship between the first field set and the data table in operation S301, the minimum set of the data table can be obtained. In this embodiment, the minimum set includes t1, t2, t3, t4, t6, t9, and t10.

[0097] Figure 4A A flowchart illustrating operation S203 of the data asset retrieval method according to an embodiment of the present disclosure is shown.

[0098] like Figure 4A As shown, the data asset retrieval method of this embodiment of the present disclosure includes operations S401 and S402, which filter key fields among multiple fields in operation S203.

[0099] In operation S401, multiple key fields are divided into a second field set according to their correspondence with the data table in the minimum set.

[0100] Based on the embodiments of operations S301 to S303, the data table of the minimum set and its corresponding field names can be used to obtain the second field set and its mapping relationship with the table:

[0101] t1: [f1, f2, f7, f9]

[0102] t2: [f1, f5, f9]

[0103] t3: [f4, f6],

[0104] t4: [f5, f8, f9]

[0105] t6: [f3, f4],

[0106] t9: [f7, f9, f 10 ],

[0107] t 10 : [f8, f 10 ]

[0108] In operation S402, the second field set is arranged and combined in pairs, and the intersection of the two second field sets in the combination is taken. The fields included in the union of the intersection are used as the key fields.

[0109] Find the intersection of the key values ​​of the two tables:

[0110] t1∩t2={f1,f9}; t1∩t4={f9}; t1∩t9={f9}; t2∩t4={f5,f9}; t2∩t9={f9}; t3∩t6={f4}; t4∩t9={f9};t4∩t 10 ={f8}; t9∩t 10 ={f 10}

[0111] Perform a union operation on the obtained intersection results to filter out the important fields:

[0112]

[0113] Figure 4B Another flowchart illustrating operation S203 of the data asset retrieval method according to an embodiment of the present disclosure is shown.

[0114] like Figure 4B As shown, the data asset retrieval method operation S203 of this embodiment includes operation S401 and operation S402, and may also include operation S403.

[0115] In operation S403, filter key fields to associate key fields with all data tables in the smallest set, and minimize the number of key fields.

[0116] In this embodiment, key fields can be intuitively obtained by constructing a relationship network between fields and tables. The relationship network is constructed as G = (V, E), where V represents the set of nodes and E represents the set of edges. Nodes represent fields or tables, and edges represent relationships between fields within tables, forming a connected subgraph.

[0117] Based on the key fields selected in operation S402, the correspondence between the key fields and the tables is as follows:

[0118] f1: [t1, t2],

[0119] f4: [t3, t6],

[0120] f5: [t2, t4],

[0121] f8: [t4, t10],

[0122] f9: [t1, t2, t4, t7, t8, t9],

[0123] f 10 : [t7, t9, t 10 ]

[0124] Then the connected subgraph includes connected components. Figure 1 include:

[0125]

[0126] Connecting element Figure 2 Including f4: [t3, t6], combined with the minimum table set {t1, t2, t3, t4, t6, t9, t... 10}, thus obtaining the association table as {t1, t2, t4, t9, t 10} and {t3, t6}.

[0127] Generate adjacency matrices for each connected subgraph, and filter key fields, as shown in the following example:

[0128] Table 2. Connecting elements Figure 1 adjacency matrix

[0129] <![CDATA[f1]]> <![CDATA[f5]]> <![CDATA[f8]]> <![CDATA[f9]]> <![CDATA[f 10 ]]> <![CDATA[t1]]> 1 1 <![CDATA[t2]]> 1 1 1 <![CDATA[t4]]> 1 1 1 <![CDATA[t7]]> 1 1 <![CDATA[t8]]> 1 <![CDATA[t9]]> 1 1 <![CDATA[t 10 ]]> 1 1

[0130] Table 3. Connectors Figure 2 adjacency matrix

[0131] <![CDATA[t3]]> <![CDATA[t6]]> <![CDATA[f4]]> 1 1

[0132] From the adjacency matrix in Table 1, it can be seen that only by passing through {f8, f9} or {f9, f 10} can then be used to define the table {t1, t2, t4, t9, t 10} Association;

[0133] From the adjacency matrix in Table 2, it can be seen that Tables {t3, t6} can be associated using only {f4};

[0134] We obtain the connected component. Figure 1 The key fields are {f8, f9} or {f9, f 10}, Connecting sub Figure 2 The key field is {f4}.

[0135] Figure 5 A flowchart illustrating operation S204 of the data asset retrieval method according to an embodiment of the present disclosure is shown.

[0136] like Figure 5 As shown, according to the data asset retrieval method operation S204 of this disclosure embodiment, the minimum set is divided into at least one associated table set based on whether there is an intersection of at least one key field associated with each data table, and the associated table set and the key field associated with the associated table set are output, which may include operations S501 to S502.

[0137] In operation S501, a relationship network of key fields and data tables in the minimum set is constructed, resulting in a connected subgraph of the relationship network. Data tables within a connected subgraph have an intersection of key fields, while data tables in different connected subgraphs do not have an intersection of key fields.

[0138] Using key fields and data tables associated with key fields as nodes, and relationships as edges, construct a relational network to form at least one connected subgraph. Data tables and key fields within a connected subgraph are connected by at least one edge. When the number of connected subgraphs is greater than 1, there are no edges connecting different connected subgraphs.

[0139] In operation S502, the associated table set constructed from the data tables within the connected subgraph is output, and the key fields within the connected subgraph are output as key fields associated with the associated table set.

[0140] Referring to the neighbor order matrices of the connected subgraphs shown in Tables 2 and 3, the connected subgraphs... Figure 1 The key fields are {f8, f9} or {f9, f 10 The corresponding associated table set is {t1, t2, t4, t9, t}. 10}, Connecting sub Figure 2 The key field is {f4}, and the associated table set is {t3, t6}.

[0141] The data asset retrieval method provided by the embodiments of this disclosure offers an efficient retrieval method, making up for the difficulties and inefficiencies in analyzing and retrieving massive amounts of data. At the same time, this technical solution is also user-friendly. Whether for developers or business personnel, the retrieval tools built based on this method can efficiently and quickly retrieve related data tables that meet their actual needs, as well as the relationships between related tables, thus eliminating the embarrassing situation of "difficulty in finding data".

[0142] Based on the above-described data asset retrieval method, this disclosure also provides a data asset retrieval device. The following will be combined with... Figure 6 The device is described in detail.

[0143] Figure 6 A schematic block diagram of a data asset retrieval apparatus according to an embodiment of the present disclosure is shown.

[0144] like Figure 6 As shown, the data asset retrieval device 600 of this embodiment includes a request response module 610, a data initial selection module 620, a key field filtering module 630, and a data re-screening module 640.

[0145] The request-response module 610 is used to respond to a retrieval request, obtain multiple fields of the data to be retrieved, and retrieve the data table associated with the multiple fields from a preset relational database. In one embodiment, the request-response module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0146] The data initial selection module 620 is used to filter data tables to obtain a minimum set of data tables covering multiple fields. In one embodiment, the data initial selection module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0147] The key field filtering module 630 is used to filter key fields among multiple fields based on preset rules. In one embodiment, the key field filtering module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0148] The data re-screening module 640 is used to divide the minimum set into at least one associated table set based on whether there is an intersection of the key fields associated with the data tables, and outputs the associated table set and the key fields associated with the associated table set. In one embodiment, the data re-screening module 640 can be used to perform the operation S240 described above, which will not be repeated here.

[0149] The data initial selection module 620 includes a first field set listing unit, a set merging unit, a combination merging unit, and an initial selection result output unit.

[0150] The first field set listing unit is used to obtain a set of tables of data tables associated with the multiple fields, and to list the first field set associated with each of the data tables.

[0151] The set merging unit is used to merge the first set of fields into pairs without repetition.

[0152] The combination and merging unit is used to combine the combinations of the first field sets in pairs without repetition, and to perform a merging operation on the first field sets in the combination.

[0153] The initial selection result output unit is used to repeat the merge operation until only one combination remains. The data table corresponding to the first field set included in the combination constitutes the minimum set.

[0154] The set merging unit includes a set comparison subunit and a first merging subunit.

[0155] The set comparison subunit is used to compare two sets of first fields in a combination, and to determine whether one set of first fields in the combination contains the other set of first fields.

[0156] The first merging subunit is used to merge the other first field set into the first field set when one of the first field sets in the combination contains the other first field set, and to retain the combination when one of the first field sets in the combination does not contain the other first field set.

[0157] The combined merging unit includes a combined comparison field element and a second merging sub-unit.

[0158] The combination comparison field element is used to merge two combinations into one combination and determine whether the combination contains a first field set or other first field sets.

[0159] The second merging subunit is used to merge the first field sets that have an inclusion relationship when there is a first field set in the combination that contains other first field sets, and to retain the merged combination; and it is used to retain the combination when there is no first field set in the combination that contains other first field sets.

[0160] The key field filtering module 630 includes a second field set listing unit and a key field selection unit.

[0161] The second field set enumeration unit is used to divide the multiple key fields into a second field set according to their correspondence with the data table in the minimum set.

[0162] The key field selection unit is used to arrange and combine the second field set in pairs, and take the intersection of the two second field sets in the combination, and take the fields included in the union of the intersection as the key field.

[0163] The key field filtering module 630 also includes a key field extraction unit.

[0164] The key field extraction unit is used to filter the key fields so that the key fields are associated with all data tables in the minimum set, and the number of key fields is minimized.

[0165] The data rescreening module 640 includes a relational network construction unit and a data output unit.

[0166] The relationship network construction unit is used to construct a relationship network between the key fields and the data tables in the minimum set, obtaining a connected subgraph of the relationship network. Data tables within the connected subgraphs have an intersection of key fields, while data tables in different connected subgraphs do not have an intersection of key fields. The relationship network construction unit uses the key fields and the data tables associated with them as nodes, and the association relationships as edges, to construct a relationship network, forming at least one connected subgraph. Data tables and key fields within the connected subgraphs are connected by at least one edge. When the number of connected subgraphs is greater than one, there are no edges connecting different connected subgraphs.

[0167] The data output unit is used to output the associated table set constructed from the data tables within the connected subgraph, and to output the key fields within the connected subgraph as key fields associated with the associated table set.

[0168] According to embodiments of this disclosure, any plurality of modules among the request response module 610, data initial selection module 620, key field filtering module 630, and data re-filtering module 640 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the request response module 610, data initial selection module 620, key field filtering module 630, and data re-filtering module 640 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any one of the three implementation methods or a suitable combination of any of them. Alternatively, at least one of the request response module 610, the data initial selection module 620, the key field filtering module 630, and the data re-screening module 640 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0169] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a data asset retrieval method according to an embodiment of the present disclosure.

[0170] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0171] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0172] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0173] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0174] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0175] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the data asset retrieval method provided in the embodiments of this disclosure.

[0176] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0177] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0178] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0179] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0180] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0181] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0182] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A data asset retrieval method, comprising: In response to a retrieval request, multiple fields of the data to be retrieved are obtained, and a data table associated with the multiple fields is retrieved from a preset relational database; Filter the data tables to obtain the smallest set of data tables that cover the multiple fields; Filter the key fields among the multiple fields based on preset rules; Based on whether there is an intersection of at least one key field associated with each of the data tables, the minimum set is divided into at least one associated table set, and the associated table set and the key field associated with the table set are output; The process of filtering the data tables to obtain a minimum set of data tables covering the multiple fields includes: Obtain a set of tables of data tables associated with the multiple fields, and list the first set of fields associated with each of the data tables; The first set of fields is grouped into pairs without repetition, and then merged. The combinations of the first field sets are grouped into pairs without repetition, and the first field sets in the combination are merged. The merge operation is repeated until only one combination remains, and the data tables corresponding to the first set of fields included in the combination constitute the minimum set. The step of combining the first set of fields pairwise and merging them includes: Compare the two sets of first fields in the combination to determine whether one set of first fields in the combination contains the other set of first fields. When one of the first field sets in a combination contains another first field set, the other first field set is merged into the first field set. The combination is retained when one of the first field sets in the combination does not contain the other first field set; The step of combining each of the first field sets in pairs and merging the first field sets in the combination includes: Merge the two combinations into one combination, and determine whether the combination contains a first field set or other first field sets. When a combination contains other sets of first fields, merge the sets of first fields that have an inclusion relationship and retain the merged combination. If no first field set in the combination contains other first field sets, the combination is retained.

2. The method according to claim 1, wherein filtering the key fields among the plurality of fields based on preset rules includes: The multiple key fields are divided into a second field set according to their correspondence with the data tables in the minimum set; The second field set is arranged in pairs, and the intersection of the two second field sets in the combination is taken. The fields included in the union of the intersection are taken as the key fields.

3. The method according to claim 2, further comprising: Filter the key fields so that the key fields are associated with all data tables in the minimum set, and the number of key fields is minimized.

4. The method according to claim 1, wherein dividing the minimum set into at least one associated table set based on whether there is an intersection of at least one key field associated with the data table, and outputting the associated table set and the key field associated with the associated table set, comprises: Construct a relationship network between the key fields and the data tables in the minimum set to obtain a connected subgraph of the relationship network. There is an intersection of key fields between the data tables in the connected subgraph, but no intersection of key fields between the data tables in different connected subgraphs. Output the associated table set constructed from the data tables within the connected subgraph, and output the key fields within the connected subgraph as key fields associated with the associated table set.

5. The method according to claim 4, wherein constructing the relationship network between the key fields and the data tables in the minimum set, and obtaining the connected subgraph of the relationship network, comprises: Using the key field and the data table associated with the key field as nodes, and the association relationship as edges, a relationship network is constructed to form at least one connected subgraph. The data table and key field in the connected subgraph are connected by at least one edge. When the number of connected subgraphs is greater than 1, there are no edges connecting different connected subgraphs.

6. A data asset retrieval device, comprising: The request response module is used to respond to a retrieval request, obtain multiple fields of the data to be retrieved, and retrieve the data table associated with the multiple fields from a preset relational database; The data initial selection module is used to filter the data tables to obtain the minimum set of data tables covering the multiple fields; The key field filtering module is used to filter key fields among the multiple fields based on preset rules; The step of filtering the data tables to obtain a minimum set of data tables covering the multiple fields includes: obtaining a set of tables of data tables associated with the multiple fields; listing the first field set associated with each of the data tables; merging the first field sets into groups of two without repetition; merging the combinations of the first field sets into groups of two without repetition; repeating the merging operation until only one combination remains, and the data tables corresponding to the first field sets included in the combination constitute the minimum set. The step of combining the first field sets in pairs and merging them includes: comparing the two first field sets in the combination to determine whether one of the first field sets in the combination contains the other first field set; when one of the first field sets in the combination contains the other first field set, merging the other first field set into the first first field set; when one of the first field sets in the combination does not contain the other first field set, retaining the combination. The step of combining each of the first field sets in pairs and merging the first field sets in the combinations includes: merging two combinations into one combination, determining whether there is a first field set in the combination that contains other first field sets; when there is a first field set in the combination that contains other first field sets, merging the first field sets that have an inclusion relationship and retaining the merged combination; when there is no first field set in the combination that contains other first field sets, retaining the combination. The data re-screening module is used to divide the minimum set into at least one associated table set based on whether there is an intersection of the key fields associated with the data table, and output the associated table set and the key fields associated with the associated table set.

7. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 5.

9. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images