A method and device for supplementing metadata information based on data lineage

Abstract

An embodiment of the present invention provides a method and device for supplementing metadata information based on data lineage, which relates to the technical field of data processing. Among them, a method for supplementing metadata information based on data lineage includes: obtaining all metadata; determining at least one target field of the field description information to be supplemented, and the supplement content corresponding to each target field, where the supplement content corresponding to any target field is the field description information required to be supplemented for this target field; obtaining multiple target metadata to be supplemented from the all metadata; searching for the metadata including all target fields from the obtained multiple target metadata as the source data; using the supplement content corresponding to each target field with the source data as the supplement starting point, supplementing the field description information of the target fields in each target metadata. It can be seen that the embodiment of the present invention can improve the supplement efficiency of the field description information of metadata.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a method and device for supplementing metadata information based on data lineage. Background Art

[0002] With the increasing maturity of data governance development, data objects, as the core assets of enterprises and institutions, can be, for example, asset data objects, media data objects, etc. And metadata, as the data describing data objects, is also an important basis for the management of data object assets and extremely important descriptive information for the construction of data object catalogs. For example: Metadata can describe the name, type, size, etc. of asset data. And the field description information of metadata is the information describing the meaning of each field of metadata. Each field of metadata can have its own field description information to assist data users in better understanding asset data. For example: For the English name field in metadata, there can be field description information such as Chinese name, data precision, etc., to further describe the English name field.

[0003] However, in the actual implementation process, it is usually found that the field description information of the collected metadata is not standardized, resulting in incomplete field description information of the metadata. Therefore, there is a need to supplement the field description information of the metadata.

[0004] In the related art, a manual supplementing method is used to supplement the field description information of the metadata. Among them, manual supplementing generally supplements the field description information of the metadata through direct supplementing on the interface or importing through an excel table.

[0005] However, when supplementing in the related art, only the metadata on the current interface can be manually supplemented, that is, the field description information is supplemented for the fields to be supplemented in a metadata. This results in that for other metadata containing the same fields to be supplemented, manual supplementing needs to be performed again.

[0006] It can be seen that the supplementing efficiency of the field description information of the metadata in the related art is low. Summary of the Invention

[0007] The purpose of the embodiments of the present invention is to provide a method and device for supplementing metadata information based on data lineage, so as to improve the supplementing efficiency of the field description information of the metadata. The specific technical solutions are as follows:

[0008] In a first aspect, the embodiments of the present invention provide a method for supplementing metadata information based on data lineage, the method including:

[0009] Obtain all metadata; wherein, the all metadata includes the metadata of each database;

[0010] Determine at least one target field of the field description information to be supplemented, and the supplementary content corresponding to each target field. The supplementary content corresponding to each target field is the field description information required to be supplemented for that target field;

[0011] Obtain multiple target metadata to be supplemented from the full amount of metadata. Among them, there is a data lineage relationship among the multiple target metadata, and the data lineage relationship means that every two target metadata have at least one same target field;

[0012] Search for the metadata that includes all target fields from the multiple target metadata obtained, as the source data;

[0013] Taking the source data as the starting point for supplementation, use the supplementary content corresponding to each target field to supplement the field description information of the target fields in each target metadata.

[0014] Optionally, each target metadata other than the source data belongs to the upstream data to be supplemented or the downstream data to be supplemented of the source data;

[0015] The step of taking the source data as the starting point for supplementation and using the supplementary content corresponding to each target field to supplement the field description information of the target fields in each target metadata includes:

[0016] Taking the source data as the starting point for supplementation, in accordance with a predetermined supplementation order, use the supplementary content corresponding to each target field to supplement the field description information of the target fields in each target metadata;

[0017] Among them, the predetermined supplementation order is supplementation from front to back, supplementation from back to front, or mixed supplementation. The supplementation from front to back is to supplement from the source data to the downstream data to be supplemented; the supplementation from back to front is to supplement from the source data to the upstream data to be supplemented; the mixed supplementation is to supplement from the source data to both the upstream data to be supplemented and the downstream data to be supplemented simultaneously.

[0018] Optionally, the determination method of the upstream data to be supplemented and the downstream data to be supplemented includes:<9000039>

[0019] Obtain the data lineage link specified by the user for each target metadata, and use the obtained data lineage link to determine the upstream and downstream types of each target metadata other than the source data. Among them, the upstream and downstream types include the upstream data to be supplemented or the downstream data to be supplemented of the source data;

[0020] Alternatively, use a pre - set script tool for parsing the data lineage link of each target metadata to parse each target metadata, obtain the data lineage link of each target metadata, and use the parsed data lineage link to determine the upstream and downstream types of each target metadata other than the source data.

[0021] Optionally, using the source data as the starting point for supplementary recording, in accordance with a predetermined supplementary recording order, use the supplementary recording content corresponding to each target field to perform supplementary recording of the field description information for the target fields in each target metadata, including:

[0022] Using the source data as the starting point for supplementary recording, in accordance with a predetermined supplementary recording order and a predetermined supplementary recording rule, use the supplementary recording content corresponding to each target field to perform supplementary recording of the field description information for the target fields in each target metadata;

[0023] Among them, the supplementary recording rule includes:

[0024] The number - encounter pause rule, the number - encounter skip rule, the number - encounter overwrite rule, or the number - encounter continue - using rule;

[0025] Among them, the number - encounter pause rule means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then the supplementary recording is paused;

[0026] The number - encounter skip rule means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then skip it and continue to perform supplementary recording on other target metadata;

[0027] The number - encounter overwrite rule means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then overwrite the existing supplementary recording content and continue to perform supplementary recording on other target metadata;

[0028] The number - encounter continue - using rule means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then use the existing supplementary recording content as the new supplementary recording content to continue the supplementary recording.

[0029] In a second aspect, an embodiment of the present invention provides a metadata information supplementary recording device based on data lineage. The device includes:

[0030] A first acquisition module, configured to acquire all - volume metadata; among them, the all - volume metadata includes the metadata of each database;

[0031] A determination module, configured to determine at least one target field for which the field description information is to be supplemented, and the supplementary recording content corresponding to each target field, where the supplementary recording content corresponding to each target field is the field description information required to be supplemented for this target field;

[0032] A second acquisition module, configured to acquire multiple target metadata to be supplemented from the full amount of metadata; wherein, there is a data lineage relationship among the multiple target metadata, and the data lineage relationship means that every two target metadata have at least one same target field;

[0033] A search module, which searches for the metadata including all target fields from the multiple target metadata acquired, as the source data;

[0034] A supplementation module, configured to use the source data as the supplementation starting point, and use the supplementation content corresponding to each target field to supplement the field description information of the target fields in each target metadata.

[0035] Optionally, each target metadata other than the source data belongs to the upstream data to be supplemented or the downstream data to be supplemented of the source data;

[0036] The supplementation module includes:

[0037] A supplementation sub-module, configured to use the source data as the supplementation starting point, and in accordance with a predetermined supplementation order, use the supplementation content corresponding to each target field to supplement the field description information of the target fields in each target metadata;

[0038] Wherein, the predetermined supplementation order is supplementation from front to back, supplementation from back to front or mixed supplementation. The supplementation from front to back is to supplement from the source data to the downstream data to be supplemented; the supplementation from back to front is to supplement from the source data to the upstream data to be supplemented; the mixed supplementation is to supplement from the source data to the upstream data to be supplemented and the downstream data to be supplemented simultaneously.

[0039] Optionally, the determination method of the upstream data to be supplemented and the downstream data to be supplemented includes:

[0040] Acquire the data lineage link of each target metadata specified by the user, and use the acquired data lineage link to determine the upstream and downstream types of each target metadata other than the source data; wherein, the upstream and downstream types include the upstream data to be supplemented or the downstream data to be supplemented of the source data;

[0041] Or, use a script tool preset for parsing the data lineage link of each target metadata to parse each target metadata, obtain the data lineage link of each target metadata, and use the parsed data lineage link to determine the upstream and downstream types of each target metadata other than the source data.

[0042] Optionally, the supplementation sub-module includes:

[0043] A supplementary recording unit, which uses the source data as the starting point of supplementary recording, and according to a predetermined supplementary recording order and a predetermined supplementary recording rule, uses the supplementary content corresponding to each target field to supplement the field description information of the target fields in each target metadata;

[0044] Among them, the supplementary recording rule includes:

[0045] A rule of pausing when encountering a number, a rule of skipping when encountering a number, a rule of overwriting when encountering a number, or a rule of continuing to use when encountering a number;

[0046] Among them, the rule of pausing when encountering a number means that during the supplementary recording process, if it is detected that the supplementary content corresponding to the target field already exists in any target metadata, the supplementary recording is paused;

[0047] The rule of skipping when encountering a number means that during the supplementary recording process, if it is detected that the supplementary content corresponding to the target field already exists in any target metadata, then skip and continue to perform supplementary recording on other target metadata;

[0048] The rule of overwriting when encountering a number means that during the supplementary recording process, if it is detected that the supplementary content corresponding to the target field already exists in any target metadata, then overwrite the existing supplementary content and continue to perform supplementary recording on other target metadata;

[0049] The rule of continuing to use when encountering a number means that during the supplementary recording process, if it is detected that the supplementary content corresponding to the target field already exists in any target metadata, then use the existing supplementary content as the new supplementary content for continued supplementary recording.

[0050] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0051] The memory is used to store a computer program;

[0052] The processor, when executing the program stored on the memory, implements the method steps of the above-mentioned metadata information supplementary recording based on data lineage.

[0053] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method steps of any one of the above-mentioned metadata information supplementary recording based on data lineage are implemented.

[0054] Beneficial effects of the embodiments of the present invention:

[0055] The method for supplementing metadata information based on data lineage provided by the embodiments of the present invention can obtain all metadata, determine at least one target field of the description information of the fields to be supplemented, and its corresponding supplementary content. From all metadata, obtain multiple target metadata with data lineage to be supplemented. From the obtained multiple target metadata, find the metadata including all target fields as the source data. Using the source data as the starting point for supplementation, use the supplementary content corresponding to each target field to supplement the description information of the target fields in each target metadata. In this way, compared with the manual supplementation method in the prior art that only supplements a single metadata, the embodiments of the present invention can obtain multiple target metadata with data lineage to be supplemented, and thus supplement the description information of the target fields in each target metadata. It can be seen that the embodiments of the present invention can improve the supplementation efficiency of the description information of the fields of metadata, thereby reducing the cost of human input and facilitating the standardization construction of metadata.

[0056] Of course, implementing any product or method of the present invention does not necessarily require achieving all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0058] Figure 1 It is a schematic flowchart of a method for supplementing metadata information based on data lineage provided by the embodiments of the present invention;

[0059] Figure 2 It is a working flowchart of supplementing metadata information in a supplementary order provided by the embodiments of the present invention;

[0060] Figure 3 It is a working flowchart of supplementing metadata information in another supplementary order provided by the embodiments of the present invention;

[0061] Figure 4 It is a working flowchart of supplementing metadata information in another supplementary order provided by the embodiments of the present invention;

[0062] Figure 5 It is a schematic structural diagram of a device for supplementing metadata information based on data lineage provided by the embodiments of the present invention;

[0063] Figure 6 It is a schematic structural diagram of an electronic device provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art based on the present invention belong to the scope of protection of the present invention.

[0065] First, the following introduces the professional terms involved in the embodiments of the present invention:

[0066] Metadata: Also known as intermediate data or relay data, it is data that describes data objects, mainly information describing data attributes, and is used to support functions such as indicating storage locations, historical data, resource searches, and file records.

[0067] Field description information of metadata: Refers to the information that describes the meaning of fields in the metadata database, and this information can assist data users in better understanding the data.

[0068] In order to improve the efficiency of supplementing the field description information of metadata, the embodiments of the present invention provide a method and device for supplementing metadata information based on data lineage.

[0069] First, the following introduces a method for supplementing metadata information based on data lineage provided by the embodiments of the invention.

[0070] A method for supplementing metadata information based on data lineage provided by the embodiments of the present invention can be applied to an electronic device. In a specific application, the electronic device can be a terminal device or a server, and the embodiments of the present invention do not limit the specific form of the electronic device.

[0071] Specifically, the execution subject of this metadata information supplementing method can be a metadata information supplementing device. Exemplarily, when this metadata information supplementing method is applied to a terminal device, the metadata information supplementing device can be a client running on the terminal device and used for supplementing metadata information. Exemplarily, when this metadata information supplementing method is applied to a server, the metadata information supplementing device can be a computer program running on the server, and this computer program can be used for supplementing metadata information.

[0072] Among them, a method for supplementing metadata information based on data lineage provided by the embodiments of the present invention may include:

[0073] Obtain full-scale metadata; wherein, the full-scale metadata includes the metadata of each database;

[0074] Determine at least one target field for the field description information to be supplemented, and the supplementary content corresponding to each target field, where the supplementary content corresponding to each target field is the field description information required to be supplemented for that target field;

[0075] Obtain multiple target metadata to be supplemented from the full amount of metadata; among them, there is a data lineage relationship among the multiple target metadata, and the data lineage relationship is that each two target metadata have at least one same target field;

[0076] From the multiple target metadata obtained, find the metadata that includes all target fields as the source data;

[0077] Using the source data as the starting point for supplementation, use the supplementary content corresponding to each target field to supplement the field description information of the target fields in each target metadata.

[0078] The method for supplementing metadata information based on data lineage provided by the embodiments of the present invention can obtain the full amount of metadata, determine at least one target field for the field description information to be supplemented and its corresponding supplementary content, obtain multiple target metadata with data lineage relationships to be supplemented from the full amount of metadata, find the metadata that includes all target fields from the multiple target metadata obtained as the source data, use the source data as the starting point for supplementation, and use the supplementary content corresponding to each target field to supplement the field description information of the target fields in each target metadata. In this way, compared with the manual supplementation method in the prior art that only supplements a single metadata, the embodiments of the present invention can obtain multiple target metadata with data lineage relationships to be supplemented, so as to supplement the field description information of the target fields in each target metadata. It can be seen that the embodiments of the present invention can improve the supplementation efficiency of the field description information of the metadata, and further reduce the cost of human input, which is convenient for the standardization construction of the metadata.

[0079] The following introduces a method for supplementing metadata information based on data lineage provided by the embodiments of the present invention with reference to the accompanying drawings.

[0080] As Figure 1 shown, a method for supplementing metadata information based on data lineage provided by the embodiments of the present invention may include the following steps:

[0081] S101, obtain the full amount of metadata; among them, the full amount of metadata includes the metadata of each database;

[0082] Among them, the embodiments of the present invention do not limit the types of data content stored in the database. For example: asset data, multimedia data, and so on. It should be noted that the metadata in the database is information describing the attributes of the data, which is used to support functions such as indicating the storage location, historical data, resource search, and file records. Exemplarily, the metadata of asset data can be data describing the storage location and file size of the asset data. In the embodiments of the present invention, the metadata is based on the table field level, with a finer granularity. Exemplarily, if a user wants to collect the metadata of asset data, then the full amount of metadata of the asset data can be obtained from the databases of each asset data. The embodiments of the present invention do not make specific limitations on the database and the full amount of metadata.

[0083] It can be understood that any method that can obtain the full amount of metadata in the process of obtaining the full amount of metadata can be applied to this solution. In the embodiments of the present invention, the supplementary recording of metadata information based on data lineage is mainly elaborated, so no more details will be given here.

[0084] S102, determine at least one target field of the field description information to be supplemented, and the supplementary content corresponding to each target field. The supplementary content corresponding to each target field is the field description information required to be supplemented for this target field;

[0085] Among them, at least one target field can be set manually, that is, manually select at least one field to be supplemented from each field to obtain at least one target field; of course, the determination method of at least one target field is not limited to manual setting. Exemplarily, if the expression forms of multiple fields in the metadata are all in English, such as: number, name, account number, state, date, and so on for each target field, since there is only an English name without a Chinese name, it is not convenient for users to use. Therefore, Chinese names need to be supplemented for each field in English form. These fields are the above-mentioned at least one target field, and the supplemented Chinese file names are the field description information. For the above-mentioned fields, field description information such as code, name, account number, state, and date is supplemented in sequence.

[0086] It can be understood that the supplementary content corresponding to each target field, that is, the field description information required to be supplemented for this target field. Of course, the field description information required to be supplemented for the target field is not limited to the Chinese name, and can also be the field type, field precision, and so on. The embodiments of the present invention do not make specific limitations on the target field and its corresponding supplementary content.

[0087] S103, obtain multiple target metadata to be supplemented from the full amount of metadata; among them, there is a data lineage relationship among the multiple target metadata, and the data lineage relationship means that each two target metadata have at least one same target field;

[0088] Among them, multiple target metadata have data lineage relationships. The data lineage relationship is that any two target metadata have at least one same target field. That is, if two metadata have a data lineage relationship, it means there is a same target field between these two metadata. Exemplarily, in metadata A, there are target fields a, b, and c; in metadata B, there is target field a. Since metadata A and metadata B have the same target field a, it can be stated that metadata A and metadata B are two target metadata with a data lineage relationship. In metadata A, there are target fields a, b, and c; in metadata B, there is target field a; in metadata C, there are target fields b and c. Then, it can be stated that both metadata B and metadata C have a data lineage relationship with metadata A, thus obtaining three target metadata.

[0089] It can be understood that any method capable of obtaining multiple target metadata to be supplemented can be applied to this solution. Exemplarily, typical application scenarios include data warehouse or data lake construction, where there is a clear lineage link relationship for each layer. In the embodiments of the present invention, the focus is on the supplementation of metadata information based on data lineage, so the embodiments of the present invention will not elaborate on this too much.

[0090] S104. From the multiple target metadata obtained, find the metadata that includes all target fields as the source data;

[0091] Among them, the source data can also be understood as the source table. In the embodiments of the present invention, it is easier to understand when described in the form of the source table, and subsequent embodiments will also be described in the form of the source table.

[0092] Exemplarily, if each target field is a - e, in target metadata A, there are target fields a and b; in target metadata B, there are target fields c, d, and e; in target metadata C, there are target fields a - e. Then, target metadata C has all target fields, so target metadata C is used as the source table.

[0093] It can be understood that the process of finding the metadata that includes all target fields is also a relatively mature part in the prior art, so the embodiments of the present invention will not elaborate on this too much.

[0094] In one implementation, each target metadata other than the source data belongs to the upstream data to be supplemented or the downstream data to be supplemented of the source data;

[0095] It should be noted that the above implementation can also be understood as that each target metadata other than the source table belongs to the upstream or downstream pending supplementary table of the source table. It can be understood that the upstream pending supplementary data can also exist in the form of an upstream pending supplementary table, and the downstream pending supplementary data can also exist in the form of a downstream pending supplementary table. For the convenience of understanding, the following embodiments will be described in terms of the upstream / downstream pending supplementary tables.

[0096] It can be understood that the source table includes all target fields, while the upstream / downstream pending supplementary tables have fewer target fields than those in the source table.

[0097] Exemplarily, if the target fields are a - e, in target metadata A, there are target fields a and b; in target metadata B, there are target fields c, d, and e; in target metadata C, there are target fields a - e. Then, since target metadata C has all target fields, target metadata C serves as the source table. According to the setting of data lineage, if target metadata A is upstream of target metadata C, then target metadata A is the upstream pending supplementary table, and if target metadata B is downstream of target metadata C, then target metadata B is the downstream pending supplementary table.

[0098] It should be noted that the determination methods for upstream and downstream pending supplementary data include: Method A and Method B:

[0099] Method A: Obtain the data lineage links of each target metadata specified by the user, and use the obtained data lineage links to determine the upstream and downstream types of each target metadata other than the source data; wherein, the upstream and downstream types include the upstream pending supplementary data or the downstream pending supplementary data of the source data;

[0100] Method B: Use a pre - set script tool for parsing the data lineage links of each target metadata to parse each target metadata, obtain the data lineage links of each target metadata, and use the parsed data lineage links to determine the upstream and downstream types of each target metadata other than the source data.

[0101] In the implementation of Method A, for the upstream data to be backfilled and the downstream data to be backfilled, the upstream and downstream types of each target metadata except the source data can be determined according to the data lineage link of each target metadata specified by the user. It should be noted that the data lineage link includes data lineage relationships, which can associate each metadata. Through the upstream and downstream relationships represented by the data lineage link, the upstream and downstream types of each target metadata except the source data can be obtained. Among them, the data lineage relationship means that each two target metadata have at least one same target field. Exemplarily, for existing target metadata A, target metadata B, and target metadata C, if target metadata A is the source table, then the upstream and downstream types of target metadata B and target metadata C can be determined according to the data lineage link of target metadata B and target metadata C specified by the user. That is, according to the specified data lineage link, target metadata B and target metadata C are both downstream tables to be backfilled, and at the same time, target metadata B is the upstream of target metadata C.

[0102] In the implementation of Method B, for the upstream data to be backfilled and the downstream data to be backfilled, a script tool preset for parsing the data lineage link of each target metadata can be used to parse each target metadata, so as to obtain the data lineage link of each target metadata and determine the upstream and downstream types of each target metadata except the source data. Among them, the script tool can be a parsing script or a Structured Query Language (SQL). It can be understood that if the data lineage link of each target metadata is not specified by the user, the script tool can be used, and the target metadata can be parsed by the parsing script and SQL to obtain the data lineage link. Among them, the parsing process can also be understood as allocating the upstream and downstream types of the target metadata. Exemplarily, for existing target metadata A, target metadata B, and target metadata C, if target metadata B is the source table, then the target metadata A and target metadata C are parsed by the parsing script and SQL to obtain the data lineage link of target metadata A and target metadata C, and the upstream and downstream types of target metadata A and target metadata C are determined. That is, target metadata A is the upstream table to be backfilled, and target metadata C is the downstream table to be backfilled.

[0103] It should be noted that in practical applications, the upstream and downstream types of metadata are not actually crucial. There are three backfilling orders in the embodiments of the present invention, and backfilling can be performed for various situations. Therefore, the embodiments of the present invention do not specifically limit the upstream and downstream types of metadata.

[0104] It can be understood that the process of generating data lineage by parsing scripts and Structured Query Language (SQL) is currently supported by relatively mature existing technologies or solutions. Therefore, the embodiments of the present invention will not elaborate on this too much.

[0105] S105. Using the source data as the starting point for supplementary recording, and using the supplementary content corresponding to each target field, supplement the field description information of the target fields in each target metadata.

[0106] It should be noted that each supplementary recording should start from the source data to supplement other data to be supplemented.

[0107] In one implementation, using the source data as the starting point for supplementary recording, and using the supplementary content corresponding to each target field, supplement the field description information of the target fields in each target metadata, which may include step C1:

[0108] Step C1. Using the source data as the starting point for supplementary recording, in accordance with a predetermined supplementary recording order, and using the supplementary content corresponding to each target field, supplement the field description information of the target fields in each target metadata.

[0109] Among them, the predetermined supplementary recording order is supplementary recording from front to back, supplementary recording from back to front, or mixed supplementary recording. Supplementary recording from front to back means starting from the source data to supplement downstream data to be supplemented; Supplementary recording from back to front means starting from the source data to supplement upstream data to be supplemented; Mixed supplementary recording means starting from the source data to supplement both upstream data to be supplemented and downstream data to be supplemented simultaneously.

[0110] It should be noted that the above three supplementary recording orders can all be selected according to the user's needs and actual situations. The embodiments of the present invention do not make specific limitations on this.

[0111] It can be understood that supplementary recording from front to back means using the source data as the starting point for supplementary recording to supplement other downstream data to be supplemented. This supplementary recording order is suitable when each target metadata except the source data belongs to downstream data to be supplemented. Exemplarily, target metadata A is the source table, target metadata B and target metadata C are downstream tables to be supplemented, and target metadata B is the upstream table to be supplemented of target metadata C. Then, when using supplementary recording from front to back as the supplementary recording order, first supplement the information of target metadata A, and then sequentially supplement the information of target metadata B and target metadata C of the downstream tables to be supplemented.

[0112] Backward supplementary recording takes the source data as the starting point for supplementary recording, and supplementary records other upstream data to be supplemented. This supplementary recording order is suitable when each target metadata except the source data belongs to the upstream data to be supplemented. Exemplarily, target metadata A is the source table, and target metadata B and target metadata C are upstream tables to be supplemented. Then, when using backward supplementary recording as the supplementary recording order, first supplement target metadata A, and then sequentially supplement target metadata B and target metadata C in the upstream tables to be supplemented.

[0113] Mixed supplementary recording takes the source data as the starting point for supplementary recording, and simultaneously supplements other upstream data to be supplemented and downstream data to be supplemented. This supplementary recording order is suitable when each target metadata except the source data has both upstream data to be supplemented and downstream data to be supplemented; it should be noted that there are also cases where supplementary recording has been performed but not all of it has been completed, which also applies to mixed supplementary recording. Exemplarily, target metadata A is the source table, target metadata B is an upstream table to be supplemented, and target metadata C is a downstream table to be supplemented. Then, when using mixed supplementary recording as the supplementary recording order, first supplement target metadata A, and then simultaneously and sequentially supplement target metadata B in the upstream table to be supplemented and target metadata C in the downstream table to be supplemented.

[0114] It should be noted that the three orders of forward supplementary recording, backward supplementary recording, or mixed supplementary recording can also be understood as three modes for the user to select the supplementary recording order, that is, the three modes of forward supplementary recording mode, backward supplementary recording mode, or mixed supplementary recording mode. It can be understood that all three modes can be selected according to the user's needs.

[0115] In another implementation, in step C1, taking the source data as the starting point for supplementary recording, according to a predetermined supplementary recording order, using the supplementary content corresponding to each target field, supplement the field description information of the target fields in each target metadata, which may include step C11:

[0116] Step C11, taking the source data as the starting point for supplementary recording, according to a predetermined supplementary recording order and a predetermined supplementary recording rule, using the supplementary content corresponding to each target field, supplement the field description information of the target fields in each target metadata;

[0117] Among them, the supplementary recording rule includes:

[0118] The rule of pausing when encountering a number, the rule of skipping when encountering a number, the rule of overwriting when encountering a number, or the rule of continuing to use when encountering a number;

[0119] Among them, the rule of pausing when encountering a number means that during the supplementary recording process, if it is detected that the supplementary content corresponding to the target field already exists in any target metadata, the supplementary recording is paused;

[0120] The rule of skipping when encountering existing data means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then skip it and continue with the supplementary recording of other target metadata;

[0121] The rule of overwriting when encountering existing data means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then overwrite the existing supplementary recording content and continue with the supplementary recording of other target metadata;

[0122] The rule of using existing data when encountering existing data means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then use the existing supplementary recording content as the new supplementary recording content and continue with the supplementary recording.

[0123] It can be understood that the rule of pausing when encountering existing data means that if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then pause the supplementary recording. The advantage of this method is that when it is detected that there is supplementary recording content corresponding to the target field in the target metadata, it directly pauses and does not operate on the target metadata with the corresponding supplementary recording content, and will not affect the subsequent target metadata to be supplemented;

[0124] The rule of skipping when encountering existing data means that if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then skip it and continue with the supplementary recording of other target metadata. The advantage of this method is that when it is detected that there is supplementary recording content corresponding to the target field in the target metadata, it directly skips and continues with the supplementary recording of other target metadata, does not operate on this target metadata, directly skips, and does not affect the supplementary recording of subsequent target metadata.

[0125] The rule of overwriting when encountering existing data means that if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then overwrite the existing supplementary recording content and continue with the supplementary recording of other target metadata. The advantage of this method is that when it is detected that there is supplementary recording content corresponding to the target field in the target metadata, it directly overwrites the existing supplementary recording content and continues with the supplementary recording of other target metadata, overwrites the existing supplementary recording content of this target metadata, will not affect the supplementary recording of subsequent target metadata, and can also ensure the consistency of the target metadata.

[0126] The rule of using existing data when encountering existing data means that if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, then use the existing supplementary recording content as the new supplementary recording content and continue with the supplementary recording. The advantage of this method is that when it is detected that there is supplementary recording content corresponding to the target field in the target metadata, if the existing supplementary recording content is better than the existing supplementary recording content, then directly use the existing supplementary recording content as the new supplementary recording content and continue with the supplementary recording, and then modify the previous supplementary recording content later, so as to ensure the consistency of the target metadata.

[0127] It should be noted that the above supplementary recording rules can all be selected according to the user's needs and actual situations, and the embodiments of the present invention do not make specific limitations in this regard. Exemplarily, if the user selects the rule of pausing when encountering a number, during the process of supplementary recording of target metadata A to target metadata B, when it is detected that the Chinese name corresponding to "a number", i.e., "a number", already exists in target metadata B, then target metadata A will pause the supplementary recording.

[0128] Exemplarily, if the user selects the rule of skipping when encountering a number, during the process of supplementary recording of target metadata A to target metadata B, when it is detected that the Chinese name corresponding to "a number", i.e., "a number", already exists in target metadata B, then target metadata A will skip target metadata B and continue to perform supplementary recording on other target metadata.

[0129] Exemplarily, if the user selects the rule of overwriting when encountering a number, during the process of supplementary recording of target metadata A to target metadata B, when it is detected that the Chinese name corresponding to "a number", i.e., "a number", already exists in target metadata B, then target metadata A will overwrite "a number" and continue to perform supplementary recording on other target metadata.

[0130] Exemplarily, if the user selects the rule of following the existing content when encountering a number, during the process of supplementary recording of target metadata A to target metadata B, when it is detected that the Chinese name corresponding to "a number", i.e., "a number", already exists in target metadata B, then target metadata A will use the existing supplementary recording content, i.e., "a number", as the new supplementary recording content and continue the supplementary recording.

[0131] It should be noted that the rule of following the existing content when encountering a number can be applicable to the situation where the existing supplementary recording content is better than the existing supplementary recording content. During each supplementary recording process, the appropriate supplementary recording order and supplementary recording rules can be selected according to the user's needs.

[0132] The method for supplementing metadata information based on data lineage provided by the embodiments of the present invention can obtain all metadata, determine at least one target field of the description information of the fields to be supplemented and the corresponding supplement content, obtain multiple target metadata with data lineage to be supplemented from all metadata, search for the metadata including all target fields from the obtained multiple target metadata as the source data, use the source data as the supplement starting point, and supplement the description information of the target fields in each target metadata with the supplement content corresponding to each target field. In this way, compared with the manual supplement method in the prior art that only supplements a single metadata, the embodiments of the present invention can obtain multiple target metadata with data lineage to be supplemented, so as to supplement the description information of the target fields in each target metadata. It can be seen that the embodiments of the present invention can improve the supplement efficiency of the description information of the metadata fields, thereby reducing the cost of human input and facilitating the standardization construction of metadata.

[0133] To better understand the above three supplement orders, the following will illustrate again the situation of supplementing metadata information in different supplement orders with reference to the accompanying drawings:

[0134] As Figure 2 shown, the following gives an exemplary illustration of the supplement order from front to back:

[0135] The backfilling order from front to back can also be understood as the user has selected the backfilling mode from front to back. As the name implies, the metadata information is backfilled in the forward direction of the data lineage. The arrows in the figure represent the data lineage links, and the metadata in the figure is based on the table field level. Five fields in Table 1.1 affect the next-level target fields, namely a, b, d, e, and f. Among them, two target fields a and b in Table 2.1 are the same as the target fields in Table 1.1, and three target fields d, e, and f in Table 2.2 are the same as the target fields in Table 1.1. And so on, the target fields in Table 3.1 are the same as those in Table 2.1, and the target fields in Table 4.1 and Table 4.2 are the same as those in Table 3.1. Then, it can be found that in the entire data lineage link, all the target fields a, b, d, e, and f are in Table 1.1. So, Table 1.1 is the source data and also the source table. When performing backfilling, set Table 1.1 as the starting point of backfilling, and backfill the Chinese names corresponding to the target fields a, b, d, e, and f. Exemplarily, a is number, b is name, d is accountnumber, e is state, and f is date. Then, it is necessary to backfill the Chinese names corresponding to the target fields, that is, a coding, b name, d account, e status, and f date. Start the backfilling mode from front to back, and backfill the Chinese names corresponding to the target fields a and b in Table 2.1, that is, coding and name, and backfill the Chinese names corresponding to the target fields d, e, and f in Table 2.2, that is, account, status, and date. And so on to backfill the Chinese names corresponding to the target fields in Table 3.x and Table 4.x.

[0136] As Figure 3 shown, the following is an exemplary description of the backfilling order from back to front:

[0137] The backfilling order from back to front can also be understood as the user has selected the backfilling mode from back to front. Different from the first backfilling mode, the backfilling mode from back to front backfills the metadata information in the reverse direction. Before the backfilling, set Table 4.1 as the starting point of backfilling and backfill the metadata in the reverse direction. Exemplarily, there are target fields a, b, c, d, e, and f in Table 4.1, a is number, b is name, c is sign, d is account number, e is state, and f is date. Then, it is necessary to backfill the Chinese names corresponding to the target fields, that is, a coding, b name, c sign, d account, e status, and f date. After starting the backfilling mode from back to front, first backfill the Chinese names corresponding to the target fields a, b, and c in Table 3.1, that is, coding, name, and sign, and backfill the Chinese names corresponding to the target fields d, e, and f in Table 3.2, that is, account, status, and date; and so on to backfill the Chinese names corresponding to the target fields in Table 2.x, and finally backfill the Chinese names corresponding to the target fields in Table 1.x.

[0138] As Figure 4 shown below, an exemplary description of the supplementary recording order for hybrid supplementary recording is as follows:

[0139] The supplementary recording order for hybrid supplementary recording, which can also be understood as the hybrid supplementary recording mode, combines the two modes of from front to back and from back to front. Before supplementary recording, Table 3.1 is set as the starting point of supplementary recording. Exemplarily, there are 6 target fields a, b, c, d, e, f in Table 3.1, where a is number, b is name, c is sign, d is account number, e is state, and f is date. Then, the corresponding Chinese names of the target fields need to be supplemented, that is, a coding, b name, c flag, d account number, e state, and f date. When using the hybrid mode for supplementary recording, from Table 4.x to Table 4.4, the supplementary recording is carried out from front to back, that is, forward supplementary recording. Among them, in Table 4.1, the corresponding Chinese names of the target fields a, b, c are supplemented, that is, a coding, b name, c flag; in Table 4.2, the corresponding Chinese names of the target fields b, c are supplemented, that is, b name, c flag; in Table 4.3, the corresponding Chinese names of the target fields d, e are supplemented, that is, d account number, e state; in Table 4.4, the corresponding Chinese names of the target fields d, e, f are supplemented, that is, d account number, e state, f date. From Table 3.1 to Table 2.x, the supplementary recording is carried out from back to front, that is, reverse supplementary recording. The corresponding Chinese names of the target fields a, b, c in Table 2.1 are supplemented, that is, a coding, b name, c flag; the corresponding Chinese names of the target fields d, e, f in Table 2.2 are supplemented, that is, d account number, e state, f date, and so on for the corresponding Chinese names of Table 1.x.

[0140] The method for supplementing metadata information based on data lineage provided by the embodiments of the present invention can obtain all metadata, determine at least one target field of the description information of the field to be supplemented and its corresponding supplementary content, obtain multiple target metadata with data lineage to be supplemented from all metadata, find the metadata including all target fields from the obtained multiple target metadata as the source data, use the source data as the starting point of supplementary recording, and use the supplementary content corresponding to each target field to supplement the description information of the target field in each target metadata. In this way, compared with the manual supplementary recording method in the prior art that only supplements a single metadata, the embodiments of the present invention can obtain multiple target metadata with data lineage to be supplemented, so as to supplement the description information of the target field in each target metadata. It can be seen that the embodiments of the present invention can improve the efficiency of supplementing the description information of the metadata fields, thereby reducing the cost of human input and facilitating the standardization construction of metadata.

[0141] Based on the above method embodiments, as Figure 5As shown in the figure, an embodiment of the present invention further provides a metadata information supplementary recording device based on data lineage, which may include:

[0142] A first acquisition module 510, configured to acquire all metadata; wherein, the all metadata includes metadata of each database;

[0143] A determination module 520, configured to determine at least one target field of the field description information to be supplemented, and the supplementary content corresponding to each target field, and the supplementary content corresponding to each target field is the field description information required to be supplemented for this target field;

[0144] A second acquisition module 530, configured to acquire a plurality of target metadata to be supplemented from the all metadata; wherein, there is a data lineage relationship among the plurality of target metadata, and the data lineage relationship means that each two target metadata have at least one same target field;

[0145] A search module 540, configured to search, from the acquired plurality of target metadata, for the metadata including all target fields as the source data;

[0146] A supplementary recording module 550, configured to use the source data as the supplementary recording starting point, and use the supplementary content corresponding to each target field to supplement the field description information of the target fields in each target metadata.

[0147] Optionally, each target metadata other than the source data belongs to the upstream supplementary recording data or the downstream supplementary recording data of the source data;

[0148] The supplementary recording module includes:

[0149] A supplementary recording sub-module, configured to use the source data as the supplementary recording starting point, and according to a predetermined supplementary recording order, use the supplementary content corresponding to each target field to supplement the field description information of the target fields in each target metadata;

[0150] Wherein, the predetermined supplementary recording order is supplementary recording from front to back, supplementary recording from back to front or mixed supplementary recording. The supplementary recording from front to back is to perform supplementary recording from the source data to the downstream supplementary recording data; the supplementary recording from back to front is to perform supplementary recording from the source data to the upstream supplementary recording data; the mixed supplementary recording is to perform supplementary recording from the source data to the upstream supplementary recording data and the downstream supplementary recording data simultaneously.

[0151] Optionally, the determination method of the upstream supplementary recording data and the downstream supplementary recording data includes:

[0152] Obtain the data lineage links of each target metadata specified by the user, and use the obtained data lineage links to determine the upstream and downstream types of each target metadata except the source data; wherein, the upstream and downstream types include the upstream data to be supplemented or the downstream data to be supplemented of the source data;

[0153] Or, use a pre-set script tool for parsing the data lineage links of each target metadata to parse each target metadata, obtain the data lineage links of each target metadata, and use the parsed data lineage links to determine the upstream and downstream types of each target metadata except the source data.

[0154] Optionally, the supplementation sub-module includes:

[0155] A supplementation unit, which uses the source data as the supplementation starting point, and according to a predetermined supplementation order and a predetermined supplementation rule, uses the supplementation content corresponding to each target field to supplement the field description information of the target fields in each target metadata;

[0156] Wherein, the supplementation rule includes:

[0157] The rule of pausing when encountering data, the rule of skipping when encountering data, the rule of overwriting when encountering data, or the rule of continuing to use when encountering data;

[0158] Wherein, the rule of pausing when encountering data means that if the supplementation content corresponding to the target field is detected in any target metadata during the supplementation process, the supplementation is paused;

[0159] The rule of skipping when encountering data means that if the supplementation content corresponding to the target field is detected in any target metadata during the supplementation process, skip it and continue to supplement other target metadata;

[0160] The rule of overwriting when encountering data means that if the supplementation content corresponding to the target field is detected in any target metadata during the supplementation process, overwrite the existing supplementation content and continue to supplement other target metadata;

[0161] The rule of continuing to use when encountering data means that if the supplementation content corresponding to the target field is detected in any target metadata during the supplementation process, use the existing supplementation content as the new supplementation content and continue to supplement.

[0162] An embodiment of the present invention also provides an electronic device, as Figure 6 shown, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604,

[0163] A memory 603 for storing computer programs;

[0164] A processor 601, when executing the program stored in the memory 603, implements a method for supplementing metadata information based on data lineage.

[0165] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0166] The communication interface is used for communication between the above electronic device and other devices.

[0167] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0168] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0169] In another embodiment provided by the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above methods for supplementing metadata information based on data lineage are implemented.

[0170] In another embodiment provided by the present invention, there is also provided a computer program product containing instructions, which when running on a computer, causes the computer to execute any of the methods for supplementing metadata information based on data lineage in the above embodiments.

[0171] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0172] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements that are not expressly listed, or elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0173] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

[0174] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included within the protection scope of the present invention.

Claims

1. A method for supplementing metadata information based on data lineage, characterized in that, The method comprises: Obtaining full metadata; wherein the full metadata includes metadata of each database; Determine at least one target field for which field description information is to be supplemented, and supplementary content corresponding to each target field, where the supplementary content corresponding to each target field is the field description information required to be supplemented for the target field; Acquire multiple target metadata to be supplemented from the full metadata; wherein the multiple target metadata have a data lineage relationship, and the data lineage relationship is that every two target metadata have at least one common target field; Searching for metadata including all target fields from the obtained multiple target metadata as source data; Taking the source data as the supplementary recording starting point, using the supplementary recording content corresponding to each target field, supplementing the field description information of the target field in each target metadata; wherein, each target metadata other than the source data belongs to the upstream data to be supplemented or the downstream data to be supplemented of the source data; taking the source data as the supplementary recording starting point, using the supplementary recording content corresponding to each target field, supplementing the field description information of the target field in each target metadata, includes: taking the source data as the supplementary recording starting point, according to a predetermined supplementary recording order, using the supplementary recording content corresponding to each target field, supplementing the field description information of the target field in each target metadata; wherein, the predetermined supplementary recording order is supplementing from front to back, supplementing from back to front, or mixed supplementing, the supplementing from front to back is supplementing from the source data to the downstream data to be supplemented; the supplementing from back to front is supplementing from the source data to the upstream data to be supplemented; the mixed supplementing is supplementing from the source data to both the upstream data to be supplemented and the downstream data to be supplemented.

2. The method according to claim 1, wherein The method for determining the upstream data to be supplemented and the downstream data to be supplemented includes: Obtaining a data lineage link specified by a user for each target metadata, and using the obtained data lineage link to determine the upstream and downstream types of each target metadata other than the source data; wherein the upstream and downstream types include upstream data to be supplemented or downstream data to be supplemented of the source data; Alternatively, a pre-set script tool for parsing the data lineage links of each target metadata is used to parse each target metadata to obtain the data lineage links of each target metadata, and the data lineage links obtained by parsing are used to determine the upstream and downstream types of each target metadata except the source data.

3. The method according to claim 1, characterized in that, The method of supplementing the target fields in the target metadata using the source data as the supplementary recording starting point, in accordance with a predetermined supplementary recording order, and using the supplementary recording content corresponding to each target field, includes: Taking the source data as the supplementary recording starting point, supplementing the field description information of the target fields in each target metadata using the supplementary recording content corresponding to each target field according to a predetermined supplementary recording order and predetermined supplementary recording rules; The supplementary recording rules include: Encounter number suspension rule, encounter number skip rule, encounter number overwrite rule or encounter number continue rule; Among them, the rule of pausing when encountering data means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, the supplementary recording is paused; The rule of skipping when encountering data means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, the supplementary recording is skipped and the supplementary recording of other target metadata is continued; The rule of overwriting when encountering data means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, the existing supplementary recording content is overwritten, and the supplementary recording of other target metadata is continued; The rule of continuing to use when encountering data means that during the supplementary recording process, if it is detected that the supplementary recording content corresponding to the target field already exists in any target metadata, the existing supplementary recording content is used as the new supplementary recording content for continued supplementary recording.

4. A metadata information supplementing device based on data lineage, characterized in that, The device includes: A first acquisition module, configured to acquire all metadata; among them, the all metadata includes the metadata of each database; A determination module, configured to determine at least one target field of the field description information to be supplemented, and the supplementary recording content corresponding to each target field, where the supplementary recording content corresponding to each target field is the field description information required to be supplemented for the target field; A second acquisition module, configured to acquire multiple target metadata to be supplemented from the all metadata; among them, there is a data lineage relationship among the multiple target metadata, and the data lineage relationship means that each two target metadata have at least one same target field; A search module, configured to search for the metadata including all target fields from the acquired multiple target metadata as the source data; A supplementary recording module, configured to use the source data as the starting point for supplementary recording, and use the supplementary recording content corresponding to each target field to supplement the field description information of the target fields in each target metadata; each target metadata other than the source data belongs to the upstream supplementary recording data or downstream supplementary recording data of the source data; the supplementary recording module includes: a supplementary recording sub-module, configured to use the source data as the starting point for supplementary recording, and according to a predetermined supplementary recording order, use the supplementary recording content corresponding to each target field to supplement the field description information of the target fields in each target metadata; where the predetermined supplementary recording order is supplementary recording from front to back, supplementary recording from back to front, or mixed supplementary recording, and the supplementary recording from front to back is to perform supplementary recording from the source data to the downstream supplementary recording data; the supplementary recording from back to front is to perform supplementary recording from the source data to the upstream supplementary recording data; the mixed supplementary recording is to perform supplementary recording from the source data to the upstream supplementary recording data and the downstream supplementary recording data simultaneously.

5. The device according to claim 4, wherein, The determination method of the upstream supplementary recording data and the downstream supplementary recording data includes: Acquire the data lineage link specified by the user for each target metadata, and use the acquired data lineage link to determine the upstream and downstream types of each target metadata other than the source data; where the upstream and downstream types include the upstream supplementary recording data or downstream supplementary recording data of the source data; Alternatively, use a pre-set script tool for parsing the data lineage link of each target metadata to parse each target metadata, obtain the data lineage link of each target metadata, and use the parsed data lineage link to determine the upstream and downstream types of each target metadata other than the source data.

6. The device according to claim 4, characterized in that The supplementary recording sub-module includes: A supplementary recording unit, configured to use the source data as the starting point of supplementary recording, and according to a predetermined supplementary recording order and a predetermined supplementary recording rule, use the supplementary recording content corresponding to each target field to perform supplementary recording of the field description information of the target fields in each target metadata; Wherein, the supplementary recording rule includes: A rule of pausing when encountering a number, a rule of skipping when encountering a number, a rule of overwriting when encountering a number, or a rule of continuing to use when encountering a number; Wherein, the rule of pausing when encountering a number means that if the supplementary recording content corresponding to the target field is detected in any target metadata during the supplementary recording process, the supplementary recording is paused; The rule of skipping when encountering a number means that if the supplementary recording content corresponding to the target field is detected in any target metadata during the supplementary recording process, skip it and continue to perform supplementary recording on other target metadata; The rule of overwriting when encountering a number means that if the supplementary recording content corresponding to the target field is detected in any target metadata during the supplementary recording process, overwrite the existing supplementary recording content and continue to perform supplementary recording on other target metadata; The rule of continuing to use when encountering a number means that if the supplementary recording content corresponding to the target field is detected in any target metadata during the supplementary recording process, use the existing supplementary recording content as the new supplementary recording content and continue the supplementary recording.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used for storing computer programs; The processor is configured to implement the method steps described in any one of claims 1-3 when executing the programs stored on the memory.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the method steps described in any one of claims 1-3.

Images