Data asset processing method and device, equipment, storage medium and program product
By comparing the properties of the first asset element of the enterprise-level data assets with the preset second asset element, and determining the processing method, the problem of long processing cycle of data assets in the prior art is solved, and processing efficiency and flexibility are improved.
Patent Information
- Application Number
- CN202411599269.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2025-03-18
AI Technical Summary
When processing enterprise-level data assets, the classification and identification periods are long, resulting in low processing efficiency and long processing periods.
By obtaining the to-be-processed data assets and the preset second asset element set, each first asset element determines the target second asset element matching its category from the second asset element set, and compares its attributes and processes it according to the comparison results.
Improve the processing efficiency of data assets, reduce the processing cycle, and achieve more flexible and dynamic asset management.
Smart Images
Figure CN119648409A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular, to a method, device, equipment, storage medium and program product for processing data assets. Background Art
[0002] The enterprise-level data architecture defines the overall structure, data types and sources, overall data storage distribution and interrelationships of the data required for the enterprise to build an information system. It defines data requirements through enterprise-level data models, guides the distribution control and integration of data assets, deploys data sharing and application environments, and specifies metadata management. The data architecture asset model (also called the data asset model) refers to the systematic and modeled abstraction and application of relevant elements in the data architecture during the information management process of the enterprise.
[0003] Currently, the way to process enterprise-level data assets is mostly to determine the asset category through extraction planning, which leads to a long data asset processing cycle. Summary of the invention
[0004] The embodiments of the present invention provide a data asset processing method, device, equipment, storage medium and program product, which can improve the processing efficiency of data assets and reduce the processing cycle.
[0005] In a first aspect, an embodiment of the present invention provides a method for processing data assets, including:
[0006] Acquire a data asset to be processed and a preset second asset element set; wherein the data asset to be processed includes at least one first asset element of a different category, each of the first asset elements includes a plurality of first attributes, and each of the second asset elements in the second asset element set has a different category, and each of the second asset elements includes a plurality of second attributes;
[0007] For each of the first asset elements, determine a second asset element that matches the category of the first asset element from the set of the second asset elements and determine it as a target second asset element;
[0008] Comparing the plurality of first attributes of the first asset element with the plurality of second attributes in the target second asset element to obtain a comparison result;
[0009] The first asset element is processed according to the comparison result.
[0010] In a second aspect, an embodiment of the present invention further provides a data asset processing device, including:
[0011] A data asset acquisition module, used to acquire a data asset to be processed and a preset second asset element set; wherein the data asset to be processed includes at least one first asset element of a different category, each of the first asset elements includes a plurality of first attributes, and each of the second asset elements in the second asset element set has a different category, and each of the second asset elements includes a plurality of second attributes;
[0012] a target second asset element determination module, configured to determine, for each of the first asset elements, from the set of the second asset elements, a second asset element that matches the category of the first asset element and determine it as a target second asset element;
[0013] A comparison result acquisition module, used for comparing the multiple first attributes of the first asset element with the multiple second attributes in the target second asset element to obtain a comparison result;
[0014] An asset element processing module is used to process the first asset element according to the comparison result.
[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for processing data assets as described in any one of the embodiments of the present invention is implemented.
[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for processing data assets as described in any one of the embodiments of the present invention.
[0017] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements a method for processing data assets as described in any one of the embodiments of the present invention.
[0018] The embodiment of the present invention discloses a method, device, equipment, storage medium and program product for processing data assets, obtaining data assets to be processed and a preset second asset element set; wherein the data assets to be processed include at least one first asset element of different categories, each first asset element includes multiple first attributes, and the categories of the second asset elements in the second asset element set are different, and each second asset element includes multiple second attributes; for each first asset element, a second asset element matching the category of the first asset element is determined from the second asset element set and determined as the target second asset element; the multiple first attributes of the first asset element are compared with the multiple second attributes in the target second asset element to obtain a comparison result; and the first asset element is processed according to the comparison result. The method for processing data assets provided by the embodiment of the present invention can improve the processing efficiency of data assets and reduce the processing cycle by determining the processing method of the first asset element based on the comparison result of the attributes of the first asset element in the data asset to be processed with the attributes of the second asset element. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0020] Figure 1 A flowchart of a data asset processing method provided by an embodiment of the present invention;
[0021] Figure 2 A flowchart of a data asset processing method provided by an embodiment of the present invention;
[0022] Figure 3 A schematic diagram of the structure of a data asset processing device provided by an embodiment of the present invention;
[0023] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0025] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. The purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.
[0026] Enterprise architecture is a method of translating an enterprise's business strategy into practical actions and IT (Information Technology) construction. Enterprise architecture involves the comprehensive planning and design of an enterprise's operations, processes, information, technology, data, and technology resources to ensure that the resources involved can support the realization of the business goals of the enterprise organization. Enterprise architecture includes business architecture and IT architecture, and IT architecture includes application architecture, data architecture, and technical architecture. The classification of architectural assets is usually planned and designed according to the classification method of enterprise architecture. Data assets are usually classified and classified according to the principles, specifications, and guidelines formulated by the enterprise. After the classification method is determined in advance, data assets are stored and managed.
[0027] The current construction of enterprise information systems focuses on the research of enterprise-level data asset classification methods, and lacks classification and identification methods for enterprise-level data architecture asset models.
[0028] Current enterprise-level asset classification methods mostly determine asset categories through advance planning, which results in a long asset classification and identification cycle, relatively fixed asset classification, and the inability to iterate dynamically and flexibly, which results in the rigid model of asset management tools.
[0029] During the planning, design and construction of enterprise-level data architecture, various data architecture assets will be generated. In order to facilitate data architecture asset management and promote the implementation of data architecture planning, data architecture assets can be modeled and managed. Enterprise-level data architecture asset models are designed based on enterprise needs to meet the business requirements of the enterprise, such as database tables, data items, data interfaces, data models, data services, data applications and other related data architecture asset elements.
[0030] Figure 1The flowchart of a data asset processing method provided by an embodiment of the present invention is applicable to the case of processing enterprise-level data architecture assets. The method can be executed by a data asset processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server. Figure 1 As shown, the method includes:
[0031] S110, obtaining the data assets to be processed and a preset second asset element set.
[0032] Among them, the data assets to be processed include at least one first asset element of different categories, each first asset element includes multiple first attributes, the categories of each second asset element in the second asset element set are different, and each second asset element includes multiple second attributes. The data assets to be processed may be enterprise-level architecture data assets. The first asset elements and the second asset elements may include the following categories: database tables, data items, data interfaces, data models, data services and data applications, which categories may be divided according to business needs. The second asset elements may be asset elements that have been pre-divided into categories according to business needs. The attributes of the asset elements, taking the database table as an example, may include attributes such as name, type, creation time, business time, and data processing method. Exemplarily, assuming that the data asset to be processed is represented by N, the first asset element it contains may be represented as: N = {N1, N2, N3, ..., N t}; Each first asset element includes multiple first attributes, which can be expressed as: N i ={A i1 , A i2 , ..., A ix The second asset element set is represented by M, and the second asset elements it contains can be represented as: M = {M1, M2, M3, ..., M k}; Each first asset element includes multiple second attributes, which can be expressed as: M j = {B j1 , B j2 , ..., B jy}. Wherein, t and k are both positive integers greater than or equal to 1.
[0033] S120: For each first asset element, determine a second asset element that matches the category of the first asset element from the second asset element set and determine it as a target second asset element.
[0034] The matching of the categories of the first asset element and the second asset element can be understood as the same category of the first asset element and the second asset element, for example, both are database table categories or data interface categories. Specifically, the method of determining the second asset element that matches the category of the first asset element from the second asset element set can be: first determine the category of each first asset element, and then extract the second asset element of the same category from the second asset element set as the target second asset element corresponding to the first asset element. For example, the first asset element N i The corresponding target second asset element is M j If no second asset element matching the category of the first asset element is found in the second asset element set, the first asset element is removed.
[0035] S130, comparing the multiple first attributes of the first asset element with the multiple second attributes in the target second asset element to obtain a comparison result.
[0036] Among them, the comparison results may include the following three situations: 1) multiple first attributes of the first asset element are exactly the same as multiple second attributes in the target second asset element; 2) the multiple first attributes of the first asset element are less than the multiple second attributes in the target second asset element, and the multiple first attributes of the first asset element are a subset of the multiple second attributes in the target second asset element; 3) the multiple first attributes of the first asset element are more than the multiple second attributes in the target second asset element, and the multiple second attributes of the target second asset element are a subset of the multiple first attributes in the first asset element.
[0037] Among them, the fact that the multiple first attributes of the first asset element are completely identical to the multiple second attributes in the target second asset element can be understood as follows: the number of first attributes contained in the first asset element is the same as the number of second attributes contained in the target asset element, and the information of the first attributes is also completely identical to the information of the second attributes. The fact that the multiple first attributes of the first asset element are less than the multiple second attributes in the target second asset element, and the multiple first attributes of the first asset element are a subset of the multiple second attributes in the target second asset element can be understood as follows: the first attributes contained in the first asset element are a true subset of the second attributes contained in the target second asset element, that is, the first attributes partially match the second attributes. The fact that the multiple first attributes of the first asset element are more than the multiple second attributes in the target second asset element, and the multiple second attributes of the target second asset element are a subset of the multiple first attributes in the first asset element can be understood as follows: the second attributes contained in the target second asset element are a true subset of the first asset element, that is, the second attributes partially match the first attributes.
[0038] S140: Process the first asset element according to the comparison result.
[0039] In this embodiment, the mismatch results of the multiple first attributes of the first asset element and the multiple second attributes of the target second asset element are different, and the corresponding processing methods are different.
[0040] Optionally, the first asset element may be processed according to the comparison result by: if the comparison result is that multiple first attributes of the first asset element are completely identical to multiple second attributes in the target second asset element, the first asset element is stored in the first data asset library corresponding to the category to which it belongs.
[0041] The first data asset library is pre-configured and corresponds to the category one by one, that is, one category corresponds to one first data asset library, which is convenient for the management of asset elements. In this embodiment, if the comparison result is that multiple first attributes of the first asset element are exactly the same as multiple second attributes in the target second asset element, the first asset element is directly stored in the first data asset library corresponding to the category to which it belongs.
[0042] Optionally, the first asset element may be processed according to the comparison result as follows: if the comparison result is that the multiple first attributes of the first asset element are less than the multiple second attributes in the target second asset element, and the multiple first attributes of the first asset element are a subset of the multiple second attributes in the target second asset element, then the attributes that are missing from the first asset element relative to the target second asset element are obtained and determined as missing attributes; data source information of the first asset element is obtained; the missing attributes are supplemented for the first asset element based on the data source information; and the first asset element after the missing attributes are supplemented is stored in the first data asset library corresponding to the category to which it belongs.
[0043] Among them, the missing attribute can be understood as the attribute belonging to the second attribute of the target second asset element, but not the first attribute of the first asset element. The data source information can be understood as the data source information of the first asset element. In this embodiment, the process of supplementing the missing attributes of the first asset element based on the data source information can be: analyzing the data source information of the first asset element, extracting its attributes that are not included in the first asset element, screening out the missing attributes from these attributes, and supplementing the missing attributes to the first attribute of the first asset element. After supplementing the missing attributes of the first asset element, the first asset element with the supplemented missing attributes is stored in the first data asset library corresponding to the category to which it belongs.
[0044] Optionally, after supplementing the missing attributes of the first asset element based on the data source information, the following step is further included: if the missing attributes are not supplemented based on the data source information, discarding the first asset element.
[0045] Among them, failure to complete missing attributes based on data source information can be understood as failure to extract some or all of the missing attributes from the data source information of the first asset element. At this time, subsequent processing of the first asset element is impossible, and the first asset element is discarded.
[0046] Optionally, the method of processing the first asset element according to the comparison result can be: if the comparison result is that the multiple first attributes of the first asset element are more than the multiple second attributes in the second asset element, and the multiple second attributes of the target second asset element are a subset of the multiple first attributes in the first asset element, then the first asset element is stored in the second data asset library corresponding to the category to which it belongs; based on the multiple first asset elements in the second data asset library, the second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library are updated.
[0047] Among them, the categories of the first asset elements in the second data asset library are the same. The second data asset library is pre-configured and corresponds one-to-one with the categories, that is, one category corresponds to one second data asset library. In this embodiment, each first asset element in the second data asset library satisfies the following conditions: the first attribute contained is more than the multiple second attributes in the corresponding target second asset element, and the second attribute of the corresponding target second asset element is a subset of the first attribute in the first asset element. Therefore, in this embodiment, it is necessary to determine the redundant attributes that meet the preset conditions based on each first asset element in the second data asset library, and supplement these redundant attributes to the second attributes in the target second asset element and the first attributes of the first asset element in the first data asset library.
[0048] Specifically, the method for updating the second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library based on multiple first asset elements in the second data asset library can be: obtaining attributes that are redundant with respect to the target second asset element of each first asset element in the second data asset library, and determining them as redundant attributes; determining the repetition rate of each redundant attribute in the first asset element in the second data asset library; if the repetition rate is greater than a first set value, adding the redundant attribute to the second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library.
[0049] Among them, the first set value can be preset, for example, set to 90%. The process of determining the repetition rate of each redundant attribute in the first asset element in the second data asset library can be: first determine the first asset element containing the largest number of redundant attributes in the second data asset library; for each redundant attribute of the first asset element, determine the number of first asset elements containing the redundant attribute in the second data asset library; determine the ratio between the number and the total number of first asset elements in the second data asset library, and obtain the repetition rate of the redundant attribute. After obtaining the repetition rate of each redundant attribute, add the redundant attributes with a repetition rate greater than the first set value to the second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library. So that the subsequent data assets to be processed are processed based on the updated second asset element.
[0050] Optionally, a method for obtaining redundant attributes of each first asset element in the second data asset library relative to a target second asset element may be: determining the number of first asset elements contained in the second data asset library; and when the number of first asset elements contained in the second data asset library reaches a second set value, obtaining redundant attributes of each first asset element in the second data asset library relative to the target second asset element.
[0051] The second set value can be preset. In this embodiment, when the number of first asset elements included in the second data asset library reaches the second set value, the redundant attributes of each first asset element in the second data asset library relative to the target second asset element are obtained, and then the repetition rate of each redundant attribute in the first asset element in the second data asset library is determined, which can improve the update accuracy of the second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library.
[0052] Optionally, the method for determining the repetition rate of each redundant attribute in the first asset element in the second data asset library may be: determining the number of first asset elements included in the second data asset library; for each redundant attribute, determining the number of times the redundant attribute appears in the first asset element in the second data asset library; dividing the number of occurrences by the number of first asset elements to obtain the repetition rate of the redundant attribute in the first asset element in the second data asset library.
[0053] The number of first asset elements included in the second data asset library can be understood as the total number of first asset elements included in the second data asset library. The number of times the first asset element of the redundant attribute appears in the second data asset library can be understood as the number of first asset elements in the second data asset library that include the redundant attribute. The process of dividing the number of occurrences by the number of first asset elements can be understood as: determining the ratio between the number of first asset elements that include the redundant attribute in the second data asset library and the total number of first asset elements included in the second data asset library.
[0054] The technical solution of this embodiment is to obtain the data assets to be processed and the preset second asset element set; wherein the data assets to be processed include at least one first asset element of different categories, each first asset element includes multiple first attributes, and each second asset element in the second asset element set has different categories, and each second asset element includes multiple second attributes; for each first asset element, a second asset element matching the category of the first asset element is determined from the second asset element set and determined as the target second asset element; the multiple first attributes of the first asset element are compared with the multiple second attributes in the target second asset element to obtain a comparison result; and the first asset element is processed according to the comparison result. The data asset processing method provided in the embodiment of the present invention can improve the processing efficiency of data assets and reduce the processing cycle by determining the processing method of the first asset element based on the comparison result of the attributes of the first asset element in the data asset to be processed with the attributes of the second asset element.
[0055] Based on the above embodiments, Figure 2 is a flow chart of a method for processing data assets provided by an embodiment of the present invention, such as Figure 2 As shown, the method comprises the following steps:
[0056] S210, obtaining the data assets to be processed and a preset second asset element set.
[0057] S220: For each first asset element, determine a second asset element that matches the category of the first asset element from the second asset element set and determine it as a target second asset element.
[0058] S230, compare the multiple first attributes of the first asset element with the multiple second attributes of the target second asset element to obtain a comparison result. If the comparison result is the first comparison result, execute S240; if the comparison result is the second comparison result, execute S250; if the comparison result is the third comparison result, execute S260.
[0059] S240: Store the first asset element in a first data asset library corresponding to the category to which it belongs.
[0060] S250, obtaining the missing attribute of the first asset element relative to the target second asset element, and determining it as the missing attribute; obtaining the data source information of the first asset element; supplementing the missing attribute to the first asset element based on the data source information; and storing the first asset element after supplementing the missing attribute in the first data asset library corresponding to the category to which it belongs.
[0061] S260: Store the first asset element in a second data asset library corresponding to the category to which it belongs.
[0062] S270, determining the number of first asset elements included in the second data asset library; when the number of first asset elements included in the second data asset library reaches a second set value, obtaining redundant attributes of each first asset element in the second data asset library relative to the target second asset element.
[0063] S280, determining the repetition rate of each redundant attribute in the first asset element in the second data asset library; if the repetition rate is greater than a first set value, adding the redundant attribute to the second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library.
[0064] Figure 3 is a structural diagram of a data asset processing device provided by an embodiment of the present invention, such as Figure 3 As shown, the device comprises:
[0065] The data asset acquisition module 310 is used to acquire the data asset to be processed and a preset second asset element set; wherein the data asset to be processed includes at least one first asset element of different categories, each first asset element includes a plurality of first attributes, and each second asset element in the second asset element set has a different category, and each second asset element includes a plurality of second attributes;
[0066] A target second asset element determination module 320 is used to determine, for each first asset element, from the second asset element set, a second asset element that matches the category of the first asset element and determines it as a target second asset element;
[0067] A comparison result acquisition module 330 is used to compare the multiple first attributes of the first asset element with the multiple second attributes in the target second asset element to obtain a comparison result;
[0068] The asset element processing module 340 is used to process the first asset element according to the comparison result.
[0069] Optionally, the comparison results include: multiple first attributes of the first asset element are exactly the same as multiple second attributes in the target second asset element; or, the multiple first attributes of the first asset element are less than the multiple second attributes in the target second asset element, and the multiple first attributes of the first asset element are a subset of the multiple second attributes in the target second asset element; or, the multiple first attributes of the first asset element are more than the multiple second attributes in the second asset element, and the multiple second attributes of the target second asset element are a subset of the multiple first attributes in the first asset element.
[0070] Optionally, the asset element processing module 340 is further configured to:
[0071] If the comparison result is that the multiple first attributes of the first asset element are completely identical to the multiple second attributes of the target second asset element, the first asset element is stored in the first data asset library corresponding to the category to which it belongs.
[0072] Optionally, the asset element processing module 340 is further configured to:
[0073] If the comparison result is that the multiple first attributes of the first asset element are less than the multiple second attributes in the target second asset element, and the multiple first attributes of the first asset element are a subset of the multiple second attributes in the target second asset element, then the attributes of the first asset element that are missing relative to the target second asset element are obtained and determined as missing attributes;
[0074] Obtaining data source information of the first asset element;
[0075] Supplementing missing attributes of the first asset element based on the data source information;
[0076] The first asset element after supplementing the missing attributes is stored in the first data asset library corresponding to the category to which it belongs.
[0077] Optionally, the asset element processing module 340 is further configured to:
[0078] If the missing attribute is not completed based on the data source information, the first asset element is discarded.
[0079] Optionally, the asset element processing module 340 is further configured to:
[0080] If the comparison result is that the multiple first attributes of the first asset element are more than the multiple second attributes in the target second asset element, and the multiple second attributes of the target second asset element are a subset of the multiple first attributes in the first asset element, the first asset element is stored in the second data asset library corresponding to the category to which it belongs; wherein the category of the first asset elements in the second data asset library is the same;
[0081] The second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library are updated based on the plurality of first asset elements in the second data asset library.
[0082] Optionally, the asset element processing module 340 is further configured to:
[0083] Obtaining redundant attributes of each first asset element in the second data asset library relative to the target second asset element, and determining them as redundant attributes;
[0084] determining a repetition rate of each redundant attribute in the first asset element in the second data asset library;
[0085] If the repetition rate is greater than the first set value, the redundant attributes are added to the second attribute in the target second asset element and the first attribute in the first asset element in the first data asset library.
[0086] Optionally, the asset element processing module 340 is further configured to:
[0087] determining a quantity of first asset elements contained in a second data asset repository;
[0088] When the number of first asset elements included in the second data asset library reaches a second set value, redundant attributes of each first asset element in the second data asset library relative to the target second asset element are obtained.
[0089] Optionally, the asset element processing module 340 is further configured to:
[0090] determining a quantity of first asset elements contained in a second data asset repository;
[0091] For each redundant attribute, determining a number of occurrences of the redundant attribute in the first asset element of the second data asset library;
[0092] The number of occurrences is divided by the number of first asset elements to obtain a repetition rate of the redundant attributes in the first asset elements in the second data asset library.
[0093] Optionally, the first asset element and the second asset element include the following categories: database table, data item, data interface, data model, data service and data application.
[0094] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not described in detail in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.
[0095] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0096] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0097] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0098] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for processing data assets.
[0099] In some embodiments, the method for processing data assets may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for processing data assets described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for processing data assets in any other appropriate manner (e.g., by means of firmware).
[0100] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0101] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0102] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0103] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0104] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0105] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0106] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements a method for processing data assets as provided in any embodiment of the present application.
[0107] In the process of implementation, the computer program product can be written in one or more programming languages or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
[0108] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A method for processing data assets, characterized in that: include: Acquire a data asset to be processed and a preset second asset element set; wherein the data asset to be processed includes at least one first asset element of a different category, each of the first asset elements includes a plurality of first attributes, and each of the second asset elements in the second asset element set has a different category, and each of the second asset elements includes a plurality of second attributes; For each of the first asset elements, determine a second asset element that matches the category of the first asset element from the set of the second asset elements and determine it as a target second asset element; Comparing the plurality of first attributes of the first asset element with the plurality of second attributes in the target second asset element to obtain a comparison result; The first asset element is processed according to the comparison result.
2. The method according to claim 1, characterized in that: The comparison result includes: multiple first attributes of the first asset element are exactly the same as multiple second attributes in the target second asset element; or, the multiple first attributes of the first asset element are less than the multiple second attributes in the target second asset element, and the multiple first attributes of the first asset element are a subset of the multiple second attributes in the target second asset element; or, the multiple first attributes of the first asset element are more than the multiple second attributes in the target second asset element, and the multiple second attributes of the target second asset element are a subset of the multiple first attributes in the first asset element.
3. The method according to claim 2, characterized in that Processing the first asset element according to the comparison result includes: If the comparison result is that the multiple first attributes of the first asset element are completely identical to the multiple second attributes of the target second asset element, the first asset element is stored in a first data asset library corresponding to the category to which it belongs.
4. The method according to claim 2, characterized in that: Processing the first asset element according to the comparison result includes: If the comparison result is that the multiple first attributes of the first asset element are less than the multiple second attributes in the target second asset element, and the multiple first attributes of the first asset element are a subset of the multiple second attributes in the target second asset element, then obtaining the missing attributes of the first asset element relative to the target second asset element and determining them as missing attributes; Obtaining data source information of the first asset element; Supplementing the missing attribute to the first asset element based on the data source information; The first asset element after supplementing the missing attributes is stored in the first data asset library corresponding to the category to which it belongs.
5. The method according to claim 4, characterized in that After supplementing the missing attribute to the first asset element based on the data source information, the method further includes: If the missing attribute is not completed based on the data source information, the first asset element is discarded.
6. The method according to claim 3 or 4, characterized in that: Processing the first asset element according to the comparison result includes: If the comparison result is that the multiple first attributes of the first asset element are more than the multiple second attributes in the target second asset element, and the multiple second attributes of the target second asset element are a subset of the multiple first attributes in the first asset element, then the first asset element is stored in a second data asset library corresponding to the category to which it belongs; wherein the categories of the first asset elements in the second data asset library are the same; The second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library are updated based on the plurality of first asset elements in the second data asset library.
7. The method according to claim 6, characterized in that Updating the second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library based on the plurality of first asset elements in the second data asset library includes: Obtaining redundant attributes of each first asset element in the second data asset library relative to the target second asset element, and determining them as redundant attributes; Determining a repetition rate of each of the redundant attributes in the first asset element in the second data asset library; If the repetition rate is greater than a first set value, the redundant attribute is added to the second attribute in the target second asset element and the first attribute of the first asset element in the first data asset library.
8. The method according to claim 7, characterized in that Acquiring redundant attributes of each first asset element in the second data asset library relative to the target second asset element, including: determining the number of first asset elements included in the second data asset library; When the number of first asset elements included in the second data asset library reaches a second set value, redundant attributes of each first asset element in the second data asset library relative to the target second asset element are obtained.
9. The method according to claim 7, characterized in that: Determining a repetition rate of each of the redundant attributes in the first asset element in the second data asset library includes: determining the number of first asset elements included in the second data asset library; For each redundant attribute, determining the number of times the redundant attribute appears in the first asset element in the second data asset library; The number of occurrences is divided by the number of the first asset elements to obtain a repetition rate of the redundant attribute in the first asset elements in the second data asset library.
10. The method according to claim 1, characterized in that The first asset element and the second asset element include the following categories: database table, data item, data interface, data model, data service and data application.
11. A data asset processing device, characterized in that: include: A data asset acquisition module, used to acquire a data asset to be processed and a preset second asset element set; wherein the data asset to be processed includes at least one first asset element of a different category, each of the first asset elements includes a plurality of first attributes, and each of the second asset elements in the second asset element set has a different category, and each of the second asset elements includes a plurality of second attributes; a target second asset element determination module, configured to determine, for each of the first asset elements, from the set of the second asset elements, a second asset element that matches the category of the first asset element and determine it as a target second asset element; A comparison result acquisition module, used for comparing the multiple first attributes of the first asset element with the multiple second attributes in the target second asset element to obtain a comparison result; An asset element processing module is used to process the first asset element according to the comparison result.
12. The device according to claim 11, characterized in that The comparison result includes: multiple first attributes of the first asset element are exactly the same as multiple second attributes in the target second asset element; or, the multiple first attributes of the first asset element are less than the multiple second attributes in the target second asset element, and the multiple first attributes of the first asset element are a subset of the multiple second attributes in the target second asset element; or, the multiple first attributes of the first asset element are more than the multiple second attributes in the second asset element, and the multiple second attributes of the target second asset element are a subset of the multiple first attributes in the first asset element.
13. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data asset processing method according to any one of claims 1 to 10 when executing the computer program.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for processing data assets as described in any one of claims 1-10 is implemented.
15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the data asset processing method according to any one of claims 1 to 10.