Data asset evaluation method and device, electronic equipment and storage medium

Abstract

The invention discloses a data asset evaluation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring original data associated with data assets from a plurality of heterogeneous systems; inputting the original data into a plurality of evaluation models, evaluating the data assets from different dimensions by using different evaluation models to obtain a plurality of evaluation results of the data assets, the plurality of evaluation results being in one-to-one correspondence with the plurality of evaluation models; and fusing the plurality of evaluation results to obtain a target evaluation result of the data assets. According to the invention, the technical problem of inaccurate data asset evaluation in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the fields of communications and data evaluation, and in particular to a data asset evaluation method, device, electronic device and storage medium. Background Art

[0002] With the development of the digital age, data has gradually become an important resource to drive the improvement of the communications industry. The data assets of the communications industry include not only traditional business data, but also complex data resources such as user network behavior data, location data, call records, etc. Therefore, it is necessary to evaluate data assets in a timely and accurate manner so as to use the evaluation results to improve network performance and user satisfaction.

[0003] In the related art, data assets are usually evaluated based on data quality. Due to the multi-source heterogeneity of data in the communications industry, evaluating data assets in the communications industry based on data quality will lead to a lack of systematicity in the evaluation results, thus affecting the accuracy of data asset evaluation.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a data asset evaluation method, device, electronic device and storage medium to at least solve the technical problem of inaccurate data asset evaluation in related technologies.

[0006] According to one aspect of an embodiment of the present invention, there is provided a data asset evaluation method, comprising: acquiring original data associated with data assets from multiple heterogeneous systems; inputting the original data into multiple evaluation models, evaluating the data assets from different dimensions using different evaluation models, and obtaining multiple evaluation results of the data assets, wherein the multiple evaluation results correspond one-to-one to the multiple evaluation models; and fusing the multiple evaluation results to obtain a target evaluation result of the data assets.

[0007] Optionally, the original data is input into multiple evaluation models, and different evaluation models are used to evaluate the data assets from different dimensions to obtain multiple evaluation results of the data assets, including: using any evaluation model to extract at least one feature from the original data, and constructing a target indicator corresponding to the evaluation model based on the at least one feature; using the evaluation model in combination with the target indicator to evaluate the data assets to obtain the evaluation results corresponding to the evaluation model.

[0008] Optionally, multiple evaluation models include: a data quality evaluation model, a data cost evaluation model, a computing performance evaluation model, a shared application evaluation model and a data security evaluation model, wherein the data quality evaluation model is used to evaluate the quality of data assets, the data cost evaluation model is used to evaluate the cost value of data assets, the computing performance evaluation model is used to evaluate the conversion efficiency of data assets, the shared application evaluation model is used to evaluate the application value of data assets, and the data security evaluation model is used to evaluate the security of data assets.

[0009] Optionally, in response to the evaluation model being a data quality evaluation model, at least one feature is extracted from the original data using any evaluation model, and a target indicator corresponding to the evaluation model is constructed based on the at least one feature, including: extracting table audit detail features, indicator audit detail features, and data quality fault reporting features from the original data using the data quality evaluation model, and constructing an accuracy indicator based on the table audit detail features, indicator audit detail features, and data quality fault reporting features; extracting workflow sequence timeout monitoring features from the original data using the data quality evaluation model, and constructing a timeliness indicator based on the workflow sequence timeout monitoring features; extracting table audit detail features, indicator audit detail features, and data quality fault reporting features from the original data using the data quality evaluation model, and constructing a timeliness indicator based on the workflow sequence timeout monitoring features; The data entry characteristics and table metadata completeness characteristics are extracted, and supporting indicators are constructed based on the data entry characteristics and table metadata completeness characteristics; the data mounting point characteristics and data domain characteristics are extracted from the original data using the data quality assessment model, and integrity indicators are constructed based on the data mounting point characteristics and data domain characteristics; the source data record characteristics and target data record characteristics are extracted from the original data using the data quality assessment model, and consistency indicators are constructed based on the source data record characteristics and target data record characteristics; the accuracy indicators, timeliness indicators, supporting indicators, integrity indicators and consistency indicators are weighted to obtain the target indicators corresponding to the data quality assessment model.

[0010] Optionally, in response to the evaluation model being a data cost evaluation model, at least one feature is extracted from the original data using any evaluation model, and a target indicator corresponding to the evaluation model is constructed based on the at least one feature, including: extracting project amount features and project required man-day features from the original data using the data cost evaluation model, and constructing a labor cost indicator based on the project amount features and project required man-day features; extracting data table size features, data table void rate features, and data cluster equipment price features from the original data using the data cost evaluation model, and constructing a labor cost indicator based on the data table size features, data table void rate features, and data cluster equipment price features. The data cost evaluation model is used to extract the program error rate characteristics, data operation and maintenance level characteristics, and data operation and maintenance personnel cost characteristics from the original data, and the operation and maintenance cost indicators are constructed based on the program error rate characteristics, data operation and maintenance level characteristics, and data operation and maintenance personnel cost characteristics; the data cost evaluation model is used to extract the data calculation energy consumption characteristics and the temporary table characteristics generated during the data calculation process from the original data, and the processing cost indicators are constructed based on the data calculation energy consumption characteristics and the temporary table characteristics; the labor cost indicators, equipment cost indicators, operation and maintenance cost indicators, and processing cost indicators are weighted to obtain the target indicators corresponding to the data cost evaluation model.

[0011] Optionally, in response to the evaluation model being a computing performance evaluation model, at least one feature is extracted from the original data using any evaluation model, and a target indicator corresponding to the evaluation model is constructed based on the at least one feature, including: extracting table component running speed features from the original data using the computing performance evaluation model, and constructing a computing speed indicator based on the table component running speed features; extracting data reference table features and data table calculation step features from the original data using the computing performance evaluation model, and constructing a computing complexity index based on the data reference table features and the data table calculation step features; extracting data operation status features and data failure features from the original data using the computing performance evaluation model, and constructing a stability index based on the data operation status features and the data failure features; and weighting the computing speed indicator, computing complexity index and stability index to obtain the target indicator corresponding to the computing performance evaluation model.

[0012] Optionally, in response to the evaluation model being a shared application evaluation model, at least one feature is extracted from the original data using any evaluation model, and a target indicator corresponding to the evaluation model is constructed based on the at least one feature, including: extracting features of the number of times the data is manually queried and the number of times the data is automatically executed from the original data using the shared application evaluation model, and constructing a usage indicator based on the features of the number of times the data is manually queried and the number of times the data is automatically executed; extracting data lineage features and data citation features from the original data using the shared application evaluation model, and constructing a reuse indicator based on the data lineage features and the data citation features; extracting data lineage features and data citation features from the original data using the shared application evaluation model, and constructing a reuse indicator based on the data lineage features and the data citation features; The data similarity features and the data usage source table similarity features are extracted from the original data, and the similarity index is constructed based on the data similarity features and the data usage source table similarity features; the data subject domain features are extracted from the original data using the shared application evaluation model, and the fusion index is constructed based on the data subject domain features; the data collection subscription features and the data browsing scoring features are extracted from the original data using the shared application evaluation model, and the service index is constructed based on the data collection subscription features and the data browsing scoring features; the usage index, reuse index, similarity index, fusion index and service index are weighted to obtain the target index corresponding to the shared application evaluation model.

[0013] Optionally, in response to the assessment model being a data security assessment model, at least one feature is extracted from the original data using any assessment model, and a target indicator corresponding to the assessment model is constructed based on the at least one feature, including: extracting table security grading and classification features from the original data using the data security assessment model, and constructing data collection indicators based on the table security grading and classification features; extracting data backup features from the original data using the data security assessment model, and constructing data storage indicators based on the data backup features; and weighting the data collection indicators and data storage indicators to obtain the target indicators corresponding to the data security assessment model.

[0014] According to another aspect of an embodiment of the present invention, a data asset evaluation device is also provided, including: an acquisition module, used to acquire original data associated with data assets from multiple heterogeneous systems; an evaluation module, used to input the original data into multiple evaluation models, and use different evaluation models to evaluate data assets from different dimensions to obtain multiple evaluation results of the data assets, wherein the multiple evaluation results correspond one-to-one to the multiple evaluation models; and a fusion module, used to fuse the multiple evaluation results to obtain a target evaluation result of the data asset.

[0015] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0016] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0017] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0018] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0019] According to another aspect of the embodiments of the present invention, a computer program is further provided. When the computer program is executed by a processor, the methods in the embodiments of the present invention are implemented.

[0020] In an embodiment of the present invention, raw data associated with data assets are obtained from multiple heterogeneous systems; the raw data are input into multiple evaluation models, and different evaluation models are used to evaluate data assets from different dimensions to obtain multiple evaluation results of data assets, wherein the multiple evaluation results correspond one-to-one to the multiple evaluation models; the multiple evaluation results are integrated to obtain a target evaluation result of the data asset. It is easy to notice that by adopting a multi-dimensional evaluation method, multiple different evaluation models are used to respectively reflect the characteristics of different dimensions, so that these different evaluation models are used to evaluate data assets from different dimensions, and evaluation results of different dimensions can be obtained, thereby ensuring the comprehensiveness of the target evaluation, achieving the purpose of accurately evaluating data assets, and thus achieving the technical effect of improving the accuracy of evaluation, thereby solving the technical problem of inaccurate data asset evaluation in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a data asset evaluation method according to an embodiment of the present application;

[0023] Figure 2 is a flow chart of a data asset evaluation method according to an embodiment of the present invention;

[0024] Figure 3 is a flow chart of an optional data asset evaluation method according to an embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of an optional data asset evaluation method according to an embodiment of the present invention;

[0026] Figure 5 It is a flow chart of a data asset evaluation device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] According to an embodiment of the present invention, an embodiment of a data asset evaluation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] The data asset evaluation method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal for implementing a data asset evaluation method is shown. Figure 1As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0031] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0032] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data asset evaluation method in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the IP address allocation method of the above-mentioned application program is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0034] Figure 2 is a flow chart of a data asset evaluation method according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0035] Step S202, obtaining original data associated with data assets from multiple heterogeneous systems.

[0036] The above-mentioned heterogeneous system may be a system that generates data in the communications industry. The heterogeneous system may be related to user services or to communication networks. The type of heterogeneous system is not limited here and can be determined as needed. The heterogeneous system may be a core network system, which may be used to process and control communication data flows in the communication network. Core network data may be extracted from the core network system, and network performance and user communication behavior may be evaluated through the core network data, thereby facilitating data asset value evaluation. The heterogeneous system may also be a business support system, which is a system that interacts with users for business. The business support system is used to manage user orders, bills, billing, and user relationships. Through the business support system, user data may be obtained, thereby facilitating user behavior and preferences, thereby accurately determining user portraits. The heterogeneous system may also be an operation support system, which may be responsible for network operation and maintenance, such as network monitoring, fault management, performance analysis, etc. Operation data may be extracted from the operation support system, and network health status and operation efficiency may be reflected through the operation data, thereby facilitating the evaluation of data asset security and computing efficiency. The heterogeneous system can also be a billing system, which can be used to process the billing information of the user's use of services, such as call records, data traffic usage, etc. The data in the billing system can be used to analyze user consumption patterns and network resource usage, thereby facilitating the evaluation of data assets and user behavior analysis. The content of the heterogeneous system is not limited here and can be determined as needed. Through the data integration and analysis of heterogeneous systems, data assets can be fully understood from different dimensions, providing a multi-dimensional perspective for data asset evaluation.

[0037] The above-mentioned data assets may refer to resources that are owned or controlled by the enterprise and can bring value. In the communications industry, data assets may include but are not limited to: user behavior data, network performance data, billing data and operation data, etc. Data assets are not limited here and can be determined as needed. User behavior data may include but are not limited to: call records, traffic usage, Internet behavior, etc. User behavior data is not limited here and can be determined as needed. User behavior data can be used to build user portraits and recommend personalized services. Network performance data may include but are not limited to: network delay, throughput, packet loss rate, etc. Network performance data is not limited here and can be determined as needed. Network performance data can be used for network optimization and fault diagnosis. Billing data may be data related to the billing information of users using communication services. Operation data may include but are not limited to: task monitoring data, system operation logs, etc. Operation data is not limited here and can be determined as needed. Data assets can facilitate business innovation and service optimization. Through the effective evaluation of data assets, it is easy to effectively utilize data resources, thereby improving communication services and enhancing user satisfaction.

[0038] The above-mentioned raw data may refer to unprocessed or unmodified data obtained from heterogeneous systems. Raw data is related to data assets and can be used to evaluate data assets. Raw data includes, but is not limited to: log files, transaction records, measurement data, user information, and system configuration data. The type of raw data is not limited here and can be determined as needed. Log files can be files that record the operation of communication systems or communication equipment, such as operation logs, error logs, etc. Transaction records can be detailed records of users using communication services, such as call records, data usage, etc. Measurement data can be performance indicators of network equipment, such as signal strength, network latency, etc. User information can be information personalized to the user, such as basic information, account information, and preference settings of the user. System configuration data can be data for configuring communication equipment, such as hardware configuration, software version, network architecture, etc., which helps to evaluate cost and efficiency. The accuracy and completeness of raw data determine the reliability and effectiveness of data asset evaluation. By analyzing raw data, data quality issues, computing power, and security risks can be identified, thereby providing a multi-dimensional perspective and basis for data asset evaluation.

[0039] In an optional embodiment, raw data can be extracted from heterogeneous systems through an application programming interface (API). Different types of API interfaces are determined based on the type of data assets. Then, a call request is initiated based on multiple APIs to call the raw data. After receiving the call request, the heterogeneous system queries the raw data from the corresponding database according to the request parameters of the call request and returns it.

[0040] In another optional embodiment, the original data can be obtained from a data warehouse or a data lake. Metadata information related to data assets is collected from heterogeneous systems in advance. The metadata information includes, but is not limited to, data type, data location, data update frequency, etc. The metadata information is not limited here and can be determined as needed. Then, a data extraction tool can be used to load the metadata information into the data warehouse or data lake regularly or in real time. The metadata information is converted and processed in the data warehouse or data lake, and then the processed data is loaded into the storage platform to provide data views and data sources for data asset evaluation. Implementing data management in the data warehouse or data lake, such as data quality monitoring, data security protection, data lifecycle management, etc., can ensure the integrity and security of the original data.

[0041] In another optional embodiment, a message queue can be established with a heterogeneous system, and the message queue can be used to asynchronously transmit raw data. An event listener is set in the heterogeneous system to capture data change events in the heterogeneous system in real time, such as data generation, data update, data deletion, etc. When a preset event occurs, the heterogeneous system sends the event data to the message queue. The preset event can be matched with the data asset in advance, so that the monitored event is related to the data asset, thereby obtaining the raw data related to the data asset. Then, the event data can be consumed from the message queue to obtain the raw data.

[0042] Step S204: input the original data into multiple evaluation models, use different evaluation models to evaluate the data assets from different dimensions, and obtain multiple evaluation results of the data assets.

[0043] Among them, multiple evaluation results correspond one-to-one to multiple evaluation models.

[0044] The above evaluation model can be used to evaluate the original data according to the indicators and weights. In the context of the communications industry, the evaluation of data assets can be used to measure multiple dimensions such as data asset value, quality, cost, computing efficiency, security and sharing degree. The evaluation model can convert the multi-dimensional evaluation into understandable and comparable scores or grades by collecting and analyzing relevant data and using statistics, mathematics, machine learning and other methods. The evaluation model can be used to quantify data assets and convert data assets into numerical values ​​that can be used for statistics. The evaluation model can also be used to identify data assets of different types and different degrees of importance through evaluation, so as to facilitate data storage, processing and analysis. The evaluation model can also be used to promote data quality improvement. The evaluation model can detect the accuracy, completeness, consistency and timeliness of the original data, so as to discover data quality problems and ensure data availability. The evaluation model can also be used to evaluate the security, privacy protection ability and compliance of the original data during the collection, processing and transmission process, ensure that the data assets meet the requirements and reduce the risk of data leakage. The evaluation model can also be used to optimize data application and sharing. The evaluation model can evaluate the use and sharing degree of data assets and promote the effective transmission and data integration of data. Evaluation models can also be used to calculate performance, and evaluate the efficiency of data processing and analysis through evaluation models. Evaluation models can also be used for closed-loop management and intelligent recommendations. Evaluation models can be used to make personalized and scenario-based data asset recommendations through feedback mechanisms, combined with user needs and scenario characteristics.

[0045] The above evaluation can be to evaluate data assets from different dimensions, so as to obtain multiple evaluation results of data assets. Evaluating data assets from multiple dimensions ensures the accuracy and comprehensiveness of the evaluation results. At the same time, the evaluation model incorporates industry indicators, so that the evaluation is not only based on the original data of heterogeneous systems, thereby improving the credibility of the evaluation.

[0046] The above multiple evaluation results correspond to multiple evaluation models one by one. Due to different data sources, data structures or types, different evaluation methods are required to ensure that different evaluation methods are performed based on different original data types and dimensions to improve the accuracy of the evaluation.

[0047] In an optional embodiment, the original data can be input into different evaluation models according to the data source or data type, and the data assets can be evaluated from different dimensions using the evaluation models. Alternatively, the original data can be divided according to pre-set evaluation rules and then input into different evaluation models to evaluate the data assets.

[0048] Furthermore, the evaluation model can be used to extract features from the input raw data and construct target indicators for data asset evaluation. Alternatively, the evaluation model can also learn the relationship between historical data and evaluation results, so that after the raw data is input into the evaluation model, the evaluation results can be predicted. Alternatively, each evaluation model can also evaluate the raw data based on the preset evaluation rules corresponding to the evaluation model to obtain the evaluation results.

[0049] Step S206, integrating multiple evaluation results to obtain a target evaluation result of the data asset.

[0050] The above-mentioned fusion may refer to combining the evaluation results of multiple evaluation models. In the process of data asset evaluation, data assets may involve multiple evaluation dimensions, such as data quality, cost-effectiveness, computing efficiency, security, and shared applications. Each evaluation model may focus on different data characteristics and business needs. Therefore, the fusion evaluation result is a comprehensive analysis of the evaluation scores given by these models from different dimensions to obtain a comprehensive evaluation, which in turn reflects the data assets. By integrating the evaluation results of various evaluation models, multiple dimensions of data assets can be combined to avoid the one-sidedness and errors that may be caused by single indicator evaluation, and provide a more comprehensive and accurate data asset evaluation. Based on the fused target evaluation results, it is easy to optimize the resource allocation strategy.

[0051] The above target evaluation results may be based on multiple evaluation results.

[0052] In an optional embodiment, multiple evaluation results can be fused by weighted fusion. The weights of the evaluation results of each evaluation model can be determined according to the pre-set weights, and the weights reflect the importance of the evaluation model in the overall evaluation. Then, the evaluation results of each evaluation model are multiplied by their corresponding weights to obtain multiple product results, and then the multiple product results are added to obtain the target score, and the target score or the evaluation level corresponding to the target score is used as the target evaluation result of the data asset.

[0053] In another optional embodiment, multiple evaluation results can be fused by using the analytic hierarchy process. The analytic hierarchy process includes at least one hierarchical structure, each evaluation model corresponds to a different level, and then the importance of each level is determined by expert judgment or historical data. Further, the relative weight of each evaluation model score is calculated, and finally the weighted scores of multiple evaluation models are summarized to obtain the total evaluation result. By using the analytic hierarchy process, the relative importance between evaluation models can be better handled.

[0054] In another optional embodiment, the fusion of multiple evaluation results can be achieved through machine learning. The evaluation results of multiple evaluation models are fused by training a machine learning model. The training data set of the machine learning model may include multi-source information such as historical evaluation results, business indicators, cost data, etc. The machine learning model can be a neural network, a decision tree, a support vector machine, etc. The machine learning model is not limited here and can be determined as needed. The machine learning model predicts the target evaluation result by learning the relationship between the historical scores of each evaluation model and the historical evaluation results of the data asset.

[0055] In an embodiment of the present invention, raw data associated with data assets are obtained from multiple heterogeneous systems; the raw data are input into multiple evaluation models, and different evaluation models are used to evaluate data assets from different dimensions to obtain multiple evaluation results of data assets, wherein the multiple evaluation results correspond one-to-one to the multiple evaluation models; the multiple evaluation results are integrated to obtain a target evaluation result of the data asset. It is easy to notice that by adopting a multi-dimensional evaluation method, multiple different evaluation models are used to respectively reflect the characteristics of different dimensions, so that these different evaluation models are used to evaluate data assets from different dimensions, and evaluation results of different dimensions can be obtained, thereby ensuring the comprehensiveness of the target evaluation, achieving the purpose of accurately evaluating data assets, and thus achieving the technical effect of improving the accuracy of evaluation, thereby solving the technical problem of inaccurate data asset evaluation in related technologies.

[0056] Optionally, the original data is input into multiple evaluation models, and different evaluation models are used to evaluate the data assets from different dimensions to obtain multiple evaluation results of the data assets, including: using any evaluation model to extract at least one feature from the original data, and constructing a target indicator corresponding to the evaluation model based on the at least one feature; using the evaluation model in combination with the target indicator to evaluate the data assets to obtain the evaluation results corresponding to the evaluation model.

[0057] By extracting features and building target indicators, the evaluation results can be made more refined and scientific, which helps to improve the efficiency of data asset evaluation.

[0058] In an optional embodiment, feature engineering can be used to extract at least one feature from the raw data, such as encoding, standardization, and discretization operations. Alternatively, features can be extracted from the raw data using database queries. Alternatively, features can be extracted from the raw data using statistical methods. Then, the target indicators corresponding to the evaluation model can be constructed with the help of historical data, or the target indicators can be determined separately by experts. Furthermore, the evaluation model can pre-learn the relationship between historical data and target indicators, so as to predict the input raw data and obtain evaluation results. Alternatively, the data assets can be evaluated by calculating the similarity between the raw data and the target indicators, and the evaluation results can be obtained.

[0059] Optionally, multiple evaluation models include: a data quality evaluation model, a data cost evaluation model, a computing performance evaluation model, a shared application evaluation model and a data security evaluation model, wherein the data quality evaluation model is used to evaluate the quality of data assets, the data cost evaluation model is used to evaluate the cost value of data assets, the computing performance evaluation model is used to evaluate the conversion efficiency of data assets, the shared application evaluation model is used to evaluate the application value of data assets, and the data security evaluation model is used to evaluate the security of data assets.

[0060] The multiple evaluation models may include, but are not limited to, data quality evaluation models, data cost evaluation models, computing efficiency evaluation models, shared application evaluation models, and data security evaluation models. The evaluation models are not limited here and can be determined as needed. Through multiple evaluation models, data assets can be evaluated from multiple perspectives, improving the accuracy of asset evaluation.

[0061] The above data quality assessment model can be used to assess the quality of data assets. Through the data quality assessment model, the accuracy, timeliness, consistency, completeness and support of data can be assessed. By identifying data quality issues, it is easier to perform data cleaning, data processing and other operations to improve the quality of data assets.

[0062] The above data cost assessment model can be used to assess the cost value of data assets. In the field of communications, the cost of data assets can include hardware, software, and operating costs. The content of the cost is not limited here and can be determined as needed. Through the data cost assessment model, the cost value of data assets can be assessed, which makes it easier to identify cost control points and take corresponding cost control measures, such as improving storage strategies, improving computing efficiency, and reducing manual intervention.

[0063] The above computing performance evaluation model can be used to evaluate the conversion efficiency of data assets. In the field of communications, the computing performance evaluation model can facilitate real-time analysis of network quality and user behavior analysis. The evaluation results of the model can facilitate the improvement of computing algorithms and improve conversion efficiency.

[0064] The above-mentioned shared application evaluation model can be used to evaluate the application value of data assets. Through the shared application evaluation model, the use of data assets can be determined, thereby optimizing the data call strategy. The shared application evaluation model can evaluate the application value of data assets by analyzing characteristics such as usage, reuse, similarity, integration, service rating and data scale.

[0065] The above data security assessment model can be used to assess the security of data assets. Through the data security assessment model, the security of data assets during storage, processing and transmission can be assessed. The data security assessment model can ensure the stability and credibility of data assets and effectively prevent data damage or leakage by analyzing data collection, transmission, storage, processing, exchange and destruction.

[0066] Optionally, in response to the evaluation model being a data quality evaluation model, at least one feature is extracted from the original data using any evaluation model, and a target indicator corresponding to the evaluation model is constructed based on the at least one feature, including: extracting table audit detail features, indicator audit detail features, and data quality fault reporting features from the original data using the data quality evaluation model, and constructing an accuracy indicator based on the table audit detail features, indicator audit detail features, and data quality fault reporting features; extracting workflow sequence timeout monitoring features from the original data using the data quality evaluation model, and constructing a timeliness indicator based on the workflow sequence timeout monitoring features; extracting table audit detail features, indicator audit detail features, and data quality fault reporting features from the original data using the data quality evaluation model, and constructing a timeliness indicator based on the workflow sequence timeout monitoring features; The data entry characteristics and table metadata completeness characteristics are extracted, and supporting indicators are constructed based on the data entry characteristics and table metadata completeness characteristics; the data mounting point characteristics and data domain characteristics are extracted from the original data using the data quality assessment model, and integrity indicators are constructed based on the data mounting point characteristics and data domain characteristics; the source data record characteristics and target data record characteristics are extracted from the original data using the data quality assessment model, and consistency indicators are constructed based on the source data record characteristics and target data record characteristics; the accuracy indicators, timeliness indicators, supporting indicators, integrity indicators and consistency indicators are weighted to obtain the target indicators corresponding to the data quality assessment model.

[0067] In an optional embodiment, when the evaluation model is a data quality evaluation model, the data quality evaluation model can extract table audit detail features, indicator audit detail features and data quality fault reporting features from the original data based on feature engineering or principal component analysis, and then construct accuracy indicators based on preset data accuracy rules, expert experience or statistical methods using table audit detail features, indicator audit detail features and data quality fault reporting features.

[0068] For example, the table audit details feature can be obtained by weighting the table based on whether it has audit metadata. The indicator audit details feature can be obtained by the ratio of the number of indicator audit failures to the total indicator audit volume. The data quality fault reporting feature can be obtained by the number of data fault reporting times. Then, the table audit details feature, indicator audit details feature, and data quality fault reporting feature are summed to obtain the accuracy index.

[0069] The accuracy index can be determined by the following formula:

[0070]

[0071] Where S11 is the accuracy index, Ai is the weighted value according to whether the table has audit metadata, Bi* is the total number of index audits, Bi is the number of index audit failures, W is the weight of the accuracy index, Ci is the number of data failure reports, Ci* is the total score when the number of data failure reports is zero, n is the score, * is the product operation, and ∑ is the sum operation.

[0072] The data quality assessment model can then extract workflow sequence timeout monitoring features from the original data based on feature engineering or principal component analysis, and then use the workflow sequence timeout monitoring features to construct timeliness indicators based on preset data timeliness rules, expert experience, or statistical methods.

[0073] For example, the monitoring feature of whether a workflow sequence is timed out is obtained by summing up the coefficients after classification transformation based on whether monitoring is configured, the proportion of the timeout duration to the running time, and the proportion of the number of timeouts to the total number of runs.

[0074] The timeliness index can be determined by the following formula:

[0075]

[0076] Where S12 is the timeliness index, Ai is the value corresponding to whether monitoring is configured, W is the weight of the timeliness index, Bi is the timeout duration, Bi* is the running duration, Ci* is the total number of runs, Ci is the number of timeouts, * is the product operation, and ∑ is the sum operation.

[0077] In addition, the data quality assessment model can also extract data entry characteristics and table metadata completeness characteristics from the original data based on feature engineering or principal component analysis, and then construct supporting indicators based on preset data support rules, expert experience or statistical methods using data entry characteristics and table metadata completeness characteristics.

[0078] For example, the data entry feature can be determined by whether the data is entered into the lake. The table metadata completeness feature can be obtained by averaging the normalized structure information of the entry into the lake and the structure information of the entry into the lake requirement.

[0079] The support index can be determined by the following formula:

[0080]

[0081] In the formula, S13 is the support index, Ai is the metadata of whether to enter the lake, W is the weight of the support index, Bi* is the value corresponding to the structural information required for entering the lake, Bi is the value corresponding to the structural information of entering the lake, * is the product operation, and ∑ is the sum operation.

[0082] In addition, the data quality assessment model can also extract data mounting point features and data domain features from the original data based on feature engineering or principal component analysis, and then construct integrity indicators based on preset data integrity rules, expert experience or statistical methods using data mounting point features and data domain features.

[0083] For example, the data mount point characteristics can be obtained through the domain information of the metadata information and whether there is a lake entry directory.

[0084] In addition, the data quality assessment model can also extract source data record features and target data record features from the original data based on feature engineering or principal component analysis, and then construct consistency indicators based on preset data consistency rules, expert experience or statistical methods using the source data record features and target data record features.

[0085] For example, the source-side data record characteristics and the target-side data record characteristics can be obtained through the source-side data quantity and the target-side data quantity, respectively.

[0086] Furthermore, the accuracy index, timeliness index, support index, integrity index and consistency index can be weighted by expert scoring to obtain the target index. Alternatively, the weights of the accuracy index, timeliness index, support index, integrity index and consistency index can be determined by principal component analysis, factor analysis or coefficient of variation method. Alternatively, the weights of each index can be learned and adjusted by intelligent algorithms for different usage scenarios, and the accuracy index, timeliness index, support index, integrity index and consistency index can be weighted to obtain the target index corresponding to the data quality assessment model.

[0087] This method of assessing data quality can facilitate the identification of data quality issues.

[0088] Optionally, in response to the evaluation model being a data cost evaluation model, at least one feature is extracted from the original data using any evaluation model, and a target indicator corresponding to the evaluation model is constructed based on the at least one feature, including: extracting project amount features and project required man-day features from the original data using the data cost evaluation model, and constructing a labor cost indicator based on the project amount features and project required man-day features; extracting data table size features, data table void rate features, and data cluster equipment price features from the original data using the data cost evaluation model, and constructing a labor cost indicator based on the data table size features, data table void rate features, and data cluster equipment price features. The data cost evaluation model is used to extract the program error rate characteristics, data operation and maintenance level characteristics, and data operation and maintenance personnel cost characteristics from the original data, and the operation and maintenance cost indicators are constructed based on the program error rate characteristics, data operation and maintenance level characteristics, and data operation and maintenance personnel cost characteristics; the data cost evaluation model is used to extract the data calculation energy consumption characteristics and the temporary table characteristics generated during the data calculation process from the original data, and the processing cost indicators are constructed based on the data calculation energy consumption characteristics and the temporary table characteristics; the labor cost indicators, equipment cost indicators, operation and maintenance cost indicators, and processing cost indicators are weighted to obtain the target indicators corresponding to the data cost evaluation model.

[0089] In an optional embodiment, the data cost assessment model can extract project amount characteristics and project man-day characteristics from the original data based on feature engineering or principal component analysis, and then construct a labor cost indicator based on expert experience or statistical methods using the project amount characteristics and project man-day characteristics.

[0090] For example, the project amount feature is obtained by comparing the project investment amount with the average project investment amount. The project man-day feature can be obtained by the man-day value required to complete the project evaluation.

[0091] In addition, the data cost evaluation model can also extract data table size characteristics, data table void rate characteristics and data cluster equipment price characteristics from the original data based on feature engineering or principal component analysis, and then construct equipment cost indicators based on expert experience or statistical methods using data table size characteristics, data table void rate characteristics and data cluster equipment price characteristics.

[0092] For example, the data table size feature can be obtained by normalizing the distribution of the data table storage capacity in the full storage space. The data table void rate feature can be obtained by the ratio of storage metadata to total storage metadata. The data cluster device price feature can be obtained by cluster pricing at different data storage locations.

[0093] In addition, the data cost evaluation model can also extract program error rate characteristics, data operation and maintenance level characteristics, and data operation and maintenance personnel cost characteristics from the original data based on feature engineering or principal component analysis, and then construct an operation and maintenance cost indicator based on expert experience or statistical methods using program error rate characteristics, data operation and maintenance level characteristics, and data operation and maintenance personnel cost characteristics.

[0094] For example, the program error rate feature can be obtained by the number of errors and the number of runs. The data operation and maintenance level feature can be obtained by normalizing the operation and maintenance level of the generated data program. The data operation and maintenance personnel cost feature can be obtained by the amount of the operation and maintenance project and the workload of the assessment.

[0095] In addition, the data cost evaluation model can also extract data calculation energy consumption characteristics and temporary table characteristics generated during the data calculation process from the original data based on feature engineering or principal component analysis, and then construct processing cost indicators based on expert experience or statistical methods using data calculation energy consumption characteristics and temporary table characteristics.

[0096] For example, the energy consumption characteristics of data computing can be obtained from the metadata of a single run time and the average run time. The characteristics of temporary tables can be obtained from the storage size ratio of temporary tables created during the execution of stored procedures.

[0097] Furthermore, through expert scoring, weight values ​​can be assigned to labor cost indicators, equipment cost indicators, operation and maintenance cost indicators, and processing cost indicators to obtain target indicators. Alternatively, the weights of labor cost indicators, equipment cost indicators, operation and maintenance cost indicators, and processing cost indicators can be determined separately through principal component analysis, factor analysis, or coefficient of variation method. Alternatively, the weights of each indicator can be adjusted in a timely manner according to different usage scenarios, and the labor cost indicator, equipment cost indicator, operation and maintenance cost indicator, and processing cost indicator can be weighted to obtain the target indicator corresponding to the data cost evaluation model.

[0098] The data cost evaluation model helps to improve the data processing process.

[0099] Optionally, in response to the evaluation model being a computing performance evaluation model, at least one feature is extracted from the original data using any evaluation model, and a target indicator corresponding to the evaluation model is constructed based on the at least one feature, including: extracting table component running speed features from the original data using the computing performance evaluation model, and constructing a computing speed indicator based on the table component running speed features; extracting data reference table features and data table calculation step features from the original data using the computing performance evaluation model, and constructing a computing complexity index based on the data reference table features and the data table calculation step features; extracting data operation status features and data failure features from the original data using the computing performance evaluation model, and constructing a stability index based on the data operation status features and the data failure features; and weighting the computing speed indicator, computing complexity index and stability index to obtain the target indicator corresponding to the computing performance evaluation model.

[0100] In an optional embodiment, the computing performance evaluation model can extract table component running speed characteristics from the original data based on feature engineering or principal component analysis, and then construct a computing speed index based on expert experience or statistical methods using the table component running speed characteristics.

[0101] For example, the running speed characteristics of a table component can be obtained by dividing the ratio of the running time of a table in a month to the average time of all tables in the month by the basic score.

[0102] In addition, the computing efficiency evaluation model can extract data reference table features and data table calculation step features from the original data based on feature engineering or principal component analysis, and then construct a computing complexity index based on expert experience or statistical methods using the data reference table features and data table calculation step features.

[0103] For example, the data reference table feature can be obtained by the ratio of the number of referenced tables to the average number of referenced tables. The data table calculation step feature can be obtained by weighting the number of stored procedure execution step records to the average number of stored procedure execution step records.

[0104] In addition, the computing performance evaluation model can extract data operation status characteristics and data failure characteristics from the original data based on feature engineering or principal component analysis, and then construct stability indicators based on expert experience or statistical methods using the data operation status characteristics and data failure characteristics.

[0105] For example, the data operation status characteristics can be obtained by the number of errors, the average number of table errors, whether monitoring is configured, and whether the calculation has timed out.

[0106] Furthermore, the calculation speed index, the calculation complexity index and the stability index can be assigned weight values ​​through expert scoring to obtain the target index. Alternatively, the weights of the calculation speed index, the calculation complexity index and the stability index can be determined respectively through principal component analysis, factor analysis or coefficient of variation method. Alternatively, the weights of each index can be adjusted in time according to different usage scenarios, and the calculation speed index, the calculation complexity index and the stability index can be weighted to obtain the target index corresponding to the computing performance evaluation model.

[0107] Optionally, in response to the evaluation model being a shared application evaluation model, at least one feature is extracted from the original data using any evaluation model, and a target indicator corresponding to the evaluation model is constructed based on the at least one feature, including: extracting features of the number of times the data is manually queried and the number of times the data is automatically executed from the original data using the shared application evaluation model, and constructing a usage indicator based on the features of the number of times the data is manually queried and the number of times the data is automatically executed; extracting data lineage features and data citation features from the original data using the shared application evaluation model, and constructing a reuse indicator based on the data lineage features and the data citation features; extracting data lineage features and data citation features from the original data using the shared application evaluation model, and constructing a reuse indicator based on the data lineage features and the data citation features; The data similarity features and the data usage source table similarity features are extracted from the original data, and the similarity index is constructed based on the data similarity features and the data usage source table similarity features; the data subject domain features are extracted from the original data using the shared application evaluation model, and the fusion index is constructed based on the data subject domain features; the data collection subscription features and the data browsing scoring features are extracted from the original data using the shared application evaluation model, and the service index is constructed based on the data collection subscription features and the data browsing scoring features; the usage index, reuse index, similarity index, fusion index and service index are weighted to obtain the target index corresponding to the shared application evaluation model.

[0108] In an optional embodiment, the shared application evaluation model can extract the characteristics of the number of times the data is manually queried and the number of times the data is automatically executed from the original data based on feature engineering or principal component analysis, and construct a usage indicator based on the characteristics of the number of times the data is manually queried and the number of times the data is automatically executed.

[0109] For example, the usage index can be calculated based on the weighted calculation of the idle time and busy time of the cluster, and can be calculated based on the number of times and time periods when the data table is manually queried, and the number of times and time periods when the data table is called by stored procedures.

[0110] In addition, the shared application evaluation model can extract data lineage characteristics and data citation characteristics from the original data based on feature engineering or principal component analysis, and then construct reuse indicators based on expert experience or statistical methods using data lineage characteristics and data citation characteristics.

[0111] For example, the reuse index can be calculated based on the number of post-models for each data asset, the average number of post-models, the number of stored procedure calls, and the average number of calls.

[0112] In addition, the shared application evaluation model can extract data similarity features and data usage source table similarity features from the original data based on feature engineering or principal component analysis, and then construct similarity indicators based on expert experience or statistical methods using the data similarity features and data usage source table similarity features.

[0113] For example, similarity indicators can be calculated through similarity algorithms, using the source data and fields in the data table as features for text input, and comparing the similarities and differences between texts.

[0114] In addition, the shared application evaluation model can extract data subject domain features from the original data based on feature engineering or principal component analysis, and then use the data subject domain features based on expert experience or statistical methods to construct a fusion index.

[0115] For example, service indicators can be calculated based on whether the data is included in the subject domain, the set of subject domains involved in the upstream source table corresponding to the data, and the total number of data domains.

[0116] In addition, the shared application evaluation model can extract data collection and subscription features and data browsing and scoring features from the original data based on feature engineering or principal component analysis, and then construct service indicators based on expert experience or statistical methods using the data collection and subscription features and data browsing and scoring features.

[0117] For example, service indicators can be calculated by the number of data collections, subscriptions, data views, and data scores.

[0118] Furthermore, the usage index, reuse index, similarity index, integration index and service index can be weighted by expert scoring to obtain the target index. Alternatively, the weights of the usage index, reuse index, similarity index, integration index and service index can be determined by principal component analysis, factor analysis or coefficient of variation method. Alternatively, the weights of each index can be adjusted in time according to different usage scenarios, and the usage index, reuse index, similarity index, integration index and service index can be weighted to obtain the target index corresponding to the shared application evaluation model.

[0119] By sharing application evaluation models, it is possible to promote the sharing and reuse of data assets and reduce data redundancy.

[0120] Optionally, in response to the assessment model being a data security assessment model, at least one feature is extracted from the original data using any assessment model, and a target indicator corresponding to the assessment model is constructed based on the at least one feature, including: extracting table security grading and classification features from the original data using the data security assessment model, and constructing data collection indicators based on the table security grading and classification features; extracting data backup features from the original data using the data security assessment model, and constructing data storage indicators based on the data backup features; and weighting the data collection indicators and data storage indicators to obtain the target indicators corresponding to the data security assessment model.

[0121] In an optional embodiment, the data security assessment model can extract table security grading and classification features from the original data based on feature engineering or principal component analysis, and then construct data collection indicators based on expert experience or statistical methods using the table security grading and classification features.

[0122] For example, table security classification characteristics can be obtained from table security classification, field classification and metadata information.

[0123] In addition, the data security assessment model can extract data backup features from the original data based on feature engineering or principal component analysis, and then use the data backup features to construct data storage indicators based on expert experience or statistical methods.

[0124] For example, the data storage index can be determined by the following formula:

[0125]

[0126] In the formula, S52 is the data storage index, W is the weight of the data storage index, Ai is the value corresponding to whether the table is backed up in multiple databases, Ai* is the minimum number of backup locations required, Bi is the value corresponding to whether the table metadata and data form a backup file storage and server directory, and * is a product operation.

[0127] Furthermore, the data collection index and the data storage index can be weighted by expert scoring, so as to obtain the target index corresponding to the data security assessment model. Alternatively, the weights of the data collection index and the data storage index can be determined by principal component analysis, factor analysis or coefficient of variation method. Alternatively, the weights of each index can be adjusted in time according to different usage scenarios, and the data collection index and the data storage index can be weighted to obtain the target index corresponding to the data security assessment model.

[0128] Through the application of data security assessment models, it is easy to identify data security risks and formulate data asset protection strategies.

[0129] Optionally, the above method also includes: obtaining historical evaluation results corresponding to multiple evaluation models; comparing the historical evaluation results with the evaluation results to obtain the change trends of data assets in multiple evaluation models; and displaying the change trends of multiple evaluation results and multiple evaluation models in a graphical interface.

[0130] In an optional embodiment, historical evaluation results corresponding to multiple evaluation models at historical time points may be obtained from a data warehouse or a data center platform.

[0131] Then, the historical evaluation results can be compared with the evaluation results through time series analysis methods, such as the autoregressive integrated moving average model or the state space model, to identify the temporal change trends and periodic patterns of data assets. Alternatively, after obtaining the historical evaluation results, the historical evaluation results can be grouped through a clustering algorithm to obtain multiple clusters, each cluster representing a different change pattern of data assets. The evaluation results are divided into clusters by comparing the evaluation results with the center of each cluster. Alternatively, the historical evaluation results can be compared with the evaluation results through a comparison algorithm to identify changes in data assets and predict the change trends of data assets in multiple evaluation models.

[0132] Furthermore, the evaluation results of data assets and the changing trend of the evaluation model can be displayed in the graphical interface through line charts, bar charts or heat maps, etc. The graphical interface can be configured manually or automatically drawn by a drawing tool according to the changing trend.

[0133] By comparing the assessment results with historical assessment results, the changing trends of data assets can be intuitively displayed.

[0134] Optionally, the above method also includes: in response to receiving a data asset recommendation instruction, determining multiple data assets to be recommended; performing feature extraction on the multiple data assets to be recommended to obtain associated features of the multiple data assets to be recommended, wherein the associated features are used to characterize the association relationship between the data assets to be recommended and the data asset recommendation instruction; determining recommendation scores of multiple data assets to be recommended based on the associated features of the multiple data assets to be recommended and the target evaluation results; determining a target data asset from the multiple data assets to be recommended based on the recommendation scores of the multiple data assets to be recommended; and outputting the target data asset.

[0135] The above-mentioned data asset recommendation instruction may be an instruction for obtaining data assets. The data asset recommendation instruction may be automatically triggered by the system or may be triggered by the user. Through the data asset recommendation instruction, data assets that meet the user's needs may be screened and recommended.

[0136] In an optional embodiment, after receiving the data asset recommendation instruction, the data asset recommendation instruction may be parsed to obtain a business identifier, so that a plurality of data assets to be recommended may be determined based on the business identifier.

[0137] Then, feature engineering can be used to extract features related to the data asset recommendation instructions from the data assets to be recommended, thereby screening out the required data assets. By using a hash algorithm or similarity analysis, the associated features are compared with the data identifiers in the data asset recommendation instructions, and the degree of association between multiple data assets to be recommended and the data asset recommendation instructions is calculated, thereby obtaining the associated features of multiple data assets to be recommended.

[0138] Furthermore, the recommendation score of each data asset to be recommended may be calculated based on the correlation degree of the correlation feature and the target evaluation results corresponding to the multiple data assets to be recommended. Optionally, the recommendation score may be calculated using a preset calculation rule or may be determined manually.

[0139] Then, based on the recommendation scores, the data assets to be recommended are sorted from large to small, and the first N data assets are selected as target data assets. N can be a positive integer, and the size of N is not limited and can be determined according to actual needs. Alternatively, a hierarchical clustering algorithm can be used to divide multiple data to be recommended into multiple clusters based on the recommendation scores, so that the cluster corresponding to the corresponding recommendation score can be determined, and the data to be recommended in the cluster can be regarded as the target data asset.

[0140] Then, the selected target data assets and their recommended scores can be output through a graphical interface or report form, which is convenient for users to quickly view and select. Alternatively, the target data assets can be pushed through SMS or email.

[0141] Through feature extraction and recommendation score calculation, it can accurately match user needs. It is suitable for scenarios such as data asset recommendation, product development, and data service customization. It provides users with personalized data asset recommendations and improves data asset utilization efficiency and user satisfaction.

[0142] Figure 3 is a flow chart of an optional data asset evaluation method according to an embodiment of the present application, such as Figure 2 As shown, the method includes:

[0143] Step S302: collecting original data associated with data assets from multiple heterogeneous systems.

[0144] Step S304, pre-processing the original data.

[0145] Step S306: input the preprocessed raw data into the data evaluation model, evaluate the data assets, and obtain evaluation results.

[0146] Step S308: visualize the evaluation results.

[0147] Step S310: recommend target data assets based on the evaluation results.

[0148] Figure 4 is a schematic diagram of an optional data asset evaluation method according to an embodiment of the present application, such as Figure 4 As shown, the method includes:

[0149] After the original data is input into the evaluation model, the feature vector is extracted. The evaluation model includes data quality evaluation model, data cost evaluation model, computing efficiency evaluation model, shared application evaluation model, and data security evaluation model. The heat score of the data asset is calculated based on the feature vector.

[0150] At the same time, metadata is constructed based on the original data. Then, based on the metadata, the feature vectors of data domain information and data lineage relationship are determined. Thus, based on the feature vectors of data domain information and data lineage relationship, business association feature vectors are constructed to obtain the three-dimensional association score of data assets.

[0151] Furthermore, the recommendation score of the data asset is calculated based on the heat score and the relevance score, thereby determining the recommended target data asset. In addition, the parameters for calculating the recommendation score can also be adjusted based on the target data asset.

[0152] According to an embodiment of the present invention, an embodiment of a data asset evaluation device is provided. It should be noted that the device can be used to execute the above-mentioned data asset evaluation method. The specific implementation scheme and application scenario in this embodiment are the same as those in the above-mentioned embodiment and will not be repeated here.

[0153] Figure 5 is a schematic diagram of a data asset evaluation device according to an embodiment of the present application, such as Figure 4 As shown, the device includes the following:

[0154] An acquisition module 50, used to acquire original data associated with data assets from multiple heterogeneous systems;

[0155] An evaluation module 52, used to input the original data into multiple evaluation models, evaluate the data assets from different dimensions using different evaluation models, and obtain multiple evaluation results of the data assets, wherein the multiple evaluation results correspond to the multiple evaluation models one by one;

[0156] The fusion module 54 is used to fuse multiple evaluation results to obtain a target evaluation result of the data asset.

[0157] Optionally, the evaluation module is also used to extract at least one feature from the original data using any evaluation model, and construct a target indicator corresponding to the evaluation model based on the at least one feature; evaluate the data assets using the evaluation model in combination with the target indicator to obtain an evaluation result corresponding to the evaluation model.

[0158] Optionally, multiple evaluation models include: a data quality evaluation model, a data cost evaluation model, a computing performance evaluation model, a shared application evaluation model and a data security evaluation model, wherein the data quality evaluation model is used to evaluate the quality of data assets, the data cost evaluation model is used to evaluate the cost value of data assets, the computing performance evaluation model is used to evaluate the conversion efficiency of data assets, the shared application evaluation model is used to evaluate the application value of data assets, and the data security evaluation model is used to evaluate the security of data assets.

[0159] Optionally, the evaluation module is also used to use the data quality evaluation model to extract table audit detail features, indicator audit detail features and data quality fault reporting features from the original data, and construct accuracy indicators based on the table audit detail features, indicator audit detail features and data quality fault reporting features; use the data quality evaluation model to extract whether the workflow sequence has timed out monitoring features from the original data, and construct timeliness indicators based on whether the workflow sequence has timed out monitoring features; use the data quality evaluation model to extract data entry lake features and table metadata completeness features from the original data, and construct supporting indicators based on the data entry lake features and table metadata completeness features; use the data quality evaluation model to extract data mount point features and data domain features from the original data, and construct integrity indicators based on the data mount point features and data domain features; use the data quality evaluation model to extract source data record features and target data record features from the original data, and construct consistency indicators based on the source data record features and target data record features; weight the accuracy indicators, timeliness indicators, supporting indicators, integrity indicators and consistency indicators to obtain the target indicators corresponding to the data quality evaluation model.

[0160] Optionally, the evaluation module is also used to use the data cost evaluation model to extract project amount characteristics and project required man-day characteristics from the original data, and construct a labor cost indicator based on the project amount characteristics and project required man-day characteristics; use the data cost evaluation model to extract data table size characteristics, data table void rate characteristics and data cluster equipment price characteristics from the original data, and construct an equipment cost indicator based on the data table size characteristics, data table void rate characteristics and data cluster equipment price characteristics; use the data cost evaluation model to extract program error rate characteristics, data operation and maintenance level characteristics and data operation and maintenance personnel cost characteristics from the original data, and construct an operation and maintenance cost indicator based on the program error rate characteristics, data operation and maintenance level characteristics and data operation and maintenance personnel cost characteristics; use the data cost evaluation model to extract data calculation energy consumption characteristics and temporary table characteristics generated during the data calculation process from the original data, and construct a processing cost indicator based on the data calculation energy consumption characteristics and temporary table characteristics; weighted processing is performed on the labor cost indicator, equipment cost indicator, operation and maintenance cost indicator and processing cost indicator to obtain the target indicator corresponding to the data cost evaluation model.

[0161] Optionally, the evaluation module is also used to use the computing efficiency evaluation model to extract table component running speed characteristics from the original data, and construct a computing speed index based on the table component running speed characteristics; use the computing efficiency evaluation model to extract data reference table characteristics and data table calculation step characteristics from the original data, and construct a computing complexity index based on the data reference table characteristics and data table calculation step characteristics; use the computing efficiency evaluation model to extract data operation status characteristics and data failure characteristics from the original data, and construct a stability index based on the data operation status characteristics and data failure characteristics; perform weighted processing on the computing speed index, computing complexity index and stability index to obtain the target indicator corresponding to the computing efficiency evaluation model.

[0162] Optionally, the evaluation module is also used to use the shared application evaluation model to extract the characteristics of the number of times the data is manually queried and the number of times the data is automatically executed from the original data, and construct a usage indicator based on the characteristics of the number of times the data is manually queried and the number of times the data is automatically executed; use the shared application evaluation model to extract the data lineage characteristics and the data citation characteristics from the original data, and construct a reuse indicator based on the data lineage characteristics and the data citation characteristics; use the shared application evaluation model to extract the data similarity characteristics and the data usage source table similarity characteristics from the original data, and construct a similarity indicator based on the data similarity characteristics and the data usage source table similarity characteristics; use the shared application evaluation model to extract the data subject domain characteristics from the original data, and construct a fusion index based on the data subject domain characteristics; use the shared application evaluation model to extract the data collection subscription characteristics and the data browsing rating characteristics from the original data, and construct a service indicator based on the data collection subscription characteristics and the data browsing rating characteristics; weighted processing is performed on the usage indicator, reuse indicator, similarity indicator, fusion indicator and service indicator to obtain the target indicator corresponding to the shared application evaluation model.

[0163] Optionally, the evaluation module is also used to extract table security grading and classification features from the original data using the data security evaluation model, and construct data collection indicators based on the table security grading and classification features; extract data backup features from the original data using the data security evaluation model, and construct data storage indicators based on the data backup features; perform weighted processing on the data collection indicators and the data storage indicators to obtain the target indicators corresponding to the data security evaluation model.

[0164] Optionally, the device further includes a comparison module. The comparison module is used to obtain historical evaluation results corresponding to multiple evaluation models; compare the historical evaluation results with the evaluation results to obtain the change trends of data assets in multiple evaluation models; and display the change trends of multiple evaluation results and multiple evaluation models in a graphical interface.

[0165] Optionally, the device further includes a recommendation module. The recommendation module is used to determine a plurality of data assets to be recommended in response to receiving a data asset recommendation instruction; extract features of the plurality of data assets to be recommended to obtain associated features of the plurality of data assets to be recommended, wherein the associated features are used to characterize the associated relationship between the data assets to be recommended and the data asset recommendation instruction; determine recommendation scores of the plurality of data assets to be recommended based on the associated features of the plurality of data assets to be recommended and the target evaluation results; determine a target data asset from the plurality of data assets to be recommended based on the recommendation scores of the plurality of data assets to be recommended; and output the target data asset.

[0166] An embodiment of the present application further provides an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention when running.

[0167] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0168] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0169] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0170] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0171] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0174] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0175] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0176] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data asset evaluation method, characterized in that: include: Acquire raw data associated with data assets from multiple heterogeneous systems; Inputting the original data into multiple evaluation models, using different evaluation models to evaluate the data assets from different dimensions, and obtaining multiple evaluation results of the data assets, wherein the multiple evaluation results correspond to the multiple evaluation models one by one; The multiple evaluation results are integrated to obtain a target evaluation result of the data asset.

2. The method according to claim 1, characterized in that The raw data is input into a plurality of evaluation models, and the data assets are evaluated from different dimensions using different evaluation models to obtain a plurality of evaluation results of the data assets, including: Extracting at least one feature from the raw data using any evaluation model, and constructing a target indicator corresponding to the evaluation model based on the at least one feature; The data assets are evaluated using the evaluation model in combination with the target indicators to obtain an evaluation result corresponding to the evaluation model.

3. The method according to claim 2, characterized in that The multiple evaluation models include: a data quality evaluation model, a data cost evaluation model, a computing performance evaluation model, a shared application evaluation model and a data security evaluation model, wherein the data quality evaluation model is used to evaluate the quality of the data assets, the data cost evaluation model is used to evaluate the cost value of the data assets, the computing performance evaluation model is used to evaluate the conversion efficiency of the data assets, the shared application evaluation model is used to evaluate the application value of the data assets, and the data security evaluation model is used to evaluate the security of the data assets.

4. The method according to claim 3, characterized in that In response to the evaluation model being the data quality evaluation model, extracting at least one feature from the original data using any evaluation model, and constructing a target indicator corresponding to the evaluation model based on the at least one feature, includes: Extracting table audit detail features, indicator audit detail features and data quality fault reporting features from the original data using the data quality assessment model, and constructing an accuracy index based on the table audit detail features, the indicator audit detail features and the data quality fault reporting features; Extracting a workflow sequence timeout monitoring feature from the raw data using the data quality assessment model, and constructing a timeliness index based on the workflow sequence timeout monitoring feature; Extracting data entry characteristics and table metadata completeness characteristics from the original data using the data quality assessment model, and constructing supporting indicators based on the data entry characteristics and the table metadata completeness characteristics; Extracting data mount point features and data domain features from the original data using the data quality assessment model, and constructing integrity indicators based on the data mount point features and the data domain features; Extracting source-end data record features and target-end data record features from the original data using the data quality assessment model, and constructing consistency indicators based on the source-end data record features and the target-end data record features; The accuracy index, the timeliness index, the support index, the integrity index and the consistency index are weighted to obtain the target index corresponding to the data quality assessment model.

5. The method according to claim 3, characterized in that: In response to the evaluation model being the data cost evaluation model, extracting at least one feature from the original data using any evaluation model, and constructing a target indicator corresponding to the evaluation model based on the at least one feature, includes: The data cost evaluation model is used to extract project amount characteristics and project required man-day characteristics from the original data, and a labor cost index is constructed based on the project amount characteristics and the project required man-day characteristics; Extracting data table size features, data table void rate features, and data cluster device price features from the original data using the data cost evaluation model, and constructing a device cost index based on the data table size features, the data table void rate features, and the data cluster device price features; Extracting program error rate characteristics, data operation and maintenance level characteristics, and data operation and maintenance personnel cost characteristics from the original data using the data cost evaluation model, and constructing an operation and maintenance cost indicator based on the program error rate characteristics, the data operation and maintenance level characteristics, and the data operation and maintenance personnel cost characteristics; Extracting data calculation energy consumption characteristics and temporary table characteristics generated during data calculation from the original data using the data cost evaluation model, and constructing a processing cost index based on the data calculation energy consumption characteristics and the temporary table characteristics; The labor cost index, the equipment cost index, the operation and maintenance cost index and the processing cost index are weighted to obtain a target index corresponding to the data cost evaluation model.

6. The method according to claim 3, characterized in that In response to the evaluation model being the computing performance evaluation model, extracting at least one feature from the original data using any evaluation model, and constructing a target indicator corresponding to the evaluation model based on the at least one feature, includes: Extracting table component operation speed characteristics from the original data using the computing efficiency evaluation model, and constructing a computing speed index based on the table component operation speed characteristics; Extracting data reference table features and data table calculation step features from the original data using the computing efficiency evaluation model, and constructing a computing complexity index based on the data reference table features and the data table calculation step features; Extracting data operation status characteristics and data fault characteristics from the original data using the computing performance evaluation model, and constructing stability indicators based on the data operation status characteristics and the data fault characteristics; The calculation speed index, the calculation complexity index and the stability index are weighted to obtain a target index corresponding to the calculation efficiency evaluation model.

7. The method according to claim 3, characterized in that In response to the evaluation model being the shared application evaluation model, extracting at least one feature from the original data using any evaluation model, and constructing a target indicator corresponding to the evaluation model based on the at least one feature, includes: Extracting the number of times the data is manually queried and the number of times the data is automatically executed from the original data using the shared application evaluation model, and constructing a usage index based on the number of times the data is manually queried and the number of times the data is automatically executed; Extracting data lineage characteristics and data citation characteristics from the original data using the shared application evaluation model, and constructing a reuse index based on the data lineage characteristics and the data citation characteristics; Extracting data similarity features and data usage source table similarity features from the original data using the shared application evaluation model, and constructing a similarity index based on the data similarity features and the data usage source table similarity features; Extracting data subject domain features from the original data using the shared application evaluation model, and constructing a fusion degree index based on the data subject domain features; Extracting data collection subscription features and data browsing scoring features from the original data using the shared application evaluation model, and constructing service indicators based on the data collection subscription features and the data browsing scoring features; The usage index, the reuse index, the similarity index, the integration index and the service index are weighted to obtain a target index corresponding to the shared application evaluation model.

8. The method according to claim 3, characterized in that In response to the assessment model being the data security assessment model, extracting at least one feature from the original data using any one of the assessment models, and constructing a target indicator corresponding to the assessment model based on the at least one feature, includes: Extracting table security classification features from the original data using the data security assessment model, and constructing data collection indicators based on the table security classification features; Extracting data backup features from the original data using the data security assessment model, and constructing data storage indicators based on the data backup features; The data collection index and the data storage index are weighted to obtain the target index corresponding to the data security assessment model.

9. A data asset evaluation device, characterized in that: include: An acquisition module, used to obtain raw data associated with data assets from multiple heterogeneous systems; An evaluation module, used to input the raw data into a plurality of evaluation models, evaluate the data assets from different dimensions using different evaluation models, and obtain a plurality of evaluation results of the data assets, wherein the plurality of evaluation results correspond one-to-one to the plurality of evaluation models; The fusion module is used to fuse the multiple evaluation results to obtain the target evaluation result of the data asset.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 8.

Images