Tenant health degree assessment method and device, storage medium and program product

Abstract

The invention provides a tenant health degree assessment method and device, a storage medium and a program product, and belongs to the technical field of big data platforms, and the method comprises the steps: analyzing the health degree of a tenant from a storage and calculation resource dimension and a data asset dimension according to the big data platform use record data of the tenant, and obtaining the health degree of the tenant; obtaining a storage and calculation resource health degree and a data asset health degree; and determining the health degree of the tenant based on the storage and calculation resource health degree and the data asset health degree. According to the tenant health degree evaluation method and device, the storage medium and the program product provided by the invention, the big data platform use data of the tenants is comprehensively analyzed from the storage and calculation resource dimension and the data asset dimension, and the tenant health degree is accurately evaluated based on a comprehensive evaluation index system; reliable tenant operation notification and risk early warning are issued according to the evaluation result of the tenant health degree, a foundation is laid for healthy and stable operation of a big data platform, and targeted guidance is provided for tenant operation.

Description

Technical Field

[0001] The present invention relates to the technical field of big data platforms, and in particular to a tenant health assessment method, device, storage medium and program product. Background Art

[0002] Enterprise-level big data platforms mostly provide services to various units in a tenant model. The health of a tenant is an indicator that describes whether the tenant can operate the big data platform normally.

[0003] At present, the assessment of tenant health is often obtained through subjective experience judgment of abnormal performance data.

[0004] However, evaluating health based on subjective experience lacks a comprehensive and integrated tenant health measurement standard and evaluation system; at the same time, it can only be evaluated when abnormal performance data appears after a platform failure, which is lagging. Summary of the invention

[0005] The present invention provides a tenant health assessment method, device, storage medium and program product, which are used to solve the defects of the prior art in assessing health based on subjective experience, and realize accurate assessment of tenant health based on a comprehensive assessment index system.

[0006] The present invention provides a tenant health assessment method, comprising: According to the tenant's big data platform usage record data, the tenant's health is analyzed from the storage and computing resource dimension and the data asset dimension to obtain the storage and computing resource health and data asset health; The health of the tenant is determined based on the health of the storage and computing resources and the health of the data assets.

[0007] According to a tenant health assessment method provided by the present invention, the storage and computing resource dimension includes a computing resource utilization index and a storage resource utilization index; The health of the storage and computing resources is determined based on the following methods: Determine computing resource usage based on the tenant's daily average memory usage and pre-allocated memory quota; Determine the storage resource utilization rate based on the storage space occupied by the tenant and the pre-allocated space quota; The health of the storage and computing resources is determined based on a weighted accumulation result of the computing resource utilization rate and the storage resource utilization rate.

[0008] According to a tenant health evaluation method provided by the present invention, the health of the storage and computing resources is determined based on the result of weighted accumulation of the computing resource usage rate and the storage resource usage rate, including: Determine a computing resource deduction value based on the computing resource usage interval in which the computing resource usage falls; Determining a storage resource deduction value based on the storage resource usage interval in which the storage resource usage falls; The health of the storage and computing resources is determined based on a result of weighted accumulation of the computing resource deduction value and the storage resource deduction value.

[0009] According to a tenant health assessment method provided by the present invention, the data asset dimension includes a model construction sub-dimension, a task operation sub-dimension and an application construction sub-dimension; The data asset health is determined based on the following: According to the big data platform usage record data of the tenant, the data asset health of the tenant is analyzed from the model construction sub-dimension, the task operation sub-dimension and the application construction sub-dimension to obtain the model construction health, the task operation health and the application construction health; The data asset health is determined based on a weighted accumulation result of the model construction health, the task operation health, and the application construction health.

[0010] According to a tenant health assessment method provided by the present invention, the model construction sub-dimensions include a model standardization rate index, a model availability rate index, a data asset efficiency index, and a data retention integrity rate index; The model building health is based on the following: Determine the model standardization rate based on the number of irregular formal models and the number of formal models; Determining model availability based on the number of formal models and the total number of full physical tables; Determine the data asset effectiveness based on the number of referenced models and the number of formal models; Determine the data retention integrity rate based on the number of models with complete storage cycles and the number of formal models; The health of the model construction is determined based on the weighted accumulation of the model standardization rate, the model availability rate, the data asset efficiency rate and the data retention integrity rate.

[0011] According to a tenant health assessment method provided by the present invention, the task operation sub-dimensions include a task timeliness rate index, a task first-time success rate index, a task stability rate index, and a task standardization rate index; The task running health is obtained based on the following method: Determine the task on-time rate based on the number of task cycles whose execution time is lower than the task benchmark completion time and the total number of task cycles; Determine a task success rate based on the number of task cycles that succeed once and the total number of task cycles; Determining a task stability rate based on the number of unstable task cycles and the total number of task cycles; Determining a task standardization rate based on the number of irregular task cycles and the total number of task cycles; The task operation health is determined based on the weighted accumulation result of the task timeliness rate, the task one-time success rate, the task stability rate and the task standard rate.

[0012] According to a tenant health assessment method provided by the present invention, the application establishment sub-dimension includes a low-value data application rate indicator and a low-efficiency data application rate indicator; The application construction health is obtained based on the following method: Determine the low-value data application rate based on the number of low-value data applications and the total number of data applications; Determining an inefficient data application rate based on the number of inefficient data applications and the total number of data applications; The task operation health is determined based on a weighted accumulation result of the low-value data application rate and the low-efficiency data application rate.

[0013] The present invention also provides a tenant health assessment device, comprising: A dimension analysis module is used to analyze the health of the tenant from the dimensions of storage and computing resources and data assets according to the tenant's big data platform usage record data, and obtain the storage and computing resource health and data asset health; The health determination module is used to determine the health of the tenant based on the health of the storage and computing resources and the health of the data assets.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the tenant health assessment method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the tenant health assessment method as described above is implemented.

[0016] The tenant health assessment method, device, storage medium and program product provided by the present invention comprehensively analyze the tenant's big data platform usage data from the storage and computing resource dimensions and the data asset dimensions, accurately assess the tenant health based on a comprehensive assessment indicator system, and issue reliable tenant operation notifications and risk warnings based on the tenant health assessment results, thereby laying the foundation for the healthy and stable operation of the big data platform and providing targeted guidance for tenant operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the tenant health assessment method provided by the present invention.

[0019] Figure 2 It is a structural schematic diagram of the tenant health assessment device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Combine the following Figure 1-Figure 2 The tenant health assessment method, device, storage medium and program product of the present invention are described.

[0022] Enterprise-level big data platforms mostly provide services to various units in a tenant model, enabling multi-tenant resource sharing while ensuring resource isolation and security. Platform resources integrate functions such as data collection, storage, processing, analysis, and visualization, covering basic resources (such as storage and computing resources) and tasks, models, and applications running on the platform. As users or owners of platform resources, tenants are the basic units for resource, personnel, and application management and isolation. Under the multi-tenant architecture, each tenant has an independent data space, permission settings, and application management environment. Administrators can flexibly configure access rights for different tenants to ensure that only authorized users can access specific data, applications, and functions. In order to standardize tenant behavior, efficiently use resources, improve tenant satisfaction, and promote the sustainable development of the platform, the platform operator should carry out systematic tenant operation and management.

[0023] On the one hand, at the current stage, tenant operations of big data platforms are mostly concentrated in one or several aspects. The health of tenants is determined based on experience to determine the degree of impact of abnormal performance data on the normal operation of tenants. There is a lack of a comprehensive and integrated tenant health measurement standard and evaluation system, which makes it difficult to support the systematic development of platform tenant operation management.

[0024] On the other hand, the existing big data platform tenant operations are mostly based on post-failure operations. This operation mode cannot detect irregularities in tenant resource use and application construction in advance, foresee potential failures in advance, and eliminate failures before they occur. Post-failure operations make it difficult to improve tenant satisfaction, irregular application construction, and insufficient utilization of tenant resource efficiency, and tenant operations lack a strong grip.

[0025] Figure 1 is a flow chart of the tenant health assessment method provided by the present invention, such as Figure 1 As shown, the tenant health assessment method includes but is not limited to the following steps 101 and 102.

[0026] It should be noted that the executors of the tenant health assessment method provided by the present invention may be the data management party, data operation and maintenance party, data analysis party, business department, etc. of the big data platform.

[0027] Step 101: Based on the tenant's big data platform usage record data, analyze the tenant's health from the storage and computing resource dimension and the data asset dimension to obtain the storage and computing resource health and data asset health.

[0028] Before comprehensively evaluating the health of big data platform tenants, we collect the usage habits and current status of platform tenants, combine the platform characteristics and tenants' daily needs, and build a tenant health evaluation index system with two evaluation dimensions. The tenant health evaluation index system generally includes three levels: dimension, sub-dimension and specific evaluation indicators.

[0029] Specifically, the tenant health evaluation index system includes the storage and computing resource dimension and the data asset dimension. After obtaining the tenant's usage record data on the big data platform, the tenant's health is analyzed from the storage and computing resource dimension and the data asset dimension to obtain the tenant's storage and computing resource health and data asset health.

[0030] Computing resources include computing resources and storage resources. Computing resources mainly refer to the hardware and software resources used to process data and perform computing tasks, such as servers, processors, CPUs, etc. Storage resources are devices or systems used to store data, such as hard disks, distributed storage systems, cloud storage services, etc., which are responsible for storing data and providing data read and write operations.

[0031] Data assets are the resources that tenants own in the big data platform, related to model building, task running, and application establishment.

[0032] In one embodiment, the tenant's big data platform usage record data is obtained based on the following steps: Obtain the daily usage data of each tenant in the big data platform and generate a data table to be evaluated; Determine a metadata table for recording the metadata composition of the data table to be evaluated; Determine the actual measured value corresponding to each specific evaluation indicator of the platform tenant under the tenant health evaluation indicator system according to the platform tenant behavior attributes of the metadata table; Based on the measured values ​​corresponding to the specific evaluation indicators, the big data platform usage record data of the tenant is determined.

[0033] Optionally, the metadata table includes but is not limited to tenant name, model code, model name, storage method, storage strategy, latest data batch, subsequent dependencies, task execution status, average task running time, application click rate, average application query time and other data.

[0034] Step 102: Determine the health of the tenant based on the health of the storage and computing resources and the health of the data assets.

[0035] Furthermore, the health of the tenant's big data platform usage record data is analyzed from the storage and computing resource dimensions and the data asset dimensions. After obtaining the storage and computing resource health and the data asset health, the health of the tenant is determined based on the sum of the storage and computing resource health and the data asset health.

[0036] Optionally, the health of the tenant may also be determined based on the weighted sum of the health of storage and computing resources and the health of data assets.

[0037] For example, the expression for determining the tenant's health based on the storage and computing resource health and the data asset health is as follows: ; (1)

[0038] in, For the health of tenants; To store and calculate the health of resources; The health of data assets.

[0039] When the tenant's health is low, it means that there is a risk that the tenant cannot operate normally on the big data platform. At this time, the tenant can be reminded by sending messages, pop-up windows, etc., to provide targeted guidance for the tenant's operations. At the same time, the background can also be used to schedule computing and storage resources in advance to avoid actual failures and maintain the healthy and stable operation of the big data platform.

[0040] The tenant health assessment method provided by the present invention comprehensively analyzes the tenant's big data platform usage data from the storage and computing resource dimensions and the data asset dimensions, accurately assesses the tenant's health based on a comprehensive assessment indicator system, and timely issues reliable tenant operation notifications and risk warnings based on the tenant health assessment results, thereby laying the foundation for the healthy and stable operation of the big data platform and providing targeted guidance for tenant operations.

[0041] Based on the above embodiment, as an optional embodiment, the storage and computing resource dimension includes a computing resource utilization index and a storage resource utilization index; The health of the storage and computing resources is determined based on the following methods: Determine computing resource usage based on the tenant's daily average memory usage and pre-allocated memory quota; Determine the storage resource utilization rate based on the storage space occupied by the tenant and the pre-allocated space quota; The health of the storage and computing resources is determined based on a weighted accumulation result of the computing resource utilization rate and the storage resource utilization rate.

[0042] It can be understood that in the tenant health evaluation index system, the storage and computing resource dimension includes two specific evaluation indicators: computing resource utilization index and storage resource utilization index. At this time, there is no sub-dimension in the storage and computing resource dimension.

[0043] The computing resource utilization rate indicator is the ratio of the actual computing resource usage to the total computing resource capacity, which reflects the efficiency and demand of platform tenants when using computing resources. The higher the computing resource utilization rate, the greater the computing demand of the tenant, and it may be necessary to increase resources or optimize resource allocation; conversely, it means that computing resources are not fully utilized and resource integration or adjustment is needed.

[0044] The storage resource utilization index is used to evaluate the proportion of storage resources that have been occupied, reflecting the data storage needs of platform tenants, and is also the basis for deciding whether to expand storage capacity or optimize storage efficiency. If the utilization rate is too high, it means that storage resources are insufficient, which will affect data storage and access efficiency; if the utilization rate is too low, it means that storage resources are excessive, and you need to consider adjusting storage resource allocation or adopting a more efficient storage solution.

[0045] The pre-allocated memory quota is a certain amount of computing resources that are pre-allocated to each tenant when using the big data platform; the pre-allocated space quota is a certain amount of space resources that are pre-allocated to each tenant when using the big data platform.

[0046] Specifically, when the storage and computing resource dimension includes two specific evaluation indicators, namely the computing resource utilization rate indicator and the storage resource utilization rate indicator, the tenant's storage and computing resource usage within a certain period is monitored to obtain the tenant's daily average memory usage and storage space occupancy quota, and then the computing resource utilization rate is determined based on the tenant's daily average memory usage and the pre-allocated memory quota, and the storage resource utilization rate is determined based on the tenant's storage space occupancy quota and the pre-allocated space quota.

[0047] Furthermore, the computing resource utilization index and the storage resource utilization index are used for the first weight of weighted accumulation in the storage resource dimension according to the hierarchical analysis method in advance, and the sum of the first weights of the computing resource utilization index and the storage resource utilization index is 1.

[0048] The usage of storage and computing resources is the basis for the stable operation of tenants and directly affects the operation of tasks. Therefore, the binary calculation (valid / invalid) in the hierarchical analysis method is used for evaluation, and the first weights of the computing resource utilization index and the storage resource utilization index are both set to 0.5.

[0049] Furthermore, when storage resources are insufficient, the scheduled task will fail to run directly; when storage resources are used at a high rate but the resource utilization rate has not reached 100%, it will not affect the task operation. If computing resources are insufficient, it will directly cause the scheduled task to run for a long time. At this time, although the execution of the task will be affected to a certain extent, it will not directly cause the task to fail. When the resource utilization rate reaches the predetermined threshold, the storage and computing resource health will be deducted from the corresponding score; when the computing resource utilization rate is less than 40%, it is considered that the resources are not fully utilized, and the health will be deducted from the corresponding score.

[0050] In one embodiment, the computing resource usage is determined based on the following expressions (2) and (3): ; (2)

[0051] ; (3)

[0052] in, To calculate resource utilization; is the daily average memory usage, To preallocate memory quota.

[0053] Optionally, the memory usage per second is obtained by the big data platform refreshing the panel at a refresh interval of once per second to monitor the memory resource usage of a single tenant for one day.

[0054] In one embodiment, the storage resource usage is determined based on the following expression (4): ; (4)

[0055] in, It is the storage resource usage.

[0056] In one embodiment, determining the health of the storage resource based on a weighted accumulation result of the computing resource usage rate and the storage resource usage rate includes: Based on the computing resource usage interval in which the computing resource usage is located, determine the computing resource deduction value; based on the storage resource usage interval in which the storage resource usage is located, determine the storage resource deduction value; based on the result of weighted accumulation of the computing resource deduction value and the storage resource deduction value, determine the health of the storage and computing resources.

[0057] Among them, the computing resource deduction value and the computing resource usage rate interval are pre-associated in the computing resource deduction rule; the storage resource deduction value and the storage resource usage rate interval are pre-associated in the storage resource deduction rule.

[0058] For example, the computing resource deduction rule is set as follows: if the computing resource usage rate is in the computing resource usage rate interval (90%, 100%], the computing resource deduction value is 50; if the computing resource usage rate is in the computing resource usage rate interval [75%, 90%], the computing resource deduction value is 30; if the computing resource usage rate is in the computing resource usage rate interval [40%, 75%), the computing resource deduction value is 0; if the computing resource usage rate is in the computing resource usage rate interval [0, 40%), the computing resource deduction value is 20.

[0059] For another example, the storage resource deduction rule is set as follows: if the storage resource usage rate is in the storage resource usage rate interval (90%, 100%], the storage resource deduction value is 50; if the storage resource usage rate is in the storage resource usage rate interval [75%, 90%], the storage resource deduction value is 30; if the storage resource usage rate is in the storage resource usage rate interval [40%, 75%), the storage resource deduction value is 0; if the storage resource usage rate is in the storage resource usage rate interval [0, 40%), the storage resource deduction value is 20.

[0060] Optionally, the determination of the storage resource health based on the result of weighted accumulation of the computing resource utilization rate and the storage resource utilization rate is based on the following expressions (5)-(7): ; (5)

[0061] ; (6)

[0062] ; (7)

[0063] in, To store and calculate the health of resources; Based on computing resource utilization Determine the computing resource deduction value; Based on storage resource usage Determine the storage resource penalty value; calculate the resource penalty value and storage resource penalty value The first weight is 50%.

[0064] The tenant health assessment method provided by the present invention divides the usage of storage and computing resources into four situations in advance. When the tenant's resource utilization rate is higher or lower than a certain threshold, the storage and computing resource health is deducted. Only when the tenant's resource utilization rate is within a certain threshold range, no points are deducted for the storage and computing resource health, thereby more accurately assessing the tenant's health status, avoiding the risk that future tasks may not be completed in a timely and smooth manner due to the tenant's excessive resource utilization rate, and avoiding the waste of resources caused by the tenant's excessively low resource utilization rate, thereby giving early warning to the tenant and providing users with more accurate and targeted guidance.

[0065] Based on the above embodiment, as an optional embodiment, the data asset dimension includes a model construction sub-dimension, a task operation sub-dimension and an application construction sub-dimension; The data asset health is determined based on the following: According to the big data platform usage record data of the tenant, the data asset health of the tenant is analyzed from the model construction sub-dimension, the task operation sub-dimension and the application construction sub-dimension to obtain the model construction health, the task operation health and the application construction health; The data asset health is determined based on a weighted accumulation result of the model construction health, the task operation health, and the application construction health.

[0066] In the tenant health assessment index system, the data asset dimension includes three sub-dimensions: model construction sub-dimension, task operation sub-dimension, and application construction sub-dimension, each of which has several specific assessment indicators.

[0067] The model construction sub-dimension records the usage of the tenant's big data platform and analyzes the health of data assets from multiple aspects such as model standardization, model availability, data asset validity, and data retention integrity.

[0068] The task operation sub-dimension is a sub-dimension that analyzes the health of data assets from multiple aspects, such as task timeliness, task one-time success rate, task stability, and task standardization, by recording the usage data of the tenant's big data platform.

[0069] The application construction sub-dimension records the tenants' use of the big data platform and analyzes the health of data assets from multiple aspects such as data application value and data application efficiency.

[0070] Specifically, when the data asset dimension includes three sub-dimensions: model construction sub-dimension, task operation sub-dimension and application construction sub-dimension, the tenant's big data platform usage record data is analyzed from these three sub-dimensions to obtain the model construction health, task operation health and application construction health corresponding to the three sub-dimensions; then the weighted accumulation result of the model construction health, task operation health and application construction health is taken as the tenant's data asset health.

[0071] Furthermore, the second weights of the model construction sub-dimension, the task operation sub-dimension and the application construction sub-dimension in the data asset dimension are determined in advance according to the expert scoring method for weighted accumulation, and the sum of the second weights of the model construction sub-dimension, the task operation sub-dimension and the application construction sub-dimension is 1.

[0072] Optionally, considering that the task running status can directly reflect the tenant situation, the second weight of the task running sub-dimension is set to the heaviest; the model is the carrier of stored data, and the model standardization and number of models directly affect the tenant's storage resources and computing resources usage, so the second weight is set lower than the task running sub-dimension; the data application construction status focuses on evaluating the value of data indirectly related to tenants, so the second weight is the lowest.

[0073] Optionally, the expression for determining the data asset health based on the weighted accumulation result of the model construction health, the task operation health and the application construction health is as follows: ; (8)

[0074] in, For data asset health; is the health of model construction, and 30% is the second weight of model construction health; is the task operation health, and 55% is the second weight of the task operation health; is the task operation health, and 15% is the second weight of the task operation health.

[0075] Optionally, after the second weight is determined for the first time using the expert scoring method, the second weight is regularly adjusted according to actual operating conditions and business development needs to ensure the accuracy of the evaluation.

[0076] The tenant health assessment method provided by the present invention, when comprehensively analyzing the health of data assets, takes into account the degree of standardization and quantity of the models of the big data platform that directly affect the use of tenants' storage resources and computing resources, as well as the task operation status that directly reflects the tenants, and also takes into account the application construction status that focuses on data value. It comprehensively analyzes the tenant's big data platform usage record data from three sub-dimensions: model establishment, task operation, and application establishment, and determines the second weight based on the expert scoring method, and regularly updates the second weight to adapt to the actual operation of the big data platform and business development needs, thereby laying a foundation for the healthy and stable operation of the big data platform.

[0077] Based on the above embodiment, as an optional embodiment, the model construction sub-dimension includes a model standardization rate indicator, a model availability rate indicator, a data asset efficiency index and a data retention integrity rate indicator; The model building health is based on the following: Determine the model standardization rate based on the number of irregular formal models and the number of formal models; Determining model availability based on the number of formal models and the total number of full physical tables; Determine the data asset effectiveness based on the number of referenced models and the number of formal models; Determine the data retention integrity rate based on the number of models with complete storage cycles and the number of formal models; The health of the model construction is determined based on the weighted accumulation of the model standardization rate, the model availability rate, the data asset efficiency rate and the data retention integrity rate.

[0078] The model normalization rate indicator is a specific evaluation indicator for measuring the model normalization; model normalization mainly focuses on the management of tenant space storage performance, and is determined based on the model's storage format, the size of the storage space occupied, and lifecycle management.

[0079] The model availability index is a specific evaluation index for measuring the availability of the model. A formally released model that has complete data metadata information on the big data platform, accurate and reasonable definition of data elements, data quality that meets availability standards, reliable data, and meets the requirements for statistical analysis is a model with model availability, that is, a formal model.

[0080] The data asset efficiency index is a specific evaluation indicator for measuring the effectiveness of data assets. When a data model is referenced by other data applications such as intelligent query reports, or is used to process subsequent models, tasks, statistical analysis associations, etc., it can be evaluated as an effective data asset.

[0081] The data retention completeness rate indicator is a specific evaluation indicator for measuring the integrity of data retention. When the model has complete retention and maintenance capabilities on the data platform, ensures the integrity of the data retention cycle, and can meet the application needs of users in multiple scenarios such as daily operations, auditing, monitoring or analysis, it is considered to have data retention integrity.

[0082] Specifically, when the model construction sub-dimension includes four specific evaluation indicators, namely, model standardization rate indicator, model availability rate indicator, data asset efficiency indicator and data retention integrity rate indicator, the actual measured values ​​of the tenant related to the above four specific evaluation indicators are first calculated based on the tenant's data platform usage record data.

[0083] For the model standardization rate indicator, the number of formal models that have complete metadata information and are officially released by the tenant in the big data platform is obtained, and the number of non-standard formal models, including the number of non-standard formal models with non-standard storage formats (that is, incomplete metadata information) and the number of non-standard formal models with non-standard lifecycle policies (that is, not officially released), is obtained. Then, the model standardization rate of the tenant is determined based on the number of formal models and the number of non-standard formal models.

[0084] For the model availability indicator, the total number of physical tables of the tenant in the big data platform is obtained, and the tenant's model availability is determined in combination with the number of formal models obtained above.

[0085] For the data asset efficiency indicator, the number of referenced models of the tenant in the big data platform after the model is officially released is obtained, and the tenant's data asset efficiency is determined in combination with the number of official models obtained above.

[0086] For the data retention completeness rate indicator, the number of models with complete storage cycles of the tenant in the big data platform is obtained, and the tenant's data retention completeness rate is determined in combination with the number of formal models obtained previously.

[0087] Furthermore, the third weights for weighted accumulation of the model standardization rate indicator, model availability rate indicator, data asset efficiency indicator and data retention integrity rate indicator in the model construction sub-dimension are determined in advance based on the expert scoring method and the degree of indicator dispersion, and the sum of the third weights is 1.

[0088] The degree of indicator dispersion is a representation of the degree of dispersion of specific evaluation indicators. If the degree of dispersion is higher, it means that the ability of the specific evaluation indicator to distinguish different evaluation objects is stronger, and the corresponding weight is higher.

[0089] After obtaining the tenant's model standardization rate, model availability rate, data asset efficiency rate and data retention integrity rate, they are weighted and accumulated in combination with the third weight, and the result is used as the health of the model construction.

[0090] Optionally, the determining of the model normalization rate based on the number of irregular formal models and the number of formal models is obtained according to the following expression: ; (9)

[0091] in, is the model norm rate; The number of formal models that have complete data metadata information and are officially released; The number of irregular formal models with irregular storage formats; The number of informal formal models that are not formalized for lifecycle policies.

[0092] Optionally, the determining of the model availability based on the number of formal models and the total number of all physical tables is obtained according to the following expression: ; (10)

[0093] in, is the model availability; is the number of formal models; The total number of all physical tables.

[0094] Optionally, the determining of the data asset efficiency based on the number of referenced models and the number of formal models is obtained according to the following expression: ; (11)

[0095] in, Be efficient for data assets; is the number of cited models that have been cited after the model was officially released; is the official model number.

[0096] Optionally, the determination of the data retention integrity rate based on the number of models with complete storage cycles and the number of formal models is obtained according to the following expression: ; (12)

[0097] in, The data retention completeness rate; The number of models for which the formal model is complete for the storage cycle; is the official model number.

[0098] The tenant health assessment method provided by the present invention, when comprehensively analyzing the health of model establishment, takes the number of formal models that have complete data metadata information and are officially released as the core, comprehensively analyzes the relationship between the number of formal models and non-standard formal models, the total number of full physical tables, the number of referenced models and the number of models with a complete storage cycle, and determines the health of model establishment mainly based on formal models. It can put forward suggestions mainly on increasing the number of formal models in terms of targeted guidance for tenants and healthy and stable operation of the platform, laying a foundation for the healthy and stable operation of the big data platform.

[0099] Based on the above embodiment, as an optional embodiment, the task operation sub-dimension includes a task timeliness rate index, a task first-time success rate index, a task stability rate index and a task standardization rate index; The task running health is obtained based on the following method: Determine the task on-time rate based on the number of task cycles whose execution time is lower than the task benchmark completion time and the total number of task cycles; Determine a task success rate based on the number of task cycles that succeed once and the total number of task cycles; Determining a task stability rate based on the number of unstable task cycles and the total number of task cycles; Determining a task standardization rate based on the number of irregular task cycles and the total number of task cycles; The task operation health is determined based on the weighted accumulation result of the task timeliness rate, the task one-time success rate, the task stability rate and the task standard rate.

[0100] The task timeliness index is a specific evaluation indicator to measure the task scheduling system's ability to control time; timeliness refers to the ratio of the number of tasks completed according to the scheduled time requirements to the total task volume.

[0101] The task one-time success rate index is a specific evaluation indicator to measure the efficiency and quality control level during task execution. It is determined based on the proportion of tasks that can achieve the expected results when they are executed for the first time. A higher one-time success rate means that the system is stable and resources are used reasonably.

[0102] The task stability rate index is a specific evaluation indicator to measure the fluctuation of tasks during execution. It can be evaluated by the frequency of failures during task execution. For example, if a scheduled task reports continuous errors within a certain period, it will be marked as a low-stability task.

[0103] The task standardization rate index is a specific evaluation indicator to measure whether the task execution process meets the predetermined specifications and standards. It can be determined based on the task's compliance with established procedures, standardized operations, and output quality. Excessive complexity and excessive running time are common manifestations of non-standard tasks.

[0104] Specifically, when the task operation sub-dimension includes four specific evaluation indicators, namely, task timeliness rate indicator, task one-time success rate indicator, task stability rate indicator and task standardization rate indicator, the actual measured values ​​of the above four specific evaluation indicators of the tenant are first calculated based on the recorded data of the tenant's data platform.

[0105] For the task timeliness index, the execution time of each scheduled task within a certain period is extracted from the tenant's big data platform usage record data, and the task benchmark completion time is determined based on the completion time of the scheduled tasks of the total number of task cycles in the statistical period. If the execution time of a task is lower than the task benchmark completion time of the task, it means that the task is completed on time, and the number of task cycles whose execution time is lower than the task benchmark completion time is counted, and then the task timeliness is determined based on the number of task cycles whose execution time is lower than the task benchmark completion time and the total number of task cycles.

[0106] For the task one-time success rate indicator, one-time success is defined as the scheduling task running successfully without error in one go without intervention. It is used to evaluate the running quality of the task. The number of task cycles in which the scheduling task is successfully scheduled is counted from the tenant's big data platform statistical usage record data, and the task one-time success rate is determined in combination with the total number of task cycles in the aforementioned statistical period.

[0107] For the task stability rate indicator, tasks that have continuous abnormalities in the statistical period (such as abnormalities for 7 consecutive days or more than 10 days) are defined as unstable tasks. The number of unstable task cycles in the statistical period is combined with the total number of task cycles in the aforementioned statistical period to determine the task stability rate.

[0108] For the task standardization rate indicator, non-standard tasks are defined as unstable tasks, large SQL tasks, and tasks whose running time exceeds the preset duration (such as 1 hour, 3 hours, etc.). The number of non-standard task cycles in the statistical period is counted, and the task standardization rate is determined in combination with the total number of task cycles in the aforementioned statistical period.

[0109] Furthermore, the fourth weights for weighted accumulation of the task timeliness index, the task one-time success rate index, the task stability rate index and the task standardization rate index in the task operation sub-dimension are determined in advance according to the expert scoring method and the degree of indicator dispersion, and the sum of the fourth weights is 1.

[0110] After obtaining the tenant's task timeliness rate, task one-time success rate, task stability rate and task standardization rate, they are weighted and accumulated in combination with the fourth weight, and the result is used as the task operation health.

[0111] Optionally, for tasks with a daily / hourly cycle, the task benchmark completion time is set to the average execution time of all daily / hourly cycle tasks in the past week, and the benchmark time range corresponding to the average execution time can be obtained by floating it by several minutes before and after the average execution time.

[0112] Optionally, for tasks with a monthly execution cycle, the task benchmark completion time is set to the average execution time of all tasks with a monthly execution cycle in the past six months, and the benchmark time range corresponding to the average execution time can be obtained by fluctuating several hours before and after the average execution time.

[0113] Optionally, the task timeliness rate is determined based on the number of task cycles whose execution time is lower than the task benchmark completion time and the total number of task cycles based on the following expression: ; (13)

[0114] in, The task timeliness rate; The number of task cycles in which the tenant's execution time is lower than the task's benchmark completion time; It is the total number of task cycles in the statistical period.

[0115] Optionally, the determination of the task one-time success rate based on the number of task cycles that succeed once and the total number of task cycles is determined based on the following expression: ; (14)

[0116] in, The first-time success rate of the task; The number of task cycles that are successfully scheduled. It is the total number of task cycles in the statistical period.

[0117] Optionally, the determining of the task stability rate based on the number of unstable task cycles and the total number of task cycles is based on the following expression: ; (15)

[0118] in, is the task stability rate; The number of unstable task cycles in which the tenant's task scheduling is abnormal within the statistical period; It is the total number of task cycles in the statistical period.

[0119] Optionally, the determining of the task standardization rate based on the number of irregular task cycles and the total number of task cycles is based on the following expression: ; (16)

[0120] in, is the task specification rate; The number of large SQL tasks in the tenant; The number of tasks that have run longer than the preset time; The number of unstable task cycles in which the tenant's task scheduling is abnormal within the statistical period; , , Together they make up the number of irregular task cycles; It is the total number of task cycles in the statistical period.

[0121] The tenant health assessment method provided by the present invention, when comprehensively analyzing the task operation health, comprehensively analyzes the timeliness rate, one-time success rate, stability rate and standardization rate of the tenants running tasks on the big data platform, and determines the task operation health based on this. It can provide guidance for improving the quality of tenant task operation in two aspects: targeted guidance for tenants and healthy and stable operation of the platform, laying the foundation for the healthy and stable operation of the big data platform.

[0122] Based on the above embodiment, as an optional embodiment, the application establishment sub-dimension includes a low-value data application rate indicator and a low-efficiency data application rate indicator; The application construction health is obtained based on the following method: Determine the low-value data application rate based on the number of low-value data applications and the total number of data applications; Determining an inefficient data application rate based on the number of inefficient data applications and the total number of data applications; The task operation health is determined based on a weighted accumulation result of the low-value data application rate and the low-efficiency data application rate.

[0123] The low-value data application rate indicator is a specific evaluation indicator to measure the contribution of data applications to the business. It can be evaluated through the click rate, usage rate or download volume of the front-end report. Applications with low long-term visits will be marked as low-value data applications.

[0124] The low-efficiency data application rate index is a specific evaluation indicator to measure the effectiveness of data application use. It can be used to query data query speed, resource consumption level and other data through the foreground application.

[0125] Specifically, when the application establishes sub-dimensions including the low-value data application rate indicator and the low-efficiency data application rate indicator, the actual measured values ​​of the tenant related to the above two specific evaluation indicators are first calculated based on the tenant's data platform usage record data.

[0126] For the low-value data application rate indicator, the access situation of each application is extracted from the tenant's big data platform usage record data. If the access frequency is less than the preset frequency (such as the cumulative number of visits within three months is less than 5 times), it means that the application is a low-value data application. The number of all low-value data applications in the tenant is obtained by statistics, and the low-value data application rate is determined based on the total number of all data applications in the tenant.

[0127] For the low-efficiency data application rate indicator, the number of times each application is accessed and the average query time are extracted from the tenant's big data platform usage record data. If the application is accessed more than the preset number of times (such as 100 times) and the average query time exceeds the preset time (such as 3 seconds) during the statistical period and the storage efficiency is low, then the application is considered to be an inefficient data application. The number of all inefficient data applications in the tenant is obtained, and the low-efficiency data application rate is determined based on the aforementioned total number of data applications.

[0128] Furthermore, the fifth weights of the low-value data application rate index and the low-efficiency data application rate index in the application establishment sub-dimension are determined in advance based on the expert scoring method and the degree of indicator dispersion for weighted accumulation, and the sum of the fifth weights is 1.

[0129] After obtaining the tenant's low-value data application rate index and low-efficiency data application rate index, they are weighted and accumulated in combination with the fifth weight, and the result is used as the task operation health.

[0130] Optionally, the determining of the low-value data application rate based on the number of low-value data applications and the total number of data applications is based on the following expression: ; (17)

[0131] in, The application rate of low-value data; The number of low-value data applications within the tenant whose access frequency is less than the preset frequency; The total number of applications for data within a tenant.

[0132] Optionally, the determining of the inefficient data application rate based on the inefficient data application quantity and the total data application quantity is determined based on the following expression: ; (18)

[0133] in, The application rate of low-value data; The number of low-value data applications within the tenant whose access volume exceeds the preset number of times, whose average query time exceeds the preset time, and whose storage efficiency is low; The total number of applications for data within a tenant.

[0134] The tenant health assessment method provided by the present invention comprehensively analyzes the value and efficiency of tenant applications being called on the big data platform when comprehensively analyzing the task operation health. It can provide guidance for improving reference value and efficiency in terms of targeted guidance for tenants and healthy and stable operation of the platform, laying the foundation for the healthy and stable operation of the big data platform.

[0135] In another embodiment, according to each dimension, each sub-dimension, each specific evaluation indicator and the corresponding weight determined in any of the above embodiments, the weights of all specific evaluation indicators are normalized to obtain a health calculation formula for a single tenant as follows: ; (19)

[0136] in, For the health of tenants; To calculate resource utilization; It is the utilization rate of storage resources; is the model norm rate; is the model availability; Be efficient for data assets; The data retention completeness rate; The task timeliness rate; The first-time success rate of the task; is the task stability rate; is the task specification rate; The application rate of low-value data; For low efficiency data application rate; is the first weight corresponding to the computing resource utilization rate; is the first weight corresponding to the storage resource usage; is the normalized weight of the third weight corresponding to the model norm; is the normalized weight of the third weight corresponding to the model availability; The normalized weight of the third weight corresponding to the efficiency of data assets; The normalized weight of the third weight corresponding to the data retention completeness rate; is the normalized weight of the fourth weight corresponding to the task timeliness rate; is the normalized weight of the fourth weight corresponding to the task's one-time success rate; is the normalized weight of the fourth weight corresponding to the task stability rate; is the normalized weight of the fourth weight corresponding to the task standardization rate; is the normalized weight of the fifth weight corresponding to the application rate of low-value data; is the normalized weight of the fifth weight corresponding to the low-efficiency data application rate.

[0137] The tenant's big data platform usage record data is used to obtain the measured values ​​of each specific evaluation indicator, and the measured values ​​are substituted into the above calculation formula (19) to obtain the tenant's health score. If the discreteness of a specific evaluation indicator is higher, the ability to distinguish different evaluation objects is stronger, so the corresponding weight should be higher.

[0138] Table 1 is the comprehensive weight after normalization of the weights of the specific evaluation indicators provided by the present invention. As shown in Table 1, the health calculation formula of a single tenant is as follows: (20)

[0139] Table 1

[0140] Optionally, weight assignment techniques such as coefficient of variation method and entropy method are used to effectively capture weight information of specific evaluation indicators.

[0141] Optionally, the weights of each specific evaluation indicator can be comprehensively obtained by integrating subjective and objective weighting methods.

[0142] Optionally, after obtaining the tenant's health score, the tenant is divided into several levels, such as excellent, good, average, poor, etc., to achieve a detailed portrayal of each tenant's usage.

[0143] The tenant health assessment method provided by the present invention establishes a systematic and comprehensive multi-dimensional and multi-indicator health assessment indicator system, and constructs a reasonable and effective health assessment analysis model to achieve quantitative assessment and analysis of tenant health. It can provide targeted guidance for the tenant operation of the big data platform and lay the foundation for the efficient and stable operation of the big data platform.

[0144] At the same time, by introducing innovative data operation management strategies and intelligent analysis tools, a tenant-based health evaluation system is constructed to improve the operational efficiency of the enterprise big data platform, optimize resource allocation strategies, improve resource utilization, tenant satisfaction and data application value, strengthen the correlation between various measurement indicators, and realize the systematization and standardization of tenant operations. In summary, the present invention can bring continuous and measurable commercial value to the enterprise big data tenant operations, fully tap the potential of the big data platform, and promote technological progress and innovation in the industry.

[0145] Figure 2 is a schematic diagram of the structure of the tenant health assessment device provided by the present invention, such as Figure 2 As shown, the tenant health assessment device includes but is not limited to a dimension analysis module 201 and a health determination module 202 .

[0146] The dimension analysis module 201 is used to analyze the health of the tenant from the dimensions of storage and computing resources and data assets according to the tenant's big data platform usage record data, and obtain the storage and computing resource health and data asset health; The health determination module 202 is used to determine the health of the tenant based on the health of the storage and computing resources and the health of the data assets.

[0147] It should be noted that the tenant health assessment device provided by the present invention can execute the tenant health assessment method described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0148] The tenant health assessment device provided by the present invention comprehensively analyzes the tenant's big data platform usage data from the storage and computing resource dimensions and the data asset dimensions, accurately assesses the tenant's health based on a comprehensive assessment index system, and issues reliable tenant operation notifications and risk warnings based on the tenant health assessment results, thereby laying the foundation for the healthy and stable operation of the big data platform and providing targeted guidance for tenant operations.

[0149] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tenant health assessment method described in any of the above embodiments, and the tenant health assessment method includes: according to the tenant's big data platform usage record data, analyzing the health of the tenant from the storage and computing resource dimension and the data asset dimension respectively to obtain the storage and computing resource health and the data asset health; based on the storage and computing resource health and the data asset health, determining the health of the tenant.

[0150] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the tenant health assessment method described in any of the above embodiments, and the tenant health assessment method includes: according to the tenant's big data platform usage record data, analyzing the health of the tenant from the storage and computing resource dimension and the data asset dimension respectively to obtain the storage and computing resource health and the data asset health; based on the storage and computing resource health and the data asset health, determining the health of the tenant.

[0151] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tenant health assessment method, characterized in that: include: According to the tenant's big data platform usage record data, the tenant's health is analyzed from the storage and computing resource dimension and the data asset dimension to obtain the storage and computing resource health and data asset health; The health of the tenant is determined based on the health of the storage and computing resources and the health of the data assets.

2. The tenant health assessment method according to claim 1, characterized in that: The storage and computing resource dimension includes a computing resource utilization index and a storage resource utilization index; The health of the storage and computing resources is determined based on the following methods: Determine computing resource usage based on the tenant's daily average memory usage and pre-allocated memory quota; Determine the storage resource utilization rate based on the storage space occupied by the tenant and the pre-allocated space quota; The health of the storage and computing resources is determined based on a weighted accumulation result of the computing resource utilization rate and the storage resource utilization rate.

3. The tenant health assessment method according to claim 2, characterized in that: The determining the health of the storage resource based on the weighted accumulation result of the computing resource utilization rate and the storage resource utilization rate includes: Determine a computing resource deduction value based on the computing resource usage interval in which the computing resource usage falls; Determining a storage resource deduction value based on the storage resource usage interval in which the storage resource usage falls; The health of the storage and computing resources is determined based on a result of weighted accumulation of the computing resource deduction value and the storage resource deduction value.

4. The tenant health assessment method according to claim 1, characterized in that: The data asset dimension includes a model construction sub-dimension, a task operation sub-dimension, and an application construction sub-dimension; The data asset health is determined based on the following: According to the big data platform usage record data of the tenant, the data asset health of the tenant is analyzed from the model construction sub-dimension, the task operation sub-dimension and the application construction sub-dimension to obtain the model construction health, the task operation health and the application construction health; The data asset health is determined based on a weighted accumulation result of the model construction health, the task operation health, and the application construction health.

5. The tenant health assessment method according to claim 4, characterized in that: The model construction sub-dimensions include model standardization rate index, model availability rate index, data asset efficiency index and data retention completeness rate index; The model building health is based on the following: Determine the model standardization rate based on the number of irregular formal models and the number of formal models; Determining model availability based on the number of formal models and the total number of full physical tables; Determine the data asset effectiveness based on the number of referenced models and the number of formal models; Determine the data retention integrity rate based on the number of models with complete storage cycles and the number of formal models; The health of the model construction is determined based on the weighted accumulation of the model standardization rate, the model availability rate, the data asset efficiency rate and the data retention integrity rate.

6. The tenant health assessment method according to claim 4, characterized in that: The task operation sub-dimensions include task timeliness rate index, task first-time success rate index, task stability rate index and task standardization rate index; The task running health is obtained based on the following method: Determine the task on-time rate based on the number of task cycles whose execution time is lower than the task benchmark completion time and the total number of task cycles; Determine a task success rate based on the number of task cycles that succeed once and the total number of task cycles; Determining a task stability rate based on the number of unstable task cycles and the total number of task cycles; Determining a task standardization rate based on the number of irregular task cycles and the total number of task cycles; The task operation health is determined based on the weighted accumulation result of the task timeliness rate, the task one-time success rate, the task stability rate and the task standard rate.

7. The tenant health assessment method according to claim 4, characterized in that: The application establishment sub-dimension includes a low-value data application rate indicator and a low-efficiency data application rate indicator; The application construction health is obtained based on the following method: Determine the low-value data application rate based on the number of low-value data applications and the total number of data applications; Determining an inefficient data application rate based on the number of inefficient data applications and the total number of data applications; The task operation health is determined based on a weighted accumulation result of the low-value data application rate and the low-efficiency data application rate.

8. A tenant health assessment device, characterized in that: include: A dimension analysis module is used to analyze the health of the tenant from the dimensions of storage and computing resources and data assets according to the tenant's big data platform usage record data, and obtain the storage and computing resource health and data asset health; The health determination module is used to determine the health of the tenant based on the health of the storage and computing resources and the health of the data assets.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the tenant health assessment method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the tenant health assessment method according to any one of claims 1 to 7 is implemented.

Images