Resource adjustment method and device of big data platform and computer program product

Abstract

The invention discloses a resource adjustment method and device of a big data platform and a computer program product. The method relates to the field of big data and comprises the steps that to-be-adjusted resources of a big data platform are determined, and the to-be-adjusted resources comprise at least one of computing resources and storage resources; obtaining resource use data of the to-be-adjusted resource in a preset period, and calculating a utilization efficiency evaluation value and an application state evaluation value of the to-be-adjusted resource based on the resource use data; determining a resource application level based on the utilization efficiency evaluation value and the application state evaluation value; and determining a resource adjustment strategy corresponding to the resource application level, and executing the resource adjustment strategy on the big data platform. Through the method and the device, the problem of low resource utilization efficiency of a big data platform in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of resource adjustment, specifically to a method, apparatus, and computer program product for a big data platform. Background Technology

[0002] With the acceleration of digital transformation in enterprises worldwide, big data technology is being widely applied across various industries. Digital transformation not only drives the optimization and innovation of business processes but also generates a surge in demand for data analysis and processing. This demand manifests as high-frequency, large-scale access to and utilization of big data platform resources, such as real-time data analysis, machine learning model training, and the storage and retrieval of massive amounts of data. However, the supply of big data platform resources is not unlimited. Especially when faced with surging data volumes and complex application scenarios, limited computing power and storage space have become a major bottleneck restricting the digital development of enterprises.

[0003] In related technologies, when users encounter performance bottlenecks while executing data processing jobs, such as slow processing speeds and long waiting times, the common approach is to request more resources to expand capacity and improve processing speed. While this approach can alleviate resource pressure in the short term, some users may over-request resources, resulting in actual usage being far below the allocated amount and wasting resources. Excessive resource requests and expansion increase investment in hardware infrastructure, raise operation and maintenance costs, and irrational resource allocation may lead to insufficient resource support for critical business operations while non-critical business operations consume excessive resources, impacting overall business efficiency and response speed.

[0004] There is currently no effective solution to the problem of low resource utilization efficiency in big data platforms in related technologies. Summary of the Invention

[0005] The main objective of this application is to provide a resource adjustment method, apparatus, and computer program product for a big data platform to solve the problem of low resource utilization efficiency in related technologies.

[0006] To achieve the above objectives, according to one aspect of this application, a resource adjustment method for a big data platform is provided. The method includes: determining resources to be adjusted in the big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; acquiring resource usage data of the resources to be adjusted within a preset period; calculating a utilization efficiency assessment value and an application status assessment value of the resources to be adjusted based on the resource usage data; determining a resource application level based on the utilization efficiency assessment value and the application status assessment value; determining a resource adjustment strategy corresponding to the resource application level; and executing the resource adjustment strategy on the big data platform.

[0007] Optionally, when the resources to be adjusted include computing resources, calculating the utilization efficiency assessment value of the resources to be adjusted based on resource usage data includes: extracting the daily computing resource usage and computing resource allocation of the resource scheduling queue from the resource usage data, wherein the computing resource usage is the number of virtual CPU cores used within a preset duration, and the computing resource allocation is the number of virtual CPU cores allocated to the resource scheduling queue within a preset period; calculating the ratio of computing resource usage to computing resource allocation to obtain the daily utilization efficiency; calculating the average daily utilization efficiency within the preset period to obtain the daily average utilization efficiency; determining the adjustment coefficient of utilization efficiency, and calculating the product of the daily average utilization efficiency and the adjustment coefficient to obtain the utilization efficiency assessment value.

[0008] Optionally, when the resources to be adjusted include computing resources, calculating the application status assessment value of the resources to be adjusted based on resource usage data includes: extracting the target number of target structured query statements within a preset period from the resource usage data, wherein the target structured query statement is a structured query statement whose execution time is greater than or equal to a duration threshold, or a structured query statement whose computing resource amount is greater than or equal to a resource amount threshold; determining the adjustment coefficient of the application status, and calculating the difference between the adjustment coefficient and the target number to obtain the application status assessment value.

[0009] Optionally, when the resources to be adjusted include storage resources, calculating the utilization efficiency assessment value of the resources to be adjusted based on resource usage data includes: extracting the storage resource usage and storage resource allocation within a preset period from the resource usage data, determining the adjustment coefficient of utilization efficiency; calculating the ratio of storage resource usage to storage resource allocation, and calculating the product of the adjustment coefficient and the ratio to obtain the utilization efficiency assessment value.

[0010] Optionally, when the resources to be adjusted include storage resources, calculating the application status assessment value of the resources to be adjusted based on resource usage data includes: extracting the storage space release capacity and storage resource allocation amount within a preset period from the resource usage data, and determining the application status adjustment coefficient; calculating the ratio of the storage space release capacity to the storage resource allocation amount, and calculating the product of the ratio and the adjustment coefficient to obtain the application status assessment value.

[0011] Optionally, determining the resource application level based on the utilization efficiency assessment value and the application status assessment value includes: determining the resource application level as Level 1 when the utilization efficiency assessment value is greater than or equal to the target assessment value threshold and the application status assessment value is greater than or equal to the target assessment value threshold; determining the resource application level as Level 2 when the utilization efficiency assessment value is less than the target assessment value threshold and the application status assessment value is greater than or equal to the target assessment value threshold, wherein the health status of the resource application at Level 1 is better than that at Level 2; determining the resource application level as Level 3 when the utilization efficiency assessment value is greater than or equal to the target assessment value threshold and the application status assessment value is less than the target assessment value threshold, wherein the health status of the resource application at Level 2 is better than that at Level 3; and determining the resource application level as Level 4 when the utilization efficiency assessment value is less than the target assessment value threshold and the application status assessment value is less than the target assessment value threshold, wherein the health status of the resource application at Level 3 is better than that at Level 4.

[0012] Optionally, determining the resource adjustment strategy corresponding to the resource application level includes: when the resource application level is Level 1, determining resource expansion as the resource adjustment strategy; when the resource application level is Level 2, determining resource recycling as the resource adjustment strategy, wherein the health status of resource application at Level 1 is better than that at Level 2; when the resource application level is Level 3 or Level 4, determining the causes of resource waste, and determining the governance strategy corresponding to the causes of resource waste as the resource adjustment strategy, wherein the health status of resource application at Level 2 is better than that at Level 3, and the health status of resource application at Level 3 is better than that at Level 4.

[0013] To achieve the above objectives, according to another aspect of this application, a resource adjustment device for a big data platform is provided. The device includes: a first determining unit, configured to determine resources to be adjusted in the big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; an acquiring unit, configured to acquire resource usage data of the resources to be adjusted within a preset period, and calculate a utilization efficiency assessment value and an application status assessment value of the resources to be adjusted based on the resource usage data; a second determining unit, configured to determine a resource application level based on the utilization efficiency assessment value and the application status assessment value; and an execution unit, configured to determine a resource adjustment strategy corresponding to the resource application level and execute the resource adjustment strategy on the big data platform.

[0014] In this embodiment, the following approach is adopted: determining the resources to be adjusted in a big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; acquiring resource usage data of the resources to be adjusted within a preset period; calculating the utilization efficiency assessment value and application status assessment value of the resources to be adjusted based on the resource usage data; determining the resource application level based on the utilization efficiency assessment value and application status assessment value; determining the resource adjustment strategy corresponding to the resource application level; and executing the resource adjustment strategy on the big data platform. By conducting a detailed evaluation of the computing and storage resources of the big data platform and formulating specific resource adjustment strategies based on the utilization efficiency assessment value and application status assessment value, the aim of avoiding waste of cluster resources, optimizing resource allocation, and reducing costs is achieved, thereby realizing the technical effect of improving the resource utilization efficiency of the big data platform and solving the technical problem of low resource utilization efficiency of the big data platform. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a resource adjustment method for a big data platform is shown.

[0017] Figure 2 This is a flowchart of a resource adjustment method for a big data platform provided according to an embodiment of this application;

[0018] Figure 3 This is a schematic diagram illustrating the determination of resource application levels according to embodiments of this application;

[0019] Figure 4 This is a schematic diagram of a resource adjustment device for a big data platform provided according to an embodiment of this application;

[0020] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0024] Example 1

[0025] According to an embodiment of this application, a method embodiment for resource adjustment of a big data platform is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a resource adjustment method for a big data platform is shown. Figure 1As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, processing devices such as MCU (Microcontroller Unit) or FPGA (Field-Programmable Gate Array), memory 104 for storing data, and transmission device 106 for communication functions. In addition, it may also include: a display, input / output interfaces (I / O interfaces), a USB (Universal Serial Bus) port (which may be included as one of the ports of a BUS (Business Bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0027] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0028] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the resource adjustment method of the big data platform in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the resource adjustment method of the big data platform described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0030] The display may be, for example, a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0031] Under the aforementioned operating environment, this application provides a resource adjustment method for a big data platform. Figure 2 This is a flowchart of a resource adjustment method for a big data platform provided according to an embodiment of this application, such as... Figure 2 As shown, the method includes:

[0032] Step S201: Determine the resources to be adjusted in the big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources.

[0033] In step S201, computing resources may include CPU (Central Processing Unit), memory, vcores (virtual CPU cores), etc., mainly used for data processing and computation. Storage resources may include database storage, etc., used for persistent data storage and access. The resources to be adjusted for the big data platform that require resource adjustments are determined based on business needs.

[0034] Step S202: Obtain resource usage data of the resource to be adjusted within a preset period, and calculate the utilization efficiency evaluation value and application status evaluation value of the resource to be adjusted based on the resource usage data.

[0035] In step S202, the preset period can be the past week, month, or any other time period, depending on the periodicity of resource usage patterns and data analysis needs. Resource usage data within the preset period is collected, including but not limited to CPU utilization, memory usage, vcore utilization, disk I / O (input / output), network bandwidth usage, and storage utilization. The average daily utilization of computing resources within the preset period is calculated, and the utilization efficiency assessment value of computing resources is quantified based on the average daily utilization rate. The application status assessment value is used to evaluate whether resource usage follows established specifications; for example, the number of slow SQL queries can be included as part of the application status assessment. The utilization efficiency assessment value of storage resources can be quantified by the average utilization rate of storage resources. The application status assessment value of storage resources can be quantified by factors such as whether there is a large amount of redundant data and whether effective data compression has been performed.

[0036] Step S203: Determine the resource application level based on the utilization efficiency assessment value and the application status assessment value.

[0037] In step S203, based on the calculated utilization efficiency assessment value and application status assessment value, the resource application level is divided into several levels, such as excellent, good, average, and unsatisfactory. Each level corresponds to a different range of utilization efficiency assessment values ​​and application status assessment values. A four-quadrant diagram can be used to map the resource utilization efficiency assessment value and application status assessment value to different quadrants to determine the resource application level. For example... Figure 3 This is a schematic diagram illustrating the determination of resource application levels according to embodiments of this application, such as... Figure 3 As shown, a rectangular coordinate system is constructed with the utilization efficiency assessment value as the horizontal axis, the application status assessment value as the vertical axis, and the target assessment value threshold as the origin. The resource application level is determined by the four quadrants divided by the rectangular coordinate system: the first quadrant corresponds to the first level, the second quadrant corresponds to the second level, the third quadrant corresponds to the third level, and the fourth quadrant corresponds to the fourth level.

[0038] Step S204: Determine the resource adjustment strategy corresponding to the resource application level, and execute the resource adjustment strategy on the big data platform.

[0039] In step S204, for resources that demonstrate good utilization efficiency and application status assessment, the current status can be maintained, or resource expansion can be prioritized if business demand further increases. Simultaneously, job priorities can be adjusted according to demand to ensure the smooth operation of critical business processes. For resources with low utilization but good application status, it is advisable to moderately reclaim resources without impacting business operations and reallocate them to more needed jobs to improve overall resource utilization. If resource utilization is low and there are instances of improper use, proactive governance measures are required, including reclaiming idle resources, optimizing slow SQL queries, and standardizing data storage to improve resource utilization efficiency and application status. After resource adjustments, resource usage is continuously monitored, and the resource application level is reassessed based on new data, forming a closed loop of continuous improvement.

[0040] The resource adjustment method for a big data platform provided in this application embodiment determines the resources to be adjusted in the big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; obtains resource usage data of the resources to be adjusted within a preset period, calculates the utilization efficiency evaluation value and application status evaluation value of the resources to be adjusted based on the resource usage data; determines the resource application level based on the utilization efficiency evaluation value and application status evaluation value; determines the resource adjustment strategy corresponding to the resource application level, and executes the resource adjustment strategy on the big data platform. By conducting a detailed evaluation of the computing and storage resources of the big data platform and formulating a specific resource adjustment strategy based on the utilization efficiency evaluation value and application status evaluation value, the method achieves the purpose of avoiding waste of cluster resources, optimizing resource allocation, and reducing costs, thereby realizing the technical effect of improving the resource utilization efficiency of the big data platform and solving the technical problem of low resource utilization efficiency of the big data platform.

[0041] The utilization efficiency assessment value of computing resources can be determined by the average daily utilization efficiency. Optionally, in the resource adjustment method of the big data platform provided in this application embodiment, when the resources to be adjusted include computing resources, calculating the utilization efficiency assessment value of the resources to be adjusted based on resource usage data includes: extracting the daily computing resource usage and computing resource allocation of the resource scheduling queue from the resource usage data, wherein the computing resource usage is the number of virtual central processing unit cores used within a preset time period, and the computing resource allocation is the number of virtual central processing unit cores allocated to the resource scheduling queue within a preset period; calculating the ratio of computing resource usage to computing resource allocation to obtain the daily utilization efficiency; calculating the average daily utilization efficiency within the preset period to obtain the average daily utilization efficiency; determining the adjustment coefficient of utilization efficiency, and calculating the product of the average daily utilization efficiency and the adjustment coefficient to obtain the utilization efficiency assessment value.

[0042] In some embodiments, daily computing resource usage data for resource scheduling queues is extracted from the resource management system of a big data platform. This includes the total number of virtual central processing unit cores (vcores) used per minute within a preset period. A vcore is a unit of measurement for CPU resources in a distributed computing framework, allowing for more flexible allocation to different tasks and users. The total number of vcores allocated to a specific resource scheduling queue within a preset period reflects the upper limit of computing resources available to that queue. The ratio of computing resource usage to allocated computing resources is calculated to obtain the daily utilization efficiency, which reflects the actual usage of computing resources relative to the allocated resources. The average daily utilization efficiency within the preset period is further calculated to obtain the daily average utilization efficiency. This daily average utilization efficiency is a long-term average assessment of resource usage efficiency, helping to provide a more comprehensive understanding of computing resource usage.

[0043] Based on the business characteristics and resource usage policies of the big data platform, an adjustment coefficient, such as 50, is pre-set to adjust the utilization efficiency assessment value, reflecting specific business needs or resource management strategies. The adjustment coefficient can be dynamically adjusted based on historical data, business volatility, resource scarcity, and other factors to ensure that the assessment value more accurately reflects the actual resource usage situation. The utilization efficiency assessment value is obtained by multiplying the average daily utilization efficiency by the adjustment coefficient.

[0044] This embodiment quantifies resource utilization efficiency to provide data support for resource management, enabling more refined and intelligent resource adjustments and optimizations. The efficiency assessment not only reflects resource usage but also considers business needs and resource management strategies, laying the foundation for the platform's sustainable development and efficient operation.

[0045] The application status assessment value of computing resources can be determined by the number of slow SQL queries. Optionally, in the resource adjustment method of the big data platform provided in this application embodiment, when the resources to be adjusted include computing resources, calculating the application status assessment value of the resources to be adjusted based on resource usage data includes: extracting the target number of target structured query statements within a preset period from the resource usage data, wherein the target structured query statement is a structured query statement whose execution time is greater than or equal to a time threshold, or whose computing resource amount is greater than or equal to a resource amount threshold; determining the adjustment coefficient of the application status, and calculating the difference between the adjustment coefficient and the target number to obtain the application status assessment value.

[0046] In some embodiments, execution records of all structured query statements (SQL) within a preset period are extracted from the computing resource usage data of the big data platform, and target SQL is filtered according to the following criteria: a threshold for execution time is set, for example, if an SQL execution time exceeds 10 minutes, it is considered a target SQL. A threshold for computing resource usage is set, such as if the number of vcores called by an SQL exceeds a preset upper limit, it is also marked as a target SQL.

[0047] The number of times the target SQL is executed within a preset period is counted; this number will serve as an important basis for subsequent evaluation. The application status adjustment factor is set considering the consumption patterns of computing resources by business needs. For example, some businesses may have a higher tolerance for long-running or resource-intensive queries. For instance, the application status adjustment factor can be set to 50. The application status evaluation value is obtained by calculating 50 minus the target number.

[0048] This embodiment uses computing resource usage data to quantitatively assess the application status of computing resources, identify and address problems in resource usage, thereby improving computing efficiency, reducing resource waste, and ensuring the healthy use and optimized allocation of platform resources.

[0049] The utilization efficiency assessment value of storage resources can be determined by the storage resource usage and the storage resource allocation. Optionally, in the resource adjustment method of the big data platform provided in this application embodiment, when the resources to be adjusted include storage resources, calculating the utilization efficiency assessment value of the resources to be adjusted based on resource usage data includes: extracting the storage resource usage and storage resource allocation within a preset period from the resource usage data, determining the adjustment coefficient of utilization efficiency; calculating the ratio of storage resource usage to storage resource allocation, and calculating the product of the adjustment coefficient and the ratio to obtain the utilization efficiency assessment value.

[0050] In some embodiments, storage resource usage data for a preset period is extracted from the resource management system of the big data platform. This includes data such as storage resource usage (the total amount of storage space actually occupied within a specified period, in GB or TB) and storage resource allocation (the upper limit of storage resources allocated to users, applications, or services within the same period, also in GB or TB). The ratio between the extracted storage resource usage and allocation is calculated, reflecting the efficiency of storage resource utilization. An adjustment factor can be determined based on the big data platform's storage strategy, business needs, and target utilization rate. For example, if the big data platform aims to improve storage efficiency, a higher adjustment factor might be set to encourage users to utilize storage resources more efficiently. The adjustment factor could be set to 50. Multiplying the storage resource usage ratio by the utilization efficiency adjustment factor yields the final utilization efficiency assessment value.

[0051] The utilization efficiency assessment value is a comprehensive indicator that reflects not only the actual usage of storage resources (i.e., the ratio of usage to allocation) but also the big data platform's expectations for storage resource utilization efficiency. By setting appropriate adjustment coefficients, the platform can incentivize users to optimize storage strategies, reduce unnecessary data redundancy, and improve storage space utilization.

[0052] This embodiment provides data support for the dynamic management and optimization of resources by quantitatively evaluating the utilization efficiency of storage resources in a big data platform. This mechanism not only identifies and addresses inefficient storage usage but also encourages users to adopt more economical and environmentally friendly data storage strategies, thereby improving the overall operational efficiency and economic benefits of the platform.

[0053] The application status assessment value of storage resources can be determined by the storage space release capacity and the storage resource allocation amount. Optionally, in the resource adjustment method of the big data platform provided in this application embodiment, when the resources to be adjusted include storage resources, calculating the application status assessment value of the resources to be adjusted based on resource usage data includes: extracting the storage space release capacity and storage resource allocation amount within a preset period from the resource usage data, determining the application status adjustment coefficient; calculating the ratio of the storage space release capacity and the storage resource allocation amount, and calculating the product of the ratio and the adjustment coefficient to obtain the application status assessment value.

[0054] In some embodiments, usage data regarding storage resources is extracted from the resource management system, such as storage space release capacity: the total amount of storage space actively released by users or applications or automatically reclaimed by the system within a preset period, in GB or TB. Storage resource allocation: the upper limit of storage resources allocated to users or applications within the same period, also in GB or TB. The ratio between storage space release capacity and storage resource allocation is calculated; this ratio reflects the efficiency of storage space management by users or applications. An adjustment factor is a preset value that characterizes the expected level or importance of storage space management efficiency for the big data platform. This factor can be determined based on the platform's storage strategy, business needs, and target release ratio, for example, set to 50. The release ratio is multiplied by the adjustment factor to obtain the final application status evaluation value.

[0055] This embodiment evaluates the storage space management efficiency of users or applications by quantitatively comparing the amount of storage space released and allocated, combined with the platform's management strategy, thereby quantifying the application status assessment value. This helps optimize the allocation of storage resources, encourages users to adopt more efficient and economical storage space usage strategies, and ultimately achieves the goals of reducing storage costs and improving the overall operating efficiency of the platform.

[0056] After determining the utilization efficiency assessment value and the application status assessment value, the resource application level corresponding to the utilization efficiency assessment value and the application status assessment value needs to be determined. Optionally, in the resource adjustment method of the big data platform provided in this application embodiment, determining the resource application level based on the utilization efficiency assessment value and the application status assessment value includes: determining the resource application level as a first level when the utilization efficiency assessment value is greater than or equal to the target assessment value threshold and the application status assessment value is greater than or equal to the target assessment value threshold; determining the resource application level as a second level when the utilization efficiency assessment value is less than the target assessment value threshold and the application status assessment value is greater than or equal to the target assessment value threshold, wherein the health status of the resource application at the first level is better than that at the second level; determining the resource application level as a third level when the utilization efficiency assessment value is greater than or equal to the target assessment value threshold and the application status assessment value is less than the target assessment value threshold, wherein the health status of the resource application at the second level is better than that at the third level; and determining the resource application level as a fourth level when the utilization efficiency assessment value is less than the target assessment value threshold and the application status assessment value is less than the target assessment value threshold, wherein the health status of the resource application at the third level is better than that at the fourth level.

[0057] In some embodiments, the target evaluation threshold can be determined based on the business characteristics, resource status, and management objectives of the big data platform, through historical data analysis and the experience of business experts. For each user or application, its utilization efficiency evaluation value and application status evaluation value are compared with the target evaluation threshold to classify resource application levels. Based on the comparison results of the evaluation values ​​and thresholds, the resource application level is determined, for example: Level 1 (Excellent resource application health): When the utilization efficiency evaluation value is greater than or equal to the target evaluation threshold, and the application status evaluation value is also greater than or equal to the target evaluation threshold, the resource application level is Level 1. This indicates that the user or application performs well in both resource utilization efficiency and application status, meeting the platform's healthy usage standards.

[0058] Level 2 (Good Resource Utilization Health): When the utilization efficiency assessment value is less than the target assessment value threshold, but the application status assessment value is greater than or equal to the target assessment value threshold, the resource utilization level is Level 2. This indicates that although resource utilization efficiency has not reached its optimal level, proper application status management still indicates a certain potential for healthy resource utilization.

[0059] Level 3 (General Health Status of Resource Application): When the utilization efficiency assessment value is greater than or equal to the target assessment value threshold, but the application status assessment value is less than the target assessment value threshold, the resource application level is Level 3. This indicates that resource utilization efficiency is relatively high, but there are problems in application status management, such as unreasonable use of storage space or excessive consumption of computing resources.

[0060] Level 4 (Poor Resource Application Health): When both the utilization efficiency assessment value and the application status assessment value are lower than their respective target assessment value thresholds, the resource application level is Level 4. This indicates serious problems in resource usage, such as inefficient resource use and poor application status management, requiring immediate remedial measures.

[0061] This embodiment utilizes a comprehensive analysis of efficiency assessment values ​​and application status assessment values ​​to enable the big data platform to more accurately assess the health status of resource applications. Based on the assessment results, different management strategies can be adopted to improve the overall resource utilization efficiency and business operation quality of the platform.

[0062] After determining the resource application level, a resource adjustment strategy corresponding to the resource application level is determined. Optionally, in the resource adjustment method of the big data platform provided in this application embodiment, determining the resource adjustment strategy corresponding to the resource application level includes: when the resource application level is the first level, determining resource expansion as the resource adjustment strategy; when the resource application level is the second level, determining resource recycling as the resource adjustment strategy, wherein the health status of resource application in the first level is better than that in the second level; when the resource application level is the third or fourth level, determining the cause of resource waste, and determining the governance strategy corresponding to the cause of resource waste as the resource adjustment strategy, wherein the health status of resource application in the second level is better than that in the third level, and the health status of resource application in the third level is better than that in the fourth level.

[0063] In some embodiments, Level 1 resource applications perform well in both utilization efficiency and application state management, indicating that users or applications are making full use of allocated resources and their operations are consistent with the platform's resource management policies. In this case, if business growth or performance demands increase, the platform can prioritize providing resource expansion for such applications or users to support their further development. Level 2 resource applications, while meeting the standards for application state management, do not meet the platform's standards for resource utilization efficiency. This indicates that some allocated resources are not being fully utilized, representing potential waste. Therefore, the platform can adopt a resource reclamation strategy, reassessing the current resource allocation and, if necessary, reclaiming excess resources to improve overall resource utilization.

[0064] At Level 3, while resource utilization efficiency meets standards, application status management has issues, such as unreasonable storage space usage or a large number of slow SQL queries. The platform can focus on addressing non-compliant storage behaviors, such as data cleanup, compression, or archiving, and optimizing computing resource usage, such as SQL optimization and resource quota adjustments. Level 4 resource applications perform poorly in both utilization efficiency and application status management, representing the most critical area to monitor in the resource health scoring system. The platform needs to implement strict resource management measures, such as restricting resource requests, reclaiming resources, or adjusting priorities, while requiring users to provide detailed resource usage reports to assist in diagnosing problems and implementing comprehensive resource usage optimization measures.

[0065] For example, Table 1 shows the resource adjustment strategies corresponding to different resource application levels of computing resources.

[0066] Table 1

[0067]

[0068]

[0069] Table 2 shows the resource adjustment strategies corresponding to different resource application levels of storage resources.

[0070] Table 2

[0071]

[0072] This embodiment, through detailed resource application level classification and corresponding resource adjustment strategies, enables the big data platform to achieve refined management of resource usage, ensuring the needs of excellent users while effectively addressing resource waste and promoting efficient and fair resource allocation.

[0073] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0074] Example 2

[0075] This application also provides a resource adjustment device for a big data platform. It should be noted that this resource adjustment device can be used to execute the resource adjustment method for a big data platform provided in this application. The resource adjustment device for a big data platform provided in this application is described below.

[0076] According to an embodiment of this application, an apparatus for implementing the resource adjustment method of the above-described big data platform is also provided. Figure 4This is a schematic diagram of a resource adjustment device for a big data platform provided according to an embodiment of this application, such as... Figure 4 As shown, the device includes:

[0077] The first determining unit 401 is used to determine the resources to be adjusted in the big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources;

[0078] The acquisition unit 402 is used to acquire resource usage data of the resource to be adjusted within a preset period, and calculate the utilization efficiency evaluation value and application status evaluation value of the resource to be adjusted based on the resource usage data.

[0079] The second determining unit 403 is used to determine the resource application level based on the utilization efficiency assessment value and the application status assessment value.

[0080] Execution unit 404 is used to determine the resource adjustment strategy corresponding to the resource application level and execute the resource adjustment strategy on the big data platform.

[0081] The resource adjustment device for a big data platform provided in this application embodiment determines the resources to be adjusted in the big data platform through a first determining unit 401, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; an acquiring unit 402 acquires the resource usage data of the resources to be adjusted within a preset period, and calculates the utilization efficiency evaluation value and application status evaluation value of the resources to be adjusted based on the resource usage data; a second determining unit 403 determines the resource application level based on the utilization efficiency evaluation value and the application status evaluation value; and an execution unit 404 determines the resource adjustment strategy corresponding to the resource application level and executes the resource adjustment strategy on the big data platform. By conducting a detailed evaluation of the computing resources and storage resources of the big data platform and formulating a specific resource adjustment strategy based on the utilization efficiency evaluation value and the application status evaluation value, the device achieves the purpose of avoiding waste of cluster resources, optimizing resource allocation, and reducing costs, thereby realizing the technical effect of improving the resource utilization efficiency of the big data platform and solving the technical problem of low resource utilization efficiency of the big data platform.

[0082] Optionally, in the resource adjustment device of the big data platform provided in this application embodiment, the acquisition unit 402 includes: a first extraction module, used to extract the daily computing resource usage and computing resource allocation of the resource scheduling queue from the resource usage data, wherein the computing resource usage is the number of virtual central processing unit cores used within a preset time period, and the computing resource allocation is the number of virtual central processing unit cores allocated to the resource scheduling queue within a preset period; a first calculation module, used to calculate the ratio of computing resource usage to computing resource allocation to obtain the daily utilization efficiency, and calculate the average daily utilization efficiency within the preset period to obtain the daily average utilization efficiency; and a first determination module, used to determine the adjustment coefficient of the utilization efficiency, and calculate the product of the daily average utilization efficiency and the adjustment coefficient to obtain the utilization efficiency evaluation value.

[0083] Optionally, in the resource adjustment device of the big data platform provided in this application embodiment, the acquisition unit 402 further includes: a second extraction module, used to extract the target number of target structured query statements within a preset period from the resource usage data, wherein the target structured query statement is an execution time greater than or equal to a time threshold, or a structured query statement whose calculated resource amount is greater than or equal to a resource amount threshold; and a second determination module, used to determine the adjustment coefficient of the application status and calculate the difference between the adjustment coefficient and the target number to obtain the application status evaluation value.

[0084] Optionally, in the resource adjustment device of the big data platform provided in this application embodiment, the acquisition unit 402 further includes: a third extraction module, used to extract the storage resource usage and storage resource allocation within a preset period from the resource usage data, and determine the adjustment coefficient of utilization efficiency; and a second calculation module, used to calculate the ratio of storage resource usage and storage resource allocation, and calculate the product of the adjustment coefficient and the ratio to obtain the utilization efficiency evaluation value.

[0085] Optionally, in the resource adjustment device of the big data platform provided in this application embodiment, the acquisition unit 402 further includes: a fourth extraction module, used to extract the storage space release capacity and storage resource allocation amount within a preset period from the resource usage data, and determine the adjustment coefficient of the application status; and a third calculation module, used to calculate the ratio of the storage space release capacity and the storage resource allocation amount, calculate the product of the ratio and the adjustment coefficient, and obtain the application status evaluation value.

[0086] Optionally, in the resource adjustment device of the big data platform provided in this application embodiment, the second determining unit 403 includes: a third determining module, used to determine the resource application level as a first level when the utilization efficiency evaluation value is greater than or equal to the target evaluation value threshold and the application status evaluation value is greater than or equal to the target evaluation value threshold; a fourth determining module, used to determine the resource application level as a second level when the utilization efficiency evaluation value is less than the target evaluation value threshold and the application status evaluation value is greater than or equal to the target evaluation value threshold, wherein the health status of the resource application at the first level is better than that at the second level; a fifth determining module, used to determine the resource application level as a third level when the utilization efficiency evaluation value is greater than or equal to the target evaluation value threshold and the application status evaluation value is less than the target evaluation value threshold, wherein the health status of the resource application at the second level is better than that at the third level; and a sixth determining module, used to determine the resource application level as a fourth level when the utilization efficiency evaluation value is less than the target evaluation value threshold and the application status evaluation value is less than the target evaluation value threshold, wherein the health status of the resource application at the third level is better than that at the fourth level.

[0087] Optionally, in the resource adjustment device of the big data platform provided in this application embodiment, the execution unit 404 includes: a seventh determining module, used to determine resource expansion as a resource adjustment strategy when the resource application level is at the first level; an eighth determining module, used to determine resource recycling as a resource adjustment strategy when the resource application level is at the second level, wherein the health status of resource application at the first level is better than that at the second level; and a ninth determining module, used to determine the cause of resource waste when the resource application level is at the third or fourth level, and determine the governance strategy corresponding to the cause of resource waste as a resource adjustment strategy, wherein the health status of resource application at the second level is better than that at the third level, and the health status of resource application at the third level is better than that at the fourth level.

[0088] It should be noted that the first determining unit 401, the acquiring unit 402, the second determining unit 403, and the execution unit 404 mentioned above correspond to steps S201 to S204 in Embodiment 1. The four units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules or units can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0089] Example 3

[0090] Embodiments of this application may provide an electronic device. Figure 5This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 Only one of the components is shown: processor 502, memory 504, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module, and display.

[0091] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0092] The processor can access information and applications stored in the memory via a transmission device to perform the following steps: determine the resources to be adjusted in the big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; obtain resource usage data of the resources to be adjusted within a preset period, and calculate the utilization efficiency assessment value and application status assessment value of the resources to be adjusted based on the resource usage data; determine the resource application level based on the utilization efficiency assessment value and application status assessment value; determine the resource adjustment strategy corresponding to the resource application level, and execute the resource adjustment strategy on the big data platform.

[0093] The processor can also access information and applications stored in memory via a transmission device to perform the following steps: extracting the daily computing resource usage and computing resource allocation of the resource scheduling queue from the resource usage data, wherein the computing resource usage is the number of virtual CPU cores used within a preset duration, and the computing resource allocation is the number of virtual CPU cores allocated to the resource scheduling queue within a preset period; calculating the ratio of computing resource usage to computing resource allocation to obtain the daily utilization efficiency; calculating the average daily utilization efficiency within the preset period to obtain the daily average utilization efficiency; determining the adjustment coefficient for utilization efficiency, and calculating the product of the daily average utilization efficiency and the adjustment coefficient to obtain the utilization efficiency evaluation value.

[0094] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: extract the target number of target structured query statements within a preset period from the resource usage data, wherein the target structured query statement is an execution time greater than or equal to a time threshold, or a structured query statement whose calculated resource amount is greater than or equal to a resource amount threshold; determine the adjustment coefficient of the application status, and calculate the difference between the adjustment coefficient and the target number to obtain the application status evaluation value.

[0095] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: extract the storage resource usage and storage resource allocation within a preset period from the resource usage data, and determine the adjustment coefficient of utilization efficiency; calculate the ratio of storage resource usage to storage resource allocation, and calculate the product of the adjustment coefficient and the ratio to obtain the utilization efficiency evaluation value.

[0096] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: extract the storage space release capacity and storage resource allocation amount within a preset period from the resource usage data, and determine the adjustment coefficient of the application status; calculate the ratio of the storage space release capacity and the storage resource allocation amount, and calculate the product of the ratio and the adjustment coefficient to obtain the application status evaluation value.

[0097] The processor can also invoke information and applications stored in memory via a transmission device to perform the following steps: If the utilization efficiency assessment value is greater than or equal to the target assessment value threshold, and the application status assessment value is greater than or equal to the target assessment value threshold, determine the resource application level as Level 1; if the utilization efficiency assessment value is less than the target assessment value threshold, and the application status assessment value is greater than or equal to the target assessment value threshold, determine the resource application level as Level 2, wherein the health status of the resource application at Level 1 is better than that at Level 2; if the utilization efficiency assessment value is greater than or equal to the target assessment value threshold, and the application status assessment value is less than the target assessment value threshold, determine the resource application level as Level 3, wherein the health status of the resource application at Level 2 is better than that at Level 3; if the utilization efficiency assessment value is less than the target assessment value threshold, and the application status assessment value is less than the target assessment value threshold, determine the resource application level as Level 4, wherein the health status of the resource application at Level 3 is better than that at Level 4.

[0098] The processor can also access information and applications stored in the memory via a transmission device to perform the following steps: when the resource application level is Level 1, resource expansion is determined as a resource adjustment strategy; when the resource application level is Level 2, resource reclamation is determined as a resource adjustment strategy, wherein the health status of resource application at Level 1 is better than that at Level 2; when the resource application level is Level 3 or Level 4, the causes of resource waste are determined, and the corresponding governance strategies for the causes of resource waste are determined as resource adjustment strategies, wherein the health status of resource application at Level 2 is better than that at Level 3, and the health status of resource application at Level 3 is better than that at Level 4.

[0099] This application provides a method for determining resources to be adjusted in a big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; acquiring resource usage data of the resources to be adjusted within a preset period; calculating the utilization efficiency assessment value and application status assessment value of the resources to be adjusted based on the resource usage data; determining the resource application level based on the utilization efficiency assessment value and application status assessment value; determining the resource adjustment strategy corresponding to the resource application level; and executing the resource adjustment strategy on the big data platform. By conducting a detailed evaluation of the computing and storage resources of the big data platform and formulating specific resource adjustment strategies based on the utilization efficiency assessment value and application status assessment value, the method achieves the goals of avoiding waste of cluster resources, optimizing resource allocation, and reducing costs, thereby improving the resource utilization efficiency of the big data platform and solving the technical problem of low resource utilization efficiency of the big data platform.

[0100] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0101] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0102] Example 4

[0103] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the resource adjustment method of the big data platform provided in Embodiment 1.

[0104] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0105] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing resource adjustment method steps of a big data platform.

[0106] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0107] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0112] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A resource adjustment method for a big data platform, characterized in that, include: The resources to be adjusted in the big data platform are identified, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; Obtain resource usage data of the resource to be adjusted within a preset period, and calculate the utilization efficiency evaluation value and application status evaluation value of the resource to be adjusted based on the resource usage data; The resource application level is determined based on the utilization efficiency assessment value and the application status assessment value. Determine the resource adjustment strategy corresponding to the resource application level, and execute the resource adjustment strategy on the big data platform.

2. The method according to claim 1, characterized in that, When the resource to be adjusted includes the computing resource, calculating the utilization efficiency assessment value of the resource to be adjusted based on the resource usage data includes: The daily computing resource usage and computing resource allocation of the resource scheduling queue are extracted from the resource usage data. The computing resource usage is the number of virtual CPU cores used within a preset time period, and the computing resource allocation is the number of virtual CPU cores allocated to the resource scheduling queue within the preset period. Calculate the ratio of the computing resource usage to the computing resource allocation to obtain the daily utilization efficiency; calculate the average daily utilization efficiency within the preset period to obtain the average daily utilization efficiency. Determine the adjustment coefficient for utilization efficiency, and calculate the product of the average daily utilization efficiency and the adjustment coefficient to obtain the utilization efficiency evaluation value.

3. The method according to claim 1, characterized in that, When the resource to be adjusted includes the computing resource, calculating the application status evaluation value of the resource to be adjusted based on the resource usage data includes: Extract the target number of target structured query statements within the preset period from the resource usage data, wherein the target structured query statement is either an execution time greater than or equal to a time threshold, or a structured query statement whose calculated resource amount is greater than or equal to a resource amount threshold. Determine the adjustment coefficient for the application status, and calculate the difference between the adjustment coefficient and the target quantity to obtain the application status evaluation value.

4. The method according to claim 1, characterized in that, When the resource to be adjusted includes the storage resource, calculating the utilization efficiency evaluation value of the resource to be adjusted based on the resource usage data includes: Extract the storage resource usage and storage resource allocation within the preset period from the resource usage data, and determine the adjustment coefficient for utilization efficiency; Calculate the ratio of the storage resource usage to the storage resource allocation, and then calculate the product of the adjustment coefficient and the ratio to obtain the utilization efficiency evaluation value.

5. The method according to claim 1, characterized in that, When the resource to be adjusted includes the storage resource, calculating the application status evaluation value of the resource to be adjusted based on the resource usage data includes: Extract the storage space release capacity and storage resource allocation within the preset period from the resource usage data to determine the application status adjustment coefficient; Calculate the ratio of the released storage space capacity to the allocated storage resources, and multiply the ratio by the adjustment coefficient to obtain the application status evaluation value.

6. The method according to any one of claims 1-5, characterized in that, Determining the resource application level based on the utilization efficiency assessment value and the application status assessment value includes: If the utilization efficiency assessment value is greater than or equal to the target assessment value threshold, and the application status assessment value is greater than or equal to the target assessment value threshold, the resource application level is determined to be the first level. If the utilization efficiency assessment value is less than the target assessment value threshold and the application status assessment value is greater than or equal to the target assessment value threshold, the resource application level is determined to be the second level, wherein the health status of the resource application at the first level is better than that at the second level. If the utilization efficiency assessment value is greater than or equal to the target assessment value threshold, and the application status assessment value is less than the target assessment value threshold, the resource application level is determined to be the third level, wherein the health status of the resource application at the second level is better than that at the third level. If the utilization efficiency assessment value is less than the target assessment value threshold and the application status assessment value is less than the target assessment value threshold, the resource application level is determined to be level four, wherein the health status of the resource application at level three is better than that at level four.

7. The method according to any one of claims 1-5, characterized in that, Determining the resource adjustment strategy corresponding to the resource application level includes: When the resource application level is at the first level, resource expansion is determined as the resource adjustment strategy; When the resource application level is the second level, resource recycling is determined as the resource adjustment strategy, wherein the health status of resource application at the first level is better than that at the second level. When the resource application level is level three or level four, the cause of resource waste is determined, and the governance strategy corresponding to the cause of resource waste is determined as the resource adjustment strategy. In this case, the health status of resource application at level two is better than that at level three, and the health status of resource application at level three is better than that at level four.

8. A resource adjustment device for a big data platform, characterized in that, include: The first determining unit is used to determine the resources to be adjusted in the big data platform, wherein the resources to be adjusted include at least one of the following: computing resources and storage resources; The acquisition unit is used to acquire resource usage data of the resource to be adjusted within a preset period, and calculate the utilization efficiency evaluation value and application status evaluation value of the resource to be adjusted based on the resource usage data. The second determining unit is used to determine the resource application level based on the utilization efficiency assessment value and the application status assessment value; An execution unit is used to determine the resource adjustment strategy corresponding to the resource application level and execute the resource adjustment strategy on the big data platform.

9. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program executes the resource adjustment method of the big data platform according to any one of claims 1 to 7 when it runs.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the resource adjustment method of the big data platform according to any one of claims 1 to 7.

Images