IaaS resource data analysis method and system, electronic device, and storage medium
Patent Information
- Application Number
- CN202610680250.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-18
- Publication Date
- 2026-09-01
AI Technical Summary
单租户模式需分别对接各云厂商接口,数据整合效率低;多租户共库模式下,资源与成本分摊易失真,数据库存储计量未合并索引长度,误差率高
[0030]本发明的附加方面和优点将在下面的描述部分中变得明显,或通过本发明的实践了解到。
Smart Images

Figure CN122673239A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and more specifically, to an IaaS resource data analysis method and system, electronic device, and storage medium. Background Technology
[0002] As enterprises deepen their digital transformation, SMEs are widely adopting subscription-based SaaS (Software as a Service) services on the cloud. This necessitates cost accounting, usage statistics, and trend forecasting for IaaS (Infrastructure as a Service) resources such as database storage, object storage, and network bandwidth. Currently, SaaS service providers generally adopt multi-cloud deployment architectures, and the inconsistent interface standards and data formats among different cloud vendors lead to high data collection and integration costs and poor reusability.
[0003] The relevant technologies are mainly divided into two modes: single-tenant independent multi-cloud deployment and multi-tenant shared database multi-cloud deployment. The single-tenant mode requires separate interfaces with each cloud vendor, resulting in low data integration efficiency. In the multi-tenant shared database mode, resource and cost allocation are prone to distortion, and database storage metering does not merge index lengths, leading to high error rates. Bandwidth calculation relies on a large number of big data nodes processing cloud vendor access logs, resulting in high latency, high costs, and incomplete data. Furthermore, existing solutions generally lack unified data standards and intelligent prediction capabilities, do not support standardized interface with MCP (Model Context Protocol) intelligent agents, and suffer from severe data silos, failing to meet the needs of upper-layer intelligent applications and global decision-making.
[0004] As a result, existing IaaS resource data analysis solutions suffer from defects such as inconsistent interfaces, inconsistent data formats, lack of trend prediction, and data silos, making it difficult to achieve accurate collection, unified calculation, intelligent prediction, and standardized services of resource data in multi-cloud and multi-tenant environments. Summary of the Invention
[0005] The present invention aims to solve the technical problems existing in the prior art or related technologies.
[0006] Therefore, the first aspect of the present invention proposes an IaaS resource data analysis method.
[0007] A second aspect of the present invention proposes an IaaS resource data analysis system.
[0008] A third aspect of the present invention provides an electronic device.
[0009] A fourth aspect of the present invention provides a storage medium.
[0010] In view of this, according to the first aspect of the present invention, an IaaS resource data analysis method is proposed, wherein the IaaS resource data analysis method includes: acquiring multi-source tenant resource data and unifying the format of the multi-source tenant resource data, the multi-source tenant resource data including tenant database storage data, tenant object storage data, tenant CDN (Content Delivery Network) traffic data, tenant Nginx (a high-performance HTTP (Hypertext Transfer Protocol) and reverse proxy server, commonly used for load balancing, static resource services and high-concurrency scenarios) access traffic data, and tenant IaaS resource cost data, the tenant IaaS resource cost data including: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost; determining the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on the tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data; and determining the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio. The total tenant cost is calculated from the total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. The total tenant cost includes database cost, object storage cost, bandwidth cost, and total resource data cost. Based on multi-source tenant resource data and a univariate linear regression time series model, the growth trend of each tenant's resource and the predicted cost of each resource are predicted for a future period. In response to the agent's query request, structured retrieval results and a mapping dataset are obtained based on the growth trend of each resource, the predicted cost of each resource, and the total tenant cost. These structured retrieval results and the mapping dataset are then fed back to the agent through the model context protocol interface.
[0011] The IaaS resource data analysis method provided by this invention includes: firstly, acquiring multi-source tenant resource data, wherein the multi-source tenant resource data includes: tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data. The tenant IaaS resource cost data includes: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. Tenant database storage data refers to the actual storage usage of the tenant in the database, which can be collected by a database collector. Tenant object storage data refers to the storage usage of the tenant in the object storage service, which can be collected by an object storage collector. Tenant CDN traffic data refers to the traffic generated when the tenant uses a Content Delivery Network (CDN) to accelerate static resources, which can be collected by a CDN collector. Tenant Nginx access traffic data refers to the traffic generated when the tenant accesses non-static resources (such as APIs) through high-performance HTTP and reverse proxy servers, which can be collected by an Nginx collector. Tenant IaaS resource cost data refers to the total cost of Infrastructure as a Service (IaaS) resources shared by multiple tenants. Total database cost refers to the cost of database services shared by all tenants. Total object storage cost refers to the cost of object storage services shared by all tenants. Total CDN acceleration cost refers to the cost of CDN acceleration services shared by all tenants. Total bandwidth cost refers to the cost of network bandwidth shared by all tenants. After obtaining multi-source tenant resource data, the multi-source tenant resource data is converted into a unified JavaScript object representation (JSON) format containing tenant ID, cloud vendor, and resource type. Then, based on the converted tenant database storage data, the single-tenant storage ratio is determined; based on the converted tenant object storage data, the object storage ratio is determined; based on the tenant CDN traffic data, the CDN traffic ratio is determined; and based on the tenant Nginx access traffic data, the Nginx bandwidth ratio is determined. Finally, based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost, the total tenant cost result is calculated. The total tenant cost result includes: database cost, object storage cost, bandwidth cost, and total resource data cost. Subsequently, based on multi-source tenant resource data and a univariate linear regression time series model, the growth trend of various tenant resources and the corresponding predicted costs are predicted in the future. The univariate linear regression time series model is a statistical model that uses historical time series data to fit a linear trend and is used to predict future values.Finally, responding to the agent's query request, the system generates structured retrieval results and a mapping dataset based on growth trends, predicted costs, and total tenant costs. The structured retrieval results are JavaScript object representation data containing entity URIs, attribute values, and relationship chains. The mapping dataset is a dataset showing the relationships between tenant resources and cloud vendors, and between costs and percentages. This structured data can be directly parsed by the agent and used for upper-level intelligent applications such as resource expansion planning, cost warnings, and operational decision-making. Finally, it is fed back to the agent through the MCP (Model Context Protocol) interface. This invention shields the differences between multiple cloud vendor interfaces through the Model Context Protocol interface, converting multi-source resource data into a unified JSON format containing tenant IDs, cloud vendors, and resource types, solving the problems of inconsistent interface standards and fragmented data formats, and reducing integration costs. Simultaneously, it predicts resource growth trends and costs over a future period based on a univariate linear regression time series model, addressing the lack of growth prediction and cost estimation capabilities. Furthermore, this invention unifies storage, bandwidth, object storage, costs, and prediction time to form a global data energy, solving the problem of data being scattered across different systems and difficult to support global analysis. This invention solves the problems in related technologies such as inconsistent interface standards, distorted cost sharing among multi-tenants, high bandwidth computing costs and high latency, insufficient intelligent agent docking capabilities, lack of trend prediction, data silos, and poor compatibility among multiple deployment modes, and achieves unified collection, accurate allocation, intelligent prediction and efficient interaction of multi-cloud resources.
[0012] In some technical solutions, optionally, the steps for acquiring multi-source tenant resource data include: using a database collector and SQL (Structured Query Language). Using a structured query language (SCL), the system accesses multiple public cloud databases to retrieve information pattern tables, obtains the total storage size and number of rows, and determines the tenant's database storage data based on these values. It then uses an object storage collector to call the model context protocol interfaces of various public cloud object storage systems to obtain a list of folders named by tenant ID, and uses the storage size of each folder in the list to determine the tenant's object storage data. Finally, it subscribes to logs reported by the front-end event tracking software development kit in a distributed message queue using a CDN collector, parses and accumulates the static resource response body size, and determines the tenant's CDN traffic data based on this size. Similarly, it subscribes to logs in the Nginx access log topic in a distributed message queue using an Nginx collector, parses and accumulates the response body size, and determines the tenant's Nginx access traffic data based on this size. Finally, it calls the application programming interfaces or model context protocol interfaces of all public clouds using a cost collector, and determines the total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost based on the returned data.
[0013] In this technical solution, the steps for acquiring multi-source tenant resource data include: firstly, a database collector, i.e., a component used to collect database storage data, is started periodically every week. Before accessing the databases of multiple public clouds (i.e., multi-cloud vendors), table analysis statements are used to refresh the storage space to obtain the latest data. The table analysis statement can be ANALYZE. The system then uses SQL to read the information schema table (information_schema, a collection of system databases / views built into relational databases, storing metadata such as table names, fields, and storage size). Data is merged using data length and index length to avoid a 20% error rate in table row count. The total storage size and number of rows are then obtained to determine the tenant's database storage data. Data length can be DATA_LENGTH, a field in the MySQL information_schema.TABLES table representing the actual number of bytes used for storing data. Index length can be INDEX_LENGTH, also a field in the MySQL information_schema.TABLES table representing the total number of bytes used for all indexes. The estimated number of rows can be TABLE_ROWS. Subsequently, an object storage collector, a component that collects object storage data, is started daily. It calls the model context protocol interfaces of various public clouds (i.e., cloud vendors' object storage) to obtain a list of folders named by tenant ID. The storage size is then determined based on the folder names to identify the tenant's object storage data. Subsequently, the CDN collector, a component that collects CDN traffic data, subscribes to logs reported by the front-end event tracking software development kit (SDK) in a distributed message queue. It parses and accumulates the static resource response body size to determine the tenant's CDN traffic data. The distributed message queue can be Kafka, an open-source distributed event streaming platform used for high-throughput, low-latency real-time data transmission. The Nginx collector, a component that collects Nginx access traffic data, subscribes to logs from Nginx access log topics (a logical concept in Kafka used for classifying, storing, and managing messages) in the distributed message queue. It parses and accumulates the response body size to determine the tenant's Nginx access traffic data. Finally, the cost collector, a component used to collect IaaS resource cost data, starts daily and calls the cloud vendor's application programming interface (API) or model context protocol (MGP) to determine the total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost based on the returned data.This invention avoids storage data errors by merging table analysis statements, information pattern tables, and data length + index length. It achieves low-resource-investment, low-latency bandwidth acquisition by subscribing to logs through a distributed message queue. It also reduces interface adaptation costs by uniformly connecting to cloud vendors through a model context protocol interface.
[0014] In some technical solutions, optionally, the steps of determining the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data include: obtaining total storage volume, total traffic, and total object storage; determining the single-tenant storage ratio based on tenant database storage data and total storage volume; determining the CDN traffic ratio based on tenant CDN traffic data and total traffic; determining the Nginx bandwidth ratio based on tenant Nginx access traffic data and total traffic; and determining the object storage ratio based on tenant object storage data and total object storage.
[0015] In this technical solution, the steps for determining the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data include: First, obtaining the total storage volume, total traffic, and total object storage, where the total storage volume is the sum of the database storage sizes of all tenants, the total traffic is the sum of the CDN traffic and Nginx access traffic of all tenants, and the total object storage is the sum of the object storage sizes of all tenants; Next, determining the single-tenant storage ratio based on the tenant database storage data and the total storage volume, by dividing the tenant database storage data by the total storage volume; Determining the CDN traffic ratio based on the tenant CDN traffic data and the total traffic, by dividing the CDN traffic data by the total traffic; Determining the Nginx bandwidth ratio based on the tenant Nginx access traffic data and the total traffic, by dividing the Nginx access traffic data by the total traffic; Determining the object storage ratio based on the tenant object storage data and the total object storage, by dividing the tenant object storage data by the total object storage. This invention achieves standardization and precision in percentage calculation, providing accurate input for multi-tenant cost allocation and solving the problem of resource allocation distortion in multi-tenant shared database mode.
[0016] In some technical solutions, optionally, the step of calculating the total tenant cost based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost includes: determining the database cost based on the single-tenant storage ratio and total database cost; determining the object storage cost based on the object storage ratio and total object storage cost; determining the CDN acceleration cost based on the CDN traffic ratio and total CDN acceleration cost; determining the Nginx bandwidth cost based on the Nginx bandwidth ratio and total bandwidth cost; determining the bandwidth cost based on the CDN acceleration cost and Nginx bandwidth cost; and determining the total resource data cost based on the database cost, object storage cost, and bandwidth cost.
[0017] In this technical solution, the steps for calculating the total tenant cost based on single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost include: determining the database cost based on the single-tenant storage ratio and total database cost, using the formula: database cost equals single-tenant storage ratio multiplied by total database cost; determining the object storage cost based on the object storage ratio and total object storage cost, using the formula: object storage cost equals object storage ratio multiplied by total object storage cost; and determining the total cost based on the CDN traffic ratio and total CDN acceleration cost. The CDN acceleration cost is calculated as follows: CDN acceleration cost equals CDN traffic share multiplied by total CDN acceleration cost. The Nginx bandwidth cost is determined based on the Nginx bandwidth share and total bandwidth cost, calculated as follows: Nginx bandwidth cost equals Nginx bandwidth share multiplied by total bandwidth cost. The bandwidth cost is also determined based on CDN acceleration cost and Nginx bandwidth cost, calculated as follows: Bandwidth cost equals CDN acceleration cost plus Nginx bandwidth cost. The total resource data cost is determined based on database cost, object storage cost, and bandwidth cost, calculated as follows: Total resource data cost equals database cost plus object storage cost plus bandwidth cost. This invention achieves accurate tenant-level cost allocation, solving the problem of cost allocation distortion in multi-tenant shared database models in related technologies, and providing data support for tenant revenue calculation.
[0018] In some technical solutions, optionally, the step of predicting the growth trend of each resource of a tenant and the corresponding prediction cost of each resource in the future period based on multi-source tenant resource data and a univariate linear regression time series model includes: predicting the growth trend of the tenant's database storage in the future period based on tenant database storage data and a univariate linear regression time series model; predicting the growth trend of the tenant's object storage in the future period based on tenant object storage data and a univariate linear regression time series model; predicting the growth trend of the tenant's CDN traffic in the future period based on tenant CDN traffic data and a univariate linear regression time series model; predicting the growth trend of the tenant's Nginx access traffic in the future period based on tenant Nginx access traffic data and a univariate linear regression time series model; and determining the prediction cost corresponding to each resource based on the growth trends of database storage, object storage, CDN traffic, and Nginx access traffic, respectively.
[0019] This technical solution involves predicting the growth trend of each tenant's resource and its corresponding predicted cost over a future period based on multi-source tenant resource data and a univariate linear regression time series model. The steps include: predicting the tenant's database storage growth trend based on tenant database storage data and a univariate linear regression time series model; predicting the tenant's object storage growth trend based on tenant object storage data and a univariate linear regression time series model; predicting the tenant's CDN traffic growth trend based on tenant CDN traffic data and a univariate linear regression time series model; predicting the tenant's Nginx access traffic growth trend based on tenant Nginx access traffic data and a univariate linear regression time series model; and determining the predicted cost for each resource based on the database storage growth trend, object storage growth trend, CDN traffic growth trend, and Nginx access traffic growth trend. The predicted cost for each resource can be obtained by multiplying the resource growth trend by the unit resource cost. This invention solves the problem of lacking data growth trend prediction and resource cost estimation capabilities in existing technologies, improving the rationality and foresight of resource planning.
[0020] In some technical solutions, optionally, the step of obtaining structured retrieval results and a mapping dataset in response to a query request from an agent, based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost of the tenant, includes: receiving the agent's query request through a model context protocol interface or a traditional open API specification interface; verifying the agent's permissions according to a preset open authorization protocol; verifying the validity and legality of the agent's access token based on the agent's permissions as the target permissions; and, based on the fact that both validity and legality verifications pass, obtaining structured retrieval results and a mapping dataset in response to the agent's query request, based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost of the tenant.
[0021] In this technical solution, the steps of obtaining structured retrieval results and mapping datasets in response to a query request from an intelligent agent, based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost of the tenant, include: receiving the intelligent agent's query request through a model context protocol interface or a traditional open API specification interface; verifying the intelligent agent's permissions according to a preset OAuth (Open Authorization) protocol; verifying the validity and legality of the intelligent agent's access token (a temporary access credential issued in the Open Authorization protocol) when the intelligent agent's permissions are the target permissions, i.e., the intelligent agent's permissions are high permissions; and, based on the successful verification of both validity and legality, obtaining structured retrieval results and mapping datasets in response to the intelligent agent's query request, based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost of the tenant. This invention achieves secure, flexible, and standardized interaction for intelligent agent queries, supports query needs in different scenarios (AI intelligent agents and ordinary programs), and solves the problem of insufficient intelligent agent integration capabilities in related technologies.
[0022] In some technical solutions, the step of acquiring multi-source tenant resource data may optionally include: acquiring multi-source tenant resource data at preset intervals, wherein the acquisition period is different for different resources.
[0023] In this technical solution, the step of acquiring multi-source tenant resource data includes: acquiring multi-source tenant resource data at preset intervals, wherein the acquisition cycle for different resources is different. For example, tenant database storage data is collected once a week, object storage data and IaaS resource cost data are collected once a day, and the total tenant cost result is calculated once a day. This invention achieves timeliness of data acquisition and resource optimization, balances data freshness and acquisition cost, and avoids unnecessary resource waste.
[0024] According to a second aspect of the present invention, an IaaS resource data analysis system is proposed, comprising: a first processing module, which is used to acquire multi-source tenant resource data and unify the format of the multi-source tenant resource data, the multi-source tenant resource data including tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data, the tenant IaaS resource cost data including: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost; a second processing module, which is used to determine the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on the tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data; and a third processing module, which... The first module calculates the total tenant cost based on single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. The total tenant cost includes database cost, object storage cost, bandwidth cost, and total resource data cost. The second module predicts the growth trend of each tenant's resource and its corresponding predicted cost over a future period based on multi-source tenant resource data and a univariate linear regression time series model. The third module responds to the agent's query request by obtaining structured retrieval results and a mapping dataset based on the growth trend of each resource, its corresponding predicted cost, and the total tenant cost. It then feeds back the structured retrieval results and mapping dataset to the agent through the model context protocol interface.
[0025] The IaaS resource data analysis system provided by this invention includes: a first processing module, a second processing module, a third processing module, a fourth processing module, and a fifth processing module. The first processing module first acquires multi-source tenant resource data, which includes: tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data. The tenant IaaS resource cost data includes: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. Tenant database storage data refers to the actual storage usage of the tenant in the database, which can be collected by a database collector. Tenant object storage data refers to the storage usage of the tenant in the object storage service, which can be collected by an object storage collector. Tenant CDN traffic data refers to the traffic generated when the tenant uses a Content Delivery Network (CDN) to accelerate static resources, which can be collected by a CDN collector. Tenant Nginx access traffic data refers to the traffic generated when the tenant accesses non-static resources through high-performance HTTP and reverse proxy servers, which can be collected by an Nginx collector. Tenant IaaS resource cost data refers to the total cost of Infrastructure as a Service (IaaS) resources shared by multiple tenants. Total database cost refers to the cost of database services shared by all tenants. Total object storage cost refers to the cost of object storage services shared by all tenants. Total CDN acceleration cost refers to the cost of CDN acceleration services shared by all tenants. Total bandwidth cost refers to the cost of network bandwidth shared by all tenants. After obtaining multi-source tenant resource data, the multi-source tenant resource data is converted into a unified JavaScript object representation (JSON) format containing tenant ID, cloud vendor, and resource type. The second processing module then determines the single-tenant storage ratio based on the converted tenant database storage data, the object storage ratio based on the converted tenant object storage data, the CDN traffic ratio based on the tenant CDN traffic data, and the Nginx bandwidth ratio based on the tenant Nginx access traffic data. The third processing module then calculates the total tenant cost result based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. The total tenant cost result includes: database cost, object storage cost, bandwidth cost, and total resource data cost. Subsequently, the fourth processing module predicts the growth trend of various tenant resources and corresponding predicted costs in the future based on multi-source tenant resource data and a univariate linear regression time series model. The univariate linear regression time series model is a statistical model that uses historical time series data to fit a linear trend and is used to predict future values.Finally, the fifth processing module responds to the agent's query request, generating structured search results and a mapping dataset based on growth trends, predicted costs, and total tenant costs. The structured search results are JavaScript object representation data containing entity URIs, attribute values, and relationship chains. The mapping dataset is a dataset showing the relationships between tenant resources and cloud vendors, and between costs and percentages. This structured data can be directly parsed by the agent and used for upper-level intelligent applications such as resource expansion planning, cost warnings, and operational decision-making. Finally, it is fed back to the agent through the MCP (Model Context Protocol) interface. This invention shields the differences between multiple cloud vendor interfaces through the Model Context Protocol interface, converting multi-source resource data into a unified JSON format containing tenant IDs, cloud vendors, and resource types, solving the problems of inconsistent interface standards and fragmented data formats, and reducing integration costs. Simultaneously, it predicts resource growth trends and costs over a future period based on a univariate linear regression time series model, addressing the lack of growth prediction and cost estimation capabilities. Furthermore, this invention unifies storage, bandwidth, object storage, costs, and prediction time to form a global data energy, solving the problem of data being scattered across different systems and difficult to support global analysis. This invention solves the problems in related technologies such as inconsistent interface standards, distorted cost sharing among multi-tenants, high bandwidth computing costs and high latency, insufficient intelligent agent docking capabilities, lack of trend prediction, data silos, and poor compatibility among multiple deployment modes, and achieves unified collection, accurate allocation, intelligent prediction and efficient interaction of multi-cloud resources.
[0026] According to a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the IaaS resource data analysis method as described above.
[0027] The electronic device provided by this invention, when the processor executes a computer program, implements the steps of the above-mentioned IaaS resource data analysis method, and can achieve the technical effects of any of the above technical solutions, which will not be elaborated here.
[0028] According to a fourth aspect of the invention, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the steps of the IaaS resource data analysis method as described above.
[0029] The storage medium provided by this invention enables the computer program to implement the steps of the above-described IaaS resource data analysis method when executed by a processor, and can achieve the technical effects of any of the above technical solutions, which will not be elaborated here.
[0030] Additional aspects and advantages of the invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description
[0031] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:
[0032] Figure 1 A flowchart illustrating an embodiment of the IaaS resource data analysis method of the present invention is shown.
[0033] Figure 2 This diagram illustrates a step in obtaining multi-source tenant resource data in an IaaS resource data analysis method according to an embodiment of the present invention.
[0034] Figure 3 The diagram illustrates a flowchart of the steps in an IaaS resource data analysis method according to an embodiment of the present invention to determine the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data.
[0035] Figure 4 This diagram illustrates a step in the IaaS resource data analysis method of an embodiment of the present invention to calculate the total cost of a tenant based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost.
[0036] Figure 5 The diagram illustrates a step in an IaaS resource data analysis method according to an embodiment of the present invention: predicting the growth trend of each resource of a tenant and the corresponding prediction cost of each resource in a future period based on multi-source tenant resource data and a univariate linear regression time series model.
[0037] Figure 6 This diagram illustrates a step in an IaaS resource data analysis method according to an embodiment of the present invention: responding to a query request from an agent, the method obtains structured retrieval results and a mapping dataset based on the growth trend of each resource, the predicted cost of each resource, and the total cost of the tenant.
[0038] Figure 7 A structural block diagram of an IaaS resource data analysis system according to an embodiment of the present invention is shown;
[0039] Figure 8 A layered architecture diagram of an IaaS resource data analysis system according to an embodiment of the present invention is shown.
[0040] Figure 9An architectural design diagram of an IaaS resource data analysis system according to an embodiment of the present invention is shown;
[0041] Figure 10 A key process timing diagram of an IaaS resource data analysis system according to an embodiment of the present invention is shown. Detailed Implementation
[0042] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.
[0043] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0044] Figure 1 A flowchart illustrating an embodiment of the IaaS resource data analysis method of the present invention is shown. The IaaS resource data analysis method includes:
[0045] Step 102: Obtain multi-source tenant resource data and unify the format of the multi-source tenant resource data. The multi-source tenant resource data includes tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data. The tenant IaaS resource cost data includes: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost.
[0046] Step 104: Determine the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on the tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data.
[0047] Step 106: Calculate the total tenant cost result based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. The total tenant cost result includes: database cost, object storage cost, bandwidth cost, and total resource data cost.
[0048] Step 108: Based on multi-source tenant resource data and a univariate linear regression time series model, predict the growth trend of each tenant's resource and the corresponding predicted cost of each resource in the future period;
[0049] Step 110: In response to the agent's query request, obtain structured retrieval results and mapping datasets based on the growth trend of each resource, the predicted cost of each resource, and the total cost of the tenant, and feed back the structured retrieval results and mapping datasets to the agent through the model context protocol interface.
[0050] The IaaS resource data analysis method provided by this invention includes: firstly, acquiring multi-source tenant resource data, wherein the multi-source tenant resource data includes: tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data. The tenant IaaS resource cost data includes: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. Tenant database storage data refers to the actual storage usage of the tenant in the database, which can be collected by a database collector. Tenant object storage data refers to the storage usage of the tenant in the object storage service, which can be collected by an object storage collector. Tenant CDN traffic data refers to the traffic generated when the tenant uses a Content Delivery Network (CDN) to accelerate static resources, which can be collected by a CDN collector. Tenant Nginx access traffic data refers to the traffic generated when the tenant accesses non-static resources (such as APIs) through high-performance HTTP and reverse proxy servers, which can be collected by an Nginx collector. Tenant IaaS resource cost data refers to the total cost of Infrastructure as a Service (IaaS) resources shared by multiple tenants. Total database cost refers to the cost of database services shared by all tenants. Total object storage cost refers to the cost of object storage services shared by all tenants. Total CDN acceleration cost refers to the cost of CDN acceleration services shared by all tenants. Total bandwidth cost refers to the cost of network bandwidth shared by all tenants. After obtaining multi-source tenant resource data, the multi-source tenant resource data is converted into a unified JavaScript object representation (JSON) format containing tenant ID, cloud vendor, and resource type. Then, based on the converted tenant database storage data, the single-tenant storage ratio is determined; based on the converted tenant object storage data, the object storage ratio is determined; based on the tenant CDN traffic data, the CDN traffic ratio is determined; and based on the tenant Nginx access traffic data, the Nginx bandwidth ratio is determined. Finally, based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost, the total tenant cost result is calculated. The total tenant cost result includes: database cost, object storage cost, bandwidth cost, and total resource data cost. Subsequently, based on multi-source tenant resource data and a univariate linear regression time series model, the growth trend of various tenant resources and the corresponding predicted costs are predicted in the future. The univariate linear regression time series model is a statistical model that uses historical time series data to fit a linear trend and is used to predict future values.Finally, responding to the agent's query request, the system generates structured retrieval results and a mapping dataset based on growth trends, predicted costs, and total tenant costs. The structured retrieval results are JavaScript object representation data containing entity URIs, attribute values, and relationship chains. The mapping dataset is a dataset showing the relationships between tenant resources and cloud vendors, and between costs and percentages. This structured data can be directly parsed by the agent and used for upper-level intelligent applications such as resource expansion planning, cost warnings, and operational decision-making. Finally, it is fed back to the agent through the MCP (Model Context Protocol) interface. This invention shields the differences between multiple cloud vendor interfaces through the Model Context Protocol interface, converting multi-source resource data into a unified JSON format containing tenant IDs, cloud vendors, and resource types, solving the problems of inconsistent interface standards and fragmented data formats, and reducing integration costs. Simultaneously, it predicts resource growth trends and costs over a future period based on a univariate linear regression time series model, addressing the lack of growth prediction and cost estimation capabilities. Furthermore, this invention unifies storage, bandwidth, object storage, costs, and prediction time to form a global data energy, solving the problem of data being scattered across different systems and difficult to support global analysis. This invention solves the problems in related technologies such as inconsistent interface standards, distorted cost sharing among multi-tenants, high bandwidth computing costs and high latency, insufficient intelligent agent docking capabilities, lack of trend prediction, data silos, and poor compatibility among multiple deployment modes, and achieves unified collection, accurate allocation, intelligent prediction and efficient interaction of multi-cloud resources.
[0051] Figure 2 The diagram illustrates a flowchart of the steps for obtaining multi-source tenant resource data in an IaaS resource data analysis method according to an embodiment of the present invention; wherein, the steps for obtaining multi-source tenant resource data include:
[0052] Step 202: Access the read information pattern tables in multiple public cloud databases through the database collector and SQL to obtain the total storage size and number of rows of the tables, and determine the tenant database storage data based on the total storage size and number of rows of the tables;
[0053] Step 204: Call the model context protocol interface of the object storage of each public cloud through the object storage collector to obtain a list of folders named by tenant ID, obtain the storage size according to the folders in the folder list, and determine the tenant object storage data;
[0054] Step 206: Subscribe to the logs reported by the front-end instrumentation software development kit in the distributed message queue through the CDN collector, parse and accumulate the static resource response body size, and determine the tenant CDN traffic data based on the static resource response body size;
[0055] Step 208: Subscribe to the Nginx access log topic in the distributed message queue through the Nginx collector, parse and accumulate the response body size, and determine the tenant's Nginx access traffic data based on the response body size;
[0056] Step 210: Use the cost collector to call the application programming interfaces or model context protocol interfaces of all public clouds, and determine the total cost of the database, the total cost of object storage, the total cost of CDN acceleration, and the total bandwidth cost based on the returned data.
[0057] In this technical solution, the steps for acquiring multi-source tenant resource data include: firstly, a database collector, i.e., a component used to collect database storage data, is started periodically every week. Before accessing the databases of multiple public clouds (i.e., multi-cloud vendors), table analysis statements are used to refresh the storage space to obtain the latest data. The table analysis statement can be ANALYZE. The system then uses SQL to read the information schema table (information_schema, a collection of system databases / views built into relational databases, storing metadata such as table names, fields, and storage size). Data is merged using data length and index length to avoid a 20% error rate in table row count. The total storage size and number of rows are then obtained to determine the tenant's database storage data. Data length can be DATA_LENGTH, a field in the MySQL information_schema.TABLES table representing the actual number of bytes used for storing data. Index length can be INDEX_LENGTH, also a field in the MySQL information_schema.TABLES table representing the total number of bytes used for all indexes. The estimated number of rows can be TABLE_ROWS. Subsequently, an object storage collector, a component that collects object storage data, is started daily. It calls the model context protocol interfaces of various public clouds (i.e., cloud vendors' object storage) to obtain a list of folders named by tenant ID. The storage size is then determined based on the folder names to identify the tenant's object storage data. Subsequently, the CDN collector, a component that collects CDN traffic data, subscribes to logs reported by the front-end event tracking software development kit (SDK) in a distributed message queue. It parses and accumulates the static resource response body size to determine the tenant's CDN traffic data. The distributed message queue can be Kafka, an open-source distributed event streaming platform used for high-throughput, low-latency real-time data transmission. The Nginx collector, a component that collects Nginx access traffic data, subscribes to logs from Nginx access log topics (a logical concept in Kafka used for classifying, storing, and managing messages) in the distributed message queue. It parses and accumulates the response body size to determine the tenant's Nginx access traffic data. Finally, the cost collector, a component used to collect IaaS resource cost data, starts daily and calls the cloud vendor's application programming interface (API) or model context protocol (MGP) to determine the total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost based on the returned data.This invention avoids storage data errors by merging table analysis statements, information pattern tables, and data length + index length. It achieves low-resource-investment, low-latency bandwidth acquisition by subscribing to logs through a distributed message queue. It also reduces interface adaptation costs by uniformly connecting to cloud vendors through a model context protocol interface.
[0058] Figure 3 This illustration shows a flowchart of the steps in an IaaS resource data analysis method according to an embodiment of the present invention to determine the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data; wherein, the steps to determine the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data include:
[0059] Step 302: Obtain total storage, total traffic, and total object storage;
[0060] Step 304: Determine the storage ratio per tenant based on the tenant database storage data and total storage volume;
[0061] Step 306: Determine the CDN traffic percentage based on tenant CDN traffic data and total traffic;
[0062] Step 308: Determine the Nginx bandwidth allocation based on the tenant's Nginx access traffic data and the total traffic;
[0063] Step 310: Determine the object storage percentage based on the tenant object storage data and the total object storage.
[0064] In this technical solution, the steps for determining the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data include: First, obtaining the total storage volume, total traffic, and total object storage, where the total storage volume is the sum of the database storage sizes of all tenants, the total traffic is the sum of the CDN traffic and Nginx access traffic of all tenants, and the total object storage is the sum of the object storage sizes of all tenants; Next, determining the single-tenant storage ratio based on the tenant database storage data and the total storage volume, by dividing the tenant database storage data by the total storage volume; Determining the CDN traffic ratio based on the tenant CDN traffic data and the total traffic, by dividing the CDN traffic data by the total traffic; Determining the Nginx bandwidth ratio based on the tenant Nginx access traffic data and the total traffic, by dividing the Nginx access traffic data by the total traffic; Determining the object storage ratio based on the tenant object storage data and the total object storage, by dividing the tenant object storage data by the total object storage. This invention achieves standardization and precision in percentage calculation, providing accurate input for multi-tenant cost allocation and solving the problem of resource allocation distortion in multi-tenant shared database mode.
[0065] Figure 4 This document illustrates a flowchart of a method for analyzing IaaS resource data according to an embodiment of the present invention, showing the steps for calculating the total tenant cost based on single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. The steps for calculating the total tenant cost based on these factors include:
[0066] Step 402: Determine the database cost based on the single-tenant storage ratio and the total database cost;
[0067] Step 404: Determine the object storage cost based on the proportion of object storage and the total cost of object storage;
[0068] Step 406: Determine the CDN acceleration cost based on the CDN traffic share and the total CDN acceleration cost;
[0069] Step 408: Determine the Nginx bandwidth cost based on the Nginx bandwidth share and the total bandwidth cost;
[0070] Step 410: Determine the bandwidth cost based on the CDN acceleration cost and the Nginx bandwidth cost;
[0071] Step 412: Determine the total cost of resource data based on database cost, object storage cost, and bandwidth cost.
[0072] In this embodiment, the step of calculating the total tenant cost based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost includes: determining the database cost based on the single-tenant storage ratio and total database cost, using the formula: database cost equals single-tenant storage ratio multiplied by total database cost; determining the object storage cost based on the object storage ratio and total object storage cost, using the formula: object storage cost equals object storage ratio multiplied by total object storage cost; and determining the total cost based on the CDN traffic ratio and total CDN acceleration cost. The CDN acceleration cost is calculated as follows: CDN acceleration cost equals CDN traffic share multiplied by total CDN acceleration cost. The Nginx bandwidth cost is determined based on the Nginx bandwidth share and total bandwidth cost, calculated as follows: Nginx bandwidth cost equals Nginx bandwidth share multiplied by total bandwidth cost. The bandwidth cost is determined based on CDN acceleration cost and Nginx bandwidth cost, calculated as follows: Bandwidth cost equals CDN acceleration cost plus Nginx bandwidth cost. The total resource data cost is determined based on database cost, object storage cost, and bandwidth cost, calculated as follows: Total resource data cost equals database cost plus object storage cost plus bandwidth cost. This invention achieves accurate tenant-level cost allocation, solving the problem of cost allocation distortion in multi-tenant shared database models in related technologies, and providing data support for tenant profitability calculation.
[0073] Figure 5 This diagram illustrates a flowchart of an IaaS resource data analysis method according to an embodiment of the present invention, showing the steps of predicting the growth trend of each resource of a tenant and the corresponding prediction cost of each resource over a future period based on multi-source tenant resource data and a univariate linear regression time series model. The steps of predicting the growth trend of each resource of a tenant and the corresponding prediction cost of each resource over a future period based on multi-source tenant resource data and a univariate linear regression time series model include:
[0074] Step 502: Predict the growth trend of tenant database storage over a future period based on tenant database storage data and a univariate linear regression time series model;
[0075] Step 504: Predict the growth trend of tenant object storage in the future based on tenant object storage data and a univariate linear regression time series model;
[0076] Step 506: Predict the growth trend of tenant CDN traffic in the future based on tenant CDN traffic data and a univariate linear regression time series model;
[0077] Step 508: Based on the tenant's Nginx access traffic data and a univariate linear regression time series model, predict the growth trend of the tenant's Nginx access traffic in the future.
[0078] Step 510: Determine the predicted cost for each resource based on the growth trends of database storage, object storage, CDN traffic, and Nginx access traffic.
[0079] In this embodiment, the steps of predicting the growth trend of each tenant's resource and the corresponding predicted cost for each resource over a future period based on multi-source tenant resource data and a univariate linear regression time series model include: predicting the tenant's database storage growth trend over a future period based on tenant database storage data and a univariate linear regression time series model; predicting the tenant's object storage growth trend over a future period based on tenant object storage data and a univariate linear regression time series model; predicting the tenant's CDN traffic growth trend over a future period based on tenant CDN traffic data and a univariate linear regression time series model; predicting the tenant's Nginx access traffic growth trend over a future period based on tenant Nginx access traffic data and a univariate linear regression time series model; and determining the predicted cost for each resource based on the database storage growth trend, object storage growth trend, CDN traffic growth trend, and Nginx access traffic growth trend, respectively. The predicted cost for each resource can be obtained by multiplying the resource growth trend by the unit resource cost. This invention solves the problem of lacking data growth trend prediction and resource cost estimation capabilities in the prior art, improving the rationality and foresight of resource planning.
[0080] Figure 6 This illustration shows a flowchart of the steps in an IaaS resource data analysis method according to an embodiment of the present invention: responding to a query request from an agent, obtaining structured search results and a mapping dataset based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost to tenants; wherein, the steps of obtaining structured search results and a mapping dataset in response to a query request from an agent, based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost to tenants, include:
[0081] Step 602: Receive the agent's query request through the Model Context Protocol interface or the traditional Open API specification interface;
[0082] Step 604: Verify the agent's permissions according to the preset open licensing protocol;
[0083] Step 606: Based on the agent's permissions as the target permissions, verify the validity and legality of the agent's access token;
[0084] Step 608: If both validity and legality are verified, then in response to the agent's query request, structured retrieval results and mapping datasets are obtained based on the growth trend of each resource, the predicted cost of each resource, and the total cost of the tenant.
[0085] In this embodiment, the steps of obtaining structured retrieval results and a mapping dataset in response to a query request from an intelligent agent, based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost of the tenant, include: receiving the intelligent agent's query request through a model context protocol interface or a traditional open API specification interface; verifying the intelligent agent's permissions according to a preset OAuth (Open Authorization) protocol; verifying the validity and legality of the intelligent agent's access token (a temporary access credential issued in the Open Authorization protocol) when the intelligent agent's permissions are the target permissions, i.e., the intelligent agent's permissions are high permissions; and, based on the successful verification of both validity and legality, obtaining structured retrieval results and a mapping dataset in response to the intelligent agent's query request, based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost of the tenant. This invention achieves secure, flexible, and standardized interaction for intelligent agent queries, supports query needs in different scenarios (AI agents and ordinary programs), and solves the problem of insufficient intelligent agent integration capabilities in related technologies.
[0086] In some technical solutions, the step of acquiring multi-source tenant resource data may optionally include: acquiring multi-source tenant resource data at preset intervals, wherein the acquisition period is different for different resources.
[0087] In this technical solution, the step of acquiring multi-source tenant resource data includes: acquiring multi-source tenant resource data at preset intervals, wherein the acquisition cycle for different resources is different. For example, tenant database storage data is collected once a week, object storage data and IaaS resource cost data are collected once a day, and the total tenant cost result is calculated once a day. This invention achieves timeliness of data acquisition and resource optimization, balances data freshness and acquisition cost, and avoids unnecessary resource waste.
[0088] Figure 7 A structural block diagram of an IaaS resource data analysis system according to an embodiment of the present invention is shown; wherein, the IaaS resource data analysis system 70 includes:
[0089] The first processing module 702 is used to acquire multi-source tenant resource data and unify the format of the multi-source tenant resource data. The multi-source tenant resource data includes tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data. The tenant IaaS resource cost data includes: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost.
[0090] The second processing module 704 is used to determine the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on the tenant database storage data, tenant object storage data, tenant CDN traffic data, and tenant Nginx access traffic data.
[0091] The third processing module 706 is used to calculate the total cost result of the tenant based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. The total cost result of the tenant includes: database cost, object storage cost, bandwidth cost, and total resource data cost.
[0092] The fourth processing module 708 is used to predict the growth trend of each resource of the tenant and the prediction cost of each resource in the future period based on multi-source tenant resource data and a univariate linear regression time series model.
[0093] The fifth processing module 710 is used to respond to the agent's query request, obtain structured retrieval results and mapping datasets based on the growth trend of each resource, the predicted cost of each resource, and the total cost of the tenant, and feed back the structured retrieval results and mapping datasets to the agent through the model context protocol interface.
[0094] The IaaS resource data analysis system 70 provided by this invention includes: a first processing module 702, a second processing module 704, a third processing module 706, a fourth processing module 708, and a fifth processing module 710. The first processing module 702 first acquires multi-source tenant resource data, which includes: tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data. The tenant IaaS resource cost data includes: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. Tenant database storage data refers to the actual storage usage of the tenant in the database, which can be collected by a database collector. Tenant object storage data refers to the storage usage of the tenant in the object storage service, which can be collected by an object storage collector. Tenant CDN traffic data refers to the traffic generated when the tenant uses a Content Delivery Network (CDN) to accelerate static resources, which can be collected by a CDN collector. Tenant Nginx access traffic data refers to the traffic generated when the tenant accesses non-static resources through high-performance HTTP and reverse proxy servers, which can be collected by an Nginx collector. Tenant IaaS resource cost data refers to the total cost of Infrastructure as a Service (IaaS) resources shared by multiple tenants. Total database cost refers to the cost of database services shared by all tenants. Total object storage cost refers to the cost of object storage services shared by all tenants. Total CDN acceleration cost refers to the cost of CDN acceleration services shared by all tenants. Total bandwidth cost refers to the cost of network bandwidth shared by all tenants. After obtaining multi-source tenant resource data, the multi-source tenant resource data is converted into a unified JavaScript object representation (JSON) format containing tenant ID, cloud vendor, and resource type. Subsequently, the second processing module 704 determines the single-tenant storage ratio based on the converted tenant database storage data, the object storage ratio based on the converted tenant object storage data, the CDN traffic ratio based on the tenant CDN traffic data, and the Nginx bandwidth ratio based on the tenant Nginx access traffic data. Subsequently, the third processing module 706 calculates the total tenant cost based on the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, object storage ratio, total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. The total tenant cost includes database cost, object storage cost, bandwidth cost, and total resource data cost. Then, the fourth processing module 708 predicts the growth trend of various tenant resources and corresponding predicted costs over a future period based on multi-source tenant resource data and a univariate linear regression time series model. The univariate linear regression time series model is a statistical model that uses historical time series data to fit a linear trend to predict future values.Finally, the fifth processing module 710 responds to the agent's query request, generating structured search results and a mapping dataset based on growth trends, predicted costs, and total tenant costs. The structured search results are JavaScript object representation data containing entity URIs, attribute values, and relationship chains. The mapping dataset is a dataset showing the relationships between tenant resources and cloud vendors, and between costs and percentages. This structured data can be directly parsed by the agent and used for upper-level intelligent applications such as resource expansion planning, cost warnings, and operational decision-making. Finally, it is fed back to the agent through the MCP (Model Context Protocol) interface. This invention shields the differences between multiple cloud vendor interfaces through the Model Context Protocol interface, converting multi-source resource data into a unified JSON format containing tenant IDs, cloud vendors, and resource types, solving the problems of inconsistent interface standards and fragmented data formats, and reducing integration costs. Simultaneously, it predicts resource growth trends and costs over a future period based on a univariate linear regression time series model, addressing the lack of growth prediction and cost estimation capabilities. Furthermore, this invention unifies storage, bandwidth, object storage, costs, and prediction time to form a global data energy, solving the problem of data being scattered across different systems and difficult to support global analysis. This invention solves the problems in related technologies such as inconsistent interface standards, distorted cost sharing among multi-tenants, high bandwidth computing costs and high latency, insufficient intelligent agent docking capabilities, lack of trend prediction, data silos, and poor compatibility among multiple deployment modes, and achieves unified collection, accurate allocation, intelligent prediction and efficient interaction of multi-cloud resources.
[0095] Figure 8This diagram illustrates the layered architecture of an IaaS resource data analysis system according to an embodiment of the present invention. The IaaS resource data analysis system is divided into five layers: an interaction layer, a basic capability layer, a data cleaning and analysis processing layer, a data acquisition layer, and a data source layer. The interaction layer is the external access system layer, mainly including: intelligent agents, IM (Instant Messaging), APP (Application), and web (World Wide Web). The basic capability layer provides interfacing capabilities for third parties, including: MCP (Model Context Protocol) intelligent interfaces, OpenAPI (OpenAPI Specification (formerly Swagger), which defines, describes, and documents RESTful HTTP interfaces, using standardized JSON / YAML formats to describe interface addresses, request parameters, response structures, authentication methods, etc., achieving readability, testability, and automatic code generation capabilities), and customized reports. The data cleaning and analysis processing layer possesses data cleaning, data analysis, and data prediction capabilities. It can calculate and predict the proportion and cost of data storage, as well as the proportion and cost of object storage. Furthermore, it calculates and predicts bandwidth usage and costs. The data acquisition layer possesses data acquisition and processing capabilities, enabling database acquisition, object storage acquisition, CDN data acquisition, and nginx log acquisition. Database acquisition can utilize direct JDBC (Java Database Connectivity) connections, object storage acquisition can use cloud vendor APIs, and CDN and nginx log acquisition can both be performed via Kafka subscriptions. The data source layer provides the infrastructure provided by external cloud service providers and exposes other internal systems to the outside world. The data source layer includes database storage data, object storage data, CDN data, and dynamic request bandwidth.
[0096] Figure 9This diagram illustrates the architecture of an IaaS resource data analysis system according to an embodiment of the present invention. The architecture of the IaaS resource data analysis system includes a ClickHouse data warehouse, which primarily comprises: a tenant data storage acquisition module, a tenant object storage acquisition module, a tenant CDN traffic acquisition module, a tenant Nginx access log acquisition module, a tenant IaaS resource cost acquisition module, a data analysis and calculation module, a data intelligent prediction module, an MCP intelligent agent interface service module, a data scheduling module, and a reporting module. Specifically, the tenant data storage acquisition module is mainly used to collect the storage data and row count of each data table from the production business system, calculating the storage size of each row in a single table using "total space size ÷ number of rows"; simultaneously, it collects the row count data corresponding to each tenant, further calculating the actual storage usage of each tenant. The tenant object storage acquisition module is mainly used to interface with multi-cloud IaaS vendors based on a tenant list, managing tenant storage data by bucket according to tenant ID, and directly querying and obtaining the object storage usage of each tenant through the interface. The tenant CDN traffic collection module primarily receives data reported by the front-end event tracking SDK, filters and extracts static resource domain-related data, including tenant ID, requested resource URL, request body size, and response body size. After accumulating the response body size, the data is stored in the CDN access summary table, ensuring that each tenant corresponds to a unique access traffic size record. The tenant Nginx access log collection module is mainly responsible for collecting access data for non-static resources. By analyzing the data, it extracts the tenant ID and the corresponding non-static resource request and response data, thereby calculating the bandwidth usage of each tenant. The tenant IaaS resource cost collection module is mainly used to collect various cost data from cloud vendors, including total database cost, total object storage cost, CDN acceleration cost, and total bandwidth cost. This data is collected weekly and stored in a local database to ensure the timeliness and completeness of the cost data. The data analysis and calculation module is primarily used to calculate the storage, bandwidth, and object storage percentages of a single tenant among all tenants based on the basic data acquired by each acquisition module. Then, it calculates the cost of each tenant item by multiplying the percentage of each data type by the total cost of the corresponding type, and finally sums up the costs of each item to obtain the total cost for each tenant. The data intelligent prediction module is mainly used to store and archive various types of data monthly, using time series forecasting methods to predict the data growth trend of a single tenant. This is used for cost prediction and data storage volume prediction, providing data support for resource scheduling and decision-making in the peripheral system. The MCP intelligent agent interface service module mainly provides the MCP interface for intelligent agents to call, supporting core functions such as tenant storage volume query and trend prediction data query, enabling efficient data interaction with intelligent agents.The data scheduling module serves as the core scheduling unit of the data intelligent analysis device. Database data collection is synchronized weekly, object storage data collection is synchronized daily, and cost data collection is synchronized daily. The reporting module integrates all analytical data, supporting querying, statistics, and data mining of various reports, providing users with intuitive and comprehensive data display and analysis support.
[0097] Figure 10 This diagram illustrates a key process sequence of an IaaS resource data analysis system according to an embodiment of the present invention. The implementation steps of each independent module are as follows: The tenant database storage data acquisition module implements the following steps: First, the database collector starts weekly. Then, it accesses the databases of multiple cloud vendors, refreshing the storage space with the latest data using the ANALYZE TABLE statement before acquisition. Next, it uses SQL statements to read the information_schema table, iterating through each table to determine its storage size and number of rows. Before data acquisition, it merges data using DATA_LENGTH + INDEX_LENGTH to avoid the 20% error rate introduced by TABLE_ROWS. After reading the schema name, table name, estimated table row count, and table storage size, it stores the data in the database. Then, it iterates through the tables in each schema, determining if the table name contains a tenant ID identifier. If a tenant ID identifier is present, it indicates that the table is stored in a tenant ID-based partitioned table. If it is a tenant ID-based partitioned table, it reads the total number of rows in the table, representing the proportion of rows for that tenant. If no tenant ID is present, it uses the SQL statement COUNT, grouped by tenant ID, to query and read the tenant ID field and the corresponding number of rows. Then, calculate the row size by dividing the total table storage size by the number of rows to determine the size of each row. Calculate the tenant database storage size by multiplying the calculated row size by the number of rows for each tenant to determine the storage size for the tenant ID. Data storage is then performed according to the table structure design, including tenant ID, tenant name, schema, table, number of rows, row size, and total table storage, and is updated monthly. Finally, the database collector completes a single data collection session. The above steps are repeated every week at midnight.
[0098] The implementation steps of the tenant object storage data collection module are as follows: First, the object storage collector starts daily at a set time, calling the MCP interface of each cloud vendor's object storage to obtain a list of folders named according to tenant ID within the bucket. The file directory name is: Tenant ID. Based on the folder named with each tenant ID, the storage size is obtained. Then, the acquired data is standardized and converted into a unified format, stored as: Tenant ID, Cloud Vendor, Object Storage Size, Total Object Storage Size, Storage Unit (MB), and updated monthly. Finally, the object storage data collection is complete. The scheduled task synchronizes and repeats the above steps once a day.
[0099] The implementation steps of the tenant CDN traffic collection module are as follows: First, the CDN collector starts and subscribes to the front-end resource access logs reported by the front-end SDK collector via Kafka. The front-end reported log data includes tenant ID, operation action, resource type, request URL, domain name, request response time, request body size, response body size, request duration, etc. After subscribing to the Kafka data, the raw data reported by the front-end is parsed, and the tenant ID, static resource URL list, and the body size of each URL response are extracted from the message body. After cleaning the data, the response body size of each URL list is added together to calculate the total size of the current tenant's CDN static resources. The unified storage format is: tenant ID, cloud provider, static resource size (MB), total CDN traffic size, and monthly storage. Finally, the CDN collection subscription and consumption ends.
[0100] The implementation steps of the tenant Nginx access traffic collection module are as follows: First, the tenant Nginx access traffic collector starts, subscribes to the access logs in Kafka with the topic identified as Nginx-access-topic, and parses out the URL, response body size, and tenant ID from the Nginx logs. The response body size of each URL is accumulated to calculate the Nginx access log for the tenant ID. The data is stored in the database with the following data structure: tenant ID, dynamic bandwidth request size (MB), total bandwidth size, and monthly storage. Finally, the Nginx access traffic collection ends.
[0101] The implementation steps of the tenant IaaS resource cost collection module are as follows: First, the IaaS resource cost collector starts up daily. Then, it calls the API interfaces or MCP interfaces of various cloud vendors to collect data on database type, object storage type, CDN bandwidth costs, and network traffic costs. The data storage format is: resource type, cloud vendor, cost, and date, stored monthly. Finally, the collector completes a single collection session and repeats the process daily.
[0102] The data calculation and analysis module implements the following steps: First, after the previously obtained data is prepared, data analysis and calculation are performed. Analyze the tenant's database storage ratio and cost. Calculate the database storage ratio by dividing the current tenant ID's database storage by the total storage based on the collected and cleaned data. Calculate the current tenant's database cost by multiplying the storage ratio by the database cost. Analyze the tenant's object storage ratio and cost. Calculate the object storage ratio by dividing the current tenant ID's object storage by the total object storage based on the cleaned object storage data. Calculate the object storage cost by multiplying the storage ratio by the object storage cost. Analyze the tenant's bandwidth ratio and cost. Calculate the bandwidth ratio by dividing the current tenant's CDN traffic by the total traffic, and then multiply the ratio by the cost to calculate the current tenant's CDN cost. Calculate the bandwidth ratio by dividing the current tenant's Nginx access bandwidth by the total traffic, and then multiply the ratio by the bandwidth cost to calculate the bandwidth cost. Calculate the total bandwidth cost by adding the CDN cost to the Nginx access bandwidth cost. Finally, the data analysis and processing are completed.
[0103] The data prediction module operates as follows: First, the module starts and regularly updates the prediction model data daily. Then, it uses a univariate linear regression equation to model the data, employing historical data and taking the time dimension to represent the collection period for each data point, predicting the tenant's data growth over the next six months. This includes predicting the growth trends and costs for database storage, object storage, CDN bandwidth, and Nginx access traffic corresponding to the tenant ID. After prediction, the data is stored in a prediction table, including the tenant ID, database growth trend, data growth cost, object storage growth trend, object storage growth cost, CDN bandwidth growth trend, CDN growth cost, and the predicted month. Finally, the data prediction module completes its single execution and updates the data daily.
[0104] The implementation steps of the data MCP service are as follows: First, the MCP service provides two types of interfaces: MCP type and traditional OpenAPI to meet different usage scenarios. The MCP interfaces include: querying various types of stored data, querying predicted data, querying single-table data, and querying various cost data. MCP authentication uses OAuth, issuing a temporary Access Token to each agent or client using the data. MCP authorization controls data access permissions based on client and agent type; cost-sensitive data can only be accessed by users with high privileges. When an agent calls the MCP interface, the data server verifies the Access Token, ensuring it is valid and legally valid for data access. Data authorization is performed for each client, and access to the corresponding data is granted upon successful verification. The server then responds with JSON data, and the access process ends.
[0105] The steps to implement the reporting service are as follows: First, use customized reports, such as trend charts, pie charts, and bar charts, to display the data. The customized report set is designed for use by non-intelligent agents and browsers.
[0106] The data scheduling module is implemented as follows: The trigger controller for scheduled tasks is defined using xxjob. Database data is collected weekly. Object storage data is collected daily. Cost data is recalculated daily.
[0107] An electronic device according to an embodiment of the present invention includes a memory processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the IaaS resource data analysis method as described above.
[0108] The electronic device provided by the present invention, when the processor executes the computer program, implements the steps of the above-described IaaS resource data analysis method, and can achieve the technical effects of any of the above embodiments, which will not be repeated here.
[0109] One embodiment of the present invention provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of the IaaS resource data analysis method as described above.
[0110] The storage medium provided by this invention enables the computer program to implement the steps of the above-described IaaS resource data analysis method when executed by a processor, and can achieve the technical effects of any of the above embodiments, which will not be elaborated further.
[0111] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance, unless otherwise expressly specified and limited. The terms "connection," "installation," and "fixing," etc., should be interpreted broadly. For example, "connection" can mean a fixed connection, a detachable connection, or an integral connection; it can mean a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0112] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0113] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for analyzing IaaS resource data, characterized in that, include: Acquire multi-source tenant resource data and unify the format of the multi-source tenant resource data. The multi-source tenant resource data includes tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data. The tenant IaaS resource cost data includes: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. Based on the tenant database storage data, the tenant object storage data, the tenant CDN traffic data, and the tenant Nginx access traffic data, the single tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio are determined. The total tenant cost is calculated based on the single-tenant storage ratio, the CDN traffic ratio, the Nginx bandwidth ratio, the object storage ratio, the total database cost, the total object storage cost, the total CDN acceleration cost, and the total bandwidth cost. The total tenant cost includes: database cost, object storage cost, bandwidth cost, and total resource data cost. Based on the multi-source tenant resource data and the univariate linear regression time series model, predict the growth trend of each resource of the tenant in the future period and the predicted cost corresponding to each resource; In response to the agent's query request, a structured retrieval result and a mapping dataset are obtained based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost of the tenant. The structured retrieval result and the mapping dataset are then fed back to the agent through the model context protocol interface.
2. The IaaS resource data analysis method according to claim 1, characterized in that, The steps for obtaining multi-source tenant resource data include: By accessing read information pattern tables in multiple public cloud databases through a database collector and SQL, the total storage size and number of rows of the tables are obtained, and the data stored in the tenant database is determined based on the total storage size and number of rows of the tables. The object storage collector calls the model context protocol interface of the object storage of each of the public clouds to obtain a list of folders named by tenant ID, obtains the storage size based on the folders in the list, and determines the object storage data of the tenant. The CDN collector subscribes to the logs reported by the front-end instrumentation software development kit in the distributed message queue, parses and accumulates the size of the static resource response body, and determines the tenant's CDN traffic data based on the size of the static resource response body. The Nginx collector subscribes to the logs of the Nginx access log topic in the distributed message queue, parses and accumulates the response body size, and determines the tenant's Nginx access traffic data based on the response body size. The cost collector calls all the application programming interfaces or model context protocol interfaces of the public cloud to determine the total cost of the database, the total cost of the object storage, the total cost of the CDN acceleration, and the total bandwidth cost based on the returned data.
3. The IaaS resource data analysis method according to claim 1, characterized in that, The step of determining the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio, and object storage ratio based on the tenant database storage data, the tenant object storage data, the tenant CDN traffic data, and the tenant Nginx access traffic data includes: Get the total storage, total bandwidth, and total object storage; The single-tenant storage ratio is determined based on the tenant database storage data and the total storage volume. The CDN traffic percentage is determined based on the tenant CDN traffic data and the total traffic. The Nginx bandwidth percentage is determined based on the tenant's Nginx access traffic data and the total traffic. The object storage percentage is determined based on the tenant object storage data and the total object storage.
4. The IaaS resource data analysis method according to claim 1, characterized in that, The step of calculating the total tenant cost based on the single-tenant storage ratio, the CDN traffic ratio, the Nginx bandwidth ratio, the object storage ratio, the total database cost, the total object storage cost, the total CDN acceleration cost, and the total bandwidth cost includes: The database cost is determined based on the single-tenant storage ratio and the total database cost. The object storage cost is determined based on the object storage percentage and the total object storage cost. The CDN acceleration cost is determined based on the CDN traffic share and the total CDN acceleration cost. The Nginx bandwidth cost is determined based on the Nginx bandwidth percentage and the total bandwidth cost. The bandwidth cost is determined based on the CDN acceleration cost and the Nginx bandwidth cost. The total cost of the resource data is determined based on the database cost, the object storage cost, and the bandwidth cost.
5. The IaaS resource data analysis method according to claim 1, characterized in that, The step of predicting the growth trend of each resource of a tenant and the prediction cost corresponding to each resource in the future period based on the multi-source tenant resource data and the univariate linear regression time series model includes: Based on the tenant database storage data and the univariate linear regression time series model, predict the tenant's database storage growth trend in the future period; Based on the tenant object storage data and the univariate linear regression time series model, predict the growth trend of tenant object storage in the future period; Based on the tenant CDN traffic data and the univariate linear regression time series model, predict the tenant's CDN traffic growth trend in the future period; Based on the tenant's Nginx access traffic data and the univariate linear regression time series model, predict the growth trend of the tenant's Nginx access traffic in the future. The predicted cost for each of the resources is determined based on the growth trends of the database storage, object storage, CDN traffic, and Nginx access traffic.
6. The IaaS resource data analysis method according to claim 1, characterized in that, The step of obtaining structured retrieval results and a mapping dataset in response to a query request from an agent, based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost of the tenant, includes: The agent receives query requests through the model context protocol interface or the traditional open API specification interface. The permissions of the intelligent agent are verified according to a preset open licensing protocol; Based on the permissions of the intelligent agent as the target permissions, verify the validity and legality of the intelligent agent's access token; If both the validity and legality are verified, then in response to the query request of the intelligent agent, a structured retrieval result and a mapping dataset are obtained based on the growth trend of each resource, the predicted cost corresponding to each resource, and the total cost result of the tenant.
7. The IaaS resource data analysis method according to any one of claims 1 to 6, characterized in that, The steps for obtaining multi-source tenant resource data include: The multi-source tenant resource data is acquired at preset intervals, wherein the acquisition period is different for different resources.
8. An IaaS resource data analysis system, characterized in that, include: The first processing module is used to acquire multi-source tenant resource data and unify the format of the multi-source tenant resource data. The multi-source tenant resource data includes tenant database storage data, tenant object storage data, tenant CDN traffic data, tenant Nginx access traffic data, and tenant IaaS resource cost data. The tenant IaaS resource cost data includes: total database cost, total object storage cost, total CDN acceleration cost, and total bandwidth cost. The second processing module is used to determine the single-tenant storage ratio, CDN traffic ratio, Nginx bandwidth ratio and object storage ratio based on the tenant database storage data, the tenant object storage data, the tenant CDN traffic data and the tenant Nginx access traffic data. The third processing module is used to calculate the total tenant cost result based on the single-tenant storage ratio, the CDN traffic ratio, the Nginx bandwidth ratio, the object storage ratio, the total database cost, the total object storage cost, the total CDN acceleration cost, and the total bandwidth cost. The total tenant cost result includes: database cost, object storage cost, bandwidth cost, and total resource data cost. The fourth processing module is used to predict the growth trend of each resource of the tenant and the prediction cost corresponding to each resource in the future period based on the multi-source tenant resource data and the univariate linear regression time series model. The fifth processing module is used to respond to the query request of the agent, obtain structured retrieval results and mapping datasets based on the growth trend of each resource, the predicted cost corresponding to each resource and the total cost of the tenant, and feed back the structured retrieval results and mapping datasets to the agent through the model context protocol interface.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the IaaS resource data analysis method as described in any one of claims 1 to 7.
10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the IaaS resource data analysis method as described in any one of claims 1 to 7.