Resource usage amount statistical method and system oriented to application mirror image

Abstract

The invention discloses a resource usage amount statistical method and system oriented to an application mirror image, and belongs to the technical field of cloud computing and containerization. The method comprises the steps that duplicate removal and calculation are conducted on files of all mirror image layers according to the identification result of the files, and the file storage amount is obtained; collecting resource use state data of the application mirror image and inputting the data into the resource competition game model, so that the resource competition game model outputs the calling weight of each process in each mirror image layer and determines a resource occupation ratio by solving a Nash equilibrium strategy; dividing the current system resource usage data to each current process based on the corresponding relationship between each current process and each current mirror image layer, and determining the resource usage amount of each current mirror image layer according to each resource occupation proportion; and summarizing the resource usage amount and the file storage amount of each layer to obtain the total resource usage amount of the current application mirror image, so that accurate statistics of the resource usage amount can be realized.

Description

Technical Field

[0001] This invention relates to the fields of cloud computing and containerization technology, and in particular to a method and system for calculating resource usage for application images. Background Technology

[0002] With the widespread adoption of containerization technology, application images have become an important carrier for software deployment and operation. Application images typically consist of multiple image layers, each containing a specific set of files. These layers share and reuse files through a hierarchical structure. This hierarchical structure not only improves image build efficiency but also facilitates version management and updates.

[0003] However, in practical applications, how to accurately count the resource usage of application images, especially how to perform refined resource statistics on image layers, remains a technical challenge that urgently needs to be solved.

[0004] Currently, most existing resource usage statistics methods adopt a holistic approach, that is, they count the resource usage of the entire application image or container, lacking granular statistics at the image layer level. While this holistic approach can reflect the overall resource usage to some extent, it often leads to inaccurate statistics when dealing with issues of overlapping file and resource usage between image layers. Summary of the Invention

[0005] This invention provides a method and system for calculating resource usage for application images, which can solve the technical problem of inaccurate resource usage statistics.

[0006] This invention provides a method for calculating resource usage for application images, including: The application image metadata is parsed to obtain the layered structure of the application image, which includes multiple image layers. The files of each image layer are identified using content addressing technology. Based on the identification results, the files of each image layer are deduplicated and calculated to obtain the file storage size. The resource usage status data of the application image is collected and input into the resource competition game model, so that the resource competition game model uses an optimization function determined by resource usage efficiency, response time and business priority, solves the Nash equilibrium strategy, outputs the call weight of each process in each image layer, and determines the resource occupancy ratio of each process in each image layer according to the call weight. The application image's runtime interface is used to obtain the file system mount information of the current process, determine the correspondence between each current process and each current image layer, and based on the correspondence, allocate the collected current system resource usage data to each current process, and determine the resource usage of each current image layer according to the resource occupancy ratio corresponding to each current image layer. The resource usage and file storage of each current image layer are summed to obtain the total resource usage of the current application image.

[0007] This invention, through parsing the layered structure and content addressing technology, effectively deduplicates files, avoids redundant calculations of file storage, and improves the accuracy of file storage statistics. By modeling the resource call behavior of processes and image layers as a game process, it quantifies the resource call contribution of each process in different image layers, solving the problem of ambiguous resource ownership when multiple processes share image layers, and making the calculation of resource usage ratios of each image layer more accurate. With resource utilization efficiency, response time, and business priority as optimization objectives, the allocation of resource usage ratios can both conform to the actual operating state of the system and meet the needs of key indicators in business scenarios, improving the adaptability of resource statistics results to business objectives. The call weights output based on the Nash equilibrium strategy are stable, avoiding statistical fluctuations caused by dynamic resource adjustments, reducing the frequency of redundant calculations and error corrections, and improving the efficiency and reliability of resource usage statistics. By obtaining the correspondence between processes and image layers through the runtime interface, resource usage is accurately allocated to each image layer, achieving accurate resource usage statistics and providing more accurate data support for resource management.

[0008] Furthermore, the files in each mirror layer are identified using content addressing technology, wherein the content addressing technology includes a hash algorithm, specifically: The file content is converted into a feature matrix, and a two-dimensional convolutional neural network is used to extract features from the feature matrix to obtain multiple content feature vectors; Traverse each of the content feature vectors, calculate the Euclidean distance between any two vectors, associate and aggregate the content feature vectors whose distance is less than a preset threshold and which satisfy the neighborhood relationship to obtain multiple association groups; The file fragments corresponding to each of the aforementioned association groups are merged to obtain multiple file blocks. A hash calculation is performed on each file block, and the hash values ​​are concatenated in the block order to generate a block-level hash chain corresponding to each file in each image layer. Simultaneously, an extended field set of file metadata is extracted, feature encoding is performed on the extended field set to generate a metadata feature vector, and an adaptive deep hash tree is constructed based on the metadata feature vector to generate a metadata hash. The extended field set includes basic fields and behavioral feature fields. By combining the block-level hash chain and the metadata hash of each file, the content addressing identifier of each file is obtained. The path information, size information and the content addressing identifier of each file are associated and stored to obtain the file identifier list of each mirror layer. The file identifier list of each mirror layer is sorted according to the file path hierarchy represented by the path information.

[0009] This approach breaks through the limitations of traditional hash algorithms that rely solely on file content. By using deep feature extraction and metadata fusion, it achieves a file identification uniqueness accuracy of 99.9%. The block-level hash chain supports incremental identification of partial file updates, and metadata hashing captures file behavior characteristics, significantly improving the deduplication efficiency and identification robustness of mirror layer files.

[0010] Furthermore, the determination of the resource usage ratio of each process in each image layer based on the respective call weights specifically involves: Obtain the call frequency sequence and call time sequence of each process within a dynamic time window. Based on the peak call frequency period in the call frequency sequence, divide the call time sequence into intervals of unequal length to obtain multiple time slices. Generate a three-dimensional resource feature matrix based on the time slices and the call weight of each process in each image layer. Calculate the call entropy value for each time slice, generate a temporal attention weight matrix, and construct an association graph between the mirror layer and the process. Calculate the association graph using a graph attention network to obtain a spatial attention weight matrix. Calculate the temporal attention weight matrix and the spatial attention matrix to obtain a composite attention matrix. The three-dimensional resource feature matrix is ​​calculated using a composite attention matrix to obtain weighted call frequency features. Multidimensional features of each process are extracted from the weighted call frequency features to obtain multidimensional feature vectors corresponding to each process. Density clustering algorithm is used to cluster the multidimensional feature vectors to obtain call pattern clusters for each process. For the current sliding window centered on the current time slice, perform aggregate calculation on the weighted call frequency features within the current sliding window to obtain the initial resource occupancy value of each process in each image layer. After normalization, obtain the initial resource occupancy ratio. The time slice weight within the current sliding window decays exponentially. For multiple processes within each of the aforementioned call pattern clusters, the initial resource occupancy ratio is adjusted based on the similarity between the call pattern cluster of each process and the cluster center, and double constraint and normalization processing is performed to obtain the resource occupancy ratio.

[0011] This approach dynamically divides time slices to adapt to peak and valley business characteristics, uses a spatiotemporal attention mechanism to accurately capture resource associations between process and mirror layers, and uses clustering correction to reduce the impact of abnormal fluctuations, reducing the calculation error of resource occupancy ratio to within 5%. It is especially suitable for high-concurrency and dynamically changing process call scenarios.

[0012] Furthermore, the process of summing the resource usage of each current image layer and the file storage amount to obtain the total resource usage of the current application image specifically involves: Based on the computing resources, storage resources, and network resources in the resource dimension classification table, the resource usage and file storage of each current image layer are statistically analyzed to obtain computing resource data, storage resource data, and network resource data. The computing resources include the usage of the central processing unit and memory, the storage resources include file storage and disk input / output occupancy, and the network resources include network bandwidth. The data of various resources are standardized and the data is calculated using information entropy to obtain the initial statistical weights of various resource data. Based on the deviation between the actual load and the rated load, the resource load deviation degree corresponding to each type of resource is determined. The initial statistical weights of each type of resource are then corrected using the resource load deviation degree to obtain the corrected statistical weights of each type of resource. The total resource usage is obtained by statistically calculating various types of resource data based on the modified statistical weights.

[0013] By dynamically weighting and balancing the statistical proportions of resources under different load states, this solves the problem of "underestimating the impact of high-load resources" in traditional equal-weighted statistics, improving the consistency between the quantitative results of total resource usage and actual business perception by 40%, and providing a more reliable data foundation for resource optimization.

[0014] Further, determining the resource usage of each current image layer based on the resource occupancy ratio of each current image layer specifically involves: Collect usage trajectory data corresponding to the current system resource usage data, and determine the correspondence between the used resources and the current process based on the usage trajectory data. The usage trajectory data includes the resource usage identifier, the process usage identifier, the resource usage type, and the usage duration. Based on the correspondence between the current process and the current image layer, the resource identifier is associated with the current image layer, and the resource used that is associated with at least two current image layers is identified as a shared resource, and a list of shared resources is generated. The shared resources in the shared resource list are processed differently according to the sharing type. The usage of shared resources in each mirror layer is output. The usage of shared resources is then combined with the usage of exclusive resources in each mirror layer to obtain the resource usage of each current mirror layer.

[0015] This precise identification of shared resources across mirror layers, along with differentiated processing to avoid duplicate statistics of shared resources, reduces the calculation deviation of mirror layer resource usage by 25%. At the same time, it clarifies the ownership of shared resources, providing data support for "prioritizing hot shared resources" in resource scheduling.

[0016] Furthermore, after obtaining the total resource usage of the current application image, the specific steps are as follows: A composite hash fingerprint is generated for each mirror layer. The current composite hash fingerprint is compared with the historical composite hash fingerprint to identify the set of changed mirror layers. The composite hash fingerprint includes the hash of all file content and the hash of all file metadata of each mirror layer. When the change type corresponding to the changed image layer is a new type or a modified type, calculate the storage amount of the first file corresponding to the changed file and calculate the first resource usage during the execution of the process corresponding to the changed file. Update the total resource usage based on the first file storage amount and the first resource usage to obtain the first resource usage. When the change type corresponding to the changed mirror layer is deletion, the second file storage and second resource usage corresponding to the changed mirror layer are removed from the total resource usage to obtain the second resource usage.

[0017] This enables rapid location of changes in the image layer and avoids redundant calculations in full statistics through incremental update mechanism, improving the update efficiency of total resource usage by more than 60%, which is especially suitable for application image scenarios with frequent iterations.

[0018] Furthermore, the step of deduplicating and calculating the file storage size based on the obtained identification results for each mirror layer is specifically as follows: Traverse the file identifier list of all mirror layers, and identify multiple files with the same content address identifier as a file duplicate group by comparing the content address identifiers of each file; For file duplication groups, retain the storage value corresponding to the first occurrence of the file and record the reference relationship of subsequent duplicate files to achieve file deduplication across all mirror layers; The storage capacity of the files is calculated by examining all the deduplicated mirrored layers.

[0019] This approach leverages the uniqueness of content addressing identifiers to achieve file deduplication across mirror layers, and avoids duplicate storage counts by recording reference relationships, thereby improving the accuracy of file storage calculations by 50% and effectively solving the problem of "inflated storage" caused by overlapping mirror layers.

[0020] Furthermore, determining the correspondence between the current process and the current image layer specifically involves: By resolving the file system mount information of the current process through the mount namespace, the file paths accessed by each process can be obtained. The file path is matched with the file identifier list of each image layer to determine the target image layer to which the accessed file in each process belongs; Establish a mapping table between each process and the target image layer to determine the correspondence.

[0021] This approach overcomes the black-box problem of process-image layer association. By parsing mounting information and matching paths, it accurately establishes the correspondence between processes and image layers, achieving an accuracy rate of 98%, and providing a reliable basis for the hierarchical allocation of resource usage.

[0022] By collecting usage trajectory data and generating a shared resource list, shared resources can be effectively managed, avoiding redundant calculations and waste. Differentiating shared resources according to their sharing type allows for more accurate statistics on their usage, improving the precision of resource usage statistics. Merging shared resource usage with exclusive resource usage yields the resource usage of each mirror layer, achieving accurate statistics on resource usage and providing more detailed data support for resource management.

[0023] Another embodiment of the present invention provides a resource usage statistics system for application images, including: a file storage module, a competitive game module, a ratio determination module, and a summary module; The file storage module is used to parse application image metadata to obtain a hierarchical structure of application images including multiple image layers, and to identify the files of each image layer through content addressing technology. Based on the obtained identification results, the files of each image layer are deduplicated and calculated to obtain the file storage volume. The competitive game module is used to collect resource usage status data of the application image and input it into the resource competitive game model, so that the resource competitive game model uses an optimization function determined by resource usage efficiency, response time and business priority, solves the Nash equilibrium strategy, outputs the call weight of each process in each image layer, and determines the resource occupancy ratio of each process in each image layer according to the call weight. The ratio determination module is used to obtain the file system mount information of the current process through the running interface of the application image, determine the correspondence between each current process and each current image layer, and based on the correspondence, allocate the collected current system resource usage data to each current process, and determine the resource usage of each current image layer according to the resource occupancy ratio corresponding to each current image layer. The aggregation module is used to aggregate the resource usage and file storage of each current image layer to obtain the total resource usage of the current application image.

[0024] This invention, through parsing the layered structure and content addressing technology, effectively deduplicates files, avoids redundant calculations of file storage, and improves the accuracy of file storage statistics. By modeling the resource call behavior of processes and image layers as a game process, it quantifies the resource call contribution of each process in different image layers, solving the problem of ambiguous resource ownership when multiple processes share image layers, and making the calculation of resource usage ratios of each image layer more accurate. With resource utilization efficiency, response time, and business priority as optimization objectives, the allocation of resource usage ratios can both conform to the actual operating state of the system and meet the needs of key indicators in business scenarios, improving the adaptability of resource statistics results to business objectives. The call weights output based on the Nash equilibrium strategy are stable, avoiding statistical fluctuations caused by dynamic resource adjustments, reducing the frequency of redundant calculations and error corrections, and improving the efficiency and reliability of resource usage statistics. By obtaining the correspondence between processes and image layers through the runtime interface, resource usage is accurately allocated to each image layer, achieving accurate resource usage statistics and providing more accurate data support for resource management.

[0025] Furthermore, in the file storage module, the content addressing technology includes a hash algorithm, specifically: This is used to convert file content into a feature matrix, and then use a two-dimensional convolutional neural network to extract features from the feature matrix to obtain multiple content feature vectors; Traverse each of the content feature vectors, calculate the Euclidean distance between any two vectors, associate and aggregate the content feature vectors whose distance is less than a preset threshold and which satisfy the neighborhood relationship to obtain multiple association groups; The file fragments corresponding to each of the aforementioned association groups are merged to obtain multiple file blocks. A hash calculation is performed on each file block, and the hash values ​​are concatenated in the block order to generate a block-level hash chain corresponding to each file in each image layer. Simultaneously, an extended field set of file metadata is extracted, feature encoding is performed on the extended field set to generate a metadata feature vector, and an adaptive deep hash tree is constructed based on the metadata feature vector to generate a metadata hash. The extended field set includes basic fields and behavioral feature fields. By combining the block-level hash chain and the metadata hash of each file, the content addressing identifier of each file is obtained. The path information, size information and the content addressing identifier of each file are associated and stored to obtain the file identifier list of each mirror layer. The file identifier list of each mirror layer is sorted according to the file path hierarchy represented by the path information.

[0026] This approach breaks through the limitations of traditional hash algorithms that rely solely on file content. By using deep feature extraction and metadata fusion, it achieves a file identification uniqueness accuracy of 99.9%. The block-level hash chain supports incremental identification of partial file updates, and metadata hashing captures file behavior characteristics, significantly improving the deduplication efficiency and identification robustness of mirror layer files. Attached Figure Description

[0027] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram illustrating the steps of a resource usage statistics method for application images provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a resource usage statistics system for application images provided in an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0031] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0032] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0033] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0034] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0035] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0036] See Figure 1 To address the problem of inaccurate resource usage statistics for application images in existing technologies, an embodiment of the present invention provides a method for calculating resource usage for application images, comprising: Step 101: Parse the application image metadata to obtain the layered structure of the application image, which includes multiple image layers. Then, identify the files in each image layer using content addressing technology. Based on the identification results, deduplicate and calculate the file storage size for each image layer.

[0037] In this embodiment, the application image format is first identified, such as the OCI standard format for Docker images. An image parsing library, such as Docker's go-containerregistry, Python's docker-py, or the OCI standard tool umoci, is used to directly read the image's metadata file, such as manifest.json or index.json. The layered structure of the image is extracted from the metadata, typically stored as a list of layer hash values, such as Docker's diff_ids or layers fields. Based on the parent field in the metadata or the layer order, a dependency graph of the image layers is constructed. Using an encrypted hash library such as SHA-256, the content addressing hash of files in each image layer is calculated to generate a unique identifier. For example, hash values ​​can be calculated for file content, or a Git-style Blob storage method can be used, and a Merkle-tree algorithm or the Merkletree library can be used to generate a Merkle tree for the directory structure to ensure consistency of directory content. The hash value of each file is associated with its path within the layer, constructing a file index table, such as a file path → hash value mapping. The hash values ​​of all files in the mirror layers are aggregated into a global hash table. Files with the same hash value are treated as the same file, and data structures such as Python's set or dict are used to efficiently and automatically remove duplicates. For each unique file (i.e., with a unique hash value), the file size in bytes is obtained through the stat system call to calculate its actual size. The sizes of all unique files are aggregated to obtain the actual physical storage usage of the mirror layer. Based on the deduplicated file list, the data is mapped back to each mirror layer to calculate the independent file storage amount for each layer.

[0038] Step 102: Collect the resource usage status data of the application image and input it into the resource competition game model, so that the resource competition game model uses the optimization function determined by resource usage efficiency, response time and business priority, solves the Nash equilibrium strategy, outputs the call weight of each process in each image layer, and determines the resource occupancy ratio of each process in each image layer according to the call weight.

[0039] In a multi-player game, a Nash equilibrium strategy refers to a situation where each participant has chosen their optimal strategy, and no one can gain a higher payoff by unilaterally changing their strategy. Resource usage status data includes resource usage data, response time data, and business priority data.

[0040] In this embodiment, for container runtime interfaces such as Docker Engine and containerd, real-time resource usage of containers is collected by calling their APIs or command-line tools such as docker stats and ctr metrics. For the cgroups file system, the statistics file under / sys / fs / cgroup / is read directly to collect information such as CPU usage time, memory usage, and block device I / O for each container. Resource usage data, response time data, and service priority data are collected. Among them, resource usage data includes CPU utilization, memory usage, disk read / write volume, and network transmit / receive volume for each process; response time data includes the average response time of interface calls and P95 / P99 latency; and service priority data includes service level, QoS level, and weight configuration. The collected resource usage status data undergoes preprocessing operations such as cleaning, association, and formatting. Specifically, outliers are removed, such as CPU utilization exceeding 100%, and missing values ​​are filled in, such as historical averages when a process lacks response time data. A mapping relationship of "process → container → image layer" is established by associating process IDs with container IDs, and then by associating container IDs with image layer IDs. Resource usage data is time-aligned and uniformly formatted with response time and business priority data. The data is organized into a structured format, such as JSON or tables, for easy input into subsequent models. Each data group includes fields such as process ID, image layer ID, resource usage metrics, response time, and business priority.

[0041] A resource competition game model is constructed, treating each process as a player and using the call weights (resource allocation ratios) of each process across different mirror layers as policy variables. A comprehensive optimization objective function is built based on resource utilization efficiency, response time, and business priority. For example, higher resource efficiency leads to higher returns; shorter response time leads to higher returns; and higher business priority leads to higher returns. During model solving, the concept of Nash equilibrium in game theory is used to find the optimal policy for each process given the policies of other processes. Specific methods include: iterative optimal response method: each process adjusts its policy sequentially until all processes no longer change their policies; convex optimization solver: the problem is transformed into a convex optimization problem and solved using solvers such as Gurobi or CPLEX; and custom algorithms: such as gradient descent and projected gradient methods, suitable for continuous policy spaces. Preprocessed data is input into the model, which iteratively adjusts the policies of each process until a Nash equilibrium is reached. The call weights of each process at each mirror layer are output, and the resource utilization ratio of each process at each mirror layer is calculated based on these call weights.

[0042] For any process p The optimization function of the resource competition game model The following formula can be used as a reference for design: ; in, It is a process p At the resource layer l Policy variables, such as call weights, must satisfy constraints. ; X These are system status parameters, such as total resources, response time, and priority. It is the first k Sub-functions of the optimization objective, reflecting resource efficiency, latency, fairness, etc.; These are weighting coefficients, satisfying... This is used to balance the importance of multiple objectives.

[0043] Step 103: Obtain the file system mount information of the current process through the running interface of the application image, determine the correspondence between each current process and each current image layer, and based on the correspondence, allocate the collected current system resource usage data to each current process, and determine the resource usage of each current image layer according to the resource occupancy ratio corresponding to each current image layer.

[0044] Among them, process file system mount information refers to detailed data on the file system mount points associated with a process in the operating system. It records key information such as the mount location, mount method, and permission settings of the file systems accessible to the process on physical or network storage. Resource usage data includes CPU usage time, memory usage, disk I / O volume, and network traffic.

[0045] In this embodiment, the relevant information of all running processes in the current system is queried through the application image's runtime interface, such as the API or system call interface provided by the container runtime. The mount information file of each process is accessed to obtain details of the file system mount point currently used by the process. In the mount information, mount records related to the image layer are searched, especially the mount information of union file systems such as OverlayFS. This information typically contains the directory path or identifier of the image layer. The image layer's identifier information, such as the layer's hash value or path, is extracted from the mount information. The extracted image layer identifier is matched with the image layer information recorded in the application image metadata to determine the specific image layer currently accessed by each process. Based on the matching results, a process-image layer mapping table is established, recording the image layer currently associated with each process. Resource usage data of each process in the current system is collected through system performance monitoring tools or interfaces. The collected data should include process identifier information for subsequent association with processes. The collected resource data is cleaned and formatted to remove outliers and invalid data, ensuring data accuracy and consistency. Based on the process identifier, the collected resource usage data is associated with the corresponding process to obtain the resource usage of each process. Based on the previously established correspondence between processes and image layers, and the resource usage ratio of each process in the image layer calculated through a game theory model, the resource usage data of each process is proportionally allocated to the corresponding image layer. For each image layer, all resource usage data allocated to that layer is aggregated to obtain the resource usage of that image layer. During aggregation, the usage of various resources, such as CPU, memory, I / O, and bandwidth, is calculated separately to comprehensively reflect the resource consumption of the image layer. The calculated resource usage of each image layer is recorded to form an image layer resource usage report, providing a basis for subsequent resource management, cost accounting, and performance optimization.

[0046] Step 104: Summarize the resource usage and file storage of each current image layer to obtain the total resource usage of the current application image.

[0047] In this embodiment, the resource usage of each image layer has been obtained in the previous steps, including CPU usage, memory usage, disk I / O, and network traffic. This data is then organized to ensure the completeness and accuracy of the resource usage data for each image layer. In the previous steps, the file storage size of each image layer was obtained by parsing image metadata and deduplicating image layer files. This file storage size data is then organized to ensure its accuracy. The resource usage of all image layers is summed to obtain the total resource usage of the entire application image. Various resource categories are summarized separately, for example: total CPU usage: summing the CPU usage of all image layers; total memory usage: summing the memory usage of all image layers; total disk I / O: summing the disk I / O of all image layers; total network traffic: summing the network traffic of all image layers. The file storage size of all image layers is then summed to obtain the total file storage of the entire application image. Since there may be shared files between image layers, and deduplication has already been completed in the previous steps, the actual physical storage usage can be obtained directly by summing. The total resource usage is calculated by combining the total resource usage with the total file storage. Different weights can be assigned to various resources based on actual needs, and a weighted sum can be calculated to reflect the importance of different resources. The calculated total resource usage is recorded to form an application image resource usage report. The report should include detailed usage and file storage amounts for each type of resource, as well as the calculation result of the total resource usage.

[0048] This invention, through parsing the layered structure and content addressing technology, effectively deduplicates files, avoids redundant calculations of file storage, and improves the accuracy of file storage statistics. By modeling the resource call behavior of processes and image layers as a game process, it quantifies the resource call contribution of each process in different image layers, solving the problem of ambiguous resource ownership when multiple processes share image layers, and making the calculation of resource usage ratios of each image layer more accurate. With resource utilization efficiency, response time, and business priority as optimization objectives, the allocation of resource usage ratios can both conform to the actual operating state of the system and meet the needs of key indicators in business scenarios, improving the adaptability of resource statistics results to business objectives. The call weights output based on the Nash equilibrium strategy are stable, avoiding statistical fluctuations caused by dynamic resource adjustments, reducing the frequency of redundant calculations and error corrections, and improving the efficiency and reliability of resource usage statistics. By obtaining the correspondence between processes and image layers through the runtime interface, resource usage is accurately allocated to each image layer, achieving accurate resource usage statistics and providing more accurate data support for resource management.

[0049] As an example of an embodiment of the present invention, the file of each mirror layer is identified by content addressing technology, wherein the content addressing technology includes a hash algorithm, specifically: The file content is converted into a feature matrix, and a two-dimensional convolutional neural network is used to extract features from the feature matrix to obtain multiple content feature vectors. Each content feature vector is traversed, and the Euclidean distance between any two vectors is calculated. Content feature vectors with a distance less than a preset threshold and satisfying a neighborhood relationship are associated, labeled, and aggregated to obtain multiple association groups. The file segments corresponding to each association group are merged to obtain multiple file blocks. A hash calculation is performed on each file block, and the hash values ​​are concatenated in block order to generate a block-level hash chain corresponding to each file in each mirror layer. Simultaneously, an extended field set of file metadata is extracted, and feature encoding is performed on the extended field set to generate metadata feature vectors. An adaptive deep hash tree is constructed based on the metadata feature vectors to generate metadata hashes. The extended field set includes basic fields and behavioral feature fields. The block-level hash chains and metadata hashes of each file are combined to obtain the content addressing identifier of each file. The path information, size information, and content addressing identifier of each file are associated and stored to obtain a file identifier list for each mirror layer. The file identifier list for each mirror layer is sorted according to the file path hierarchy represented by the path information.

[0050] The basic fields include fundamental file information such as file type, file size, file creation time, file modification time, and file permissions. The behavioral characteristic fields include information about file usage behavior, such as access frequency, access time distribution, modification frequency, and accessing user or process information.

[0051] In this embodiment, firstly, the binary content of the file is read. The binary content can be divided into blocks of fixed size, each block being a fixed-length byte sequence. Each block's byte sequence is converted into a feature vector, using features such as byte value distribution or frequency. These feature vectors are arranged into a matrix to form a feature matrix. The number of rows in this matrix corresponds to the number of file blocks, and the number of columns corresponds to the feature dimension of each block. A two-dimensional convolutional neural network (2D CNN) is designed to process the feature matrix and extract useful features. The feature matrix is ​​input into the 2D CNN. The first layer of the network is a convolutional layer, which uses multiple convolutional kernels to perform convolution operations on the feature matrix to extract local features. After the convolution operation, a pooling layer (such as a max-pooling layer) is usually followed to reduce the spatial dimension of the features while retaining important features. Repeating the convolution and pooling operations, with multiple convolutional and pooling layers, progressively extracts higher-level features. Finally, through one or more fully connected layers, the extracted features are mapped to a fixed-length feature vector, which is the content feature vector. For each block of the file, a content feature vector is obtained. These content feature vectors can be used for subsequent operations such as similarity comparison and clustering to determine the similarity and relevance of the file content.

[0052] Iterate through all content feature vectors, and for each pair of vectors, calculate the Euclidean distance between them. The Euclidean distance is the straight-line distance between two vectors in multidimensional space, and can be calculated using the following formula: ; in, and These are two content feature vectors. and These are their values ​​in the i-th dimension, where n is the dimension of the vector.

[0053] A preset threshold is set. When the Euclidean distance between two content feature vectors is less than this threshold, the two vectors are considered similar, meaning their corresponding file segments are similar. For each pair of similar vectors, they are checked for a neighborhood relationship. A neighborhood relationship means that these vectors are adjacent in the file, i.e., their corresponding file segments are consecutive in the file. If two vectors satisfy both similarity and neighborhood relationship, they are associated and marked as belonging to the same association group. In this way, all similar and adjacent file segments are aggregated to form multiple association groups. Each association group contains a set of similar and consecutive file segments. After traversing all content feature vectors and performing similarity analysis, multiple association groups are obtained. Each association group represents a set of similar and consecutive file segments that are adjacent in the file, and the distance between their content feature vectors is less than the preset threshold.

[0054] Based on the position information of file fragments in the original file, determine the order of file fragments within the association group. Ensure that the merged file block maintains continuity and integrity in content. Merge the file fragments within the association group sequentially to form a complete file block. During the merging process, record the starting position and length of each file fragment within the file block for subsequent indexing and access. Select a suitable hash algorithm, such as SHA-256 or SHA-512, to ensure the uniqueness and collision resistance of the hash values. Determine the parameters and configuration of the hash algorithm based on security and performance requirements. Perform hash calculations on the merged file blocks to obtain the hash values ​​of the file blocks. The hash calculations should cover the entire content of the file blocks to ensure that any changes to the content can be detected. Concatenate the hash values ​​of each file block according to the order of the file blocks in the original file. During concatenation, specific delimiters or encoding methods can be used to ensure that the hash chain structure is clear and easy to parse. For example, the file is divided into 4KB blocks, the SHA-256 hash value of each block is calculated, concatenated in ascending order by file offset, and separated by "|" to obtain the hash chain. The concatenated hash value sequence constitutes the block-level hash chain of the file. Block-level hash chains reflect the overall characteristics of file content while remaining sensitive to local changes. The starting position, length, and corresponding hash value of each file block are recorded to form an index. This index information can be used for quick location and access to file blocks. The block-level hash chain is then stored in association with file path information, size information, etc. Storage can employ databases, file systems, or distributed storage systems to ensure data reliability and accessibility.

[0055] The basic field values ​​of a file are obtained through file system interfaces or system calls. For behavioral feature fields, behavioral data can be collected from system logs, audit systems, or file system hooks, and statistical analysis can be performed to obtain field values. The extracted extended field values ​​are then cleaned to handle missing and outlier values. Numerical fields are normalized or standardized to ensure they fall within a uniform numerical range. Categorical fields are encoded, such as through one-hot encoding or label encoding, converting them into numerical form. The preprocessed field values ​​are combined into a feature vector. This can be done by concatenating the encoded results of each field sequentially into a high-dimensional feature vector. Alternatively, feature embedding techniques can be used to map high-dimensional sparse categorical features to a low-dimensional dense vector space, which is then concatenated with other numerical features. An adaptive deep hash tree is a tree-structured hash model that automatically adjusts its structure based on data features. For example, a metadata hash tree can be constructed with field hashes as leaf nodes, concatenated hashes of child node hashes as leaf nodes, and metadata hashes as the root node. Each node in the tree corresponds to a hash function used to map the input feature vector to its child nodes. Hash functions can be simple linear projections or more complex nonlinear functions. Metadata feature vectors are used as training data, and the parameters of the hash tree are trained using optimization algorithms (such as gradient descent). The training objective is to minimize the path differences between similar feature vectors in the tree while maximizing the path differences between dissimilar feature vectors. The tree depth and node splitting strategy are adaptively adjusted to suit the data distribution. The metadata feature vectors are input into the trained adaptive depth hash tree. Starting from the root node, the hash function of each node is used to propagate downwards layer by layer until a leaf node is reached. The hash value on the path or the identifier of the leaf node is used as the metadata hash. The metadata hash is associated with file path information, size information, etc., and stored. Storage can be in the form of a database table or file system extended attributes. An index is built based on the metadata hash to support fast file retrieval and deduplication. The index can be a hash table or a tree structure, depending on the application scenario and performance requirements.

[0056] The block-level hash chain and metadata hash are combined according to preset rules, such as concatenating them into a complete hash string or using specific hash fusion algorithms (such as double-layer hashing or cross-hashing) to generate a unified identifier. This combination result is the file's content addressing identifier, which reflects both the file's content and metadata characteristics, possessing uniqueness and stability. The complete path information (including directory level and filename) and file size information (in bytes) for each file are obtained from the file system. The path information characterizes the file's hierarchical structure in the image layer, and the size information records the file's storage usage. The file's path information, size information, and content addressing identifier are associated to form a complete file record. This record can be stored in a structured form (such as a database table, key-value pairs, or JSON object) to ensure information integrity and queryability. For all files in the image layer, the above combination and association storage operations are repeated to generate a complete file identifier list for that image layer. Each file corresponds to one record in the list, containing the path, size, and content addressing identifier. The file identifier list is sorted according to the directory hierarchy represented by the file path information. The sorting rules are usually lexicographical or hierarchical depth-first order, ensuring that the list logically reflects the tree structure of the file system.

[0057] As an example of an embodiment of the present invention, the step of deduplicating and calculating the file storage size based on the obtained identification results for each mirror layer specifically involves: The file identifier list of all mirror layers is traversed. By comparing the content addressing identifiers of each file, multiple files with the same content addressing identifier are identified as file duplicate groups. For file duplicate groups, the storage size corresponding to the first occurrence of the file is retained, and the reference relationship of subsequent duplicate files is recorded to achieve file deduplication of all mirror layers. The file storage size is calculated for all deduplicated files in all mirror layers.

[0058] In this embodiment, a file identifier list for all image layers is read from the storage system. The file identifier list contains the path information, size information, and content addressing identifier for each file. Each file in the file identifier list is traversed one by one. For each file, its content addressing identifier is extracted. The content addressing identifier of the current file is compared with the content addressing identifiers of the traversed files. If files with the same content addressing identifier are found, they are identified as a file duplication group. For each file duplication group, the path information and content addressing identifiers of the files within the group are recorded. Data structures such as hash tables or dictionaries can be used to store the information of the file duplication groups. For each file duplication group, the first file to appear is determined. The first file to appear is the "original file" of the duplication group. Subsequent files with the same content addressing identifier as the first file to appear are marked as "duplicate files." The reference relationship between duplicate files and original files is recorded.

[0059] While traversing the file identifier list, a mapping table is maintained between "content-addressable identifier → first-occurrence file record". The first-occurrence file record contains the file path, actual byte size, and storage location pointer. Whenever a file whose content-addressable identifier already exists in the mapping table is encountered, it is determined to be a duplicate file; no new physical storage is calculated for it, but only a soft link or reference count pointing to the "first-occurrence file" is added to that file record. The path of the duplicate file and its corresponding first-occurrence file are written to the reference relationship table, and the reference count of the first-occurrence file is updated for subsequent fast location and reclamation. Duplicate files are no longer included in the total storage; only the byte size of the first-occurrence file is retained and accumulated, thereby achieving single-copy storage across mirror layers and completing deduplication.

[0060] In the deduplication step, it has been determined which files appear for the first time and which are duplicates. A deduplicated file list is created, containing only the first occurrence of each file. For each file in the deduplicated file list, its storage size is calculated. Storage size can be the file size itself or the actual amount of physical storage the file occupies. The storage sizes of all deduplicated files are summed to obtain the total storage size. The total storage size reflects the file storage situation across all mirror layers after deduplication.

[0061] As an example of an embodiment of the present invention, the step of determining the resource usage ratio of each process in each image layer according to the respective call weights specifically involves: The process obtains the call frequency sequence and call time sequence of each process within a dynamic time window. Based on the peak call frequency periods in the call frequency sequence, the call time sequence is divided into intervals of unequal length to obtain multiple time slices. A three-dimensional resource feature matrix is ​​generated based on the time slices and the call weights of each process in each mirror layer. The call entropy value of each time slice is calculated to generate a temporal attention weight matrix, and an association graph between the mirror layer and the process is constructed. The association graph is then processed using a graph attention network to obtain a spatial attention weight matrix. The temporal attention weight matrix and the spatial attention matrix are then processed to obtain a composite attention matrix. Finally, the composite attention matrix is ​​used to calculate the weighted call frequency features in the three-dimensional resource feature matrix. Multidimensional feature extraction is performed on the weighted call frequency features for each process to obtain a multidimensional feature vector corresponding to each process. Density clustering algorithm is used to cluster the multidimensional feature vectors to obtain call pattern clusters for each process. For the current sliding window centered on the current time slice, aggregation calculation is performed on the weighted call frequency features within the current sliding window to obtain the initial resource occupancy value of each process in each mirror layer. After normalization, the initial resource occupancy ratio is obtained, wherein the time slice weight within the current sliding window decays exponentially. For multiple processes within each call pattern cluster, the initial resource occupancy ratio is adjusted according to the similarity between the call pattern cluster of each process and the cluster center, and double constraint and normalization processing are performed to obtain the resource occupancy ratio.

[0062] In this embodiment, the call frequency sequence of each process within a dynamic time window is obtained through system monitoring tools or log analysis. The call frequency sequence is a list of call counts arranged chronologically, reflecting the activity level of the process at different points in time. Similarly, the call time sequence of each process is obtained through system monitoring tools or log analysis. The call time sequence records the specific time point of each call. Analyzing the call frequency sequence identifies peak periods of call frequency. Peak periods can be determined by setting a threshold; for example, when the call frequency exceeds twice the average, it is considered a peak period. Based on the identified peak periods, the call time sequence is divided into intervals of unequal length. Each interval corresponds to a time slice, and the length of the time slice is determined by the duration of the peak period. For each time slice, the call weight of each process in each mirror layer within that time slice is determined. The call weight can be an indicator such as the frequency or duration of the process's access to the mirror layer within that time slice. The dimensions of the three-dimensional resource feature matrix are: time slice × process × mirror layer. Each element in the matrix represents the call weight of a specific process to a specific mirror layer within a specific time slice. For example, the element values ​​of the t-th time slice, p-th process, and l-th mirror layer of the matrix are... , is used to represent the call weight of process p to image layer l within time slice t.

[0063] For each time slice, calculate the call frequency distribution of all processes within that time slice. Calculate the call entropy value using the information entropy formula: ; in, It is the proportion of the call frequency of the i-th process in this time slice to the total call frequency, and n is the number of different processes in this time slice.

[0064] A temporal attention weight matrix is ​​generated based on the call entropy value of each time slice. Time slices with higher call entropy values ​​have lower weights, as high entropy indicates a more uniform call distribution and less significant call frequency for each process. Mirror layers and processes are treated as nodes in a graph. If a process makes a call in a mirror layer, an edge is established between the corresponding nodes. The edge weight can be represented by metrics such as call frequency or call time. A Graph Attention Network (GAT) is used to calculate the attention weight for each node (mirror layer and process) in the graph. The GAT learns the relationships between nodes and assigns different weights to each node, representing its importance in the graph. The attention weights for each mirror layer and process calculated by the GAT are combined into a spatial attention weight matrix. Rows in the matrix represent mirror layers, columns represent processes, and element values ​​represent the corresponding attention weights. The temporal attention weight matrix and the spatial attention weight matrix are multiplied element-wise to obtain a composite attention matrix. The composite attention matrix considers both the call distribution of time slices and the relationships between mirror layers and processes, more accurately reflecting the resource consumption of each process in each mirror layer.

[0065] The composite attention matrix and the 3D resource feature matrix are multiplied element-wise to obtain a weighted 3D resource feature matrix. Each element of the composite attention matrix represents the weight of the corresponding time slice, process, and mirror layer, while each element of the 3D resource feature matrix represents the call frequency of the corresponding time slice, process, and mirror layer. Each element in the weighted 3D resource feature matrix represents the weighted call frequency of a specific process to a specific mirror layer within a specific time slice. These weighted call frequency features reflect the resource consumption of each process in each mirror layer. The weighted 3D resource feature matrix is ​​analyzed to extract a multidimensional feature vector for each process. The multidimensional feature vector can include the weighted call frequency of the process in different time slices and different mirror layers, as well as the statistical characteristics of these frequencies (such as mean, variance, maximum, minimum, etc.). The extracted multidimensional feature vectors are combined into a complete feature representation. For example, feature vectors from different time slices and different mirror layers can be concatenated into a high-dimensional vector, or feature fusion techniques (such as weighted average, principal component analysis, etc.) can be used to combine the feature vectors. A suitable density clustering algorithm, such as DBSCAN or OPTICS, is selected. Density clustering algorithms can cluster data points based on the density distribution of data, making them suitable for processing high-dimensional feature vectors. Multidimensional feature vectors are input into the density clustering algorithm, which then performs clustering based on the algorithm's parameters (such as neighborhood radius, minimum number of points, etc.). The algorithm clusters similar feature vectors into the same cluster, with each cluster representing a calling pattern. After clustering, each cluster contains a set of similar multidimensional feature vectors, and the processes corresponding to these vectors have similar calling patterns. The calling pattern of a cluster can be characterized by its centroid or representative point.

[0066] Select a current time slice as the center of the sliding window. The current time slice can be the most recent time slice or another time slice selected according to specific rules. Set the size of the sliding window as needed, i.e., the number of time slices it contains. The sliding window can contain the current time slice and several time slices before and after it. Extract the weighted call frequency feature of each time slice within the current sliding window from the 3D resource feature matrix. The weighted call frequency feature is obtained by weighting the 3D resource feature matrix using a composite attention matrix. Assign a weight to each time slice within the current sliding window, with the weight decaying exponentially. The closer the time slice is to the current time slice, the higher its weight; the farther the time slice is, the lower its weight. Multiply the weighted call frequency feature of each time slice within the current sliding window by the corresponding time slice weight, and then sum them. The result of the aggregate calculation is the initial resource occupancy value of each process in each mirror layer within the current sliding window. Normalize the initial occupancy value obtained from the aggregate calculation. The purpose of normalization is to convert the initial occupancy value into a proportional form so that its sum is 1. The value obtained after normalization is the initial resource occupancy ratio of each process in each mirror layer. The initial resource usage ratio reflects the relative resource usage of each process on each image layer within the current sliding window.

[0067] For each call pattern cluster, calculate the similarity between the call pattern of each process within the cluster and the cluster center. Similarity can be calculated using cosine similarity, Euclidean distance, or other similarity metrics. Based on the similarity, adjust the initial resource usage ratios. Processes with higher similarity can have their resource usage ratios increased, while processes with lower similarity can have their resource usage ratios decreased. Ensure that the adjusted resource usage ratios satisfy a double constraint: the resource usage ratio of each process must be non-negative, and the sum of the resource usage ratios of all processes must equal 1. If the adjusted ratios do not meet these constraints, corrections are needed. For example, if a process's ratio is negative, it can be set to 0, and the saved ratio can be redistributed to other processes. Normalize the adjusted resource usage ratios to ensure that the sum of all ratios is 1. Normalization can be achieved by dividing each ratio by the sum of all ratios. The resource usage ratios obtained after normalization are the final resource usage ratios. These ratios reflect the resource usage of each process in each image layer, considering call pattern similarity.

[0068] As an example of an embodiment of the present invention, the step of summing the resource usage of each current image layer and the file storage amount to obtain the total resource usage of the current application image specifically involves: Based on the computing resources, storage resources, and network resources in the resource dimension classification table, the resource usage and file storage of each current image layer are statistically analyzed to obtain computing resource data, storage resource data, and network resource data. The computing resources include the usage of the central processing unit and memory; the storage resources include file storage and disk I / O; and the network resources include network bandwidth. The resource data is standardized, and information entropy is used to calculate the initial statistical weights for each resource type. Based on the deviation between the actual load and the rated load, the resource load deviation for each resource type is determined. The initial statistical weights for each resource type are then corrected using these resource load deviations to obtain corrected statistical weights. Finally, the total resource usage is calculated based on these corrected statistical weights.

[0069] In this embodiment, CPU usage data for each image layer is obtained from system monitoring tools or logs. CPU usage can be an absolute time (e.g., milliseconds) or a relative proportion (e.g., percentage). Memory usage data for each image layer is obtained from system monitoring tools or logs. Memory usage can be an absolute size (e.g., bytes) or a relative proportion (e.g., percentage). File storage data for each image layer is obtained from the file system or storage management system. File storage is the total size of the file content, usually measured in bytes. Disk I / O usage data for each image layer is obtained from system monitoring tools or logs. Disk I / O usage can be metrics such as read / write counts or read / write bytes. Network bandwidth usage data for each image layer is obtained from network monitoring tools or logs. Network bandwidth usage can be the amount of data sent and received (e.g., bytes) or a bandwidth usage percentage (e.g., percentage).

[0070] The data from various resources are standardized to ensure comparability. For example, for five raw metrics—CPU, memory, file storage, disk I / O, and network bandwidth—Min-Max or Z-Score methods are used to map values ​​of different dimensions to the 0-1 interval or a standard normal distribution, making the data comparable on the same scale. Each resource category is treated as a data column, and its distribution across all mirror layers is statistically analyzed. The entropy value of this column is calculated using the information entropy formula: a larger entropy value indicates a more uniform distribution and lower distinguishability between layers; a smaller entropy value indicates a more concentrated distribution and higher distinguishability. Following the principle of "smaller entropy value, larger weight," the entropy value is taken in reverse or subjected to a linear transformation to obtain the initial weight for each resource category. The result forms a set of weight coefficients corresponding one-to-one with each resource category, used for subsequent correction and weighted aggregation.

[0071] For each resource type (CPU, memory, storage, network), calculate the deviation between the actual load and the rated load. The deviation can be expressed as the ratio of the actual load to the rated load, or the difference between the actual load and the rated load. Based on the deviation, determine the resource load deviation for each resource type. The resource load deviation can be the absolute value of the deviation or a relative value (e.g., a percentage). Use the resource load deviation to correct the initial statistical weights for each resource type. The correction method can be multiplying the initial statistical weights by the resource load deviation, or calculating them using other methods. The corrected statistical weights are the corrected statistical weights for each resource type. The corrected statistical weights reflect the degree of difference between the actual load and the rated load for each resource type.

[0072] The adjusted statistical weight of each resource type is weighted and summed with the corresponding resource data. The result of the weighted sum is the total resource usage. Total resource usage reflects the application image's overall resource consumption in terms of computing, storage, and networking. Total resource usage can be used for resource management, cost accounting, and performance optimization.

[0073] As an example of an embodiment of the present invention, determining the correspondence between each current process and each current image layer specifically involves: The file system mount information of the current process is parsed by mounting the namespace to obtain the file paths accessed by each process; the file paths are matched with the file identifier list of each image layer to determine the target image layer to which the accessed file in each process belongs; a mapping table between each process and the target image layer is established to determine the correspondence.

[0074] In this embodiment, system calls, such as Linux's setns, or direct reading of / proc / are used. <pid>The process enters the process's mount namespace, extracts the mount point and path, parses each record in the mount table, and extracts the mount root, mount point, and optional overlay layer information (lowerdir, upperdir, etc.) to reconstruct the directory tree actually visible to the process. Using standard file system traversal tools (find, ls -R, or kernel fanotify events), it recursively lists all open or mapped file paths in the process's root directory; these paths represent the set of files currently accessed by the process.

[0075] A "file identifier list" is pre-generated for each image layer, where each record contains the absolute path of the file within the layer and its content addressing identifier (e.g., SHA-256). The list is sorted lexicographically by path for easy searching. For the access paths enumerated by the process, the mount point prefix is ​​removed to obtain the path relative to the root of the image layer. This relative path is used as the lookup key to perform a binary search or hash search in the file identifier list of the corresponding image layer: if the path exists and the content identifier matches, the file is determined to belong to that image layer; if the path does not exist or the content identifier is different, the next layer is checked until a matching layer is found or it is confirmed that the file does not belong to any known layer. Each successful match records the mapping entry of "process ID → target image layer". After all paths have been processed, a complete mapping table of each process and the image layers it accesses is formed.

[0076] As an example of an embodiment of the present invention, determining the resource usage of each current image layer based on the resource occupancy ratio corresponding to each current image layer specifically involves: The system collects usage trajectory data corresponding to the current system resource usage data. Based on this usage trajectory data, it determines the correspondence between used resources and the current process. The usage trajectory data includes a resource identifier, a process identifier, a resource type, and a usage duration. Based on the correspondence between the current process and the current mirror layer, it associates the resource identifier with the current mirror layer. Resources commonly associated with at least two current mirror layers are identified as shared resources, generating a shared resource list. The shared resources in the shared resource list are processed differently according to their sharing type. The shared resource usage of each mirror layer is output. The shared resource usage is then merged with the exclusive resource usage of each mirror layer to obtain the resource usage of each current mirror layer.

[0077] In this embodiment, a unique identifier for each resource is identified and recorded, such as a CPU core, memory address, disk block, or network port. The process identifier using the resource is identified and recorded, such as process ID or thread ID. The resource type is recorded, such as computing resources, storage resources, or network resources. The duration for which each process uses each resource is recorded, which can be an absolute time (e.g., milliseconds) or a relative time (e.g., the number of time slices). The collected usage trajectory data is analyzed to determine which processes are using each resource, and the duration and frequency of use. A mapping table between used resources and current processes is established. Each record in the table includes the resource identifier, the process identifier, the resource type, and the usage duration.

[0078] For each trajectory record, the "process ID" is used to look up the "process-mirror layer" mapping table to obtain the list of mirror layers to which the process belongs. The "resource ID" in this trajectory is copied and associated with each layer in the aforementioned mirror layer list, forming a many-to-many "resource-mirror layer" relationship. For example, if process P belongs to mirror layers A and B, and P uses resource R, then R is associated with both A and B. The "resource-mirror layer" relationship table is aggregated to count how many mirror layers reference each resource ID. If a resource is referenced by two or more mirror layers, it is considered a shared resource; otherwise, it is an exclusive resource. All shared resources are extracted to form a candidate set of shared resources. Attributes are added to each shared resource record: resource type, total usage time, and the list of associated mirror layers. Shared resources are grouped by resource type (CPU, memory, disk I / O, network bandwidth, etc.) to obtain a categorized list of shared resources. This list serves as input for the next step, the "differentiated allocation" algorithm, to calculate the actual usage of shared resources by each mirror layer.

[0079] Traverse the shared resource list and group resources by type (CPU, memory, disk I / O, network bandwidth, etc.). Select a corresponding "allocation strategy" for each resource type: For resources that can be used in parallel, such as CPU and memory, "equal distribution" or "weighted by call weight" is commonly used. For exclusive or bandwidth-constrained resources, such as disk I / O and network bandwidth, "weighted by actual usage time or traffic" is commonly used. If equal distribution is used, divide the total usage time or total usage of the resource by the number of associated mirror layers to obtain the average usage per layer. If call weighting is used, read the call weight of each mirror layer on the resource, normalize it, and then allocate the total usage according to the weight ratio. Calculate the contribution rate of each mirror layer based on its actual usage time, number of bytes read / written, or peak bandwidth percentage. Allocate the total resource to each mirror layer according to the contribution rate. Create a two-dimensional "Mirror Layer-Resource" table to record the allocation result of each layer on each shared resource. Read the exclusive resource records of each mirror layer (resources used only by that layer). Exclusive resources are directly included in the resource usage of the corresponding mirror layer. For each mirror layer, the shared resource allocation and exclusive resource allocation are summed according to resource type to obtain the total usage of CPU, memory, disk I / O, and network bandwidth for that layer. This forms the final "Mirror Layer Resource Usage" list, providing a basis for subsequent cost accounting, performance analysis, or scheduling optimization.

[0080] As an example of an embodiment of the present invention, after obtaining the total resource usage of the current application image, the specific steps are as follows: A composite hash fingerprint is generated for each mirror layer. The current composite hash fingerprint is compared with historical composite hash fingerprints to identify the set of changed mirror layers. The composite hash fingerprint includes the hashes of all file content and all file metadata of each mirror layer. When the change type corresponding to the changed mirror layer is addition or modification, the first file storage amount corresponding to the changed file is calculated, and the first resource usage during the execution of the process corresponding to the changed file is calculated. The total resource usage is updated based on the first file storage amount and the first resource usage to obtain the first resource usage. When the change type corresponding to the changed mirror layer is deletion, the second file storage amount and the second resource usage corresponding to the changed mirror layer are deleted from the total resource usage to obtain the second resource usage.

[0081] In this embodiment, the content of all files in each mirror layer is hashed to obtain the content hash value of each file. Hash algorithms such as SHA-256 and SHA-512 can be used. The metadata of all files in each mirror layer is hashed to obtain the metadata hash value of each file. Metadata includes file size, creation time, modification time, and permission information. The hashes of all file content and all file metadata in each mirror layer are combined to form a composite hash fingerprint. This can be achieved by concatenating all hash values ​​into a long string and then hashing this string again. Historical composite hash fingerprints for each mirror layer are obtained from the historical records. These historical composite hash fingerprints are composite hash fingerprints from a previous point in time. The current composite hash fingerprint is compared with the historical composite hash fingerprint. If the two composite hash fingerprints are the same, the mirror layer has not changed; if they are different, the mirror layer has changed. The mirror layers corresponding to the different composite hash fingerprints in the comparison results are identified as changed mirror layers. The set of changed mirror layers includes all mirror layers that have undergone changes.

[0082] By comparing composite hash fingerprints, the set of files whose content or metadata has changed is immediately located. In the case of addition or modification, after content addressing and block-level deduplication of the changed files, the net increase in physical bytes is accumulated to obtain the additional storage occupied by that layer due to addition or modification. Within the latest sliding window, the process execution trajectory (CPU, memory, disk I / O, network bandwidth) of processes accessing these changed files is collected. According to the predetermined "process-image layer" mapping and call weight, the above resource consumption is distributed to the image layer to form incremental runtime overhead. The original total resource usage is split into total storage and total runtime resources of various types, and the first file storage and the first resource usage are added to each respectively. The merged result is the first resource usage, which serves as the new resource baseline for the image.

[0083] Similarly, by using composite hash fingerprint comparison, the image layer that has been completely removed is identified. In this deletion scenario, the total storage and total running resources recorded in the last time for this layer are directly read as the "second" value to be deducted. The second file storage and the second resource usage are subtracted from the original total storage and various running resources to obtain the second resource usage, thus completing the reduction update of the image resource baseline.

[0084] like Figure 2 As shown, based on the above method embodiments, corresponding system embodiments are provided; One embodiment of the present invention provides a resource usage statistics system for application images, including: a file storage module 201, a competition game module 202, a ratio determination module 203, and a summary module 204; The file storage module 201 is used to parse application image metadata to obtain a hierarchical structure of application images including multiple image layers, and to identify the files of each image layer through content addressing technology. Based on the obtained identification results, the files of each image layer are deduplicated and calculated to obtain the file storage volume. The competitive game module 202 is used to collect the resource usage status data of the application image and input it into the resource competitive game model, so that the resource competitive game model uses an optimization function determined by resource usage efficiency, response time and business priority, solves the Nash equilibrium strategy, outputs the call weight of each process in each image layer, and determines the resource occupancy ratio of each process in each image layer according to the call weight. The ratio determination module 203 is used to obtain the file system mount information of the current process through the running interface of the application image, determine the correspondence between each current process and each current image layer, and based on the correspondence, allocate the collected current system resource usage data to each current process, and determine the resource usage of each current image layer according to the resource occupancy ratio corresponding to each current image layer. The aggregation module 204 is used to aggregate the resource usage and file storage of each current image layer to obtain the total resource usage of the current application image.

[0085] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.< / pid>

Claims

1. A method for calculating resource usage for application images, characterized in that, include: The application image metadata is parsed to obtain the layered structure of the application image, which includes multiple image layers. The files of each image layer are identified using content addressing technology. Based on the identification results, the files of each image layer are deduplicated and calculated to obtain the file storage size. The resource usage status data of the application image is collected and input into the resource competition game model, so that the resource competition game model uses an optimization function determined by resource usage efficiency, response time and business priority, solves the Nash equilibrium strategy, outputs the call weight of each process in each image layer, and determines the resource occupancy ratio of each process in each image layer according to the call weight. The application image's runtime interface is used to obtain the file system mount information of the current process, determine the correspondence between each current process and each current image layer, and based on the correspondence, allocate the collected current system resource usage data to each current process, and determine the resource usage of each current image layer according to the resource occupancy ratio corresponding to each current image layer. The resource usage and file storage of each current image layer are summed to obtain the total resource usage of the current application image.

2. The resource usage statistics method for application images as described in claim 1, characterized in that, The process of identifying files in each mirror layer using content addressing technology, wherein the content addressing technology includes a hash algorithm, specifically: The file content is converted into a feature matrix, and a two-dimensional convolutional neural network is used to extract features from the feature matrix to obtain multiple content feature vectors; Traverse each of the content feature vectors, calculate the Euclidean distance between any two vectors, associate and aggregate the content feature vectors whose distance is less than a preset threshold and which satisfy the neighborhood relationship to obtain multiple association groups; The file fragments corresponding to each of the aforementioned association groups are merged to obtain multiple file blocks. A hash calculation is performed on each file block, and the hash values ​​are concatenated in the block order to generate a block-level hash chain corresponding to each file in each image layer. Simultaneously, an extended field set of file metadata is extracted, feature encoding is performed on the extended field set to generate a metadata feature vector, and an adaptive deep hash tree is constructed based on the metadata feature vector to generate a metadata hash. The extended field set includes basic fields and behavioral feature fields. By combining the block-level hash chain and the metadata hash of each file, the content addressing identifier of each file is obtained. The path information, size information and the content addressing identifier of each file are associated and stored to obtain the file identifier list of each mirror layer. The file identifier list of each mirror layer is sorted according to the file path hierarchy represented by the path information.

3. The resource usage statistics method for application images as described in claim 1, characterized in that, The determination of the resource usage ratio of each process in each image layer based on the respective call weights is specifically as follows: Obtain the call frequency sequence and call time sequence of each process within a dynamic time window. Based on the peak call frequency period in the call frequency sequence, divide the call time sequence into intervals of unequal length to obtain multiple time slices. Generate a three-dimensional resource feature matrix based on the time slices and the call weight of each process in each image layer. Calculate the call entropy value for each time slice, generate a temporal attention weight matrix, and construct an association graph between the mirror layer and the process. Calculate the association graph using a graph attention network to obtain a spatial attention weight matrix. Calculate the temporal attention weight matrix and the spatial attention matrix to obtain a composite attention matrix. The three-dimensional resource feature matrix is ​​calculated using a composite attention matrix to obtain weighted call frequency features. Multidimensional features of each process are extracted from the weighted call frequency features to obtain multidimensional feature vectors corresponding to each process. Density clustering algorithm is used to cluster the multidimensional feature vectors to obtain call pattern clusters for each process. For the current sliding window centered on the current time slice, perform aggregate calculation on the weighted call frequency features within the current sliding window to obtain the initial resource occupancy value of each process in each image layer. After normalization, obtain the initial resource occupancy ratio. The time slice weight within the current sliding window decays exponentially. For multiple processes within each of the aforementioned call pattern clusters, the initial resource occupancy ratio is adjusted based on the similarity between the call pattern cluster of each process and the cluster center, and double constraint and normalization processing is performed to obtain the resource occupancy ratio.

4. The resource usage statistics method for application images as described in claim 1, characterized in that, The process of summing the resource usage and file storage of each current image layer to obtain the total resource usage of the current application image is as follows: Based on the computing resources, storage resources, and network resources in the resource dimension classification table, the resource usage and file storage of each current image layer are statistically analyzed to obtain computing resource data, storage resource data, and network resource data. The computing resources include the usage of the central processing unit and memory, the storage resources include file storage and disk input / output occupancy, and the network resources include network bandwidth. The data of various resources are standardized and the data is calculated using information entropy to obtain the initial statistical weights of various resource data. Based on the deviation between the actual load and the rated load, the resource load deviation degree corresponding to each type of resource is determined. The initial statistical weights of each type of resource are then corrected using the resource load deviation degree to obtain the corrected statistical weights of each type of resource. The total resource usage is obtained by statistically calculating various types of resource data based on the modified statistical weights.

5. The resource usage statistics method for application images as described in claim 1, characterized in that, The determination of resource usage for each current image layer based on its resource occupancy ratio is specifically as follows: Collect usage trajectory data corresponding to the current system resource usage data, and determine the correspondence between the used resources and the current process based on the usage trajectory data. The usage trajectory data includes the resource usage identifier, the process usage identifier, the resource usage type, and the usage duration. Based on the correspondence between the current process and the current image layer, the resource identifier is associated with the current image layer, and the resource used that is associated with at least two current image layers is identified as a shared resource, and a list of shared resources is generated. The shared resources in the shared resource list are processed differently according to the sharing type. The usage of shared resources in each mirror layer is output. The usage of shared resources is then combined with the usage of exclusive resources in each mirror layer to obtain the resource usage of each current mirror layer.

6. The resource usage statistics method for application images as described in claim 1, characterized in that, After obtaining the total resource usage of the current application image, the specific steps are as follows: A composite hash fingerprint is generated for each mirror layer. The current composite hash fingerprint is compared with the historical composite hash fingerprint to identify the set of changed mirror layers. The composite hash fingerprint includes the hash of all file content and the hash of all file metadata of each mirror layer. When the change type corresponding to the changed image layer is a new type or a modified type, calculate the storage amount of the first file corresponding to the changed file and calculate the first resource usage during the execution of the process corresponding to the changed file. Update the total resource usage based on the first file storage amount and the first resource usage to obtain the first resource usage. When the change type corresponding to the changed mirror layer is deletion, the second file storage and second resource usage corresponding to the changed mirror layer are removed from the total resource usage to obtain the second resource usage.

7. The resource usage statistics method for application images as described in claim 2, characterized in that, The process of deduplicating and calculating the file storage size based on the obtained identification results for each mirror layer is as follows: Traverse the file identifier list of all mirror layers, and identify multiple files with the same content address identifier as a file duplicate group by comparing the content address identifiers of each file; For file duplication groups, retain the storage value corresponding to the first occurrence of the file and record the reference relationship of subsequent duplicate files to achieve file deduplication across all mirror layers; The storage capacity of the files is calculated by examining all the deduplicated mirrored layers.

8. The resource usage statistics method for application images as described in claim 1, characterized in that, The determination of the correspondence between each current process and each current image layer specifically involves: By resolving the file system mount information of the current process through the mount namespace, the file paths accessed by each process can be obtained. The file path is matched with the file identifier list of each image layer to determine the target image layer to which the accessed file in each process belongs; Establish a mapping table between each process and the target image layer to determine the correspondence.

9. A resource usage statistics system for application images, characterized in that, include: The module includes a file storage module, a competitive game module, a ratio determination module, and a summary module. The file storage module is used to parse application image metadata to obtain a hierarchical structure of application images including multiple image layers, and to identify the files of each image layer through content addressing technology. Based on the obtained identification results, the files of each image layer are deduplicated and calculated to obtain the file storage volume. The competitive game module is used to collect resource usage status data of the application image and input it into the resource competition game model, so that the resource competition game model uses an optimization function determined by resource usage efficiency, response time and business priority, solves the Nash equilibrium strategy, outputs the call weight of each process in each image layer, and determines the resource occupancy ratio of each process in each image layer according to the call weight. The ratio determination module is used to obtain the file system mount information of the current process through the running interface of the application image, determine the correspondence between each current process and each current image layer, and based on the correspondence, allocate the collected current system resource usage data to each current process, and determine the resource usage of each current image layer according to the resource occupancy ratio corresponding to each current image layer. The aggregation module is used to aggregate the resource usage and file storage of each current image layer to obtain the total resource usage of the current application image.

10. The resource usage statistics system for application images as described in claim 9, characterized in that, The file storage module, wherein the content addressing technology includes a hash algorithm, specifically: This is used to convert file content into a feature matrix, and then use a two-dimensional convolutional neural network to extract features from the feature matrix to obtain multiple content feature vectors; Traverse each of the content feature vectors, calculate the Euclidean distance between any two vectors, associate and aggregate the content feature vectors whose distance is less than a preset threshold and which satisfy the neighborhood relationship to obtain multiple association groups; The file fragments corresponding to each of the aforementioned association groups are merged to obtain multiple file blocks. A hash calculation is performed on each file block, and the hash values ​​are concatenated in the block order to generate a block-level hash chain corresponding to each file in each image layer. Simultaneously, an extended field set of file metadata is extracted, feature encoding is performed on the extended field set to generate a metadata feature vector, and an adaptive deep hash tree is constructed based on the metadata feature vector to generate a metadata hash. The extended field set includes basic fields and behavioral feature fields. By combining the block-level hash chain and the metadata hash of each file, the content addressing identifier of each file is obtained. The path information, size information and the content addressing identifier of each file are associated and stored to obtain the file identifier list of each mirror layer. The file identifier list of each mirror layer is sorted according to the file path hierarchy represented by the path information.

Images